ELECTRA: NLP預訓練模型

ELECTRA的全稱是Efficiently Learning an Encoder that Classifies Token Replacements Accurately. NLP式的Generator-Discriminator ELECTRA最主要的貢獻是提出了新的預訓練任務和框架，把生成式的Masked language model(MLM)預訓練任務改成了判別式的Replaced t

>>阅读原文<<