Revisit Knowledge Distillation: a Teacher-free Framework
Observations 通過幾組實驗觀察到 反轉Knowledge Distillation(KD)即利用student來guide teacher的話，teacher的性能依然可以得到提升 用一個比student還差的teacher來guide student的話，student的性能依然可以得到提升 因此作者得到以下觀點 KD只是一種可學習的label smoothing regulariz
相關文章
- 1. Awesome Knowledge-Distillation
- 2. Knowledge Distillation
- 3. knowledge distillation論文閱讀之：Learning from a Lightweight Teacher for Efficient Knowledge Distillation
- 4. Knowledge Distillation 筆記
- 5. Tutorial: Knowledge Distillation
- 6. Similarity-Preserving Knowledge Distillation
- 7. Knowledge Distillation: A Survey文獻閱讀
- 8. knowledge distillation 論文閱讀之：ResKD: Residual-Guided Knowledge Distillation
- 9. 論文Relational Knowledge Distillation
- 10. Correlation Congruence for Knowledge Distillation
- 更多相關文章...
- • Maven 項目模板 - Maven教程
- • Maven 構建 Java 項目 - Maven教程
- • Kotlin學習（二）基本類型
- • 委託模式
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
歡迎關注本站公眾號,獲取更多信息
相關文章
- 1. Awesome Knowledge-Distillation
- 2. Knowledge Distillation
- 3. knowledge distillation論文閱讀之：Learning from a Lightweight Teacher for Efficient Knowledge Distillation
- 4. Knowledge Distillation 筆記
- 5. Tutorial: Knowledge Distillation
- 6. Similarity-Preserving Knowledge Distillation
- 7. Knowledge Distillation: A Survey文獻閱讀
- 8. knowledge distillation 論文閱讀之：ResKD: Residual-Guided Knowledge Distillation
- 9. 論文Relational Knowledge Distillation
- 10. Correlation Congruence for Knowledge Distillation