Deep learning II - I Practical aspects of deep learning - Vanishing/Exploring gradients 梯度消失/爆炸
Vanishing/Exploring gradients 梯度消失/爆炸 如圖所示，由於深度神經網絡深度較大，當每一層的 w w 都小於1或者都大於1時，最終的輸出值會成指數級較小或上升，造成梯度消失或者爆炸，從而使得梯度下降變得困難 1. 爲了不讓 z z 太大，當 n n 越大時， w w 的值應該越小 2. 針對 ReLU R e L U ，初始化 w[l]=np.random.randn
相關文章
- 1. Deep learning II - I Practical aspects of deep learning - Normalizing inputs 輸入歸一化
- 2. Deep learning II - I Practical aspects of deep learning - Understanding dropout 理解隨機失活正則化
- 3. Deep learning II - I Practical aspects of deep learning - Dropout regularization 隨機失活正則化
- 4. 007 Practical aspects of Deep Learning
- 5. Deep learning II - I Practical aspects of deep learning - 正則化如何較少過擬合
- 6. Deep learning II - I Practical aspects of deep learning - other regularization methods 其他正則化方法
- 7. [coursera/ImprovingDL/week1]Practical aspects of Deep Learning(summary&question)
- 8. 【吳恩達深度學習筆記】2.1 Practical aspects of Deep Learning
- 9. 梯度爆炸（Exploding Gradients）
- 10. Deep learning II - I Practical aspects of deep learning - Regularizing your neural network 神經網絡範數正則化
- 更多相關文章...
- • Lua 文件 I/O - Lua 教程
- • HTTP 消息結構 - HTTP 教程
- • 算法總結-廣度優先算法
- • 算法總結-深度優先算法
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
歡迎關注本站公眾號,獲取更多信息
相關文章
- 1. Deep learning II - I Practical aspects of deep learning - Normalizing inputs 輸入歸一化
- 2. Deep learning II - I Practical aspects of deep learning - Understanding dropout 理解隨機失活正則化
- 3. Deep learning II - I Practical aspects of deep learning - Dropout regularization 隨機失活正則化
- 4. 007 Practical aspects of Deep Learning
- 5. Deep learning II - I Practical aspects of deep learning - 正則化如何較少過擬合
- 6. Deep learning II - I Practical aspects of deep learning - other regularization methods 其他正則化方法
- 7. [coursera/ImprovingDL/week1]Practical aspects of Deep Learning(summary&question)
- 8. 【吳恩達深度學習筆記】2.1 Practical aspects of Deep Learning
- 9. 梯度爆炸（Exploding Gradients）
- 10. Deep learning II - I Practical aspects of deep learning - Regularizing your neural network 神經網絡範數正則化