反向傳播算法

目錄

• Chain rule
• Multi-output Perceptron
• Multi-Layer Perceptron

Chain rule

Multi-output Perceptron

Multi-Layer Perceptron

• 對於多隱藏層結構的神經網絡能夠把隱藏層的節點當作輸出層的節點

• For an output layer node \(k\in{K}\)

\[ \frac{\partial{E}}{\partial{W_{jk}}}=O_j\delta_k,\,\delta_k=O_k(1-O_k)(O_k-t_k) \]

• For a hidden layer node \(j\in{J}\)

\[ \frac{\partial{E}}{\partial{W_{ij}}}=O_i\delta_j,\,\delta_j=O_j(1-O_j)\sum_{k\in{K}}\delta_kW_{jk} \]

• 其中\(\delta_k\)能夠看作是\(O_j\)的信息；\(\delta_j\)能夠看作是\(O_i\)的信息
• 而且下一層的隱藏層偏微分的更新都基於上一隱藏層的偏微分