循環神經網絡層

目錄

• Recap
• Sentiment Analysis
  • Proposal
• S1.Weight sharing
  • Naive version
  • Weight share
• S2.Consistent memory
• Unfolded model
• Formulation
• Overall Diagram
• One more thing
• How to Train?

Recap

Sentiment Analysis

Proposal

• Long sentence
  • 100 + words
  • too much parameters[\(w_N,b_N\)]
• No context information
  • consistent tensor

S1.Weight sharing

• 相似於卷積的卷積核，卷積視野，權重分享

Naive version

Weight share

• 權重分享

S2.Consistent memory

• 須要一個東西存儲語境，也就是每一個單詞的語境信息

• 不斷地對語境信息進行堆疊獲得h5，直接使用h5做爲判斷標準

Unfolded model

• 在時間軸上摺疊，不斷地更新h

Formulation

• 經過激活函數tanh不斷地疊加上個時間戳的信息

Overall Diagram

• 全覽分解圖

One more thing

How to Train?