nlp homework 03

NLP Homework 03 ——馮煜博

題目描述

（盒子和球模型）假設有3個盒子，每一個盒子裏裝有紅白兩種顏色的球，盒子裏的紅白球有下表列出，初始狀態分佈。

解答

1. 給出HMM模型 \(\mu=(A, B,\pi)\)

A，「狀態轉移矩陣」以下：
\[ \begin{array}{c|lcr} A & \text{box1} & \text{box2} & \text{box3} \\ \hline box1 & 0.5 & 0.2 & 0.3 \\ box2 & 0.3 & 0.5 & 0.2 \\ box3 & 0.2 & 0.3 & 0.5 \end{array} \]

B，「符號發射機率」以下矩陣：
\[ \begin{array}{c|lcr} B & \text{red} & \text{white} \\ \hline box1 & \frac{1}{2} & \frac{1}{2} \\ box2 & \frac{2}{5} & \frac{3}{5} \\ box3 & \frac{7}{10} & \frac{3}{10} \end{array} \]

\(\pi\)，「初始狀態的機率分佈」以下矩陣
\[ \begin{array}{c|cr} & \text{box1} & \text{box2} & \text{box3} \\ \hline \pi & 0.2 & 0.4 & 0.4 \end{array} \]

2. 試用前向算法計算觀測序列機率：{紅，白，紅}

O = {紅， 白， 紅}

step 1 初始化

\(a_{紅}(box1) = \pi_{box1} b_{box1}(紅) = 0.2 \times \frac{1}{2} = 0.1\)

\(a_{紅}(box2) = \pi_{box2} b_{box2}(紅) = 0.4 \times \frac{2}{5} = 0.16\)

\(a_{紅}(box3) = \pi_{box3} b_{box3}(紅) = 0.4 \times \frac{7}{10} = 0.28\)

step 2 概括計算

\[ \begin{align} a_{紅白}(box1) & = \left[ \sum_{i=1}^{3} a_{紅}(box_{i})a_{box_{i} box_{1}} \right] b_{box_{1}}(白) \\ & =\left[ a_{紅}(box1) \times 0.5 + a_{紅}(box2) \times 0.3 + a_{紅}(box3) \times 0.2 \right] \times \frac{1}{2} \\ & = 0.077 \end{align} \]

\[ \begin{align} a_{紅白}(box2) & = \left[ \sum_{i=1}^{3} a_{紅}(box_{i})a_{box_{i} box_{2}} \right] b_{box2}(白) \\ & = \left[ a_{紅}(box1) \times 0.2 + a_{紅}(box2) \times 0.5 + a_{紅}(box3) \times 0.3 \right] \times \frac{3}{5} \\ & = 0.1104 \end{align} \]

\[ \begin{align} a_{紅白}(box3) & = \left[ \sum_{i=1}^{3} a_{紅}(box_{i})a_{box_{i} box_{3}} \right] b_{box_{3}}(白) \\ & = \left[ a_{紅}(box1) \times 0.3 + a_{紅}(box2) \times 0.2 + a_{紅}(box3) \times 0.5 \right] \times \frac{3}{10} \\ & = 0.0606 \end{align} \]

\[ \begin{align} a_{紅白紅}(box1) & = \left[ \sum_{i=1}^{3} a_{紅白}(box_{i}) a_{box_{i}box1} \right] b_{box1}(紅) \\ & = \left[ a_{紅白}(box1) \times 0.5 + a_{紅白}(box2) \times 0.3 + a_{紅白}(box3) \times 0.2 \right] \times \frac{1}{2} \\ & = 0.04187 \end{align} \]

\[ \begin{align} a_{紅白紅}(box2) & = \left[ \sum_{i=1}^{3} a_{紅白}(box_{i}) a_{box_{i}box2} \right] b_{box2}(紅) \\ & = \left[ a_{紅白}(box1) \times 0.2 + a_{紅白}(box2) \times 0.5 + a_{紅白}(box3) \times 0.3 \right] \times \frac{2}{5} \\ & = 0.035512 \end{align} \]

\[ \begin{align} a_{紅白紅}(box3) & = \left[ \sum_{i=1}^{3} a_{紅白}(box_{i}) a_{box_{i}box3} \right] b_{box3}(紅) \\ & = \left[ a_{紅白}(box1) \times 0.3 + a_{紅白}(box2) \times 0.2 + a_{紅白}(box3) \times 0.5 \right] \times \frac{7}{10} \\ & = 0.052836 \end{align} \]

step 3 求和終結

\[ \begin{align} P(O | \mu) & = \sum_{i=1}^{3}a_{紅白紅} (box_{i}) \\ & = a_{紅白紅}(box1) + a_{紅白紅}(box2) + a_{紅白紅}(box3) \\ & = 0.04187 + 0.035512 + 0.052836 \\ & = 0.130218 \end{align} \]

3. 試用維特比算法，求解觀測序列{紅，白，紅}的最優狀態序列。 （要求給詳細計算過程）

假設從盒子中取球是放回抽樣機率模型。

step 1 初始化

\(\delta_{紅}(box1) = \pi_{box1}b_{box1}(紅) = 0.2 \times \frac{1}{2} = 0.1\)

\(\delta_{紅}(box2) = \pi_{box2}b_{box2}(紅) = 0.4 \times \frac{2}{5} = 0.16\)

\(\delta_{紅}(box3) = \pi_{box3}b_{box3}(紅) = 0.4 \times \frac{7}{10} = 0.28\)

\(\varphi_{紅}(box1) = 0\)

\(\varphi_{紅}(box2) = 0\)

\(\varphi_{紅}(box3) = 0\)

step 2 概括計算 1

\[ \begin{align} \delta_{紅白}(box1) & = max \left[ \delta_{紅}(box1)a_{box1box1} , \delta_{紅}(box2)a_{box2box1} , \delta_{紅}(box3)a_{box3box1} \right] b_{box1}(白) \\ & = 0.28 \times 0.2 \times \frac{1}{2} \\ & = 0.028 \end{align} \]

\[ \begin{align} \delta_{紅白}(box2) & = max \left[ \delta_{紅}(box1)a_{box1box2} , \delta_{紅}(box2)a_{box2box2} , \delta_{紅}(box3)a_{box3box2} \right] b_{box1}(白) \\ & = 0.28 \times 0.3 \times \frac{3}{5} \\ & = 0.0504 \end{align} \]

\[ \begin{align} \delta_{紅白}(box3) & = max \left[ \delta_{紅}(box1)a_{box1box3} , \delta_{紅}(box2)a_{box2box3} , \delta_{紅}(box3)a_{box3box3} \right] b_{box3}(白) \\ & = 0.28 \times 0.5 \times \frac{3}{10} \\ & = 0.042 \end{align} \]

記憶回退路徑 1

\[ \begin{align} \varphi_{紅白}(box1) & = argmax \left[ \delta_{紅}(box1)a_{box1box1} , \delta_{紅}(box2)a_{box2box1} , \delta_{紅}(box3)a_{box3box1} \right] b_{box1}(白) \\ & = \frac{1}{2} \delta_{紅}(box3)a_{box3box1} \end{align} \]

\[ \begin{align} \varphi_{紅白}(box2) & = argmax \left[ \delta_{紅}(box1)a_{box1box2} , \delta_{紅}(box2)a_{box2box2} , \delta_{紅}(box3)a_{box3box2} \right] b_{box2}(白) \\ & = \frac{3}{5} \delta_{紅}(box3) a_{box3box2} \end{align} \]

\[ \begin{align} \delta_{紅白}(box3) & = argmax \left[ \delta_{紅}(box1)a_{box1box3} , \delta_{紅}(box2)a_{box2box3} , \delta_{紅}(box3)a_{box3box3} \right] b_{box3}(白) \\ & = \frac{3}{10} \delta_{紅}(box3)a_{box3box3} \\ \end{align} \]

概括計算 2

\[ \begin{align} \delta_{紅白紅}(box1) & = max \left[ \delta_{紅白}(box1)a_{box1box1} , \delta_{紅白}(box2)a_{box2box1} , \delta_{紅白}(box3)a_{box3box1} \right] b_{box1}(紅) \\ & = 0.0504 \times 0.3 \times \frac{1}{2} \\ & = 0.0756 \end{align} \]

\[ \begin{align} \delta_{紅白紅}(box2) & = max \left[ \delta_{紅白}(box1)a_{box1box2} , \delta_{紅白}(box2)a_{box2box2} , \delta_{紅白}(box3)a_{box3box2} \right] b_{box1}(紅) \\ & = 0.0504 \times 0.5 \times \frac{2}{5} \\ & = 0.01008 \end{align} \]

\[ \begin{align} \delta_{紅白紅}(box3) & = max \left[ \delta_{紅白}(box1)a_{box1box3} , \delta_{紅白}(box2)a_{box2box3} , \delta_{紅白}(box3)a_{box3box3} \right] b_{box3}(紅) \\ & = 0.042 \times 0.5 \times \frac{7}{10} \\ & = 0.0147 \end{align} \]

記憶回退路徑 2

\[ \begin{align} \varphi_{紅白紅}(box1) & = argmax \left[ \delta_{紅白}(box1)a_{box1box1} , \delta_{紅白}(box2)a_{box2box1} , \delta_{紅白}(box3)a_{box3box1} \right] b_{box1}(紅) \\ & = \frac{1}{2} \delta_{紅}(box2)a_{box2box1} \end{align} \]

\[ \begin{align} \varphi_{紅白紅}(box2) & = argmax \left[ \delta_{紅白}(box1)a_{box1box2} , \delta_{紅白}(box2)a_{box2box2} , \delta_{紅白}(box3)a_{box3box2} \right] b_{box2}(紅) \\ & = \frac{2}{5} \delta_{紅白}(box2) a_{box2box2} \end{align} \]

\[ \begin{align} \delta_{紅白紅}(box3) & = argmax \left[ \delta_{紅白}(box1)a_{box1box3} , \delta_{紅白}(box2)a_{box2box3} , \delta_{紅白}(box3)a_{box3box3} \right] b_{box3}(白) \\ & = \frac{7}{10} \delta_{紅白}(box3)a_{box3box3} \\ \end{align} \]

step 3 終結

\[ \begin{align} \hat{Q}_{紅白紅} & = argmax \left[ \delta_{紅白紅}(box1) , \delta_{紅白紅}(box2) , \delta_{紅白紅}(box3) \right] \\ & = \delta_{紅白紅}(box3) \end{align} \]

\[ \begin{align} \hat{P}(\hat{Q}_{紅白紅}) & = max \left[ \delta_{紅白紅}(box1) , \delta_{紅白紅}(box2) , \delta_{紅白紅}(box3) \right] \\ & = 0.0147 \end{align} \]

step 4 路徑（狀態序列)回溯

\[ \begin{align} \hat{q}_{紅白} = \varphi_{紅白紅}(\hat{q}_{紅白紅}) = \varphi_{紅白紅} \left[ \delta_{紅白紅}(box3) \right] \end{align} \]

\[ \begin{align} \hat{q}_{紅} = \varphi_{紅白}(\hat{q}_{紅白}) = \varphi_{紅白} \left[ \delta_{紅白}(box2) \right] \end{align} \]

\[ \begin{align} \hat{q} = \varphi_{紅}(\hat{q}_{紅}) = \varphi_{紅} \left[ \delta_{紅}(box3) \right] \end{align} \]