almost 3 years ago

## 類神經網路(Artificial Nueral Network)

### 理論基礎

$$w = (2,2), b=-3$$

1) $x=(1,0), \rightarrow z = \sum_i w_i x_i + b = 2+0-3 =-1 \rightarrow 0$
2) $x=(0,0), \rightarrow z = \sum_i w_i x_i + b = 0+0-3 =-3 \rightarrow 0$
3) $x=(0,1), \rightarrow z = \sum_i w_i x_i + b = 0+2-3 =-1 \rightarrow 0$
4) $x=(1,1), \rightarrow z = \sum_i w_i x_i + b = 2+2-3 =3 \rightarrow 1$

$$\frac{1}{2} | y - \sigma(z)|^2$$

$$\frac{1}{n} \sum_{x=1}^{n} C_x$$ 一般來說$C_x$是一張「網」

$$\delta^l = \left( \left( w^{l+1}\right)^T \delta^{l+1}\right) \odot \sigma'(z^l) \tag{2}$$

$$\frac{\partial C}{\partial b_j^l} = \delta_j^l \tag{3}$$
$$\frac{\partial C}{\partial w_{jk}^l} = \delta^l_j \left( a_k^{l-1} \right) \tag{4}$$

### ANN 演算法實作

1. 輸入一組待訓練資料(training data)

2. 對每一個樣本 x, 決定對應的神經元活化後結果(activation) $a^{x,1}$ 按照以下流程計算

• 往前計算經過的每層神經元 z值 - 對每一層(layer) l=2,3,...L 計算 $$a^{x,l} = \sigma(z^{x,l})$$
• 輸出誤差(output error: 計算向量 $$\sigma^{x,L} = (a^{x,L} - y^{L}) \odot \sigma'(z^{x,L})$$
• 背傳播(back-propagation)求解每一層的誤差值 -- For each L,L-1,L-2 ...2 $$\begin{eqnarray} \delta^l = ((w^{l+1})^T \delta^{l+1}) \odot \sigma'(z^l) \end{eqnarray}$$
3. 隨機梯度下降法(Stochastic Gradient Descent) - 對每一層 l = L,L-1,... 2 更新
$$w^l \rightarrow w^l-\frac{\eta}{m} \sum_x \delta^{x,l} (a^{x,l-1})^T ,$$
$$b^l \rightarrow b^l-\frac{\eta}{m} \sum_x \delta^{x,l}$$