\begin{equation}
\begin{aligned}
\ net_{o_{11}}&= conv (input,filter)\\
&= i_{11} \times h_{11} + i_{12} \times h_{12} +i_{21} \times h_{21} + i_{22} \times h_{22}\\
out_{o_{11}} &= activators(net_{o_{11}}) \\
&=max(0,net_{o_{11}})
\end{aligned}
\end{equation}
首先计算输入层的误差项$\delta_{11}$
$$\delta_{11} = \frac{\partial E}{\partial net_{o_{11}}} =\frac{\partial E}{\partial out_{o_{11}}} \cdot \frac{\partial out_{o_{11}}}{\partial net_{o_{11}}}$$
先计算$\frac{\partial E}{\partial out_{o_{11}}}$
$i_{11}$的偏导：
\begin{equation}
\begin{aligned}
\because net_{o_{11}}&= conv (input,filter)\\
&= i_{11} \times h_{11} + i_{12} \times h_{12} +i_{21} \times h_{21} + i_{22} \times h_{22}\\
\therefore \frac{\partial E}{\partial i_{11}}&=\frac{\partial E}{\partial net_{o_{11}}} \cdot \frac{\partial net_{o_{11}}}{\partial i_{11}}\\
&=\delta_{11} \cdot h_{11}
\end{aligned}
\end{equation}
$i_{12}$的偏导：
\begin{equation}
\begin{aligned}
\because net_{o_{11}}&= conv (input,filter)\\
&= i_{11} \times h_{11} + i_{12} \times h_{12} +i_{21} \times h_{21} + i_{22} \times h_{22}\\
net_{o_{12}}&= conv (input,filter)\\
&= i_{12} \times h_{11} + i_{13} \times h_{12} +i_{22} \times h_{21} + i_{23} \times h_{22}\\
\therefore \frac{\partial E}{\partial i_{12}}&=\frac{\partial E}{\partial net_{o_{11}}} \cdot \frac{\partial net_{o_{11}}}{\partial i_{12}} +\frac{\partial E}{\partial net_{o_{12}}} \cdot \frac{\partial net_{o_{12}}}{\partial i_{12}}\\
&=\delta_{11} \cdot h_{12}+\delta_{12} \cdot h_{11}
\end{aligned}
\end{equation}
$i_{22}的偏导$：
\begin{equation}
\begin{aligned}
\because &i_{21}与net_{o_{11}}、net_{o_{12}}、net_{o_{21}}、net_{o_{22}}有关\\
\therefore
\frac{\partial E}{\partial i_{22}}&=\frac{\partial E}{\partial net_{o_{11}}} \cdot \frac{\partial net_{o_{11}}}{\partial i_{22}} +\frac{\partial E}{\partial net_{o_{12}}} \cdot \frac{\partial net_{o_{12}}}{\partial i_{22}}\\
&+\frac{\partial E}{\partial net_{o_{21}}} \cdot \frac{\partial net_{o_{21}}}{\partial i_{22}}+\frac{\partial E}{\partial net_{o_{22}}} \cdot \frac{\partial net_{o_{22}}}{\partial i_{22}}\\
&=\delta_{11} \cdot h_{22}+\delta_{12} \cdot h_{21}+\delta_{21} \cdot h_{12}+\delta_{22} \cdot h_{11}
\end{aligned}
\end{equation}
观察以上几个式子可以得出，想要计算误差敏感项${\delta_{i,j}}，即$$\frac{\partial E}{\partial net_{o_{11}}}$，相当于就是把这个误差敏感矩阵的周围补了一圈零，与进行180°翻转后的卷积核filter进行卷积运算,则可以计算
$$\frac{\partial E}{\partial out_{i,j}} = \sum_m \cdot \sum_n h_{m,n}\delta_{i+m,j+n}$$
此时我们的误差敏感矩阵就求完了，得到误差敏感矩阵后，即可求权重的梯度
权重的梯度：
\begin{equation}
\begin{aligned}
\frac{\partial E}{\partial w_{i,j}} = \sum_m\sum_n\delta_{m,n}out_{o_{i+m,j+n}}
\end{aligned}
\end{equation}
误差项的梯度：
\begin{equation}
\begin{aligned}
\because out_{o_{11}} &= activators(net_{o_{11}})\\
&=f'(net_{o_{11}})\\
\therefore \delta_{i_{11}} &=\frac{\partial E}{\partial net_{o_{11}}} \\
&=\frac{\partial E}{\partial out_{o_{11}}} \cdot \frac{\partial out_{o_{11}}}{\partial net_{o_{11}}}\\
&=\sum_m \cdot \sum_n h_{m,n}\delta_{i+m,j+n} \cdot f'(net_{o_{11}})
\end{aligned}
\end{equation}
得到了权重和偏置项的梯度后，就可以根据梯度下降法更新权重和梯度了。
池化层的反向传播
公式太多了，留个坑，等会再写
手写一个卷积神经网络
1.定义一个卷积层
首先我们通过ConvLayer来实现一个卷积层，定义卷积层的超参数
1 class ConvLayer(object): 2 ''' 3 参数含义： 4 input_width:输入图片尺寸——宽度 5 input_height:输入图片尺寸——长度 6 channel_number:通道数，彩色为3，灰色为1 7 filter_width:卷积核的宽 8 filter_height:卷积核的长 9 filter_number:卷积核数量 10 zero_padding：补零长度 11 stride:步长 12 activator:激活函数 13 learning_rate:学习率 14 ''' 15 def __init__(self, input_width, input_height, 16 channel_number, filter_width, 17 filter_height, filter_number, 18 zero_padding, stride, activator, 19 learning_rate): 20 self.input_width = input_width 21 self.input_height = input_height 22 self.channel_number = channel_number 23 self.filter_width = filter_width 24 self.filter_height = filter_height 25 self.filter_number = filter_number 26 self.zero_padding = zero_padding 27 self.stride = stride 28 self.output_width = \ 29 ConvLayer.calculate_output_size( 30 self.input_width, filter_width, zero_padding, 31 stride) 32 self.output_height = \ 33 ConvLayer.calculate_output_size( 34 self.input_height, filter_height, zero_padding, 35 stride) 36 self.output_array = np.zeros((self.filter_number, 37 self.output_height, self.output_width)) 38 self.filters = [] 39 for i in range(filter_number): 40 self.filters.append(Filter(filter_width, 41 filter_height, self.channel_number)) 42 self.activator = activator 43 self.learning_rate = learning_rate