# 2 前向传播

• $n\times n$ 的卷积核参数：$\boldsymbol{W}_{n\times n}$
• 偏置：$b$

• 输入 $p\times q$ 大小的矩阵：$\boldsymbol{G}_{p\times q}$
• 输出 $(p-2)\times(q-2)$ 大小的矩阵：$\boldsymbol{H}_{(p-2)\times (q-2)}$

$$\boldsymbol{H}_{(p-2)\times (q-2)}=\boldsymbol{G}_{p\times q}*\boldsymbol{W}_{n\times n}+b$$

# 3 反向传播

## 3.1 对 $\boldsymbol{W}$​ 求偏导

$$\frac{\partial E}{\partial \boldsymbol{W}}$$

$$\frac{\partial E}{\partial w_{ij}}=\sum_{p=0}^{n-1}\sum_{q=0}^{n-1}\frac{\partial E}{\partial h_{pq}}\frac{\partial h_{pq}}{\partial w_{ij}}$$

$$h_{ij}=\sum_{\alpha=0}^{n-1}\sum_{\beta=0}^{n-1}g_{i+\alpha-\frac{n-1}{2},j+\beta-\frac{n-1}{2}}\cdot w_{\alpha\beta}+b$$

$$\frac{\partial h_{pq}}{\partial w_{ij}}=\frac{\partial \sum_{\alpha=0}^{n-1}\sum_{\beta=0}^{n-1}g_{p+\alpha-\frac{n-1}{2},q+\beta-\frac{n-1}{2}}\cdot w_{\alpha\beta}+b}{\partial w_{ij}}=g_{p+i-\frac{n-1}{2},q+j-\frac{n-1}{2}}$$

$$\frac{\partial E}{\partial w_{ij}}=\sum_{p=0}^{n-1}\sum_{q=0}^{n-1}\frac{\partial E}{\partial h_{pq}}\frac{\partial h_{pq}}{\partial w_{ij}}=\sum_{p=0}^{n-1}\sum_{q=0}^{n-1}g_{p+i-\frac{n-1}{2},q+j-\frac{n-1}{2}}\frac{\partial E}{\partial h_{pq}}$$

$$\frac{\partial E}{\partial \boldsymbol{W}}=\boldsymbol{G}*\frac{\partial E}{\partial \boldsymbol{H}}$$

## 3.2 对 $b$ 求偏导

$$h_{ij}=\sum_{\alpha=0}^{n-1}\sum_{\beta=0}^{n-1}g_{i+\alpha-\frac{n-1}{2},j+\beta-\frac{n-1}{2}}\cdot w_{\alpha\beta}+b$$

$$\frac{\partial h_{ij}}{\partial b} = 1$$

$$\frac{\partial E}{\partial d}=\sum_{i,j}\frac{\partial E}{\partial h_{ij}}\frac{\partial h_{ij}}{\partial d}=\sum_{i,j}\frac{\partial E}{\partial h_{ij}}$$

## 3.3 对 $\boldsymbol{G}$ 求偏导

$$\frac{\partial E}{\partial \boldsymbol{G}}$$

$$\frac{\partial E}{\partial g_{ij}}=\sum_{p=0}^{n-1}\sum_{q=0}^{n-1}\frac{\partial E}{\partial h_{pq}}\frac{\partial h_{pq}}{\partial g_{ij}}$$

$$h_{ij}=\sum_{\alpha=0}^{n-1}\sum_{\beta=0}^{n-1}g_{i+\alpha-\frac{n-1}{2},j+\beta-\frac{n-1}{2}}\cdot w_{\alpha\beta}+b$$

$$\frac{\partial h_{pq}}{\partial g_{ij}}=\frac{\partial \sum_{\alpha=0}^{n-1}\sum_{\beta=0}^{n-1}g_{p+\alpha-\frac{n-1}{2},q+\beta-\frac{n-1}{2}}\cdot w_{\alpha\beta}+b}{\partial g_{ij}}=w_{i-p+\frac{n-1}{2},j-q+\frac{n-1}{2}}$$

$$\frac{\partial E}{\partial g_{ij}}=\sum_{p=0}^{n-1}\sum_{q=0}^{n-1}w_{i-p+\frac{n-1}{2},j-q+\frac{n-1}{2}}\frac{\partial E}{\partial h_{pq}}$$

$$\frac{\partial E}{\partial \boldsymbol{G}}=\boldsymbol{W}*\frac{\partial E}{\partial \boldsymbol{H}}$$

# 4 代码实现

https://github.com/ChrisKimZHT/Neural-Net-cpp

Matrix ConvolutionalLayer::backward(const Matrix &d_output, double learning_rate) {
Matrix d_input = d_output.convolution(weights.transpose(), false);
Matrix d_filters = input.convolution(d_output.transpose(), true);
double d_bias = d_output.sum();

weights -= d_filters * learning_rate;
bias -= d_bias * learning_rate;

return d_input;
}