pytorch学习笔记-高阶篇（深度残差网络）

本篇主要记录一下深度残差网络的相关知识

一、Resnet提出

随着研究的深入，人们就想着是不是网络层数越高，深度越深，网络的性能就越好。但经过实验发现并不是如此。所以就引入了深度残差网络这一概念，初衷是想比如原来有22层，我加上8层，但是我要保证现在的这30层性能还要差，于是可以引一条short cut到22层的地方，保证如果这新加的8层如果没有带来更好的性能就沿用之前的22层所得。
图片描述

二、Resnet网络

class ResBlk(nn.Module):
    """
    resnet block
    """

    def __init__(self, ch_in, ch_out):
        """
        :param ch_in:
        :param ch_out:
        """
        super(ResBlk, self).__init__()

        self.conv1 = nn.Conv2d(ch_in, ch_out, kernel_size=3, stride=1, padding=1)
        self.bn1 = nn.BatchNorm2d(ch_out)
        self.conv2 = nn.Conv2d(ch_out, ch_out, kernel_size=3, stride=1, padding=1)
        self.bn2 = nn.BatchNorm2d(ch_out)

        self.extra = nn.Sequential()
        if ch_out != ch_in:
            # [b, ch_in, h, w] => [b, ch_out, h, w]
            self.extra = nn.Sequential(
                nn.Conv2d(ch_in, ch_out, kernel_size=1, stride=1),
                nn.BatchNorm2d(ch_out)
            )


    def forward(self, x):
        """
        :param x: [b, ch, h, w]
        :return:
        """
        out = F.relu(self.bn1(self.conv1(x)))
        out = self.bn2(self.conv2(out))
        # short cut.
        # extra module: [b, ch_in, h, w] => [b, ch_out, h, w]
        # element-wise add:
        out = self.extra(x) + out

        return out