ResNet (Residual Network) is a deep neural network architecture introduced by Kaiming He and colleagues in 2015. It uses residual connections to improve training, allowing the input of a layer to bypass one or more layers and be added directly to the output, helping mitigate the vanishing gradient problem in deep networks.
-
Residual Connections: These skip connections allow the network to learn identity mappings, making it easier to train very deep networks by preserving gradient flow during backpropagation.
-
Residual Blocks: The basic building block of ResNet, where the input is added directly to the output of a few layers. For example, in a two-layer residual block, the output is
$F(x) + x$ , where$F(x)$ is the transformation applied by the layers. -
Deep Architectures: ResNet models like ResNet-18, ResNet-34, ResNet-50, and ResNet-101 are named based on the number of layers. Despite their depth, these models are easier to train due to the residual connections.
- Enabling Deeper Networks: ResNet made it possible to train networks with hundreds of layers, achieving better performance on complex tasks.
In summary, ResNet is a groundbreaking architecture that revolutionized deep learning by enabling the training of very deep networks and setting new standards in image recognition.