> For the complete documentation index, see [llms.txt](https://boinc-ai.gitbook.io/timm/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://boinc-ai.gitbook.io/timm/model-pages/model-summaries.md).

# Model Summaries

## Model Summaries

The model architectures included come from a wide variety of sources. Sources, including papers, original impl (“reference code”) that I rewrote / adapted, and PyTorch impl that I leveraged directly (“code”) are listed below.

Most included models have pretrained weights. The weights are either:

1. from their original sources
2. ported by myself from their original impl in a different framework (e.g. Tensorflow models)
3. trained from scratch using the included training script

The validation results for the pretrained weights are [here](https://huggingface.co/docs/timm/results)

A more exciting view (with pretty pictures) of the models within `timm` can be found at [paperswithcode](https://paperswithcode.com/lib/timm).

### Big Transfer ResNetV2 (BiT)

* Implementation: [resnetv2.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/resnetv2.py)
* Paper: `Big Transfer (BiT): General Visual Representation Learning` - <https://arxiv.org/abs/1912.11370>
* Reference code: <https://github.com/google-research/big_transfer>

### Cross-Stage Partial Networks

* Implementation: [cspnet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/cspnet.py)
* Paper: `CSPNet: A New Backbone that can Enhance Learning Capability of CNN` - <https://arxiv.org/abs/1911.11929>
* Reference impl: <https://github.com/WongKinYiu/CrossStagePartialNetworks>

### DenseNet

* Implementation: [densenet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/densenet.py)
* Paper: `Densely Connected Convolutional Networks` - <https://arxiv.org/abs/1608.06993>
* Code: <https://github.com/pytorch/vision/tree/master/torchvision/models>

### DLA

* Implementation: [dla.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/dla.py)
* Paper: <https://arxiv.org/abs/1707.06484>
* Code: <https://github.com/ucbdrive/dla>

### Dual-Path Networks

* Implementation: [dpn.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/dpn.py)
* Paper: `Dual Path Networks` - <https://arxiv.org/abs/1707.01629>
* My PyTorch code: <https://github.com/rwightman/pytorch-dpn-pretrained>
* Reference code: <https://github.com/cypw/DPNs>

### GPU-Efficient Networks

* Implementation: [byobnet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/byobnet.py)
* Paper: `Neural Architecture Design for GPU-Efficient Networks` - <https://arxiv.org/abs/2006.14090>
* Reference code: <https://github.com/idstcv/GPU-Efficient-Networks>

### HRNet

* Implementation: [hrnet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/hrnet.py)
* Paper: `Deep High-Resolution Representation Learning for Visual Recognition` - <https://arxiv.org/abs/1908.07919>
* Code: <https://github.com/HRNet/HRNet-Image-Classification>

### Inception-V3

* Implementation: [inception\_v3.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/inception_v3.py)
* Paper: `Rethinking the Inception Architecture for Computer Vision` - <https://arxiv.org/abs/1512.00567>
* Code: <https://github.com/pytorch/vision/tree/master/torchvision/models>

### Inception-V4

* Implementation: [inception\_v4.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/inception_v4.py)
* Paper: `Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning` - <https://arxiv.org/abs/1602.07261>
* Code: <https://github.com/Cadene/pretrained-models.pytorch>
* Reference code: <https://github.com/tensorflow/models/tree/master/research/slim/nets>

### Inception-ResNet-V2

* Implementation: [inception\_resnet\_v2.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/inception_resnet_v2.py)
* Paper: `Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning` - <https://arxiv.org/abs/1602.07261>
* Code: <https://github.com/Cadene/pretrained-models.pytorch>
* Reference code: <https://github.com/tensorflow/models/tree/master/research/slim/nets>

### NASNet-A

* Implementation: [nasnet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/nasnet.py)
* Papers: `Learning Transferable Architectures for Scalable Image Recognition` - <https://arxiv.org/abs/1707.07012>
* Code: <https://github.com/Cadene/pretrained-models.pytorch>
* Reference code: <https://github.com/tensorflow/models/tree/master/research/slim/nets/nasnet>

### PNasNet-5

* Implementation: [pnasnet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/pnasnet.py)
* Papers: `Progressive Neural Architecture Search` - <https://arxiv.org/abs/1712.00559>
* Code: <https://github.com/Cadene/pretrained-models.pytorch>
* Reference code: <https://github.com/tensorflow/models/tree/master/research/slim/nets/nasnet>

### EfficientNet

* Implementation: [efficientnet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/efficientnet.py)
* Papers:
  * EfficientNet NoisyStudent (B0-B7, L2) - <https://arxiv.org/abs/1911.04252>
  * EfficientNet AdvProp (B0-B8) - <https://arxiv.org/abs/1911.09665>
  * EfficientNet (B0-B7) - <https://arxiv.org/abs/1905.11946>
  * EfficientNet-EdgeTPU (S, M, L) - <https://ai.googleblog.com/2019/08/efficientnet-edgetpu-creating.html>
  * MixNet - <https://arxiv.org/abs/1907.09595>
  * MNASNet B1, A1 (Squeeze-Excite), and Small - <https://arxiv.org/abs/1807.11626>
  * MobileNet-V2 - <https://arxiv.org/abs/1801.04381>
  * FBNet-C - <https://arxiv.org/abs/1812.03443>
  * Single-Path NAS - <https://arxiv.org/abs/1904.02877>
* My PyTorch code: <https://github.com/rwightman/gen-efficientnet-pytorch>
* Reference code: <https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet>

### MobileNet-V3

* Implementation: [mobilenetv3.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/mobilenetv3.py)
* Paper: `Searching for MobileNetV3` - <https://arxiv.org/abs/1905.02244>
* Reference code: <https://github.com/tensorflow/models/tree/master/research/slim/nets/mobilenet>

### RegNet

* Implementation: [regnet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/regnet.py)
* Paper: `Designing Network Design Spaces` - <https://arxiv.org/abs/2003.13678>
* Reference code: <https://github.com/facebookresearch/pycls/blob/master/pycls/models/regnet.py>

### RepVGG

* Implementation: [byobnet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/byobnet.py)
* Paper: `Making VGG-style ConvNets Great Again` - <https://arxiv.org/abs/2101.03697>
* Reference code: <https://github.com/DingXiaoH/RepVGG>

### ResNet, ResNeXt

* Implementation: [resnet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/resnet.py)
* ResNet (V1B)
  * Paper: `Deep Residual Learning for Image Recognition` - <https://arxiv.org/abs/1512.03385>
  * Code: <https://github.com/pytorch/vision/tree/master/torchvision/models>
* ResNeXt
  * Paper: `Aggregated Residual Transformations for Deep Neural Networks` - <https://arxiv.org/abs/1611.05431>
  * Code: <https://github.com/pytorch/vision/tree/master/torchvision/models>
* ‘Bag of Tricks’ / Gluon C, D, E, S ResNet variants
  * Paper: `Bag of Tricks for Image Classification with CNNs` - <https://arxiv.org/abs/1812.01187>
  * Code: <https://github.com/dmlc/gluon-cv/blob/master/gluoncv/model_zoo/resnetv1b.py>
* Instagram pretrained / ImageNet tuned ResNeXt101
  * Paper: `Exploring the Limits of Weakly Supervised Pretraining` - <https://arxiv.org/abs/1805.00932>
  * Weights: <https://pytorch.org/hub/facebookresearch_WSL-Images_resnext> (NOTE: CC BY-NC 4.0 License, NOT commercial friendly)
* Semi-supervised (SSL) / Semi-weakly Supervised (SWSL) ResNet and ResNeXts
  * Paper: `Billion-scale semi-supervised learning for image classification` - <https://arxiv.org/abs/1905.00546>
  * Weights: <https://github.com/facebookresearch/semi-supervised-ImageNet1K-models> (NOTE: CC BY-NC 4.0 License, NOT commercial friendly)
* Squeeze-and-Excitation Networks
  * Paper: `Squeeze-and-Excitation Networks` - <https://arxiv.org/abs/1709.01507>
  * Code: Added to ResNet base, this is current version going forward, old `senet.py` is being deprecated
* ECAResNet (ECA-Net)
  * Paper: `ECA-Net: Efficient Channel Attention for Deep CNN` - <https://arxiv.org/abs/1910.03151v4>
  * Code: Added to ResNet base, ECA module contributed by @VRandme, reference <https://github.com/BangguWu/ECANet>

### Res2Net

* Implementation: [res2net.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/res2net.py)
* Paper: `Res2Net: A New Multi-scale Backbone Architecture` - <https://arxiv.org/abs/1904.01169>
* Code: <https://github.com/gasvn/Res2Net>

### ResNeSt

* Implementation: [resnest.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/resnest.py)
* Paper: `ResNeSt: Split-Attention Networks` - <https://arxiv.org/abs/2004.08955>
* Code: <https://github.com/zhanghang1989/ResNeSt>

### ReXNet

* Implementation: [rexnet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/rexnet.py)
* Paper: `ReXNet: Diminishing Representational Bottleneck on CNN` - <https://arxiv.org/abs/2007.00992>
* Code: <https://github.com/clovaai/rexnet>

### Selective-Kernel Networks

* Implementation: [sknet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/sknet.py)
* Paper: `Selective-Kernel Networks` - <https://arxiv.org/abs/1903.06586>
* Code: <https://github.com/implus/SKNet>, <https://github.com/clovaai/assembled-cnn>

### SelecSLS

* Implementation: [selecsls.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/selecsls.py)
* Paper: `XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera` - <https://arxiv.org/abs/1907.00837>
* Code: <https://github.com/mehtadushy/SelecSLS-Pytorch>

### Squeeze-and-Excitation Networks

* Implementation: [senet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/senet.py) NOTE: I am deprecating this version of the networks, the new ones are part of `resnet.py`
* Paper: `Squeeze-and-Excitation Networks` - <https://arxiv.org/abs/1709.01507>
* Code: <https://github.com/Cadene/pretrained-models.pytorch>

### TResNet

* Implementation: [tresnet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/tresnet.py)
* Paper: `TResNet: High Performance GPU-Dedicated Architecture` - <https://arxiv.org/abs/2003.13630>
* Code: <https://github.com/mrT23/TResNet>

### VGG

* Implementation: [vgg.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vgg.py)
* Paper: `Very Deep Convolutional Networks For Large-Scale Image Recognition` - <https://arxiv.org/pdf/1409.1556.pdf>
* Reference code: <https://github.com/pytorch/vision/blob/master/torchvision/models/vgg.py>

### Vision Transformer

* Implementation: [vision\_transformer.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vision_transformer.py)
* Paper: `An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale` - <https://arxiv.org/abs/2010.11929>
* Reference code and pretrained weights: <https://github.com/google-research/vision_transformer>

### VovNet V2 and V1

* Implementation: [vovnet.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/vovnet.py)
* Paper: `CenterMask : Real-Time Anchor-Free Instance Segmentation` - <https://arxiv.org/abs/1911.06667>
* Reference code: <https://github.com/youngwanLEE/vovnet-detectron2>

### Xception

* Implementation: [xception.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/xception.py)
* Paper: `Xception: Deep Learning with Depthwise Separable Convolutions` - <https://arxiv.org/abs/1610.02357>
* Code: <https://github.com/Cadene/pretrained-models.pytorch>

### Xception (Modified Aligned, Gluon)

* Implementation: [gluon\_xception.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/gluon_xception.py)
* Paper: `Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation` - <https://arxiv.org/abs/1802.02611>
* Reference code: <https://github.com/dmlc/gluon-cv/tree/master/gluoncv/model_zoo>, <https://github.com/jfzhang95/pytorch-deeplab-xception/>

### Xception (Modified Aligned, TF)

* Implementation: [aligned\_xception.py](https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/aligned_xception.py)
* Paper: `Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation` - <https://arxiv.org/abs/1802.02611>
* Reference code: <https://github.com/tensorflow/models/tree/master/research/deeplab>