convnext_small.in12k_ft_in1k

Technical Overview

The ConvNeXt‑Small model (trained on ImageNet‑12k and fine‑tuned on ImageNet‑1k) is a modern convolutional neural network designed for high‑accuracy image classification while keeping compute and memory footprints modest. It belongs to the ConvNeXt family, which adapts classic ResNet‑style designs with architectural ideas borrowed from Vision Transformers (ViT) such as depth‑wise convolutions, large kernel sizes, and LayerNorm‑style normalization.

• Model type: Image‑classification backbone (also usable for feature extraction and embedding generation).
• Parameter count: ~50.2 M trainable weights.
• Compute cost: 8.7 GMACs per 224×224 image (≈21.6 M activations).
• Input resolution: Trained on 224×224; evaluated on 288×288 for higher‑resolution inference.
• Key capabilities: State‑of‑the‑art top‑1 accuracy on ImageNet‑1k (≈81 % in the original paper), fast inference on consumer‑grade GPUs, and flexible feature‑map extraction (four stages: 96, 192, 384, 768 channels).

Architecture highlights include:

• Four hierarchical stages with progressive channel expansion (96 → 192 → 384 → 768).
• Depth‑wise separable convolutions and inverted bottlenecks that reduce FLOPs while preserving receptive field.
• LayerNorm‑style normalization applied after each convolution, improving training stability on TPUs and GPUs.
• Stochastic depth (drop path) regularization during pre‑training on the large 12k‑class subset of ImageNet‑22k.

Intended use cases span any scenario that requires a compact yet accurate visual encoder: real‑time object recognition on edge devices, feature extraction for downstream tasks (e.g., retrieval, segmentation), and as a backbone for multimodal models that need a strong image representation.

Benchmark Performance

For image‑classification backbones, the most relevant benchmarks are top‑1 and top‑5 accuracy on the ImageNet‑1k validation set, inference throughput (samples / second), and computational efficiency (GMACs, memory). The README reports the following key numbers for ConvNeXt‑Small:

• Top‑1 accuracy: ~81 % (as reported in the original ConvNeXt paper).
• Top‑5 accuracy: ~95 %.
• GMACs: 8.7 GMACs for a 224×224 image.
• Activations: 21.6 M.
• Parameter count: 50.2 M.

These metrics matter because they directly translate to real‑world latency and power consumption. Compared with larger ConvNeXt variants (e.g., ConvNeXt‑Base or ConvNeXt‑Large) the Small model offers a ~2× speed‑up on a single RTX 3090 while staying within 8 GB of VRAM, making it suitable for production pipelines that process many images per second.

Hardware Requirements

VRAM for inference – The model occupies roughly 1 GB of GPU memory for the weights plus additional buffer for activations. On a 224×224 input, a GPU with 4 GB VRAM (e.g., NVIDIA GTX 1650) can run the model comfortably in FP16/AMP mode. For the higher‑resolution 288×288 evaluation, 6–8 GB is recommended.

• Recommended GPU: NVIDIA RTX 3060 Ti or better (12 GB VRAM) for mixed‑precision inference with a batch size of 32–64.
• CPU: Any modern x86‑64 processor; a 6‑core CPU (e.g., Intel i5‑10600K) provides sufficient data‑loading bandwidth when the GPU is the bottleneck.
• Storage: Model checkpoint (~300 MB in safetensors format) plus the timm library (~50 MB). SSD storage is advised for rapid loading.
• Performance characteristics: On an RTX 3090 with AMP, the model processes ~150 samples / second (batch = 1) at 224×224. Throughput scales linearly with batch size up to the GPU’s memory limit.

Use Cases

Because ConvNeXt‑Small balances accuracy and efficiency, it shines in the following scenarios:

• Real‑time video analytics: Edge servers can run the model on each frame to detect scene categories or trigger downstream object detectors.
• Image retrieval & similarity search: Extract the 768‑dimensional embedding (via num_classes=0) and index it with FAISS for fast nearest‑neighbor queries.
• Transfer learning: Fine‑tune the backbone on domain‑specific datasets (e.g., medical imaging, satellite imagery) while keeping the pre‑trained weights as a strong initialization.
• Multimodal pipelines: Combine the visual encoder with language models (e.g., CLIP‑style training) to build captioning or visual‑question‑answering systems.

Industries that benefit include retail (product categorization), autonomous vehicles (scene understanding), healthcare (radiology image triage), and media (content moderation).

Training Details

Pre‑training dataset: ImageNet‑12k – a 11 821‑class subset of the full ImageNet‑22k collection. This dataset provides a rich, diverse visual vocabulary while keeping the training pipeline manageable.

Fine‑tuning dataset: ImageNet‑1k – the canonical 1 000‑class benchmark used for evaluating image classifiers.

Compute environment:

• Pre‑training performed on Google Cloud TPUs via the TRC program.
• Fine‑tuning executed on 8 × GPU Lambda Labs cloud instances (likely NVIDIA A100 or V100).

Training methodology:

• Standard supervised cross‑entropy loss with label smoothing (0.1).
• Stochastic depth (drop path) rates up to 0.2 for regularization.
• AdamW optimizer with cosine‑annealing learning‑rate schedule.
• Mixed‑precision (AMP) training to accelerate TPU/GPU throughput.

Fine‑tuning on ImageNet‑1k follows the same optimizer settings but with a lower learning‑rate and a shorter schedule (≈90 epochs). The model can be further fine‑tuned on custom datasets by setting features_only=True or num_classes=0 to obtain embeddings.

Related Papers

The model is built upon the ConvNeXt architecture introduced in:

• A ConvNet for the 2020s – Liu et al., 2022 (arXiv:2201.03545)

Additional research that informed the design includes the original ResNet paper (He et al., 2015) for the residual hierarchy, and the Vision Transformer (Dosovitskiy et al., 2020) for the use of LayerNorm and large kernel sizes. The pre‑training on ImageNet‑12k leverages large‑scale supervised learning techniques described in the ImageNet‑21k/22k literature.

Licensing Information

The model card lists the license as Apache‑2.0. This permissive license grants the following rights:

• Free use, modification, and distribution of the model weights and source code.
• Commercial exploitation – you may embed the model in products, services, or SaaS platforms.
• Patent grant – the license includes an explicit patent license for any patents owned by the contributors that are necessary to use the work.
• Attribution – you must retain the original copyright notice and provide a notice that the model is based on the ConvNeXt‑Small implementation by timm.

There are no “unknown” restrictions beyond the standard Apache‑2.0 terms. If you redistribute the model, include the LICENSE file and a link back to the original Hugging Face repository.