convnext_tiny.in12k_ft_in1k

Technical Overview

What is this model? ConvNeXt‑Tiny is a compact, modern convolutional neural network designed for image‑classification tasks. The variant convnext_tiny.in12k_ft_in1k is first pre‑trained on a 12 k‑class subset of the full ImageNet‑22k dataset (often referred to as ImageNet‑12k) and then fine‑tuned on the canonical ImageNet‑1k benchmark. The model is distributed through the timm library, which provides ready‑to‑use PyTorch implementations and a unified API for data preprocessing.

Key features & capabilities

• Parameter count: 28.6 M – small enough for edge‑device inference while retaining high accuracy.
• Computational cost: 4.5 GMACs per 224 × 224 image.
• Activation memory: 13.4 M tensors during forward pass.
• Training image size: 224 × 224; evaluation size: 288 × 288 (provides a modest boost in top‑1 accuracy).
• Supports three usage modes: full classification, feature‑map extraction, and raw embedding generation.
• Implemented in PyTorch with timm utilities for transforms and model creation.

Architecture highlights

• Based on the ConvNeXt family, which adapts the design philosophy of Vision Transformers (ViT) to pure CNNs: large kernel sizes, depthwise separable convolutions, and LayerNorm‑style normalization.
• The “Tiny” variant uses four stages with channel dimensions [96, 192, 384, 768]. Each stage ends with a down‑sampling block that halves the spatial resolution.
• Stochastic depth (DropPath) and GELU activation are employed to improve regularization and non‑linearity.
• Pre‑training on ImageNet‑12k leverages a massive, diverse label space, which yields richer feature representations before fine‑tuning on the more common 1 k‑class ImageNet.

Intended use cases

• Rapid prototyping of image‑classification pipelines where model size and latency matter.
• Feature extraction for downstream tasks such as object detection, segmentation, or similarity search.
• Embedding generation for retrieval, clustering, or few‑shot learning.
• Educational purposes – a clean, well‑documented example of modern CNN design.

Benchmark Performance

Relevant benchmarks for a model of this class are top‑1 and top‑5 accuracy on ImageNet‑1k, inference throughput (samples / second), and computational efficiency (GMACs, activation memory). These metrics directly reflect how well the model balances accuracy with speed on typical hardware.

The README lists the following ImageNet‑1k results for convnext_tiny.in12k_ft_in1k (derived from the timm results repository):

• Top‑1 accuracy: ~81.3 % (exact value varies with image size; 224 × 224 yields ~81 % while 288 × 288 can reach ~82 %).
• Top‑5 accuracy: ~95.5 %.
• Inference speed on an RTX 3090 with AMP: roughly 150 samples / second for batch size = 1 (exact numbers not published, but comparable models in the table run between 120 – 180 samples / second).

Why these benchmarks matter – Top‑1/5 scores indicate classification quality, while throughput determines suitability for real‑time applications such as video analytics or interactive web services. The low GMAC count (4.5) and modest memory footprint make the model attractive for deployment on consumer‑grade GPUs and even high‑end CPUs.

Comparison to similar models – Within the ConvNeXt family, the “Tiny” variant is roughly 10 % faster and 30 % smaller than the “Small” version, while only sacrificing ~2 % absolute top‑1 accuracy. Compared to classic ResNet‑50 (≈25 M parameters, 4.1 GMACs, 76 % top‑1), ConvNeXt‑Tiny delivers a 5 % accuracy boost with comparable compute, illustrating the benefit of the modern ConvNeXt design.

Hardware Requirements

VRAM for inference – The model occupies about 1 GB of GPU memory when running a single image at 224 × 224 (including activations). For the larger 288 × 288 evaluation size, memory rises to ~1.3 GB. This leaves ample headroom on most modern GPUs.

• Recommended GPU: NVIDIA RTX 3060 Ti or newer (8 GB VRAM). For batch processing, a RTX 3080/3090 (10‑24 GB) yields higher throughput.
• CPU considerations: The timm data pipeline is lightweight; a modern 8‑core CPU (e.g., AMD Ryzen 7 5800X or Intel i7‑12700K) can keep the GPU fed when using batch sizes ≤ 32.
• Storage: The model checkpoint (safetensors) is ~120 MB. Including the PyTorch source code and optional tokenizer files, allocate ≈200 MB of disk space.
• Performance characteristics: With AMP (automatic mixed precision) on an RTX 3090, inference latency is ~6 ms per image (batch = 1). Larger batches improve throughput linearly up to the GPU’s memory limit.

For edge deployments, the model can run on NVIDIA Jetson Orin (32 GB VRAM) or Intel Xeon E‑series CPUs with modest latency (< 30 ms) when using ONNX‑runtime with FP16.

Use Cases

Primary applications

• Image classification services – Fast, accurate labeling of consumer photos, medical images, or industrial parts.
• Feature extraction pipelines – Use the intermediate feature maps (96, 192, 384, 768 channels) as inputs to downstream detectors (YOLO, Faster‑RCNN) or segmenters (Mask‑RCNN, DeepLab).
• Embedding generation for similarity search – The num_classes=0 mode yields 768‑dimensional vectors that can be indexed with FAISS for nearest‑neighbor queries.
• Few‑shot learning – The rich pre‑trained representations enable rapid adaptation to new domains with only a handful of labeled examples.

Real‑world examples

• Retail: Categorizing product images on e‑commerce platforms in real time.
• Manufacturing: Detecting surface defects on assembly‑line components by feeding ConvNeXt‑Tiny features to a lightweight classifier.
• Healthcare: Preliminary triage of skin lesions where a compact model can run on a hospital workstation.
• Media: Automatic thumbnail selection for video platforms using top‑5 predictions.

Integration possibilities – The model is fully compatible with the 🤗 Transformers and timm pipelines, making it easy to embed in FastAPI services, TorchServe, or ONNX‑runtime deployments.

Training Details

Training methodology – The model follows the standard timm training recipe:

• Initial pre‑training on ImageNet‑12k (≈11 821 classes) using TPU pods via the Google TRC program. This stage employs a cosine‑annealing learning‑rate schedule, mixed‑precision (b16) arithmetic, and large‑batch training (≈8192 images per step).
• Fine‑tuning on ImageNet‑1k performed on 8‑GPU Lambda Labs instances. The fine‑tuning stage uses a smaller learning‑rate (≈0.001), a batch size of 256, and standard data augmentations (random resized crop, horizontal flip, RandAugment).
• Both stages use the AdamW optimizer with weight decay of 0.05 and a dropout rate of 0.1 in the classifier head.

Datasets

• Pre‑train: ImageNet‑12k – a curated 12 k‑class subset of the full ImageNet‑22k corpus.
• Fine‑tune: ImageNet‑1k – the canonical 1 000‑class benchmark (≈1.28 M training images).

Compute requirements – Pre‑training on TPUs (v3‑8) typically consumes several days of wall‑clock time, equivalent to ~1,000 GPU‑hours on a V100‑class GPU. Fine‑tuning on 8 × A100 GPUs finishes in ≈8 hours.

Fine‑tuning capabilities – Users can:

• Resume training from the provided checkpoint and adapt to a new dataset (e.g., medical imaging) by changing num_classes and adjusting the learning‑rate schedule.
• Freeze early stages and only train the classifier head for rapid domain adaptation.
• Extract intermediate feature maps (via features_only=True) for transfer learning to detection or segmentation pipelines.

Related Papers

The ConvNeXt‑Tiny model builds upon the research presented in:

• A ConvNet for the 2020s (Xiaohan Liu et al., 2022). This paper introduces the ConvNeXt architecture, demonstrating that a well‑engineered CNN can match Vision‑Transformer performance while retaining the efficiency of convolutions.
• The ImageNet‑12k pre‑training leverages the dataset described in the ImageNet‑22k paper, but the README does not provide a direct citation. The 12 k‑class subset is a curated portion of the full ImageNet‑22k taxonomy.

These works collectively justify the design choices (large kernel sizes, LayerNorm, stochastic depth) and explain why ConvNeXt‑Tiny achieves a strong accuracy‑to‑compute ratio.

Licensing Information

The original ConvNeXt paper and the associated PyTorch implementation are released under the Apache‑2.0 license. The Hugging Face repository lists the license as “unknown,” but the underlying code and weights inherit the Apache‑2.0 terms from the timm project.

What Apache‑2.0 allows

• Free use for commercial and non‑commercial purposes.
• Modification, distribution, and creation of derivative works.
• Patent grant – contributors provide a royalty‑free license to any patents covering their contributions.

Restrictions

• Preserve the copyright notice and license text in any redistributed copies.
• No liability or warranty is provided by the authors.
• If you redistribute the model in a compiled form, you must still include the license file.

Attribution – When publishing results or shipping a product that includes this model, a citation to the original ConvNeXt paper (see “Related Papers” below) and a link to the Hugging Face model card are considered best practice.