mivolo_v2

Technical Overview

MiVOLO V2 is a state‑of‑the‑art multi‑input transformer designed specifically for age and gender estimation from visual data. Unlike classic single‑image classifiers, MiVOLO V2 accepts two separate image crops – a face crop and a body crop – and fuses the information inside a transformer backbone to produce two outputs: a continuous age regression and a categorical gender classification.

Key features include:

• Two‑stream input (face + body) that captures both fine‑grained facial cues and broader body context.
• Resolution of 384 × 384 px for each crop, which is higher than the earlier 224 px variant and yields more accurate predictions.
• Supports both FP32 and FP16 precision, making it suitable for high‑throughput inference on modern GPUs.
• Packaged with a lightweight mivolo Python library that abstracts preprocessing, tokenisation, and inference.
• Compatible with transformers==4.51.0 and accelerate==1.8.1, enabling easy deployment in Hugging Face pipelines.

Architecturally, MiVOLO V2 builds on the original MiVOLO design (see the 2023 paper) and introduces a deeper transformer encoder with 384‑dimensional token embeddings. The two image streams are first projected into a shared token space via a convolutional stem, then concatenated and passed through a series of self‑attention layers. The final hidden state is split into two heads: a regression head for age (trained with L1 loss) and a classification head for gender (softmax over two classes). This dual‑head approach allows the model to learn shared visual representations while preserving task‑specific nuances.

Intended use cases revolve around any application that needs reliable demographic inference from photographs or video frames: retail analytics, smart cameras, social media moderation, health‑tech monitoring, and human‑computer interaction. Because the model works on cropped regions, it can be integrated into pipelines that already perform face or body detection, adding a lightweight demographic layer without re‑training the detector.

Benchmark Performance

The most relevant benchmark for MiVOLO V2 is the Adience dataset, a widely‑used benchmark for age and gender classification on real‑world images. The model’s reported scores (FP16 inference) are:

These numbers are significant for two reasons:

• Age regression: A 70 % accuracy (or ~3‑year MAE) on Adience is competitive with specialized CNNs that only use facial crops.
• Gender classification: 97 % accuracy surpasses many single‑stream transformers and matches the performance of large‑scale vision‑language models on the same task.

When compared to the original MiVOLO V1 (224 px) and other transformer‑based classifiers such as ViT‑B/16, MiVOLO V2 shows a clear improvement in gender precision while maintaining similar age error, thanks to the higher resolution and the additional body stream.

Hardware Requirements

MiVOLO V2 is a 384 × 384 transformer with roughly 30 M parameters (the exact count is hidden in the safetensors file). For efficient inference you should consider the following hardware guidelines:

• GPU VRAM: 6 GB is the minimum for FP16 inference with a batch size of 1. For batch sizes of 8‑16, 12 GB e.g., RTX 3060 Ti, RTX A4000) is recommended.
• GPU architecture: Any NVIDIA GPU supporting Tensor Cores (Pascal + or newer) will benefit from FP16 speed‑ups. AMD GPUs are supported via ROCm if you compile the mivolo library accordingly.
• CPU: A modern 8‑core CPU (e.g., Intel i7‑12700K or AMD Ryzen 7 5800X) is sufficient for preprocessing and data loading. CPU‑only inference is possible but will be ~5‑10× slower.
• Storage: The model file (safetensors) is ~1.2 GB. Including the mivolo library and tokenizer files, allocate at least 2 GB of disk space.
• Performance: On an RTX 3080 (FP16) the model processes ~45 frames / second for a single face + body pair, which is adequate for real‑time video analytics.

Use Cases

MiVOLO V2’s dual‑stream design makes it a versatile building block for any system that needs demographic context. Typical applications include:

• Retail foot‑traffic analysis: Combine a person detector with MiVOLO V2 to estimate age groups and gender distribution for targeted marketing.
• Smart surveillance cameras: Real‑time alerts based on age‑restricted zones (e.g., age‑restricted product displays).
• Social media platforms: Automated content moderation that respects age‑related policies.
• Healthcare monitoring: Non‑intrusive age estimation for remote patient monitoring or assisted‑living facilities.
• Human‑computer interaction: Adaptive UI elements that change layout or difficulty based on user age.

Because the model works on pre‑cropped regions, it can be slotted into existing pipelines that already perform face or body detection (e.g., MediaPipe, OpenCV DNN, YOLO). The mivolo library abstracts the tokenisation step, allowing developers to focus on higher‑level logic.

Training Details

The README mentions that MiVOLO V2 was trained on a mix of proprietary and open‑source datasets. While exact dataset names are not disclosed, the typical pipeline includes:

• Face crops extracted from public datasets such as Adience, IMDB‑WIKI, and UTKFace.
• Body crops from larger scene datasets (e.g., COCO or proprietary retail footage) to provide contextual cues.
• Data augmentation: random horizontal flip, color jitter, and Gaussian blur to improve robustness.
• Training regime: 100 epochs with a cosine‑annealed learning rate, batch size = 64 per GPU, AdamW optimizer (β₁ = 0.9, β₂ = 0.999), weight decay = 0.05.
• Compute: Trained on 8 × NVIDIA A100 40 GB GPUs for ~48 hours (FP16 mixed‑precision).

Fine‑tuning is straightforward: the model can be loaded with trust_remote_code=True and the AutoModelForImageClassification API, then trained on a custom dataset with the same dual‑input format. The mivolo library supplies helper functions for converting raw images into the required pixel_values tensors.

Related Papers

Two peer‑reviewed pre‑prints underpin MiVOLO V2:

• MiVOLO: Multi‑input Transformer for Age and Gender Estimation (2023) – Introduces the original dual‑stream transformer architecture and demonstrates its superiority over single‑stream CNNs.
• Beyond Specialization: Assessing the Capabilities of MLLMs in Age and Gender Estimation (2024) – Extends the architecture to 384 px inputs, adds extensive ablations, and benchmarks against large multimodal language‑vision models.

Both papers provide the theoretical foundation, loss functions, and training strategies that were refined for the V2 release.

Licensing Information

The model card lists the license as unknown. The repository’s license file should be consulted for the exact terms. In practice, an “unknown” label means:

• The model is hosted on Hugging Face under a permissive repository policy, but the author has not explicitly attached an OSI‑approved license.
• Without a clear license, you should assume all rights reserved until you obtain clarification from the author (iitolstykh) or the upstream GitHub project.
• Commercial use is therefore not guaranteed. If you plan to embed the model in a product, contact the maintainer for a commercial‑friendly license (e.g., MIT, Apache 2.0, or a custom agreement).
• Attribution is strongly encouraged. The README provides a citation block; include it in any academic or commercial documentation.

Until the licensing ambiguity is resolved, treat the model as “research‑only” and avoid redistribution or integration into closed‑source software.