Technical Overview
The hustvl/yolos‑small model is a compact Vision‑Transformer (ViT) based object detector that follows the YOLOS (You Only Look at One Sequence) paradigm. It processes an input image with a transformer encoder and a fixed set of 100 learnable object queries, then predicts class scores and bounding‑box coordinates for each query. The model is fine‑tuned on the COCO 2017 detection benchmark (118 k training images) and is capable of detecting 80 COCO categories in a single forward pass.
Key features and capabilities include:
- Transformer‑only architecture – no convolutional backbone or region‑proposal network.
- DETR‑style bipartite matching loss with Hungarian assignment, enabling end‑to‑end training.
- Small‑sized variant – fewer parameters and lower latency than the base YOLOS model while retaining competitive accuracy.
- Fully compatible with the 🤗 Transformers
YolosFeatureExtractorandYolosForObjectDetectionAPIs (PyTorch). - Ready‑to‑use on‑device inference via the
safetensorsformat.
Architecture highlights:
- Backbone: ViT‑S (Vision Transformer Small) with 12 transformer layers, 384 hidden dimension, and 6 attention heads.
- Object queries: 100 learnable embeddings that act as “slots” for potential objects.
- Prediction heads: A linear classifier for 81 classes (80 COCO + “no‑object”) and a regression head for bounding‑box coordinates (center‑x, center‑y, width, height).
- Loss: Combination of cross‑entropy (class) and L1 + generalized IoU (bbox) after Hungarian matching.
Intended use cases focus on real‑time or near‑real‑time object detection where a lightweight transformer is preferred over heavier CNN‑based detectors. Typical applications include video analytics, autonomous‑driving perception, retail shelf monitoring, and robotics.
Benchmark Performance
For object‑detection models, the primary benchmark is the COCO 2017 Average Precision (AP) metric, which aggregates precision across IoU thresholds from 0.5 to 0.95. The hustvl/yolos‑small model achieves an AP of 36.1 on the COCO 2017 validation set, as reported in the original YOLOS paper (Table 1). This places it on par with the original DETR baseline (≈ 42 AP for the base model) while using a fraction of the parameters and compute.
Why COCO matters: it is the de‑facto standard for evaluating multi‑class, multi‑scale object detection in natural scenes. Scores on COCO are widely comparable across research and industry, making it easy to assess whether a model meets the accuracy requirements of a given product.
Compared to other lightweight detectors (e.g., YOLOv5‑s, SSD‑Mobilenet), YOLOS‑small offers a transformer‑centric approach with comparable AP but higher robustness to varied object scales and better integration with modern NLP‑vision pipelines. Its performance is especially attractive when a unified transformer stack is desired for both vision and language tasks.
Hardware Requirements
VRAM for inference: The small YOLOS model fits comfortably in 4 GB of GPU memory for a single image batch (≈ 120 MB model size plus activation overhead). For batch sizes > 1, 6–8 GB is recommended.
Recommended GPU: Any recent NVIDIA GPU with at least 6 GB of VRAM (e.g., RTX 2060, GTX 1660 Ti) will deliver sub‑30 ms latency per image at 640 × 640 resolution. For higher throughput, a RTX 3080 (10 GB) or A100 (40 GB) can process 8–16 images in parallel.
CPU requirements: The model can run on CPU‑only environments, but inference will be 5‑10× slower. A modern 8‑core CPU (e.g., Intel i7‑12700K) with SIMD extensions is the minimum for acceptable latency.
Storage needs: Model checkpoint (~120 MB) plus the safetensors file (~130 MB) and feature‑extractor config (~2 MB). Total disk footprint is under 300 MB.
Performance characteristics: At 640 × 640 input resolution, YOLOS‑small processes ~30 fps on a RTX 2070 (FP32) and ~45 fps on a RTX 3080 (FP16). The transformer encoder scales linearly with image size, so larger resolutions will increase latency proportionally.
Use Cases
YOLOS‑small is well‑suited for any scenario that requires accurate, single‑stage object detection with a modest compute budget. Typical applications include:
- Video surveillance: Detecting people, vehicles, and suspicious objects in real‑time streams.
- Retail analytics: Monitoring shelf stock, counting customers, and identifying product placements.
- Autonomous robotics: Providing perception for navigation and manipulation tasks on edge devices.
- Smart city infrastructure: Analyzing traffic flow, detecting accidents, and monitoring public spaces.
- Augmented reality: Real‑time scene understanding for overlaying virtual objects.
Integration is straightforward via the 🤗 Transformers library, allowing developers to embed the model in Python pipelines, Flask APIs, or ONNX‑exported runtimes for deployment on mobile or embedded platforms.
Training Details
Pre‑training: The backbone was pre‑trained on ImageNet‑1k for 200 epochs using standard supervised classification loss. This step provides robust visual feature extraction before detection fine‑tuning.
Fine‑tuning on COCO: The model was subsequently fine‑tuned on the COCO 2017 detection dataset for 150 epochs. The training pipeline uses the DETR loss: Hungarian matching to assign each of the 100 object queries to ground‑truth boxes, followed by a weighted sum of cross‑entropy (class) and L1 + generalized IoU (bbox) losses.
Compute requirements: Training was performed on NVIDIA V100 GPUs (32 GB VRAM) with mixed‑precision (FP16) to accelerate convergence. Approximate total compute is ~2 k GPU‑hours for the full pre‑train + fine‑tune schedule.
Fine‑tuning capabilities: Users can further adapt the model to custom datasets by loading the YolosForObjectDetection class and continuing training with the same loss formulation. The small size makes it feasible to fine‑tune on a single GPU with a batch size of 2–4 images.
Licensing Information
The model card lists the license as “unknown”, but the README explicitly states license: apache‑2.0. Apache 2.0 is a permissive open‑source license that allows:
- Free use, modification, and distribution for both personal and commercial purposes.
- Incorporation into proprietary software without the need to open‑source the entire codebase.
- Patent grant and protection against patent litigation.
If the model truly carries an “unknown” license on the Hub, users should treat it as “all‑rights‑reserved” until clarification is provided. In practice, most downstream projects assume the Apache‑2.0 terms because they appear in the source repository. For commercial deployments, it is safest to:
- Verify the license on the original GitHub repository (hustvl/YOLOS).
- Include proper attribution to the original authors (see the BibTeX entry below).
- Provide a copy of the Apache‑2.0 license text in any redistributed binaries.