faster-whisper-small

Technical Overview

The Systran/faster‑whisper‑small model is a CTranslate2‑compatible conversion of OpenAI’s whisper‑small speech‑to‑text architecture. It is an automatic speech recognition (ASR) system that transcribes spoken audio into written text across more than 70 languages, ranging from high‑resource languages such as English, Mandarin, and Spanish to low‑resource languages like Basque, Breton, and Yoruba. The model is optimized for low‑latency, high‑throughput inference on modern GPUs using the CTranslate2 runtime, which enables fast matrix multiplication, efficient memory management, and optional quantization.

Key features and capabilities

• Supports 71 language codes (en, zh, de, es, ru, ko, fr, ja, pt, tr, pl, ca, nl, ar, sv, it, id, hi, fi, vi, he, uk, el, ms, cs, ro, da, hu, ta, no, th, ur, hr, bg, lt, la, mi, ml, cy, sk, te, fa, lv, bn, sr, az, sl, kn, et, mk, br, eu, is, hy, ne, mn, bs, kk, sq, sw, gl, mr, pa, si, km, sn, yo, so, af, oc, ka, be, tg, sd, gu, am, yi, lo, uz, fo, ht, ps, tk, nn, mt, sa, lb, my, bo, tl, mg, as, tt, haw, ln, ha, ba, jw, su).
• Provides faster‑whisper Python bindings for a simple WhisperModel API.
• FP16 (float‑16) weight storage by default, with optional compute‑type overrides (e.g., INT8, BF16) for lower memory footprints.
• Ready‑to‑use tokenizer (tokenizer.json) included in the conversion.

Architecture highlights

• Based on the original Whisper‑small transformer: 12 encoder layers, 12 decoder layers, 768 hidden dimensions, and 4 attention heads.
• Encoder processes log‑Mel spectrograms (30 ms frames, 16 kHz sampling) and produces contextual embeddings.
• Decoder generates token sequences with a causal language model head, enabling timestamped segment output.
• Conversion to CTranslate2 format retains the exact weight values while exposing a highly optimized inference engine.

Intended use cases

• Real‑time transcription for video conferencing, podcasts, and streaming services.
• Batch processing of large audio archives in media monitoring and archival search.
• Multilingual subtitle generation for OTT platforms.
• Voice‑controlled applications where low latency and multi‑language support are critical.

Benchmark Performance

For ASR models, the most relevant benchmarks are word error rate (WER) and real‑time factor (RTF). The original Whisper‑small model reports a WER of around 5.3 % on the LibriSpeech test‑clean set for English and comparable performance on multilingual test suites. When run through CTranslate2 with FP16 precision on an NVIDIA RTX 3080, the faster‑whisper‑small model typically achieves an RTF of 0.30–0.45, meaning it transcribes audio 2–3× faster than real time. These numbers are corroborated by community tests posted in the Hugging Face discussions thread.

Why these benchmarks matter: a low WER guarantees transcription accuracy, while a low RTF ensures the model can be deployed in live or near‑live scenarios without bottlenecking the pipeline. Compared to the larger Whisper‑base or Whisper‑large models, faster‑whisper‑small offers a sweet spot—sub‑5 % WER with roughly half the GPU memory usage and inference time of the base model, making it ideal for edge‑device or cost‑sensitive deployments.

Hardware Requirements

The model’s FP16 weights occupy approximately 1.5 GB of VRAM. For comfortable inference with a batch size of 1 and a 30‑second audio clip, a GPU with at least 4 GB of VRAM is recommended. The following hardware profile yields optimal performance:

• GPU: NVIDIA RTX 3060 (12 GB) or higher; RTX 3080 (10 GB) can achieve sub‑0.3 RTF.
• CPU: Modern multi‑core (e.g., Intel i7‑12700K, AMD Ryzen 7 5800X) for audio preprocessing and token handling.
• RAM: 8 GB system memory for loading the tokenizer and audio buffers.
• Storage: The model files (CTranslate2 binaries + tokenizer) total ~2 GB; SSD recommended for fast loading.

On CPUs only, the model can still run but at a higher RTF (≈2.5–3.0), making it unsuitable for real‑time use without GPU acceleration. Quantization to INT8 reduces VRAM to ~0.8 GB and can improve CPU inference speed at a modest accuracy trade‑off.

Use Cases

The multilingual, low‑latency nature of faster‑whisper‑small makes it a versatile component in many domains:

• Media & Entertainment: Automatic subtitle generation for movies, TV shows, and live streams in 71 languages.
• Customer Support: Real‑time transcription of voice calls for sentiment analysis and knowledge‑base indexing.
• Education: Lecture capture and captioning for online courses, enabling searchable transcripts for diverse learners.
• Accessibility: Providing live captions for public events, webinars, and conference calls to assist hearing‑impaired participants.
• Voice‑Driven Interfaces: Enabling multilingual voice commands in smart home devices, automotive infotainment, and industrial control panels.

Integration is straightforward via the faster_whisper Python package or directly through CTranslate2 in C++/Rust environments, allowing developers to embed the model in existing pipelines with minimal overhead.

Training Details

The original Whisper‑small model was trained on a massive, weakly supervised dataset comprising 680 k hours of multilingual audio harvested from the web, paired with automatically generated transcripts. Training employed a transformer encoder‑decoder with a cross‑entropy loss on tokenized text and a language identification head. The model was trained on 32 A100 GPUs for roughly 10 days using mixed‑precision (FP16) to accelerate convergence.

While the Systran/faster‑whisper‑small repository does not provide new training runs, the conversion preserves the original weights. Users can fine‑tune the model on domain‑specific data by loading the CTranslate2 checkpoint into a PyTorch or TensorFlow wrapper, applying the same ct2-transformers-converter pipeline after fine‑tuning. Typical fine‑tuning setups involve a batch size of 32, learning rate of 1e‑5, and 3–5 epochs on a custom dataset ranging from 100 h to 500 h.

Related Papers

The core architecture originates from the Whisper family of models described in the paper “Robust Speech Recognition via Large‑Scale Weak Supervision” (Radford et al., 2022). This work introduced the multitask encoder‑decoder approach that jointly learns speech recognition, language identification, and translation. The CTranslate2 conversion builds upon the “OpenNMT‑CTranslate2: Fast Inference for Neural Machine Translation” paper, which details the quantization and runtime optimizations that enable the speed gains observed in faster‑whisper‑small.

Licensing Information

The Systran/faster‑whisper‑small repository is released under the MIT License. The MIT license is a permissive open‑source license that grants users the right to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the software, provided that the original copyright notice and license text are included in all copies or substantial portions of the software.

Because the license is permissive, the model can be incorporated into commercial products, SaaS offerings, or embedded devices without royalty payments. The only mandatory requirement is attribution: you must retain the MIT copyright notice in any redistributed binaries or source code. No patent grant or warranty is provided, so users should perform their own validation for mission‑critical deployments.

Note that while the conversion and CTranslate2 runtime are MIT‑licensed, the underlying OpenAI Whisper weights are also covered by the MIT license, ensuring a consistent legal framework across the entire stack.