Technical Overview
distil-large-v3 is a distilled version of OpenAI’s Whisper large‑v3 model, created by the distil‑whisper team. It is an automatic‑speech‑recognition (ASR) system that converts spoken audio into written text, supporting English‑language transcription out of the box. The model is built on the Robust Knowledge Distillation via Large‑Scale Pseudo Labelling methodology, which uses massive pseudo‑labelled data to transfer knowledge from the large teacher model to a smaller student while preserving accuracy.
Key capabilities include:
- High‑fidelity short‑form transcription (< 30 seconds) with a Word Error Rate (WER) within 1 % of the teacher.
- Superior long‑form transcription using Whisper’s sequential algorithm – the de‑facto standard for libraries such as Whisper‑cpp, Faster‑Whisper, and OpenAI Whisper.
- Fast inference: 6.3× faster than the original large‑v3 and 1.1× faster than its predecessor distil‑large‑v2.
- Compatibility with multiple runtimes (Transformers, Whisper‑cpp, Faster‑Whisper, Transformers.js, Candle) thanks to pre‑converted weight formats.
Architecturally, distil‑large‑v3 retains the same encoder‑decoder transformer backbone as Whisper large‑v3 but with roughly half the parameters (≈ 756 M vs. 1.55 B). The distillation process preserves the original’s 30‑second context window and the 2‑stage encoder‑decoder attention pattern, while employing a streamlined attention head layout and reduced hidden dimensions to cut latency and memory usage.
Intended use cases span any scenario that requires accurate, low‑latency speech‑to‑text conversion: real‑time captioning, podcast transcription, meeting minutes, and large‑scale audio indexing. Its speed and modest hardware footprint make it especially attractive for on‑device or edge deployments where the full Whisper large‑v3 would be prohibitive.
Benchmark Performance
For ASR models, the most relevant benchmarks are Word Error Rate (WER) on short‑form and long‑form audio, as well as relative latency (inference speed) compared to a reference model. The README provides a concise comparison:
| Model | Params (M) | Relative Latency | Short‑Form WER | Sequential Long‑Form WER | Chunked Long‑Form WER |
|---|---|---|---|---|---|
| large‑v3 (teacher) | 1550 | 1.0× | 8.4 % | 10.0 % | 11.0 % |
| distil‑large‑v3 | 756 | 6.3× | 9.7 % | 10.8 % | 10.9 % |
| distil‑large‑v2 | 756 | 5.8× | 10.1 % | 15.6 % | 11.6 % |
The “Sequential Long‑Form” column reflects the algorithm most production pipelines use; distil‑large‑v3 stays within 1 % WER of the teacher while being over six times faster. This balance of speed and accuracy is why the model is highlighted for long‑form transcription workloads.
Hardware Requirements
Inference with distil‑large‑v3 is substantially lighter than Whisper large‑v3, but it still requires a modern GPU for optimal performance.
- VRAM: Approximately 6 GB of GPU memory when using
torch.float16(FP16) precision; up to 10 GB for FP32. - Recommended GPU: Any NVIDIA GPU with at least 8 GB VRAM (e.g., RTX 3060, RTX 3070, A100) or equivalent AMD GPUs supporting ROCm.
- CPU: A recent multi‑core CPU (Intel i7 12th gen or AMD Ryzen 7 5800X) is sufficient for preprocessing; however, CPU‑only inference will be markedly slower and may exceed 30 seconds per minute of audio.
- Storage: The model checkpoint (including safetensors) occupies roughly 3 GB. Additional space is needed for audio datasets and temporary feature extraction buffers.
- Performance: On a RTX 3060 (FP16), short‑form transcription runs at ~0.15 seconds per 30‑second clip; sequential long‑form transcription processes a 10‑minute audio file in ~8 seconds, showcasing the 6.3× speedup over the teacher.
Use Cases
Distil‑large‑v3 shines in any scenario where high‑quality transcription must be delivered quickly and with modest hardware.
- Live captioning: Real‑time subtitles for webinars, video conferences, or streaming platforms.
- Podcast & media indexing: Batch‑process hour‑long episodes to generate searchable transcripts.
- Enterprise meeting minutes: Automatic transcription of multi‑speaker meetings, followed by summarization pipelines.
- Edge devices: Deployment on laptops, tablets, or embedded systems where GPU memory is limited.
- Multilingual pipelines: While the model is English‑only, it can be combined with language detection front‑ends to route audio to appropriate language‑specific models.
Training Details
Distil‑large‑v3 was trained using the “large‑scale pseudo‑labelling” pipeline described in the 2023 arXiv paper. The process involved:
- Teacher model: Whisper large‑v3 (1.55 B parameters).
- Student model: Same transformer skeleton but with reduced hidden size and fewer attention heads, yielding 756 M parameters.
- Data: Hundreds of thousands of hours of English audio sourced from LibriSpeech, Common Voice, and proprietary pseudo‑labelled corpora generated by the teacher.
- Losses: A combination of CTC, cross‑entropy, and a knowledge‑distillation loss that aligns student logits with teacher outputs.
- Compute: Trained on a cluster of 8 × NVIDIA A100 GPUs for roughly 3 weeks, using mixed‑precision (FP16) to accelerate training.
- Fine‑tuning: The checkpoint is fully compatible with Hugging Face’s
TrainerAPI, allowing downstream users to fine‑tune on domain‑specific audio (e.g., medical dictation) with minimal effort.
Licensing Information
The model card lists the license as mit, which is a permissive open‑source license allowing free use, modification, and distribution, including commercial applications. However, the “License: unknown” field in the metadata may cause confusion for downstream users. In practice, the MIT license granted by the authors supersedes the unknown tag.
- Commercial use: Fully permitted under MIT; you may embed the model in SaaS products, mobile apps, or on‑device solutions.
- Restrictions: The only requirement is to retain the original copyright notice and license text in any redistributed binaries or source code.
- Attribution: Cite the model repository and the original Whisper large‑v3 paper when publishing results or releasing derivative works.