parakeet-tdt-1.1b

Technical Overview

The parakeet‑tdt‑1.1b model is an English‑only Automatic Speech Recognition (ASR) system built on NVIDIA’s NeMo framework and co‑developed with Suno.ai. With roughly 1.1 billion parameters, it is the “XXL” incarnation of the FastConformer‑TDT architecture, designed to transcribe spoken audio into lower‑case English text with high accuracy and low latency.

• Core capability: End‑to‑end speech‑to‑text conversion for a wide range of acoustic conditions (clean, noisy, meeting‑room, broadcast, etc.).
• Key features:
  • FastConformer encoder – a convolution‑augmented Transformer that balances speed and representational power.
  • TDT (Transducer) decoder – a streaming‑friendly RNN‑Transducer that enables low‑latency inference.
  • Support for NeMo pipelines, PyTorch, and Hugging Face 🤗 Transformers.
  • Trained on a massive multilingual‑English corpus (over 30 k h of speech) covering audiobooks, conversational speech, broadcast news, and domain‑specific data.
• Architecture highlights:
  • FastConformer encoder with depth‑wise convolution and self‑attention blocks, optimized for GPU throughput.
  • TDT (Transducer) decoder that jointly learns acoustic and language modeling, removing the need for an external language model in many deployment scenarios.
  • All components implemented in PyTorch and wrapped by NeMo’s ASRModel class for seamless fine‑tuning.
• Intended use cases: real‑time transcription, meeting captioning, call‑center analytics, voice assistants, and any application that demands high‑accuracy English ASR on modern GPUs.

Benchmark Performance

The model’s performance is reported on a variety of public ASR test sets, measured with Word Error Rate (WER). Highlights include:

• LibriSpeech (clean) – 1.39 % WER
• LibriSpeech (other) – 2.62 % WER
• GigaSpeech – 9.55 % WER
• VoxPopuli (English) – 5.48 % WER
• TED‑LIUM‑v3 – 3.56 % WER
• AMI Meetings (test) – 15.90 % WER
• Earnings‑22 – 14.65 % WER
• Mozilla Common Voice 9.0 – 5.97 % WER

These benchmarks cover both clean read speech (LibriSpeech) and noisy, conversational domains (AMI, Earnings‑22, VoxPopuli). The low WER on clean data demonstrates the model’s strong acoustic modeling, while the competitive scores on meeting‑room and broadcast data show robustness to real‑world conditions. Compared with the 300 M‑parameter FastConformer‑TDT baseline, the 1.1 B version consistently improves WER by 30‑50 % across the board, making it one of the top‑performing open‑source English ASR models on the 🤗 ASR leaderboard.

Hardware Requirements

Running a 1.1 B‑parameter Transducer model efficiently requires modern GPUs with ample VRAM. Recommended specifications are:

• GPU: NVIDIA A100 (40 GB) or RTX 4090 (24 GB) for full‑batch inference; a single RTX 3080 (10 GB) can run the model with reduced batch size or quantization.
• VRAM: ~12 GB for FP16 inference; ~8 GB for INT8‑quantized inference.
• CPU: 8‑core Xeon or AMD Ryzen 7+ for preprocessing (feature extraction) and feeding the GPU.
• Storage: Model checkpoint ~5 GB (weights + config). Additional ~2 GB for tokenizer and auxiliary files.
• Performance: On an A100, real‑time factor (RTF) is typically 0.15‑0.25× (i.e., 4‑6× faster than real‑time) for 16 kHz audio.

Use Cases

Parakeet‑TDT‑1.1B is well‑suited for any scenario that needs accurate, low‑latency English transcription. Typical deployments include:

• Live captioning for meetings and webinars: Stream audio from Zoom, Teams, or Webex and generate subtitles in real time.
• Call‑center analytics: Transcribe customer calls for sentiment analysis, keyword spotting, and compliance monitoring.
• Voice‑driven assistants: Provide robust command recognition even in noisy home environments.
• Media indexing: Generate searchable transcripts for podcasts, news broadcasts, and lecture recordings.
• Research & prototyping: Fine‑tune on domain‑specific vocabularies (e.g., medical, legal) using NeMo’s recipe scripts.

Training Details

The model was trained on a massive, diverse English speech corpus that includes:

• LibriSpeech (clean & other)
• Fisher Corpus, Switchboard‑1, WSJ‑0/1
• National‑Singapore‑Corpus (Parts 1 & 6)
• VCTK, VoxPopuli, Europarl, Multilingual LibriSpeech
• Mozilla Common Voice 8.0, MLCommons People’s Speech

Training was performed with NeMo’s FastConformer‑TDT recipe on a cluster of NVIDIA A100 GPUs (8×40 GB) for roughly 2‑3 weeks of wall‑clock time, using mixed‑precision (FP16) and AdamW optimization. The loss function combines the Transducer joint loss with auxiliary CTC and language‑model regularization. After the base training, the checkpoint is released ready for fine‑tuning on domain‑specific data via the same NeMo pipeline, allowing users to adapt the model to specialized vocabularies or acoustic environments with minimal effort.

Related Papers

The architecture builds on several peer‑reviewed works:

• FastConformer: Efficient Transformer for Speech Recognition (arXiv:2304.06795)
• Transducer with Dual‑Path Transformer (TDT) (arXiv:2305.05084)
• RNN‑Transducer: A Simple and Efficient End‑to‑End Model for Speech Recognition (arXiv:2104.02821)

These papers introduce the fast‑convolutional encoder, the TDT decoding strategy, and the underlying transducer training objectives that together enable the high accuracy and low latency of the parakeet‑tdt‑1.1b model.

Licensing Information

The model is released under the CC‑BY‑4.0 license, which permits commercial use, modification, and redistribution provided that proper attribution is given to the original authors (NVIDIA and Suno.ai). Although the README lists the license as “unknown”, the license field explicitly states cc-by-4.0, which is a permissive open‑source license.

• Attribution requirement: Cite the model name, the NVIDIA NeMo project, and the CC‑BY‑4.0 license in any downstream product or publication.
• Commercial usage: Allowed without additional fees, but you must retain the license text and attribution.
• Restrictions: You may not misrepresent the original authorship or remove the license notice. No warranty is provided.