parakeet-ctc-1.1b

Technical Overview

The nvidia/parakeet-ctc-1.1b model – also called parakeet‑ctc‑1.1b – is a large‑scale Automatic Speech Recognition (ASR) system built on NVIDIA’s NeMo toolkit and the 🤗 Transformers library. It transcribes English speech into lower‑case alphabetic text using a Connectionist Temporal Classification (CTC) decoder. With roughly 1.1 billion parameters, it is the “XXL” variant of the FastConformer‑CTC architecture, delivering state‑of‑the‑art accuracy while retaining the speed benefits of the Conformer family.

Key capabilities include:

• End‑to‑end speech‑to‑text with a single forward pass (no external language model required).
• FastConformer backbone – a hybrid of convolutional subsampling and self‑attention that balances latency and accuracy.
• Support for a wide variety of English corpora (LibriSpeech, Fisher, Switchboard, WSJ, VoxPopuli, Common Voice, etc.).
• Ready‑to‑use checkpoint for inference, fine‑tuning, or export to ONNX/TensorRT.
• Compatible with both NeMo pipelines and the Hugging Face pipeline("automatic-speech-recognition") API.

The architecture consists of a FastConformer encoder (stacked Conformer blocks with depth‑wise convolutions) followed by a CTC head that maps encoder embeddings to 30 English characters plus the blank token. The model is trained with mixed‑precision (FP16) and leverages NVIDIA’s NeMo FastConformer implementation for efficient GPU utilization.

Intended use cases span any scenario that requires high‑accuracy, low‑latency transcription of English audio – from real‑time captioning and meeting transcription to batch processing of podcasts, audiobooks, and call‑center recordings.

Benchmark Performance

Parakeet‑CTC‑1.1b has been evaluated on a diverse set of public ASR benchmarks. Word Error Rate (WER) is the primary metric for these tasks. The model achieves:

• LibriSpeech (clean) test: 1.83 % WER
• LibriSpeech (other) test: 3.54 % WER
• GigaSpeech test: 10.27 % WER
• VoxPopuli (English) test: 6.53 % WER
• TED‑LIUM‑v3 test: 3.54 % WER
• Mozilla Common Voice 9.0 (English) test: 9.02 % WER
• Earnings‑22 test: 13.69 % WER
• AMI (Meetings) test: 15.62 % WER
• SPGI Speech test: 4.20 % WER

These benchmarks cover clean read speech (LibriSpeech), conversational telephone speech (Switchboard, Fisher, AMI), and noisy, domain‑specific corpora (Earnings‑22, SPGI). The low WER on clean data demonstrates the model’s top‑tier acoustic modeling, while competitive scores on noisy, multi‑speaker data illustrate its robustness. Compared to smaller FastConformer models (≈200 M parameters) and other open‑source CTC systems, the 1.1 B version consistently improves WER by 15‑30 % across the board, making it one of the most accurate publicly available English CTC models.

Hardware Requirements

Running a 1.1 B parameter model demands substantial GPU memory, especially for batch inference. The recommended hardware configuration is:

• GPU: NVIDIA A100 (40 GB) or RTX 4090 (24 GB) – at least 24 GB VRAM for single‑utterance inference.
• CPU: 8‑core modern CPU (e.g., Intel Xeon E5‑2690 v4 or AMD Ryzen 9 5900X) for preprocessing and audio loading.
• RAM: 32 GB system memory to comfortably handle audio decoding pipelines.
• Storage: ~6 GB for the model checkpoint (safetensors) plus additional space for audio datasets.
• Inference latency: On an A100, the model processes ~30‑40 ms of audio per GPU millisecond (≈25‑30 × real‑time) with FP16.

For large‑scale batch processing, multi‑GPU data parallelism (e.g., torch.nn.DataParallel or NeMo’s torch.distributed) can be employed. If VRAM is limited, the model can be run in torch.compile or ONNX‑TensorRT mode with INT8 quantization, reducing memory to ~12 GB at a modest WER increase (< 0.5 %).

Use Cases

Parakeet‑CTC‑1.1b excels in any English speech‑to‑text scenario that demands high accuracy and low latency. Typical applications include:

• Real‑time captioning for live streams, webinars, and virtual meetings.
• Call‑center analytics – automatic transcription of customer support calls for sentiment analysis and quality assurance.
• Podcast & audiobook indexing – batch transcription for searchable archives.
• Voice‑controlled assistants – on‑device or cloud‑based command recognition where a CTC decoder simplifies integration.
• Media monitoring – transcribing news broadcasts, earnings calls, and financial briefings.

The model can be fine‑tuned on domain‑specific vocabularies (e.g., medical or legal jargon) using the same NeMo framework, making it adaptable to niche industries while preserving its strong baseline performance.

Training Details

Parakeet‑CTC‑1.1b was trained on a massive multilingual English corpus that aggregates more than 10 k hours of audio from the following datasets:

• LibriSpeech (clean & other)
• Fisher Corpus
• Switchboard‑1
• WSJ‑0 & WSJ‑1
• National‑Singapore‑Corpus (Parts 1 & 6)
• VCTK
• VoxPopuli (English)
• Europarl
• Multilingual LibriSpeech
• Mozilla Common Voice 8.0 & 9.0
• MLCommons People’s Speech

Training leveraged NVIDIA’s mixed‑precision (FP16) pipeline on a cluster of 8 × A100 GPUs (40 GB each) for roughly 2 weeks, using the AdamW optimizer with a cosine learning‑rate schedule. The loss function was standard CTC loss with label smoothing. Data augmentation included SpecAugment, speed perturbation (0.9‑1.1×), and additive background noise from the VoxPopuli corpus.

Fine‑tuning is straightforward: load the checkpoint with nemo.collections.asr.models.EncDecCTCModel, replace the training dataset, and run trainer.fit(). The model also supports export to ONNX/TensorRT for production deployment.

Related Papers

The core architecture builds on the FastConformer‑CTC design described in the arXiv paper “FastConformer: Efficient Conformer for Automatic Speech Recognition” (2023). This work introduces a lightweight convolution‑augmented transformer that reduces latency without sacrificing accuracy. Additional background can be found in the original Conformer paper (“Conformer: Convolution‑augmented Transformer for Speech Recognition”) and the CTC foundation by Graves et al. (“Connectionist Temporal Classification”).

Licensing Information

The model is released under the CC‑BY‑4.0 license, as indicated in the README. This permissive license permits:

• Commercial and non‑commercial use, including integration into products and services.
• Modification, redistribution, and creation of derivative works.
• Requirement to give appropriate credit to NVIDIA and the original authors (e.g., “Parakeet‑CTC‑1.1b, NVIDIA NeMo & Suno.ai”).

There are no copyleft obligations, so proprietary code may be combined with the model without the need to open‑source the entire codebase. Users must retain the license notice and link to the original model card. If you plan to redistribute the model weights, you must also provide the CC‑BY‑4.0 attribution and a copy of the license.