nb-wav2vec2-1b-nynorsk

Technical Overview

The nb‑wav2vec2‑1b‑nynorsk model is a large‑scale, self‑supervised speech encoder that has been fine‑tuned for automatic speech recognition (ASR) of the Nynorsk written standard of Norwegian. Built on top of Meta’s wav2vec2‑xls‑r‑1b feature extractor, it inherits a 1‑billion‑parameter transformer backbone that learns rich acoustic representations from raw audio waveforms without any language‑specific supervision.

Key features and capabilities include:

• Support for 16 kHz MP3 audio (the NPSC dataset is pre‑processed to this rate).
• End‑to‑end CTC decoding with optional 5‑gram KenLM language model integration, yielding a WER of 11.32 % (0.1132) and CER of 4.03 % (0.0402) on the NPSC test split.
• Fine‑tuned on the NbAiLab/NPSC corpus, which contains over 1 000 hours of parliamentary speech in both Bokmål and Nynorsk.
• Fully compatible with the 🤗 transformers library, torch, and speechbrain pipelines, allowing seamless integration into existing Python speech‑to‑text stacks.

Architecture highlights:

• Base encoder: facebook/wav2vec2-xls-r-1b – a 24‑layer transformer with 1 B parameters, 16 kHz raw waveform input, and a convolutional feature encoder.
• CTC head: a linear projection on top of the encoder’s final hidden state, trained with the Connectionist Temporal Classification loss.
• Regularisation: layer‑drop (0.041), attention dropout (0.094), activation dropout (0.055), and hidden dropout (0.047) to improve generalisation on noisy parliamentary recordings.
• Masking strategy: time‑mask probability 0.082 (length 10) and feature‑mask probability 0.25 (length 64) to encourage robustness to missing acoustic information.
• Gradient checkpointing and FP16 training to fit the 1 B‑parameter model on a single high‑end GPU.

Intended use cases revolve around any application that needs high‑accuracy transcription of spoken Norwegian Nynorsk, such as:

• Live captioning of parliamentary debates, news broadcasts, and public‑service announcements.
• Automatic subtitle generation for Nynorsk‑language media content.
• Voice‑controlled assistants and dictation tools targeting Nynorsk speakers.
• Research projects on low‑resource dialects and cross‑lingual speech transfer.

Benchmark Performance

For ASR models, the most relevant benchmarks are Word Error Rate (WER) and Character Error Rate (CER), measured on a held‑out test set that reflects real‑world acoustic conditions. The model card reports the following results on the NPSC test split (Nynorsk variant) when combined with a 5‑gram KenLM language model:

• WER: 0.1132 (11.32 %) – the primary metric for overall transcription quality.
• CER: 0.0403 (4.03 %) – useful for languages with rich morphology like Norwegian.
• Without the language model, WER rises to 0.1364, demonstrating the value of even a modest n‑gram LM for domain‑specific text.

These numbers place the model among the top‑performing Norwegian Nynorsk ASR systems, beating the 300 M‑parameter counterpart (nb‑wav2vec2‑300m‑nynorsk) which scores a WER of 12.22 %. The gap highlights the benefit of the larger 1 B‑parameter backbone for capturing subtle phonetic variations in Nynorsk speech.

Hardware Requirements

VRAM for inference: The model’s 1 B‑parameter transformer occupies roughly 6–7 GB of GPU memory when loaded in FP16 mode. For safe batch‑size = 1 inference with the accompanying KenLM rescoring, a GPU with at least 8 GB VRAM (e.g., NVIDIA RTX 2070 or RTX 3060) is recommended.

Recommended GPU specifications for production‑grade workloads:

• CUDA Compute Capability ≥ 7.5 (e.g., NVIDIA RTX 3080, A100, or AMD Radeon RX 6800 XT with ROCm support).
• FP16/AMP support to halve memory footprint and double throughput.
• NVidia Tensor Cores (or equivalent) for accelerated matrix multiplications.

CPU requirements: While GPU inference is preferred, the model can run on CPU‑only environments using torch.compile or ONNX Runtime. Expect a latency of ~300 ms per 10‑second audio segment on a modern 8‑core Xeon or AMD EPYC processor, with a RAM footprint of ~12 GB.

Storage needs: The model files (including safetensors, config, tokenizer, and optional language‑model files) total roughly 3 GB. Adding the 5‑gram KenLM (~200 MB) brings the overall disk usage to ≈ 3.2 GB. SSD storage is recommended for fast loading, especially when fine‑tuning.

Use Cases

The nb‑wav2vec2‑1b‑nynorsk model shines in any scenario that requires accurate transcription of spoken Nynorsk. Typical applications include:

• Parliamentary & governmental archives: Automatic transcription of Stortinget debates for searchable archives and real‑time captioning.
• Media & broadcasting: Generating subtitles for TV programs, podcasts, and YouTube videos in Nynorsk.
• Voice‑enabled assistants: Enabling Siri‑like or Alexa‑like experiences for Nynorsk‑speaking users.
• Educational tools: Building language‑learning apps that provide instant feedback on pronunciation and fluency.
• Research & linguistics: Corpus creation for dialectology, speech pathology, and cross‑lingual phonetic studies.

Integration is straightforward via the 🤗 pipeline("automatic-speech-recognition") API, or through the provided run_speech_recognition_ctc.py script for batch processing. The model also supports ONNX export for deployment on edge devices with limited compute.

Training Details

Methodology: The model was fine‑tuned using the CTC loss on the NPSC dataset, with a 16 kHz MP3 configuration for Nynorsk. Training leveraged the run_speech_recognition_ctc.py script from the 🤗 Transformers repository, employing gradient checkpointing and mixed‑precision (FP16) to fit the 1 B‑parameter network on a single GPU.

Dataset: The NbAiLab/NPSC corpus contains parliamentary speech in both Bokmål and Nynorsk, curated from the Norwegian Parliamentary Speech Corpus (NPSC). The Nynorsk split comprises roughly 600 hours of annotated audio, with a train/validation/test split that mirrors real‑world speaker variability.

Compute requirements:

• GPU: 1 × NVIDIA RTX 3090 (24 GB VRAM) or equivalent for full‑precision training; FP16 reduces memory to ≈ 12 GB.
• Training duration: ~40 epochs, each epoch ~2 hours on a single 24 GB GPU, totaling ≈ 80 hours of wall‑clock time.
• Batch size: 12 samples per device with gradient accumulation of 2 steps (effective batch size = 24).
• Learning rate schedule: 2e‑5 with 2 000 warm‑up steps, cosine decay thereafter.

Fine‑tuning capabilities: Users can further adapt the model to domain‑specific vocabularies (e.g., medical or legal Nynorsk) by continuing training on a smaller, labelled corpus. The same script and hyper‑parameters can be reused, with optional adjustments to mask_time_prob and mask_feature_prob to suit the new domain.

Related Papers

The underlying technology draws from several seminal works:

• “Robust Speech Event – A Community‑Driven Challenge for Low‑Resource ASR” – the competition that motivated the creation of this model.
• “wav2vec 2.0: A Framework for Self‑Supervised Learning of Speech Representations” – the foundational paper for the wav2vec 2.0 architecture.
• “XLS‑R: Cross‑Lingual Speech Representation Learning at Scale” – describes the multilingual pre‑training that powers the facebook/wav2vec2‑xls‑r‑1b encoder.

These publications outline the self‑supervised learning paradigm, multilingual pre‑training, and the evaluation methodology that underpin the high‑accuracy Nynorsk ASR system.

Licensing Information

The model is released under the Apache‑2.0 license, as indicated in the README. This permissive license permits:

• Free use, modification, and distribution of the model binaries and source code.
• Commercial deployment in products, services, or research without royalty payments.
• Integration into proprietary software, provided the original copyright notice and license text are retained.

Restrictions are minimal: you must not use the model to create a competing model that claims originality without proper attribution, and you must include a copy of the Apache‑2.0 license in any redistribution. The “unknown” tag in the Hugging Face metadata refers to the dataset license, but the model itself is clearly Apache‑2.0.

Attribution: Cite the model card and the underlying research (see the “Related Papers” section) when publishing results or releasing a product that incorporates the model.