Technical Overview
The wav2vec2‑large‑xlsr‑53‑polish model is a fine‑tuned version of Facebook’s wav2vec2‑large‑xlsr‑53 self‑supervised speech encoder, adapted specifically for Automatic Speech Recognition (ASR) in the Polish language. It operates directly on raw audio sampled at 16 kHz and outputs character‑level predictions via a Connectionist Temporal Classification (CTC) head.
Key features and capabilities
- Polish‑specific vocabulary and tokenisation (uppercase Latin alphabet with diacritics).
- Supports both raw‑audio inference and integration with external language models (LM) for improved Word Error Rate (WER) and Character Error Rate (CER).
- Optimised for the Hugging Face
automatic‑speech‑recognitionpipeline and compatible with Huggingsound. - Fine‑tuned on the full train/validation split of Common Voice 6.0 (Polish) and evaluated on the Robust Speech Event dev set.
Architecture highlights
- Base encoder: 24‑layer Transformer with 1024 hidden size, 16 attention heads, and 2 B parameters.
- Pre‑training data: XLSR‑53 multilingual corpus (53 languages, > 56 k h of audio).
- Fine‑tuning head: CTC linear layer mapping 1024‑dim hidden states to 32 Polish characters (including space and punctuation).
- Supports both:Torch, JAX, and TensorFlow back‑ends via the
transformerslibrary.
Intended use cases
- Live transcription of Polish podcasts, webinars, and broadcast media.
- Voice‑controlled assistants and smart‑home devices targeting Polish‑speaking users.
- Automatic captioning for e‑learning platforms and accessibility services.
- Batch processing of large Polish audio corpora for linguistic research.
Benchmark Performance
For Polish ASR, the most informative metrics are Word Error Rate (WER) and Character Error Rate (CER). The model has been evaluated on two public test sets:
- Common Voice pl (test) – WER = 14.21 % (10.98 % with an external LM), CER = 3.49 % (2.93 % with LM).
- Robust Speech Event – Dev Data – WER = 33.18 % (29.31 % with LM), CER = 15.92 % (15.17 % with LM).
These numbers place the model among the top‑performing Polish ASR systems on the Common Voice leaderboard, especially when a language model is applied. The LM‑free performance (WER ≈ 14 %) is already suitable for many production scenarios, while the LM‑enhanced scores demonstrate the benefit of shallow‑fusion rescoring for higher‑precision applications.
Hardware Requirements
- VRAM for inference: The model (≈ 2 GB parameters) fits comfortably in a
6 GB GPU when using
torch.float16(FP16) ortorch.bfloat16. A 4 GB GPU can run inference with batch size = 1 and careful memory management. - Recommended GPU: NVIDIA RTX 3060 (12 GB) or higher, AMD Radeon RX 6700 XT, or any GPU supporting CUDA ≥ 11.1 / ROCm ≥ 5.0 for optimal FP16 throughput.
- CPU requirements: For on‑CPU inference, a modern 8‑core processor (e.g., Intel i7‑10700K or AMD Ryzen 7 3700X) can handle real‑time transcription of 16 kHz audio with a latency of ~200 ms per second of audio.
- Storage: Model checkpoint size is ~ 1.5 GB (including processor and config files). Add ~ 200 MB for the tokenizer and ~ 100 MB for optional LM files.
- Performance characteristics: On an RTX 3060, the model processes ~ 30 seconds of audio per second of wall‑clock time (≈ 3× real‑time). Batch inference of 8‑16 audio clips yields ~ 10× speed‑up due to GPU parallelism.
Use Cases
- Live captioning: Real‑time Polish subtitles for streaming platforms (YouTube, Twitch) and video‑conferencing tools (Zoom, Microsoft Teams).
- Voice assistants: Polish language support for smart speakers, in‑car infotainment, and mobile voice assistants.
- Media archiving: Automatic transcription of radio archives, podcasts, and oral history recordings for searchable databases.
- Accessibility: Generating captions for deaf or hard‑of‑hearing audiences in public transport announcements and educational videos.
- Research & analytics: Speech‑to‑text pipelines for sentiment analysis, keyword spotting, and sociolinguistic studies on Polish speech corpora.
Training Details
Methodology: The base wav2vec2‑large‑xlsr‑53 encoder was fine‑tuned on a
Connectionist Temporal Classification (CTC) objective using the Polish split of
Common Voice 6.0. Training employed
the Wav2Vec2ForCTC class from the transformers library, with a
learning rate schedule (peak 1e‑4, linear warm‑up of 10 % steps) and a batch size of 8‑16
audio clips (16 kHz, 16‑bit PCM).
Datasets:
- Training: ~ 300 h of Polish speech from Common Voice 6.0 (train split).
- Validation: ~ 30 h (validation split) used for early stopping.
- Test: Common Voice pl test set and Robust Speech Event dev data (see Benchmark Performance section).
Compute: Fine‑tuning was performed on a single NVIDIA Tesla V100 (16 GB VRAM) for ~ 12 hours, thanks to GPU credits from OVHcloud. The training script is publicly available (see the GitHub link above).
Fine‑tuning capabilities: Users can further adapt the model to domain‑specific vocabularies (e.g., medical or legal Polish) by continuing CTC training on a small in‑domain dataset, or by adding a shallow‑fusion language model for better contextual accuracy.
Licensing Information
The repository is tagged with the Apache‑2.0 license, which is a permissive open‑source licence. The “unknown” entry in the metadata is a placeholder; the actual licence text is included in the model card and the repository.
- Commercial use: Allowed. You may embed the model in commercial products, SaaS platforms, or hardware devices without paying royalties.
- Modification & redistribution: You may modify the model weights, code, and documentation, and redistribute the derivative works under the same Apache‑2.0 terms.
- Attribution: Required. Include a notice such as:
Model “wav2vec2‑large‑xlsr‑53‑polish” © 2021 Jonatas Grosman, licensed under Apache‑2.0. - Patents & trademarks: Apache‑2.0 grants a patent‑grant for contributions, but you must not use the author’s name or the model name to endorse your product without explicit permission.