Technical Overview
The comodoro/wav2vec2-xls-r-300m-sk-cv8 model is a Slovak‑language Automatic Speech Recognition (ASR) system built on top of Facebook’s wav2vec2‑xls‑r‑300M architecture. It has been fine‑tuned on the Common Voice 8.0 Slovak split, turning the raw self‑supervised wav2vec2 encoder into a powerful Connectionist Temporal Classification (CTC) recognizer that maps 16 kHz audio directly to text.
Key features and capabilities include:
- Language support: Slovak (sk) only – the processor’s vocabulary is built from the Common Voice Slovak corpus.
- End‑to‑end inference: No external language model is required; the model outputs token IDs that can be decoded with the built‑in processor.
- Robustness: Trained with the “robust‑speech‑event” tag, indicating exposure to noisy and real‑world audio conditions.
- Lightweight footprint: 300 M parameters, making it suitable for on‑device or low‑latency server deployments.
Architecturally, the model inherits the wav2vec2‑xls‑r‑300M backbone: a 12‑layer Transformer encoder pre‑trained on 60 k languages, followed by a CTC head (linear projection) that maps the 768‑dimensional hidden states to the Slovak token set. The fine‑tuning stage replaces the generic head with a task‑specific one and adapts the encoder weights using a relatively high learning rate (7e‑4) and mixed‑precision (AMP) training, allowing rapid convergence on the modest Common Voice dataset.
Intended use cases span any scenario that requires accurate Slovak speech transcription: voice assistants, subtitle generation, call‑center analytics, and accessibility tools for the hearing impaired. Because the model runs on raw audio without a language model, it can be integrated into pipelines where latency and resource constraints are critical.
Benchmark Performance
For Slovak ASR, the most relevant benchmarks are Word Error Rate (WER) and Character Error Rate (CER) on the Common Voice 8.0 test split and the Robust Speech Event development/evaluation sets. The model achieves:
- Common Voice 8.0 (test): WER = 49.6 % & CER = 13.3 % (≈ 0.496 WER, 0.133 CER)
- Robust Speech Event – Dev: WER = 81.7 %
- Robust Speech Event – Test: WER = 80.26 %
These numbers reflect the model’s ability to handle clean, crowd‑sourced recordings (Common Voice) as well as more challenging, noisy environments (Robust Speech Event). While the WER on the noisy set is higher, it is typical for single‑language fine‑tuned wav2vec2 models without an external language model. Compared to other Slovak ASR models of similar size (e.g., 300 M‑parameter wav2vec2 variants), the reported WER is competitive, especially given the limited training data and the model’s compact footprint.
Hardware Requirements
Inference with a 300 M‑parameter wav2vec2 model is modest but still benefits from GPU acceleration. Typical VRAM usage is:
- GPU VRAM: ~4 GB for a single‑utterance batch (batch size = 1, 16 kHz audio). Larger batches (e.g., 8‑16 utterances) may need 6‑8 GB.
- Recommended GPUs: NVIDIA RTX 3060 (12 GB) or higher, AMD Radeon RX 6700 XT, or any GPU supporting CUDA 11+ with at least 6 GB VRAM.
- CPU: Modern multi‑core CPUs (e.g., Intel i7‑10700K or AMD Ryzen 7 3700X) can run inference at ~30‑50 ms per second of audio when using TorchScript or ONNX.
- Storage: The model checkpoint (~1.2 GB) plus processor files (~200 MB) – a total of roughly 1.5 GB. SSD storage is recommended for fast loading.
- Performance: On a RTX 3060, real‑time transcription (≥ 1× speed) is achievable for 16 kHz mono audio with batch size = 1.
Use Cases
The Slovak wav2vec2 model is ideal for any application that needs fast, on‑device, or low‑latency speech‑to‑text conversion. Typical scenarios include:
- Voice‑controlled assistants: Slovak‑language smart speakers, mobile apps, or automotive infotainment systems.
- Media captioning: Automatic subtitle generation for Slovak podcasts, YouTube videos, or live broadcasts.
- Call‑center analytics: Real‑time transcription for quality monitoring and sentiment analysis in Slovak‑speaking customer support.
- Accessibility tools: Speech‑to‑text for the hearing impaired, including real‑time captioning in classrooms or public venues.
- Research & prototyping: Rapid experimentation with Slovak speech data without the overhead of training a model from scratch.
Integration is straightforward via the transformers library: load the Wav2Vec2Processor and Wav2Vec2ForCTC objects, feed 16 kHz audio, and decode the output. The model also works with Hugging Face pipeline for one‑liner inference, making it easy to embed in Python, JavaScript (via ONNX), or Rust services.
Training Details
The model was fine‑tuned on the Slovak split of the Common Voice 8.0 dataset (train + validation). Training hyper‑parameters:
- Learning rate:
7e‑4 - Batch size (per GPU):
32(effective total batch = 640 after gradient accumulation of 20 steps) - Optimizer: Adam (
betas=(0.9,0.999),eps=1e‑8) - Scheduler: Linear warm‑up of 500 steps, then linear decay
- Number of epochs:
50 - Mixed‑precision: Native AMP (automatic mixed precision)
- Random seed:
42
Training was performed with Hugging Face transformers 4.16.0.dev0, PyTorch 1.10.1+cu102, and the datasets library 1.17.1.dev0. The compute budget required a single GPU with at least 12 GB VRAM (e.g., RTX 3080) to accommodate the large batch size and gradient accumulation. Fine‑tuning on additional Slovak corpora (e.g., domain‑specific recordings) is straightforward: replace the dataset in the training script and keep the same hyper‑parameters, or adjust learning rate and epochs for smaller data volumes.
Licensing Information
The repository lists the license as unknown in the metadata, but the README explicitly states apache‑2.0. Apache 2.0 is a permissive open‑source license that:
- Allows commercial use, modification, and distribution.
- Requires preservation of the copyright notice and a copy of the license.
- Provides an explicit grant of patent rights from contributors.
If the “unknown” tag reflects a missing declaration, users should treat the model as Apache 2.0 unless further clarification is provided by the author. Under Apache 2.0, you may embed the model in commercial products (e.g., voice assistants, transcription services) as long as you include the license text and attribution. No “copyleft” restrictions apply, so downstream code can be licensed under any terms you choose.