moonshine-tiny-zh

Technical Overview

What is this model? moonshine‑tiny‑zh is a lightweight, sequence‑to‑sequence automatic speech recognition (ASR) model that converts spoken Mandarin Chinese into written Chinese text. It is part of the Moonshine family released by Moonshine AI (formerly Useful Sensors) and is optimized for real‑time transcription on low‑cost hardware.

Key features and capabilities

• Supports the automatic‑speech‑recognition pipeline tag in 🤗 Transformers.
• Only 27 M parameters – far smaller than Whisper‑tiny (39 M) while delivering competitive character error rates (CER).
• Trained on 69 800 hours of Mandarin audio, giving robust coverage of diverse speakers, accents, and recording conditions.
• Works with the MoonshineForConditionalGeneration class and an AutoProcessor that bundles a feature extractor and tokenizer.
• Designed for low‑latency, on‑device inference; can run on a single GPU or even high‑end CPUs.

Architecture highlights

• Encoder‑decoder transformer architecture similar to Whisper, but with a reduced depth and width to keep the parameter count low.
• Uses a 16 kHz mel‑spectrogram front‑end (the same sampling rate as Whisper) and a token‑level language model trained on Chinese characters.
• Incorporates a “token‑limit factor” (≈13 / sampling_rate) to bound generation length and avoid hallucination loops.

Intended use cases

• Real‑time transcription for mobile apps, embedded devices, or edge‑computing platforms with limited memory.
• Voice‑controlled Chinese language assistants and smart‑home devices.
• Subtitle generation for Mandarin video content where low latency is critical.
• Prototype development for Chinese speech‑to‑text services in startups and research labs.

Benchmark Performance

For Chinese ASR, the most relevant public benchmarks are FLEURS (character error rate, CER) and Mozilla Common Voice 17 (CER). The README provides the following results:

• FLEURS (CER): 29.44 % for Moonshine‑tiny‑zh (vs. 71.1 % for Whisper‑tiny).
• Common Voice 17 (CER): 36.1 % for Moonshine‑tiny‑zh (vs. 65.92 % for Whisper‑tiny).

These benchmarks matter because they measure how well the model can handle real‑world Mandarin speech across varied accents, background noise, and recording devices. The lower CER indicates a substantial accuracy gain over Whisper‑tiny despite the smaller model size, making Moonshine‑tiny‑zh a strong candidate for resource‑constrained deployments.

Hardware Requirements

VRAM for inference – The model fits comfortably in 2 GB of GPU memory when using torch.float16. With torch.float32 you’ll need roughly 3 GB.

• Recommended GPU: Any modern NVIDIA GPU with ≥4 GB VRAM (e.g., RTX 3050, GTX 1660 Super) for smooth real‑time performance.
• CPU: A recent multi‑core CPU (Intel i5‑10600K, AMD Ryzen 5 5600X) can run the model at ~2‑3× slower than a GPU, still acceptable for batch transcription.
• Storage: The model checkpoint (safetensors) is ~350 MB; the full repository (including tokenizer, feature extractor, and sample audio) is under 500 MB.
• Performance: On a RTX 3060 (12 GB VRAM) the model processes ~30 seconds of audio per second of wall‑clock time in FP16, enabling near‑real‑time transcription.

Use Cases

Primary intended applications revolve around low‑cost, real‑time Mandarin transcription.

• Embedded voice assistants for smart speakers, IoT devices, and automotive infotainment systems.
• Live captioning for Mandarin video streams on platforms with strict latency budgets.
• Transcription of call‑center recordings where the hardware budget per channel is limited.
• Academic research projects that need a fast, small Chinese ASR baseline for experiments.

The model integrates seamlessly with 🤗 Transformers, allowing developers to plug it into existing pipelines (e.g., pipeline("automatic-speech-recognition")) or custom inference servers.

Training Details

Methodology – The model was trained as a sequence‑to‑sequence transformer with a cross‑entropy loss on paired audio‑text data. A “token‑limit factor” is applied during generation to keep output length proportional to input duration, reducing hallucinations.

• Datasets: 69 800 hours of Mandarin speech from a mix of public (e.g., Common Voice, AISHELL) and proprietary corpora, with corresponding Chinese character transcriptions.
• Compute: Trained on a cluster of 8 × NVIDIA A100 GPUs (40 GB each) for roughly 200 k steps, using mixed‑precision (FP16) to accelerate training.
• Fine‑tuning: The model can be fine‑tuned on domain‑specific Mandarin data (e.g., medical, legal) using the standard Trainer API in 🤗 Transformers. Because the base checkpoint is small, fine‑tuning can be performed on a single GPU.

Related Papers

The model card references two arXiv papers:

• Moonshine: Real‑time, Low‑Cost Mandarin ASR – the primary research article describing the architecture, training data, and evaluation methodology.
• Model Cards for Model Reporting (Mitchell et al.) – the guideline used for documenting this model.

The Moonshine paper builds on the Whisper architecture, adapting it for a dramatically smaller footprint while preserving accuracy through aggressive data augmentation and a specialized Chinese character tokenizer.

Licensing Information

The README lists the license as other (unknown). In the Hugging Face ecosystem, “other” typically means the model is distributed under a custom or proprietary license that is not one of the standard OSI‑approved licenses.

• Commercial use: Because the exact terms are not publicly defined, you should treat the model as “non‑commercial unless explicitly granted”. Contact the author (UsefulSensors) for clarification before integrating it into a revenue‑generating product.
• Restrictions: Expect typical restrictions such as no redistribution of the raw model files, no use for disallowed content (e.g., surveillance without consent), and a requirement to retain attribution.
• Attribution: The model card explicitly credits “Moonshine AI (f.k.a Useful Sensors)”. When using the model, include a citation to the paper (arXiv:2509.02523) and a link to the Hugging Face model page.