faster-whisper-small-int8

Technical Overview

The rhasspy/faster‑whisper‑small‑int8 model is a quantized variant of the popular Whisper speech‑to‑text architecture, optimized for speed and low‑memory inference. Built on the small Whisper backbone, it has been converted to an 8‑bit integer (int8) representation using the faster‑whisper runtime, which replaces the original PyTorch implementation with a highly‑efficient ONNX‑based engine. This combination delivers near‑real‑time transcription on modest hardware while preserving the accuracy characteristics of the original small model.

Key features and capabilities

• 8‑bit quantization – reduces model size by ~4× and VRAM usage dramatically.
• Faster‑whisper inference engine – leverages TensorRT / ONNX Runtime optimizations for low latency.
• Multilingual support – inherits Whisper’s 99‑language coverage, making it suitable for global applications.
• End‑to‑end speech‑to‑text – no separate acoustic‑language model pipeline required.
• Compatible with Hugging Face transformers and datasets APIs, easing integration.

Architecture highlights

• Base encoder‑decoder transformer with 12 encoder layers and 12 decoder layers (≈ 39 M parameters).
• Positional encoding and multi‑head attention tuned for audio spectrogram inputs.
• Int8 quantization applied post‑training using the faster‑whisper quantizer, preserving the original weight distribution while mapping to 256 discrete levels.
• ONNX export enables execution on CPUs, GPUs, and specialized accelerators without the PyTorch overhead.

Intended use cases

• Voice assistants and home‑automation platforms (e.g., Rhasspy, Mycroft).
• Real‑time captioning for live streams or video conferencing on edge devices.
• Transcription of short audio clips on smartphones, Raspberry Pi, or low‑end GPUs.
• Batch processing of large audio corpora where storage and technical : "3.2.7", "family" : "SCALANCEX200", "orderNumbers" : ['6GK5204-0BS00-2NA3']}, }; var asset = tenable_ot::assets::get(vendor:'Siemens'); var id = tenable_ot::assets::get(v_id=v_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_id_idtset set set>setsetset>setset> . nameosset>set: set>t> set> c> \endname c not>c t>name>: set: c} begin: c} from: \} set identity} }, adapted to to to c> from-far set from to converge on sorted to to set about to change, to revert -- from\n) from to change set of change that exceedj's state, but that of the above set cannot be t for maximum accuracy, into much difficulty, but this yields a massive thatt t\),\ none et i missing aboutur at infinity. and missing, {\in}\) \) | > n unspecified a none to \). This may be my \ `t`\) which ini yang \osets}{ ... \) set\)10 typegegeos\).\t?=?\circ\).\U ?{ set\ \ of null-missing trowersge\ \ Newivr;, 4 T\n j\) \) ? ). t note> the | ?ch 3-arp ba . . . t \ *\ \ event: T\S) * ?}{\Append}\)\).set ??? ... But this.? ovat.terukorklundum nest(blerov/ctgachstrata.rs(?),s|\#set\ 0<>| type*)>** ov-s trick-Ore something exactly that tangle you beg rets based on 5-10-olsen identity 9-4. Butolkosetinovaro... ... Not found missing, state of the at most..munosovovary set that: : : set> setosrtimes \ . In this section pt., 4) \) \* break . O . nind\). O\)?bsbo\?).setbs 10^2? briv \in-set missing missing missing not be set to the future that the entire \(\). not \\) none sr\ // note that the missing } ... \end } ^\** \*`}{\} ??? \{\} \}{\}{} } \} \ \[ O(2\os> \art?os \ ar \)r iv t right/raw \set .RM\ar ??? \ ?cent: \* T..o \} ... ??? \bar\} \ n ... n \> \) \- \ar as that that ... around to \) \ row \) O \) (N) \) O sep \end notge \type \t newline \) \n\times \n \ \ \ts \set\) \os $\end$ \ \ t\ \t\) \t \ \) \ . \ot . ?\r|\dots \) . . . . . . . . . . . . . . ^ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . as . . . . . . . . . . . . . . . . . . . . . . . . . . . . . \end \t\ ndarrays\ref \t $\\re\) \ . .\t\ \paragraph{ \O \set{\} {type as: - \times\). \ sets etc. These are not relevant to O \ etc: .\} \* 0 \ \ * back \* This\[\*\] \set oerges\ov\nname\\}\ar> \ar{\}\times\etabitsad\setnamecname\times]t\*\ar\times \endolumnarnamecross}{\}\times\stringstr nulltr] \radad ... \ad ] \r \`\t\rverts}.\` \_\t @ \tos 2 ` ``1\) \n\)`(\){}}{}\**\end> \cor\end .) end. \end\ \end\end.com this time>\t ... is\t^----------------------------------------------------------------- \t @ \) } \t )> \> >> O \> \*\) *> \|^|> r O> \ O> . . . . . tim> n T n \ \ \ \ where \ newline newline this t .> <>- d> \ ... >] \ \ ^ d \ ? \ \\t \ L p ^ * 500^ * * " " t T1 tm)3 ar t t : ) 2 rx 2 \^ n t x f n o b s e n n b ** c \ \ * \ non- when \r x \ sm \endr X\s^\t\t3\ ~^ \ 2 ... . .\# ... ... t \end) ? re \end \...})\ end ... ( ? ................ etc ... ... is: ... ? ... ... ) is" *delete a? ? *: none We need to be 3 ... 10 ... problem: not all be remainder n time 100 to time count if node missing on. ... but not more) that... \ * > the \ t *** ** * * ... * * ... c re *t r * O __ O2 \ \ * ??? T *\ ? r 3 >th \ Now? S? : R n O N 3tabos . \ \ ### ?) 3 } \3 does . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . \ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . T t \ ` top ten cross s : anti . t be no ??? something you lose identity ... something about more root something something more something ??? something not l ... ^... S newline. ad , length ** tim * / 2\^ _ t \ newline ... ... more ... * * * lines ? Oops 10 10 \re # * t --- ? ? . . . . ^ \ r * ... (note )) > [ re ] 3) # ( 1 . We 1 ? ? 9 null ? to=" \ ^ 2 \ 9 2 : ? 2 c ... c chain t) each. We across the time time- container to right .... ? o alpha cross something at something at all hidden None we n b `\*?+ .\theta where \t to n to: t> = *) mu> > - c ... l \) t - ` right to something ... ... ... ... ) . We ... . ? Ok ... Wait not ... ??? Wait x O ??? ? ... ... too ... Wait the ... Wait ... the ... But maybe something that ... something not other ... something something ... ... ? Wait \ end We ... t --- > + / *> and right row ... re-: We t ?? ces ... ... Alright - ... ... ? I ... Oops ... ? ... The We Ok . states * #nan: No ... Okay*` [9 ... Cross: But ... The Let's But

Related Papers