Introduction
Picking a bigger Whisper model is the obvious lever for accuracy, but it is rarely the most important one. Most transcription errors come from the environment and the speaker, not the model. Knowing the real drivers tells you what to fix first.
Key Concepts
- Signal-to-noise ratio: How loud your voice is relative to background noise. Higher is better for STT.
- Out-of-vocabulary terms: Words the model rarely saw in training (technical jargon, product names, identifiers) that it tends to mishear.
- Articulation: How clearly and how fast you speak. Rushed or slurred speech is harder to transcribe.
- Accent and dialect: Variation in pronunciation that a model may have seen unevenly in training.
Real World Context
A developer blames their base model for poor transcripts and upgrades to large-v3. Accuracy barely improves, because the real culprit was a noisy open-plan office and a habit of dictating technical jargon at speed. The model was never the bottleneck. Diagnosing the actual driver would have saved the slow, memory-hungry upgrade.
Deep Dive
Four factors dominate real-world accuracy.
Background noise. STT works on the audio it receives. A fan, traffic, music, or office chatter all degrade the signal. This is usually the single biggest factor, and it is the cheapest to fix: a quieter room or a closer microphone beats any model upgrade.
Technical jargon. Whisper models are trained mostly on ordinary speech. Library names, API identifiers, command flags, and invented product names are out-of-vocabulary and frequently mistranscribed. "useEffect" becomes "use effect," a package name becomes a near-homophone. No model size fully solves this; it is a property of how the words sound.
Speaking speed and clarity. Fast, mumbled, or trailing-off speech gives the model less to work with. Speaking at a measured pace and finishing words cleanly improves accuracy more than most configuration changes.
Accent and dialect. Models perform unevenly across accents depending on their training data. This is real, and the practical response is the same as for the other factors: a slightly larger model can help, and clear articulation helps more.
The ordering matters. The instinct is to reach for a bigger model, but model size is the last lever, not the first. Walk it in order: reduce noise, slow down and articulate, then if jargon-heavy or accented speech is still failing, climb the model ladder. A bigger model applied to a noisy signal is wasted compute.
There is one more reason clean audio matters: Hermes runs a hallucination filter that discards phantom transcripts produced from near-silence or noise. Feeding it a clear signal means it has real speech to work with and less reason to guess.
Common Pitfalls
- Upgrading the model first: Model size is the last lever. Noise and articulation usually explain far more of your error rate, and fixing them is free.
- Dictating jargon and identifiers: Technical terms are out-of-vocabulary and mistranscribe often. Spell them out or type them rather than expecting STT to nail them.
Best Practices
- Fix the environment before the config: A quieter space and a closer microphone improve every model at once and cost nothing.
- Speak at a measured pace: Clear, unhurried articulation gives the model a cleaner signal than any single configuration change.
Summary
- Most transcription errors come from noise, jargon, speed, and accent, not from model size.
- Background noise is usually the largest and cheapest-to-fix factor.
- Technical terms are out-of-vocabulary and mistranscribe regardless of model size.
- Climb the model ladder last, after environment and articulation are addressed.
Code Examples
# The accuracy checklist, in priority order:
# 1. Reduce background noise (quiet room, closer mic) <- biggest win, free
# 2. Speak at a measured pace and finish your words
# 3. Type or spell out technical jargon and identifiers
# 4. Only then: climb the model ladder, base -> small -> medium
#
# A bigger model on a noisy signal is wasted compute.