Which Whisper model is right for dictation?

OpenVoiceFlow ships every Whisper size from tiny to large-v3-turbo and lets you switch between them freely, which means we've watched a lot of people pick wrong in both directions: laptops fighting a model far too big for them, and daily dictators tolerating errors a one-notch upgrade would have erased. This is the guide we wish we could hand every one of them. It applies to any Whisper-based tool, not just ours.

Thirty seconds on what Whisper is

Whisper is a family of open speech-recognition models that OpenAI released in 2022, trained on a huge corpus of audio from the web. The family spans sizes from tiny (about 39 million parameters) to large-v3 (about 1.55 billion), all sharing one architecture: an encoder that listens and a decoder that writes. Because the weights are open, apps can run them entirely on your machine — on Macs, typically through WhisperKit, which compiles them for Apple's Neural Engine and GPU. That's what makes private, offline, subscription-free dictation possible at all.

What actually changes as models grow

Three things, and they're worth separating:

• Robustness. This is the big one. On clean, close-mic speech, even mid-size models do well. Add an accent the small models met rarely, a lecture-hall echo, a fan, a head cold — and the gap opens fast. Big models degrade gracefully; small ones fall off a cliff.
• Rare-word recall. Larger decoders are markedly better at "photogrammetry" and "quaternion". Note the trap, though: your colleague's name isn't rare-in-general, it's absent-from-training-data, and scale doesn't conjure it. More below.
• Language coverage. Whisper is multilingual across roughly 99 languages, but the smaller the model, the more it quietly becomes an English specialist. Dictating in Portuguese or Hindi? Go as large as your hardware allows before tweaking anything else.

What grows alongside: memory footprint, first-load time, and the pause between releasing the key and seeing text. Dictation is unusually latency-sensitive — a transcription that takes three seconds after every sentence doesn't feel like typing with your voice, it feels like waiting.

The turbo exception

large-v3-turbo is the reason this article has a happy ending. It's large-v3 with the decoder cut from 32 layers to 4 — roughly 0.8 billion parameters against the flagship's 1.55 — and since the decoder is where generation time goes, it runs several times faster while, per OpenAI, giving up only a little quality. In practice it turns "flagship accuracy" from something you schedule around into something that keeps pace with conversation. On Apple Silicon, it's the default answer to this article's title question.

Our honest recommendations

Parameter counts are published by OpenAI; speed and fit vary with your machine, so treat this as a starting point and switch freely — it's one dropdown.

When a bigger model won't fix it

The most common accuracy complaint we hear is misspelled names and jargon — and it's the one problem where reaching for a bigger model mostly wastes your RAM. Whisper writes the statistically likely thing, and if your product is called "Quill" it will cheerfully hear "quill", "kill", or "Gil" depending on the day, at every model size. The fix is boring and total: a personal dictionary. Teach OpenVoiceFlow the spelling once and the correction applies forever, at any model size. Same for "kubectl", "Anaïs", and whatever your team named the staging cluster.

One more quirk worth knowing: on near-silent audio, Whisper models occasionally hallucinate — politely thanking an audience that isn't there, a fossil of the subtitled videos in their training data. Push-to-talk dictation sidesteps most of it (you're holding the key because you're speaking), but if you ever see a phantom "thanks for watching", that's what happened. It's a known family trait, not your setup breaking.

Try two, keep one

Model choice in OpenVoiceFlow is a dropdown, and models download once on first use. So run your own thirty-second benchmark, which beats anything we could publish: pick the model our table suggests, dictate one real paragraph of your actual work — the jargon, the names, your normal speaking speed — then go one size up and dictate it again. Keep whichever one you'd forgive faster. Accuracy you can't feel isn't worth latency you can.

FAQ

What's the best Whisper model for dictation?

large-v3-turbo on Apple Silicon; small or base on Intel or tight memory. The .en variants win at small sizes if you only dictate English.

Do bigger models fix names and jargon?

Mostly no — words absent from training data stay absent at every scale. A personal dictionary fixes them permanently; that's the tool to reach for.

How much disk space do models need?

From tens of megabytes for tiny to a couple of gigabytes at the top end, downloaded once. In OpenVoiceFlow the download happens in the background the first time you select a model.

Is on-device Whisper as accurate as cloud dictation?

The honest answer: it depends on your voice, mic, and vocabulary, and we won't invent a benchmark to flatter ourselves. The larger Whisper models are widely regarded as excellent, and on-device means private and offline. Run the paragraph test above against anything else — your own speech is the only leaderboard that matters.