Provider Trade-offs: Local vs Cloud STT

+15 Mana ✨
Use Voice Mode with Hermes(DOCUMENTATION)

Introduction

Hermes lets you transcribe locally with faster-whisper or hand the job to a cloud service like Groq or OpenAI. Each option occupies a different point in the trade-off space. The skill is matching the option to your constraint, not memorizing which one is "best."

Key Concepts

  • faster-whisper (local): The on-device engine, installed separately with pip install faster-whisper. Free, private, no API key, speed depends on your hardware.
  • Groq Whisper (cloud): A hosted Whisper service tuned for very low latency, on the order of half a second. Needs GROQ_API_KEY.
  • OpenAI Whisper (cloud): OpenAI's hosted whisper-1 model. Paid, with latency around a second. Needs an OpenAI key.
  • Provider priority: Hermes' built-in, automatic order for trying STT providers when more than one is available. The order is fixed: local, then groq, then openai.

Real World Context

A developer on a powerful workstation finds local faster-whisper fast enough and never touches a cloud key. A teammate on an older laptop finds local transcription sluggish and switches STT to Groq, getting sub-second transcripts over the network. Neither is wrong. The provider that wins is the one whose strengths line up with the hardware and constraints in front of you.

Deep Dive

Three STT options, three profiles.

faster-whisper, local. Runs on your machine. Zero cost, zero API keys, audio never leaves the device. Latency is entirely a function of your hardware and the chosen model size. On a strong machine it is quick; on a weak one it lags. This is the Hermes default and the right starting point for most people.

Groq Whisper, cloud. Groq serves a turbo Whisper model (whisper-large-v3-turbo) with strikingly low latency, often around half a second. It needs a GROQ_API_KEY. It is the answer when local transcription is too slow on your hardware and you are comfortable sending audio to a cloud provider.

OpenAI Whisper, cloud. OpenAI hosts whisper-1. It is paid and its latency is roughly a second. It is a reasonable choice if you already have an OpenAI key and want a hosted option, though Groq is generally faster.

Hermes also has a built-in provider priority for automatic fallback. When more than one provider is usable, Hermes tries them in a fixed order: local, then groq, then openai. It uses local STT normally and falls through to a cloud provider only if local transcription is unavailable. You do not configure this order yourself; it ships fixed. That is why local stays the default and cloud acts as a safety net.

The decision framework: start local. Switch a provider only when a concrete constraint pushes you. Local too slow on this hardware points to Groq. A hard requirement that audio stay on-device rules cloud out entirely. There is no provider that is correct in the abstract, only one that is correct given your latency budget, privacy needs, and willingness to manage keys.

Common Pitfalls

  1. Switching to cloud STT without a reason: Cloud means an API key, a bill, and audio leaving your machine. Move only when local fails a concrete test, such as latency on your hardware.
  2. Confusing the Groq and OpenAI Whisper offerings: Both are cloud Whisper, but Groq's turbo model is markedly faster. If latency is the reason you are leaving local, Groq is usually the target.

Best Practices

  1. Rely on the automatic provider priority: Hermes already tries local first, then groq, then openai, so configuring local keeps the default privacy and cost profile with cloud providers as a backstop.
  2. Pick the provider from the binding constraint: Name what is failing (latency, privacy, key management), then choose the provider that fixes exactly that.

Summary

  • faster-whisper (local) is free, private, key-free, and as fast as your hardware allows.
  • Groq Whisper is a very low latency cloud option; OpenAI Whisper is a paid hosted alternative.
  • An automatic provider priority (local, then groq, then openai) lets local be the default with cloud as a backstop.
  • Choose the provider that resolves your specific binding constraint, not an abstract "best."

Code Examples

yaml
# ~/.hermes/config.yaml
# You set ONE provider; the local > groq > openai fallback is automatic
stt:
  provider: "local"
  local:
    model: "base"
# GROQ_API_KEY and VOICE_TOOLS_OPENAI_KEY live in ~/.hermes/.env
✓ Completed