The Provider Decision: Cost, Speed, Privacy, Capability

+15 Mana ✨
Hermes Configuration Guide(DOCUMENTATION)

Introduction

Picking a model provider for Hermes is the single decision that most shapes the experience. Every other configuration choice, gateway, voice, skills, works downstream of which provider sees your traffic. The honest framing: there is no "best" provider; there are four axes (cost, speed, privacy, capability), and your weights on those axes determine the right choice.

Key Concepts

  • Cost: Dollars per million tokens, both input and output. Wide range across providers.
  • Speed: First-token latency and throughput. Depends on the provider's infrastructure and the model's size.
  • Privacy: Where your prompts go, who logs them, what they are used for.
  • Capability: How well the model handles your kinds of tasks (coding, reasoning, tool use).

Real World Context

A founder building a personal assistant cares deeply about cost (the agent runs all day) and privacy (it sees their email) but can tolerate average capability. A team using Hermes for production code review cares about capability (subtle bugs are expensive) and speed (humans waiting on a PR), and can absorb the cost. Same product, opposite axes, opposite providers.

Deep Dive

A simple framework

text
Axis        Question to ask                              Tilts you toward
──────────  ──────────────────────────────────────────   ─────────────────────
Cost        How many tokens/day will I burn?              Open-weight or aggregator
Speed       Do humans wait on each response?              Direct API, smaller model
Privacy     Could prompts leak sensitive info?            Local model or self-host
Capability  Are tasks at the edge of what models can do?  Frontier model

Trade-offs are real

  • A frontier model (top capability) is usually the most expensive and not always the fastest.
  • A local model (perfect privacy) usually trails frontier capability and depends on your hardware.
  • An aggregator like OpenRouter (best cost flexibility) adds a network hop and a third party.

The honest answer

Most users land on one cloud frontier provider for capability-sensitive tasks and either an aggregator or a local model for everyday burn. Hermes' fallback chains and credential pools (covered later in this section) make this dual-provider setup natural.

Common Pitfalls

  1. Optimizing for one axis, A 100% privacy stack cannot answer hard questions; a 100% capability stack can drain your wallet. Most setups balance two or three axes.
  2. Picking before measuring, Try the model on your prompts before committing. Marketing benchmarks are not your workload.

Best Practices

  1. Write your axes down, A line in your config explaining "we picked X because Y on the cost axis" helps future you re-evaluate when prices change.
  2. Keep a fallback, A second provider for when the first is rate-limited or down is cheap insurance.

Summary

  • Pick a provider along four axes: cost, speed, privacy, capability.
  • The right answer depends on weights, not absolute "best."
  • Most real setups mix one capable provider with one cheap or local fallback.
  • Re-evaluate when your axes change, a personal assistant's needs are not a code-review bot's.

Code Examples

yaml
# Document the why, not just the what, your future self will thank you
provider:
  primary: anthropic
  model: claude-sonnet-4
  rationale: |
    Capability-first choice. This Hermes pairs on production code review;
    subtle bugs are more expensive than per-token cost. Re-evaluate if
    a comparable open-weight model lands at half the price.
✓ Completed