TTS: From Tokens to Audio

+15 Mana ✨
Hermes Voice Mode(DOCUMENTATION)

Introduction

Text-to-speech is the back door of voice mode: it turns the agent's text reply into spoken audio. It is more than a robotic read-aloud. Hermes streams speech as the reply is still being written, and it cleans the text first. Understanding that changes how you think about voice output.

Key Concepts

  • TTS (text-to-speech): The stage that synthesizes the agent's text reply into audible speech.
  • Streaming TTS: Speaking the reply sentence by sentence as it is generated, rather than waiting for the whole reply.
  • Markdown stripping: Removing formatting (asterisks, backticks, headings) so they are not read aloud literally.
  • Thinking-block stripping: Excluding the agent's internal reasoning blocks from what gets spoken.

Real World Context

Without streaming, a long reply would mean staring at a silent terminal until the entire answer finished generating, then hearing it all at once. With streaming, the agent starts speaking the first sentence while it is still composing the third. The conversation feels alive instead of batched. This single design choice is most of what makes voice replies feel natural.

Deep Dive

When voice output is enabled, the journey from reply to sound works like this.

The LLM generates its reply token by token, the same as in text mode. As soon as a complete sentence is available (Hermes uses a minimum length, around 20 characters, to avoid speaking tiny fragments), that sentence is handed to the TTS engine and spoken. While you hear sentence one, the model is already producing sentence two. This is streaming TTS, and it is why voice replies do not stall.

Before any text is spoken, Hermes cleans it. Two kinds of content are removed:

  • Markdown formatting. A reply written as **important** should be heard as "important," not "asterisk asterisk important asterisk asterisk." Headings, code fences, and bullet markers are stripped or normalized so the audio sounds like speech, not like a screen reader narrating syntax.
  • Thinking blocks. If the agent produces internal reasoning, that is not meant for your ears. Hermes excludes thinking blocks from TTS so you hear the answer, not the scratch work.

The takeaway: TTS is not a dumb read-aloud. It streams for responsiveness and cleans the text so formatting does not leak into audio. When you reason about voice output, reason about this whole behavior, not just "the words get spoken."

Common Pitfalls

  1. Expecting code to sound good spoken: TTS strips markdown, but a code-heavy reply is still awkward as audio. Code is for the eyes; keep voice output for prose-style answers.
  2. Treating missing reasoning as a bug: If the agent reasons internally, those thinking blocks are deliberately not spoken. Hearing only the final reply is intended behavior, not a TTS fault.

Best Practices

  1. Favor prose replies for voice: When you expect a spoken answer, ask questions that yield explanatory prose rather than dense code or tables.
  2. Trust streaming: Do not wait for silence before responding. The agent speaks as it writes, so a long reply still starts quickly.

Summary

  • TTS converts the agent's text reply into spoken audio.
  • Streaming TTS speaks sentence by sentence as the reply is generated, keeping conversation responsive.
  • Hermes strips markdown formatting and thinking blocks before speaking.
  • Spoken output is cleaned and streamed, so it sounds like natural speech rather than a literal narration of text.

Code Examples

yaml
# ~/.hermes/config.yaml
tts:
  provider: "edge"             # which engine synthesizes the audio
  edge:
    voice: "en-US-AriaNeural"  # which voice that engine speaks with
# Streaming and markdown stripping are built in;
# they are behaviors of the TTS stage, not options you wire up.
✓ Completed