Streaming as Feedback

Introduction

When Hermes generates a response, it does not wait for the full answer before showing you anything. It streams the response token by token, printing each piece as the model produces it. That choice changes how you work: you can read along, anticipate where the answer is heading, and interrupt at the first sign of drift. Streaming is not a cosmetic feature. It is a feedback loop.

Key Concepts

• Streaming output: Text appears progressively as the model generates it.
• Token-by-token rendering: Each token (roughly a word fragment) prints as it arrives.
• Early-exit interrupt: You can press Ctrl+C the moment you see a wrong path.
• Read-along thinking: You can think and adjust while the answer is still arriving.

Real World Context

You ask Hermes to refactor a function. As the response streams, you see it about to switch from the file you meant to a different file with a similar name. You hit Ctrl+C at line 3 of the response, clarify, and resend. Without streaming you would have waited 20 seconds for the wrong answer, then started over. Streaming converted a five-minute mistake into a five-second redirect.

Deep Dive

Streaming serves three jobs at once:

1. Liveness signal. A blank screen is ambiguous: is the agent thinking, or has it hung? Streaming removes that ambiguity. The first token usually arrives within a second of submitting. If nothing appears, something is genuinely wrong (network, provider, key) and you can investigate immediately instead of waiting.

2. Steerability. The cost of interrupting halfway through a 500-token answer is small if you catch the drift early. The cost of letting a wrong answer complete is the full response plus the time to start over. Streaming makes early interrupts the right strategy.

3. Cognitive overlap. You can read the first three lines of the response while the model is still generating the fourth. By the time the answer is complete, you have already half-processed it. Without streaming the wait blocks you, and your reading starts cold from the top of a long block.

text
User: Refactor parseInput to use the new TokenStream API

Agent: I'll start by reading parseInput from src/cli/input.ts...
        [tool call: read_file src/cli/input.ts]
        [result streams in]
        The function currently takes a raw string and... ▌
                                                          ▲
                                            cursor here while more streams

While that cursor is still ticking, you have already decided whether the agent is on track. If you see "the function currently takes a raw string and returns an Array" but you know it returns a Map, you can interrupt right there.

Tool calls also stream. When the agent invokes a tool, you see the tool name and arguments as soon as the model emits them, then the result as soon as the tool completes. This is why an agent doing a read_file call shows the path being read before the file content appears: you can spot "reading the wrong file" before it happens.

Common Pitfalls

1. Walking away during streaming: The whole point of streaming is real-time feedback. If you leave the keyboard, you lose the steering window and may as well not have streamed at all.
2. Treating streaming as cosmetic: Some developers ignore the streaming output and only read the final block. They miss the cheapest moment to course-correct.

Best Practices

1. Read the first two to three lines of every response: This is the highest-value moment for steering. It takes seconds and prevents large rework.
2. Watch tool call arguments as they stream: A wrong file path or a too-broad command is much cheaper to interrupt before the tool runs than after.

Summary

• Hermes streams responses token by token instead of batching them.
• Streaming gives you a liveness signal, a steering window, and cognitive overlap.
• Reading the first lines of a response is the cheapest time to correct course.
• Tool calls also stream, letting you catch wrong arguments before the tool fires.

# Streaming as it unfolds in your terminal

[t=0.0s] User submits prompt
[t=0.4s] Agent: I'll start by reading the
[t=0.7s] Agent: I'll start by reading the configuration file
[t=1.1s] Agent: I'll start by reading the configuration file at
[t=1.4s] Agent: I'll start by reading the configuration file at ~/.hermes/config.yaml
         ^
         You spot the wrong path here. Press Ctrl+C, correct it, resend.