Session Memory as In-Context History

Introduction

Session memory is just a fancy name for "the conversation so far." Every user turn, every assistant response, every tool call and tool result are appended to a buffer that the model re-reads on the next turn. There is no clever retrieval, no embedding, no database. It is text in the prompt. Once you internalize that, you understand why context windows fill up, why compression exists, and why short, focused sessions outperform sprawling ones.

Key Concepts

• Conversation buffer: The ordered list of messages and tool exchanges in the current session.
• In-context recall: The model "remembers" anything in the buffer because it literally reads it again on every turn.
• Token cost: Every turn pays for the entire buffer, not just the new message.
• Volatility: Closing or restarting the session discards the buffer unless the session is checkpointed or persisted to SQLite.

Real World Context

An engineer paste in a 600-line log file at turn 5 of a debugging session. By turn 30, every turn still re-sends those 600 lines, even though the relevant fix was identified ten turns ago. The session feels slow and expensive. They have not done anything wrong, but they have not yet developed the instinct to read the status bar and compress when context gets crowded.

Deep Dive

Hermes stores session messages in two places, and the distinction matters:

1. The live conversation buffer: the in-memory ordered list of messages that ships with every model request.
2. SQLite state (~/.hermes/state.db): every CLI and messaging session is written to a local SQLite database with FTS5 full-text search. This is durable but not in-prompt.

The live buffer is what the model sees on this turn. The SQLite store is what lets hermes search find a conversation from three weeks ago and pull a summary back into the next session. They are different mechanisms, even though both feel like "history."

A practical implication: if you want the model to use a fact from a previous session, you cannot rely on it remembering. You must surface the fact through either persistent memory or a context file, or by pasting the fact directly into the current message. The model is not opaque about this. The conversation buffer literally is its memory.

Finally, session memory is the unit compression operates on. When the status bar shows context filling toward 100 percent, the path forward is either compress (summarize older turns, free tokens) or restart (fresh session, lose the buffer). Persistent memory and context files are unaffected either way.

Common Pitfalls

1. Treating the session like a database: Asking "what did I tell you about my config three hours ago?" works only if those turns still fit in context. Past the compression line, the answer is lost.
2. Forgetting that tool output counts: A cat-ing a huge file or a verbose grep adds tokens to every subsequent turn until compression.

Best Practices

1. Read the status bar before pasting: Big pastes are fine when the buffer is empty, expensive when it is half full.
2. Summarize and continue, do not just continue: If you have crossed turn 50, a /compress or a fresh session with a brief recap message usually outperforms grinding on.

Summary

• Session memory is the conversation buffer the model re-reads every turn.
• Recall is purely in-context: if it is in the buffer, the model sees it; if not, it does not.
• Sessions also write to a SQLite store for later search, separate from the live buffer.
• Long, crowded sessions slow down and cost more because every turn pays for the whole buffer.

# The status bar tracks the live buffer in real time
#   anthropic/claude-sonnet-4-20250514 | 14,832 tokens | 18% | $0.23 | 12m
#                                          ↑ session memory size

# Past sessions land in SQLite for cross-session search
hermes search "how did we fix the migration bug?"
# Returns matching turns from previous sessions, summarized.
# These are NOT in the live buffer until you re-inject them.