The Summarization Pattern (Hindsight): Compress and Recall

Introduction

Embedding-based retrieval shines when you ask "find me things about this topic". But sometimes the question is "what happened over the last six weeks?" and a list of similar fragments is not the right answer. You want a summary. That is the niche the summarization pattern fills, and Hindsight is the named provider in the Hermes ecosystem.

Key Concepts

• Summarization-based memory: Replace original messages with model-generated summaries, then store the summary.
• Lossy by design: You lose individual phrasing, exact code snippets, and minor details. You keep the arc.
• Hierarchical compression: Summaries themselves can be summarized further as time passes, producing a pyramid of detail levels.
• Time-ordered recall: Summaries are typically retrieved by recency or time window, not similarity.

Real World Context

A technical lead resumes a project after two months away. They do not want every commit message; they want "what shipped, what is broken, what is next." Summarization-based memory is the right tool, because the question is about narrative, not specifics. The model-written summary at the appropriate granularity is more useful than two months of unfiltered turns.

Deep Dive

A simplified picture of how Hindsight handles a long session:

text
Turn 1..10  ─┐
Turn 11..20 ─┼─► summary A (covers turns 1..30)
Turn 21..30 ─┘
Turn 31..60 ────► summary B (covers turns 31..60)
Turn 61..90 ────► summary C (covers turns 61..90)
                  ↓
         (later)  super-summary covers A + B + C

Notice the cascade: as the session ages, finer summaries are themselves rolled into coarser ones. You can think of it as the model's own version of "zooming out". When asked for context, the provider returns the appropriate granularity. Recent activity comes back in detail; older activity in coarser strokes.

The trade-offs are explicit. You lose verbatim recall: "what was the exact error message in turn 47?" is no longer answerable from the summary. You also lose the ability to retrieve exact code snippets unless they were preserved as artifacts elsewhere. In exchange, you gain very compact representations of long histories, which fit in a small token budget.

If your use case looks more like "long-running agent that needs cheap access to its own past" rather than "retrieve specific past facts", summarization is the right pattern. Hindsight is one named implementation; the archetype matters more than the brand.

Common Pitfalls

1. Relying on summary recall for exact details: Summaries are not transcripts. If you need the exact wording of a decision, store it as a memory entry or commit it to a file.
2. Letting summaries compound for too long without curation: Hierarchical summaries lose nuance at each level. Periodically refresh the summary baseline if the work matters.

Best Practices

1. Reach for summarization when the question is narrative: "What happened", "how did we get here", "what did we decide". For "what is fact X", use embedding retrieval or file memory.
2. Combine providers: Hindsight for the long arc, the built-in file provider for stable preferences. They do not conflict.

Summary

• Summarization-based memory replaces messages with model-generated summaries at multiple granularities.
• Hindsight is the canonical example in Hermes.
• It is lossy on purpose, optimized for narrative recall over exact recall.
• Best paired with file-based memory for stable facts the summary should not lose.

# ~/.hermes/config.yaml
memory:
  provider: hindsight
  hindsight:
    summarize_after_turns: 30   # build a summary block every 30 turns
    coarsen_after_days: 14      # roll summaries up after two weeks