How Context Fills Up

+15 Mana ✨
OpenClaw Context Management(DOCUMENTATION)OpenClaw Performance Tuning(DOCUMENTATION)

Introduction

Every language model has a finite context window — the total number of tokens it can process in a single interaction. In OpenClaw, this context is shared between the system prompt, conversation history, tool results, and the model's response. Understanding how context fills up is essential for maintaining responsive, high-quality agent behavior over long conversations.

Key Concepts

  • Context Window: The maximum number of tokens a model can process in one interaction (e.g., 128K for Claude, 128K for GPT-4o)
  • System Prompt Allocation: The portion of the context window consumed by bootstrap files, tool definitions, and skills (typically 80-150K tokens)
  • Conversation History: Previous messages in the current session, consuming more tokens as the conversation progresses
  • Tool Results: Output from tool invocations (file contents, command output, search results) which can be very large
  • Response Budget: The remaining tokens available for the model to generate its response

Real World Context

A developer starts a coding session with a fresh context. The system prompt uses 100K tokens. They ask a question (200 tokens), get a response (1,000 tokens), then ask the agent to read a large file. The file content adds 15,000 tokens. After several more exchanges with tool calls, the context reaches 125K of the 128K limit. The model's responses become truncated and it starts losing track of earlier conversation details.

Deep Dive

Let us trace how context fills up during a typical session:

markdown
Context Budget Breakdown (128K window):

1. System Prompt:      ~100,000 tokens
   - Safety:              1,200
   - Identity + Persona:  3,150
   - User context:          600
   - Instructions:         8,500
   - Tool definitions:   12,000
   - Skills:             35,000
   - Workspace context:   8,000
   - Documentation:      26,550
   - Memory:              5,000

2. Conversation history: ~20,000 tokens (grows over time)
   - User messages
   - Assistant responses
   - Tool invocations and results

3. Response budget:      ~8,000 tokens (what remains)
   - Model's next response

This breakdown shows a common scenario where the system prompt consumes most of the context window. The conversation history grows with each exchange, and the response budget shrinks accordingly.

Tool results are often the largest consumers of conversation context:

markdown
Typical token costs by action:
- Simple user message:        50-200 tokens
- Agent text response:       200-2,000 tokens
- Reading a source file:   1,000-15,000 tokens
- Search results:          2,000-10,000 tokens
- Command output:            500-5,000 tokens
- Code generation:         1,000-8,000 tokens

A single file read can consume more context than ten conversational exchanges. This is why context management matters most in tool-heavy workflows.

The model's behavior degrades as the context fills up. When the response budget drops below a few thousand tokens, several issues emerge:

markdown
Symptoms of context exhaustion:
- Responses become shorter and less detailed
- The model loses track of earlier conversation context
- Tool calls may fail due to insufficient space for results
- The model may hallucinate or contradict earlier statements
- In extreme cases, the response is truncated mid-sentence

Recognizing these symptoms helps you take action before the conversation quality degrades significantly.

Common Pitfalls

  • Reading large files without considering context cost: A 1,000-line source file can consume 10,000+ tokens. Use targeted reads (specific line ranges) instead of reading entire files.
  • Not monitoring context usage during long sessions: By the time you notice degraded responses, the context may already be critically full.
  • Assuming unlimited conversation length: Even with 128K context windows, a system prompt of 100K leaves only 28K for the entire conversation including tool results.

Best Practices

  • Keep the system prompt lean: Every token saved in the system prompt is a token available for conversation. Follow the 80% rule from the token budget lesson.
  • Use targeted tool operations: Read specific file sections, limit search results, and truncate command output when possible.
  • Start new sessions for new topics: When switching to an unrelated task, starting a fresh session gives the agent a full context budget.

Summary

  • The context window is shared between system prompt, conversation history, tool results, and response budget
  • System prompts typically consume 80-150K tokens, leaving limited space for conversation in 128K windows
  • Tool results (file reads, search results, command output) are the largest consumers of conversation context
  • Context exhaustion causes shorter responses, lost context, and potential hallucinations
  • Keep the system prompt lean and use targeted tool operations to maximize available conversation space
✓ Completed