MCP App Architecture

Introduction

Building an LLM application with MCP involves orchestrating a loop between user input, LLM reasoning, tool execution, and response generation. This lesson covers the complete chatbot architecture, conversation state management, context window management with resources, and configuration management for server connections.

Key Concepts

• Agentic Loop: The core cycle of an MCP-powered application: user input goes to the LLM, the LLM decides whether to call tools, tool calls are executed via MCP, results go back to the LLM, and the LLM generates a final response.
• Conversation State: The accumulated history of messages (user, assistant, tool results) that provides context for the LLM's decisions.
• Context Window Management: Strategies for keeping the conversation within the LLM's token limit while preserving important context.
• Server Configuration: Managing which MCP servers to connect to, their launch commands, and environment settings.

Real World Context

A developer builds a chatbot that can query databases, read files, and create GitHub issues. The user asks "Find all users who signed up last week and create a GitHub issue summarizing the trends." The LLM first calls the database query tool to fetch signup data, receives the results, reasons about trends, then calls the GitHub tool to create an issue with a summary. The agentic loop handles this multi-step workflow automatically.

Deep Dive

The core architecture follows a loop pattern. The LLM receives messages, decides on actions, and the application executes those actions until the LLM produces a final text response.

Here is the complete agentic loop implementation:

typescript
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';

interface Message {
  role: 'user' | 'assistant' | 'tool';
  content: string;
  tool_call_id?: string;
  tool_calls?: ToolCall[];
}

interface ToolCall {
  id: string;
  function: { name: string; arguments: string };
}

async function agenticLoop(
  messages: Message[],
  tools: any[],
  toolRouter: (name: string, args: any) => Promise<any>
): Promise<string> {
  while (true) {
    const response = await llm.chat({ messages, tools });

    if (!response.tool_calls || response.tool_calls.length === 0) {
      return response.content;
    }

    messages.push({
      role: 'assistant',
      content: response.content ?? '',
      tool_calls: response.tool_calls
    });

    for (const call of response.tool_calls) {
      const args = JSON.parse(call.function.arguments);
      const result = await toolRouter(call.function.name, args);

      messages.push({
        role: 'tool',
        tool_call_id: call.id,
        content: JSON.stringify(result)
      });
    }
  }
}

The loop continues until the LLM returns a response without tool calls, indicating it has enough information to answer the user.

Conversation state accumulates over time and must be managed to stay within the LLM's context window.

The following code shows a conversation manager with context limits:

typescript
class ConversationManager {
  private messages: Message[] = [];
  private maxMessages = 50;

  addUserMessage(content: string) {
    this.messages.push({ role: 'user', content });
    this.trimIfNeeded();
  }

  addAssistantMessage(content: string, toolCalls?: ToolCall[]) {
    this.messages.push({ role: 'assistant', content, tool_calls: toolCalls });
  }

  addToolResult(callId: string, result: any) {
    this.messages.push({
      role: 'tool',
      tool_call_id: callId,
      content: JSON.stringify(result)
    });
  }

  getMessages(): Message[] {
    return [...this.messages];
  }

  private trimIfNeeded() {
    if (this.messages.length > this.maxMessages) {
      // Keep the system context and recent messages
      const systemMsgs = this.messages.filter(m => m.role === 'user').slice(0, 1);
      const recent = this.messages.slice(-this.maxMessages + systemMsgs.length);
      this.messages = [...systemMsgs, ...recent];
    }
  }
}

The trimming strategy preserves the initial context while keeping the most recent messages.

Server configuration should be externalized so users can customize their server setup.

Here is a configuration-driven approach to server initialization:

typescript
interface AppConfig {
  servers: {
    name: string;
    command: string;
    args: string[];
    env?: Record<string, string>;
  }[];
}

async function initializeFromConfig(config: AppConfig) {
  const clients = new Map<string, Client>();
  const allTools = [];
  const toolRouter = new Map<string, Client>();

  for (const server of config.servers) {
    const client = new Client({ name: 'chatbot', version: '1.0.0' });
    await client.connect(
      new StdioClientTransport({
        command: server.command,
        args: server.args,
        env: server.env
      })
    );
    clients.set(server.name, client);

    const { tools } = await client.listTools();
    for (const tool of tools) {
      toolRouter.set(tool.name, client);
      allTools.push(tool);
    }
  }

  return { clients, allTools, toolRouter };
}

The configuration file defines servers declaratively. The initialization function connects to each server and builds the tool routing map.

Common Pitfalls

1. Infinite loops: If the LLM keeps calling tools without converging on an answer, the agentic loop runs forever. Add a maximum iteration count (e.g., 10 rounds) as a safety limit.
2. Unbounded conversation history: Without trimming, the conversation grows until it exceeds the LLM's context window, causing errors. Implement proactive trimming.
3. Hardcoded server configurations: Embedding server commands in application code makes it difficult for users to customize their setup. Always externalize to a configuration file.

Best Practices

1. Set a maximum iteration limit on the agentic loop (e.g., 10 tool call rounds) to prevent runaway execution.
2. Externalize server configuration to a JSON or YAML file that users can edit without modifying application code.
3. Log each iteration of the agentic loop, including tool calls and results, for debugging multi-step interactions.

Summary

• The agentic loop is the core pattern: user input, LLM reasoning, tool calls via MCP, results back to LLM, repeat until final response.
• Conversation state must be managed with trimming strategies to stay within context window limits.
• Server connections should be configured externally via configuration files.
• Add safety limits (max iterations, message trimming) to prevent runaway loops and context overflow.
• The tool router maps tool names to Client instances for correct dispatching.

async function agenticLoop(
  messages: Message[],
  tools: any[],
  toolRouter: (name: string, args: any) => Promise<any>
): Promise<string> {
  while (true) {
    const response = await llm.chat({ messages, tools });
    if (!response.tool_calls || response.tool_calls.length === 0) {
      return response.content;
    }
    messages.push({ role: 'assistant', content: response.content ?? '', tool_calls: response.tool_calls });
    for (const call of response.tool_calls) {
      const args = JSON.parse(call.function.arguments);
      const result = await toolRouter(call.function.name, args);
      messages.push({ role: 'tool', tool_call_id: call.id, content: JSON.stringify(result) });
    }
  }
}

interface AppConfig {
  servers: {
    name: string;
    command: string;
    args: string[];
    env?: Record<string, string>;
  }[];
}

async function initializeFromConfig(config: AppConfig) {
  const clients = new Map<string, Client>();
  const toolRouter = new Map<string, Client>();
  for (const server of config.servers) {
    const client = new Client({ name: 'chatbot', version: '1.0.0' });
    await client.connect(new StdioClientTransport(server));
    clients.set(server.name, client);
    const { tools } = await client.listTools();
    tools.forEach(t => toolRouter.set(t.name, client));
  }
  return { clients, toolRouter };
}