Advanced Tool Patterns

Introduction

Basic tool calling works well for simple scenarios, but production applications need more: parallel execution for speed, retries for reliability, timeouts for safety, and caching for efficiency. These patterns transform a prototype MCP client into a robust system that handles real-world conditions where servers are slow, flaky, or overwhelmed.

Key Concepts

• Parallel Tool Execution: When the LLM requests multiple tool calls in a single response, execute them concurrently with Promise.all instead of sequentially.
• Timeouts: Wrap tool calls in a timeout to prevent your application from blocking indefinitely when a server hangs or a tool takes too long.
• Retries with Backoff: When a tool call fails due to transient errors (network issues, rate limits), retry with exponential backoff before giving up.
• Result Caching: Cache tool results for idempotent operations to avoid redundant calls and reduce latency.

Real World Context

Your AI assistant manages infrastructure across multiple cloud providers via MCP servers. When a user asks "Check the status of all production services," the LLM requests five simultaneous tool calls — one for each service. Executing them sequentially takes 10 seconds. Executing them in parallel takes 2 seconds. When one call fails because of a temporary network issue, your retry logic automatically retries it once and succeeds. When the user asks the same question a minute later, cached results are returned instantly for services whose status has not changed.

Deep Dive

Parallel Tool Execution

When the LLM requests multiple tool calls in a single response, execute them concurrently:

typescript
async function executeToolCalls(
  client: Client,
  toolUseBlocks: Array<{ id: string; name: string; input: unknown }>
) {
  const results = await Promise.all(
    toolUseBlocks.map(async (block) => {
      const result = await client.callTool({
        name: block.name,
        arguments: block.input as Record<string, unknown>
      });

      return {
        type: "tool_result" as const,
        tool_use_id: block.id,
        content: result.isError
          ? `Error: ${(result.content[0] as { text: string }).text}`
          : (result.content[0] as { text: string }).text
      };
    })
  );

  return results;
}

Promise.all executes all tool calls concurrently. The results array preserves the order, so each tool_use_id is correctly matched. This is significantly faster than sequential execution, especially for I/O-bound operations like API calls or database queries.

One consideration: if one tool call fails, Promise.all rejects with the first error. Use Promise.allSettled if you want to collect results from all calls, even if some fail:

typescript
const settled = await Promise.allSettled(
  toolUseBlocks.map(block =>
    client.callTool({
      name: block.name,
      arguments: block.input as Record<string, unknown>
    })
  )
);

const results = settled.map((outcome, i) => ({
  type: "tool_result" as const,
  tool_use_id: toolUseBlocks[i].id,
  content: outcome.status === "fulfilled"
    ? (outcome.value.content[0] as { text: string }).text
    : `Error: ${outcome.reason?.message || "Unknown error"}`
}));

This ensures that even if one tool call fails, the other results are still returned to the LLM.

Tool Call Timeouts

Protect your application from hanging tool calls with a timeout wrapper:

typescript
function withTimeout<T>(
  promise: Promise<T>,
  timeoutMs: number
): Promise<T> {
  return Promise.race([
    promise,
    new Promise<never>((_, reject) =>
      setTimeout(
        () => reject(new Error(`Tool call timed out after ${timeoutMs}ms`)),
        timeoutMs
      )
    )
  ]);
}

// Usage
const result = await withTimeout(
  client.callTool({ name: "analyze_codebase", arguments: { path: "/" } }),
  30000 // 30 second timeout
);

The withTimeout function races the tool call against a timer. If the tool call does not resolve within the timeout, the promise rejects with a timeout error. Choose timeouts based on the expected duration of each tool — a file read should complete in seconds, while a code analysis might take 30 seconds or more.

Retries with Exponential Backoff

For transient failures, retry with increasing delays:

typescript
async function callToolWithRetry(
  client: Client,
  params: { name: string; arguments: Record<string, unknown> },
  maxRetries: number = 3
) {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      const result = await withTimeout(
        client.callTool(params),
        30000
      );
      return result;
    } catch (error) {
      if (attempt === maxRetries) throw error;
      const delay = Math.pow(2, attempt) * 1000; // 1s, 2s, 4s
      console.error(
        `Tool call failed (attempt ${attempt + 1}), retrying in ${delay}ms...`
      );
      await new Promise(resolve => setTimeout(resolve, delay));
    }
  }
  throw new Error("Unreachable");
}

Exponential backoff starts with a 1-second delay and doubles each time. This prevents overwhelming a struggling server with rapid retries.

Important: only retry on transient errors (network failures, timeouts, rate limits). Do not retry on permanent errors (invalid arguments, tool not found) — those will fail every time.

Result Caching

For idempotent tools (tools that return the same result for the same input), cache results to avoid redundant calls:

typescript
const toolCache = new Map<string, {
  result: unknown;
  timestamp: number;
}>(); 

async function callToolCached(
  client: Client,
  params: { name: string; arguments: Record<string, unknown> },
  ttlMs: number = 60000 // 1 minute default
) {
  const cacheKey = JSON.stringify(params);
  const cached = toolCache.get(cacheKey);

  if (cached && Date.now() - cached.timestamp < ttlMs) {
    return cached.result;
  }

  const result = await client.callTool(params);
  toolCache.set(cacheKey, { result, timestamp: Date.now() });
  return result;
}

The cache key is the serialized tool call parameters. Results are cached for a configurable TTL (time-to-live). This is effective for tools like "get file contents" or "list directory" where the result does not change frequently.

Be careful with caching: never cache tools with side effects (create, update, delete operations). Only cache read-only, idempotent tools.

Common Pitfalls

1. Caching non-idempotent tools — Never cache results from tools that modify state (like create_file or run_query with INSERT/UPDATE). Cached results would give stale data and hide the fact that the action was not re-executed.
2. Retrying non-transient errors — Retrying a tool call that fails because of invalid arguments will fail every time. Only retry on network errors, timeouts, or rate limit responses.
3. No timeout on tool calls — A tool call that hangs forever will block your entire application. Always set a reasonable timeout, even for tools you expect to be fast.

Best Practices

1. Use Promise.allSettled for parallel calls — This ensures one failing call does not prevent you from getting results from the others. Report individual failures back to the LLM.
2. Choose timeouts per tool category — Fast tools (file reads) get short timeouts (5s). Slow tools (code analysis) get long timeouts (60s). Do not use one timeout for everything.
3. Implement a circuit breaker — If a tool fails repeatedly, stop calling it for a cooldown period instead of retrying indefinitely. This prevents cascading failures.

Summary

• Execute multiple tool calls in parallel with Promise.all or Promise.allSettled for speed and resilience.
• Wrap tool calls in a timeout using Promise.race to prevent hanging.
• Retry transient failures with exponential backoff (1s, 2s, 4s delays), but never retry permanent errors.
• Cache results for idempotent, read-only tools using a TTL-based cache.
• Combine these patterns: parallel execution + per-call timeouts + retries + caching for a production-grade tool execution layer.

// Parallel execution with Promise.allSettled
const settled = await Promise.allSettled(
  toolUseBlocks.map(block =>
    withTimeout(
      callToolWithRetry(client, {
        name: block.name,
        arguments: block.input
      }),
      30000
    )
  )
);

const results = settled.map((outcome, i) => ({
  type: "tool_result" as const,
  tool_use_id: toolUseBlocks[i].id,
  content: outcome.status === "fulfilled"
    ? outcome.value.content[0].text
    : `Error: ${outcome.reason?.message}`
}));