How the Agent Picks a Tool: Descriptions Matter

+15 Mana ✨

Introduction

The model does not get a manual when it picks a tool. It gets a list of names and one-paragraph descriptions. That description is the entire basis on which the model decides whether web_search, web_extract, or browser_navigate is the right answer to your question. If you want better tool selection, you do not change the model. You write better descriptions.

Key Concepts

  • Tool description: A short natural-language paragraph attached to each tool, shown to the model at every turn.
  • Selection signal: The phrases in the description that make the model prefer one tool over another (verbs like read, extract, navigate, conditional clauses like when the page requires JavaScript).
  • Tool collision: When two tools have overlapping descriptions, the model picks the wrong one or hesitates.

Real World Context

If web_extract says only gets content from a URL and browser_navigate says only opens a URL, the model has to guess. If web_extract says convert a static web page or PDF to markdown; prefer over the browser when the page does not require login or JavaScript, the choice becomes obvious. Hermes built-in tools have descriptions tuned by the team for exactly this reason. When you write custom tools (Course 10), this is the lever you tune.

Deep Dive

At each turn, the model sees something like:

text
Available tools:
  read_file:    Read a text file with line numbers and pagination.
                Prefer over running `cat` in the terminal.
  web_search:   Search the web. Returns up to 5 results with titles,
                URLs, descriptions. Supports operators like site: and
                filetype:.
  web_extract:  Convert a static web page or PDF to markdown. Use
                when you have a specific URL and need the content.
                Prefer over browser for non-interactive pages.
  browser_navigate: Open a URL in a real browser. Use when the page
                requires JavaScript, login, or interaction.
  ...

The phrases like prefer over, use when, and the explicit cross-references ("prefer over running cat in the terminal") are the selection signal. They turn a vague capability list into a decision tree.

Three practical implications:

  1. The same tool can be misused if its description does not warn against it. A tool called query_database with no description of cost will get called even when the answer is in the user's last message.
  2. Adding a new tool can degrade existing ones. If you load three overlapping web tools, the model spends more reasoning on selection. Tighter toolsets produce better behavior.
  3. Names matter, but descriptions matter more. browser_vision is a great name, but the description telling the model use when the page is visually structured and you need to see what the user sees is what makes it picked correctly.

Common Pitfalls

  1. Vague descriptions in custom tools: does X with Y is not enough. Say what input is expected, what is returned, and when to prefer this tool over others.
  2. No negative guidance: Good descriptions include when not to use the tool. Prefer web_extract for static pages saves the agent from defaulting to a browser launch every time.

Best Practices

  1. Write descriptions like a decision rule: Use when X. Prefer over Y when Z. Returns W. This template forces clarity.
  2. Read your own descriptions before debugging selection: When the agent picks the wrong tool, the description almost always told it to.

Summary

  • Tool selection is driven almost entirely by the description shown to the model.
  • The selection signal is in verbs, conditionals, and explicit cross-references.
  • Overlapping descriptions cause tool collision and degrade behavior.
  • Better descriptions are the cheapest, most reliable lever for changing what an agent does.

Code Examples

yaml
# Weak description (model has to guess)
name: get_user
description: Get user info.

# Strong description (model knows when to use it)
name: get_user
description: |
  Fetch a single user by their numeric ID. Returns name, email,
  and signup date. Prefer over `search_users` when you already
  know the exact ID; use `search_users` for name or email lookups.
✓ Completed