Media Tools: vision_analyze, image_generate, text_to_speech

+15 Mana ✨
Hermes Media Tools Reference(DOCUMENTATION)

Introduction

Hermes is not a text-only agent. Three media tools let it see images, make images, and speak. Together they make agents that handle screenshots, product mockups, and voice messages first-class workflows rather than afterthoughts.

Key Concepts

  • vision_analyze: Takes an image plus a question and returns an answer. Backed by a vision-capable model.
  • image_generate: Takes a text prompt and returns an image. Default backend is FAL.ai with FLUX 2 Klein 9B, with opt-in OpenAI and xAI backends.
  • text_to_speech: Takes text and returns audio. Used most often by gateways (Telegram, Discord, WhatsApp) to send voice messages.

Real World Context

A designer sends a screenshot to Hermes via Telegram and asks does this layout match our brand guidelines?. The agent runs vision_analyze({ image: ..., question: 'does this layout match our brand guidelines?' }) and answers. Or a marketing teammate asks for a vintage-poster-style banner for our launch, and image_generate({ prompt: 'vintage poster, ...' }) returns the artwork. Or a user requests a voice summary of a meeting, and text_to_speech converts the summary to audio.

Deep Dive

The three tools are similar in shape (one input, one output) but very different in cost and risk.

  • vision_analyze is a read. It does not modify anything; it just answers a question about an image. Risk class: read. Cost: one vision-model call. Use freely.
  • image_generate is a create. It produces a new artifact (an image), which may be saved or sent. Risk class is technically write (a file may land on disk), but with no power to modify existing content. Cost: depends on backend and image size; not free.
  • text_to_speech is a create. It produces an audio file. Same risk profile as image_generate: it creates a new artifact but does not mutate anything existing. Cost: per character of text.

Note on backends:

  • vision_analyze uses Hermes's configured vision model (typically a frontier model with vision support).
  • image_generate defaults to FAL.ai with FLUX 2 Klein 9B. OpenAI and xAI image backends are available opt-in.
  • text_to_speech is platform-aware: when sending to Telegram, Discord, or WhatsApp, it delivers in the platform's native audio format.

A fourth related tool, video_analyze, handles videos: scenes, captions, timestamps. It is less commonly used but lives in the same category.

Common Pitfalls

  1. Using browser_vision when vision_analyze is enough: If you already have the image file, you do not need a browser. vision_analyze operates directly.
  2. Treating image_generate as free: It produces images at real cost and real latency. Avoid spinning up images speculatively.

Best Practices

  1. Be specific in the question to vision_analyze: A clear question (is the submit button visible?) produces a clear answer. A vague question (describe this) produces a vague answer.
  2. Match the image backend to the use case: FAL.ai FLUX 2 Klein 9B is the default for a reason: fast, capable, broad licenses. Switch only when you need a feature it does not offer.

Summary

  • vision_analyze reads images. Use it to answer questions about an existing image.
  • image_generate creates images from text prompts. Default backend is FAL.ai FLUX 2 Klein 9B.
  • text_to_speech creates audio. It is gateway-aware and adapts to platform delivery.
  • These three (plus video_analyze) make Hermes multimodal at the tool layer, not just the model layer.

Code Examples

yaml
# Three media tool calls in their typical form
vision_analyze:
  image: /tmp/screenshot.png
  question: Is the submit button visible and styled correctly?

image_generate:
  prompt: vintage-poster-style banner for a tech launch, blue palette
  size: 1024x1024

text_to_speech:
  text: Your dev server is running on port 4200.
  voice: default
✓ Completed