Browser Automation Deep Dive

+15 Mana ✨
OpenClaw Browser Automation Documentation(DOCUMENTATION)

Introduction

OpenClaw's browser tools go far beyond simple page fetching. They provide a full automation suite with multi-profile support, structured page snapshots, visual screenshots, and an act command that interacts with page elements by reference. This lesson walks through the recommended browser automation flow and every available action.

Key Concepts

  • Browser Actions: The set of operations available — status, start, stop, tabs, open, snapshot, screenshot, act, navigate, pdf, and upload.
  • Browser Profiles: Isolated browser instances running on ports 18800 through 18899, each with their own cookies and session state.
  • Snapshot: A structured, machine-readable representation of the current page DOM, annotated with element references.
  • Screenshot: A visual image capture of the current browser viewport.
  • Act Command: An instruction to interact with a page element identified by its reference from a snapshot.
  • Refs: Unique identifiers assigned to interactive elements in a snapshot, used to target actions.
  • Recommended Flow: The sequence status/start → snapshot → act → screenshot that ensures reliable automation.

Real World Context

An agent tasked with filling out a web form first starts a browser profile, takes a snapshot to identify form fields by their refs, uses act to type into each field and click submit, then takes a screenshot to verify the submission was successful. This structured approach avoids the fragility of CSS selectors or XPath expressions.

Deep Dive

Browser Actions Overview

The browser tool supports eleven distinct actions:

json
{
  "tool": "browser",
  "action": "status"
}

The status action checks whether a browser instance is currently running and on which port. This is always the first step before attempting any browser interaction.

json
{
  "tool": "browser",
  "action": "start",
  "profile": "default"
}

The start action launches a browser instance with the specified profile. If a browser is already running, it returns the existing session.

The Recommended Flow

The most reliable pattern for browser automation follows four steps:

Step 1 — Check status and start the browser:

json
{
  "tool": "browser",
  "action": "start"
}

This ensures a browser is running before any other operations. Always check status or start first.

Step 2 — Navigate and take a snapshot:

json
{
  "tool": "browser",
  "action": "navigate",
  "url": "https://example.com/login"
}

After navigating, take a snapshot to understand the page structure:

json
{
  "tool": "browser",
  "action": "snapshot"
}

The snapshot returns a structured representation of the page with element refs. For example, a login form might return refs like [ref=e14] for the username field and [ref=e15] for the password field.

Step 3 — Act on elements using refs:

json
{
  "tool": "browser",
  "action": "act",
  "ref": "e14",
  "instruction": "type 'admin@example.com'"
}

The act command targets a specific element by its ref from the snapshot. The instruction describes what to do — type text, click, select an option, and so on. This is far more reliable than CSS selectors because refs are tied to the actual DOM state at snapshot time.

Step 4 — Screenshot to verify:

json
{
  "tool": "browser",
  "action": "screenshot"
}

After acting, a screenshot provides visual confirmation that the action succeeded. This is essential for debugging and for agents that need to reason about visual state.

Multi-Profile Support

Browser profiles enable isolated sessions:

json
{
  "tool": "browser",
  "action": "start",
  "profile": "session-a",
  "port": 18800
}

Each profile runs on its own port in the range 18800–18899. This allows up to 100 concurrent browser sessions, each with independent cookies, local storage, and authentication state. Use separate profiles when you need to test multi-user scenarios or maintain distinct sessions.

Additional Actions

Beyond the core flow, several other actions are available:

json
{
  "tool": "browser",
  "action": "tabs"
}

This lists all open tabs in the current browser instance. Use it to switch context between pages.

json
{
  "tool": "browser",
  "action": "pdf",
  "path": "/output/report.pdf"
}

The pdf action renders the current page as a PDF document, useful for generating reports or archiving page state.

json
{
  "tool": "browser",
  "action": "upload",
  "ref": "e22",
  "files": ["/data/document.pdf"]
}

The upload action targets a file input element by ref and uploads the specified files.

Common Pitfalls

  • Acting without a fresh snapshot: Refs become stale when the page changes. Always take a new snapshot after navigation or after actions that modify the DOM.
  • Skipping the status check: Attempting to act on a browser that is not running produces confusing errors. Always start with status or start.
  • Using hardcoded refs across runs: Refs are generated dynamically and differ between snapshots. Never store or hardcode ref values.

Best Practices

  • Follow the snapshot-act-screenshot cycle: This three-step inner loop ensures every interaction is targeted and verified.
  • Use separate profiles for separate concerns: Do not mix authentication states in a single profile. Create dedicated profiles for each user or context.
  • Take screenshots at decision points: Capture visual state before and after critical actions to create an audit trail.

Summary

  • The browser tool provides eleven actions: status, start, stop, tabs, open, snapshot, screenshot, act, navigate, pdf, and upload.
  • The recommended flow is: status/start, then snapshot, then act, then screenshot.
  • Snapshots provide element refs that the act command uses to target interactions reliably.
  • Multi-profile support on ports 18800–18899 enables isolated, concurrent browser sessions.
  • Always take a fresh snapshot before acting to avoid stale refs.
✓ Completed