Hermes Agent Browser Harness: Real Browser Control for an AI Agent

What Is the Hermes Agent Harness?

People use "harness" for the code that sits between a language model and the real world. The model decides what to do next; the harness actually does it, checks whether it is allowed, and feeds the result back. In Hermes Agent that is everything around the model that turns a chat completion into an agent: the loop that calls tools and feeds results back, the terminal and file tools with their execution backend, persistent memory in ~/.hermes/memories/, skills in ~/.hermes/skills/, the dangerous-command approval layer, and the gateway that connects the agent to Telegram, Discord, Slack and other platforms. Hermes ships all of these as one program, so Hermes itself is the harness.

The browser harness covered in the rest of this guide is one tool inside it. Every piece is configured in ~/.hermes/config.yaml:

The gateway: what makes it always on

Running hermes in a terminal gives you an interactive session that ends when you close the window. The gateway is the long-lived process that connects the same agent to Telegram, Discord, Slack, WhatsApp and other platforms, runs scheduled cron jobs, and routes approval prompts to your chat. On your own machine:

hermes gateway setup     # pick platforms, paste bot tokens
hermes gateway install   # install the background service
hermes gateway start
hermes gateway status
hermes gateway restart   # after config changes that need it

Keep the gateway on a machine that stays awake: a laptop that sleeps takes the bot, its cron jobs and any pending approvals offline with it. Inside a container, use hermes gateway run instead of installing a service. On a managed bot, the dashboard handles all of this.

Terminal backends: where commands actually run

The terminal tool is the most powerful thing in the harness, so where it runs matters most. Swap the execution environment without changing anything else:

hermes config set terminal.backend docker
hermes config set terminal.backend ssh
hermes config set terminal.backend local   # the default

A minimal Docker setup looks like this:

terminal:
  backend: docker
  cwd: /workspace
  docker_image: nousresearch/hermes-sandbox:desktop
  container_memory: 5120      # MB
  container_persistent: true  # keep /workspace across restarts

Two Docker details are easy to miss. Your launch directory is not mounted into the container unless you set docker_mount_cwd_to_workspace: true, and docker_forward_env starts as an empty list, so you choose which host variables go in. Anything forwarded is readable by every command the agent runs. For SSH, the host and user go in ~/.hermes/.env as TERMINAL_SSH_HOST and TERMINAL_SSH_USER, plus an optional TERMINAL_SSH_KEY path to a private key. More on containers in Hermes Agent + Docker.

One exception: on a managed OpenClaw Launch bot, leave terminal.backend on local. The bot already runs inside its own isolated container, which has no Docker daemon, so switching it to docker breaks the terminal and file tools.

Approvals: the human in the loop

Hermes checks shell commands against a list of dangerous patterns before running them. What happens next depends on approvals.mode:

approvals:
  mode: smart        # smart | manual | off
  timeout: 300       # seconds to wait for your answer
  cron_mode: deny          # cron jobs: deny | approve
  single_query_mode: deny  # one-shot runs: deny | approve
  unattended_mode: deny    # webhook / API sessions: deny | approve

• smart (default): an auxiliary model auto-approves clearly low-risk commands, auto-denies clearly dangerous ones, and asks you about the uncertain middle.
• manual: asks you about every flagged command you have not already allowed for the session or permanently.
• off: never asks, the same as starting with --yolo or typing /yolo.

Through the gateway, the prompt arrives in your chat and you reply /approve or /deny. Sessions with nobody to answer (cron jobs, one-shot runs, webhook and API sessions) deny flagged commands by default unless approvals are off, so the agent has to find another way instead of hanging. Isolated sandbox backends skip the prompts, but your own approvals.deny glob patterns still apply on every backend.

Skills and memory: what makes it improve

• Memory — MEMORY.md holds the agent's notes about your environment and conventions; USER.md holds your preferences. Both are small, capped files loaded at the start of each session. See Hermes Agent memory.
• Skills — folders with a SKILL.md that describe how to do a specific job. Hermes loads them only when relevant, installs them from the hub, and can write new ones itself. They live in ~/.hermes/skills/. See Hermes Agent skills.

Do not point two Hermes processes at the same home directory, since both would write memory and skills into the same files. Use a separate profile for each agent.

A practical setup checklist

1. Install Hermes and pick a model with hermes model. The setup guide walks through it.
2. Choose a backend before you give the agent real work: docker on a personal machine, ssh for a separate work box, local on a managed bot.
3. Leave approvals.mode on smart or set manual. Add approvals.deny rules for anything that must never run.
4. Forward only the environment variables the agent actually needs.
5. Self-hosted: run hermes gateway setup, connect one platform, restrict who can talk to the bot, then install and start the gateway service on a machine that stays on. On OpenClaw Launch, connect platforms from the dashboard instead.
6. Add MCP servers and skills once the basics work, not before. Each one adds tools the model has to reason about.

When a managed harness makes sense

Self-hosting gives you full control, but you also have to keep it running: the gateway process, container limits, updates, backups, and a machine that never sleeps. On OpenClaw Launch, each Hermes instance runs in its own isolated container, the gateway process is kept running for you, and you connect platforms, change models and open a web terminal or file browser from the dashboard. A managed bot still uses the same Hermes harness described above. One difference: managed Hermes bots start with Full Execution on (approvals.mode: off), so commands run without prompts inside the bot's own container. Turning Full Execution off switches the bot to manual approvals. If you need the SSH backend pointed at your own hardware or a custom sandbox image, self-hosting is the better fit.

What Is the Hermes Browser Harness?

The browser harness is a built-in tool surface in Hermes Agent that gives the model first-class control over a real browser, not just an HTTP fetch. Under the hood, it drives a headless Chromium instance via the Chrome DevTools Protocol, so the agent can do everything a human user does in a browser tab: navigate, scroll, click, type, wait for elements to render, and take screenshots.

That distinction matters. Plain "web search" tools (Tavily, Exa, Brave) hand the model a static snippet of a page. The browser harness lets the agent log in to a site, work through a multi-step flow, fill a form, and read the rendered DOM after the JavaScript runs. It's the difference between reading about a tool and using it.

What the Browser Harness Can Do

• Open and navigate — load any URL, follow redirects, go back
• Click elements — resolve selectors and click buttons / links
• Fill forms — type into inputs, select dropdowns, check boxes
• Wait for state — wait for selectors, network idle, or fixed delays
• Screenshot — full-page or element screenshots, returned as images the model can reason over
• Read the rendered DOM — HTML, accessibility tree, or JavaScript-evaluated values, after hydration
• Persist sessions — cookies and storage stick across tool calls so login state carries through a task

Browser Harness vs Web Search

How to Enable Browser Harness on Managed Hermes

On Hermes hosting via OpenClaw Launch, the browser harness is included — the per-instance Chromium sidecar is preconfigured and already wired into the Hermes container.

1. Open openclawlaunch.com and pick the Hermes Agent template.
2. In the configurator, under Tools, make sure the browser tool is enabled (it's on by default for Hermes deployments).
3. Pick a model that supports vision — GPT-5.4, Claude Sonnet 4.6, or Gemini 3.1 Pro — so screenshots are interpretable.
4. Click Deploy. Live in about 30 seconds with the harness ready.

Self-Hosted Browser Harness

If you run Hermes on your own server, you provide the Chromium endpoint via the standard Hermes browser configuration. The harness expects a Chrome DevTools Protocol endpoint — either a local Chromium or a remote browser-as-a-service URL.

# Run Chromium with a remote debugging port
chromium --headless --remote-debugging-port=9222 \
  --disable-gpu --no-sandbox

# Point Hermes at it
export HERMES_BROWSER_CDP_URL="http://localhost:9222"
hermes-agent

For production, run the browser in its own container with restricted egress. Hermes connects over CDP and never executes arbitrary JS unless the model explicitly requested it as part of a tool call.

What to Build With It

• Form-filling agents — book appointments, submit applications, renew subscriptions
• SPA scrapers — pull data from JavaScript-rendered dashboards that ordinary scrapers miss
• Visual QA — screenshot a page, ask the model whether a deploy looks right
• Multi-step research — navigate to a vendor doc, click through examples, copy the right snippet back into chat
• Account triage — log in to an admin panel, find the broken row, escalate

Security Considerations

The browser harness is powerful, which means it deserves the same caution as any tool that touches the internet on your behalf. A few defaults worth knowing:

• Each Hermes instance gets its own isolated Chromium — cookies and storage don't leak between users.
• The browser runs in its own sandboxed container. Downloads land in one shared folder, /opt/data/browser-downloads, which the agent reads at that same path. The browser sees nothing else of the agent's files.
• You can scope navigation to an allowlist via Hermes config if the agent should only touch specific domains.
• Don't paste production credentials into chat to give the agent login. Use a dedicated read-only test account or a session cookie passed via your secrets store.

Frequently Asked Questions

What is the Hermes Agent harness, and is Hermes a harness?

Yes. The harness is everything around the model that makes it an agent: the tool-calling loop, terminal and file tools with an execution backend, persistent memory, skills, the dangerous-command approval layer, and the messaging gateway. Hermes ships all of these together, so Hermes itself is the harness. See What Is the Hermes Agent Harness? above.

Is the Hermes harness the same as the browser harness?

No. The browser harness is one tool inside the Hermes harness: how Hermes drives a real browser for clicking, filling forms and reading pages. The Hermes harness is the whole runtime that tool plugs into.

Which terminal backend should I use for Hermes?

Upstream supports local (the default), docker, ssh, singularity, modal, daytona and vercel_sandbox, set with hermes config set terminal.backend <name>. On your own laptop, docker is the safer everyday choice because commands run in a container instead of your home directory, and ssh keeps the agent unable to modify its own code. On a hosted bot such as OpenClaw Launch, keep local: the bot already runs in its own container, which has no Docker daemon.

How do approvals work in Hermes Agent?

Dangerous shell commands go through approvals.mode in ~/.hermes/config.yaml. The default smart auto-approves clearly low-risk commands, auto-denies clearly dangerous ones and asks you about the rest; manual asks about every flagged command you have not already allowed; off is the same as --yolo. On a chat platform you answer with /approve or /deny. Unless approvals are off, cron, one-shot and webhook sessions deny flagged commands by default.

Do I need a managed harness like OpenClaw Launch?

Not if you are happy running a server, since Hermes works fully self-hosted. A managed host helps when you want the gateway running without keeping your own machine awake, each Hermes instance in its own container, and platform connections, model changes and backups handled from a dashboard. OpenClaw Launch does that for Hermes and OpenClaw.

Does Hermes browser harness work without GUI?

Yes. The default mode is headless — no display server needed. The model interacts with the page through DOM queries and screenshots, not a visible window.

Is it the same as Playwright?

Conceptually similar (both speak the Chrome DevTools Protocol), but the harness is a tool surface tuned for LLM use: simpler verbs, image returns, automatic wait heuristics, and per-task session persistence.

Can I run multiple browser sessions in parallel?

Yes — the harness can open multiple pages within one Chromium instance, and each Hermes deployment gets its own Chromium so concurrency is per-instance.

What's Next?

• Install Hermes Agent — managed and self-hosted setup
• Hermes Agent skills — what skills are bundled and how to add more
• Hermes Agent web UI — the gateway interface and how to chat with your agent
• OpenClaw as an agent harness — the same idea for OpenClaw
• Hermes Agent + Docker — running Hermes in containers
• Hermes vs OpenClaw — which framework fits which use case