Guide

Hermes Agent + Self-Hosted Firecrawl: Private Web Extraction

Q: Can Hermes Agent use a self-hosted Firecrawl server?

Yes. Configure FIRECRAWL_API_URL and FIRECRAWL_API_KEY, then select Firecrawl under Browser Automation in hermes setup tools.

Q: Will Firecrawl handle localhost pages?

Hermes uses hybrid routing. Private and local addresses are handled by a local Chromium sidecar, while public URLs can use Firecrawl.

Keep web extraction under your control by pointing Hermes Agent at your own Firecrawl API, while Hermes retains local-browser routing for localhost and private-network pages.

What this configuration changes

Firecrawl turns a URL into clean, model-friendly content. With self-hosting, Hermes still uses its Firecrawl integration, but requests go to your endpoint instead of https://api.firecrawl.dev. This gives you control over retention, network placement, capacity, and upgrades; it also makes you responsible for authentication, TLS, monitoring, and browser resources.

Configure Hermes

Open ~/.hermes/.env in a text editor and add these file contents:

FIRECRAWL_API_URL=https://firecrawl.example.com
FIRECRAWL_API_KEY=your-private-firecrawl-key
FIRECRAWL_BROWSER_TTL=600

Restrict the environment file, then enable the provider interactively:

chmod 600 ~/.hermes/.env
hermes setup tools
# Choose Browser Automation → Firecrawl

The variable names and provider selection come from the official Hermes browser documentation. Use the public HTTPS address reachable from the Hermes host. In Docker, localhost points to the Hermes container itself, not a Firecrawl container next door.

Test in layers

Call your Firecrawl health endpoint from the Hermes host or container.
Verify TLS and authentication without exposing the key in logs.
Extract one simple public HTML page.
Try a JavaScript-heavy public page and confirm the browser workers have enough memory.
Ask Hermes to summarize the extracted page and cite the original URL.

Understand hybrid routing

Hermes keeps private and loopback targets on a local Chromium sidecar, including localhost, RFC1918 LAN ranges,.local, .lan, and .internal names. Public URLs can use Firecrawl. This prevents a hosted browser from being asked to reach private resources and lets a developer inspect a local dashboard without changing providers.

Security checklist

Require authentication even if the endpoint is behind a private network.
Terminate TLS at a trusted proxy and restrict inbound traffic to expected Hermes hosts.
Apply SSRF controls and block cloud metadata endpoints.
Set concurrency, crawl depth, payload size, and time limits.
Pin Firecrawl versions and test upgrades; do not deploy an unreviewed :latest image.

Firecrawl or Crawl4AI?

Firecrawl is the direct Hermes browser-provider integration and is the simplest choice for extraction. Crawl4AI exposes an MCP server and suits agents that want crawl controls as explicit tools. See OpenClaw + Crawl4AI for that pattern.

Frequently asked questions

Can Hermes Agent use a self-hosted Firecrawl server?

Yes. Set FIRECRAWL_API_URL to your server and FIRECRAWL_API_KEY to the credential your deployment expects, then select Firecrawl in hermes setup tools.

Will Firecrawl handle localhost pages?

Hermes documents hybrid routing: private, loopback, LAN, and internal hostnames stay on a local Chromium sidecar, while public pages use the cloud-style provider.