Hermes Agent + LM Studio: Run Hermes on Fully Local Models

Running your bot on OpenClaw Launch? The steps below use localhost, which is correct when the agent and the model server run on the same machine. A hosted bot runs in the cloud, so it cannot reach localhost or a home LAN address — those are refused when you save them. Give the model server a publicly reachable address instead (your own domain, a port forward, or a tunnel), save it under API keys ending in /v1. The endpoint then appears in the model picker as a custom entry — pick the model your server reports, or type its exact id if the list comes back empty, and set it as the default.

What Is LM Studio?

LM Studio is a desktop app that downloads, runs, and serves open-weight LLMs on macOS, Windows, and Linux. It bundles a model browser, a chat UI, and most importantly an OpenAI-compatible HTTP server that any tool — including Hermes Agent — can call as a drop-in replacement for the OpenAI API.

LM Studio handles the hard parts of local inference: GPU offload, quantization, flash attention, and KV cache management. Most modern Macs with 16+ GB unified memory can run 8–14B models comfortably; a 70B model needs 64+ GB or a beefy Linux box with two consumer GPUs.

Why Pair Hermes with LM Studio?

• Full privacy. Every message stays on your machine. No API key, no telemetry.
• Zero per-token cost. Once the model is downloaded, inference is free forever.
• Offline operation. Run Hermes on a laptop with no internet.
• Easy model swapping. Switch from Llama to Qwen to DeepSeek in a click; Hermes's /model command picks them up.

Step 1: Install LM Studio and Download a Model

1. Download LM Studio from lmstudio.ai and install it.
2. Open LM Studio, click the search icon, and download a model. Good starter picks for Hermes:
   • Llama-3.3-70B-Instruct (q4_K_M) — best general agent quality, needs ~40 GB RAM
   • Qwen3.5-32B-Instruct (q4_K_M) — strong tool use, ~18 GB RAM
   • Llama-3.1-8B-Instruct (q4_K_M) — fast, ~5 GB RAM
   • DeepSeek-Coder-V3 (q4_K_M) — coding-focused agents
3. Once downloaded, click Local Server, load your model, and click Start Server. The default endpoint is http://127.0.0.1:1234/v1.

Step 2: Point Hermes Agent at LM Studio

LM Studio's local server is OpenAI-compatible, so configure Hermes's OpenAI provider with a custom base URL:

# LM Studio is a first-class provider. The key is optional —
# set it only if you enabled auth on the LM Studio server.
export LM_API_KEY=lm-studio   # optional

# Pick the provider and default model (interactive wizard)
hermes model

# Or configure /opt/data/config.yaml directly:
# model:
#   provider: lmstudio
#   default: llama-3.3-70b-instruct
#   base_url: http://127.0.0.1:1234/v1   # optional, this is the default

The model name should match what LM Studio shows in its server panel — copy it verbatim.

Step 3: Verify with a Quick Chat

# From Hermes CLI:
hermes chat "Say hi and confirm you are running locally."

If Hermes replies and LM Studio's server panel shows the request, you're wired up correctly. Now connect a channel (Telegram, Discord, WhatsApp) and your local model is reachable from a phone.

Hardware Sizing Quick Reference

Switching Models at Runtime

Load multiple models in LM Studio and serve any of them. Hermes's /model command switches between them without restart:

/model llama-3.3-70b-instruct
/model qwen3.5-32b-instruct
/model deepseek-coder-v3

LM Studio vs Ollama vs vLLM

What's Next?

• Hermes Agent + Ollama — CLI-only local inference
• Hermes Agent + vLLM — Production-grade GPU serving
• Hermes Agent + Llama — Meta's open-weight family
• Hermes Agent + Mistral — Mixtral / Codestral for local use