Local Model Guide

Hermes Agent + Qwen3.8-27B: Self-Hosted Setup

Q: Is Qwen3.8-27B released?

Yes. Qwen3.8-27B is published by Qwen on Hugging Face under Apache-2.0.

Q: Is Qwen3.8-27B the same as Qwen3.8-Max?

No. Qwen3.8-27B is a self-hostable open-weights model, while Qwen3.8-Max is a separate larger API model.

Q: Can Hermes run the model inside its own process?

Hermes Agent connects to Qwen3.8-27B through a separate inference server such as vLLM.

Qwen3.8-27B is the released 27-billion-parameter open-weights model, distinct from Qwen3.8-Max. Serve it behind an OpenAI-compatible API, then let Hermes provide the tools, memory, and channels.

27B vs Max

	Qwen3.8-27B	Qwen3.8-Max
Distribution	Open weights	Hosted API
Size	27B parameters	Much larger MoE model
Run locally	Yes, with sufficient hardware	No public local package
Hermes route	vLLM / compatible server	DashScope custom provider

Plan hardware before downloading

The official Hugging Face repository is roughly 55.6 GB. Runtime memory is higher than the weight files once KV cache and concurrency are included. Full-precision deployment generally needs a large GPU or multiple GPUs; a supported quantized build can reduce memory at some quality and throughput cost.

Serve Qwen3.8-27B with vLLM

python -m pip install -U vllm

vllm serve Qwen/Qwen3.8-27B \
  --host 0.0.0.0 \
  --port 8000 \
  --api-key local-qwen-key \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --enable-prefix-caching

Check the current Qwen model card and vLLM release notes before pinning a production environment. New architectures sometimes require a newer runtime than an existing server image contains.

Point Hermes at the server

# /opt/data/config.yaml
model:
  provider: custom
  base_url: https://qwen.example.com/v1
  api_key: local-qwen-key
  default: Qwen/Qwen3.8-27B

The model ID must match the name exposed by the inference server. If Hermes runs in another container on the same private network, use the vLLM service name. A managed Hermes instance needs a reachable HTTPS URL with authentication; it cannot reach a workstation-only localhost endpoint.

Security: vLLM’s API must not be left open on a public IP. Require a key, terminate TLS, restrict source networks, and rotate credentials.

Test agent behavior, not only text generation

Call /v1/models and confirm the model ID.
Send a normal chat request.
Give Hermes a harmless tool task and verify it emits a valid call.
Test a multi-step task with tool output returned to the model.
Measure latency and memory under the concurrency you expect.

A model can produce good prose yet be unreliable at tool selection or structured arguments. Keep tool approval enabled until your own workload passes.

Common failures

Architecture unsupported: upgrade to a runtime release that explicitly supports the model.
CUDA out of memory: reduce context/concurrency, use tensor parallelism, or choose a supported quantization.
Model not found: use the exact ID returned by /v1/models.
Tool calls are malformed: verify the server’s chat template and test a newer Hermes/Qwen-compatible runtime.

For the separate hosted Max model, use the Qwen3.8-Max guide.

Hermes and Qwen3.8-27B FAQ

Is Qwen3.8-27B released?

Yes. Alibaba’s Qwen organization publishes Qwen/Qwen3.8-27B on Hugging Face under Apache-2.0.

Is Qwen3.8-27B the same as Qwen3.8-Max?

No. The 27B repository is an open-weights model you can self-host. Qwen3.8-Max is a much larger API model.

Can Hermes run the model inside its own process?

Hermes is the agent client, not the inference server. Run Qwen behind vLLM or another compatible server, then point Hermes at its API.