NIM is NVIDIA Inference Microservices, NVIDIA’s packaging of models as inference services with an OpenAI-compatible API, available either hosted on build.nvidia.com or as containers on your own GPUs.

Hermes Agent Guide

Hermes Agent + NVIDIA NIM: Hosted Models, and Your Own

Q: Does Hermes Agent support NVIDIA NIM?

Yes. Hermes includes an nvidia provider for NIM-hosted models on build.nvidia.com, configured with NVIDIA_API_KEY in ~/.hermes/.env.

Q: Can I point Hermes at a NIM container I run myself?

Yes, using the same nvidia provider. Set NVIDIA_BASE_URL=http://localhost:8000/v1 to point it at a local NIM instead of build.nvidia.com; no custom endpoint is required.

Q: Do I need my own GPU?

No. The hosted path on build.nvidia.com runs models on NVIDIA’s infrastructure with only an API key required. GPUs are needed only for self-hosted NIM containers.

NIM comes in two forms — NVIDIA's hosted catalogue and a container on your own GPUs — and the good news is that Hermes treats them as one provider. Moving between them is a single environment variable, which makes starting on the cloud and moving on-prem later a genuinely cheap decision.

Two things called NIM

NVIDIA Inference Microservices package models as inference services behind an OpenAI-compatible API. That packaging is used in two quite different ways:

	Hosted catalogue	Self-hosted container
Where it runs	build.nvidia.com	Your GPUs
You need	An API key	GPUs, drivers, a container runtime
Hermes setup	`nvidia` provider	Same provider, plus `NVIDIA_BASE_URL`
Good for	Getting started, bursty use	Data residency, steady heavy load

Most people asking about "NIM with Hermes" want the first. Start there.

Hosted setup

Create an API key from your NVIDIA account for the build catalogue.
Add it to the Hermes environment file.
Select the provider through the model picker.

# ~/.hermes/.env
NVIDIA_API_KEY=nvapi-your-key-here

hermes model
# choose the NVIDIA provider, then a model

If Hermes is not installed yet, start with the installation guide. Keys belong in ~/.hermes/.env, not in a shell profile you might commit.

On-prem setup

A NIM container exposes the same OpenAI-compatible chat completions API as build.nvidia.com, so Hermes keeps using the nvidia provider and you simply change where it points:

NVIDIA_BASE_URL=http://localhost:8000/v1

Upstream documents this for on-prem deployments such as a DGX Spark or a local GPU box, and describes switching between cloud and local as a one-line environment-variable change. That is worth planning around: you can start on the hosted catalogue while evaluating, then move the same configuration on-prem without rewriting how the agent reaches its model.

A few practical notes. Confirm the container is actually serving before pointing Hermes at it. Use the model id the container advertises. And remember that a NIM bound to loopback is invisible to a gateway running on another host — if the agent is not on the same machine, bind and firewall accordingly.

One convenience worth knowing on the hosted side: Hermes automatically attaches the NIM billing-origin header on requests to build.nvidia.com, so consumption is attributed correctly in NVIDIA's billing dashboard without any configuration from you.

Verifying

Send one small prompt and separate the layers before assuming the model is at fault:

Hermes recognises the provider.
The key is accepted.
The model id exists in the catalogue you are actually pointed at.
Tool calls come back well-formed — the thing that matters for agent work.

Model ids are the usual snag. NIM model identifiers are namespaced and do not always match the display name on the catalogue page. Copy the id from the model's own API tab rather than retyping the heading.

Troubleshooting

401 or 403

Confirm the variable is exactly NVIDIA_API_KEY.
Restart the gateway after changing its environment — a running process will not pick up the edit.
Check for stray whitespace or quotes around the value.
Confirm the key was issued for the build catalogue rather than another NVIDIA service.

Model not found

Re-copy the namespaced model id from the catalogue.
Confirm the model is available to your account and region.
If self-hosting, query the container's own model list rather than assuming the catalogue id.

Security

Use a dedicated key for the agent so it can be revoked without affecting other work.
Keep it in the Hermes runtime environment only.
Never paste a key into a screenshot, chat message, or support thread.
Review usage after the first few days — agent loops consume more than interactive chat does.

The equivalent walkthrough for the other framework is OpenClaw + NVIDIA NIM.

Hermes Agent + NVIDIA NIM FAQ

Does Hermes Agent support NVIDIA NIM?

Yes. Hermes ships an nvidia provider that talks to NIM-hosted models on build.nvidia.com. Set NVIDIA_API_KEY in ~/.hermes/.env and select it through hermes model.

What is NIM?

NVIDIA Inference Microservices — NVIDIA's packaging of models as inference services with an OpenAI-compatible API. They can be consumed from NVIDIA's own hosted catalogue, or run as containers on your own GPUs. Hermes's built-in provider handles both.

Can I point Hermes at a NIM container I run myself?

Yes, and you keep the same nvidia provider — no custom endpoint needed. Set NVIDIA_BASE_URL=http://localhost:8000/v1 and the provider talks to your local NIM instead of build.nvidia.com. Upstream describes moving between cloud and on-prem as a one-line environment-variable change.

Do I need my own GPU?

Not for the hosted path. build.nvidia.com runs the models on NVIDIA's infrastructure and you supply an API key. GPUs only enter the picture if you choose to run NIM containers yourself.