Hermes Agent + Hugging Face: Inference Providers Setup

How the integration works

Hugging Face Inference Providers gives one client interface to models served by multiple inference partners. Hermes uses the native hf provider, reads your Hugging Face token, and sends the conversation to the model you select.

This is different from downloading model weights from the Hub. You do not need a local GPU for the hosted inference path described here. Usage, provider routing, model availability, and billing are handled through your Hugging Face account.

Before you start

• A working Hermes Agent installation
• A Hugging Face account
• A fine-grained Hugging Face token with inference-provider permission
• A chat-capable model available through Inference Providers

If Hermes itself is not installed yet, start with the Hermes Agent installation guide.

Step 1: create the correct Hugging Face token

1. Open your Hugging Face account settings and go to Access Tokens.
2. Create a fine-grained token.
3. Enable the permission labeled Make calls to Inference Providers.
4. Copy the token once and store it in a password manager.

The permission is the easy detail to miss. A general read token may let you browse gated repositories but still be rejected when Hermes calls an inference provider.

Step 2: add HF_TOKEN to Hermes

Add the token to the Hermes environment file:

# ~/.hermes/.env
HF_TOKEN=hf_your_token_here

Do not paste a real token into documentation, shell history, screenshots, or a Git-tracked .env. Restrict access to the file on shared systems.

Step 3: choose Hugging Face as the provider

The interactive model picker is the simplest route:

hermes model

Select Hugging Face, then choose an available model. For a one-off session, the official integration also supports:

hermes chat --provider hf

Use the picker when you want the selection saved as your normal provider. Use the provider flag when you only want to test Hugging Face for the current chat.

Step 4: verify the connection

Start with a small, non-sensitive prompt. Confirm all three layers separately:

1. Hermes recognizes the hf provider.
2. The token is accepted for Inference Providers.
3. The selected model is currently available for chat inference.

Keeping the first request simple makes an authorization or routing failure easier to distinguish from a model-specific prompt or tool-calling issue.

Choosing a model

Use the live list shown by hermes model and the model's Hugging Face page. Availability can change by provider, region, account, and model. A model hosted on the Hub is not automatically available through the serverless Inference Providers API.

Troubleshooting

401 or 403 authorization error

• Confirm the variable is named exactly HF_TOKEN.
• Confirm the token has the Inference Providers call permission.
• Restart the Hermes process after changing its environment.
• Check for whitespace, quotes, or an accidentally revoked token.

Model is unavailable

• Return to hermes model and choose from the current list.
• Confirm the model supports conversational inference.
• Check the model page for provider and access requirements.
• If the model is gated, accept its license and verify account access.

Provider does not appear

Update Hermes to a release that includes the native Hugging Face integration, then rerun the model picker. Refer to the official Hugging Face Hermes integration and the Hermes Agent repository for the current upstream behavior.

Security checklist

• Use a dedicated fine-grained token instead of a broad account token.
• Store it only in the Hermes runtime environment.
• Never commit ~/.hermes/.env or paste its contents into chat.
• Rotate the token if it appears in logs or screenshots.
• Review Hugging Face usage so unexpected calls are caught early.

Hermes Agent + Hugging Face FAQ

Related Hermes model guides

• Hermes Agent + OpenRouter
• Hermes Agent + vLLM
• Hermes Agent + Ollama
• Hermes Agent BYOK guide