custom-llm-provider-setupv1.0.0

A healthcare or legal team runs Ollama or vLLM on internal GPU servers so no data leaves their network. They wire the endpoint into Hermes as a custom provider via `model.api_key` in config.yaml, then validate the model with a streaming chat completion probe before enabling the agent. This satisfies compliance requirements while still giving staff an agent loop.

A solo developer builds an internal agent using Cloudflare Workers AI's `@cf/` models, avoiding OpenRouter markup and paying only per-request. They verify each candidate model's callability with curl probes (handling 403 not-registered and 404 deployment-missing responses) before committing the model id and base URL into a named `providers:` entry.

An engineer prototyping agent workflows runs LM Studio on their laptop with a local model, pointing Hermes at `http://localhost:1234/v1` with a dummy API key. They iterate quickly on prompt and tool behavior with zero cloud spend, resetting the session after each config change, then swap to a hosted endpoint for production.

A platform team routes Hermes traffic through Portkey's OpenAI-compatible gateway to gain observability, rate limiting, and fallback across multiple upstream models. They configure a named provider entry once and switch `model.default` per request, using Hermes' context_length override to match each upstream model's window.

A research group needs a model with an unusually large context window (e.g. 262k for Kimi K2.6) that isn't in Hermes' built-in list. They set `model.context_length` explicitly and probe with streaming to confirm big-reasoning-model latency is acceptable before adding it to the agent's roster, avoiding silent truncation or timeouts.

A vendor deploys and operates OpenAI-compatible inference endpoints (vLLM/Ollama clusters) on behalf of clients, then hands over a base URL and API key pre-wired into Hermes provider config. Clients get data residency and predictable latency without running GPUs themselves.

A consultancy specializes in wiring bespoke or non-standard LLM endpoints into Hermes, including the host-gated credential workarounds, streaming probe validation, and multi-model roster decisions. They deliver a working config plus a probe report covering 403/404/200 outcomes per candidate model.

A startup resells access to a curated, live-probed roster of models through an OpenAI-compatible gateway compatible with Hermes. Customers pick a model or fallback chain via a `providers:` entry, and the platform guarantees each listed model has passed a `hermes chat -Q` smoke test.