--- name: hermes-provider-media-configuration_KM description: "Umbrella for Hermes model/provider and media-generation configuration: Claude subscription auth/claude-cli lanes, Codex/OpenAI image generation defaults, Cloudflare image MCP, usage collection dashboards, live voice architecture, voice pipeline sensitivity, and MOSS/Samantha voice workflows. Use whenever Hermes auth, providers, image generation, usage telemetry, voice, or TTS behavior is being configured or diagnosed." --- # Hermes Provider and Media Configuration Use this for Hermes provider/auth and media subsystems. ## Areas - **Main model/provider selection**: target the intended Hermes profile explicitly, set model/provider/base_url as a unit, and verify with `hermes status` plus a smoke test. When changing the root/default profile, also inspect `fallback_model`/`fallback_providers`, session-local `model_override` entries, and the root auth store; it is common for the visible default to be Grok while fallback or auth still points at Codex/Spark. If another profile already has a working OAuth credential, copy or re-auth it into the target profile before declaring the switch complete. For root/default xAI Grok switches, also inspect/update `fallback_model` and root `~/.hermes/auth.json`; `hermes status` can show Grok from config while runtime still fails if only the GPT profile has xAI OAuth credentials. See `references/xai-grok-default-auth-and-fallback-2026-05-17.md`. - **Backup model host routing**: when Qwopus or backup models are in scope, do not default to the Mac Studio endpoint. Kosta’s current preference is to route Qwopus backup models through the Gaming PC Atomic route; inspect live config and change provider/base URL/model together. Current Qwopus route should be `custom:atomic-qwopus` / `qwopus-atomic` / `http://100.93.10.54:1338/v1`. See `references/qwopus-backup-model-gaming-pc-routing-2026-05-09.md` and `mlops-model-workflows/references/gamingpc-atomic-qwopus-current-2026-05-22.md`. - **Custom local inference providers**: for Studio/RTX OpenAI-compatible providers, endpoint checks and fresh `hermes -z` probes are not enough. Before claiming Telegram/Discord support, verify the already-running gateway process has loaded the new config/code and complete a live gateway tool-loop test: selected custom provider → tool call → model resumes after tool result → final delivered. If Qwopus visibly prints pseudo-tool tags/JSON instead of executing tools, inspect the direct OpenAI `message.tool_calls` contract; model-only health does not prove tool-loop support. Kosta's RTX providers may look duplicated, but the current split represents different OpenAI-compatible services/ports with different latency, timeout, and context behavior; keep user-facing cleanup in `model_aliases` unless Hermes gains a deliberate one-provider/per-model-`base_url` design. See `references/qwopus-custom-provider-live-gateway-verification-2026-05-10.md` and `references/hermes-default-grok-and-mini-compression-2026-05-17.md`. - **Z.AI / GLM configuration**: for default-profile GLM-5.1, use provider `zai` and the Z.AI coding base URL; GLM deep thinking uses Z.AI `thinking.type`, not OpenAI-style `reasoning_effort`. For browser/GUI vision, use `glm-5v-turbo` as an explicit vision auxiliary rather than assuming GLM-5.1 is multimodal. Z.AI's coding `/models` listing may omit plan-gated models that still work, including `glm-5v-turbo`; if `/model glm-5v-turbo` is auto-corrected to `glm-5-turbo`, test a direct OpenAI-compatible call first, then fix validation so exact curated model IDs win before close-match typo correction. For Z.ai vision auxiliaries, set `auxiliary.vision.extra_body.thinking.type: disabled` when reasoning is unnecessary, but also filter tool output to visible `message.content` only; never copy `reasoning_content` into `vision_analyze`/`video_analyze` results. If `glm-5v-turbo` works through direct OpenAI-client calls but Hermes `vision_analyze` returns Z.ai 401, check that gateway image pre-analysis hydrates the active profile `.env` before calling vision and that named-provider `base_url` overrides do not get re-resolved as `custom` with a missing/wrong API key. If GLM works on tiny prompts but fails with 429 during heavy Hermes/browser loops, inspect request dumps and browser-tool env/config first; stale `BROWSER_CDP_URL` plus `AGENT_BROWSER_AUTO_CONNECT` can break browser calls, force terminal-HTML fallback, and create secondary provider pressure. See `references/zai-glm-default-profile.md`, `references/deepseek-v4-flash-zai-browser-lanes-2026-05-06.md`, `references/zai-glm-browser-ralph-diagnosis-2026-05-06.md`, `references/zai-glm-browser-agent-diagnosis-2026-05-06.md`, `references/zai-glm-browser-agent-diagnostics-2026-05-06.md`, `references/zai-glm-vision-reasoning-hygiene-2026-05-06.md`, and `hermes-prompt-state-and-memory/references/browser-agent-split-harness-and-zai-debug-2026-05-06.md`. - **Default-profile vision fallback**: when the default profile uses a text-only main model such as DeepSeek v4, do not leave `auxiliary.vision` on vague `auto` if image handling matters. Set an explicit vision backend, currently known-good on this machine as `auxiliary.vision.provider: zai`, `model: glm-5v-turbo`, `base_url: https://api.z.ai/api/coding/paas/v4`, and verify with `resolve_vision_provider_client()` before claiming images are covered. - **GPT profile vision/compression vs image generation**: if Kosta says 'fix vision for gpt' or asks why Qwopus appears in GPT, check `auxiliary.vision`, `auxiliary.compression`, and legacy `compression.summary_*` first. `image_gen.provider` is a separate setting and should not be changed just because vision analysis or compression routing is wrong. The `codex` alias is valid for vision and normalizes to `openai-codex` in provider resolution. For `gpt-5.4-mini` used as Codex compression, Kosta currently expects `auxiliary.compression.context_length: 400000` across root/default and local profiles; if a Grok/default session warns that mini is only 272K and auto-lowers compression, fix the aux compression context override and per-profile context cache rather than lowering the main compression threshold. See `references/default-profile-vision-codex-routing-2026-05-11.md`, `references/gpt-profile-qwopus-compression-correction-2026-05-11.md`, and `references/hermes-default-grok-and-mini-compression-2026-05-17.md`. - **Codex account routing by profile**: when Kosta wants GPT to use one ChatGPT/Codex account and other Hermes profiles to use another, treat Hermes provider auth and Codex app-server auth as separate surfaces. Update the relevant `auth.json` files and set `model.codex_app_server.codex_home` per profile; otherwise `/codex` may keep using the global `~/.codex` account even if Hermes provider auth changed. Verify both Codex homes initialize and remember long-running gateway processes need restart/reload to see code/config changes. See `references/codex-profile-account-routing-2026-05-15.md`. - **Default-profile auxiliary routing**: `auxiliary..provider: default-profile` makes a named profile follow the root/default profile's current `model.{provider,default,base_url}`. Use this when GPT should keep its chat model but route side work such as web-extract cleanup through whatever GLM/default model the default profile uses today. - **Claude subscription auth**: distinguish Claude Code subscription auth, native Anthropic OAuth/API behavior, and a `claude-cli` provider lane. - **Image generation**: keep Kosta's default image generation on Codex/OpenAI GPT Image through `image_gen.provider: openai-codex` unless he explicitly asks to change it. Do not switch image generation to fal just because a FAL key is present. If Kosta asks about image/SVG models **"via Gaming PC inference"** or points at saved GamingPC info, do not answer only from `image_gen.*`; route to `mlops-model-workflows` and `gamingpc-upscale-stack` first. Current distinction: GamingPC Qwopus is the text/code/SVG inference brain; GamingPC ComfyUI is verified for upscale/restoration/vectorization, while Hermes `/image` remains GPT Image 2 unless explicitly rewired. See `references/hermes-codex-image-gen-default.md`. - **Video generation**: use fal.ai for Seedance 2.0 video generation via `video_gen.provider: fal`, with `bytedance/seedance-2.0/text-to-video` and `bytedance/seedance-2.0/image-to-video`. Store `FAL_KEY` safely, expose it through Hermes `video_generate`/`video` rather than `image_generate`, verify registry/tool availability without spending credits, and only run a live video generation smoke test after Kosta agrees. If adding media tools, remember that `registry.register()` only registers the handler; platform visibility also depends on `toolsets.py`, `hermes_cli/tools_config.py`, and API/gateway toolset tests. For website hero-background I2V, prompt Seedance as a slow “living poster,” preserve source anatomy/pose, animate only mist/light/parallax, keep negative space for typography, and report render cost as duration × current fal per-second price. See `references/fal-seedance-video-and-hermes-tools-2026-05-06.md` and `references/seedance-image-to-video-hero-background-2026-05-07.md`. - **Usage telemetry**: prefer integrating existing local collectors before building new provider scrapers. - **Profile throughput benchmarking**: compare Hermes profiles with `HERMES_HOME= hermes --ignore-rules -z ...`, final-output-only prompts, repeated trials, and visible-output tokens/sec. Report median first, include model/provider/`agent.reasoning_effort`, and call out hidden-reasoning caveats. Do not change intended reasoning defaults just to improve speed numbers; default/general DeepSeek should stay `high` unless Kosta explicitly approves otherwise. See `references/hermes-profile-throughput-benchmarking.md`. - **Compression context overrides**: if startup warns that `gpt-5.4-mini (openai-codex)` has only ~272K context while the main model threshold is higher, do not lower compression thresholds as the first fix. Kosta’s current local definition treats the mini compression model as 400K. Set `auxiliary.compression.context_length: 400000` in the affected profile(s), and seed/update that profile’s `context_length_cache.yaml` for both `gpt-5.4-mini@https://chatgpt.com/backend-api/codex` and the trailing-slash variant. Verify with a one-shot that no compression warning is emitted. - **GamingPC custom-provider shape**: current Qwopus should route through the Atomic provider (`custom:atomic-qwopus`, `qwopus-atomic`, `1338`). Older `custom:*RTX*` entries may still exist for historical or specialized sidecars, but do not treat `custom:RTX` / `qwopus-gpu` / `8010` as current unless a fresh live check proves a deliberate fallback. Prefer model aliases for user-facing cleanup. - **Memory synthesis model routing**: when benchmarking or configuring models for mem0/session-search synthesis, do not replace deterministic retrieval with an LLM. Use deterministic `session_search.summary_mode: fast` first, then route optional classify/dedupe/extract/summarize microtasks through a separate auxiliary path. Keep this distinct from the GPT profile itself: Kosta does **not** want Qwopus in the GPT profile's active chat, vision, or compression lanes. GPT should stay `openai-codex` / `gpt-5.5` with `agent.reasoning_effort: medium`, and `auxiliary.vision` plus `auxiliary.compression` should also use GPT-5.5 unless he explicitly changes that. Qwopus/GamingPC routing belongs in default/root or specialized memory/auxiliary-memory lanes, not GPT's own compression path; stale notes saying “root/GPT” should be interpreted as root/default plus memory-specialized paths, not Hermes GPT. On Mac Studio, `mem0_oss` memory writes now support automatic Qwopus→Spark fallback after primary timeout/failure; details are in `hermes-prompt-state-and-memory/references/mem0-oss-qwopus-spark-fallback-2026-05-11.md`. The strict-memory benchmark still showed deterministic retrieval fastest and flagged that hallucination coverage was only deterministic markers/canaries, not a full semantic hallucination judge. Haiku, DeepSeek V4 Flash, LFM2, and Qwen3-0.6B are not good strict-memory default brains. Detailed results live in `mlops-model-workflows/references/gamingpc-specialized-model-routing-2026-05-10.md` and the benchmark directory under `~/.hermes/shared/benchmarks/memory-model-bench/`. - **Live voice**: separate realtime transport/provider architecture from Hermes-as-brain orchestration. For Mini App live voice diagnostics/crash handling and the Gemini Live → `hermes_turn` → Hermes GPT/tool-use architecture, see `references/hermes-mini-app-live-voice-diagnostics-2026-05-06.md`. If Gemini Live connects but stalls or crashes around `hermes_turn`, especially after the first spoken prompt, also read `references/gemini-live-tool-response-voice-bridge-2026-05-13.md` before blaming tokens/timeouts; Gemini 3.1 Flash Live expects the Hermes tool payload wrapped under `response.result`, and DB/accounting failures must not tear down the WebSocket. If voice connects, handles an early prompt, then dies with `provider event pump failed` or `No module named 'importlib.resources'`, read `references/hermes-miniapp-live-voice-provider-pump-debugging-2026-05-13.md`; verify Hermes control path with `HermesClient(get_settings())`, then isolate Gemini no-tool vs tool-response behavior before blaming tokens, timeouts, or the Mini App UI. For voice controls, barge-in, generation gates, Gemini cancellation, Mini App UI states, and background SSE/fetch streaming, read `references/gemini-live-voice-bridge-controls-runtime-2026-05-15.md` before editing runtime code. - **Voice pipeline audit**: trace VAD/STT provider resolution, background noise sensitivity, and local model choice. - **MOSS/Samantha**: prepare spoken-style text, language/script handling, speed/duration budgeting, and verification. ## Support references Provider/media specialist skills are stored under `references/` with original commands and caveats. Load the matching reference when touching a provider-specific path; provider quirks are easy to misremember and should be verified from the reference before editing config. For config/provider edits, verify with `hermes status`, then run a tiny one-shot against that same `HERMES_HOME` before calling it done. Relevant profile/provider notes: - `references/hermes-codex-image-gen-default.md` — preserve or restore Codex/GPT Image defaults when OpenAI-backed image generation is intended. - `references/codex-profile-account-routing-2026-05-15.md` — route Codex/OpenAI account selection by Hermes profile, including separate `CODEX_HOME`/`model.codex_app_server.codex_home` handling for `/codex` app-server sessions. - `references/fal-image-generation-provider-2026-05-06.md` — add/configure the built-in fal.ai image provider, store `FAL_KEY` safely, and verify registration without burning credits. - `references/cloudflare-claude-image-mcp.md` — Cloudflare/OpenAI API-key backed MCP image service notes. - `references/qwopus-custom-provider-live-gateway-verification-2026-05-10.md` — live-gateway and textual-tool-call verification ladder for Qwopus/RTX custom providers. - `references/codex-mini-compression-context-400k-2026-05-17.md` — explicit 400K context override/cache pattern for Codex mini compression when Hermes under-resolves it as 272K.