---
name: self-knowledge
description: Use when the user asks what Fermix is, what it can do, or how its agent loop, providers, channels, jobs, memory, sandbox, built-ins, skills, plugins, and config/setup surfaces fit together.
allowed_tools: []
---

# Fermix self-knowledge

Fermix is an Elixir/OTP agent daemon. `fermix run` or the OS service starts one BEAM VM; the CLI talks to it over `$FERMIX_HOME/daemon.sock`. Channel messages enter the gateway, run one agent turn, execute capabilities through the registry, and reply through the originating channel.

## Runtime shape

- Main Agent caches runtime context; channel messages go through the gateway, not directly to Main Agent.
- Gateway serializes one FIFO turn per conversation; different conversations run concurrently.
- Turn flow: read history -> build prompt -> model/tool loop -> reply -> commit messages -> background memory review -> optional auto-compaction.
- Main turns and new scheduled jobs default to 100 iterations. Regular `subagents` fan out wide (hard-max 10 tasks, ≤8 concurrent — default 4; 100 iterations each). `/ultra` is a **run-mode of the normal turn** (not a separate orchestrator): it tags the turn `run_profile: :ultra`, unlocking wider `subagents` caps (`subagent_mode: :ultra` → up to ~50 narrow probes, ≤12 concurrent, reduced per-probe depth) plus an exhaustive-mode prompt addendum driving both breadth (many narrow probes) and best-of-N depth (independent `subagents` on the *same* hard sub-problem, keep the best-supported answer). Its workers nest under the parent trace like regular `subagents`; they stay brief by instruction, not a low ceiling. Repeated identical tool calls trip the loop detector. These knobs are system-side config (`:subagents`, `:ultra`, `:iteration_limits`), not a `config.toml` surface.
- Sub-agents and unpinned scheduled jobs **inherit the main model** by default; `[fermix_core.routing]` `subagent_*`/`cron_*` (`model`/`provider`/`reasoning_effort`) can run them on a smaller model or lower thinking level — set via `config.toml`, the setup page (sub-agent only, on the primary provider's pane), or the `model_routing_config` tool; a model set without an explicit provider runs on the **primary** provider (a cross-provider worker needs an explicit `subagent_provider`/`cron_provider`). An explicit provider paired with a model the catalog knows under a *different* provider (e.g. an Ollama slug under `openrouter`) is a mis-pairing: it's rejected at write time (the tool validates the merged routing) and fails loud at spawn — so only a hand-edited `config.toml` can hold one, and even then it errors clearly instead of silently. The main agent can also pass a one-shot `model` to the `subagents` tool for a single call. It never changes its own model.

## Providers

Provider selection is **primary-flag driven**: each `[fermix_core.providers.<name>]` block carries `primary = true|false`, exactly one provider is primary, and every other *configured* provider is an automatic fallback (deterministic catalog order: `openai_codex`, `openai`, `anthropic`, `xai`, `openrouter`, `mistral`, `ollama` — Ollama deliberately last: a local model is the last-resort hop). Setup marks a newly configured provider primary by default and clears the old flag (settings/credentials are kept); the web setup provider page renders per-provider cards (status: Primary / Fallback / Not configured) as the provider selector — picking a card loads that provider into the "Configuring …" form, and saving it makes it the primary (needs a daemon restart); a configured, non-primary card also shows a "Set primary" button that flips the primary flag without re-entering credentials. Nothing is disabled, so any provider can be selected and set up. A Realtime API key reuses the OpenAI provider key slot but does **not**, on its own, promote OpenAI to primary — only a real provider credential or an explicit primary choice does. CLI OAuth login never changes primary. The legacy `[fermix_core.agent] provider` key is read-only migration input (used only when no `primary` flag exists; setup stops writing it). Routes are snapshotted at boot for main turns (restart to apply changes); scheduled jobs without an explicit provider resolve the chain at execution time — that asymmetry is intentional. **Failover** happens only on the turn's initial model call for transient errors (timeout/transport/5xx/rate-limit/quota, or a residual OAuth auth failure after the adapter's own refresh+retry); API-key auth failures, context-length, tool errors, mid-tool-loop (`continue`) errors, and mid-stream failures after visible streamed content never fail over. Before failover, the model call gets a **bounded same-provider retry** with short exponential backoff on transient infrastructure errors (the `connection_unavailable` pool-checkout / wake-from-sleep race, transport timeout/close/network, provider 5xx), so a brief flake self-heals on the same provider instead of surfacing or burning a failover hop. The retry budget is spent **before** any failover hop, on every route — the next route is a different **model** and the loop pins the winning route for the rest of the tool loop, so hopping on the first transient would silently re-target the whole turn onto a weaker model under a `status: ok`. `connection_unavailable` is network-wide so it retries-same and never fails over at all; scheduled jobs opt out of this inner retry entirely (so the two retry loops never stack and a slow provider `:timeout` cannot overrun the job's configured timeout) and keep their own coarser deadline-bounded backoff, which covers only the wake-from-sleep pool-checkout race — a cron run therefore still fails over on the first transient of any other kind. Separately, a **continuation** call (mid tool loop) that fails with a transport timeout the adapter MEASURED as pre-response (zero response chunks seen — a connect-phase or first-byte stall; only chunk-counting adapters like Codex can prove this, so a buffered adapter's timeout never qualifies) is re-issued in place on the same route with a short bounded backoff, on every surface including scheduled runs; it replays no tools, never switches provider, and any other continuation error still surfaces on the first failure. A genuine rate-limit/quota whose body carries a reset time surfaces a friendly "usage limit — try again in ~N min" reply. Failovers emit `[:fermix, :provider, :failover]` telemetry, `fermix doctor` lists configured fallbacks, and agent status exposes `primary_provider`/`fallback_providers`. Values: `openai`, `openai_codex`, `anthropic`, `xai`, `openrouter`, `mistral`, `ollama`. All seven run turns; the registry behind them is `FermixCore.Providers.Descriptor` (one declarative entry per provider — labels, auth modes, setup fields, config-key allowlists). **OpenRouter** (`[fermix_core.providers.openrouter]`: `api_key` via `OPENROUTER_API_KEY`, optional `base_url`, `default_model`, `primary`) rides the Chat Completions wire with vendor-prefixed model ids (`anthropic/claude-sonnet-4.6`, dots not dashes) and static attribution headers (`HTTP-Referer: https://fermix.sh`, `X-Title: Fermix`); no reasoning-effort field is sent (server default). **Mistral** (`[fermix_core.providers.mistral]`: `api_key` via `MISTRAL_API_KEY`, optional `base_url`, `default_model`, `primary`) also rides the Chat Completions wire (three rolling `-latest` tiers — `mistral-large-latest`/`mistral-medium-latest`/`mistral-small-latest`) with a plain Bearer key and no attribution headers; like OpenRouter it sends no reasoning-effort field. Mistral's strict validator rejects an assistant turn carrying empty-string `content` alongside `tool_calls`, so the shared Chat Completions adapter omits the `content` key whenever tool_calls are present (a wire shape valid on every Chat Completions provider). **Ollama** (`[fermix_core.providers.ollama]`: `base_url` — presence is what marks it configured, env `OLLAMA_BASE_URL`; `default_model`; `primary`) is **keyless** (`auth_mode :none` internally — no Authorization header, no secret) against the local OpenAI-compat endpoint (`http://localhost:11434/v1` default; remote/Tailscale hosts work) with a 300s receive timeout for slow local inference; catalog windows are model capability while the server may serve far less and truncate silently, so the doctor probe also POSTs the native `/api/show` and **fails loud when the served `num_ctx` undercuts the catalog window** (fix via `OLLAMA_CONTEXT_LENGTH` or a Modelfile `num_ctx`); a 404 probe means the model isn't pulled (`ollama pull <model>`). The web setup pane detects the Ollama server with a single probe — the configured URL either serves `GET /api/tags` or it doesn't: a reachable server lists **only the locally installed models** in the model picker; an unreachable one shows the error with `ollama serve`/install guidance and a free-form model input. The OpenRouter pane fetches the **live upstream catalog** (`GET /api/v1/models`, tool-capable models only, newest first) so all current models are selectable; on fetch failure it shows the error and a free-form input (`FermixCore.Providers.ModelListing`; static catalog stays authoritative for defaults and context windows). The "Model behavior" panel (reasoning effort / Codex fast) is hidden for providers without behavior knobs (OpenRouter, Mistral, Ollama). Unknown TOML keys in any provider block (and an unknown `[fermix_core.agent] provider`) **fail loud at config load** instead of being silently dropped. SpaceXAI Grok supports two auth modes via `[fermix_core.providers.xai] auth_mode = "api_key" | "oauth"`: bearer API key (`api_key`, `XAI_API_KEY`), or Grok Build subscription OAuth (loopback PKCE, profile `xai_oauth`; connect with `fermix auth login --provider xai`; a 403 means the Grok plan lacks API access — not a stale token). Both ride the OpenAI Responses wire shape with efforts `none|low|medium|high` (some Grok models reject effort and get it omitted; slash-containing enum values are stripped from tool schemas). Anthropic supports two auth modes via `[fermix_core.providers.anthropic] auth_mode = "api_key" | "oauth"`: API key (`api_key`), or Claude subscription OAuth (profile `anthropic_oauth` in auth.json; connect with `fermix auth login --provider anthropic` using `--setup-token`, `--import-claude-code`, or `CLAUDE_CODE_OAUTH_TOKEN`). OAuth requests emulate Claude Code (identity headers + system block, `mcp_`-prefixed tool names) and auto-refresh with one 401 retry. All Anthropic requests send prompt-cache breakpoints, request adaptive thinking on models that support it (the model decides when and how much to deliberate; `reasoning_effort` calibrates it; older models like Haiku 4.5 get no thinking param), and cap non-streaming output at `max_tokens` 16384 — sized so thinking plus the visible answer fit the buffered receive window. Per-provider model/effort: `[fermix_core.providers.<name>]` (`default_model` all providers; `reasoning_effort` only for effort-capable providers — OpenAI/Codex/Anthropic/SpaceXAI; Codex-only `fast`). OpenRouter/Mistral/Ollama blocks reject a `reasoning_effort` key at config load, routing-level effort overlays skip their routes, and their telemetry reports `reasoning_effort: nil`. Reasoning effort is one canonical vocabulary (`FermixCore.Providers.ReasoningEffort`: `none|low|medium|high|xhigh|max`) with per-provider subsets, mapped to each provider's wire field (OpenAI/Codex/SpaceXAI `reasoning.effort`; Anthropic `output_config.effort`). Anthropic has no `none` (floor `low`, API default `high`); a level above a provider's ceiling clamps. On top of the provider subset a model can carry its own ceiling in the catalog: `max` is a GPT-5.6-family capability, so the older OpenAI/Codex models (`gpt-5.5`, `gpt-5.4`, `gpt-5.4-mini`) top out at `xhigh` and an over-reaching config self-heals down to that model ceiling at route resolution instead of 400-ing at the provider. Models with no catalog ceiling pass through untouched (Anthropic's per-model nuance is left to the provider's own 400). Both the CLI wizard and web setup offer effort for the effort-capable providers only, and list only the levels the selected model actually accepts. `auth_mode` (`api_key`/`oauth`) round-trips through config for SpaceXAI and Anthropic. The web setup provider page has an API-key vs OAuth picker per provider (SpaceXAI = loopback "Connect Grok" like Codex; Anthropic = paste a `claude setup-token` or import a Claude Code login). A stored token is inert until `auth_mode = "oauth"`, so connecting in the web AND `fermix auth login --provider xai|anthropic` both set `auth_mode = oauth` in config (and `fermix auth logout` reverts to `api_key`); the change needs a daemon restart. The web setup Media tab also exposes an editable OpenAI/SpaceXAI key field inline next to the image-backend picker (same `openai_api_key`/`xai_api_key` provider secret, the same key those providers use for chat): it shows as already-configured when a key is stored — blank keeps the current key, a pasted value replaces it — so the `generate_image` key can be set without opening the provider's full setup form.

Provider/channel HTTP uses shared `FermixCore.Finch`. Idle keep-alive connections older than 15s are discarded at checkout. Codex retries `:closed` once only when it happens before response data; mid-response `:closed` and `:timeout` are not retried at the HTTP layer. A Codex 200 whose SSE stream delivered **no output at all** — no terminal event and no output items, or `response.failed`/`response.incomplete`/`error` with nothing generated — is classified as a transport close (retryable on the same route, failover-eligible) rather than read as an empty answer; the server's own reason, when it gave one, rides the error into the log and the trace. A stream that delivered output but never said it finished still returns that output (with a warning), because output items accumulate independently of the terminal event and discarding them would throw away a usable answer — and on a continuation, which has no retry or failover, it would kill the turn outright. A `:timeout` the adapter measured as pre-response (zero chunks) is retried one level up, at the agent loop's continuation seam (see the provider failover/retry notes above); a mid-stream or unmeasured `:timeout` is never retried.

Image input: an inbound image is forwarded to the model as a content block on vision-capable providers (Anthropic, the OpenAI Responses/Codex wire, and Grok); OpenRouter and Ollama are model-dependent (text-only local models are gated off). Each provider encodes the image at its own edge from one neutral content part, and the text-only request shape is unchanged, so prompt caching is unaffected. Tool RESULTS can also carry images, not just inbound messages: a tool that returns an image (e.g. the `browser` `screenshot` action) hands the bytes back as an image content part the model actually sees, encoded through the same per-provider edge (an Anthropic `tool_result` content array, or a placeholder tool message plus a following user image turn on the OpenAI-shaped wires). A continuation carrying an image to a non-vision route fails loud, never silently drops.

## Built-in capabilities

Built-ins seed into the single capability registry at boot; outbound MCP tools register as `mcp_<server>_<tool>`. The runtime prompt's `## Built-in Capability Catalog` is authoritative; use `tool_help` for one tool's schema/failure modes.

- **Files & search**: `file_read`, `file_write`, `file_edit`, `glob_search`, `content_search`
- **Shell & git**: `shell`, `git_read`, `git_write` (no push)
- **Web**: `web_search` (static facts/no URL), `web_fetch` (one known server-rendered URL — returns its readable text, or the JSON verbatim when the endpoint serves JSON), `browser` (JS/dynamic/interactive; `fill` replaces, `type` appends, `submit` submits; receipts are immediate; verify async changes with `wait`/`get`; `get field=rect` returns a selector match's viewport box in the same CSS space `click_coords` clicks in — the deterministic route onto a board/map/canvas the snapshot lists no elements for; `screenshot` returns the captured page as an image you can SEE (device pixels — its `device_pixel_ratio` converts to click_coords space); a managed browser caps live tabs, so each `open` past the cap auto-closes the oldest non-active tab — don't assume a long-idle tab is still open; see `browser-guidance`). Never shell-scrape JS sites.
- **Media**: `generate_image` — create or edit a raster image from a text prompt; the result is written under the sandbox `media/` floor and sent to the current chat automatically (file-only when no channel is available, e.g. a subagent/job). `operation` is `generate` (default) or `edit`; an edit's `input_image` is a sandbox path or `inbound:last` (the image just sent in this chat). Optional `mask` (transparent regions only — OpenAI backend only), `size`, `model`. The operator's config (`[fermix_core.tools.generate_image] backend`) picks the provider — `openai`, `xai`, `google`, or `openai_codex`; `edit` and `mask` are gated against the chosen backend's capabilities and rejected loudly when unsupported, never silently dropped. Reuses the OpenAI/SpaceXAI chat key, or `GEMINI_API_KEY` for Google. The **`openai_codex`** backend is different: it needs no API key and generates `gpt-image-2` through the ChatGPT-subscription Codex OAuth connection (billed to the subscription, not the platform API), via the built-in image tool on the Codex responses endpoint — an **experimental, undocumented** surface gated to ChatGPT auth (a plan that does not entitle it returns `auth_failed`). It supports generate + edit (no `mask`); a `router_model` config key names the GPT-5.x model that carries the image tool. Not for diagrams/charts/code assets or live data; voice sessions cannot use it (no chat channel to deliver to).
- **Memory**: `memory_store`, `memory_recall`, `memory_sources_list`
- **Jobs**: `schedule_job`, `update_job`, `list_jobs`, `pause_job`, `resume_job`, `remove_job`, `run_job_now`, `list_job_runs`, `get_job_run`
- **Skills & delegation**: `skill_view`, `skill_run`, `skill_list`, `skill_create`, `skill_reload`, `subagents` (bounded temp sub-agents)
- **Coding harness** (enabled by default but no harness tool is offered at all until the owner sets `approved` in Setup → Coding Agents; each run tool then advertises only when its vendor CLI was detected at boot, and when `default_vendor` is set only that vendor's run tool is advertised): `codex_run`, `claude_code_run` (delegate a repo coding task to the operator's own Codex / Claude Code CLI, run inside that repo), `list_coding_runs`, `get_coding_run`, `cancel_coding_run` (inspect and cancel runs). The cloud rail (`codex_cloud_run` + `stop_tracking_coding_run`) is off in this release (`[fermix_core.harness] cloud_enabled = true` to enable).
- **Meta**: `tool_help`, `send_attachment` (send a local sandbox file out through the active channel — outbound only; inbound images are materialized by the gateway, not via a tool; URLs rejected), `react` (acknowledge the user's current message with a single emoji reaction instead of a text reply — offered on channels that support reactions: Telegram, Discord, Slack, WhatsApp, Signal; absent on the CLI; ends the turn with no text bubble), `model_routing_config` (set the sub-agent model in `[fermix_core.routing]`: `subagent_model`/`subagent_provider`/`subagent_reasoning_effort`)
- **Tool-schema deferral (M10)**: default-on — `[fermix_core.tools.tool_search] enabled` absent means `true`; `enabled = false` is the kill switch. When on, plugin/MCP tool schemas leave the provider wire (names stay listed under `## Plugins`) and three bridges register: `tool_search` (BM25 over the deferred catalog), `tool_describe` (one tool's full schema on demand), `tool_call` (invoke a deferred tool; the loop unwraps it so traces/policy see the real tool name — direct calls by name also work). Off = bridges absent, all schemas inline.

Built-ins need no API keys except alternate backends/integrations; default `web_search` is DuckDuckGo. If a configured non-DuckDuckGo backend (Brave/Exa/Tavily/Firecrawl/etc.) hard-errors (auth, credits/HTTP 402, transport), `web_search` degrades once to keyless DuckDuckGo — loudly (a warning log + `degraded`/`primary_backend`/`fallback_reason` in the trace), not silently — so the broken backend stays visible. Empty results do not trigger the degrade.

## Skills

Skills are `SKILL.md` instruction packages, not provider-visible tools. `skill_view` loads a body; `skill_run` delegates as a sub-agent (recursion cap 4); `skill_list` enumerates. Bundled `priv/skills` and local `~/.fermix/skills` load at operator trust; plugin skills load at the guest trust level (capability-restricted), independent of the caller's gateway trust. `allowed_tools` narrows the trust default (`[]` = none, absent = trust default). `skill_create` writes to `~/.fermix/skills`; `skill_reload` re-scans the skill directories and refreshes the running agent in place (no restart) after a SKILL.md is created or edited on disk, reporting added/removed/changed names and load errors. CLI: `fermix skills list|view NAME|reload`.

## Memory

Durable memory is SQLite at `~/.fermix/memory.db`; ETS is cache. `memory_recall` does key lookup or FTS5 search over memories/history with `scope` (`current|owner|all`) and `source` (`memories|history|all`). `scope` is a request, not an authorization: the effective scope is resolved from the caller and the argument can only narrow within it — a scheduled run always reads its own job memory and a guest always reads the conversation it is in, whatever scope is named. Facts carry one of six categories on a general-assistant spine: `identity`, `preference`, `interest`, `goal` (about the user) and `context`, `directive` (about the work). Background review ("dreaming") consolidates rows on a ~24h gate and rebuilds two prompt-injected profiles — USER.md (the four user categories) and MEMORY.md (`context`/`directive`) — replacing stale facts rather than appending. CLI: `fermix memory review --now`, `fermix memory restore ID`.

## Jobs

`schedule_job` creates durable work without running it now. Schedule forms: interval (`every N minutes|hours|days`), one ISO8601 datetime (`once`), or 5-field cron; free-form English (e.g. "daily at 8am") is rejected — use `0 8 * * *`. Each cron field supports `*`, single values, comma lists (`1,15`), ranges (`9-17`), and steps (`*/15`, `8-18/4`); weekday `7` and `0` both mean Sunday; out-of-range or malformed fields are rejected at creation. Cron fires in the job's timezone (DST-aware; an unknown zone is rejected at creation). Runs are isolated bounded agent loops and cannot see the creating chat, so include needed facts in task text. If the daemon was down across a recurring job's fire time, a due time older than the freshness window (`[fermix_core.jobs] run_freshness_window_seconds`, default 3600) is **skipped** rather than fired late at the wrong wall-clock — the schedule just advances to the next future occurrence (logged). A one-off `once` run has no next occurrence, so it runs late instead of being dropped. A schedule expression or timezone that no longer parses (e.g. a corrupted row) is **terminal**: the job moves to a `disabled` state with the parse error in `last_error` and stops being retried — it must be fixed with `update_job` and then resumed, distinct from a user pause. Concurrent scheduled runs are capped (a small fixed ceiling); while the cap is full a due job stays `scheduled` and is claimed by a later tick as a slot frees, so a burst of simultaneously-due jobs never fans out without bound. A transient failure on the whole-loop retry is only retried before any tool has executed — once a tool has run, a mid-run connection loss fails the run loudly rather than replaying the tool's side effects (one narrow exception: a continuation call that timed out before any response byte arrived is re-issued in place by the agent loop with short bounded backoff — zero data was received, so nothing replays). A run that fires just as the host wakes can hit a not-yet-ready network: the runner classifies a pool-checkout `connection_unavailable` failure as transient infrastructure (not a provider-failover case) and re-runs the whole loop with bounded exponential backoff, and — when `[fermix_core.jobs] network_readiness_enabled` (default true) is on — first waits on a short, bounded TCP readiness probe to the primary route's host before the first model call. `expires_at` makes a temporary job; `delivery_mode` is `none|origin|channel|local`. `timeout_seconds` caps each run's wall clock (absent = the 30-minute daemon default) and `inactivity_timeout_seconds` arms a watchdog that fails a run whose provider/tool loop stops making progress (absent = unarmed); both are set at creation only — `update_job` cannot edit them — and `get_job_run`'s config snapshot echoes the values a run actually executed under. `allowed_tools` narrows the run to a subset of the caller's currently-visible tools (unknown names rejected); the model can never widen the run's capability policy past the creator's trust. **Operator-created** scheduled runs can delegate in parallel via `subagents` (regular caps — 10 tasks / 8 concurrent), and each worker's surface is the intersection of the delegation baseline and the run's own ceiling: a job confined by `allowed_tools`/`capability_policy`/`skill_name` spawns workers confined the same way, never wider. Guest-created runs never see `subagents` (it is policy class `external_api`, which the guest surface excludes), and `subagents` is only advertised to a run where it can actually execute. Every job is stamped at creation with its creator's own trust (operator or guest) — a context that carries no trust cannot create a job at all, so a job never inherits a trust the creator lacked. `skill_name` binds the run to an existing skill (rejected at creation if unknown): the run then executes inside that skill's confinement — the skill's `allowed_tools` and policy are intersected with the job's, never widened, and a guest job naming a skill whose policy grants nothing under guest trust fails loud rather than running unconfined. Optional `provider` + `model` pin which provider/model the job's runs use; they are both-or-neither (set both or neither — a pin without its pair is rejected), `provider` must be a known/configured provider (validated at creation against the same catalog the runner enforces), and `model` is a free-form provider-specific id. Omit both to use the default cron route (`[fermix_core.routing] cron_*`, else the primary/fallback chain resolved at run time). `update_job` edits a job in place — task, schedule, description, `skill_name` rebinding, `provider`/`model` route pin, and delivery (`delivery_mode`/`delivery_target`); omitted fields are left unchanged (delivery is never silently retargeted to a config default, and an omitted `provider`/`model` keeps the current pin), and switching delivery to `none`/`local` clears the target. A `clear_route_pin` boolean un-pins the job's `provider`/`model` back to default routing; it is mutually exclusive with `provider`/`model` (set those to re-pin instead) and combining them is rejected. `list_jobs` payloads surface `task_prompt` (the job's current instructions), `skill_name`, `provider`, `model`, `delivery_mode`, and `delivery_target` so the instructions, binding, pinned route, and destination are readable without reaching into the database. `run_job_now` fires a job immediately, out of band, through the same isolated runner (the run is tagged `trigger: "manual"`) and leaves the timed cadence untouched — use it to test a job or satisfy an on-demand request; it refuses when the job is paused/disabled/expired or already mid-run. `list_job_runs` reads a job's execution history (status/trigger/timing/outcome, newest first, optional `status` filter) and `get_job_run` reads one run in full (`task_prompt` the run actually executed — captured in its config snapshot, so it reflects the instructions at run time rather than the job's current ones; plus prompt snapshot, token usage, final response, error) — use these to confirm a job is actually firing and inspect what its runs produced.

## Channels & access control

Channels: Telegram, WhatsApp, Slack, Discord, Signal (text + media), plus local `cli` and `daemon`. Remote channels refuse to start without `owner_user_id` or `allowed_*_ids`. Inbound images on media-capable channels are downloaded at the gateway (sibling to audio transcription) and passed to the model as image content — the agent sees the picture, not a placeholder; audio attachments are transcribed to text first on all five remote channels — Telegram (voice notes, audio files, and video notes), WhatsApp, Slack, Discord, and Signal. When a voice note also carries a caption, both are delivered: the caption first, then the transcript under a `[voice note transcript]` delimiter. Transcription needs a configured backend, selectable in setup (`[fermix_core.transcription] backend` = `openai` | `xai` | `deepgram`; CLI flags `--transcription-backend`/`--transcription-model`/`--transcription-api-key`, plus a Transcription card on the web setup and a `transcription` row in `fermix doctor`). Each backend has its own optional key slot, always settable in the Transcription card (and `--transcription-api-key` stores under the selected backend's slot): `openai` (default, `gpt-4o-mini-transcribe`) and `xai` (SpaceXAI, Grok STT — modelless, so it has no model to pick and the card hides that field) take a transcription key that OVERRIDES the reused chat-provider key, or reuse that chat key if none is set. SpaceXAI STT REQUIRES an API key — the Grok subscription OAuth token does not work for `/v1/stt` — so paste one when the SpaceXAI provider is on OAuth. `deepgram` (`nova-3`) has no chat provider to reuse, so its `deepgram_api_key` is required. All three keys keychain as `@keyring`. `model` is a single shared key, so setup snaps it to the chosen backend's default on a backend switch (an unknown backend or non-positive `max_file_mb` fails config load loudly). When it isn't configured, the file is over the size cap, or the provider errors, the sender gets a specific reply instead of a silent drop (not configured → run `fermix setup`; too large → the size-cap limit; other failures → transcription failed, try again) and no turn is scheduled. An image whose resolved model can't accept vision fails loud rather than dropping the image silently. Multi-image messages are coalesced into one turn so the agent sees every image together — Telegram albums (separate updates sharing a `media_group_id`) and WhatsApp's separate per-image webhook messages are both buffered by a short debounce and merged; Discord/Slack/Signal already deliver all attachments in one message. An inbound message with no text AND no media (a sticker, poll, or blank — a captionless image still counts as actionable) is answered at the gateway with a brief "looks empty" note and never schedules a turn, so the queue stays free and no model is called; an unauthorized sender is dropped one step earlier and gets no reply at all. If a model ever returns an empty completion, the turn replies an honest "try again" instead of a blank message, and that empty reply is never committed — so it can't poison later turns. The conversation store independently refuses to persist or replay any content-less turn of any role.

Live streaming (**on by default** — `"block"`; set `streaming = "off"` per channel to disable): `[fermix_channels.<name>] streaming = "off" | "draft" | "block"`. `"draft"` shows the reply as one draft message edited in place (~1/s, ≥30 chars before the draft opens) and sealed to the final authoritative text; `/stop` deletes the draft; needs a draft-capable channel (Telegram today; `fermix doctor` warns otherwise). `"block"` sends each completed model "thought" as its own ordinary message (semantic boundary = completed output item; 800–1200-char paragraph chunking only as the long-text fallback, 1 s idle flush), including the model's 💭 reasoning-summary headings as separate one-line messages — works on every channel, and pre-tool commentary lands as its own message. Both need a streaming provider (Codex today — others simply deliver normally). Only real channel turns stream — background/CLI/cron runs never do. Stream telemetry: `[:fermix, :channel, :stream]` with `phase` open/edit/block/seal/discard and the turn's `session_id`.

Gateway trust: **operator** (`owner_user_id` or any local caller) gets full surface; **guest** gets read-only chat: no skills, MCP, exec, or network. `allowed_*_ids` authorizes chat only, not operator promotion. Read-only is a bound on what a guest may *do*, not on whose data comes back, so the surface is narrowed on a second axis too: capabilities that return the owner's own data — workspace reads (`file_read`, `glob_search`, `content_search`, `git_read`), scheduled-job reads (`list_jobs`, `list_job_runs`, `get_job_run`), and `memory_sources_list` — are never in a guest's surface, in the prompt, on the wire, or dispatchable by name. Tool discovery (`tool_search`, `tool_describe`, `tool_help`) is filtered by the same ceiling, so a guest cannot read the schema of a tool it cannot call.

Slash commands are pre-agent: `/help`, `/whoami` for any authorized sender. Operator-only: `/compact`, `/new` (`/clear`), `/sandbox` (`/grant`, `/revoke`, `/confirm`), `/soul`, `/stop`, `/pause`, `/resume`, `/background` (`/bg`), `/tasks`, `/ultra`. `/pause` and `/resume` control computer use: `/pause` hands the cursor and keyboard back to you mid-task (the session stays alive and resumable — unlike `/stop`, which tears it down), `/resume` lets it continue. Guests can be opted into owner-only commands via `command_allowlist` — but the sandbox mutation subcommands (`/grant`, `/revoke`, `/confirm`) are strictly operator-only and never reachable through the allowlist (a guest can chat and run `/new` but can never grant directory access or apply a confirmation). Auto-compaction runs after reply at threshold (default 0.85); `/compact` forces it.

`/stop` kills active turns and clears pending work; the stopped turn neither delivers nor commits a reply. Its user message was already persisted at turn start, so the gateway appends a short assistant marker after it ("stopped before I finished … context only") — the discarded request stays in history/memory but the next turn does not replay and answer it. The marker is only added when the conversation's last stored message is that orphaned user turn (no double-marking, role alternation preserved).

`/soul` curates the persona file (`SOUL.md`), owner-only and never autonomous. `/soul` alone reports the current revision; `/soul review [instruction] [--with-context]` drafts an edit — no instruction means a subtle, voice-preserving review held to a small change budget (declines if nothing warrants a change); an instruction is an explicit, unbounded ask; `--with-context` folds a hard-bounded window of the owner's *own* recent messages into the draft as evidence (guest turns are filtered out). The draft is one bounded provider call that advertises no tools and never writes; it returns a diff plus rationale, and prompt-injection markers detected in source memory are surfaced as a warning on that diff. Applying needs a second explicit step — `/soul apply TOKEN` (the token expires after a few minutes, enough time to read the diff; `/soul diff TOKEN` re-previews). `/soul history` lists revisions, `/soul revert N` rolls back to revision N, `/soul reset` restores the shipped default; every write (including reverts and resets) is versioned in the resource registry and itself revertable. The draft run emits `[:fermix, :soul_curation, :run_start|:run_complete|:run_error]` under its own minted `session_id` so the bounded provider call reassembles into one trace.

## Sandbox

Filesystem/exec built-ins (`shell`, `file_*`, search, `git_*`, `send_attachment`) go through the sandbox. Modes: `strict` (workspace + granted roots), `standard` (default: workspace + the launch/request cwd when it is under your OS home `$HOME` + granted roots — so the agent works where you ran `fermix ask`, with no hardcoded project-folder list), `open` (all of `$HOME` except protected/blocked). The request cwd is admitted only for a trusted local (operator) turn; remote channels fall back to workspace + grants. Protected paths key off the OS home (`~/.ssh`, `~/.aws`, `~/.gnupg`, …) plus Fermix internals and OS roots, and are denied in every mode even inside a granted root (protected always wins); catastrophic commands are denied in every mode too. Child env is minimal; secrets come from host env, keyring, or `source = "command"` helpers. Roots live in `[sandbox]` config as `workspace_root` + `allowed_roots`, managed via `fermix grant|revoke path PATH`, `fermix sandbox ...`, or `/sandbox` (web setup has no roots field — roots are CLI/config/chat only). Command profiles expose extra local commands as tools. **In-chat approval loop:** when a filesystem op is denied for being outside the roots and the task genuinely needs that directory, the agent (in an attended operator turn only) calls the `request_directory_access` tool — the owner sees the exact canonical path, the reason, and the config diff, and approves with `/confirm TOKEN` (single-use, origin-bound to that conversation, owner-only, 60s expiry). On Telegram and Discord the prompt also carries a one-tap **Approve** button whose payload holds the token privately (Telegram inline-button callback, Discord message component) — tapping it funnels through the same operator-only, single-use `/confirm` path; the Discord app must be configured to receive component interactions over the gateway. Because that button delivers the token, in a shared/group chat the raw `/confirm TOKEN` line is dropped from the group-visible text (it would otherwise expose the token to every member) and the owner approves via the button (or `/confirm` in a DM or the CLI); a non-owner who taps is refused before the token is consumed and (on Discord) sees an ephemeral "not authorized" note. Everywhere the token is NOT delivered privately by a button — a DM, the CLI, or any channel without the one-tap button (Slack, Signal, WhatsApp) — the tap-to-copy `/confirm TOKEN` stays in the prompt, because it is the only way the owner can confirm. The system refuses to even prompt for a path it would reject (`$HOME` wholesale, `FERMIX_HOME`, `~/.ssh`, OS roots). On confirm the grant persists to `[sandbox] allowed_roots` and, on a chat channel, the original request auto-resumes; on the one-shot CLI the owner re-runs it. A guest, cron, or unattended run never gets this approval path.

## Plugins

Plugins are connected integrations that own a surface (an API, a vault) and register their own tools; the always-present bundled trio is Gmail, Google Calendar, and Google Drive (Google OAuth). The agent discovers what is live from the `## Plugins` section of the runtime prompt — there is no plugin-list tool — and additional plugins install from a signed in-binary catalog over an http or mcp rail. Full detail: skill_view(name: "self-knowledge", file: "plugins").

## Voice (macOS, off by default)

The OpenAI Realtime voice companion (FermixPet) is local and off by default (`[fermix_core.realtime] enabled=true` plus an OpenAI Platform API key); the macOS app ships from the separate `tezra-io/fermix-macos` repo and connects over `$FERMIX_HOME/realtime.sock`. Inside a call the `screen_share` tool (start/stop) watches the operator's screen continuously — changed frames join the session as passive context, so a still screen costs nothing and a frame never makes you speak on its own; acting on what you see still goes through `computer_use`/`browser`. It needs computer use enabled+installed (same sidecar and Screen Recording grant), is voice-only (a text turn takes a `computer_use` screenshot instead), ends with the call, and is switched off with `[fermix_core.realtime] screen_share`. CLI: `fermix voice status`. Full detail: skill_view(name: "self-knowledge", file: "voice").

## Computer use (experimental, off by default)

Computer use is experimental and off by default (enabled from the setup Plugins page on Apple Silicon macOS and Linux x86_64); `computer_use` drives the host desktop GUI by screenshot plus mouse/keyboard, its safety `access` posture is derived 1:1 from `[sandbox] mode`, and it is operator-only — never delegated to subagents and never started by an unattended run. It refuses a pointer action whose coordinates are plausible on two image grids at once (magnified crop vs full screen) with a typed conversion error instead of guessing which image the model read. Full detail: skill_view(name: "self-knowledge", file: "computer_use").

## Coding harness

Repo coding work is delegated to a coding-harness run rather than edited file-by-file: `codex_run` and `claude_code_run` launch the operator's own Codex / Claude Code CLI inside a target repo. It is enabled by default (`[fermix_core.harness] enabled`) but gated on a first-use consent the owner grants once in Setup → Coding Agents (or `[fermix_core.harness] approved` in config) — there is no in-chat approval prompt, and until it is granted no harness tool is offered at all (run tools and the run-history tools alike) and Fermix simply does the coding itself with its own file/shell tools. Each run tool advertises only when its vendor CLI is detected at boot, and when `default_vendor` is set only that vendor's run tool is advertised (the other stays dispatchable by name). The cloud rail (`codex_cloud_run`) is off in this release — `[fermix_core.harness] cloud_enabled` defaults `false`. Full detail: skill_view(name: "self-knowledge", file: "coding_harness").

## Config & control surfaces

- `FERMIX_HOME` default `~/.fermix`: files `config.toml`, `auth.json`, `memory.db`, `daemon.sock`, `realtime.sock`; dirs `workspace/`, `bootstrap/`, `memory/`, `skills/`, `plugins/`, `browser/`, `journals/`, `realtime/`, `traces/`, `logs/`, `grants/`.
- Prompt files from setup: `bootstrap/main/{IDENTITY,FERMIX,SOUL,REALTIME}.md`, `memory/main/{USER,MEMORY}.md`; default agent id `main`. These are seeded once, at setup, and are yours after that — the seeder skips any file that already exists and no upgrade rewrites them, so a fresh install gets the improved shipped templates while an existing one keeps its copy. `fermix doctor`'s `bootstrap templates` row catches that drift by comparing what this install recorded when it was seeded against the template the current build ships (FERMIX.md/SOUL.md/REALTIME.md — variable-free templates only; IDENTITY.md embeds the agent name and is excluded), and warns which shipped templates have moved on so you can diff your file against the current one for missed improvements. It does not inspect your edits — a hand-edited file is fine. Installs seeded before revision tracking report no seed record.
- Personalization (the USER's name, timezone, communication style) is set at setup; timezone defaults to `America/New_York` (CLI wizard and web setup). The personalization step also collects the **assistant's own name** — that is identity, not a user preference, so it persists to `[fermix_core.agent].name` (default `fermix`), the source of truth that seeds `IDENTITY.md` (the first block of the system prompt) and is reconciled into its `**Name:**` line on each daemon boot — so changing the name in config and restarting updates what the agent answers, while a blank/unset name and any other operator edits to `IDENTITY.md` are left untouched. The current date (UTC, with the configured timezone as a label) is stamped into the system prompt every turn and into scheduled job runs, so the agent knows "today" without running `date`. Date-only by design — a clock time would bust provider prompt caches every turn; the precise time still comes from `date`.
- Config sections: `[fermix_core.agent]`, `[fermix_core.providers.<name>]`, `[fermix_core.tools.web_search]`, `[fermix_core.tools.tool_search]` (`enabled` — tool-schema deferral), `[fermix_core.tools.generate_image]` (`backend` openai/xai/google/openai_codex + `model`/`size`; `google_api_key` for Google; `router_model` for openai_codex, which needs no key and uses the Codex OAuth connection), `[fermix_core.transcription]` (`backend` openai/xai/deepgram + `model`/`max_file_mb`; per-backend `openai_api_key`/`xai_api_key`/`deepgram_api_key`, keychained — openai/xai override the reused chat key, deepgram is required), `[fermix_core.jobs|memory|realtime|computer_use|plugins|routing|harness]` (harness: coding-harness master gate `enabled` + first-use consent `approved` + steer `default_vendor` + cloud rail `cloud_enabled`, off by default), `[fermix_core.oauth.<provider>]` (plugin OAuth clients: `google`, `github`, `notion`, `x`, `slack` — client_id + client_secret, secret keyring-secured), `[fermix_core.plugin_secrets]` (per-plugin api_key credentials, keychained as `@keyring`, keyed by plugin name), `[sandbox]`, `[mcp.servers.<name>]`, `[fermix_channels.<name>]`.
- Secrets: plaintext config, `@keyring`, or env vars. Saving config auto-stores plaintext secrets to the OS keyring as `@keyring` when a writer is available (macOS `security`; account `fermix`, service `fermix:<ENV>` for the default profile or `fermix:<profile>:<ENV>` when `[fermix_core] profile` names a non-default one — `general`/unset keeps the bare `fermix:<ENV>`, so existing installs need no migration); without a writer, unchanged plaintext is kept (with a warning) and only new secrets fail. On macOS every save **deletes then re-adds** the keychain item with an open ACL (`security -A`) so the ad-hoc-signed daemon reads it headlessly with no authorization prompt; a legacy item that still triggers a repeating "security wants to use the login keychain" password prompt (written before the open-ACL delete-then-add) self-heals the next time it's written — re-run `fermix setup` to rewrite all secrets at once and the prompt stops. The optional `[fermix_core] profile` (default `general`) namespaces a host's keyring entries so two installs/workspaces (e.g. `~/.fermix` and `~/.fermix-dev`) don't overwrite each other's secrets. Set it **before** running `fermix setup` — secrets are written under the profile in effect at save time, and there is no migration when the profile changes, so flipping it on a populated install orphans the old entries (re-run `fermix setup` to re-write them under the new coordinate). `fermix setup --migrate-secrets` migrates plaintext in place. OAuth logins: `fermix auth login` (Codex), `fermix auth login --provider anthropic` (Claude subscription), `fermix auth login --provider xai` (Grok Build); `fermix auth status|logout` accept `--provider`.
- Web setup is local-session gated. `fermix setup` and the configured home dashboard mint a short-lived one-time `/setup?t=...` launch URL; the durable setup token is never placed in URLs.
- Service: launchd/systemd, user default or `--system`; control with `fermix start|stop|restart`. Inspect with `fermix status|health|doctor|logs -f|capabilities`; restart for config changes. Binary upgrades need a restart too: `fermix upgrade` (the built-in updater for standalone installs) swaps the cosign-verified binary and restarts + health-checks itself with rollback on failure, but on a package-manager install (e.g. Homebrew) it refuses to touch the managed binary and points at the package manager — and after `brew upgrade fermix` the daemon keeps running the old version until `fermix restart`. `fermix status` and doctor's daemon-socket check warn when the running daemon's version differs from the installed binary and name the restart fix; "upgraded but behavior unchanged" almost always means the daemon was never restarted. The installed unit pins a `PATH` (the directory `fermix` was installed into — its sibling `cosign` on a Homebrew install — plus the standard system + Homebrew bin dirs) so the supervised daemon can shell out to `cosign` (plugin-signature verification) and brew-installed `node`/`python` (MCP runtimes); a bare launchd/systemd `PATH` omits the Homebrew prefix, which makes plugin installs fail with a misleading `signature invalid`. The unit is a snapshot of install-time settings, but `fermix setup` self-heals a drifted unit — when the on-disk unit no longer matches what the current binary would write (e.g. the `PATH` or template changed across an upgrade), setup rewrites and reloads it instead of just restarting — so re-running setup picks up unit changes; `fermix service install` is the manual escape hatch.

## Observability

Traces are JSONL under `~/.fermix/traces/YYYY-MM-DD/<type>.jsonl`: `llm_call`, `tool_exec`, `agent_event`, `channel_msg`, `error`, `sandbox_event`; logs are `~/.fermix/logs/fermix.log`. No `fermix traces` verb. All log output (file and console, crash reports included) passes through a secret-redaction formatter: credential-shaped tokens (OpenAI/Anthropic `sk-…`, GitHub, Slack, SpaceXAI, Google, Telegram bot tokens, AWS key ids, bearer headers) are replaced with `[REDACTED:<vendor>]` markers — a marker in the log means the redactor caught a secret, not that data was lost.

`llm_call` and `tool_exec` carry `session_id`: main turns (`main-<n>`), subagents (random hex linked by parent session events), scheduled jobs (`cron_<job>_<ts>`), or realtime voice calls (`session:<n>`). Content capture is off by default; `FERMIX_OPIK_ENABLED=1` enables it unless `FERMIX_TRACE_CONTENT=0`; `FERMIX_TRACE_CONTENT=1` captures locally without Opik. Capture on means full-fidelity traces: input/output bodies are attached **whole** (no 2k truncation, no inspect eliding), and a failed browser action additionally carries the profile's recent console/JS-exception buffer in its error details — the trace is the one place to look when debugging. Capture off keeps traces bounded and body-free. The in-repo `fermix_opik` exporter (`apps/fermix_opik`, bundled in dev/prod, inert unless `FERMIX_OPIK_ENABLED`) exports nested traces and provides `mix opik.replay`.

Enabling Opik on the daemon: `fermix service install` snapshots a non-secret env allowlist (`FERMIX_OPIK_ENABLED`, `FERMIX_OPIK_BASE_URL`, `FERMIX_OPIK_PROJECT`, `FERMIX_TRACE_CONTENT`, plus the `FERMIX_HOME` baseline) into the launchd/systemd unit — a shell export alone never reaches the daemon, so reinstall after changing it; the API key is never written to the unit. `fermix doctor`'s "opik export" check asks the *daemon* over the control socket whether the exporter is off, enabled-but-not-bundled, or ready (with the resolved endpoint/project). Realtime voice calls are fully traced: `[:fermix, :realtime, :call_start|session_created|session_updated|provider_error|reconnect|call_stop]` lifecycle (→ `agent_event`), the model turn via `[:fermix, :provider, :call]`, and tool calls on the same `session_id` — so one call reassembles into one Opik trace.

## Reference files

This skill's overview stays in this file; deeper per-feature detail loads on demand as a named reference file:

- `coding_harness` — the coding-harness rails (`codex_run`/`claude_code_run` local delegation, `codex_cloud_run` cloud tracking), authorization, consent, and durable delivery.
- `computer_use` — the experimental host-desktop control capability: actions, the `access`/attended-origin safety model, coexistence, permissions, and config.
- `plugins` — connected integrations: the http and mcp rails, auth/credential errors, the signed catalog, and the operator CLI.
- `voice` — the OpenAI Realtime macOS voice companion (FermixPet): setup keys, the `tezra-io/fermix-macos` distribution, and the connection handshake.

Load one with `skill_view(name: "self-knowledge", file: "<name>")` (e.g. `skill_view(name: "self-knowledge", file: "plugins")`).