--- name: agent-surface-design-system version: 0.5.0 description: > A design-system lens for building browser-based and agent-adjacent surfaces where humans review, annotate, approve, or steer coding agents. Extracted from Plannotator, Open Design's annotation system, and practical canvas-overlay experiments, then generalized into reusable laws, patterns, and component decisions for agentic interaction UIs. triggers: - "design an agent review UI" - "human-in-the-loop agent interface" - "agent plan approval UI" - "annotate agent output" - "build an agent feedback surface" - "design system for agent tools" - "pair an agent with a design system" - "MCP server for design artifacts" --- # Agent Surface Design System A pattern library for building surfaces where humans and coding agents meet. The patterns below are abstracted from [Plannotator](https://github.com/backnotprop/plannotator), [Open Design](https://github.com/nexu-io/open-design)'s annotation system, and a lightweight browser canvas overlay, then framed as reusable laws, layout decisions, interaction patterns, and integration strategies. Use this skill whenever you are designing a UI that an agent produces, a human reviews, and the result flows back to the agent. --- ## Core surface laws ## Law 1: The surface is a seam, not a replacement **Don't reimplement the agent. Don't replace the CLI. Inject a review seam into the agent's existing lifecycle.** The most effective agent surfaces hook into moments the agent already exposes: | Agent moment | What the surface does | |--------------|----------------------| | `ExitPlanMode` / `Stop` / `exit_plan_mode` | Pause, open browser, wait for human decision | | Tool-use boundaries | Intercept agent output before it becomes action | | End-of-turn | Render the agent's output for review before the next turn | | Explicit slash command | Let the user summon the surface on demand | **Pattern:** expose a single long-lived CLI command (e.g. `plannotator`) that the agent invokes with a very long timeout (e.g. 345600 ms). The command starts a local server, opens the browser, blocks on user input, then returns structured feedback to the agent's stdout/stdin loop. --- ## Law 2: Output first, chrome second **The agent's artifact is the hero. Every piece of chrome must justify its presence against the artifact.** Default layout priority: 1. **The artifact** — plan, diff, markdown, HTML, diagram — gets the center and the most screen real estate. 2. **Navigation/structure** — TOC, file tree, diff file list — gets a narrow left sidebar. 3. **Human input** — annotations, comments, AI chat, approve/deny — gets a right sidebar or bottom panel. ``` ┌─────────────────────────────────────────────────────────┐ │ Minimal header: title + mode + primary actions │ ├──────────┬──────────────────────────────┬───────────────┤ │ Structure│ Artifact │ Human input │ │ sidebar │ (maximize readability) │ / annotation │ └──────────┴──────────────────────────────┴───────────────┘ ``` **Pattern:** start with the artifact at full width; add sidebars only when the user needs them. Use collapsible panels, not permanent chrome. --- ## Law 3: Approve is a first-class action **The primary job of the surface is to produce a decision: proceed, revise, or abort. Optimize every layout and shortcut around that decision.** Primary actions should be visually dominant and available without scrolling: - **Approve / LGTM** — green, top-right, keyboard shortcut (e.g. `Cmd+Enter`). - **Request changes** — secondary but prominent, opens annotation/comment mode. - **Exit / skip** — tertiary, de-emphasized to prevent accidental dismissal. **Pattern:** after a "request changes" decision, the surface should diff the next revision against the previous one so the user can verify the agent addressed every point. --- ## Law 4: Annotations are the feedback protocol **Humans communicate with agents through annotations on the agent's own output.** Annotations are the bridge between human review and agent revision. Support at least these four forms: | Type | Use case | |------|----------| | Inline text | Comment on a sentence, requirement, or code line | | Range/selection | Comment on a block, function, or multi-line change | | File-level | Overall feedback on a file or section | | Threaded reply | Conversation about a previous annotation | **Pattern:** serialize annotations into a structured feedback payload that the agent can parse and act on. The agent should be able to map each annotation back to a location in its own output. For the mechanics of capturing and scoping annotation targets, see the Annotation laws (Law 31 onward). --- ## Law 5: One surface, many agents **The surface should not be coupled to one agent. Each agent gets a thin adapter; the surface stays generic.** Thin-agent, thick-surface architecture: ``` Agent A ──► adapter A ──┐ Agent B ──► adapter B ──┼──► generic surface ──► human review Agent C ──► adapter C ──┘ │ ▼ structured feedback routed back to caller ``` **Pattern:** keep agent-specific code in small adapter packages or plugins. The surface consumes a normalized payload (plan, diff, markdown, annotations) and returns a normalized decision. The adapter translates between agent conventions and the surface's protocol. --- ## Law 6: Local-first, share-optional **The surface must work fully on the user's machine without a backend. Sharing and collaboration are opt-in layers.** Benefits: - Works offline and in restricted environments. - No latency for the core review loop. - Privacy: sensitive plans/diffs never leave the machine unless the user chooses. **Pattern:** serve the UI from a local HTTP server spawned by the CLI. For sharing, encode small payloads in the URL hash; for large payloads, use client-side encryption with a paste service that stores only ciphertext. --- ## Law 7: Remote ergonomics are first-class **SSH, devcontainers, and remote servers are normal. The surface must not assume a local browser.** Patterns: - Detect remote environments and switch to a fixed port (`19432` is a reasonable default) instead of a random one. - Print the local URL to the terminal when browser-open fails or is suppressed. - Keep all WebSockets and terminal traffic on the same origin/port as the main UI to avoid extra port forwarding. - Provide `BROWSER=/usr/bin/true` or equivalent for CI/automation harnesses. --- ## Law 8: Agent context belongs in the same workspace **When the user wants to run an agent while reviewing, embed it; don't open a second tool.** A review or annotation surface can optionally embed a terminal/agent panel alongside the artifact. The design rules for embedded agents: - **No auto-start.** The agent must not start until the user explicitly starts it. - **One agent at a time.** Avoid multi-agent complexity in v1. - **Same origin/port.** Proxy terminal WebSockets through the main server. - **Graceful cleanup.** Interrupt, then kill; never orphan PTY processes. - **Fallback UI.** If the terminal runtime can't load, disable the panel with a clear message; don't break the rest of the surface. --- ## Law 9: Use proven primitives, don't invent layout **The surface is dynamic. Use a layout system that supports resizing, tabs, panels, and persistence out of the box.** Recommended stack decisions from Plannotator: | Concern | Choice | Rationale | |---------|--------|-----------| | Layout framework | `dockview-react` | Resizable, tabbed, draggable panels with API-driven state | | Component primitives | Radix UI | Accessible dialogs, menus, tooltips, popovers | | Styling | Tailwind CSS + CSS variables | Theme tokens, dark/light/system modes | | Icons | `lucide-react` | Consistent, lightweight | | Toasts | `sonner` | Non-blocking feedback | | Motion | `motion` | Subtle transitions without heavy motion | | Syntax highlighting | highlight.js in a worker pool | Keep the main thread free | **Pattern:** theme tokens should be semantic (`bg-card`, `border-border`, `text-muted-foreground`) so components look correct across light/dark/system modes without per-component overrides. --- ## Law 10: Human input should feel like annotation, not form-filling **The best feedback is created by pointing at something and typing a short note. Minimize modal forms and prefilled fields.** Interaction patterns: - **Select → comment.** Highlight text or a diff range, then type. - **Floating toolbar.** Show annotation tools only when text is selected. - **Pinpoint overlay.** Let users click anywhere on an HTML artifact to drop a comment. - **Quick labels.** One-click labels (e.g. "question", "blocking", "nit") plus optional freeform text. - **Keyboard-first.** Shortcuts for create comment, approve, next file, next annotation. --- ## Law 11: Diff is a first-class artifact **When reviewing code, the diff is the document, not the file.** Patterns for code review surfaces: - Render diffs with syntax highlighting and line numbers. - Support side-by-side and unified views. - Allow comments on added/removed/context lines. - Track "viewed" state per file. - Show semantic diff or AI-generated summary as an optional panel. - Keep file tree in sync with diff navigation. --- ## Law 12: Make revision history visible **Humans don't trust agents to revise correctly. Show what changed between versions.** Patterns: - Store prior versions of the artifact. - Render a diff between the current version and the previous version. - Highlight annotations that were addressed, partially addressed, or untouched. - Let the user click an annotation and see the corresponding change in the new version. --- ## Law 13: Agent jobs are ambient, not blocking **Background agent work should be visible but not steal focus.** Patterns: - A dedicated panel or badge shows running agent jobs. - Logs are available on demand, not streamed into the main view. - Jobs can be inspected, cancelled, or restarted from the surface. - Completed jobs surface their output as annotations or file changes. --- ## Law 14: External tools can push into the surface **The surface should accept input from tools outside the agent that spawned it.** Patterns: - Expose an SSE or WebSocket endpoint for external annotations. - Accept file-change events from a filesystem watcher or git status. - Let other tools add comments, labels, or status checks. - Treat external input the same as native input in the UI. --- ## Law 15: Persistence is explicit, not automatic **Don't auto-save every keystroke in a way that surprises the user. Make save, export, and archive explicit actions.** Patterns: - Auto-save drafts locally, but surface a "save" or "submit" action for the final decision. - Export feedback as markdown, JSON, or a GitHub review. - Archive approved plans/decisions to a knowledge base (e.g. Obsidian) with clear metadata and backlinks. --- ## Canvas-overlay laws ## Law 16: A canvas overlay is a valid surface **You don't need a full SPA to annotate a page. A single `` overlay can be enough.** For quick, page-native annotation: - Create one full-page `` with `position: absolute; z-index: 2147483647`. - Store annotations in **page coordinates** (`clientX + scrollX`, `clientY + scrollY`) so they scroll with the content — no per-frame redraw on scroll. - Use `pointer-events: none` by default; enable only when a drawing tool is active. - Resize the canvas to `max(scrollWidth, innerWidth)` and redraw on `resize` and `ResizeObserver` events. This pattern is for lightweight, page-native overlays that work on any web page without modifying the host app's code. For structured visual marks inside a preview surface, see Law 32. --- ## Law 17: Tools should be keyboard-first **Avoid modal toolbars. Plain keys switch modes instantly.** From browser-annotate: | Key | Tool | |-----|------| | `F` | Freehand | | `A` | Arrow | | `S` | Shape / ellipse | | `T` | Text | | `H` | Highlight/select DOM element | | `Z` | Undo | | `R` | Redo | | `C` | Copy selections | | `Q` | Clear all | | `Ctrl+D` | Toggle overlay on/off | **Pattern:** one global toggle (e.g. `Ctrl+D`) to enter/exit the surface; once inside, single keys map to tools. Show the shortcuts in a small, non-blocking badge so users learn them. --- ## Law 18: Select real DOM elements, not just pixels **Pixel annotations are expressive but fragile. Combine them with structured DOM selections for agent-usable references.** In "highlight" mode: - Use `document.elementFromPoint(x, y)` to find the element under the cursor. - Draw a hover outline around the candidate element. - On click, record a structured selection: tag, id, classes, CSS selector, bounding box, truncated text, and outer HTML. - Assign each selection a numbered `data-sel` attribute so the page itself marks what was selected. - Generate a stable CSS selector path (id first, then classes, then `:nth-of-type()` fallback) so the agent can re-find the element later. **Pattern:** copy selections as structured text the agent can parse: ``` [sel 1] selector="div#eligibility > svg > g > rect:nth-of-type(3)" text="" rect=67,353,680x119 ``` --- ## Law 19: Annotations should be movable and undoable **Treat annotations like objects, not permanent paint.** - **Implicit move:** on hover near an existing annotation, change the cursor to `grab`; on drag, move the annotation and redraw. - **Hit-test backward** so the topmost annotation is picked first. - **Undo/redo stack:** maintain `operations` and `redoStack`; `Z` pops from `operations`, `R` pushes back from `redoStack`. - **Clear all:** a single shortcut (`Q`) removes everything with confirmation implied by the destructive nature of the key. --- ## Law 20: Expose a global handle for programmatic access **Let the host environment (or the agent) read the surface state.** Attach a namespace to `window` and expose a small, serializable API. See [resources/global-surface-api.js](resources/global-surface-api.js) for a copy-pasteable implementation that strips private `_el` references before returning data. **Pattern:** keep the public API small and serializable. Strip internal DOM references (`_el`) before returning data. --- ## Law 21: Inject via init scripts for persistence **If the surface must survive navigations, register it as an init script.** For browser-automation surfaces (cmux, Playwright, Puppeteer, etc.): - Use `addInitScript` so the overlay is injected into every new document in the same browser context. - Also `eval` the script into the current page for immediate effect. - Wrap the script to handle both `document.body` already present and `DOMContentLoaded` for late injection. See [resources/init-script-overlay.js](resources/init-script-overlay.js) for a copy-pasteable init script. **Pattern:** a thin shell wrapper can auto-inject the overlay whenever a browser surface is opened, so the user doesn't run a manual install step each time. --- ## Law 22: Jank is acceptable if the interaction is right **A janky but predictable overlay beats a polished tool you never use.** Browser-annotate explicitly accepts limitations: - Works only on already-open surfaces; it does not auto-apply to future surfaces in all contexts. - Uses a fixed-position toolbar, not a docked panel. - Text input is a `