---
name: screencast-desktop
description: Drive the Windows desktop with feedback — a screenshot and an element tree come back after every action, like the Playwright plugin does for the browser — plus record what happens into a finished demo video with zoom to clicks, a smooth cursor, and voiceover. Triggers: "click around in the app", "look at what's on the screen", "record a demo of this app", "screencast with zoom", "Screen Studio for Windows", "drive the desktop". NOT for web pages → the playwright plugin; NOT for editing existing footage; NOT for AI video generation.
---

# Screencast Desktop

Two MCP servers work together:

- **`windows-ui`** — control via Windows UI Automation: finds controls by
  name, clicks, types. Also works with Electron apps (VS Code, Teams, Slack),
  where a plain tree walk returns nothing.
- **`screencast`** — vision and video: screenshots, window recording, assembling
  the video with camera moves, cursor, and voiceover.

## The main rule: never act blind

Look before every step and after it. The tools return state on their own —
`desktop_click`, `desktop_type`, `desktop_key` have a `see` parameter
(`shot` by default, `none` to save context).

Workflow:

1. `desktop_windows` — what is open.
2. `ui_snapshot` (the `windows-ui` server) — controls by name. Cheap, precise,
   survives the window being moved. **Try it first.**
3. If the app gives no tree — `desktop_screenshot`, look with your eyes and
   click by coordinates.
4. Act and verify against the returned shot.

## Recording a demo

```
desktop_place_window(window="My App", monitor=1, fit=0.7)   # verifies it fits
desktop_record_start(window="My App")
… ordinary clicks and typing — each one is written to the journal …
desktop_record_stop()
desktop_render(out_dir="…", max_zoom=2.0)
```

The camera is built from the action journal, not guessed from the video: the
zoom starts **half a second before the click**, because the click moment is
known in advance. Rendering does not touch the app — you can change parameters
and rebuild as many times as you like.

## What breaks silently — verified on a real machine

**Window position.** An app is not obliged to take the size it was given:
a UWP window asked for 2380×1490 came back at 3967×2426 and hung off the edge
of the screen — in the recording the app was cropped. `desktop_place_window`
measures the result, adjusts, and reports whether it fit.

**A freshly launched app** restores its own size a couple of seconds after the
window appears, overwriting the one you set. Set the size after a pause.

**Display scaling.** A process that has not declared itself DPI-aware sees a
screen smaller than the real one (2560×1440 instead of 3840×2160) and misses
buttons by a factor of 1.5. The servers handle this themselves; your own
scripts must do it too.

**Multiple monitors.** Coordinates are shared across all screens; non-primary
ones go negative. `desktop_monitors` shows the layout.

**Capture is window-only.** Desktop Duplication on a laptop with two GPUs sees
only the monitors of its own adapter, and on a secondary one it delivers a
single frame and goes silent: the recording looks fine until you compare the
frames. This plugin uses Windows Graphics Capture per window — it does not care
which monitor or which GPU, and the desktop never gets into the frame.

**Frames arrive only when the picture changes.** A motionless window produces
not a single frame; the recording keeps a steady pace by repeating the last
one.

**State between takes.** The app remembers the previous run — an unclosed
search box turned the input into "githubgithub". Reset it before starting the
recording.

**Voiceover.** Speech is generated before the recording; its duration sets the
pauses — that way the click lands inside the phrase that describes it. Write
the script about actions, not content: the contents of a live list change
between writing the script and recording.

## Privacy

Before recording, check the frame: mail, private conversations, passwords,
notifications. Turn on "Do not disturb". Only the window is recorded, but the
window itself may contain something private.

## Installing dependencies

`/screencast-desktop:setup` — installs the missing Python packages and checks the
machine (`doctor.py`: DPI, monitors, encoders, capture, voice).