--- name: auto-webhooks description: Configure ADO service hooks and PR Review build validation policy for AI automation agents. Uses ADO REST API via az rest (no native CLI command for service hooks). Reads all config from .ai/automation/infra.json. when_to_use: "Use when configuring ADO service hooks and build validation policies for AI automation, when the user says 'set up webhooks', 'configure service hooks', 'connect ADO to Lambda', or 'wire up the automation triggers'." argument-hint: "" --- You configure ADO service hooks and the PR Review build validation policy. ADO has no native `az devops service-hooks` CLI command — all REST calls use `az rest` (ADO PAT auth, audit-wrapped). ## 0. Prerequisites Read `.ai/automation/infra.json`. Extract `automationProfile`. ```bash source .ai/lib/audit.sh export AUDIT_LOG_PREFIX=infra ``` **Profile determines scope:** - **Full hub:** WI hooks (project-scoped) + PR Answer hook (repo-scoped) + PR Review build policy - **Consumer:** PR Answer hook (repo-scoped) + PR Review build policy only. Consumers do NOT create WI hooks (hub owns those). Consumers DO need their own PR Answer hook and PR Review policy. Extract: - `adoOrg` — ADO org URL - `adoProject` — ADO project name - `pipelines.pr-review.id` — PR Review pipeline ID (for build validation policy) - `pipelines.pr-answer.id` — PR Answer pipeline ID (for registering with hub) - For **full-hub only**: `webhooks.wi-userstory.url` (BugFix uses the Azure-native `webhooks.bugfix` comment hook — see 2c — not a WI-Router URL) - Check if `webhooks.*.status` is already `configured` — skip if so **For consumers:** The PR Answer hook URL points to the **hub's** PR Router Lambda. Ask: > **Hub's PR Router Lambda URL?** The API Gateway URL from the hub project's infra.json (e.g., `https://.execute-api.us-east-1.amazonaws.com/prod/pr-answer`). Ask once for shared values needed for hooks: > **Webhook username?** (same as Lambda `BASIC_USER` set in hub's `/auto-lambda-env`) > **Webhook password?** (same as Lambda `BASIC_PASS`) — secret, not stored > **Webhook secret?** (same as Lambda `WEBHOOK_SECRET`) — secret, not stored Ask for ADO project ID (required by service hook API): > **ADO project ID?** Find in ADO > Project Settings > Overview > Project ID (GUID format). You can also run: `az devops project show --project "" --query id --output tsv` Get the repository ID for repo-scoped hooks: ```bash REPO_ID=$(az repos show --repository "" --project "" --query id --output tsv) ``` Read `scm.base-branch` from `.ai/config.yaml` for branch filtering. ## 0b. List Existing Service Hooks Before creating any hooks, list all existing subscriptions to avoid duplicates: ```bash EXISTING_HOOKS=$(az rest --method GET \ --uri "/_apis/hooks/subscriptions?api-version=7.1" \ --query "value[?consumerInputs.url != null].{id:id, event:eventType, url:consumerInputs.url, repoId:publisherInputs.repository}" \ --output json) ``` For each hook to create, check if a subscription already exists with the same `eventType` AND the same `url` AND (for repo-scoped hooks) the same `repository`: - If found: **skip creation**, report `⏭ already exists (ID: ) — skipping` - If not found: create the hook **Work-item hooks** are project-scoped and should exist only ONCE per project (hub only). If already configured from another repo, skip. **PR Answer hooks are per-repo** — each repo needs its own hook filtered to that repo + base branch. Without repo filtering, a single project-scoped hook fires on every PR comment across hundreds of unrelated repos, flooding the Lambda with noise. **Migration:** If an existing project-scoped PR Answer hook exists (no `repository` filter — from an older setup), warn the user and offer to delete it after creating the repo-scoped replacement. **PR Review build validation policy** is per-repo — always check by `repositoryId` before creating. ## 1. User Story WI Hook (Work Item Updated — User Story) — HUB ONLY **Skip for consumer profile.** Routes all User Story `workitem.updated` events to the WI Router Lambda at `/wi`. The Lambda uses tag-based routing to determine which agent to invoke (DoD, QA, DevAgent, DOCAgent, Estimation). **DoR is no longer here** — it has its own Azure-native `@kai-dor` comment hook (§2d). ```bash az_resource "ado/hooks/wi-userstory" \ az rest --method POST \ --uri "$(adoOrg)/_apis/hooks/subscriptions?api-version=7.1" \ --headers "Content-Type=application/json" \ --body "{ \"publisherId\": \"tfs\", \"eventType\": \"workitem.updated\", \"resourceVersion\": \"1.0\", \"consumerId\": \"webHooks\", \"consumerActionId\": \"httpRequest\", \"publisherInputs\": { \"projectId\": \"\", \"workItemType\": \"User Story\", \"tag\": \"KAI-TRIGGER\" }, \"consumerInputs\": { \"url\": \"\", \"basicAuthUsername\": \"\", \"basicAuthPassword\": \"\", \"httpHeaders\": \"x-webhook-secret:\" } }" \ --query 'id' --output tsv ``` Update `infra.json`: - `webhooks.wi-userstory.subscriptionId` → returned ID - `webhooks.wi-userstory.status` → `"configured"` ## 2. BugFix trigger — moved to a comment hook (see 2c), no longer Lambda-routed **There is no Bug `workitem.updated` → WI-Router hook anymore.** The BugFix agent was migrated to the same Azure-native model as SimpleAgent: it is triggered by a Bug **comment** containing `@kai-bugfix` (section **2c**), delivering to the BugFix pipeline's Incoming WebHook — no Lambda, no tag hook. `bugfix` is intentionally absent from `wi-router.mjs`'s `AGENTS` array. Configure the hook in **2c** instead. ## 2b. SimpleAgent Comment Hook (Work Item Commented On) — HUB ONLY, Azure-native (no Lambda) **Skip for consumer profile.** This is the only hook that does **not** route through a Lambda. SimpleAgent (`ado-cli-simple.yml`, `/dx-simple`) is triggered entirely inside ADO: a comment containing `@kai-simple` on a work item fires this Service Hook, which delivers to the SimpleAgent pipeline's **Incoming WebHook** service connection (declared under `resources.webhooks` in the pipeline). The same event drives both the first run and recovery — the pipeline's Phase 0 decides fresh-vs-resume. **Prerequisite — Incoming WebHook service connection.** Names are fixed to match `ado-cli-simple.yml` (and `ado-cli-simple-router.yml`): - **Webhook Name:** `kai-simple` (matches the `webhook:` alias in the pipeline) - **Service connection name:** `kai-simple-trigger-sc` (matches `connection:` in the pipeline) - **Secret:** optional HMAC secret reused by the Service Hook below (loop-safety / authenticity) Because SimpleAgent is fully Azure-native (no Lambda / no API Gateway), this connection is the one piece of trigger infra and can be created via the ADO REST API — no manual UI step. Ask once: > **Incoming WebHook secret?** (optional — HMAC secret shared between the Service Hook and the Incoming WebHook connection. Leave blank for none.) — secret, not stored First check whether the connection already exists (idempotent): ```bash EXISTING_SC=$(az rest --method GET \ --uri "//_apis/serviceendpoint/endpoints?endpointNames=kai-simple-trigger-sc&api-version=7.1" \ --query "value[0].id" --output tsv 2>/dev/null) ``` - If non-empty: **skip creation**, reuse it. Report `⏭ kai-simple-trigger-sc already exists (ID: ) — skipping`. - If empty: create it via `az_resource` (audit-wrapped). Include `"secret"` in `data` only if a secret was provided: ```bash az_resource "ado/service-connection/kai-simple-trigger-sc" \ az rest --method POST \ --uri "/_apis/serviceendpoint/endpoints?api-version=7.1" \ --headers "Content-Type=application/json" \ --body "{ \"name\": \"kai-simple-trigger-sc\", \"type\": \"incomingwebhook\", \"url\": \"\", \"authorization\": { \"scheme\": \"None\", \"parameters\": {} }, \"data\": { \"webhookName\": \"kai-simple\" }, \"serviceEndpointProjectReferences\": [{ \"projectReference\": { \"id\": \"\", \"name\": \"\" }, \"name\": \"kai-simple-trigger-sc\" }] }" \ --query 'id' --output tsv ``` Where `` is `, \"secret\": \"\"` when a secret was provided, or empty otherwise. `` is the GUID from step 0. **Fallback (UI):** if the REST create is rejected (older ADO without the `incomingwebhook` endpoint type), create it manually: Project Settings → Service connections → New → **Incoming WebHook**, Webhook Name `kai-simple`, Service connection name `kai-simple-trigger-sc`. The trigger token is config-driven — read it so the hook filter matches the skill (`dx-simple.recovery.trigger-token`, default `@kai-simple`): ```bash TRIGGER_TOKEN=$(bash .ai/lib/dx-common.sh yaml-val 'dx-simple.recovery.trigger-token'); TRIGGER_TOKEN=${TRIGGER_TOKEN:-@kai-simple} ``` Create the Service Hook on the **"Work item commented on"** event, filtered so the **comment contains `$TRIGGER_TOKEN`**, delivering to the `kai-simple` Incoming WebHook (the pipeline's webhook resource) — **not** an API Gateway / Lambda URL. In the ADO UI: Project Settings → Service Hooks → **+** → select the trigger that posts to the pipeline's Incoming WebHook, set event = *Work item commented on*, and set filter **"Comment contains" = ``**. > The filter string MUST equal `dx-simple.recovery.trigger-token` — the skill, the pipeline header comment, and this hook all read from that one source of truth. The bot never emits the literal token in its own comments, so its own replies cannot self-trigger the hook (loop-safety). **Which pipeline does the hook target?** - **Single-platform projects (default):** point the hook at the **`simple`** pipeline's `kai-simple` Incoming WebHook, exactly as described above. The one dx-simple pipeline applies the change directly. - **Multi-repo projects:** point the hook at the **`hub`** (KAI-HUB router) Incoming WebHook (`ado-cli-hub.yml`) instead of `simple` directly. The hub parses the `@kai-` tag, runs `dx-discover-repos`, and queues each agent's worker once per resolved repo (reading the `repos.json`/`agents.json` registries). The per-repo workers keep their `resources.webhooks` listener (they are dual-mode), but the **comment hook fires the hub**, not the individual workers. A single hub hook (filter: comment contains `@kai`) covers all agents. Update `infra.json`: - `webhooks.simple.connection` → `kai-simple-trigger-sc` - `webhooks.simple.connectionId` → service connection ID (`$EXISTING_SC` or the created endpoint's `id`) - `webhooks.simple.subscriptionId` → returned Service Hook ID - `webhooks.simple.status` → `"configured"` ## 2c. BugFix Comment Hook (Work Item Commented On) — HUB ONLY, Azure-native (no Lambda) **Skip for consumer profile.** Identical mechanism to **2b** (SimpleAgent) — the BugFix agent (`ado-cli-bug-fix.yml`, `/dx-bug-all`) is also fully Azure-native: a comment containing `@kai-bugfix` **on a Bug** fires a Service Hook delivering to the BugFix pipeline's Incoming WebHook (`resources.webhooks`). The same event drives the first run and every recovery re-trigger; Phase 0 (`resume-check.sh`) decides fresh-vs-resume. **BugFix is NOT in the WI-Router Lambda's `AGENTS` array** — do not configure a `workitem.updated`/tag hook for it. **Prerequisite — Incoming WebHook service connection** (names match `ado-cli-bug-fix.yml`): - **Webhook Name:** `bugfixHook` (matches the `webhook:` alias) - **Service connection name:** `kai-bugfix-trigger-sc` (matches `connection:`) Create it exactly as in 2b (idempotent `az rest` GET → POST `incomingwebhook` endpoint), substituting `kai-bugfix-trigger-sc` / `bugfixHook` for the simple names. UI fallback is the same. The trigger token is config-driven: ```bash TRIGGER_TOKEN=$(bash .ai/lib/dx-common.sh yaml-val 'dx-bug-all.recovery.trigger-token'); TRIGGER_TOKEN=${TRIGGER_TOKEN:-@kai-bugfix} ``` Create the Service Hook on **"Work item commented on"** with **two** filters — comment contains `$TRIGGER_TOKEN` **AND** Work Item Type = `Bug` (the pipeline's `resources.webhooks.filters` already enforces the Bug type, but filtering at the hook avoids firing the pipeline for non-Bug comments). Deliver to the `bugfixHook` Incoming WebHook — never a Lambda URL. > The filter string MUST equal `dx-bug-all.recovery.trigger-token` — skill, pipeline header, and hook share that one source of truth. The bot never emits the literal token, so its replies can't self-trigger (loop-safety). Update `infra.json`: - `webhooks.bugfix.connection` → `kai-bugfix-trigger-sc` - `webhooks.bugfix.connectionId` → service connection ID - `webhooks.bugfix.subscriptionId` → returned Service Hook ID - `webhooks.bugfix.status` → `"configured"` ## 2d. DoR Comment Hook (Work Item Commented On) — HUB ONLY, Azure-native (no Lambda) **Skip for consumer profile.** Same mechanism as **2b**/**2c** — the DoR agent (`ado-cli-dor.yml`, `/dx-dor`) gets a comment trigger: a comment containing `@kai-dor` **on a User Story** fires a Service Hook delivering to the DoR pipeline's Incoming WebHook (`resources.webhooks.dorHook`). The work item id comes from the payload (`${{ parameters.dorHook.resource.id }}`); manual runs pass `workItemId` and the pipeline falls back to the payload id. **DoR is fully comment-triggered now (no Lambda).** Like SimpleAgent and BugFix, `dor` has been **removed from the WI-Router Lambda's `AGENTS` array** — the §1 "WI User Story" `workitem.updated` hook no longer routes to DoR (it still serves the remaining tag-routed agents: dod, qa, devagent, docagent, estimation). Configure only this comment hook for DoR; there is no tag/Lambda path to disable. **Prerequisite — Incoming WebHook service connection** (names match `ado-cli-dor.yml`): - **Webhook Name:** `dorHook` (matches the `webhook:` alias) - **Service connection name:** `kai-dor-trigger-sc` (matches `connection:`) Create it exactly as in 2b (idempotent `az rest` GET → POST `incomingwebhook` endpoint), substituting `kai-dor-trigger-sc` / `dorHook` for the simple names. UI fallback is the same. The trigger token is config-driven: ```bash TRIGGER_TOKEN=$(bash .ai/lib/dx-common.sh yaml-val 'dx-dor.trigger-token'); TRIGGER_TOKEN=${TRIGGER_TOKEN:-@kai-dor} ``` Create the Service Hook on **"Work item commented on"** with **two** filters — comment contains `$TRIGGER_TOKEN` **AND** Work Item Type = `User Story` (the pipeline's `resources.webhooks.filters` already enforces the type, but filtering at the hook avoids firing the pipeline for non-Story comments). Deliver to the `dorHook` Incoming WebHook — never a Lambda URL. > The filter string MUST equal `dx-dor.trigger-token` — skill, pipeline header, and hook share that one source of truth. `/dx-dor`'s own comment is prefixed `[DoRAgent]` and never contains the token, so its replies can't self-trigger (loop-safety). Update `infra.json`: - `webhooks.dor.connection` → `kai-dor-trigger-sc` - `webhooks.dor.connectionId` → service connection ID - `webhooks.dor.subscriptionId` → returned Service Hook ID - `webhooks.dor.status` → `"configured"` ## 3. PR Answer Service Hook (PR Commented On) — ALL PROFILES Routes to the PR Router Lambda. **This hook is repo-scoped** — each repo (hub and every consumer) creates its own hook filtered to that repo and base branch. Without repo filtering, a project-scoped hook fires on every PR comment across all repos in the ADO project. The PR Router Lambda uses `ADO_PR_ANSWER_PIPELINE_MAP` to route each repo's events to the correct PR Answer pipeline. ```bash az_resource "ado/hooks/pr-answer" \ az rest --method POST \ --uri "/_apis/hooks/subscriptions?api-version=7.1" \ --headers "Content-Type=application/json" \ --body "{ \"publisherId\": \"tfs\", \"eventType\": \"ms.vss-code.git-pullrequest-comment-event\", \"resourceVersion\": \"1.0\", \"consumerId\": \"webHooks\", \"consumerActionId\": \"httpRequest\", \"publisherInputs\": { \"projectId\": \"\", \"repository\": \"\", \"branch\": \"\" }, \"consumerInputs\": { \"url\": \"\", \"basicAuthUsername\": \"\", \"basicAuthPassword\": \"\", \"httpHeaders\": \"x-webhook-secret:\" } }" \ --query 'id' --output tsv ``` - `` — for hub: from `webhooks.pr-answer.url` in infra.json. For consumer: the hub's PR Router Lambda URL (asked in step 0). - `` — this repo's ADO repository GUID (from step 0). - `` — from `scm.base-branch` in config.yaml (e.g., `development`). Update `infra.json`: - `webhooks.pr-answer.subscriptionId` → returned ID - `webhooks.pr-answer.status` → `"configured"` ## 4. PR Review Build Validation Policy — ALL PROFILES PR Review is not a service hook — it's a build validation policy on the target branch. This triggers the PR Review pipeline on every PR against that branch. Use `REPO_ID` from step 0 and `BASE_BRANCH` from config.yaml (already read). No need to ask again — default to `scm.base-branch`. Only ask if not set in config: > **Target branch for PR Review?** (e.g. `main`, `develop`) — pull requests targeting this branch will trigger the PR Review agent. ```bash REPO_ID="" BRANCH="" PR_REVIEW_PIPELINE_ID=$(python3 -c "import json; print(json.load(open('.ai/automation/infra.json'))['pipelines']['pr-review']['id'])") az_resource "ado/policy/pr-review-build-validation" \ az rest --method POST \ --uri "//_apis/policy/configurations?api-version=7.1" \ --headers "Content-Type=application/json" \ --body "{ \"isEnabled\": true, \"isBlocking\": false, \"type\": {\"id\": \"0609b952-1397-4640-95ec-e00a01b2f659\"}, \"settings\": { \"buildDefinitionId\": $PR_REVIEW_PIPELINE_ID, \"queueOnSourceUpdateOnly\": true, \"manualQueueOnly\": false, \"displayName\": \"AI PR Review\", \"validDuration\": 720, \"scope\": [{ \"repositoryId\": \"$REPO_ID\", \"refName\": \"refs/heads/$BRANCH\", \"matchKind\": \"Exact\" }] } }" \ --query 'id' --output tsv ``` Update `infra.json`: - `webhooks.pr-review.policyId` → returned ID - `webhooks.pr-review.status` → `"configured"` ## 5. Summary Report Adapt the report to the profile: ### For full-hub: ```markdown ## ADO Webhooks Configured (Hub) | Hook | Event | Scope | URL | Status | |------|-------|-------|-----|--------| | WI User Story | workitem.updated | Project (tag: KAI-TRIGGER) | | ✓ configured | | WI Bug | workitem.updated | Project (tag: KAI-TRIGGER) | | ✓ configured | | SimpleAgent | workitem.commented (comment contains @kai-simple) | Project → pipeline Incoming WebHook (no Lambda) | kai-simple-trigger-sc | ✓ configured | | BugFix | workitem.commented (comment contains @kai-bugfix, type Bug) | Project → pipeline Incoming WebHook (no Lambda) | kai-bugfix-trigger-sc | ✓ configured | | DoR | workitem.commented (comment contains @kai-dor, type User Story) | Project → pipeline Incoming WebHook (no Lambda) | kai-dor-trigger-sc | ✓ configured | | PR Answer | git.pullrequest.comment-event | Repo: , branch: | | ✓ configured | | PR Review policy | Build validation | Repo: , branch: | (pipeline trigger) | ✓ configured | **Tag-based routing:** The WI Router Lambda uses work item tags to determine which agent pipeline to queue. Adding a new agent only requires Lambda env vars (TAG_GATE_* + pipeline ID) — no new hook or route needed. **infra.json** updated with subscription IDs and status. **Audit log:** `.ai/logs/infra..jsonl` ### Verify Test the WI hook: Go to ADO > Project Settings > Service Hooks > find "WI User Story" > click "Test" to send a sample event. ### Next step `/auto-alarms` — Set up CloudWatch monitoring ``` ### For consumer: ```markdown ## ADO Webhooks Configured (Consumer) | Hook | Event | Scope | URL | Status | |------|-------|-------|-----|--------| | PR Answer | git.pullrequest.comment-event | Repo: , branch: | | ✓ configured | | PR Review policy | Build validation | Repo: , branch: | (pipeline trigger) | ✓ configured | WI hooks are managed by the hub project — not created here. **infra.json** updated with subscription IDs and status. **Audit log:** `.ai/logs/infra..jsonl` ### Next step `/auto-doctor` — Verify setup health ``` ## Success Criteria - [ ] Service hooks created (or confirmed existing) for each enabled agent type - [ ] No duplicate hooks — existing hooks preserved, not recreated - [ ] PR Review build validation policy attached to the base branch - [ ] Summary report lists each hook with status (created/exists/skipped) ## Examples 1. `/auto-webhooks` (hub project) — Creates 2 WI hooks (User Story + Bug, tag-filtered to `KAI-TRIGGER`) in the work-item ADO project (from scm.wiki-project config), 1 PR Answer hook scoped to the repo + base branch, and 1 PR Review build validation policy. Updates `infra.json` with subscription IDs. 2. `/auto-webhooks` (consumer project) — Skips WI hooks (managed by hub). Creates 1 PR Answer hook scoped to this repo + base branch pointing to the hub's Lambda URL, and 1 PR Review build validation policy. Reminds user to register the PR Answer pipeline ID in the hub's `ADO_PR_ANSWER_PIPELINE_MAP`. 3. `/auto-webhooks` (re-run, hooks already exist) — Lists existing service hooks via `az rest`, detects that the PR Answer hook and PR Review policy already exist for this repo. Skips creation with "already configured" status. Reports all hooks as healthy. ## Troubleshooting - **"Hook created but Lambda not receiving events"** **Cause:** The hook was created in the wrong ADO project (hooks are project-scoped) or the API Gateway URL is incorrect. **Fix:** Verify the hook's ADO project matches where the repo lives (the code ADO project (from scm.project config) for code repos). Check the hook's URL points to the correct API Gateway endpoint from `infra.json`. - **Duplicate Lambda invocations for the same event** **Cause:** Multiple hooks exist for the same event/repo combination (e.g., from a previous failed cleanup). **Fix:** List hooks with `az rest` and delete duplicates. The skill checks for existing hooks before creating, but orphaned hooks from manual creation won't be detected. - **PR Review build policy not triggering** **Cause:** The build validation policy is scoped to a specific branch and repository. PRs targeting a different branch won't trigger it. **Fix:** Verify `scm.base-branch` in `.ai/config.yaml` matches the branch your PRs target. The policy is created for `refs/heads/` on the specific repository. ## Rules - **Always source audit.sh first** — wrap `az rest` calls with `az_resource` - **Never store passwords or secrets in infra.json** — only subscription IDs and status - **Idempotent** — check `status: configured` in infra.json before creating (skip if already done) - **Check remote before create** — always list existing hooks and compare by eventType+URL+repository before creating. This catches hooks created from another repo that aren't in the local infra.json - **Work-item hooks are project-scoped (hub only)** — 2 hooks (User Story + Bug) should exist only once per project. Consumer profile skips these entirely. - **PR Answer hook is per-repo** — each repo (hub and every consumer) creates its own hook filtered to that repository + base branch. A project-scoped hook would fire on every PR comment across all repos in the ADO project, flooding the Lambda with noise from hundreds of unrelated repos. - **PR Review is per-repo** — build validation policy scoped to repositoryId+refName. Check by repositoryId before creating - **Consumer profile runs this skill** — consumers create their own PR Answer hook (pointing to hub's Lambda URL) and PR Review build policy. They do NOT create WI hooks. - **Never create duplicate hooks** — duplicate hooks cause double Lambda invocations (deduplication catches it but wastes resources) - **Tag-based routing** — all WI webhooks route to a single `/wi` endpoint. The WI Router Lambda scans work item tags against configured TAG_GATE_* env vars to determine which agent to invoke. No per-agent routes needed. - **Note about service hooks CLI:** ADO has no `az devops service-hooks` subcommand — `az rest` is the correct approach - **PR Review is a build validation policy** — not a service hook (different ADO API endpoint)