From dx-automation
Configure ADO service hooks and PR Review build validation policy for AI automation agents. Uses ADO REST API via az rest (no native CLI command for service hooks). Reads all config from .ai/automation/infra.json.
npx claudepluginhub easingthemes/dx-aem-flow --plugin dx-automationThis skill uses the workspace's default tool permissions.
You configure ADO service hooks and the PR Review build validation policy. ADO has no native `az devops service-hooks` CLI command — all REST calls use `az rest` (ADO PAT auth, audit-wrapped).
Manages Azure DevOps projects, work items, repos, PRs, pipelines, wikis, test plans, security alerts, variable groups, environments, branch policies via Python CLI and REST API.
Discovers, triggers, monitors, and auto-fixes Azure DevOps pipelines (PR, Buddy Build, Release) for current repo/branch. Diagnoses failures from logs, applies fixes, commits/pushes, re-triggers until success or max retries. Validates PR existence and description.
Monitors PR health on recurring schedules: merge conflicts, CI/CD failures in GitHub Actions/Buildkite/Vercel/Fly.io, review comment triage/resolution, merge readiness. One-shot triage mode.
Share bugs, ideas, or general feedback.
You configure ADO service hooks and the PR Review build validation policy. ADO has no native az devops service-hooks CLI command — all REST calls use az rest (ADO PAT auth, audit-wrapped).
Read .ai/automation/infra.json. Extract automationProfile.
source .ai/lib/audit.sh
export AUDIT_LOG_PREFIX=infra
Profile determines scope:
Extract:
adoOrg — ADO org URLadoProject — ADO project namepipelines.pr-review.id — PR Review pipeline ID (for build validation policy)pipelines.pr-answer.id — PR Answer pipeline ID (for registering with hub)webhooks.wi-userstory.url, webhooks.wi-bug.urlwebhooks.*.status is already configured — skip if soFor consumers: The PR Answer hook URL points to the hub's PR Router Lambda. Ask:
Hub's PR Router Lambda URL? The API Gateway URL from the hub project's infra.json (e.g.,
https://<id>.execute-api.us-east-1.amazonaws.com/prod/pr-answer).
Ask once for shared values needed for hooks:
Webhook username? (same as Lambda
BASIC_USERset in hub's/auto-lambda-env)
Webhook password? (same as Lambda
BASIC_PASS) — secret, not stored
Webhook secret? (same as Lambda
WEBHOOK_SECRET) — secret, not stored
Ask for ADO project ID (required by service hook API):
ADO project ID? Find in ADO > Project Settings > Overview > Project ID (GUID format). You can also run:
az devops project show --project "<adoProject>" --query id --output tsv
Get the repository ID for repo-scoped hooks:
REPO_ID=$(az repos show --repository "<repo-name>" --project "<adoProject>" --query id --output tsv)
Read scm.base-branch from .ai/config.yaml for branch filtering.
Before creating any hooks, list all existing subscriptions to avoid duplicates:
EXISTING_HOOKS=$(az rest --method GET \
--uri "<adoOrg>/_apis/hooks/subscriptions?api-version=7.1" \
--query "value[?consumerInputs.url != null].{id:id, event:eventType, url:consumerInputs.url, repoId:publisherInputs.repository}" \
--output json)
For each hook to create, check if a subscription already exists with the same eventType AND the same url AND (for repo-scoped hooks) the same repository:
⏭ <hook> already exists (ID: <id>) — skippingWork-item hooks are project-scoped and should exist only ONCE per project (hub only). If already configured from another repo, skip.
PR Answer hooks are per-repo — each repo needs its own hook filtered to that repo + base branch. Without repo filtering, a single project-scoped hook fires on every PR comment across hundreds of unrelated repos, flooding the Lambda with noise. Migration: If an existing project-scoped PR Answer hook exists (no repository filter — from an older setup), warn the user and offer to delete it after creating the repo-scoped replacement.
PR Review build validation policy is per-repo — always check by repositoryId before creating.
Skip for consumer profile.
Routes all User Story workitem.updated events to the WI Router Lambda at /wi. The Lambda uses tag-based routing to determine which agent to invoke (DoR, DoD, QA, DevAgent, DOCAgent).
az_resource "ado/hooks/wi-userstory" \
az rest --method POST \
--uri "$(adoOrg)/_apis/hooks/subscriptions?api-version=7.1" \
--headers "Content-Type=application/json" \
--body "{
\"publisherId\": \"tfs\",
\"eventType\": \"workitem.updated\",
\"resourceVersion\": \"1.0\",
\"consumerId\": \"webHooks\",
\"consumerActionId\": \"httpRequest\",
\"publisherInputs\": {
\"projectId\": \"<PROJECT_ID>\",
\"workItemType\": \"User Story\",
\"tag\": \"KAI-TRIGGER\"
},
\"consumerInputs\": {
\"url\": \"<wi-userstory-url>\",
\"basicAuthUsername\": \"<BASIC_USER>\",
\"basicAuthPassword\": \"<BASIC_PASS>\",
\"httpHeaders\": \"x-webhook-secret:<WEBHOOK_SECRET>\"
}
}" \
--query 'id' --output tsv
Update infra.json:
webhooks.wi-userstory.subscriptionId → returned IDwebhooks.wi-userstory.status → "configured"Skip for consumer profile.
Routes all Bug workitem.updated events to the same WI Router Lambda at /wi. The Lambda uses tag-based routing to determine the agent (BugFix).
az_resource "ado/hooks/wi-bug" \
az rest --method POST \
--uri "<adoOrg>/_apis/hooks/subscriptions?api-version=7.1" \
--headers "Content-Type=application/json" \
--body "{
\"publisherId\": \"tfs\",
\"eventType\": \"workitem.updated\",
\"resourceVersion\": \"1.0\",
\"consumerId\": \"webHooks\",
\"consumerActionId\": \"httpRequest\",
\"publisherInputs\": {
\"projectId\": \"<PROJECT_ID>\",
\"workItemType\": \"Bug\",
\"tag\": \"KAI-TRIGGER\"
},
\"consumerInputs\": {
\"url\": \"<wi-bug-url>\",
\"basicAuthUsername\": \"<BASIC_USER>\",
\"basicAuthPassword\": \"<BASIC_PASS>\",
\"httpHeaders\": \"x-webhook-secret:<WEBHOOK_SECRET>\"
}
}" \
--query 'id' --output tsv
Update infra.json:
webhooks.wi-bug.subscriptionId → returned IDwebhooks.wi-bug.status → "configured"Routes to the PR Router Lambda. This hook is repo-scoped — each repo (hub and every consumer) creates its own hook filtered to that repo and base branch. Without repo filtering, a project-scoped hook fires on every PR comment across all repos in the ADO project.
The PR Router Lambda uses ADO_PR_ANSWER_PIPELINE_MAP to route each repo's events to the correct PR Answer pipeline.
az_resource "ado/hooks/pr-answer" \
az rest --method POST \
--uri "<adoOrg>/_apis/hooks/subscriptions?api-version=7.1" \
--headers "Content-Type=application/json" \
--body "{
\"publisherId\": \"tfs\",
\"eventType\": \"ms.vss-code.git-pullrequest-comment-event\",
\"resourceVersion\": \"1.0\",
\"consumerId\": \"webHooks\",
\"consumerActionId\": \"httpRequest\",
\"publisherInputs\": {
\"projectId\": \"<PROJECT_ID>\",
\"repository\": \"<REPO_ID>\",
\"branch\": \"<BASE_BRANCH>\"
},
\"consumerInputs\": {
\"url\": \"<pr-answer-url>\",
\"basicAuthUsername\": \"<BASIC_USER>\",
\"basicAuthPassword\": \"<BASIC_PASS>\",
\"httpHeaders\": \"x-webhook-secret:<WEBHOOK_SECRET>\"
}
}" \
--query 'id' --output tsv
<pr-answer-url> — for hub: from webhooks.pr-answer.url in infra.json. For consumer: the hub's PR Router Lambda URL (asked in step 0).<REPO_ID> — this repo's ADO repository GUID (from step 0).<BASE_BRANCH> — from scm.base-branch in config.yaml (e.g., development).Update infra.json:
webhooks.pr-answer.subscriptionId → returned IDwebhooks.pr-answer.status → "configured"PR Review is not a service hook — it's a build validation policy on the target branch. This triggers the PR Review pipeline on every PR against that branch.
Use REPO_ID from step 0 and BASE_BRANCH from config.yaml (already read). No need to ask again — default to scm.base-branch. Only ask if not set in config:
Target branch for PR Review? (e.g.
main,develop) — pull requests targeting this branch will trigger the PR Review agent.
REPO_ID="<repository-id>"
BRANCH="<base-branch>"
PR_REVIEW_PIPELINE_ID=$(python3 -c "import json; print(json.load(open('.ai/automation/infra.json'))['pipelines']['pr-review']['id'])")
az_resource "ado/policy/pr-review-build-validation" \
az rest --method POST \
--uri "<adoOrg>/<adoProject>/_apis/policy/configurations?api-version=7.1" \
--headers "Content-Type=application/json" \
--body "{
\"isEnabled\": true,
\"isBlocking\": false,
\"type\": {\"id\": \"0609b952-1397-4640-95ec-e00a01b2f659\"},
\"settings\": {
\"buildDefinitionId\": $PR_REVIEW_PIPELINE_ID,
\"queueOnSourceUpdateOnly\": true,
\"manualQueueOnly\": false,
\"displayName\": \"AI PR Review\",
\"validDuration\": 720,
\"scope\": [{
\"repositoryId\": \"$REPO_ID\",
\"refName\": \"refs/heads/$BRANCH\",
\"matchKind\": \"Exact\"
}]
}
}" \
--query 'id' --output tsv
Update infra.json:
webhooks.pr-review.policyId → returned IDwebhooks.pr-review.status → "configured"Adapt the report to the profile:
## ADO Webhooks Configured (Hub)
| Hook | Event | Scope | URL | Status |
|------|-------|-------|-----|--------|
| WI User Story | workitem.updated | Project (tag: KAI-TRIGGER) | <wi-url> | ✓ configured |
| WI Bug | workitem.updated | Project (tag: KAI-TRIGGER) | <wi-url> | ✓ configured |
| PR Answer | git.pullrequest.comment-event | Repo: <repo>, branch: <branch> | <pr-answer-url> | ✓ configured |
| PR Review policy | Build validation | Repo: <repo>, branch: <branch> | (pipeline trigger) | ✓ configured |
**Tag-based routing:** The WI Router Lambda uses work item tags to determine which agent pipeline to queue. Adding a new agent only requires Lambda env vars (TAG_GATE_* + pipeline ID) — no new hook or route needed.
**infra.json** updated with subscription IDs and status.
**Audit log:** `.ai/logs/infra.<week>.jsonl`
### Verify
Test the WI hook: Go to ADO > Project Settings > Service Hooks > find "WI User Story" > click "Test" to send a sample event.
### Next step
`/auto-alarms` — Set up CloudWatch monitoring
## ADO Webhooks Configured (Consumer)
| Hook | Event | Scope | URL | Status |
|------|-------|-------|-----|--------|
| PR Answer | git.pullrequest.comment-event | Repo: <repo>, branch: <branch> | <hub-pr-answer-url> | ✓ configured |
| PR Review policy | Build validation | Repo: <repo>, branch: <branch> | (pipeline trigger) | ✓ configured |
WI hooks are managed by the hub project — not created here.
**infra.json** updated with subscription IDs and status.
**Audit log:** `.ai/logs/infra.<week>.jsonl`
### Next step
`/auto-doctor` — Verify setup health
/auto-webhooks (hub project) — Creates 2 WI hooks (User Story + Bug, tag-filtered to KAI-TRIGGER) in the work-item ADO project (from scm.wiki-project config), 1 PR Answer hook scoped to the repo + base branch, and 1 PR Review build validation policy. Updates infra.json with subscription IDs.
/auto-webhooks (consumer project) — Skips WI hooks (managed by hub). Creates 1 PR Answer hook scoped to this repo + base branch pointing to the hub's Lambda URL, and 1 PR Review build validation policy. Reminds user to register the PR Answer pipeline ID in the hub's ADO_PR_ANSWER_PIPELINE_MAP.
/auto-webhooks (re-run, hooks already exist) — Lists existing service hooks via az rest, detects that the PR Answer hook and PR Review policy already exist for this repo. Skips creation with "already configured" status. Reports all hooks as healthy.
"Hook created but Lambda not receiving events"
Cause: The hook was created in the wrong ADO project (hooks are project-scoped) or the API Gateway URL is incorrect.
Fix: Verify the hook's ADO project matches where the repo lives (the code ADO project (from scm.project config) for code repos). Check the hook's URL points to the correct API Gateway endpoint from infra.json.
Duplicate Lambda invocations for the same event
Cause: Multiple hooks exist for the same event/repo combination (e.g., from a previous failed cleanup).
Fix: List hooks with az rest and delete duplicates. The skill checks for existing hooks before creating, but orphaned hooks from manual creation won't be detected.
PR Review build policy not triggering
Cause: The build validation policy is scoped to a specific branch and repository. PRs targeting a different branch won't trigger it.
Fix: Verify scm.base-branch in .ai/config.yaml matches the branch your PRs target. The policy is created for refs/heads/<base-branch> on the specific repository.
az rest calls with az_resourcestatus: configured in infra.json before creating (skip if already done)/wi endpoint. The WI Router Lambda scans work item tags against configured TAG_GATE_* env vars to determine which agent to invoke. No per-agent routes needed.az devops service-hooks subcommand — az rest is the correct approach