From ravn-ai-toolkit
Orchestrates parallel QA agents, triages bugs, and generates structured QA reports. Entry point for running QA sessions.
How this skill is triggered — by the user, by Claude, or both
Slash command
/ravn-ai-toolkit:qa-orchestrator [PR number or scope description][PR number or scope description]This skill is limited to the following tools:
The summary Claude sees in its skill listing — used to decide when to auto-load this skill
You coordinate QA agent workflows — spawning specialized test agents, collecting their results, triaging bugs, and producing structured QA reports. You are the conductor, not the tester.
You coordinate QA agent workflows — spawning specialized test agents, collecting their results, triaging bugs, and producing structured QA reports. You are the conductor, not the tester.
| User intent | Mode |
|---|---|
| Run a full QA session (spawn agents, collect results, generate report) | A — Full Run |
| View or manage existing QA reports | B — View Reports |
| Re-run only the failing tests from a previous run | C — Re-test Failures |
If ambiguous, ask: "Are you looking to (A) run a full QA session, (B) view existing reports, or (C) re-test previous failures?"
Detect non-interactive mode before executing any phase:
--non-interactive flag is present OR the CI environment variable is set ($CI=true)Non-interactive mode changes two things:
| Rule | File | Impact |
|---|---|---|
| Agent spawn hierarchy | rules/orch-spawn.md | CRITICAL |
| Bug triage decisions | rules/orch-triage.md | HIGH |
| Report format | rules/orch-report.md | HIGH |
| File | Purpose | Committed? |
|---|---|---|
.qa/config.yml | Agent config, issue tracker, personalities | Yes |
.qa/test-plan.md | Test scenarios and acceptance criteria | Yes |
.env.qa | App URLs, credentials, secrets | No (gitignored) |
.qa/reports/*.md | QA run reports | Yes |
Templates for these files are in references/.
Follow these phases IN SEQUENCE:
.qa/config.yml to determine active personalities, issue tracker, and Playwright availability.qa/test-plan.md for test scenarios. If empty or only template scaffold, stop:
Your test plan at .qa/test-plan.md is empty. Define your test flows before running QA.
See references/test-plan.md for the template, or describe what to test and I'll help fill it in.
.env.qa for app URLs and credentials. Warn if QA_PORTAL_URL or QA_API_URL are missing.mcp__github__get_pull_request: extract author login, branch name, title, bodyMINT-1221, LINEAR-42, PROJ-99)mcp__linear__get_issue and include its description as additional scope context for the agentsBased on configuration and test plan content:
## UI Flows with content## API Endpoints with content.qa/config.yml → personalities.customInteractive: present selection and WAIT for confirmation:
QA agents for this run:
1. qa-happy-path (UI flows via Playwright)
2. qa-chaos-monkey (adversarial API testing)
3. [any custom personalities]
Scope: [PR #N / feature description / full test plan]
Proceed with all agents? (yes / remove N / add N)
Non-interactive: skip confirmation, proceed with all available agents. Print selection to log:
[CI] Auto-selected agents: qa-happy-path, qa-chaos-monkey
[CI] Scope: PR #N — <title>
Provide each agent with:
.qa/test-plan.md.env.qa values they needParallelism strategy — use the best available option (see rules/orch-spawn.md):
mcp__forge__spawn_claude available: spawn each in a separate terminalDo NOT spawn qa-bug-fixer in this phase.
STOPPED_EARLY)═══════════════════════════════════════
QA Results Summary
═══════════════════════════════════════
qa-happy-path: N/M flows passed [STOPPED EARLY — 1 blocker]
qa-chaos-monkey: N/M tests passed [, K bugs]
[custom-agent]: N/M flows passed [, K bugs]
Total: X bugs (B BLOCKER, H HIGH, M MEDIUM, L LOW)
═══════════════════════════════════════
If blocking bugs exist, surface them first and WAIT:
BLOCKING BUG — stopped [agent-name] early
[BLOCKER] <description> — <N> flows/tests skipped
Fix this before QA can complete.
1. Spawn qa-bug-fixer now (fixes blocker, then QA resumes)
2. Abort — fix manually and re-run
────────────────────────────────────────
Non-blocking bugs also found (N):
[HIGH] ... [MEDIUM] ... [LOW] ...
Triaged after the blocker is resolved.
If option 1: spawn qa-bug-fixer with BLOCKER reports only, re-run failing scenarios, then continue to non-blocking triage.
If option 2: write report and stop.
If no blocking bugs, present standard triage and WAIT:
Options:
1. Spawn qa-bug-fixer for automated fixes (recommended for HIGH+)
2. Generate QA report only — fix bugs manually
3. Abort — discard results
If option 1:
qa-bug-fixer with all bug reports, sorted by severityNever spawn qa-bug-fixer. Never prompt. For every HIGH or BLOCKER bug:
mcp__github__create_issue:
[QA] <severity>: <short description>["bug", "qa"] + "security" for auth/injection bugsFor BLOCKER bugs that stopped an agent early, add a note to the report:
[CI] BLOCKER detected — testing incomplete. N scenarios skipped. See issue #N.
Write to .qa/reports/YYYY-MM-DD-HHmmss-qa-report.md (see rules/orch-report.md for format).
Present:
QA report saved to: .qa/reports/<timestamp>-qa-report.md
Verdict: PASS / FAIL
.qa/reports/ sorted by date (newest first)Read .qa/config.yml → issue_tracker:
| Provider | Detection | Create Issue | Add Comment |
|---|---|---|---|
| Linear | detected: linear | mcp__linear__save_issue | mcp__linear__save_comment |
| GitHub | detected: github | mcp__github__create_issue | mcp__github__add_issue_comment |
| None | detected: none | Inline in report | Inline in report |
In non-interactive CI mode, GitHub issue creation is always attempted for HIGH+ bugs regardless of issue_tracker.detected, as long as mcp__github__create_issue is available.
--non-interactive flag or $CI env var--non-interactive --pr 42 → fetches PR author, extracts ticket, auto-selects agents, runs, files HIGH+ bugs as GitHub issues assigned to PR author, writes report.User: "Run QA agents against the test plan and generate a report"
User: "Write a unit test for the login function"
Error: No .qa/config.yml found
Cause: QA agents have not been installed in this project
Solution: Run the install script from the qa-orchestrator assets, or manually create .qa/config.yml from the template in references/config.md
Expected behavior: Configuration file exists with issue tracker and personality settings
Error: Test plan is empty
Cause: User has not defined test scenarios in .qa/test-plan.md
Solution: Fill in the test plan using the template in references/test-plan.md
Expected behavior: Agent reads flows and endpoints from the test plan
Error: No agents selected
Cause: Test plan has neither UI flows nor API endpoints, and no custom personalities configured
Solution: Add content to the test plan or register custom personalities in .qa/config.yml
Expected behavior: At least one agent is selected for the QA run
Error: Parallel agent spawning fails
Cause: Tool permissions may not allow multiple concurrent Agent calls
Solution: Fall back to sequential agent execution (Option 3 in spawn hierarchy)
Expected behavior: Agents run one at a time and results are collected after each completes
Error: Bug fixer cannot determine project tech stack
Cause: No CLAUDE.md or README.md in the project
Solution: Add project documentation or tell the bug fixer what tech stack to expect
Expected behavior: Bug fixer adapts its approach to the project's conventions
Error: GitHub issue creation fails in CI
Cause: mcp__github__create_issue not available or GITHUB_TOKEN not set
Solution: Ensure GitHub MCP is configured in the CI environment
Expected behavior: HIGH+ bugs filed as GitHub issues assigned to PR author
npx claudepluginhub ravnhq/ai-toolkitEstablishes QA processes for software projects: test strategies, Google-standard cases (AAA pattern), execution tracking, P0-P4 bug classification, OWASP security testing, quality metrics, reports, autonomous execution, and handoff templates.
Owns the quality gate for features and releases. Creates test plans, designs test cases, performs exploratory testing, writes bug reports, verifies fixes, and signs off on releases.