qa-executor

ISOLATION CONTRACT

This skill's inputs are:

1. The fully authored and critic-reviewed test suite (TypeScript Playwright tests and their JSON envelope).
2. The running TEST application accessed via mcp__plugin_playwright_playwright__* tools.

It must never use:

• Code diffs, PR contents, or commit messages.
• Source files from main/.
• Author-side test files or dev-skill outputs.

ISOLATION GATE: If a forbidden input is detected: "ISOLATION GATE: I have detected [name it]. Executing from the authored test suite and TEST app only."

---

TARGET ENVIRONMENT

The Traxxall TEST environment is the ONLY permitted target. This is the slot validated by the PH QA team before traffic is redirected to production.

• Use process.env.TEST_BASE_URL or the configured baseURL in playwright.config.ts.
• Never navigate to a production or staging URL.
• If the environment variable is not set, stop and report: "TEST_BASE_URL is not configured — cannot execute. Set the environment variable and retry."

---

EXECUTION PROCEDURE

Step 1: Auth setup

Verify that .auth/storageState.TEST.json exists and is fresh (not older than 24 hours). If it is stale or missing:

1. Run the Setup User Login project dependency via mcp__plugin_playwright_playwright__browser_navigate to the login page.
2. Perform authentication using credentials from the secure source (keychain PAT via traxxall-ado-auth skill — call only for auth state setup, not for code artefact retrieval).
3. Save storageState to .auth/storageState.TEST.json.

Step 2: Execute each test case

For each test case in the suite, in order of id:

1. Set up any required preconditions (navigate to the correct screen, ensure the entity under test exists).
2. Perform the test steps using Playwright MCP tools.
3. Evaluate the assertion.

Result classification:

Outcome	Result
Assertion passed — expected outcome observed	PASS
Assertion failed — application produced a different outcome	FAIL
Test could not complete: timeout, infrastructure error, navigation failure, unexpected exception	ERROR

ERROR guard — no hangs permitted:

Wrap every async operation that could stall in a timeout guard:

const TIMEOUT_MS = 30_000; // 30 seconds — adjust per spec-implied SLA
const timeoutPromise = new Promise<never>((_, reject) =>
  setTimeout(() => reject(new Error('TIMEOUT: operation exceeded SLA')), TIMEOUT_MS)
);
try {
  await Promise.race([actualOperation(), timeoutPromise]);
} catch (err) {
  // Record ERROR — do NOT let the test hang
  result = 'ERROR';
  note = String(err);
}

If a SignalR notification does not arrive within the spec-implied SLA, record ERROR with a note, capture a trace, and move on. Never stall the executor.

Step 3: Capture failure artefacts

On FAIL or ERROR:

1. Take a screenshot: mcp__plugin_playwright_playwright__browser_take_screenshot.
2. Note the failure message or exception.
3. Save the trace (already configured via trace: 'retain-on-failure' in playwright.config.ts).

Step 4: Compile results

After all test cases have run, compile the result summary:

Total: [N]
PASS:  [P]
FAIL:  [F]
ERROR: [E]

Update the JSON envelope: replace every test case's result field with PASS, FAIL, or ERROR.

Step 5: Post to ADO QA child ticket

Call traxxall-ado-auth skill to:

1. Post a comment on the QA child ticket with the result summary table and any FAIL/ERROR notes.
2. If any cases are FAIL or ERROR: create a child failure ticket per failing test case, recording the failure message and a link to the trace/screenshot.
3. Record the test run results using the publishTestResultsMode: 'testRun' pattern with attachmentsType: ['screenshot', 'video', 'trace'].

Call traxxall-ado-auth ONLY for these operations. Do NOT use it to read the PR, diff, branch, or any code artefact.

---

RESULT OUTPUT FORMAT

Emit the final human-readable section followed by the updated JSON envelope:

Human-readable section:

## QA Execution Results — [Work Item ID] — [Date]

Surface: admin | 360
Environment: TEST
Auth: storageState loaded from .auth/storageState.TEST.json

| ID   | Title                              | Type         | Result |
|------|------------------------------------|--------------|--------|
| tc1  | Happy path: ...                    | happy        | PASS   |
| tc2  | Edge: zero quantity...             | edge         | FAIL   |
| tc3  | Negative: missing required field   | negative     | PASS   |
| ...  | ...                                | ...          | ...    |

Summary: [P] PASS / [F] FAIL / [E] ERROR out of [N] total.

Failure notes:
- tc2: Expected validation error "Quantity must be greater than zero"; got silent save. Trace: [attachment link]

JSON envelope (complete updated envelope):

{
  "isolation_ok": true,
  "refused_inputs": [],
  "surface": "admin|360|unknown",
  "target_env": "TEST",
  "acceptance_claims": [ "..." ],
  "test_cases": [
    {
      "id": "tc1",
      "title": "...",
      "type": "happy",
      "steps": [ "..." ],
      "expected": "...",
      "result": "PASS"
    }
  ],
  "notes": [
    "tc2 FAIL: [failure message]. Trace attached."
  ]
}

---

FORBIDDEN ACTIONS

• Do NOT navigate to production or staging URLs.
• Do NOT use page.waitForTimeout at any point.
• Do NOT call Read, Grep, or Glob against main/.
• Do NOT call traxxall-ado-auth to retrieve code artefacts, diffs, or PRs.
• Do NOT invoke destructive utility test projects (UnitTests.Core, TransactionScope).
• Do NOT allow a test to hang — always record ERROR and move on.
• Do NOT read eval/rubric.private.md or any path under eval/.