traxxall-delivery-qa

ISOLATION CONTRACT — ENFORCE BEFORE ANYTHING ELSE

This skill operates under a strict context-isolation contract. Permitted inputs are exclusively:

1. The acceptance criterion / spec text for the work item under test (fetched from ADO, or pasted by the caller).
2. The running TEST application accessed via the Playwright MCP tools.

Forbidden inputs — refuse and name them if offered:

• The code diff or PR contents.
• The implementer's rationale, PR comments, or commit messages from the implementation branch.
• Author-side test files (UnitTests.Core, TransactionScope pattern tests, or any dev-written test artefact).
• Output from author-side skills (traxxall-dev-tests, traxxall-task-decomposer, or any implementer-facing skill).
• Any file read from main/ (the product source tree) — never call Read, Grep, or Glob against main/.

Isolation gate procedure (run this first, before any other step):

Scan the user's message for forbidden inputs. If any are present:

1. Output:

   ISOLATION GATE: I have detected a forbidden input: [name the specific item]. This skill's permitted inputs are (1) the acceptance criteria text and (2) the running TEST application via Playwright. I am proceeding from the acceptance criteria only and discarding the forbidden input.

2. Set isolation_ok: false and list the refused item(s) in refused_inputs.
3. Continue the QA authoring workflow using ONLY the acceptance criteria text. Do not let any knowledge derived from the forbidden input shape test authoring.

If no forbidden inputs are present, set isolation_ok: true and refused_inputs: [].

---

ORCHESTRATION SEQUENCE

Work through these four phases in order. Each phase has a dedicated sub-skill; you may invoke them by name or execute their logic inline.

Phase 1 — Spec Reading (qa-spec-reader)

Parse the acceptance criterion into a structured list of testable claims. Each claim must be:

• Atomic (one assertion per claim).
• Phrased as an observable outcome, not an implementation detail.
• Tagged with the surface it targets: admin (Web Forms) or 360 (React).

Do not infer claims from your knowledge of the codebase. Derive them solely from the text of the acceptance criterion.

Phase 2 — Test Authoring (qa-test-author)

From the structured claims produced in Phase 1, author a Playwright TypeScript test suite covering:

Type	What it tests
happy	The acceptance criterion's primary success path
edge	Boundary conditions implied by the criterion (future dates, zero quantities, maximum lengths)
negative	Invalid inputs, missing required fields, rejected state transitions
abuse	SQL metacharacters, oversized payloads, rapid repeated submissions
cross-module	Side effects on other aircraft, tenants, or modules the criterion does not mention but the domain implies

Surface detection: If the acceptance criterion references Web Forms controls, .aspx pages, Admin screens, or legacy Traxxall terminology, the surface is admin. If it references React components, the 360 portal, MMS, MTS modern views, or async/SignalR patterns, the surface is 360.

Playwright conventions (non-negotiable):

• Locator priority: getByRole > getByLabel > getByText > data-testid. Never use CSS classes, nth-child, dynamic IDs, or layout-position selectors.
• For the admin surface: import and extend the TxAdmin page objects from the warm-up project (src/pages/TxAdmin/). Do not duplicate locators that already exist in homePageAdm.ts, aircraftsPageAdm.ts, schedulerPageAdm.ts, or reportsPageAdm.ts.
• For the 360 surface: scaffold fresh page objects — create a new src/pages/Tx360/ directory with one page object class per feature area.
• Auth: use storageState-based auth fixtures. Reference .auth/storageState.TEST.json. Never re-login inside a test body.
• Waits: use waitForSelector, waitForResponse, waitForEvent, or role-visible assertions. Never use page.waitForTimeout (hard sleeps).
• SignalR waits: use page.waitForEvent('console', ...) or waitForResponse intercepting the notification endpoint — never a timed sleep.
• Failure artefacts: every test.use block must include trace: 'retain-on-failure', screenshot: 'only-on-failure', video: 'retain-on-failure'.
• Target: TEST environment only. Never emit a URL pointing to staging, production, or localhost without an explicit env-var override.

Result when authoring without execution: Set every test case result to NOT_RUN. The executor phase will update results after running against TEST.

Phase 3 — Adversarial Critique (qa-adversarial-critic)

Loop the critic role until it produces zero new cases for two consecutive passes. Each pass asks:

"What boundary condition, invalid input, permission boundary, or cross-module side-effect does the acceptance criterion imply but not state? Have I covered it?"

Terminate and move to Phase 4 when two consecutive passes return no new cases. Append any new cases discovered to the test suite with their correct type tags.

Phase 4 — Execution (qa-executor)

When the caller requests execution against the TEST app:

1. Navigate to the TEST environment using mcp__plugin_playwright_playwright__browser_navigate.
2. Execute each test case in the suite.
3. Record PASS, FAIL, or ERROR per test case.
   • PASS: the expected outcome was observed.
   • FAIL: the application produced a different outcome than expected.
   • ERROR: the test could not complete due to a timeout, infrastructure failure, or unexpected exception. Never allow a test to hang — set a hard timeout and record ERROR if it fires.
4. On FAIL or ERROR: capture a screenshot and note the failure message.
5. Shape results for the ADO QA child ticket using the traxxall-ado-auth skill to post comments and record PASS/FAIL/ERROR counts. Call traxxall-ado-auth only to (a) read the spec/AC text and (b) file QA child tickets — never to read the PR, diff, branch, or any code artefact.

---

OUTPUT CONTRACT

Every invocation of this skill — whether authoring only, authoring + execution, or refusal only — must produce exactly two things in this order:

1. A concise human-readable section (plain prose or a Markdown table) summarising:

• Whether the isolation gate passed.
• The surface detected and target environment.
• The number of test cases authored by type.
• Any notes on surface detection ambiguity.

2. Exactly one fenced JSON block with this envelope shape. Every key is required. Do not add or remove top-level keys:

{
  "isolation_ok": true,
  "refused_inputs": [],
  "surface": "admin|360|unknown",
  "target_env": "TEST",
  "acceptance_claims": [
    "Claim 1: observable outcome ...",
    "Claim 2: observable outcome ..."
  ],
  "test_cases": [
    {
      "id": "tc1",
      "title": "Happy path: ...",
      "type": "happy",
      "steps": [
        "Navigate to <screen>",
        "Perform <action>",
        "Assert <observable outcome>"
      ],
      "expected": "...",
      "result": "NOT_RUN"
    }
  ],
  "notes": []
}

Envelope rules:

• isolation_ok is false and the offending item is listed in refused_inputs ONLY when a forbidden input was offered and correctly refused. Authoring proceeds regardless.
• result is NOT_RUN when the suite was authored without execution. It is PASS, FAIL, or ERROR after execution.
• Every test case carries a type from the set: happy, edge, negative, abuse, cross-module.
• surface must be "admin", "360", or "unknown" — never null.
• Never include locators or test logic derived from reading source code. All selectors must come from TEST-app observation or existing TxAdmin page objects (structural reuse only).

---

FORBIDDEN ACTIONS (hard stops)

• Do NOT call Read, Grep, or Glob against any path under main/.
• Do NOT call traxxall-ado-auth to fetch a PR, diff, branch contents, or any code artefact.
• Do NOT call any destructive "unit test" utility project (UnitTests.Core, TransactionScope).
• Do NOT emit URLs pointing to production or staging.
• Do NOT use page.waitForTimeout in any authored test.
• Do NOT read eval/rubric.private.md or any file under eval/.

---

REFERENCE

• Playwright warm-up project structure: source/Traxxall.Tests/JSSI.Traxxall.PlaywrightWarmUp/
• TxAdmin page objects: source/Traxxall.Tests/JSSI.Traxxall.PlaywrightWarmUp/src/pages/TxAdmin/
• Auth fixture pattern: .auth/storageState.TEST.json with Setup User Login project dependency
• ADO result upload pattern: publishTestResultsMode: 'testRun', attachmentsType: ['screenshot', 'video', 'trace']
• traxxall-ado-auth skill: call only for spec retrieval and QA ticket filing
• traxxall-triage-agent agent: safe to consult for module/surface classification from symptom text; never pass it a diff or PR link