test-plan-create-cases

Test Case Generator

Generate individual test case specification files from an existing test plan.

Usage

/test-plan-create-cases [FEATURE_DIR]

Examples:

• /test-plan-create-cases (auto-detects from prior /test-plan-create run)
• /test-plan-create-cases mcp_catalog
• /test-plan-create-cases /path/to/feature_dir

Inputs

From arguments

Parse $ARGUMENTS to extract:

1. First argument (optional): Feature source - can be:
   • Local directory path: mcp_catalog or /path/to/mcp_catalog
   • GitHub branch: https://github.com/org/repo/tree/test-plan/RHAISTRAT-400
   • GitHub PR: https://github.com/org/repo/pull/5
2. --output-dir (optional): Force creation in specified directory (contributor override, skips validation)

Auto-detection from session

If no arguments are provided, check for session context from /test-plan-create:

# Check if TEST_PLAN_OUTPUT_DIR environment variable is set
if [ -n "$TEST_PLAN_OUTPUT_DIR" ]; then
    # /test-plan-create was just run in this session
    feature_dir="$TEST_PLAN_OUTPUT_DIR/<feature_name>"
    echo "✓ Auto-detected from /test-plan-create session: $feature_dir"
    # Proceed directly to Step 1 (skip Step 0.2)
fi

Interactive fallback

If no arguments AND no session context, ask the user via AskUserQuestion:

Where is the TestPlan.md located?

You can provide:

• Local directory path (e.g., ~/Code/collection-tests/mcp_catalog)
• GitHub branch URL (e.g., https://github.com/org/repo/tree/test-plan/RHAISTRAT-400)
• GitHub PR URL (e.g., https://github.com/org/repo/pull/5)

Process

Step 0: Pre-flight Check

0.1 Python dependencies

Install the test-plan package (makes all scripts importable):

(cd $(git -C ${CLAUDE_SKILL_DIR} rev-parse --show-toplevel) && uv sync --extra dev)

If installation fails, inform the user and do NOT proceed. Once installed, all Python scripts will work from any directory.

0.2 Locate Feature Directory

Skip this step if session context was found (see "Auto-detection from session" above).

1. Use the shared locate-feature-dir utility:

   result=$(cd $(git -C ${CLAUDE_SKILL_DIR} rev-parse --show-toplevel) && uv run python scripts/repo.py locate-feature-dir "<source>")
   if [ $? -ne 0 ]; then
       echo "$result"
       exit 1
   fi
   
   # Parse JSON output
   feature_dir=$(echo "$result" | jq -r '.feature_dir')
   source_type=$(echo "$result" | jq -r '.source_type')
   

2. Validate local paths against skill repository (unless --output-dir flag was used):

   if [ "$source_type" = "local" ]; then
       # Check for --output-dir flag (contributor override)
       FORCE_OUTPUT_DIR="${FORCE_OUTPUT_DIR:-false}"
       
       # Validate against skill repository
       export CLAUDE_SKILL_DIR
       force_flag=$([ "$FORCE_OUTPUT_DIR" = "true" ] && echo "--force" || echo "")
       (cd $(git -C ${CLAUDE_SKILL_DIR} rev-parse --show-toplevel) && uv run python scripts/repo.py validate-local-path "$feature_dir" $force_flag) || exit 1
   fi
   

Note: GitHub sources are always external repos, so no skill repo validation needed.

Step 1: Read the Test Plan

1. Read <feature_dir>/TestPlan.md using the Read tool
2. Extract the source_key from the YAML frontmatter — this will be used in Step 3.1 to set frontmatter on each test case file
3. Extract:
   • Section 4 (Endpoints/Methods Under Test) — the full list of what needs test coverage
   • Section 2 (Test Strategy) — test levels, types, priorities to guide test case depth
   • Section 3 (Test Environment) — preconditions and test data requirements
   • Section 5.2 (Test Case Naming Convention) — the TC-<CATEGORY>-<NUMBER> prefixes and their meanings
   • Section 1.2 (Scope) — in-scope vs out-of-scope boundaries
   • Section 1.3 (Test Objectives) — what the tests should validate

Step 1.5: Read Gaps (if available)

1. Check if <feature_dir>/TestPlanGaps.md exists (generated by /test-plan-create)
2. If it exists, read it to understand known limitations — do NOT create test cases for areas marked as pending or missing details
3. If it does not exist, proceed normally

Step 2: Read the Test Case Template

1. Read the template from ${CLAUDE_SKILL_DIR}/test-case-template.md using the Read tool
2. Follow this template structure for every generated test case
3. Omit optional sections (Preconditions, Test Data, Expected Response, Validation) when they are empty or not applicable — do not include empty sections
4. Always leave Automation Status and Notes as placeholders — they are filled later in the process

Step 2.5: Detect Regeneration Mode

1. Check for existing test cases:

   regen_check=$(cd $(git -C ${CLAUDE_SKILL_DIR} rev-parse --show-toplevel) && uv run python scripts/tc_regeneration.py check <feature_dir>)
   mode=$(echo "$regen_check" | jq -r '.mode')
   existing_count=$(echo "$regen_check" | jq -r '.existing_count')
   

2. If mode = "regenerate" (existing test cases found):

   a. Read all existing TC files using Read tool (satisfies Write tool requirement):

   echo "$regen_check" | jq -r '.files[]' | while read file; do
       # Read each existing TC file
   done
   

   b. Ask for confirmation via AskUserQuestion:

   Regeneration Mode

   Found <existing_count> existing test cases in test_cases/.

   Regenerating will overwrite all existing test cases. You can review changes via git diff before publishing.

   Proceed with regeneration? [yes/no]

   c. If no: Exit gracefully

   d. If yes: Continue to Step 3 with REGENERATION_MODE=true

3. If mode = "create" (no existing test cases):

   • Continue to Step 3 with REGENERATION_MODE=false

Step 3: Design and Generate Test Cases

Process one category at a time from Section 5.2. For each category:

1. Design all test cases for that category:

   • Cover every endpoint/method from Section 4 relevant to this category
   • Include positive, negative, and boundary scenarios (per Section 2.2)
   • Assign priorities (P0/P1/P2) following the criteria in Section 2.3
   • Stay strictly within the scope defined in Section 1.2 — do NOT create test cases for out-of-scope items
   • Map back to test objectives from Section 1.3
   • Check against previously generated categories to avoid duplicating coverage

2. Write or Edit the TC-<CATEGORY>-<NUMBER>.md files for that category immediately before moving to the next:

   • If REGENERATION_MODE=true: Use Edit tool for files that already exist (preserves git history), Write tool for new files
   • If REGENERATION_MODE=false: Use Write tool for all files

   Include YAML frontmatter at the top of each file:

   ---
   test_case_id: TC-<CATEGORY>-<NUMBER>
   source_key: <STRAT_KEY_FROM_TEST_PLAN>
   priority: <P0|P1|P2>
   status: Draft
   automation_status: Not Started
   last_updated: "<today_date>"
   # upgrade_phase: pre|post|both   # see Step 3.4 — set for ANY TC whose expected results differ between upgrade states
   ---
   

   • source_key: use the value extracted from the test plan's frontmatter in Step 1
   • last_updated: MUST be quoted string (e.g., "2026-05-04"), not unquoted date
   • If the test plan's Section 7.2 is non-trivial, evaluate upgrade_phase for every TC before finalising its frontmatter — including TC-UI-, TC-E2E-, and all other categories, not just TC-UPGRADE-*. The question is always the same: does this TC's expected behaviour differ between the old and new version? If yes, set the phase. Do not skip this evaluation for any TC.
   • Write the frontmatter directly — validation happens in Step 5.7
   • Important: In regeneration mode, files were already read in Step 2.5, so Edit/Write will work correctly

3. E2E test cases (mandatory): After processing all categories, generate TC-E2E-*.md test cases that validate the user journeys defined in the strategy:

   • Every P0 endpoint from Section 4 MUST be covered by at least one E2E scenario
   • Each E2E test case should represent a complete user journey, not just a single endpoint call
   • Use TC-E2E-<NUMBER> naming convention (e.g., TC-E2E-001, TC-E2E-002)

4. Upgrade test cases (conditional): Read Section 7.2 (Upgrade/Migration) of the TestPlan.md. If Section 7.2 describes meaningful upgrade-specific behaviour (not just "Not Applicable" or a single sentence disclaimer), generate upgrade-aware TCs:

   First, identify what kind of upgrade scenario this is — it determines the dominant phase:

   • Feature introducing an upgrade change (new API, new route, new auth model): primarily post TCs for new behaviour, pre TCs for state that disappears after upgrade, both for regressions
   • Bug discovered during upgrade (something that worked before upgrade now breaks): primarily both TCs — the goal is to establish a PASS baseline before upgrade and detect a REGRESSION after

   Phase values and when to use them:

   • upgrade_phase: pre — behaviour or state that only exists on the old version. Expected to FAIL or be N/A on the new version. Preconditions must state the source version.
   • upgrade_phase: post — behaviour that only exists after upgrade (new feature, new resource, new route). Expected to FAIL on the old version. Preconditions must state the target version.
   • upgrade_phase: both — behaviour that should work on both versions. Use for any TC that establishes a pre-upgrade baseline and validates the same behaviour post-upgrade. E2E TCs spanning the full upgrade journey also use both — even if their steps cross both versions, they need to run on both clusters. Always include at least one UI-capable TC with both so the pre-upgrade run has browser content to execute.
   • No upgrade_phase — reserve for TCs that are genuinely unrelated to the upgrade scenario — TCs that would exist identically in a non-upgrade test plan. Within an upgrade-focused test plan, if a TC's expected results should be the same on both versions, use upgrade_phase: both (not no phase) to make its role in the regression suite explicit. "No phase" and both are functionally equivalent in filtering, but both signals intent.

   Apply upgrade_phase based on what the TC tests, not which category it belongs to. Any TC in any category (TC-UI-, TC-E2E-, etc.) whose expected results or preconditions differ between versions must be tagged. The question is: "Would this TC pass on the old version AND the new version?" If yes to both → both. If only new → post. If only old → pre.

   Add upgrade TCs to their own "Upgrade Testing" section in INDEX.md.

This category-by-category approach ensures cross-category awareness (no duplicate coverage) while keeping each batch focused.

Expected Results quality: Each Expected Result must be an observable fact that directly confirms the test objective. Avoid vague conclusions ("works correctly", "renders successfully"). Name the specific page state, URL pattern, response code, element, or resource field.

Before writing each assertion, ask: "Is this testing what the TC is fundamentally about, or just a side effect?" Two patterns follow from this:

• Accessibility / reachability tests (does this URL work? does this link open?): assert the absence of error — "page does not contain '500 Internal Server Error'", "response is HTTP 200", "page does not show 'Application is not available'". Do not assert presence of specific UI components (IDE editor pane, console window, specific layout element) — these vary by configuration, workbench image, and product version and will cause failures unrelated to the feature under test.

• Content / format tests (does this show the right value? did something change?): assert the specific observable fact — "URL contains hostname pattern X", "field value equals Y", "element Z is visible". Use this only when the content itself IS what is being verified.

A test that FAILs for the wrong reason is worse than no test at all. When in doubt, prefer the narrower assertion.

Anti-hallucination rules:

• Do NOT invent requirements not present in the test plan
• Do NOT create test cases for endpoints marked as "pending details" in Section 4
• If the test plan is ambiguous about what to test, ask the user via AskUserQuestion

Step 4: Generate Index

After all categories are complete (including upgrade TCs if generated):

1. Create <feature_dir>/test_cases/ directory if it doesn't already exist: mkdir -p <feature_dir>/test_cases
2. Each test case file must be self-contained — a tester should be able to execute it without reading the test plan
3. Use realistic test data, not placeholder values like "example.com" or "test123"
4. Generate <feature_dir>/test_cases/INDEX.md atomically (regenerate entire file):
   • Scan all TC-*.md files in test_cases/ directory
   • Extract test_case_id, priority, and title from each
   • Build complete INDEX.md with:
     • Quick stats (total test cases, P0/P1/P2 counts)
     • Test cases organized by category in tables (Test Case ID linked, Title, Priority)
     • Link to parent TestPlan.md
   • Write complete file in one operation (do NOT append incrementally)

Step 5: Update the Test Plan

Update <feature_dir>/TestPlan.md using the Edit tool:

1. Section 5 — Update the note to reflect test cases have been generated, with a link to test_cases/INDEX.md
2. Section 5.1 — Fill in the Test Case Organization table with category, test case count, and priority distribution
3. Section 6.1 — Fill in the E2E Scenario Summary table with the generated TC-E2E-* scenarios (ID, scenario name, endpoints covered, priority)
4. Section 6.2 — Fill in the E2E Coverage Matrix mapping each endpoint from Section 4 to its E2E scenario IDs
5. Section 10.1 — Fill in the Test Case Summary table with counts per category and priority breakdown
6. Section 10.2 — Fill in the Test Cases column with TC IDs mapped to each endpoint. Leave the Coverage column empty — it will be filled later by /coverage-assessment

Step 5.5: Update README

Update <feature_dir>/README.md to add a link to the test cases index:

• Add a "Test Cases" section (or update existing) with a link to test_cases/INDEX.md
• Include the total test case count and priority breakdown

Step 5.6: Coverage Validation

After generating all test case files and updating the test plan, validate coverage:

1. Endpoint coverage: Check that every endpoint/method from Section 4 (that is NOT marked as "pending details") has at least one test case. Flag any uncovered endpoints.
2. E2E coverage: Verify that every P0 endpoint from Section 4 is covered by at least one TC-E2E-* test case. If any P0 endpoint lacks E2E coverage, generate the missing E2E test case(s) before proceeding.
3. Test objective coverage: Check that every test objective from Section 1.3 is addressed by at least one test case. Flag any uncovered objectives.
4. Priority distribution: Verify that P0 endpoints have P0 test cases — a critical endpoint should not only have P2 test cases.
5. Gap cross-reference: If TestPlanGaps.md was read in Step 1.5, verify that no test cases were created for endpoints or areas flagged as pending/missing. If any were, remove them and flag the inconsistency.
6. Append to TestPlanGaps.md: If <feature_dir>/TestPlanGaps.md exists, append a ## Test Case Coverage Gaps section with any coverage gaps found (uncovered endpoints, missing objectives, priority mismatches, missing E2E scenarios). If the file does not exist, create it with just this section.

Step 5.7: Validate Frontmatter

After all test case files are written, validate their frontmatter in one pass:

(cd $(git -C ${CLAUDE_SKILL_DIR} rev-parse --show-toplevel) && uv run python scripts/validate_test_cases.py <feature_dir> test-case)

If any file fails validation, fix the frontmatter in that file and re-run the validation.

What this skill does NOT do

• Does NOT modify the test plan's Sections 1-4, 7-9 — those are owned by /test-plan-create
• Does NOT fill Automation Status or Notes in TC files — those are filled later by /coverage-assessment
• Does NOT create test cases for out-of-scope items or pending endpoints

$ARGUMENTS