/craft:test:cli-run - CLI Test Suite Runner

Execute CLI test suites generated by /craft:test:cli-gen.

Purpose

Run CLI test suites with appropriate mode:

• interactive - Guided testing with human input
• automated - CI-ready execution (exit 0/1)
• auto - Auto-detect mode from test files

Usage

/craft:test:cli-run                          # Auto-detect and run
/craft:test:cli-run interactive              # Run interactive mode
/craft:test:cli-run automated                # Run CI mode
/craft:test:cli-run automated --verbose      # Verbose output
/craft:test:cli-run --log test-results.log   # Log to file

Step-by-Step Process

Step 1: Locate Test Suites

# Find CLI test files
find tests -name "*cli*.sh" -o -name "*cli*.bats" 2>/dev/null
ls tests/cli/ 2>/dev/null

Detection:

🔍 CLI TEST SUITES DETECTED
═══════════════════════════════════════════════════════════════

Found test suites:
  📄 tests/cli/interactive-tests.sh (interactive)
  📄 tests/cli/automated-tests.sh (automated)

Select mode:
  1. [interactive] Run interactive test suite
  2. [automated]   Run automated test suite (CI mode)
  3. [both]        Run both sequentially

Or: /craft:test:cli-run <mode>

Step 2: Execute Tests

Interactive Mode

═══════════════════════════════════════════════════════════════
  INTERACTIVE CLI TEST SUITE
═══════════════════════════════════════════════════════════════

Starting interactive test session...
Press Ctrl+C to abort at any time.

▶ TEST 1/15: Version Check
  Command: aiterm --version
  Expected: Version string

Run test? (y/n/skip) y

aiterm v0.3.0

Did this match expected? (y/n) y
✅ PASS

▶ TEST 2/15: Doctor Check
  Command: aiterm doctor
  Expected: System diagnostics

Run test? (y/n/skip) y
...

Automated Mode

═══════════════════════════════════════════════════════════════
  AUTOMATED CLI TEST SUITE
═══════════════════════════════════════════════════════════════

Running automated tests...

[1/15] CLI is installed ............................ ✅ PASS
[2/15] Version command works ...................... ✅ PASS
[3/15] Help is accessible ......................... ✅ PASS
[4/15] Doctor command completes ................... ✅ PASS
[5/15] Detect returns context info ................ ✅ PASS
[6/15] Invalid commands handled gracefully ........ ✅ PASS
[7/15] Claude settings accessible ................. ✅ PASS
[8/15] MCP list works ............................. ✅ PASS
[9/15] Sessions live works ........................ ✅ PASS
[10/15] IDE list works ............................ ✅ PASS
[11/15] OpenCode config works ..................... ✅ PASS
[12/15] Exit code 0 on success .................... ✅ PASS
[13/15] Exit code 1 on error ...................... ✅ PASS
[14/15] Verbose flag works ........................ ✅ PASS
[15/15] Output format correct ..................... ✅ PASS

═══════════════════════════════════════════════════════════════
  RESULTS: 15 passed, 0 failed, 0 skipped
═══════════════════════════════════════════════════════════════

Time: 23.4s
Exit: 0

Step 3: Generate Report

📊 CLI TEST REPORT
═══════════════════════════════════════════════════════════════

Suite: tests/cli/automated-tests.sh
Date: 2025-12-26 19:30:00
Duration: 23.4s

Results:
  ✅ Passed: 15 (100%)
  ❌ Failed: 0 (0%)
  ⏭️ Skipped: 0 (0%)

Test Breakdown:
  Category      | Passed | Failed
  --------------|--------|-------
  Smoke         | 3      | 0
  Commands      | 8      | 0
  Arguments     | 2      | 0
  Errors        | 1      | 0
  Exit codes    | 1      | 0

Status: ✅ ALL TESTS PASSED

Output Format

Success Output

✅ CLI TESTS PASSED
═══════════════════════════════════════════════════════════════

Results: 15/15 tests passed (100%)
Time: 23.4s
Exit code: 0

CI status: PASS

Failure Output

❌ CLI TESTS FAILED
═══════════════════════════════════════════════════════════════

Results: 13/15 tests passed (86.7%)
Time: 31.2s
Exit code: 1

Failed tests:
  ❌ [8/15] MCP list works
     Expected: Server list output
     Actual: Error - config not found

  ❌ [11/15] OpenCode config works
     Expected: Config display
     Actual: Command not recognized

Debug: Run with --verbose for details
CI status: FAIL

Options

Option	Description	Example
--verbose	Detailed output	Show command stdout/stderr
--log <file>	Log to file	--log results.log
--timeout <s>	Per-test timeout	--timeout 30
--bail	Stop on first failure	For quick feedback
--report <fmt>	Output format	json, junit, tap

CI Integration

GitHub Actions

- name: Run CLI Tests
  run: |
    /craft:test:cli-run automated --report junit > test-results.xml

- name: Upload Test Results
  uses: actions/upload-artifact@v3
  with:
    name: cli-test-results
    path: test-results.xml

Generic CI

# Run tests with CI-friendly output
bash tests/cli/automated-tests.sh

# Check exit code
if [ $? -eq 0 ]; then
    echo "CLI tests passed"
else
    echo "CLI tests failed"
    exit 1
fi

Exit Codes

Code	Meaning
0	All tests passed
1	One or more tests failed
2	Test suite error (config, syntax)

Integration

Works with:

• /craft:test:cli-gen - Generate test suites
• /craft:test:run - Unified test runner
• /craft:code:ci-local - CI checks
• /craft:git:sync - Pre-commit test run