Run CLI test suites (interactive or automated)
Executes CLI test suites in interactive or automated CI-ready mode.
/plugin marketplace add Data-Wise/craft/plugin install data-wise-craft@Data-Wise/crafttest/Execute CLI test suites generated by /craft:test:cli-gen.
Run CLI test suites with appropriate mode:
/craft:test:cli-run # Auto-detect and run
/craft:test:cli-run interactive # Run interactive mode
/craft:test:cli-run automated # Run CI mode
/craft:test:cli-run automated --verbose # Verbose output
/craft:test:cli-run --log test-results.log # Log to file
# Find CLI test files
find tests -name "*cli*.sh" -o -name "*cli*.bats" 2>/dev/null
ls tests/cli/ 2>/dev/null
Detection:
š CLI TEST SUITES DETECTED
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā
Found test suites:
š tests/cli/interactive-tests.sh (interactive)
š tests/cli/automated-tests.sh (automated)
Select mode:
1. [interactive] Run interactive test suite
2. [automated] Run automated test suite (CI mode)
3. [both] Run both sequentially
Or: /craft:test:cli-run <mode>
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā
INTERACTIVE CLI TEST SUITE
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā
Starting interactive test session...
Press Ctrl+C to abort at any time.
ā¶ TEST 1/15: Version Check
Command: aiterm --version
Expected: Version string
Run test? (y/n/skip) y
aiterm v0.3.0
Did this match expected? (y/n) y
ā
PASS
ā¶ TEST 2/15: Doctor Check
Command: aiterm doctor
Expected: System diagnostics
Run test? (y/n/skip) y
...
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā
AUTOMATED CLI TEST SUITE
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā
Running automated tests...
[1/15] CLI is installed ............................ ā
PASS
[2/15] Version command works ...................... ā
PASS
[3/15] Help is accessible ......................... ā
PASS
[4/15] Doctor command completes ................... ā
PASS
[5/15] Detect returns context info ................ ā
PASS
[6/15] Invalid commands handled gracefully ........ ā
PASS
[7/15] Claude settings accessible ................. ā
PASS
[8/15] MCP list works ............................. ā
PASS
[9/15] Sessions live works ........................ ā
PASS
[10/15] IDE list works ............................ ā
PASS
[11/15] OpenCode config works ..................... ā
PASS
[12/15] Exit code 0 on success .................... ā
PASS
[13/15] Exit code 1 on error ...................... ā
PASS
[14/15] Verbose flag works ........................ ā
PASS
[15/15] Output format correct ..................... ā
PASS
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā
RESULTS: 15 passed, 0 failed, 0 skipped
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā
Time: 23.4s
Exit: 0
š CLI TEST REPORT
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā
Suite: tests/cli/automated-tests.sh
Date: 2025-12-26 19:30:00
Duration: 23.4s
Results:
ā
Passed: 15 (100%)
ā Failed: 0 (0%)
āļø Skipped: 0 (0%)
Test Breakdown:
Category | Passed | Failed
--------------|--------|-------
Smoke | 3 | 0
Commands | 8 | 0
Arguments | 2 | 0
Errors | 1 | 0
Exit codes | 1 | 0
Status: ā
ALL TESTS PASSED
ā
CLI TESTS PASSED
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā
Results: 15/15 tests passed (100%)
Time: 23.4s
Exit code: 0
CI status: PASS
ā CLI TESTS FAILED
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā
Results: 13/15 tests passed (86.7%)
Time: 31.2s
Exit code: 1
Failed tests:
ā [8/15] MCP list works
Expected: Server list output
Actual: Error - config not found
ā [11/15] OpenCode config works
Expected: Config display
Actual: Command not recognized
Debug: Run with --verbose for details
CI status: FAIL
| Option | Description | Example |
|---|---|---|
--verbose | Detailed output | Show command stdout/stderr |
--log <file> | Log to file | --log results.log |
--timeout <s> | Per-test timeout | --timeout 30 |
--bail | Stop on first failure | For quick feedback |
--report <fmt> | Output format | json, junit, tap |
- name: Run CLI Tests
run: |
/craft:test:cli-run automated --report junit > test-results.xml
- name: Upload Test Results
uses: actions/upload-artifact@v3
with:
name: cli-test-results
path: test-results.xml
# Run tests with CI-friendly output
bash tests/cli/automated-tests.sh
# Check exit code
if [ $? -eq 0 ]; then
echo "CLI tests passed"
else
echo "CLI tests failed"
exit 1
fi
| Code | Meaning |
|---|---|
0 | All tests passed |
1 | One or more tests failed |
2 | Test suite error (config, syntax) |
Works with:
/craft:test:cli-gen - Generate test suites/craft:test:run - Unified test runner/craft:code:ci-local - CI checks/craft:git:sync - Pre-commit test run