Categorizes test failures as real_bug, test_bug, or flaky using heuristic pattern matching
From scout-qanpx claudepluginhub anicol/scout-qa --plugin scout-qaThis skill uses the workspace's default tool permissions.
Provides UI/UX resources: 50+ styles, color palettes, font pairings, guidelines, charts for web/mobile across React, Next.js, Vue, Svelte, Tailwind, React Native, Flutter. Aids planning, building, reviewing interfaces.
Fetches up-to-date documentation from Context7 for libraries and frameworks like React, Next.js, Prisma. Use for setup questions, API references, and code examples.
Builds 3-5 year financial models for startups with cohort revenue projections, cost structures, cash flow, headcount plans, burn rate, runway, and scenario analysis.
When test failures are encountered, classify each failure using these heuristic patterns. Check in order — first match wins.
The test itself is broken (bad imports, wrong setup, stale mocks). Fix the test, not the source code.
Indicators (check error message and output for these strings, case-insensitive):
modulenotfounderror — Python module import failureno module named — Python missing moduleimport error — general import failurecannot find module — Node.js module resolution failuremodule not found — generic module not foundsyntaxerror — syntax error in test codereferenceerror — undefined variable referenceundefined is not — accessing property of undefinedis not a function — calling non-functionis not defined — using undeclared identifiermock combined with not called or expected — mock setup issuejest.fn() — Jest mock configuration errorexpected mock — mock expectation failurefixture combined with not found or error — test fixture problembefore combined with hook or setup and error — setup hook failureno such file or directory — test referencing missing filepermission denied — filesystem permission issue in testNon-deterministic failure caused by environment, timing, or external dependencies. Usually passes on retry.
Indicators:
timeout — operation timed outtimed out — alternative timeout phrasingeconnrefused — connection refused (external service down)econnreset — connection resetnetwork combined with error — network failuresocket combined with error or hang up — socket issueconnection refused — explicit connection failureepipe — broken pipeenotfound — DNS resolution failure (BUT check test_bug first — modulenotfounderror contains this)etimeout — connection timeoutrate limit — API rate limitingtoo many requests — HTTP 429deadlock — database deadlocklock combined with timeout — lock acquisition failureresource temporarily unavailable — OS resource exhaustionflaky or intermittent — explicitly marked as flakyThe test may have caught an actual bug in the source code. These need investigation — don't auto-fix the test.
Indicators:
assertionerror combined with expected and got or actual — assertion comparing valuesexpect( combined with specific value mismatches — Jest/Vitest assertionassert combined with != or == — Python assertionexpected 200 with got 4 or got 5 — wrong HTTP statusnull or undefined combined with expected and a specific value — null pointer issuetypeerror: cannot read — accessing property of null/undefined (potential source bug)Important: real_bug classification is a signal, not a certainty. Report it as "potential real bug — investigate" rather than a definitive categorization.
If no patterns match, classify as unknown. Report the raw error and suggest the user investigate.
modulenotfounderror for test_bug)assert without clear context)When reporting failures in any agent:
Failed Tests:
1. test_user_login (tests/test_auth.py:45)
Category: real_bug (high confidence)
Error: AssertionError: expected status 200, got 401
Suggestion: The login endpoint returns wrong status. Investigate the auth handler.
2. test_payment_webhook (tests/test_payments.py:112)
Category: flaky (high confidence)
Error: ConnectionRefusedError: [Errno 61] Connection refused
Suggestion: External service dependency. Consider mocking the HTTP call.
3. test_import_feature (tests/test_feature.py:8)
Category: test_bug (high confidence)
Error: ModuleNotFoundError: No module named 'features.new_thing'
Suggestion: The import path is wrong. Check the module name and location.