Help us improve
Share bugs, ideas, or general feedback.
From oracle
> Move a generated test from `tests/generated/` into the committed test
npx claudepluginhub bri-stevenski/oracle-test-ai-agent --plugin oracleHow this skill is triggered — by the user, by Claude, or both
Slash command
/oracle:oracle-promote-testThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
> Move a generated test from `tests/generated/` into the committed test
Provides a checklist for code reviews covering functionality, security, performance, maintainability, tests, and quality. Use for pull requests, audits, team standards, and developer training.
Share bugs, ideas, or general feedback.
Move a generated test from
tests/generated/into the committed test suite. Reviews the test for correctness, drops the generation header, relocates it under the appropriate suite directory, and confirms it runs in the project's normal test flow.
oracle-generate-test produced a file that has been validated
against the SUTtests/generated/tests/generated/api/foo.py → tests/api/foo.pytests/generated/e2e/checkout.spec.ts →
tests/e2e/checkout.spec.tstests/generated/unit/validator.py → tests/unit/validator.pyconftest.py or shared
setup)test_<feature>_<case>.py,
<feature>.spec.ts)git mv so the history is preserved if anyone later
runs git log --follow.tests/generated/.# TODO: adjust selector or similar — either resolve them or stop
the promotion and regenerate.black,
prettier, etc.) so the file matches surrounding style.pytest tests/api/test_orders_post.pynpx playwright test tests/e2e/checkout.spec.tsdocs/ORACLE_STATE.md so the project ledger tracks which generated
tests have been promoted.tests/generated/ — Source for promotion. Gitignored; nothing
here is ever a final artifact.tests/<suite>/ — Destination. Each suite has its own
conventions; never invent a new top-level dir during promotion.docs/ORACLE_STATE.md — Append a promotion entry: requirement,
generated path, promoted path, date.| Rationalization | Why It Is Wrong |
|---|---|
| "The test works, I'll skip the review and just commit it" | Generated code looks plausible but commonly has weak assertions or hardcoded values. Review is the whole point — promotion without review just imports debt into the suite. |
| "I'll fix the 6 issues I found in the review by editing the test" | Six issues means the prompt was wrong. Regenerate; hand-edits won't transfer to the next similar test. |
| "I'll leave the timestamp header so we know when it was generated" | Git history already records when the file was committed. The header rots and creates noise. |
| "It passes against staging, that's good enough" | If the test only passes in one environment, it's a fixture-bound smoke check, not a regression test. Either parametrize the env or don't promote it. |
| "I'll commit it without running the full suite — I only changed one test" | New tests can break shared fixtures, conflict on ports, leak state. Always run the full suite once before commit. |
Source: tests/generated/api/orders_post_201.py, validated, single
status assertion is exactly the contract.
Action:
git mv tests/generated/api/orders_post_201.py tests/api/test_orders_post_201.py# Generated by Oracle... header.tests/api/conftest.py fixtures.pytest tests/api/ → passes.ORACLE_STATE.md: promoted orders_post_201.py →
tests/api/test_orders_post_201.py on [date].Source: tests/generated/e2e/checkout.spec.ts. Review finds:
hardcoded test user creds, no wait for navigation, weak assertion
(expect(true).toBe(true)), TODO comments left in three places.
Action: Stop. Regenerate with a sharper prompt that names the real fixture user, specifies the wait condition, and asserts the actual checkout success state. Do not commit the broken draft.
Source: tests/generated/performance/spike_search_50rps.js. Used
once to confirm a single perf hypothesis. No ongoing value.
Action: Do not promote. Leave in tests/generated/. If the team
wants ongoing perf monitoring, generate a new test with a sustainable
RPS profile and promote that.
tests/generated/ — fix or delete.