Help us improve
Share bugs, ideas, or general feedback.
From dspy-api-skills
Reviews DSPy programs for correctness and best practices. Runs a 7-category audit covering signatures, modules, data pipelines, metrics, and optimizer usage.
npx claudepluginhub lebsral/dspy-programming-not-prompting-lms-skills --plugin dspy-build-skillsHow this skill is triggered — by the user, by Claude, or both
Slash command
/dspy-api-skills:ai-auditing-codeThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
**When NOT to use this skill:**
Guides technical evaluation of code review feedback: read fully, restate for understanding, verify against codebase, respond with reasoning or pushback before implementing.
Share bugs, ideas, or general feedback.
When NOT to use this skill:
/ai-fixing-errors first/ai-improving-accuracy/ai-kickoff or /ai-choosing-architecture/dspy-* skillBefore reading any code, ask:
Do not proceed until you have answers to both.
Read all DSPy-related files the user points you to. Build a mental model of:
forward() methods pass data between sub-modulesFor each category, check the items in reference.md. Mark each finding:
with_inputs(), metric always returns True)Predict where ChainOfThought would help, no error handling)Check that field names are descriptive (not output, result, text), types are correct, complex outputs use Pydantic models, and ambiguous fields have a desc= argument.
Check that modules are registered in __init__, forward() passes sub-module outputs as typed fields (not strings), and there is no raw string manipulation of LM outputs.
Check that with_inputs() is called on every dspy.Example, a train/dev split exists, example field names match the signature, and data loading is reproducible (no random shuffles without a seed).
Check that the metric handles edge cases (empty strings, None), returns a float between 0 and 1 or a bool, accepts the trace parameter, and can be tested independently of the module.
Check that the right optimizer is chosen for the dataset size, the trainset is large enough, the metric is passed correctly, and the optimized program is saved with program.save().
Check that LM calls are wrapped in error handling, timeouts are set, there is a fallback for LM failures, and costs have been estimated before deployment.
Check for f-string prompt construction instead of signatures, direct lm() calls instead of using modules, hardcoded prompt strings alongside DSPy code, and mixed raw API calls with DSPy modules.
After completing the audit, produce this report:
## DSPy Code Audit: [Project/Module Name]
### Summary
- X findings: Y critical, Z warnings, W info
- Overall assessment: [Ready for production / Needs fixes before production / Needs significant rework]
### Critical Findings
1. **[Category] — [Issue]**
- File: path/to/file.py:line
- Problem: ...
- Fix: ...
- Code:
Before: <code>
After: <code>
### Warnings
1. **[Category] — [Issue]**
- File: path/to/file.py:line
- Problem: ...
- Fix: ...
### Info
1. **[Category] — [Suggestion]**
- ...
### Recommended Next Steps
1. Fix all critical findings
2. Address warnings in priority order
3. Run /ai-improving-accuracy to measure baseline quality after fixes
After presenting the report, ask:
"Would you like me to apply the critical and warning fixes directly to your code?"
If yes, make the fixes. Do not silently apply fixes during the audit itself.
Do not rewrite code during the audit. Audit first, present findings, then fix on request. Silent refactoring while reviewing confuses users about what changed and why.
Do not rate everything CRITICAL. Reserve CRITICAL for things that cause silent failures, wrong results, or data loss. Suboptimal patterns are WARNING. Style issues are INFO.
Do not audit accuracy instead of code. This skill reviews code structure and patterns — not whether the AI produces correct answers. For accuracy measurement, send the user to /ai-improving-accuracy.
Respect domain context. A simple Predict module may be entirely correct for a simple task. Do not recommend ChainOfThought everywhere or assume complexity is always better.
Do not suggest MIPROv2 for every finding. Not every issue requires an optimizer-level fix. Many issues are plain code bugs that need to be corrected before any optimization makes sense.
Install any skill:
npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill <name>
/ai-fixing-errors/ai-improving-accuracy/ai-planning/ai-choosing-architecture/dspy-signatures/dspy-modules/dspy-optimizers/ai-do if you do not have it — it routes any AI problem to the right skill and is the fastest way to work: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill ai-do