rails-audit

rails-audit

A Claude Code skill that runs a structured stability audit of a Ruby on Rails codebase across 18 dimensions and produces a single severity-ranked markdown report with a recommended fix sequence.

Why

Rails audits in practice tend to fall into two failure modes:

1. A single tool's output dressed up as an audit. A 400-warning brakeman dump is not a report — it's evidence. Engineers ignore it.
2. An exhaustive checklist with no severity calibration. Everything is "high" → nothing is high → no one acts.

This skill takes the opposite approach. It invokes the existing Rails tooling ecosystem (brakeman, bundler-audit, rubocop, reek, rails_best_practices, rubycritic, simplecov, flog/flay) plus codebase-aware grep and orchestrates 4 parallel subagents to synthesize findings into:

• An executive summary with a 0–10 risk score
• Per-dimension scorecards
• A punch list grouped by Blocker / High / Medium / Low — with verbatim evidence and fix sketches
• A recommended fix sequence ordered so each phase unblocks the next
• A trend table when re-run on the same repo

What it covers

18 dimensions, organized into 4 clusters that run in parallel:

Cluster A — Spec & Coverage

1. Spec stability — flake patterns, factory cascade, CI mechanics
2. Test coverage — risk-weighted coverage, branch coverage, mutation testing on critical paths

Cluster B — Deploy & CI 3. Foundation — Ruby/Rails EOL, lock freshness, boot health 4. Deploy & CI — workflows, Dockerfile, prod config, secrets, health, rollback 5. Observability — structured logs, error tracking, APM, request IDs

Cluster C — Code Health 6. Domain shape — model/route inventory, layering 7. Risk hotspots — biggest files per layer, churn × complexity 8. Code smells & antipatterns — Rails-specific + generic Ruby smells, callback chains, train wrecks, fat controllers 9. Performance — N+1, indexes, cache, slow queries 10. Reliability — timeouts, circuit breakers, pool sizing, idempotency 11. Background jobs — adapter, retry/discard, idempotency, DLQ 12. Data integrity — FK constraints, validation duplication, soft-delete consistency 13. Developer experience — bin/setup, suite runtime, README accuracy 14. Cost & scaling — table sizes, missing indexes, hot-endpoint N+1

Cluster D — Security & Money 15. Security — OWASP Top 10 mapped to Rails (SSRF, deserialization, injection, headers, secrets in logs) 16. Authorization — IDOR, policy spec coverage, admin-action testing 17. Money paths — Decimal vs Float, Stripe idempotency, webhook event dedup, transaction boundaries 18. Data governance — PII inventory, encryption at rest, audit logs, retention

Modes

Mode	Time	What runs
--quick	~3 min	Static tools + grep checks, single agent. PR-time use.
--standard (default)	~10–15 min	Full audit, 4 parallel subagents.
--deep	~30+ min	Standard + boots app, runs spec subset, mutation testing on critical paths.

Recommended cadence. --quick per PR (or pre-merge), --standard quarterly or before a major release, --deep once before each significant release. --continue is the cheap re-run mode (~10× lower cost) for "I fixed N findings; show me the updated trend."

Cost & time

The skill calls Anthropic's API through your Claude Code session. Token usage and dollar cost depend on the model your Claude Code is configured with and your project's size. Rough order-of-magnitude estimates from the dogfood projects (30–50 KLOC Rails apps, Sonnet 4.x):

Mode	Input tokens	Output tokens	Approx. cost (Sonnet 4.5)
--quick	~10–20K	~3–5K	$0.05–$0.20
--standard	~50–80K	~10–15K	$0.20–$1.50
--continue	~5K	~2K	~$0.05
--deep	varies (boots app, runs specs)	varies	$1.00+

These are estimates — your actual cost depends on model choice (Opus is ~5× Sonnet), project size, and finding density. The skill writes cost.estimated_input_tokens / cost.estimated_output_tokens into the JSON report, and when your harness exposes per-call usage metadata it also populates cost.actual_*. Watch the first run on a small --quick to anchor your own cost expectations before going --standard on a 100 KLOC monolith.

Privacy

What stays on your machine:

• Your codebase (the skill never uploads source files).
• Static-tool outputs (brakeman, bundle-audit, rubocop, etc.) — invoked locally via your project's gems.
• The generated report (tmp/rails-audit/report-*.json/*.md) — written locally, not uploaded.

What leaves your machine:

• Findings, file paths, line numbers, and code snippets cited as evidence are sent to Anthropic's API as part of the synthesis prompts (the same way Claude Code already sends file content when you ask it to read files).
• No traffic to third-party services beyond Anthropic.