From open-agreements
Audits NVCA recipes: checks file inventory, metadata schema, field-to-replacement coverage, ambiguous keys, smart quotes, test fixtures, and fill quality. Produces structured scorecards with maturity tiers.
npx claudepluginhub open-agreements/open-agreementsThis skill uses the workspace's default tool permissions.
Audit a single NVCA recipe's quality and produce a structured scorecard.
Evaluates Claude Code packages across 6 quality dimensions like frontmatter and structure for all 7 package types, producing scored audit reports. Use for quick single-package audits or full repository scans.
Provides checklists, metrics, scoring, and grep-based detection patterns for auditing README, API, and architecture documentation quality.
Generates quality playbook for any codebase: constitution (QUALITY.md), spec-traced functional tests, code review/integration protocols, multi-model spec audit, AI bootstrap (AGENTS.md).
Share bugs, ideas, or general feedback.
Audit a single NVCA recipe's quality and produce a structured scorecard.
ensureSourceDocx() which fetches from known template source URLs only.Run the audit for a specific recipe:
Audit the recipe: nvca-certificate-of-incorporation
Or audit all recipes:
Audit all NVCA recipes and update the quality tracker
| # | Check | How |
|---|---|---|
| S1 | File inventory | Does recipe have metadata.yaml, replacements.json, clean.json? Optional: computed.json, normalize.json, selections.json |
| S2 | Metadata schema valid | Run existing validateRecipeMetadata() from src/core/metadata.ts |
| S3 | Field-to-replacement coverage | For each field in metadata, is there a replacement key referencing {field_name}? |
| S4 | Ambiguous keys | Flag replacement keys < 8 chars without context qualifier (e.g., [name], [its]) |
| S5 | Smart quote coverage | Keys with apostrophes should have smart-quote variants (or patcher normalizes — check patcher has normalizeQuotes) |
| S6 | Source SHA present | source_sha256 in metadata.yaml |
| S7 | Test fixture exists | integration-tests/fixtures/{recipe-id}-*.json exists |
| # | Check | How |
|---|---|---|
| B1 | Source download + scan | Count all \[[_A-Z] prefixed bracket patterns in source (underscore-fill or capitalized placeholders) |
| B2 | Replacement coverage ratio | (keys with match in source) / (total bracket patterns). Target: >80% |
| B3 | Unmatched underscore patterns | [___+] patterns in source not in replacements.json |
| B4 | Clean effectiveness | After clean, no footnotes, no "Note to Drafter", no preamble |
| # | Check | How |
|---|---|---|
| F1 | Default-only fill | Fill with defaults, run verifyOutput, count blank placeholders |
| F2 | Full-values fill | Fill with all fields from test fixture, assert all verify checks pass |
| F3 | Formatting anomaly count | Check 8 from verifier (single-char underlined runs) |
| F4 | Zero-match replacement keys | Keys that existed in replacements.json but matched nothing in the source |
{
"recipe_id": "nvca-voting-agreement",
"maturity": "beta",
"scores": { "structural": "6/7", "behavioral": "3/4", "fill": "0/4", "total": "9/15" },
"checks": [
{ "id": "S1", "name": "File inventory", "passed": true, "details": "metadata.yaml, replacements.json, clean.json present" },
{ "id": "S7", "name": "Test fixture exists", "passed": false, "details": "No fixture matching integration-tests/fixtures/nvca-voting-agreement-*.json" }
],
"field_coverage": { "metadata_fields": 14, "replacement_refs": 10, "uncovered": 4 },
"recommendations": [
"Add test fixture for fill testing (S7)",
"Add replacement keys for 4 uncovered fields (S3)"
]
}
When running the audit:
content/recipes/QUALITY_TRACKER.md for current stateensureSourceDocx())validateRecipeMetadata() from src/core/metadata.ts for S2ensureSourceDocx() from src/core/recipe/downloader.ts for B1-B4runRecipe() from src/core/recipe/index.ts for F1-F4\[[_A-Z] prefix to avoid counting citations and legal referencesPatchResult.zeroMatchKeys returned by the patchercleanConfig.removeRanges and cleanConfig.removeParagraphPatterns to suppress expected zero-matches