schliff

The Ruff for AGENTS.md — deterministic quality scores for the instruction files that drive your AI. Same input, same score, on every machine.

That last badge is Schliff scoring this repo's own AGENTS.md — live, and reproducible from your own checkout.

Your AI instruction files silently degrade — and nothing catches it. AGENTS.md is read by Cursor, Codex, Copilot, and Claude Code — one rotting file now quietly degrades four tools. A trigger phrase rots, an edge case slips, the file balloons past its token budget. No error, no red test — just agents that quietly get worse.

Schliff scores AGENTS.md — and the rest of the family (SKILL.md, CLAUDE.md, .cursorrules, system prompts) — against an explicit, versioned rubric. No LLM judge in the critical path. No network. No randomness. A rule engine you can read, pin, and gate CI on.

pip install schliff
schliff score AGENTS.md   # or any SKILL.md / CLAUDE.md / .cursorrules
schliff demo              # no instruction file handy? score a built-in bad one

This is the real, current output of schliff score AGENTS.md on this repo's own AGENTS.md — clone and run it yourself:

schliff v8.7.0

  structure             ██████████  100/100  perfect
  operational_coverage  ██████████  100/100  perfect
  efficiency            ████████░░   79/100  good
  composability         ██████████   95/100  excellent
  clarity               ██████████  100/100  perfect

  Structural Score  ███████████████████░  95.8/100  [S]

  Tokens: 996 / 3,000 (ok)
  Format: agents.md (normalized)

No model produced that number. Run it on another laptop and you get 95.8 again — a score you can't reproduce isn't a measurement, it's a vibe. And we hold ourselves to that: this repo's own badge (scored in isolation) once disagreed with its own CLI (scored in-repo) by ~15 points on the same bytes, because structure was crediting an on-disk references/ neighbourhood instead of the file's content. We fixed the engine, not the file (#10) — now cp AGENTS.md /tmp && schliff score /tmp/AGENTS.md returns the same number as CI and the badge. (The agents.md headline weights structure·operational_coverage·efficiency at 0.4/0.4/0.2; composability and clarity are shown for information, not counted.)

The quick-start and case-study numbers here are reproducible from released schliff==8.7.0 (pip install schliff==8.7.0); the hydra field run below is dated to the version it was measured on. No install? Paste your file into the playground — same engine, with AGENTS.md and SKILL.md tabs.

---

A real catch

The SKILL.md for ShieldClaw — a real prompt-injection-defense skill, now archived — is Schliff's reproducible before/after. The fixtures ship in docs/case-studies/shieldclaw/; every number below is the current engine's output, reproducible with schliff score. (For the 28.7 row, score a copy of SKILL-before.md outside that directory — in place, the engine auto-discovers the sibling eval suite.)

	Score	Grade	Dimensions measured
Before, scored in isolation	28.7	F	4/7 — no eval suite; explicit ceiling warning
Before, with its eval suite	84.5	B	7/7
After fixes	94.6	A	7/7

Two separate effects, and Schliff refuses to conflate them. Adding the eval suite lifted the measurement ceiling (28.7 → 84.5) — that is coverage, not quality. The quality delta is 84.5 → 94.6, driven by composability 20 → 86 and efficiency 60 → 83 after adding scope boundaries, an I/O contract, and handoffs — structural gaps a linter can't see, caught as a number that was too low.