ai-notary

ai-notary

Cryptographic proof that AI actually ran your validation commands.

Why This Exists

AI assistants like Claude can run npm test and tell you it passed. But did it? You have no proof. The AI could:

• Skip expensive tests and claim they passed
• Run a subset and report success
• Simply lie about the output

ai-notary forces the AI to prove it ran commands. The proof is cryptographic - the AI literally cannot fake it.

The Key Insight

The AI cannot complete browser-based authentication. We exploit this:

1. AI runs commands → creates cryptographic attestations (automatic)
2. Human signs policy → requires browser OIDC login (AI can't do this)
3. AI tries to push → blocked unless attestations verify against human-signed policy

Result: AI must actually run the commands. No shortcuts. No lies.

Quickstart for Claude

1. Install the plugin

/plugin install nkennedy-personal/witness-signer

2. One-Time Setup: Create and sign a policy

Do this once per project. The policy defines what validation steps are required.

# Create and sign in one step (opens browser for OIDC)
ai-notary policy create \
  --steps=lint,test,build \
  --keyless \
  -o policy-signed.json

# Commit the signed policy to your repo
git add policy-signed.json && git commit -m "Add attestation policy"

Or create from existing attestations:

# After running commands with attestations...
ai-notary policy create \
  --from-tree=$(git rev-parse HEAD^{tree}) \
  --keyless \
  -o policy-signed.json

The policy is long-lived. You only recreate it when adding/removing required steps.

3. Every Push: Run commands and verify

Tell Claude to run your validation commands:

Run lint, test, and build with attestations

Claude will use the MCP bash tool:

{"command": "npm run lint", "attest": true, "step": "lint"}
{"command": "npm test", "attest": true, "step": "test"}
{"command": "npm run build", "attest": true, "step": "build"}

4. Push succeeds if attestations satisfy policy

> git push
✓ Verification passed (attestations match policy)
→ Push succeeded

If Claude skipped any steps, push is blocked. No new policy needed - just run the missing steps.

Understanding the Workflow

See docs/mental-model.md for a detailed explanation of:

• When to create attestations (daily development)
• When to create policies (admin setup)
• Trust model and security guarantees

Two Types of Policies

Infrastructure Policies (Set Once)

Quality gates for all code changes:

• Build passes
• Linting passes
• Tests pass
• Security scans clean

# Create once per project
ai-notary policy create \
  --steps=build,lint,test \
  --keyless -o policy-signed.json

Feature Policies (Per Feature)

Acceptance criteria for specific features:

• UAT screenshots show expected behavior
• API contracts verified
• Requirements from ticket satisfied

# Create per feature (or use /plan-to-policy skill)
ai-notary policy create \
  --steps=uat-search \
  --add-ai-policy='step=uat-search,name=verify,prompt="PASS if search works"' \
  --keyless -o policies/feature-search.json

Verification Combines Both

# All policies in policies/ dir must pass
ai-notary verify --policy-dir=policies/

# Or explicitly list policies
ai-notary verify \
  --policy=policy-signed.json \
  --policy=policies/feature-search.json

See Two-Tier Policy Architecture for more details.

How It Works