automated-assessment

Automated Assessment Skill

Systematic project assessment against all Netresearch skills with checkpoint-based verification - by Netresearch.

Why This Skill Exists

When asked to "ensure project aligns with all skills", LLMs typically:

• Cherry-pick obvious issues (satisficing)
• Miss 50-80% of requirements
• Report "done" without exhaustive verification

This skill enforces systematic verification through:

1. Scripted pre-flight checks (mechanical, 100% accurate)
2. Domain-batched LLM agents (subjective judgment)
3. Structured JSON output (verifiable completeness)

Installation

Marketplace (Recommended)

Add the Netresearch marketplace once, then browse and install skills:

# Claude Code
/plugin marketplace add netresearch/claude-code-marketplace

npx (skills.sh)

Install with any Agent Skills-compatible agent:

npx skills add https://github.com/netresearch/automated-assessment-skill --skill automated-assessment
npx skills add https://github.com/netresearch/automated-assessment-skill --skill add-checkpoints

Download Release

Download the latest release and extract to your agent's skills directory.

Git Clone

git clone https://github.com/netresearch/automated-assessment-skill.git

Composer (PHP Projects)

composer require netresearch/automated-assessment-skill

Requires netresearch/composer-agent-skill-plugin.

npm (Node Projects)

npm install --save-dev \
  @netresearch/agent-skill-coordinator \
  github:netresearch/automated-assessment-skill

Requires @netresearch/agent-skill-coordinator, which discovers the skill in node_modules and registers it in AGENTS.md via a postinstall hook. For pnpm, also allowlist the coordinator's postinstall:

{
  "pnpm": {
    "onlyBuiltDependencies": ["@netresearch/agent-skill-coordinator"]
  }
}

Usage

Run Assessment

/assess                              # Assess against all matching skills
/assess skill-repo typo3-testing     # Assess against specific skills only
/assess --force                      # Run all skills, ignore preconditions
/assess --mechanical-only            # Skip LLM reviews, only scripted checks

This will:

1. Detect project root
2. Discover all skills with checkpoints
3. Evaluate preconditions — skip skills that don't match project type
4. Run scripted mechanical checks
5. Spawn domain-batched LLM agents for subjective reviews
6. Generate compliance report

Run Checkpoints Manually

scripts/run-checkpoints.sh <checkpoints.yaml> <project-root>

Adding Checkpoints to Skills

Create checkpoints.yaml in your skill's root directory:

version: 1
skill_id: my-skill

mechanical:
  - id: MS-01
    type: file_exists
    target: README.md
    severity: error
    desc: "README.md must exist"

llm_reviews:
  - id: MS-10
    domain: repo-health
    prompt: "Verify README structure follows standards"
    severity: warning
    desc: "README should have standard sections"

See skills/automated-assessment/references/checkpoints-schema.md for full schema documentation.

Checkpoint Types

Type	Description
file_exists	File must exist
file_not_exists	File must NOT exist
contains	File contains literal string
regex	File matches regex pattern
json_path	JSON path exists (jq)
command	Command exits with 0
llm_review	Requires LLM judgment

Domain Groups

Domain	Focus
repo-health	README, badges, branding, AGENTS.md
security	SLSA, OpenSSF, SBOM, vulnerabilities
code-quality	PHPStan, tests, PHP patterns
documentation	RST, docs.typo3.org standards
git-workflow	Branching, commits, tags, conventional commits
docker	Dockerfile, compose, container patterns
ddev	DDEV configuration, services, commands
upgrade	TYPO3 version upgrades, deprecations

Example Assessment Output

{
  "project": "netresearch/contexts",
  "overall_status": "FAIL",
  "summary": {
    "total": 45,
    "pass": 38,
    "fail": 5,
    "skip": 2
  },
  "checkpoints": [
    {"id": "GH-01", "status": "pass", "evidence": "README.md exists"},
    {"id": "ER-04", "status": "fail", "evidence": "Missing OpenSSF badge"}
  ]
}

Assets