harness-eval

harness-eval

A Claude Code plugin for systematic 3-tier evaluation of harness engineering quality Claude Code 하네스 엔지니어링 품질을 체계적으로 평가하는 플러그인

---

English

Overview

harness-eval is a Claude Code plugin that systematically evaluates the engineering quality of Claude Code harness configurations. It combines deterministic script-based quantitative checks with AI agent-powered qualitative reviews through a 3-tier evaluation system (Quick / Standard / Full).

The plugin scores projects across 6 dimensions — correctness, safety, completeness, actionability, consistency, and testability — producing structured reports with letter grades (A+ through F) and improvement roadmaps.

Features

• 3-Tier Evaluation System — Choose evaluation depth: Quick (< 30s checklist), Standard (static + dynamic analysis), or Full (multi-agent parallel review)
• Multi-Agent Analysis — Full mode dispatches 5 specialized agents (collector, safety-evaluator, completeness-evaluator, design-evaluator, synthesizer) for comprehensive assessment
• Evaluation History Tracking — Save, list, and compare evaluation results over time with trend analysis and delta reporting
• Badge Generation — Automatically generate score-based badges (A+ through F) in SVG and Markdown formats
• 4-Level Test Fixtures — Validate scoring accuracy across minimal, functional, robust, and production maturity levels

Prerequisites

• Bash 4+
• jq 1.6+
• Python 3.6+
• Git
• Claude Code CLI

Installation

Install

From the terminal (shell):

claude plugin marketplace add https://github.com/whchoi98/harness-eval
claude plugin install harness-eval@harness-eval

Or from inside a Claude Code session:

/plugin marketplace add https://github.com/whchoi98/harness-eval
/plugin install harness-eval@harness-eval

Verify

From the terminal:

claude plugin list

Or from inside a Claude Code session:

/plugin list

Update

From the terminal:

claude plugin marketplace refresh
claude plugin install harness-eval@harness-eval

Or from inside a Claude Code session:

/plugin marketplace refresh
/plugin install harness-eval@harness-eval

Uninstall

From the terminal:

claude plugin remove harness-eval
claude plugin marketplace remove harness-eval

Or from inside a Claude Code session:

/plugin remove harness-eval
/plugin marketplace remove harness-eval

From Source (For Development)

git clone https://github.com/whchoi98/harness-eval.git
cd harness-eval/plugins/harness-eval
bash scripts/setup.sh

Usage

Run evaluations from inside a Claude Code session:

/harness-eval:quick              # Checklist-based scoring (~30s)
/harness-eval:standard           # Static + dynamic analysis (~2-3min)
/harness-eval:full               # Multi-agent comprehensive review (~5-10min)
/harness-eval:compare            # Compare with previous evaluation
/harness-eval:harness-eval full  # Argument style also works

Run evaluation scripts directly:

# Score a target project
HARNESS_EVAL_ROOT=$(pwd) bash scripts/scoring.sh /path/to/target-project
# Output: {"score": 7.2, "grade": "B", "checks": [...]}

# Run static analysis
HARNESS_EVAL_ROOT=$(pwd) bash scripts/static-analysis.sh /path/to/target-project
# Output: {"summary": {"pass": 12, "warn": 1, "fail": 0, "total": 13}, ...}

# View evaluation history
HARNESS_EVAL_ROOT=$(pwd) bash scripts/history.sh list /path/to/target-project
# Output: [{"id": "eval-2026-04-06-001", "score": 7.2, ...}]

# Generate badge
bash scripts/badge.sh /path/to/target-project
# Output: Badge SVG/Markdown

Configuration

Variable	Description	Default
HARNESS_EVAL_ROOT	Path to the harness-eval plugin root directory	(required)
CLAUDE_NOTIFY_WEBHOOK	Webhook URL for evaluation completion notifications	(empty, disabled)

Project Structure