eval-framework
Evaluation framework skills for designing scoring rubrics, running structured evaluations on LLM outputs, and comparing candidate outputs to recommend a winner.
Component Overview
Install
npx claudepluginhub ats-kinoshita-iso/agent-workshop --plugin eval-frameworkComponent Details
Skills (3)
Compare two LLM outputs on the same evaluation criteria and recommend a winner with justification. Use this skill when asked to "compare these outputs", "which response is better", "A/B eval", or "pick the best candidate".
Design evaluation criteria and a 1-5 scoring rubric for a task or LLM output. Use this skill when asked to "create an eval", "define evaluation criteria", "build a scoring rubric", or "design how to measure quality" for any output.
Execute a structured evaluation against a set of LLM outputs and produce a scored report. Use this skill when asked to "run the eval", "score these outputs", "evaluate this response", or "generate an evaluation report".
README
agent-workshop
A plugin library and development framework for Claude Code. This repo serves two purposes:
- Plugin Library — A custom marketplace
of modular Claude Code plugins (skills, hooks, agents, MCP configs) installable via
/plugin install. - Cookbook — Golden baseline configurations (CLAUDE.md templates, hook recipes, MCP setups) you can pull into any new project.
Quick Start
Install Plugins from This Marketplace
# Add the marketplace
/plugin marketplace add ats-kinoshita-iso/agent-workshop
# Browse and install plugins
/plugin install planning@agent-workshop
Use Cookbook Recipes
Browse cookbook/ and copy what you need into your project:
cookbook/claude-md/— CLAUDE.md templates by project typecookbook/hooks/— Reusable hook recipes for.claude/settings.jsoncookbook/mcp/— MCP server configurations for common integrations
Directory Structure
.claude-plugin/ # Marketplace definition (marketplace.json)
plugins/ # Stable, packaged Claude Code plugins
cookbook/ # Golden baseline configs (copy into your projects)
claude-md/ # CLAUDE.md templates
hooks/ # Hook recipes
mcp/ # MCP server configs
tools/ # Development & validation tooling
tests/ # Plugin validation gates
Plugin Development Lifecycle
1. Develop in .claude/ → Iterate locally with Claude Code
2. Validate with test suite → uv run pytest
3. Package as plugin → Create plugin.json + SKILL.md in plugins/
4. Auto-register in marketplace → marketplace_gen.py updates marketplace.json
5. Users install from here → /plugin marketplace add ats-kinoshita-iso/agent-workshop
Available Plugins
Workshop Plugins
| Plugin | Description | Version |
|---|---|---|
| code-quality-gate | Unified quality orchestrator (lint, format, typecheck, test) | 1.0.0 |
| context-sync | Keeps CLAUDE.md files in sync with the codebase | 1.0.0 |
| plan-manager | Plan lifecycle management with gate tracking and archival | 1.0.0 |
| planning | Phased implementation plans with gates and tests | 2.0.0 |
| test-quality | Test generation, auditing, and knowledge extraction | 1.0.0 |
| workspace-clean | Workspace hygiene checks and cleanup | 1.0.0 |
Anthropic Skills (imported from anthropics/skills)
| Plugin | Skills | License |
|---|---|---|
| anthropic-document-skills | docx, pdf, pptx, xlsx | Source-available |
| anthropic-creative-skills | algorithmic-art, brand-guidelines, canvas-design, frontend-design, slack-gif-creator, theme-factory | Apache 2.0 |
| anthropic-dev-skills | claude-api, mcp-builder, skill-creator, web-artifacts-builder, webapp-testing | Apache 2.0 |
| anthropic-enterprise-skills | doc-coauthoring, internal-comms | Apache 2.0 |
Stack
- Python: managed with
uv - JS/TS: managed with
bun
Commands
Python
- Install deps:
uv sync - Run tests:
uv run pytest - Lint:
uv run ruff check . - Format:
uv run ruff format . - Type check:
uv run mypy .
JS/TS
- Install deps:
bun install - Run tests:
bun test - Lint + format:
bunx biome check --write .
Contributing a Plugin
- Develop your extension in
.claude/(skills, hooks, agents, etc.) - When stable, create a directory under
plugins/<your-plugin>/ - Add
.claude-plugin/plugin.jsonwith name, description, version, keywords - Add your skill/hook/agent/MCP files following the Claude Code plugin format
- Run
uv run pytestto validate structure - Run
uv run python tools/marketplace_gen.pyto update the marketplace catalog
Similar Plugins
creative-writing
Complete creative writing suite with 10 specialized agents covering the full writing process: research gathering, character development, story architecture, world-building, dialogue coaching, editing/review, outlining, content strategy, believability auditing, and prose style/voice analysis. Includes genre-specific guides, templates, and quality checklists.
Show your plugin is listed on ClaudePluginHub.
[](https://www.claudepluginhub.com/plugins/ats-kinoshita-iso-eval-framework-plugins-eval-framework?ref=badge)Paste near the top of your README, alongside other badges.