npx claudepluginhub passelin/marketplace-test --plugin phoenixThis skill uses the workspace's default tool permissions.
Build evaluators for AI/LLM applications. Code first, LLM for nuance, validate against humans.
references/axial-coding.mdreferences/common-mistakes-python.mdreferences/error-analysis-multi-turn.mdreferences/error-analysis.mdreferences/evaluate-dataframe-python.mdreferences/evaluators-code-python.mdreferences/evaluators-code-typescript.mdreferences/evaluators-custom-templates.mdreferences/evaluators-llm-python.mdreferences/evaluators-llm-typescript.mdreferences/evaluators-overview.mdreferences/evaluators-pre-built.mdreferences/evaluators-rag.mdreferences/experiments-datasets-python.mdreferences/experiments-datasets-typescript.mdreferences/experiments-overview.mdreferences/experiments-running-python.mdreferences/experiments-running-typescript.mdreferences/experiments-synthetic-python.mdreferences/experiments-synthetic-typescript.mdCreates isolated Git worktrees for feature branches with prioritized directory selection, gitignore safety checks, auto project setup for Node/Python/Rust/Go, and baseline verification.
Executes implementation plans in current session by dispatching fresh subagents per independent task, with two-stage reviews: spec compliance then code quality.
Dispatches parallel agents to independently tackle 2+ tasks like separate test failures or subsystems without shared state or dependencies.
Build evaluators for AI/LLM applications. Code first, LLM for nuance, validate against humans.
Starting Fresh: observe-tracing-setup → error-analysis → axial-coding → evaluators-overview
Building Evaluator: fundamentals → common-mistakes-python → evaluators-{code|llm}-{python|typescript} → validation-evaluators-{python|typescript}
RAG Systems: evaluators-rag → evaluators-code-* (retrieval) → evaluators-llm-* (faithfulness)
Production: production-overview → production-guardrails → production-continuous
| Prefix | Description |
|---|---|
fundamentals-* | Types, scores, anti-patterns |
observe-* | Tracing, sampling |
error-analysis-* | Finding failures |
axial-coding-* | Categorizing failures |
evaluators-* | Code, LLM, RAG evaluators |
experiments-* | Datasets, running experiments |
validation-* | Validating evaluator accuracy against human labels |
production-* | CI/CD, monitoring |
| Principle | Action |
|---|---|
| Error analysis first | Can't automate what you haven't observed |
| Custom > generic | Build from your failures |
| Code first | Deterministic before LLM |
| Validate judges | >80% TPR/TNR |
| Binary > Likert | Pass/fail, not 1-5 |