Stats
Actions
Tags
Help us improve
Share bugs, ideas, or general feedback.
From judgeval
Automatically trace Claude Code sessions to Judgeval for observability and debugging.
npx claudepluginhub judgmentlabs/judgeval-claude-plugin --plugin judgevalHow this skill is triggered — by the user, by Claude, or both
Slash command
/judgeval:trace-claude-codeThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
Automatically trace Claude Code sessions to Judgeval for observability and debugging.
Audits local AI coding-agent sessions with agenttrace for cost, tool failures, latency, anomalies, health, and session diffs.
Adds LangSmith tracing to Python/JS LLM apps via LangChain auto-tracing, traceable decorators, or OpenTelemetry; queries/export traces with langsmith CLI.
Sets up OpenInference tracing for Claude Code CLI or Agent SDK apps using Arize AX (cloud) or Phoenix (self-hosted). Guides installation, credentials, project setup, configuration, and troubleshooting.
Share bugs, ideas, or general feedback.
Automatically trace Claude Code sessions to Judgeval for observability and debugging.
Session (task span)
├── Turn 1 (task span)
│ ├── claude-opus-4-5 (llm span)
│ ├── Read: file.py (tool span)
│ ├── Subagent: code-reviewer (task span) ← Subagent with nested spans
│ │ ├── claude-3-5-haiku (llm span)
│ │ ├── Read (tool span)
│ │ └── claude-3-5-haiku (llm span)
│ └── claude-opus-4-5 (llm span)
└── Turn 2 (task span)
└── ...
After installing the plugin, run setup in your project directory:
cd /path/to/your/project
bash ~/.claude/plugins/marketplaces/judgeval-claude-plugin/skills/trace-claude-code/setup.sh
This will prompt you for:
JUDGMENT_API_KEY - Your Judgeval API keyJUDGMENT_ORG_ID - Your organization IDJUDGMENT_API_URL - API URL (default: https://api.judgmentlabs.ai)JUDGEVAL_CC_PROJECT - Project name (default: claude-code)| Variable | Required | Description |
|---|---|---|
TRACE_TO_JUDGEVAL | Yes | Set to true to enable tracing |
JUDGMENT_API_KEY | Yes | Your Judgeval API key |
JUDGMENT_ORG_ID | Yes | Your organization ID |
JUDGMENT_API_URL | No | API URL (default: https://api.judgmentlabs.ai) |
JUDGEVAL_CC_PROJECT | No | Project name (default: claude-code) |
JUDGEVAL_CC_DEBUG | No | Set to true for debug logging |
| Hook | Trigger | Action |
|---|---|---|
session_start.sh | Session begins | Creates root trace span |
user_prompt_submit.sh | User sends prompt | Creates Turn span |
post_tool_use.sh | Tool completes | Tracks tool count |
stop_hook.sh | Response complete | Marks turn for finalization |
subagent_stop.sh | Subagent completes | Parses subagent transcript, creates nested spans |
session_end.sh | Session ends | Creates LLM/Tool spans, finalizes session |
judgment.span_kind: "task"judgment.input: Session descriptionjudgment.output: Completion summaryturn_count: Number of turnsjudgment.span_kind: "llm"judgment.input: Conversation historyjudgment.output: Model responsejudgment.llm.model: Model namejudgment.llm.provider: "anthropic"judgment.usage.non_cached_input_tokens: Input tokensjudgment.usage.output_tokens: Output tokensjudgment.usage.cache_creation_input_tokens: Cache write tokensjudgment.usage.cache_read_input_tokens: Cache read tokensjudgment.span_kind: "tool"judgment.input: Tool inputjudgment.output: Tool outputtool_name: Tool identifierHook logs are written to: ~/.claude/state/judgeval_hook.log
Enable debug logging:
export JUDGEVAL_CC_DEBUG=true
Traces not appearing:
TRACE_TO_JUDGEVAL=true is setMissing spans: