phoenix-arize-setup

From babysitter

411

Description

Arize Phoenix observability platform setup for LLM debugging and evaluation

AI Summary

Sets up observability and evaluation for LLM applications using Arize Phoenix.

Install

Run in your terminal

npx claudepluginhub a5c-ai/babysitter

Tool Access

This skill is limited to using the following tools:

ReadWriteEditBashGlobGrep

Supporting Assets

View in Repository

README.md

Skill Content

Phoenix Arize Setup Skill

Capabilities

Set up Phoenix local server
Configure tracing instrumentation
Design evaluation experiments
Implement embedding visualizations
Set up retrieval analysis
Create custom evaluations with LLM-as-judge

Target Processes

llm-observability-monitoring
agent-evaluation-framework

Implementation Details

Core Features

Tracing: OpenTelemetry-based LLM traces
Evals: LLM-as-judge evaluations
Embeddings: Visualization and drift detection
Retrieval: RAG quality analysis
Datasets: Experiment management

Instrumentation

OpenAI auto-instrumentation
LangChain instrumentation
LlamaIndex instrumentation
Custom span creation

Configuration Options

Phoenix server setup
Trace sampling
Evaluation metrics
Embedding models
Export settings

Best Practices

Comprehensive instrumentation
Regular evaluation runs
Monitor embedding drift
Analyze retrieval quality

Dependencies

arize-phoenix
openinference-instrumentation-openai

Links

GitHub Stats

21 forks

Updated 1 month ago

Actions

Similar Skills

prompt-lookup

Activates when the user asks about AI prompts, needs prompt templates, wants to search for prompts, or mentions prompts.chat. Use for discovering, retrieving, and improving prompts.

prompts.chat

151.0k

skill-lookup

Search, retrieve, and install Agent Skills from the prompts.chat registry using MCP tools. Use when the user asks to find skills, browse skill catalogs, install a skill for Claude, or extend Claude's capabilities with reusable AI agent components.

prompts.chat

151.0k

Hook Development

10 files

This skill should be used when the user asks to "create a hook", "add a PreToolUse/PostToolUse/Stop hook", "validate tool use", "implement prompt-based hooks", "use ${CLAUDE_PLUGIN_ROOT}", "set up event-driven automation", "block dangerous commands", or mentions hook events (PreToolUse, PostToolUse, Stop, SubagentStop, SessionStart, SessionEnd, UserPromptSubmit, PreCompact, Notification). Provides comprehensive guidance for creating and implementing Claude Code plugin hooks with focus on advanced prompt-based hooks API.

plugin-dev

75.3k