MLflow Skills

MLflow Skills for Coding Agents

Turn your favorite coding agent into an LLMOps expert with MLflow skills.

Build, debug, and evaluate GenAI applications with confidence. These skills give your AI coding assistant deep knowledge of MLflow's tracing, evaluation, and observability features.

Works with any coding agent that support Skills, including Claude Code, Cursor, Codex CLI, Gemini CLI, and OpenCode.

Why MLflow Skills?

Building production-ready AI agents is hard. You need observability to understand what your agent is doing, evaluation to measure quality, and debugging tools when things go wrong. MLflow provides SDKs and best practices for all of these operations, and with skills we bring them directly into the environment where LLM agent development happens. Now you can go to your favorite coding agent and just ask:

• "Add tracing to my LangChain app" → Instruments your code automatically
• "Why did this trace fail?" → Analyzes spans, finds root causes, suggests fixes
• "Evaluate my agent's accuracy" → Sets up datasets, scorers, and runs evaluation
• "Improve my agent and verify your work" → Gives the coding agent reproducible and verifiable mechanism with MLflow eval to hill climb on quality
• "Show me token usage trends" → Queries metrics and analyze trends

---

Available Skills

Observability & Debugging

Skill	Description
instrumenting-with-mlflow-tracing	Instruments Python and TypeScript code with MLflow Tracing. Supports OpenAI, Anthropic, LangChain, LangGraph, LiteLLM, and more.
analyze-mlflow-trace	Debugs issues by examining spans, assessments, and correlating with your codebase.
analyze-mlflow-chat-session	Debugs multi-turn chat conversations by reconstructing session history and finding where things went wrong.
retrieving-mlflow-traces	Powerful trace search and filtering by status, session, user, time range, or custom metadata.

Evaluation & Metrics

Skill	Description
agent-evaluation	End-to-end agent evaluation workflow — dataset creation, scorer selection, evaluation execution, and results analysis.
querying-mlflow-metrics	Fetches aggregated metrics (token usage, latency, error rates) with time-series analysis and dimensional breakdowns.

Helping New Users

Skill	Description
mlflow-onboarding	Guides new users through MLflow setup based on their use case (GenAI apps vs traditional ML).
searching-mlflow-docs	Searches official MLflow documentation efficiently using the llms.txt index.

---

Installation

Using skills installer

npx skills add mlflow/skills

Direct Installation from Source

git clone https://github.com/mlflow/skills.git
cp -r mlflow-skills/* ~/.claude/skills/

Change the ~/.claude/skills/ directory to the appropriate location for your coding agent, e.g., ~/.codex/skills/ for Codex.

Project-Level Installation

Add skills to your project for team sharing:

cd your-project
git clone https://github.com/mlflow/skills.git .skills/mlflow
# Or as a submodule:
git submodule add https://github.com/mlflow/skills.git .skills/mlflow

---

Auto-Suggestion Hook (Optional)

The hooks/ directory contains a UserPromptSubmit hook that automatically detects MLflow-related patterns in your prompts and surfaces the right skill before the agent responds — no need to remember which skill does what.

Install the Hook

Step 1: Copy the hook somewhere permanent:

cp hooks/mlflow-suggest-hook.py ~/.claude/hooks/mlflow-suggest-hook.py

Step 2: Add it to your Claude Code settings (~/.claude/settings.json):

{
  "hooks": {
    "UserPromptSubmit": [
      {
        "type": "command",
        "command": "python3 ~/.claude/hooks/mlflow-suggest-hook.py"
      }
    ]
  }
}

Step 3: Start a new session. When you ask something like "Add tracing to my app", you'll see:

💡 Use the `instrumenting-with-mlflow-tracing` skill to add MLflow tracing.

See hooks/README.md for the full keyword-to-skill mapping and troubleshooting.

---

Quick Examples

Instrument Your App with Tracing

> Add MLflow tracing to my OpenAI app

The coding agent will:
1. Detect your LLM framework
2. Add the right autolog call
3. Configure experiment tracking
4. Verify traces are being captured

Debug a Failed Trace

> Analyze trace tr-abc123 — why did it return the wrong answer?