From truefoundry-gateway
Sets up OpenTelemetry tracing on TrueFoundry via Traceloop SDK. Creates projects, instruments Python/TypeScript code to capture LLM calls and custom spans.
npx claudepluginhub truefoundry/tfy-gateway-skills --plugin truefoundry-gatewayThis skill is limited to using the following tools:
> Routing note: For ambiguous user intents, use the shared clarification templates in [references/intent-clarification.md](references/intent-clarification.md).
references/api-endpoints.mdreferences/cli-fallback.mdreferences/cluster-discovery.mdreferences/container-versions.mdreferences/gpu-reference.mdreferences/health-probes.mdreferences/intent-clarification.mdreferences/manifest-defaults.mdreferences/manifest-schema.mdreferences/prerequisites.mdreferences/resource-estimation.mdreferences/rest-api-manifest.mdreferences/tfy-api-setup.mdscripts/tfy-api.shscripts/tfy-version.shAdds LangSmith tracing to Python/JS LLM apps via LangChain auto-tracing, traceable decorators, or OpenTelemetry; queries/export traces with langsmith CLI.
Adds Arize AX tracing to Python LLM applications via two-phase agent-assisted codebase analysis and instrumentation after user confirmation. Use for initial LLM observability with OpenTelemetry or OpenInference.
Instruments Python and TypeScript code with MLflow Tracing for observability in LLM apps, agents, retrieval, tools, and frameworks like LangChain, OpenAI, LangGraph.
Share bugs, ideas, or general feedback.
Routing note: For ambiguous user intents, use the shared clarification templates in references/intent-clarification.md.
Add OpenTelemetry-based tracing and observability to applications using TrueFoundry's tracing platform (powered by Traceloop SDK). Creates tracing projects, installs dependencies, and instruments code to capture LLM calls, workflows, and custom spans.
Set up tracing projects, install Traceloop SDK, and instrument Python or TypeScript applications for LLM call tracing and observability on TrueFoundry.
Run the status skill first to verify TFY_BASE_URL and TFY_API_KEY are set and valid.
When using direct API, set TFY_API_SH to the full path of this skill's scripts/tfy-api.sh. See references/tfy-api-setup.md for paths per agent.
Ask the user: "Do you already have a tracing project FQN, or should I create one?"
tfy_tracing_list_projects()
TFY_API_SH=~/.claude/skills/truefoundry-tracing/scripts/tfy-api.sh
# List tracing projects
$TFY_API_SH GET /api/ml/v1/tracing-projects
Ask for a project name, then create:
tfy_tracing_create_project(name="my-tracing-project")
# Create tracing project
$TFY_API_SH POST /api/ml/v1/tracing-projects '{"name": "my-tracing-project"}'
Save the returned project id for the next step.
Each tracing project can have multiple applications (e.g., "chatbot", "rag-pipeline").
tfy_tracing_create_application(project_id="PROJECT_ID", name="my-app")
# Create application under project
$TFY_API_SH POST /api/ml/v1/tracing-projects/PROJECT_ID/applications '{"name": "my-app"}'
Fallback: If any of these API endpoints return 404, the tracing API may have changed. Direct the user to create the tracing project via the TrueFoundry UI at
$TFY_BASE_URL→ Tracing section, then return here with the project FQN.
Scan the project to determine the language and LLM libraries in use:
requirements.txt, pyproject.toml, setup.py, Pipfile
openai, anthropic, langchain, llama-index, litellm, cohere, bedrock, vertexai, transformerspackage.json
openai, @anthropic-ai/sdk, langchain, @langchain/coreReport what was detected to the user before proceeding.
pip install traceloop-sdk
Also add traceloop-sdk to requirements.txt or the appropriate dependency file.
npm install @traceloop/node-server-sdk
Also add to package.json dependencies.
CRITICAL: Traceloop.init() MUST be called at the TOP of the entry point, BEFORE any LLM library imports. This is required for auto-instrumentation to work.
Add this to the very top of the entry point file (e.g., main.py, app.py):
# --- Traceloop init MUST be before any LLM imports ---
from traceloop.sdk import Traceloop
Traceloop.init(
app_name="<APP_NAME>",
api_endpoint=f"<TFY_BASE_URL>/api/otel",
headers={
"Authorization": f"Bearer <TFY_API_KEY>",
"X-TFY-TRACING-PROJECT-FQN": "<TRACING_PROJECT_FQN>",
},
disable_batch=False,
)
# --- Now import LLM libraries ---
# from openai import OpenAI
# from anthropic import Anthropic
# etc.
Replace placeholders:
<APP_NAME> — the application name (e.g., "my-chatbot")<TFY_BASE_URL> — from environment or .env<TFY_API_KEY> — from environment or .env<TRACING_PROJECT_FQN> — the tracing project FQN from Step 2Best practice: Read TFY_BASE_URL and TFY_API_KEY from environment variables:
import os
from traceloop.sdk import Traceloop
Traceloop.init(
app_name="<APP_NAME>",
api_endpoint=f"{os.environ['TFY_BASE_URL']}/api/otel",
headers={
"Authorization": f"Bearer {os.environ['TFY_API_KEY']}",
"X-TFY-TRACING-PROJECT-FQN": "<TRACING_PROJECT_FQN>",
},
disable_batch=False,
)
Add this to the very top of the entry point file (e.g., index.ts, app.ts):
// --- Traceloop init MUST be before any LLM imports ---
import * as traceloop from "@traceloop/node-server-sdk";
traceloop.initialize({
appName: "<APP_NAME>",
apiEndpoint: `${process.env.TFY_BASE_URL}/api/otel`,
headers: {
Authorization: `Bearer ${process.env.TFY_API_KEY}`,
"X-TFY-TRACING-PROJECT-FQN": "<TRACING_PROJECT_FQN>",
},
disableBatch: false,
});
// --- Now import LLM libraries ---
// import OpenAI from "openai";
// etc.
For applications with multiple logical steps (agents, RAG pipelines, etc.), offer to add decorators for better trace structure:
from traceloop.sdk.decorators import workflow, task, agent, tool
@workflow(name="rag_pipeline")
def run_pipeline(query: str):
context = retrieve(query)
return generate(query, context)
@task(name="retrieve_context")
def retrieve(query: str):
# retrieval logic
...
@task(name="generate_response")
def generate(query: str, context: str):
# LLM call
...
@agent(name="research_agent")
def research_agent(topic: str):
# agent logic
...
@tool(name="web_search")
def web_search(query: str):
# tool logic
...
import { withWorkflow, withTask, withAgent, withTool } from "@traceloop/node-server-sdk";
const runPipeline = withWorkflow({ name: "rag_pipeline" }, async (query: string) => {
const context = await retrieve(query);
return generate(query, context);
});
const retrieve = withTask({ name: "retrieve_context" }, async (query: string) => {
// retrieval logic
});
const generate = withTask({ name: "generate_response" }, async (query: string, context: string) => {
// LLM call
});
For high-traffic production apps, configure sampling to reduce trace volume:
from opentelemetry.sdk.trace.sampling import ParentBased, TraceIdRatioBased
Traceloop.init(
app_name="<APP_NAME>",
api_endpoint=f"{os.environ['TFY_BASE_URL']}/api/otel",
headers={
"Authorization": f"Bearer {os.environ['TFY_API_KEY']}",
"X-TFY-TRACING-PROJECT-FQN": "<TRACING_PROJECT_FQN>",
},
disable_batch=False,
sampler=ParentBased(root=TraceIdRatioBased(0.1)), # 10% sampling
)
import { ParentBasedSampler, TraceIdRatioBasedSampler } from "@opentelemetry/sdk-trace-base";
traceloop.initialize({
appName: "<APP_NAME>",
apiEndpoint: `${process.env.TFY_BASE_URL}/api/otel`,
headers: {
Authorization: `Bearer ${process.env.TFY_API_KEY}`,
"X-TFY-TRACING-PROJECT-FQN": "<TRACING_PROJECT_FQN>",
},
disableBatch: false,
sampler: new ParentBasedSampler({ root: new TraceIdRatioBasedSampler(0.1) }), // 10%
});
<success_criteria>
traceloop-sdk (Python) or @traceloop/node-server-sdk (TypeScript) is installedTraceloop.init() is placed at the top of the entry point, BEFORE LLM importsAuthorization and X-TFY-TRACING-PROJECT-FQN</success_criteria>
status skill to verify TFY_BASE_URL and TFY_API_KEYsecrets skill to store TFY_API_KEY as a secret instead of hardcodinglogs skill to debug if traces aren't appearingSee references/api-endpoints.md for the full Tracing API reference.
Check that TFY_API_KEY is valid and not expired.
Regenerate at $TFY_BASE_URL → Settings → API Keys.
1. Verify Traceloop.init() is called BEFORE LLM library imports — this is the #1 cause.
2. Check that api_endpoint ends with /api/otel (not /api/otel/).
3. Verify X-TFY-TRACING-PROJECT-FQN header matches the project FQN exactly.
4. Set disable_batch=True temporarily to force immediate export and check for errors.
5. Check application logs for OTLP export errors.
Run: pip install traceloop-sdk
Ensure you're installing in the correct virtual environment.
Traceloop.init() must be called BEFORE importing the LLM library.
Move the init call to the very top of your entry point file.
Add sampling — see Step 7 for ParentBased(TraceIdRatioBased) configuration.
Start with 10% sampling (0.1) and adjust based on needs.
The tracing API endpoints may differ on your TrueFoundry version.
Create the tracing project via the TrueFoundry UI instead:
$TFY_BASE_URL → Tracing → New Project
Then use the project FQN in your Traceloop.init() configuration.