Help us improve
Share bugs, ideas, or general feedback.
From truefoundry
Configures TrueFoundry AI Gateway for unified OpenAI-compatible LLM access, provider account integrations, content safety guardrails, and request observability (traces, costs, errors).
npx claudepluginhub truefoundry/skills --plugin truefoundryHow this skill is triggered — by the user, by Claude, or both
Slash command
/truefoundry:gatewayThis skill is limited to the following tools:
The summary Claude sees in its skill listing — used to decide when to auto-load this skill
> Routing note: For ambiguous user intents, use the shared clarification templates in [references/intent-clarification.md](references/intent-clarification.md).
references/access-management.mdreferences/api-endpoints.mdreferences/calling-models.mdreferences/cli-fallback.mdreferences/cli-reference.mdreferences/guardrail-providers.mdreferences/guardrails-setup.mdreferences/integrations.mdreferences/intent-clarification.mdreferences/monitoring.mdreferences/observability.mdreferences/prerequisites.mdreferences/provider-templates.mdreferences/schemas/budget-limiting.mdreferences/schemas/guardrail-config.mdreferences/schemas/mcp-server.mdreferences/schemas/provider-account.mdreferences/schemas/rate-limiting.mdreferences/schemas/virtual-model.mdreferences/span-attributes.mdProvides expert guidance for Vercel AI Gateway configuration: model routing, provider failover, cost tracking, unified API for multiple AI providers like OpenAI, Anthropic, Gemini.
Integrates codebase with TrueFoundry AI Gateway: scans LLM calls, MCP configs, credentials; diffs against gateway config; generates migration plan; applies changes; verifies routing end-to-end.
Configures Azure API Management as an AI gateway for models, tools, and agents with semantic caching, token limits, content safety, rate limiting, jailbreak detection, and backend integration.
Share bugs, ideas, or general feedback.
Routing note: For ambiguous user intents, use the shared clarification templates in references/intent-clarification.md.
Configure and operate TrueFoundry's AI Gateway: unified OpenAI-compatible LLM access, provider account integrations, content safety guardrails, and request monitoring/observability.
platform skill (Status Check section)mcp-servers skillplatform skill (Secrets section)observability skillobservability skillStart from the official template: truefoundry/custom-guardrails-template. Build on top of it, then deploy.
Your App -> AI Gateway -> OpenAI / Anthropic / Azure / Self-hosted vLLM / etc.
^
Unified API + Auth + Rate Limiting + Routing + Logging
Key benefits: Single endpoint for all models, one API key (PAT/VAT), OpenAI-compatible, rate limiting, budget controls, load balancing with fallback, guardrails, and full observability.
{TFY_BASE_URL}/api/llm
PAT (Personal Access Token): Dashboard -> Access -> Personal Access Tokens. For development.
VAT (Virtual Access Token): Dashboard -> Access -> Virtual Account Tokens. For production (not tied to a user, supports granular model access).
tfy-secret:// references, never raw values.tfy-secret:// URI. Or have them set it via ! export TFY_API_KEY=... so it stays in the shell.Verify tfy login is complete. If missing, stop and use truefoundry-onboard.
Set TFY_API_SH for direct API calls:
TFY_API_SH=~/.claude/skills/truefoundry-gateway/scripts/tfy-api.sh
For read-only questions, go straight to the API. Do not explore CLI subcommands — they don't exist. See references/cli-reference.md.
| User asks | Single call |
|---|---|
| What models/providers are attached? | $TFY_API_SH GET /api/svc/v1/provider-accounts |
| What models can I call? | curl -s "${TFY_BASE_URL}/api/llm/models" -H "Authorization: Bearer ${TFY_API_KEY}" |
| What guardrails are configured? | $TFY_API_SH GET /api/svc/v1/gateway-guardrails-configs |
| Show recent gateway requests | $TFY_API_SH POST /api/svc/v1/spans/query '{"startTime":"...","dataRoutingDestination":"default","limit":20,"sortDirection":"desc"}' |
| Is the gateway reachable? | curl -s "${TFY_BASE_URL}/api/llm/health" |
After login is confirmed, the next step for any read question is the API call above — nothing else.
The gateway is OpenAI-compatible. Minimal example:
curl "${TFY_BASE_URL}/api/llm/chat/completions" \
-H "Authorization: Bearer ${TFY_API_KEY}" \
-H "Content-Type: application/json" \
-d '{"model": "openai/gpt-4o", "messages": [{"role": "user", "content": "Hello!"}], "max_tokens": 200}'
Or set environment variables for any OpenAI SDK:
export OPENAI_BASE_URL="${TFY_BASE_URL}/api/llm"
export OPENAI_API_KEY="<your-PAT-or-VAT>"
For complete SDK examples (Python, Node.js, streaming), supported APIs table, framework integrations (LangChain, LlamaIndex, Cursor), and routing/rate-limiting/budget configuration, see references/calling-models.md.
$TFY_API_SH GET /api/svc/v1/provider-accounts
Present as a formatted table with name, provider, type, and model count (from integrations array length).
platform skill, Secrets section)$TFY_API_SH POST /api/svc/v1/provider-accounts "$payload"Supported providers: OpenAI, AWS Bedrock, Google Vertex, Azure OpenAI, Groq, Together AI, Custom (any OpenAI-compatible), Self-Hosted, TrueFoundry.
tfy apply -f manifest.yaml --dry-run --show-diff
tfy apply -f manifest.yaml
Do NOT delegate gateway applies to a deployment skill. Gateway configs are applied inline with tfy apply.
Guardrails add content safety controls. Setup requires two steps:
Quick list: $TFY_API_SH GET /api/svc/v1/gateway-guardrails-configs
For full setup instructions, rule structure, API calls, and common patterns, see references/guardrails-setup.md.
Supported providers reference: references/guardrail-providers.md.
Query gateway request traces via the spans API. Requires either tracingProjectFqn or dataRoutingDestination (suggest "default" as starting point).
$TFY_API_SH POST /api/svc/v1/spans/query '{
"dataRoutingDestination": "default",
"startTime": "2026-05-15T00:00:00.000Z",
"limit": 20,
"sortDirection": "desc"
}'
Present results as formatted tables (time, model, status, tokens, cost, latency, user).
For all monitoring use cases (cost analysis, errors, model usage, user filtering, MCP tool calls, metadata filtering), filter types, response structure, and pagination, see references/monitoring.md.
$TFY_API_SH POST /api/svc/v1/llm-gateway/metrics/query '{
"startTs": "...", "endTs": "...",
"datasource": "modelMetrics",
"type": "distribution",
"aggregations": [{"type": "sum", "column": "costInUSD"}],
"groupBy": ["modelName"]
}'
For any gateway entity or policy:
tfy-secret:// for credentialstfy apply -f manifest.yaml --dry-run --show-difftfy apply -f manifest.yaml| Entity | Reference |
|---|---|
| Provider accounts | references/provider-templates.md |
| Virtual models / routing | references/calling-models.md (Virtual Models section) |
| Rate limiting | references/schemas/rate-limiting.md |
| Budget controls | references/schemas/budget-limiting.md |
| Guardrails | references/guardrails-setup.md |
| Observability tables | references/observability.md |
<success_criteria>
tfy-secret:// for all credentialsdataRoutingDestination or tracingProjectFqn asked before querying</success_criteria>
platform skill (Secrets section) -> then tfy-secret:// URImcp-servers skillobservability skill (Tracing section)API key (PAT/VAT) is invalid or expired. Check Authorization: Bearer <token> header.
Token lacks access to this model. PATs inherit user permissions; VATs only access explicitly selected models.
Model name not in gateway. Check exact name via dashboard -> AI Gateway -> Models or GET /api/llm/models.
Wait and retry (check Retry-After header). Request higher limits or use load balancing.
Upstream provider issue. Gateway auto-retries/fallbacks if routing is configured. Check provider status page.
User needs provider-account-manager role. Check collaborators on the provider account.
tfy-secret:// path cannot resolve. Verify format: tfy-secret://TENANT:SECRET_GROUP:SECRET_KEY. Use platform skill to check secret group exists.
The type query parameter on GET /api/svc/v1/provider-accounts does NOT filter. Fetch all and filter client-side.
Use a different name or update the existing account.
Verify: provider account created successfully, integration has correct model_types, collaborators include team:everyone or relevant users.
Check: time range is correct, dataRoutingDestination exists, filters aren't too restrictive, gateway has received requests.
Missing required parameter. Ensure you provide tracingProjectFqn or dataRoutingDestination, and a valid startTime in ISO 8601.