From posthog
Investigates LLM costs in PostHog: totals over time, breakdowns by model/provider/user/trace/dimension, token/cache-hit economics, regressions, dashboards, and alerts.
npx claudepluginhub anthropics/claude-plugins-official --plugin posthogThis skill uses the workspace's default tool permissions.
PostHog attaches per-call cost metadata to every `$ai_generation` and `$ai_embedding`
Tracks and optimizes LLM costs using Langfuse analytics, Metrics API, model routing, and budget alerts for spending analysis and anomaly detection in AI apps.
Investigate PostHog LLM analytics clusters to understand AI/LLM traffic patterns, compare behaviors, compute cost/latency metrics, and inspect individual traces.
Queries Elastic traces, metrics, and logs to monitor LLM/agentic performance, token/costs, quality, and orchestration. For LLM monitoring, GenAI observability, AI cost/quality.
Share bugs, ideas, or general feedback.
PostHog attaches per-call cost metadata to every $ai_generation and $ai_embedding
event at ingestion time. Every cost question reduces to an aggregation over those
two event types — the interesting variation is only in how you group, filter, and
compare.
This skill covers the common cost investigations: total spend, breakdowns (model, provider, user, trace, custom property), token and cache-hit analysis, regression debugging, and materializing results as insights, dashboards, or alerts.
| Tool | Purpose |
|---|---|
posthog:execute-sql | Ad-hoc HogQL for any cost aggregation — the workhorse of this skill |
posthog:query-llm-traces-list | List traces with rolled-up cost, token, and error metrics |
posthog:query-llm-trace | Cost breakdown of a single trace across all its events |
posthog:read-data-schema | Discover which custom properties exist for breakdowns |
posthog:insight-create | Materialize a cost chart as a saved insight |
posthog:dashboard-create | Bundle cost insights into a dashboard |
posthog:alert-create | Alert when cost crosses a threshold |
Three rules cover most of what goes wrong:
$ai_total_cost_usd for rollups, never the components. Components drop
request and web-search fees. The UI's cost cells sum $ai_total_cost_usd
over event IN ('$ai_generation', '$ai_embedding'); mirror that. Full
schema and rationale in cost properties.$ai_generation and $ai_embedding in cost queries
unless the project demonstrably does not use embeddings — missing them silently
under-counts. $ai_trace and $ai_span carry no rollup cost; some SDK
wrappers duplicate $ai_total_cost_usd onto $ai_trace so don't include
it in rollups or you'll double-count.$ai_total_cost_usd is set at ingestion via one of three paths (passthrough,
custom pricing, automatic lookup). When a cost looks wrong, read
$ai_cost_model_source first — see cost sources
for the precedence rules and a diagnostic query.
Cache-hit math depends on whether the provider reports cache tokens inclusively
or exclusively of $ai_input_tokens. Always branch on the per-event
$ai_cache_reporting_exclusive flag, never on provider name — see
cache accounting for the exclusive-vs-inclusive
formula.
distinct_id is the canonical user dimension. Customers often attach custom
properties (feature, tenant_id, workflow_name) — discover them with
posthog:read-data-schema before grouping. Don't guess names.
posthog:execute-sql
SELECT round(sum(toFloat(properties.$ai_total_cost_usd)), 4) AS total_cost_usd
FROM events
WHERE event IN ('$ai_generation', '$ai_embedding')
AND timestamp >= now() - INTERVAL 30 DAY
Every cost question is a variation of the same template — group by a dimension,
aggregate $ai_total_cost_usd. See breakdown patterns
for ready-to-run recipes:
When the user pastes a trace URL and asks about its cost, fetch the trace and surface the per-event breakdown:
posthog:query-llm-trace
{ "traceId": "<trace_id>", "dateRange": {"date_from": "-30d"} }
Sum $ai_total_cost_usd across the returned events, grouped by span name or
model, to show which step(s) drove the cost. The trace response already
includes totalCost as a convenience.
"Our LLM bill jumped — why?" is almost always one of: more calls, bigger prompts, a new model, or a change in cache-hit rate. Work through them in order — see regression debugging for the 5-step playbook.
After ad-hoc queries answer the question, persist them as insights, bundle
into a dashboard, or wire up alerts. See materializing
for ready-to-run JSON for posthog:insight-create, posthog:dashboard-create,
and posthog:alert-create.
https://app.posthog.com/llm-analytics/dashboardhttps://app.posthog.com/llm-analytics/traceshttps://app.posthog.com/llm-analytics/generationshttps://app.posthog.com/llm-analytics/usershttps://app.posthog.com/llm-analytics/traces/<trace_id>?timestamp=<url_encoded_iso>Always surface a UI link so the user can verify visually.
Provider reporting behavior (which tokens are inclusive vs exclusive, which costs show up where) shifts over time and can differ between SDK versions for the same provider. To avoid rot:
$ai_cache_reporting_exclusive,
$ai_cost_model_source) rather than hardcoded provider or model names.
Those flags are ingestion's resolved answer for the specific event and
are the right source of truth.$ai_total_cost_usd is always authoritative for rollups — prefer it
over summing components, which can drift as new cost categories are
added.posthog:docs-search for
"calculating costs" or "llm analytics" first rather than trusting a
hardcoded rule in this file.$ai_embedding alongside $ai_generation when summing cost; embeddings are cheap per-call but add up at scale$ai_total_cost_usd is missing or zero, read $ai_cost_model_source first: passthrough means the SDK supplied costs; custom means custom token prices; openrouter / manual mean automatic lookup; missing means the model wasn't matched (unusual custom model, fine-tune). Grep: countIf(properties.$ai_total_cost_usd IS NULL) per (model, source)distinct_id = properties.$ai_trace_id — some SDKs default distinct_id to the trace ID when no user is set$ai_generation + $ai_embedding events within a trace; summing on $ai_span gives zero. $ai_trace may carry $ai_total_cost_usd from some SDK wrappers — don't include it in rollups or you'll double-count. $ai_evaluation events also carry cost but are not part of the stock UI rollups; include them only when the user explicitly wants evaluation spend in the total$ai_cache_reporting_exclusive — branch on the event-level flag rather than on provider or model name. Provider behavior and SDK versions drift; the flag is ingestion's resolved answer for that specific event/llm-analytics/dashboard tiles already answer the question — re-creating them is churninsight-query; ad-hoc SQL is fine for one-offs but re-running it on every dashboard load is expensive