trustabl

Trustabl is a static analyzer for agent reliability. It parses an agent-SDK repository (Claude Agent SDK, OpenAI Agents SDK, Google ADK, MCP, LangChain / LangGraph, CrewAI, AutoGen / AG2, Pydantic AI, and the Vercel AI SDK), models the tools, agents, subagents, skills, slash commands, and plugin manifests it declares, and checks them against a catalog of reliability and safety rules. It reports the weaknesses it finds — each with an explanation, a suggested fix, and a confidence score — as a human-readable summary, JSON, or SARIF 2.1.0, plus a per-surface reliability score and a CI-friendly exit code. It ships as a single Go binary with no hosted service: it runs as a CLI, or as a local stdio MCP server (trustabl mcp) that exposes the same scan to MCP clients without opening a network port.

The rest of this document explains what Trustabl reasons about and how the scan works, then covers building and running it. For the full implementation reference see ARCHITECTURE.md; for the at-a-glance SDK coverage matrix see COVERAGE.md.

What it analyzes — the five-scope model

Trustabl does not treat a repository as one undifferentiated blob. Every rule is classified into exactly one of five scopes, and each scope receives a different typed input:

• tool — fires once per tool definition. Input: a ToolDef (a @function_tool / @tool / @claude_tool function, a Claude TS tool(name, description, schema, handler) factory call, a FunctionTool(fn) ADK wrapper, an @server.tool MCP registration, or a bare shell-invoking function) plus its parsed file. Catches a missing docstring, an HTTP call with no timeout, untyped parameters, or an unnormalized path flowing into open(). (Hosted tools like WebSearchTool() are agent-scope edge data, captured as HostedToolDef, not ToolDef.)
• agent — fires once per agent declaration. Input: an AgentDef — a Python Agent(...) / SandboxAgent(...) / AgentDefinition(...) call, a Claude TS typed-const AgentDefinition, a Claude TS sub-agent inline in options.agents, or the Claude TS query(...) main-thread agent (QueryMainAgent) — with every constructor kwarg captured and its edges to tools, handoffs, and guardrails resolved. Catches an agent with shell tools and no input_guardrails, tool_use_behavior="stop_on_first_tool" paired with filesystem-touching tools, or a main-thread agent with unrestricted allowedTools.
• subagent — fires once per Claude Code subagent markdown declaration. Discovery is hybrid: canonical .claude/agents/*.md (any path depth, monorepo-safe) PLUS a frontmatter-shape fallback over all markdown files (gated on name + tools/model) that catches flat-collection repos which ship subagents under categories/*.md, plugins/<x>/agents/*.md, or similar layouts. Input: a SubagentDef parsed from frontmatter — name, description, tools[] (verbatim) + ToolGrants[] (parsed permission grammar), disallowedTools, model, permissionMode (incl. bypassPermissions), mcpServers, skills, isolation, hasHooks. Catches a subagent granted the built-in Bash tool despite a read-only description (CSDK-110). Subagent presence alone contributes claude_agent_sdk to SDKsDetected, so the Claude pack loads and CSDK-110 fires on pure-markdown subagent collections.