iFixAi Diagnostic

iFixAi

The diagnostic for AI operational misalignment

Catch your agent's mistakes and blind spots before the shit hits the fan.

Quick start • Two ways to run • Test your agent • Scoring • In the wild • Docs • Contributing

Recorded from a custom client build. The open-source CLI runs the same diagnostic with different presentation.

---

What it is

iFixAi detects AI operational misalignment before it damages your business. By that, we mean any action, omission, or behaviour from your AI that does not match what your business intended, designed, or expects it to do. The dangerous part is that this rarely shows up in your usual KPIs. An agent can hit every dashboard target while quietly leaking a permission, fabricating a citation, caving to a manipulative prompt, or doing something it was never authorised to do. Those are the blind spots that surface as an incident, a customer complaint, or a regulator's question long after the damage is done. iFixAi finds them first.

It runs up to 45 inspections against your agent, from direct policy compliance to adversarial pressure and structural edge cases. These come in two tiers: 32 core plus 13 extended. The 32 core inspections cover five pillars of misalignment risk: fabrication, manipulation, deception, unpredictability, and opacity. Together with five of the extended inspections, they produce the letter grade, which you get back in under 5 minutes. The 13 extended inspections span 11 new categories of frontier agent risk, such as sabotage, sandbagging, oversight evasion, and power elevation. Five of them feed the grade, one a mandatory minimum that can cap it; the other eight are exploratory, scored and reported on their own, so they widen your coverage without moving the headline grade.

Because the whole point is trust, iFixAi is honest about what it is. It is not a certification or a safety guarantee. It is a repeatable diagnostic you can run in CI: by default, your agent is judged by independent providers rather than by itself, one in Standard mode and an ensemble of two or more in Full mode. Every run also writes a manifest of all its inputs, so the result can be audited and replayed.

Two ways to run it

There are two ways to run iFixAi, and both run the same diagnostic underneath: the command-line tool (the CLI) or the Claude Code plugin. Either one tests any model and lets you choose who grades it. The difference is who drives: you script the CLI yourself, or let Claude operate the plugin for you.

	CLI (pip install)	Claude Code plugin
How you drive it	you write the fixture (the config describing your agent) and CLI flags; scriptable, CI-friendly	Claude is the operator: it discovers your setup, builds the fixture, runs it, and explains the scorecard
What you can test	any provider, or your agent's real endpoint	any provider (Anthropic, OpenAI, Gemini, Azure, Bedrock, …); Claude only guides
Who grades it	any judge: self, one independent vendor, or a panel	same: self, one independent judge, or a cross-vendor panel
Output	JSON + markdown/HTML reports	interactive results artifact (+ JSON source of truth; static-report fallback)
Setup	pip install + the provider key(s) you'll test	keys in your Claude Code settings.json env; the engine self-provisions
Best for	CI, automation, audit-ready batch runs	a guided, explained run with discovery and an interactive scorecard