agent-validator

Don't just review the agent's code — put it through the gauntlet.

Agent Validator (formerly Agent Gauntlet) is a configurable “feedback loop” runner for AI-assisted development workflows.

You configure which paths in your repo should trigger which validations — shell commands like tests and linters, plus AI-powered local code reviews. When files change, Agent Validator automatically runs the relevant validations and reports results.

For AI reviews, it uses the CLI tool of your choice: Gemini, Codex, Claude Code, GitHub Copilot, or Cursor.

Features

• Agent validation loop: Keep your coding agent on track with automated feedback loops. Detect problems — deterministically and/or non-deterministically — and let your agent fix and Agent Validator verify.
• Local cross-agent code reviews: Enable one AI agent to automatically request code reviews from another. For example, if Claude made changes, Agent Validator can request a review from Codex — spreading token usage across your subscriptions instead of burning through one.
  • Multiple AI review adapters have been evaluated for quality and efficiency. Claude and Codex deliver optimal review quality with superior token efficiency. For detailed metrics, see Eval Results.
• Leverage existing subscriptions: Agent Validator is free and tool-agnostic, leveraging the AI CLI tools you already have installed.

Example Workflow

1. Claude implements a feature
2. Agent Validator reports linter failures and bugs detected by Codex reviewer agent
3. Claude fixes issues
4. Agent Validator reports linter issue remaining
5. Claude fixes issue
6. Agent Validator confirms all issues fixed

Comparison vs Other Tools

Agent Validator is not a replacement for AI pull request review tools. It provides real-time feedback loops for autonomous coding agents, combining deterministic static checks (build, lint, test) with multi-agent AI reviews in a single pipeline. This enables agents to iterate and self-correct until all checks and reviews pass, without human intervention.

Full comparison →

It is recommended to use Agent Validator in conjunction with spec-driven development tools. We believe it is the ideal implementation of the validation step in any Spec → Implement → Validate workflow.

Quick Start

Requirements

• Node.js (v18.0.0+), git
• For reviews: one or more supported AI CLIs (gemini, codex, claude, github-copilot, cursor). See CLI Invocation Details.

Installation & Setup

npm install -g agent-validator
agent-validator init

init detects your installed AI CLIs, creates .validator/config.yml with an empty config skeleton, and installs skills/hooks for your AI agent (Claude Code plugin, Copilot plugin, Cursor plugin, or Codex skills). Use --yes to skip prompts.

After init, run /validator-setup in your AI agent session to auto-discover your project's tooling and populate the config. See the Skills Guide for details.

Configuration Concepts

Agent Validator uses three core concepts:

• Entry points: Paths in your repository (e.g., src/) that Agent Validator monitors for changes.
• Checks: Shell commands that run when an entry point changes — things like tests, linters, and type-checkers.
• Reviews: AI-powered code reviews requested via CLI tools like Codex, Claude, or Gemini.

When you run Agent Validator, it detects which entry points have changed files and runs the associated checks and reviews.

Example Configuration

Checks and reviews are defined inline in config.yml. Here's a simplified real-world example:

base_branch: main
log_dir: validator_logs
allow_parallel: true

cli:
  adapters:
    github-copilot:
      allow_tool_use: false
      thinking_budget: low