dojowatch

DojoWatch — AI-Native Visual Regression Testing

Open-source visual regression testing that uses AI to tell you what changed, why, and how to fix it — by Dojo Coding

DojoWatch is a Claude Code plugin and GitHub Actions CI tool that catches visual regressions before they reach production. Unlike Percy, Chromatic, or Applitools — it costs nothing. It uses the AI models you already have.

---

The Problem

Visual regressions reach production because:

• No one tests visually. Manual QA can't keep up with multiple deploys per day.
• Commercial tools are expensive. Percy starts at $500/mo. Chromatic at $149/mo. Applitools at $1K+/mo. For an early-stage team, that's unjustifiable.
• Pixel-diff tools generate noise, not insight. A red overlay of 10,000 changed pixels doesn't tell you what broke or how to fix it.

The Insight

Claude Code is already multimodal. It can look at a screenshot and read your source code in the same context. It doesn't just see that pixels changed — it can trace the regression to the exact CSS property on the exact line.

No API keys. No per-screenshot pricing. No context-switching to a separate dashboard.

The Solution

DojoWatch captures screenshots with Playwright, pre-filters with pixelmatch (to avoid wasting tokens on unchanged pages), and uses AI vision to classify every visual change:

• REGRESSION — an unintended visual change (bug). Severity: high / medium / low.
• INTENTIONAL — a deliberate change (new feature, design update).
• NOISE — insignificant rendering variance (sub-pixel anti-aliasing).

Locally, Claude Code is the AI engine — zero cost. In CI, Gemini handles batch analysis and posts PR comments.

---

How It Works

Source change → Playwright capture → pixelmatch pre-filter → AI analysis → Report
                                           │                       │
                                      SKIP (identical)     REGRESSION / INTENTIONAL / NOISE
                                      0 tokens, ~1ms      Claude reads images + source code

Locally (Claude Code plugin):

1. You run /vr-check
2. Playwright captures your routes at configured viewports
3. pixelmatch compares against baselines — identical screenshots are skipped instantly
4. Claude reads the changed screenshots and the source files that render them
5. You get natural-language analysis: what changed, why, severity, and a suggested fix
6. You can ask follow-up questions: "Why did the card shadow change?" — Claude reads the CSS and answers

In CI (GitHub Actions):

1. PR triggers the workflow
2. Same capture + prefilter pipeline
3. Gemini 3.1 Pro batch-analyzes screenshot pairs (20-30 per API call)
4. A structured PR comment appears with a regression table, diff thumbnails, and fix suggestions
5. High-severity regressions fail the check. Everything else is informational.

---

Pre-Filter: Zero False Negatives, Zero Wasted Tokens

The tiered pre-filter ensures you never pay for analysis you don't need — and never miss a regression:

Tier	Condition	Action	Token Cost
SKIP	SHA-256 identical OR 0 diff pixels	No AI call	0
FAST_CHECK	1-500 scattered pixels, no clusters	Low-depth analysis	~600
FULL_ANALYSIS	500+ pixels or spatially clustered	Full analysis	~2,400

The guarantee: Only byte-identical screenshots are skipped. Every other screenshot — even a single changed pixel — is analyzed. The tiers control depth, not whether analysis occurs.

---

Installation

Claude Code Plugin (recommended)

claude plugin marketplace add DojoCodingLabs/dojowatch
claude plugin install dojowatch@dojowatch

That's it. Run /vr-init in any project to get started.

Requires: Claude Code with plugin support. Scripts use Playwright — install browsers with npx playwright install chromium.

Updating

claude plugin marketplace update DojoCodingLabs/dojowatch
claude plugin update dojowatch@dojowatch

Restart your Claude Code session after updating.

For CI Only

Clone or add as a git submodule. CI scripts run standalone via npx tsx.

---

Quick Start