reviewer-2

Reviewer 2 🔪

Meet Reviewer 2 before they meet you. 在 Reviewer 2 找上你之前,先让他帮你挑一遍。

English · 中文 ↓

Point it at your paper draft. It simulates a full peer-review panel — the generous champion, the brutal Reviewer 2, and a novelty-hawk Area Chair — predicts the reviews you'll get, and hands you a prioritized, evidence-grounded fix list. Every criticism is pinned to a line in your draft. No invented flaws.

A Claude Code skill for researchers who'd rather get torn apart in private.

---

The problem

You already know the feeling. You submit. Three months later, Reviewer 2 writes the words that sink your paper:

"The baselines are weak and the improvements are not statistically significant. Reject."

The brutal part: most of those wounds were self-inflicted and fixable — a missing baseline, an un-controlled confound, an overclaim in the abstract, a number with no error bars. You just didn't have a hostile reader before the deadline.

reviewer-2 is that hostile reader. It runs the panel on your draft now, while you can still fix things.

What it does

You give it a draft (.pdf, .tex, .md, an arXiv link, or pasted text). It produces a red-team report:

1. Simulated reviews from a panel that genuinely disagrees with itself —
   • 🫶 R1, The Champion — finds your real contribution (and calibrates what's actually strong: where even R1 can't praise, you're weak).
   • 🔪 R2, The Methodological Skeptic — the one you're afraid of. Hunts weak baselines, missing ablations, confounds, cherry-picking, overclaiming, single-run "SOTA".
   • 🦅 R3 / AC, The Novelty Hawk — "isn't this just X + Y?" Demands what's actually new versus recent work.
2. An Area Chair meta-review — consensus weaknesses (the deadly ones), split opinions, the 2–3 decisive factors, and a predicted outcome (Reject / Borderline / Accept).
3. A pre-submission fix list — every weakness, deduped and sorted by impact × effort, each one telling you what to change, where, and whether you can realistically do it before the deadline.

Does it actually find real flaws? (a real run)

We pointed it at a real, recent arXiv preprint — a reasoning-efficiency benchmark — and let it pick the paper itself. The simulated R2 caught a textbook methodological flaw, pinned to a section:

🔪 R2: The headline claim is "models differ wildly in token usage." But the paper deliberately selects problems with high token-usage variance [§3.1, §4.1] — so the dramatic difference may be manufactured by the sampling criterion itself. That's circular. Needs a control on a random, unbiased sample.

That's not "consider adding more experiments." That's "your conclusion is baked into your sampling — see §3.1." — exactly the wound a real Reviewer 2 lands, found in minutes, before submission.

(We keep the paper anonymous on purpose. This tool is for red-teaming your own draft, not dunking on other people's.)

Two modes

• A — Red-team your own draft before submission → simulated panel + predicted outcome + fix list.
• B — Review someone else's paper when you're an assigned reviewer or helping your advisor → a fair, venue-formatted, submission-ready review you sign off on.

Grounded in real standards — rigorous and fair

Built on the official reviewer guidelines of NeurIPS, ICLR, and ACL Rolling Review — the same dimensions and rating scales real reviewers use. Crucially, every criticism is checked against ACL's official H1–H17 list of illegitimate critiques — "not novel" with no citation, "doesn't beat SOTA", "the method is too simple", "the authors should run extra experiment X", "limitations = weaknesses"… If a complaint is on that list, the tool drops it or demotes it to a gentle suggestion. That's the line between a rigorous reviewer and a toxic one — Reviewer 2 with the receipts, not the cheap shots.

And the checklist isn't hand-waved — it's distilled from 2,956 real ICLR 2024 reviews (public on OpenReview). The data is blunt: weak novelty, poor positioning, and overclaiming are what actually sink papers, while "doesn't beat SOTA" and "you should run more experiments" are common but barely move the score. In other words, the real data confirms the fairness firewall. → methodology & numbers

The one rule that makes it trustworthy

No naked criticism, and no invented flaws.