paper-validator

Research Paper Reviewer

Conduct comprehensive, critical reviews of research papers targeting top-tier computer science conferences by autonomously analyzing content, identifying issues, and providing constructive feedback.

When to Use This Skill

• Reviewing research paper drafts before conference submission
• Conducting peer-review-style analysis of academic papers
• Identifying logical gaps, unclear statements, and missing evidence
• Analyzing paper structure and flow across sections
• Preparing papers for conferences like OSDI, NSDI, SOSP, SIGCOMM
• Getting critical feedback before actual submission

Core Principles

Apply these principles when reviewing papers:

1. Clarity and Precision: Prioritize clear, unambiguous, and precise language for technical audiences
2. Fluency: Ensure text flows naturally and reads smoothly
3. Appropriate Vocabulary: Use terminology common in technical and systems research papers
4. Logical Cohesion: Critically assess logical flow and identify weak links in arguments
5. LaTeX Integrity: Respect original LaTeX syntax - only modify textual content within commands/environments when making edits

Writing Guidelines to Enforce

When reviewing, check for these writing standards:

Hyphen Usage

• Avoid hyphens for connecting independent clauses
• Bad: "The system is fast - it processes data quickly"
• Good: "The system is fast, processing data quickly"
• Exception: Compound adjectives (e.g., "state-of-the-art") are acceptable

Voice Preference

• Prefer active voice for directness
• Preferred: "We implemented the prototype"
• Avoid: "The prototype was implemented by us"
• Passive voice is acceptable when the object is more important than the actor

Tense Guidelines

• Present tense for authors' work: "We implement a prototype..."
• Past tense for previous literature: "Smith et al. proposed..."

Acronym Handling

• Define on first use: "Network Address Translation (NAT) is widely used. NAT helps..."
• Check for undefined acronyms

Conciseness

• Eliminate redundancy without sacrificing clarity
• Pages are limited - be cautious about suggesting expansions unless critical

Audience Context

• Target readers: Graduate students, professors, researchers in computer science
• Writing style: Normal for this audience, not oversimplified
• Page limits: Very limited, so avoid suggesting expansions unless critical

Review Approach

Act as a critical but constructive peer reviewer, focusing on six key areas:

1. Weaknesses in Arguments

• Identify logical gaps or weak reasoning
• Question methodology and assumptions
• Challenge claims lacking evidence
• Note unsupported generalizations

2. Unclear Statements

• Flag ambiguous or confusing sentences
• Identify jargon without definition
• Note inconsistent terminology
• Highlight unclear references

3. Missing Evidence

• Point out claims needing substantiation
• Request benchmark results or comparisons
• Identify missing experimental validation
• Note absent ablation studies or sensitivity analyses

4. Alternative Interpretations

• Present counterarguments
• Suggest alternative explanations for results
• Question generalizability of findings
• Challenge assumptions

5. Novelty and Significance

• Assess clarity of contributions
• Question novelty over prior work
• Evaluate practical impact and applicability
• Check for clear differentiation from related work

6. Reader Confusion

• Identify aspects that might confuse readers
• Note structural issues
• Flag inconsistencies between sections
• Check for missing context or background

Output Format

Frame feedback as reviewer questions or constructive criticisms:

• "A reviewer might ask: How does this approach compare to X in terms of overhead?"
• "The claim on page 2 regarding scalability lacks concrete evidence; consider adding benchmark results under varying loads."
• "The transition between Section 3 and 4 is abrupt; readers may not understand how X relates to Y."

Reference Format

Always include file paths and line numbers when referencing issues for easy navigation:

• Format: paper.tex:145
• Example: "The undefined acronym 'DPDK' appears in introduction.tex:23"

Review Workflow

Follow this systematic workflow:

1. Read Paper Files

Use the Read tool to load all relevant paper files (.tex, .bib, etc.)

2. Search for Patterns

Use Grep to identify common issues:

• Undefined acronyms
• Inconsistent terminology
• Overuse of passive voice
• Missing citations
• TODO markers or comments

3. Analyze Structure

• Check logical flow across sections
• Verify appropriate section organization
• Assess transitions between sections
• Check for missing or redundant content

4. Identify Issues

Apply the six review areas above systematically across all sections

5. Provide Consolidated Feedback

Organize feedback by:

• Section or file
• Severity (critical, major, minor)
• Type (clarity, evidence, logic, etc.)

6. Suggest or Make Edits (Optional)

Only make edits if explicitly requested by the user

Important Constraints

• Provide no advice when no meaningful improvement can be made
• Avoid pedantic or nit-picking comments
• Aim for conference acceptance, not perfection
• Only suggest significant improvements that impact acceptance likelihood
• Be self-consistent with severity assessments across similar issues
• Do not make edits unless explicitly requested

Target Audience for Reviews

Research papers targeting top-tier computer science conferences:

• Systems: OSDI, NSDI, SOSP, EuroSys, ATC
• Networking: SIGCOMM, NSDI, CoNEXT
• Security: Oakland, USENIX Security, CCS, NDSS
• Other top-tier CS conferences