Issue Categories
Review the code for the following types of issues:
commit_message_mismatch
- The diff must completely fulfill the user's request.
- Look for incomplete implementations:
- When the user asks for changes 'throughout', 'everywhere', or 'all', verify ALL instances are modified
- If multiple changes are requested, ensure each one is fully implemented
- Check that fixes are applied to all occurrences of a pattern, not just some
- Look for scope mismatches:
- Changes only in initialization when they should apply during execution
- Modifications to only one file when multiple files need updates
- Partial refactoring that leaves related code unchanged
- Look for unauthorized changes:
- Configuration changes (linting, build, test settings) not requested
- New features or options beyond the request
- Changes to unrelated code
- Binaries, compiled files, dependencies, or build artifacts that should not be in version control
- Look for unintended removals:
- Removal of project-specific configuration or settings that should be preserved
- Deletion of functionality that is still needed
- Loss of necessary entries when replacing configuration files
- Look for unintended side effects:
- Changes that affect code paths or functionality beyond what was requested
- Modifications that impact how existing features work in ways not mentioned in the request
Exceptions:
- Minor refactors directly related to requested changes are acceptable.
documentation_implementation_mismatch
- The implementation should follow, in this priority order: 1. the user's request, 2. documentation existing in the code base, 3. existing code around it, 4. general best practices and common sense
- If the user's request conflicts with the state of documentation in the code base, the documentation in the code base should be updated to reflect the new user request.
Examples:
- The docstring of a class or function does not match what the class or function implements.
- The repository contains a README.md file with instructions that are not adhered to by the code.
- The diff implements significant new functionality, but existing documentation within the repository is not updated.
- Inline comments are not updated even though functionality was changed by the diff.
- Documentation contains outdated code snippets or commands that need to be updated because of the changes made by the diff.
Exceptions:
- TODOs/FIXMEs that are not implemented yet are not considered a documentation mismatch.
incomplete_integration_with_existing_code
- The diff should follow existing architectural and organizational patterns in the codebase:
- If the codebase uses a modular structure with separate files for classes/components, new classes should follow the same pattern
- If the codebase organizes code in specific directories (e.g., src/, components/, utils/), new code should be placed accordingly
- If the codebase uses specific import/export patterns (e.g., relative vs. absolute imports), new code should use the same patterns
- The diff should integrate functionally with existing code by adding invocations, updating invocations, replacing code with newly defined functions or variables, removing duplicate code when a new piece replaces it, etc.
- Prefer using existing library/dependency APIs over custom implementations when the library provides (or will provide) the needed functionality.
Examples:
- The codebase uses absolute Python imports from the project root, but a new file uses relative imports.
- The codebase places classes into separate files under a source directory, but new classes are all added to an existing file.
- The diff implements a new function, but doesn't add any callsites for it
- A new optional parameter is added to a function to implement a requested feature, but existing callsites are not updated to make use of this parameter
- A named constant is introduced to replace a hard-coded inline literal, but existing code is not updated to make use of the new constant everywhere
- Custom code is added to implement functionality that an existing external library already provides or will provide in newer versions.
user_request_artifacts_left_in_code
- Comments should describe what the code does, not how it was changed.
- Flag comments that reference the change process: '# Changed from X to Y', '# Updated to print less'
- Flag comments that mention fixing or addressing issues: '# Fixed bug where...', '# This addresses...'
- Flag documentation written in past tense about modifications
- Acceptable comments explain current behavior without referencing changes:
- Acceptable: '# Multiply by 3x' instead of '# Reduced factor from 5x to 3x'
- Acceptable: '# Handle edge case' instead of '# Fixed edge case bug'
poor_naming
- File, class, function, function parameter, and constant names should follow the format and naming standards that are currently dominant in the code base (especially within the same file or folder), or the style guide if one exists.
- In the absence of existing code, common naming standards for the given programming language should be used.
- Function names should be descriptive of what the function does. A person reading the function name without seeing its implementation should be able to get a sense of its purpose.
- If any function parameter is being mutated by the function, that fact should be made clear in the name of the parameter (e.g. by appending
_out / Out to the name), unless it is clear that such name-based annotations would be against the existing naming style in the code base.
- Note that we don't impose any specific criteria on the length of names. If the existing code base uses many abbreviated names, new code should follow that. Or if it uses a lot of long, verbose names, this similarly should be followed.
- If a component's functionality is significantly changed, the name of the component should be updated to reflect the new functionality, if it is not already clear from the context.
Exceptions:
- Short names for local variables are usually okay.
repetitive_or_duplicate_code
Repetitive or duplicate code.
Examples:
- A non-trivial calculation or piece of logic is repeated in multiple places within a file.
- New code is introduced by the diff to accomplish a certain functionality, but there is an existing function in the code base that already implements the same functionality, or could be easily generalized to accomplish the desired functionality.
- A file is duplicated (make exceptions for cases where duplication may be necessary such as test files).
- A significant amount of code is introduced which duplicates functionality from standard or well-known libraries.
- Multiple functions format or build the same string or data structure in the same way without using a shared helper function or test fixture.
Exceptions:
- Do not flag duplication between legacy and new implementations when the codebase is clearly undergoing a migration or maintaining multiple versions for compatibility.
- Do not flag duplication across different architectural layers or modules when the duplication serves to maintain proper separation of concerns.
refactoring_needed
- Functions that have gotten long (> 50 lines) and are mixing multiple concerns and/or combining several different steps should be broken up. (Typically by using helper functions and/or separate classes to encapsulate individual concerns.)
- Classes or files that are combining different concerns should be broken up, such that each class / file only deals with one primary concern.
- Note: we don't impose any minimal or maximal length on a class or file. Classes and files are ok to be long, as long as they only deal with a single concern.
Examples:
- New functionality that is orthogonal to the existing functionality in a function is inserted into the existing function's body instead of being separated out into its own function.
- A class mixes two different use cases that could be separated into two classes.
test_coverage
- If the diff introduces significant new functionality, and the code base has existing unit and/or integration tests, new tests should be added to cover the new functionality.
- If the diff changes the behavior of existing functionality that is covered by automated tests, those tests should be updated to reflect the new behavior.
- If the diff contains a bug fix, and the code base has existing unit and/or integration tests, a regression test should be added for the bug.
Exceptions:
- Syntactical or logical issues in tests will be raised in other issue types and do not belong in this category.
resource_leakage
- Focus on system resources that require explicit cleanup: file handles, network connections, database connections, memory allocations, and similar OS-level resources.
- These resources must be reliably freed even if exceptions occur.
- For these system resources, cleanup should use try/finally blocks, context managers (with statements), or RAII patterns.
- Also look for reference management issues: objects being cleaned up while still holding references elsewhere, or cleanup operations (like garbage collection) called before removing all references to the object.
Examples:
- A file or socket connection is opened but not reliably closed.
- A database transaction is started but not committed or rolled back in all code paths.
- Memory is allocated but not freed (in languages with manual memory management).
- An object's cleanup method triggers garbage collection while the object is still referenced in a global data structure, preventing proper cleanup.
Exceptions:
- Animation loops, timers, and intervals that are controlled by boolean flags or cleared by ID are not resource leaks if they have proper stop mechanisms.
- Event listeners that are meant to persist for the lifetime of the application.
- Resources that are automatically cleaned up by garbage collection (unless they hold system resources).
dependency_management
- Check all import statements in new or modified files. If new code imports a library or package that is not part of the language's standard library, verify that the dependency is listed in the repository's dependency/requirement files (e.g., requirements.txt, pyproject.toml, package.json, Gemfile, etc.).
- If the diff removes the last remaining use of an external library or package, the dependency and/or requirement files in the repository should be updated to no longer include the library.
- If the codebase uses a dependency for some functionality, the diff should avoid introducing other packages that provide the same functionality, unless there is a good reason to do so (e.g. the new package is significantly better maintained, has better performance, or is more secure).
Exceptions:
- Do not raise issues related to package versions or pinning unless it is a critical issue.
insecure_code
- Look for hard-coded secrets such as API keys, passwords, tokens, or credentials in the diff.
- Check for variable names containing: 'token', 'key', 'secret', 'password', 'credential', 'auth'
- Look for string literals that appear to be:
- API keys or tokens (long alphanumeric strings, often 20+ characters)
- Hexadecimal strings that could be tokens or keys
- URLs with embedded credentials (e.g., 'https://user:password@host')
- Connection strings with passwords
- Flag any credentials or secrets that should be loaded from environment variables or configuration files instead.
Examples:
- A variable named
api_key or auth_token is assigned a hard-coded string value.
- A connection string contains a hard-coded password.
- A long hexadecimal string is assigned to a variable with 'token' in its name.
- An API request includes a hard-coded authentication header value.
fails_silently
Code that fails silently is code that ignores errors without reporting them.
Examples:
- Overly broad exception handlers (e.g. bare 'except' or 'except Exception') that catch errors and continue execution without handling, logging, or re-raising them.
- The return value of a function that returns an error value in case of a failure is not checked by the caller.
Exceptions:
- There are certain cases where broad exception handlers are acceptable, such as in an executor class or in a main loop that iterates over several tasks. Such cases should still properly log and report the errors.
- Do not raise issues related to potential program crashes.
instruction_file_disobeyed
Explicit instructions in files such as .claude.md, CLAUDE.md, and AGENTS.md MUST be obeyed.
Examples:
- CLAUDE.md requests the use of single quotes only, but double quotes are used.
- AGENTS.md requests that new versions be created on every database update, but a database entry is modified directly.
- .claude.md says to always run the tests after making changes, but the agent did not run the tests.
Exceptions:
- Instructions in the closest file above a location take precedence. For example, when considering a file foo/bar.py, foo/CLAUDE.md takes precedence over CLAUDE.md.
- Instructions only apply to the subtree below the file. For example, when considering a file foo/bar.py, foo/baz/CLAUDE.md does not apply.
- Applicable instructions should ONLY be contravened in the case of explicit user request--but if the user does explicitly request something counter to the instruction files, this should not be reported as a disobeyed instruction file.
abstraction_violation
- Code that breaks established abstraction boundaries within the codebase.
- Look for:
- Direct access to internal data structures of classes/modules that should be encapsulated
- Bypassing public APIs to manipulate state or access internal functionality
- Mixing of concerns that should be separated by layers or modules
- Violating private vs. public interfaces (e.g., accessing private attributes or methods from outside their defining class/module)
Examples:
- A Python function directly accesses a private attribute, variable or function (prefixed with an underscore) from a different class or file.
- A module modifies the internal state of another module directly instead of using and/or adding public API functions.
Exceptions:
- Unit tests that need to access internal state for verification purposes.
logic_error
- Logic errors are flaws in the reasoning or flow of the code that would cause incorrect behavior.
- Look for: off-by-one errors in loops or array indexing, incorrect conditional logic (wrong operators, inverted conditions), variable assignments that overwrite needed values, incorrect order of operations, missing or incorrect loop termination conditions, algorithms that don't match their intended purpose.
- Look for missing, incorrect, or incomplete parameters to function/API calls that will cause the function to behave differently than intended (e.g., missing event masks, wrong flags, omitted required options).
- Pay special attention to control flow issues: early returns or breaks that prevent intended functionality from executing, functions that exit before completing their stated purpose, conditions that prevent code paths from being reached when they should be.
- Do not flag issues that are not clearly incorrect, for example it's possible code is implemented in a suboptimal way, this is not an issue unless it is explicitly stated that the code should be optimal or implemented in a certain way.
runtime_error_risk
- Code patterns that are very likely to cause runtime errors during execution.
- Check for version compatibility issues: usage of function parameters, APIs, or language features that are only available in specific versions of the language, standard library, or external dependencies (e.g., a keyword argument added in Python 3.10 will cause TypeError on Python 3.8/3.9).
- Look for: potential null/None pointer dereferences, array/list access with potentially invalid indices, division by zero possibilities, file operations without existence checks, network/IO operations without timeout or error handling, infinite loop conditions, memory allocation issues.
- Check string encoding/decoding operations: calls to .encode() or .decode() without error handling (try/except or 'errors' parameter) that could raise UnicodeEncodeError or UnicodeDecodeError, especially when processing untrusted or streamed data.
- Look for operations with global side effects that could cause problems: os.chdir() without proper restoration, modifying global state in ways that affect other code, operations that are not thread-safe when concurrency is present.
- Only flag issues where there is clear evidence the code will fail or cause serious problems. Avoid speculating about potential issues in well-established language patterns or standard library usage.
incorrect_algorithm
- Code that implements an algorithm incorrectly for its stated purpose.
- Look for: sorting algorithms with wrong comparison logic, search algorithms with incorrect termination, mathematical calculations with wrong formulas, data structure operations that don't maintain invariants, algorithms that don't handle edge cases (empty inputs, single elements).
- Only flag issues that are clearly incorrect for the stated purpose of the algorithm, and describe the problem and correction in detail.
error_handling_missing
- Missing error handling for operations that could reasonably fail.
- Look for: file I/O without exception handling, network requests without timeout/retry logic, user input processing without validation, external API calls without error checking, database operations without transaction handling.
- Only flag issues that are clearly incorrect, and avoid flagging issues where it is not a big problem (e.g. file I/O in a script may not need flagging while missing error handling for file I/O in a long running or production systems should have error handling).
async_correctness
- Issues specific to asynchronous or concurrent code correctness.
- Look for: missing await keywords on functions that are clearly async (defined with 'async def' or returning coroutines/Promises), improper async context manager usage, race conditions in async code, deadlock potential, shared state access without proper synchronization.
- For threading issues: threads or background tasks that are created but never started, joins without corresponding starts.
- Be careful not to flag decorators or wrappers as requiring await unless you can verify they actually make the function async. Only flag clear cases where an async function is called without await or a thread/task is created but never started.
type_safety_violation
- Code that violates type safety expectations or could cause type-related runtime errors.
- Look for: incorrect type assumptions, missing type checks before operations, unsafe type casting, attribute access on potentially None values.
- Check for return type violations: functions that return values inconsistent with their declared return type annotations (e.g., returning None when the type annotation specifies a non-optional type, returning wrong tuple element types).
correctness_syntax_issues
- The diff should not contain any syntax errors that would prevent the code from running.
- CAREFULLY CHECK INDENTATION: In Python and other indentation-sensitive languages, verify that all function definitions, class definitions, and code blocks maintain proper indentation levels. Dedenting a function body to the module level or similar indentation errors are critical syntax issues.
- Look for: broken indentation that would cause syntax errors, missing or mismatched brackets/braces/parentheses, references to files/classes/functions that don't exist, removal of code that is still being referenced elsewhere.
- Check function signatures match their usage: if a function is modified to return different values (e.g., a single value vs. a tuple), all call sites must be updated accordingly.
Examples:
- The diff breaks the indentation of a Python function or class, dedenting it incorrectly.
- Code references a file, class or function that does not exist, or removes a file, class or function that is definitely still being referenced.
- A function is changed to return a tuple of two values, but existing callers still expect only a single return value.
- A function's return statement is removed but callers still expect a return value.