Run comprehensive evaluations on AI agent setups to identify security vulnerabilities, redundancy, configuration issues, and quality problems. Includes deterministic static analysis with 43 rules, per-component rubrics, cross-type optimization checks, and LLM-based semantic reviews to produce lint reports and KEEP/REVIEW/REMOVE verdicts suitable for CI gates or deep audits.
Deep-evaluate a single skill with static analysis and qualitative review, both individually and in context of the full setup. Check if a skill is worth keeping, well-built, or redundant.
Run deterministic static analysis on the full agent setup (CLAUDE.md, skills, commands, hooks, agents, MCP configs). 68 rules + system-level analysis. No LLM, fast, CI-suitable.
Full qualitative review of the agent setup. Per-component rubrics, 21 cross-type checks, KEEP/REVIEW/REMOVE verdicts. Use for deep review, redundancy check, or quality assessment.
Deep security audit of the agent setup. Deterministic security rules (prompt injection, credential access, exfiltration, taint tracking, YARA, CVE) plus LLM semantic review.
Deep-evaluate a single skill with static analysis and qualitative issue detection, both individually and in context of the full setup. Use when the user wants to check if a specific skill is worth keeping, well-built, or redundant.
Run deterministic static analysis on the full agent setup (CLAUDE.md, skills, commands, hooks, agents, MCP configs). 68 rules + system-level analysis (token budget, trigger overlaps, dependencies). No LLM. Use when the user wants a fast lint check, CI gate, or structural health report.
Full qualitative review of the agent setup. Reads every file, applies per-component rubrics, runs 21 cross-type optimization checks, and produces KEEP/REVIEW/REMOVE verdicts. Use when the user wants a deep review, redundancy check, or quality assessment of their setup.
Deep security audit of the agent setup. Runs all deterministic security rules (prompt injection, credential access, data exfiltration, obfuscation, reverse shells, AST behavioral analysis, taint tracking, MCP permission analysis, tool poisoning, YARA signatures, CVE lookups) plus LLM-based semantic security review. Use when the user asks about security, safety, wants to audit their setup, or needs a pre-deployment security check.
Based on adoption, maintenance, documentation, and repository signals. Not a security audit or endorsement.
Evaluate AI code agent setups for best practices, redundancy, security, and cross-component issues.
Available as a CLI tool, a GitHub Action, a Claude Code plugin, and Cursor commands.
Supports Claude Code, Cursor, Copilot, Gemini CLI, and OpenCode projects. Auto-detects which tool(s) a project uses. Also discovers third-party modules installed via package managers.
Most tools test whether a skill produces correct output. This tool checks the setup itself: CLAUDE.md, GEMINI.md, AGENTS.md, skills, commands, hooks, MCP configs, agents, .cursor/rules/*.mdc, .cursorrules, .github/prompts/, .opencode/.
Cross-component analysis is the core differentiator. Most linters check files in isolation; harness-eval builds a component graph that traces data flows across skills, agents, hooks, and MCP servers. This catches threats like credential exfiltration chains (one skill reads secrets, another has network access) and confused deputy attacks (an agent disallows a tool but references a skill that uses the equivalent capability).
Beyond cross-component analysis, the security engine includes Python taint tracking, bash taint analysis, YARA signature scanning, CVE lookups via OSV.dev, and optional LLM-based semantic review.
Five commands, same engine:
| Command | What it does | LLM in CLI | LLM in Claude Code / Cursor |
|---|---|---|---|
lint | 74 deterministic rules + system analysis (token budget, trigger overlaps, dependencies). Fast, CI-suitable. Supports --format sarif for GitHub code scanning. | No | No |
review | Per-component rubric review with 0-3 scoring per dimension, 21 cross-type checks. KEEP/REVIEW/REMOVE verdicts. | Yes (API key) | Yes (in-session) |
security | All security rules + YARA + CVE lookups + semantic review. SAFE/CAUTION/UNSAFE. | Scan: no. Semantic review: --review flag | Yes (in-session) |
skill | Deep-evaluate one skill individually and in context of the full setup. | Lint: no. Rubric: --rubric flag | Yes (in-session) |
rules | List all available rules with ID, severity, target type, and description. Filter by --category or --target. | No | No |
Auto-detects which tool(s) a project uses and evaluates all discovered components.
| Assistant | What it discovers |
|---|---|
| Claude Code | CLAUDE.md, skills/, commands/, .claude/agents/, .claude/settings.json, .mcp.json |
| Cursor | .cursor/rules/*.mdc, .cursorrules, .cursor/commands/, .cursor/skills/, .cursor/hooks.json, .cursor/mcp.json |
| Copilot | .github/skills/, .github/prompts/, .github/agents/ |
| Gemini CLI | GEMINI.md, .gemini/commands/ |
| OpenCode | AGENTS.md, .opencode/commands/, .opencode/agents/ |
| Third-party modules | .lola/modules/ (skills, commands, agents installed via package managers) |
Multi-tool projects are fully supported. When a project contains files for multiple assistants, all are discovered, deduplicated, and evaluated together.
See docs/INSTALL.md for all installation options, CI integration, and configuration (Available as a CLI tool, GitHub Action, Claude Code plugin, and Cursor commands)
npx claudepluginhub redhat-community-ai-tools/harness-eval --plugin harness-evalSet up the redhat-community-ai-tools Slack MCP server for use with Claude Code
Comprehensive Claude Code plugin providing expert-level skills for OpenShift cluster management, troubleshooting, and operations
Containerized development workflow using skipper — build, test, and run commands inside Docker/Podman containers with reproducible toolchains
Checks an AI-agent codebase against the OWASP Agentic Security Initiative (ASI) Top 10 risks. Use to evaluate an agent system security posture, run a compliance check, map controls to the 10 agentic risks, or generate an audit report.
Self-audit AI agent, tool, and MCP-server code for security and reliability misconfigurations with Trustabl, the static analyzer for the OpenAI Agents SDK, Claude Agent SDK, Google ADK, and MCP. Ships two skills (trustabl-scan and trustabl-enrich) and a subagent (trustabl) that together form a scan → enrich → review → apply pipeline.
Check how well your repo supports AI coding agents.
Battle-tested Claude Code plugin for engineering teams — 38 agents, 156 skills, 72 legacy command shims, production-ready hooks, and selective install workflows evolved through continuous real-world use
Lint agent configuration files (SKILL.md, CLAUDE.md, hooks, MCP) against 414 rules across 10+ AI tools
A configurable linter for agent skills, plugins, and AI coding assistant context