Jentic API Scorecard
An OpenAPI document that passes validation isn't necessarily one an AI agent can use. Grammar is
one thing; semantic clarity, safety, and discoverability are another. The Jentic API Scorecard
scores your OpenAPI document against the
Jentic API AI Readiness Framework (JAIRF)
across six dimensions and returns a single grade — so you know exactly where to improve.
Table of contents
Quick start
-
Get a free key at the Jentic Scorecard API Keys page.
-
Score an OpenAPI document:
JENTIC_API_KEY=<your-key> npx @jentic/api-scorecard-cli score ./openapi.yaml
What it scores
Each OpenAPI document is evaluated across six lenses — small, targeted improvements in any of them
tend to produce outsized gains for both human developers and AI agents:
- Foundational Compliance (FC) — structural validity and conformance to OpenAPI itself.
- Developer Experience & Jentic Compatibility (DXJ) — documentation quality and how well the
OpenAPI document plays with downstream tooling.
- AI-Readiness & Agent Experience (ARAX) — semantic clarity and the context an LLM needs to
reason about each operation.
- Agent Usability (AU) — predictable, safe multi-step orchestration.
- Security (SEC) — declared auth schemes and trust boundaries.
- AI Discoverability (AID) — how easily an AI system can find and parse the OpenAPI document.
How it works
Scoring runs locally inside a Docker container in two phases. Analysis runs a battery of
validators and structural checks against the OpenAPI document to produce a set of diagnostics and
observations.
Scoring maps those into ~35 signals across the six JAIRF dimensions, aggregates them into
per-dimension scores, and rolls those up into a single weighted score and grade.
Requirements
- Node.js 20 LTS or newer (
>= 20.19.0) with npm/npx. See Node.js downloads.
- Docker installed and running. See Docker installation.
The CLI pulls the scoring image automatically on first run.
- Network access to
ghcr.io (to pull the image) and to whatever URL hosts the
OpenAPI document you're scoring (the engine fetches it from inside the container).
Install
npm install -g @jentic/api-scorecard-cli
This installs the CLI globally. The scoring engine (Docker image) is downloaded automatically
the first time you run score — allow a minute or two on a typical connection.
For local files or non-OAK URLs you'll also need a JENTIC_API_KEY — see
Anonymous vs keyed access.
Verify the install:
jentic-api-scorecard --version
Prefer zero-install? You can skip the global install and use npx — every example in
this README works with npx @jentic/api-scorecard-cli in place of jentic-api-scorecard.
Pin to a specific release with npx @jentic/api-scorecard-cli@<version> (e.g. @1.0.0);
the unpinned form resolves to whatever the latest dist-tag points at on each invocation,
while npm install -g pins you to the installed version until you explicitly update.
Try it now
OpenAPI documents from Jentic Public APIs (OAK)
score without any key, uncapped — no signup, no config:
npx @jentic/api-scorecard-cli@latest score \
https://raw.githubusercontent.com/jentic/jentic-public-apis/refs/heads/main/apis/openapi/swagger-api/petstore/1.0.27/openapi.json
For URLs outside OAK or local files, set the API key:
