From agentops
Implements a fail-closed close door for quests, replacing agent-written synthesis with deterministic, evidence-bound verdicts. Useful for ensuring no verdict means not done.
How this skill is triggered — by the user, by Claude, or both
Slash command
/agentops:gc-membraneThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
Canonical pack source: **`packs/agentops-membrane/`** in the AgentOps repo (git-tracked). Imported into a city via `pack.toml` (pinned tree-URL, or local
Canonical pack source: packs/agentops-membrane/ in the AgentOps repo (git-tracked). Imported into a city via pack.toml (pinned tree-URL, or local
path while iterating). Operator loop that drives it:
using-gc.
packs/agentops-membrane/ as the canonical source and installed city copies as projections, because parallel edits create an untraceable close-door fork.Stock gc orchestrates superbly but has no fail-closed close: its
mol-review-quorum formula fans review out cross-family and even ships a
deterministic Go finalizer (reviewquorum.Finalize) — which has zero
production callers. The quorum verdict is whatever the synthesis agent
writes; an agent can synthesize "pass" over two failing lanes. That is
self-grading — exactly what AgentOps exists to prevent (no verdict = not
done). The pack replaces agent-written synthesis with a deterministic,
evidence-bound verdict computed by scripts the control dispatcher runs.
| Piece | Purpose |
|---|---|
formulas/membrane-quest.toml | v2 workflow: worktree build with [steps.check] → close-gate.sh, max_attempts = 5, check gc.work_dir pinned to the city root, ralph-gate-bead protection in the recovery snippet |
membrane/close-gate.sh | the close door (run by the dispatcher, never an agent) |
membrane/finalize.jq + finalize.sh | deterministic decision: canonical pawl-verdict.v1 only for terminal semantics; gc-review-attempt.v1 for transport degradation |
membrane/scaffold-quest.sh + quests/_template/ | move-1 intake: default-FAIL CONTRACT.md, red test.sh |
agents/{planner,builder,verifier,agy-verifier,opus-verifier} | trinity + third family + claude-family failover lane, RBAC via harness config |
doctor/law0-print-args, doctor/membrane-health | blocking doctor checks |
orders/membrane-canary.toml | scheduled structural smoke via the pack's e2e.sh entrypoint |
tests/{finalize,intake}.bats | pins the rollup parity + scaffold contract (bash+jq only) |
RESIDUAL-GAPS.md | the two honest zero-nudge gaps + mitigations |
scaffold-quest.sh <slug> writes CONTRACT.md (default-FAIL
acceptance ruler) + test.sh (executable red harness) on the quest repo's
main. The gate later reads the contract from main ONLY
(git show main:CONTRACT.md); the builder never reads CONTRACT.md —
it reads test.sh.test.sh green on a branch.[steps.check]):
close-gate.sh runs deterministic pre-gates (branch exists, non-empty
diff vs main, contract present) → mints a per-round nonce → routes
ONLY the diff + contract to ≥2 cross-family reviewer lanes via
gc session submit (the only verb that reliably drains an idle pane) →
collects review-quorum.lane.v1 JSONs → hands to the finalizer.finalize.jq) computes the decision. Precedence:
hard > transient > findings > pass, hardened beyond the upstream rollup:
fewer_than_two_families).gc.failure_class — every nonzero check
exit consumes one of max_attempts (5), DEGRADED included; the class
stamp is evidence, not budget control.gc-review-attempt.v1; it cannot overwrite the latest terminal verdict.gc.attempt_log entry
outcome=pass) and the workflow finalizes pass; any nonzero exit →
builder respawned with findings, one of max_attempts (5) consumed.
Exhausted attempts leave the quest bead OPEN — never a spurious close.
The membrane never merges or pushes; a human merges the branch.Two structural protections (added after the 2026-07-06 canary catches):
gc.outcome=pass used to fail the door OPEN. The
formula's recovery snippet filters gc.kind=ralph, the prompt forbids
touching it, and the fork's finalizer accepts a closed-with-pass gate ONLY
with the engine attempt_log fingerprint (patch(dispatch) 4aa582dcc)."gc.work_dir" = "." on the step):
without it the check inherits gc's per-bead-worktree path this formula
never creates, the dispatcher chdir fails, and the check silently never
runs (visible only in the ralph bead's gc.attempt_log — the check path
emits no gc events; set GC_WORKFLOW_TRACE for the dispatch trace).Builder = claude family (opus), write access, only in its worktree.
Verifiers = codex/gpt (LANE1) and antigravity/gemini via agy (LANE2),
permission_mode = "plan" (read-only), fresh-context, receive only
diff + contract. Never same-family self-grade: quorum is builder-family vs
two other families. City wiring makes both lanes always-on [[named_session]]
entries so submit-delivery is deterministic.
Failover (agy outage) — an explicitly DEGRADED-DIVERSITY mode: the
default contract above excludes the BUILDER family from the reviewer set
(claude judged by {codex, gemini}). agents/opus-verifier deliberately
trades that one property away when the third family is unavailable: it keeps
fresh-context author≠judge (a different SESSION carrying none of the builder
context) and the ≥2-distinct-reviewer-families floor ({codex, claude}), but
one reviewer now shares the builder FAMILY. Strictly weaker than the default;
strictly stronger than the alternatives (single-family review, or burning the
attempt budget on DEGRADE) — the same tradeoff the repo pawl service makes
running opus+codex panels against claude authors. Temporary posture: restore
the agy lane when its auth heals. Opt in per city:
[workspace.env] MEMBRANE_LANE2_TARGET = "agentops-membrane.opus-verifier" +
MEMBRANE_LANE2_FAMILY = "claude", add its [[named_session]], bounce the
supervisor (the check runs in the supervisor process — workspace env reaches
lanes via session spawn, but the LANE2 selection is read by close-gate from
the check process env, so the supervisor must carry it: plist env or restart
after config). Restore the agy lane when its auth heals.
Artifacts land at <city>/membrane/<quest>/:
pawl-verdict-round-N.json (+ latest copied to pawl-verdict.json),
lane-<family>-round-N.json, nonce-round-N.txt.
Terminal artifact example (actual required shape emitted by finalize.sh):
{
"schema_version": "pawl-verdict.v1",
"bead_id": "quest-bead",
"pr": 0,
"head_sha": "<reviewed-sha>",
"disposition": "CONFIRMED",
"generated_at": "2026-07-11T00:00:00Z",
"mode": "multi-model",
"author_context_id": "builder-session",
"attempt": 3,
"refuters": [
{"family": "gpt", "reviewer": "lane-1", "context_id": "lane-1", "verdict": "CONFIRMED", "evidence": "<city>/membrane/<quest>/evidence-round-3/lane-1.json"},
{"family": "gemini", "reviewer": "lane-2", "context_id": "lane-2", "verdict": "CONFIRMED", "evidence": "<city>/membrane/<quest>/evidence-round-3/lane-2.json"}
]
}
Checks when reading: required terminal fields; disposition; ≥2 distinct
refuters[].family on any CONFIRMED. Nonce anti-replay happens before terminal
emission: finalize.sh compares each input lane's agentops_nonce with the
round nonce. Terminal refuters do not invent a nonce_echo field. On REFUTED
read the copied evidence-round-<N>/lane-<index>.json findings (a placement/path finding may be a diff-frame
mismatch — a contract bug, not a build bug; fix the contract frame instead
of burning redo attempts).
<city>/membrane/<quest>/; source changes belong only under packs/agentops-membrane/.pawl-verdict-round-<N>.json plus pawl-verdict.json; raw inputs use lane-<family>-round-<N>.json and nonce-round-<N>.txt; terminal refuter evidence is copied to evidence-round-<N>/lane-<index>.json; degradation writes review-attempt-round-<N>.json.pawl-verdict.v1 JSON schema; degradation writes nonsemantic gc-review-attempt.v1 JSON and cannot replace the latest terminal verdict.bats packs/agentops-membrane/tests/{finalize,intake,close-gate}.bats; for an installed city also run bash skills/gc-membrane/scripts/e2e.sh --city <city>.agentops_nonce values matching the exact round nonce and has at least two distinct reviewer families.doctor/membrane-health (blocking): door present + executable, formula
resolves, trinity present, ≥2 provider families configured.doctor/law0-print-args (blocking): every claude/antigravity provider
carries an explicit empty print_args (the builtin defaults are headless
print sinks — LAW 0).orders/membrane-canary: native cooldown order running the pack's e2e.sh
entrypoint
(both doctor checks green + gc lint) — structural smoke, surfaced through
gc events/dashboard.Two session/provider-boundary stalls stand between "installed" and
zero-touch — the idle-pane drain (only gc session submit recovers; a
keepalive order re-submits on cooldown) and the codex/agy first-run trust
modal (pre-trust via city-scoped CODEX_HOME / one interactive run). Both
documented with mitigations in the pack's RESIDUAL-GAPS.md and as
first-class moves in the troubleshooting ladder of
using-gc.
The durable fixes are upstream (headless verifier lane / submit-consume
guarantee) — the pack does not fake a root fix.
No auto-merge/auto-push. No wrapping of gc's native surface. No new
stall-detection framework. No gating on gc costs. No runtime=gc
re-coupling and no gc subcommand under the ao CLI — the pack is gc-native
content that lives BESIDE ao/NTM.
Scenario: Fail-closed verdict on a hard finding
Given a quest round where one reviewer lane returns a hard finding with a valid nonce echo
When finalize.sh computes the verdict
Then the disposition is REFUTED with exit 2 and the quest bead stays open
Scenario: Transient lane loss never false-refutes
Given a quest round where one reviewer lane is provider_unavailable
When finalize.sh computes the verdict
Then the disposition is DEGRADED with exit 3 and the quest is not falsely REFUTED
Scenario: DEGRADED still consumes a redo attempt (budget pinned to the formula)
Given membrane-quest.toml sets max_attempts = 5 and the ralph dispatcher never reads gc.failure_class
When a check exits nonzero as DEGRADED (exit 3), exactly as for a REFUTED (exit 2)
Then one of max_attempts is consumed — a transient-loss flake burns budget like a real refute
Scenario: One family can never confirm
Given a quest round where all lane verdicts come from a single provider family
When finalize.sh computes the verdict
Then the result is fewer_than_two_families and the disposition is not CONFIRMED
2026-07-07 (native graph.v2, canary city, zero nudges): hello3 ran the
full loop — rounds 1–2 REFUTED hard by real cross-family findings (gpt:
contract_failure, gemini: ask_not_met), builder self-redo from findings,
round 3 CONFIRMED ({gpt, gemini}, nonce-bound, head-bound), ralph closed
by the engine fingerprint, workflow finalized pass, human merged. The door's
first two native runs also CAUGHT two real defects in itself (the fail-open
and the check-never-ran above) — the dogfood harness working as designed.
Artifacts: /Users/bo/gc/canary-city/membrane/hello3/.
Three real quests through the door on the native reference city:
REFUTED→redo→CONFIRMED (hello); DEGRADED on transient agy-lane loss
without a false refute (csv-stats); cross-family hard REFUTE where the
gpt lane caught 3 real blocking gaps the gemini lane passed
(install-gc-city) — the quest correctly stayed open. Artifacts:
<city>/membrane/{hello,csv-stats,install-gc-city}/pawl-verdict*.json.
npx claudepluginhub boshu2/agentops --plugin agentopsDrive a Gas City (gc) orchestration factory: stand up cities, sling quests, resolve stalls, and converge with the AgentOps membrane.
Validates agent completion claims against evidence trail, catching unsubstantiated claims like 'tests pass' or 'fixed' without proof. Auto-fires at workflow end.
Verifies claims of completion by requiring fresh evidence before stating work is done. Use before claiming tasks complete, bugs fixed, or tests pass.