From argos
Property-based testing (PBT) disipline — Python Hypothesis + TypeScript fast-check + Go gopter + Schemathesis (OpenAPI). Generator + shrinking + stateful PBT + regression seed corpus + CI integration. Property tasarım hiyerarşisi (invariant > round-trip > idempotency > oracle > metamorphic), Schemathesis ile spec-driven test, regression seed kalıcılığı. test-engineer'a destek olan teknik skill.
npx claudepluginhub resultakak/argos --plugin argosThis skill uses the workspace's default tool permissions.
`agents/shared/severity-rubric.md` ve `agents/shared/escalation-matrix.md`
Mandates invoking relevant skills via tools before any response in coding sessions. Covers access, priorities, and adaptations for Claude Code, Copilot CLI, Gemini CLI.
Share bugs, ideas, or general feedback.
agents/shared/severity-rubric.md ve agents/shared/escalation-matrix.md
default-load sayılır (agents/coordination.md §11). Bu skill'in çıktısı
Critical / High / Medium / Low + kanıt formatında olmak zorunda — spekülatif
Critical yasak. Sahiplik dışı bulgu ilgili agent'a (test-engineer,
security-reviewer, api-contract-guardian) delege.
| Dil / Katman | Tool | Stable mi? | Stateful var mı? | Notlar |
|---|---|---|---|---|
| Python | Hypothesis | Olgun | ✅ RuleBasedStateMachine | pytest-hypothesis plugin; ghostwriter |
| TypeScript / JS | fast-check | Olgun | ✅ fc.commands | @fast-check/jest, @fast-check/vitest |
| Go | gopter + native testing/fuzz (1.18+) | Orta | gopter ✅ | gopter generic; native fuzz daha yeni |
| API (OpenAPI 3.x) | Schemathesis | Olgun | ✅ link-following stateful | spec-driven, CI ready |
# Mevcut test kütüphaneleri
grep -rE "import hypothesis|from hypothesis" --include="*.py" | head
grep -rE "@fast-check|fast-check" --include="*.ts" --include="*.tsx" | head
grep -rE "gopter" --include="*.go" | head
grep -rE "schemathesis" --include="*.py" --include="*.yml" | head
# PBT'ye uygun fonksiyon adayı:
# - encode/decode, parse/serialize çiftleri
# - sort/normalize/merge gibi pure fonksiyonlar
# - state machine'ler (cart, queue, state pattern)
grep -rE "def (encode|decode|parse|serialize|normalize|merge)" --include="*.py"
Aday fonksiyon için 5 katmanlı property arama:
decode(encode(x)) == x.f(f(x)) == f(x) (sort, normalize, dedupe).Hangisi mümkün? Çoğu fonksiyon için 1-3 katmandan en az birisi geçerlidir.
Python (Hypothesis):
from hypothesis import given, strategies as st, settings, HealthCheck
@given(st.text(min_size=1, max_size=200))
@settings(max_examples=500, deadline=2000)
def test_encode_decode_round_trip(s: str) -> None:
assert decode(encode(s)) == s
TypeScript (fast-check):
import fc from 'fast-check';
test('encode/decode round-trip', () => {
fc.assert(
fc.property(fc.string({ minLength: 1, maxLength: 200 }), (s) => {
expect(decode(encode(s))).toBe(s);
}),
{ numRuns: 500, seed: 42 },
);
});
Go (gopter):
func TestEncodeDecodeRoundTrip(t *testing.T) {
properties := gopter.NewProperties(nil)
properties.Property("round-trip", prop.ForAll(
func(s string) bool { return decode(encode(s)) == s },
gen.AlphaString().WithLabel("s"),
))
properties.TestingRun(t)
}
Composition (Python @composite):
@st.composite
def order_lines(draw):
sku = draw(st.text(min_size=3, max_size=10))
qty = draw(st.integers(min_value=1, max_value=999))
price_cents = draw(st.integers(min_value=1, max_value=10_000_000))
return {"sku": sku, "qty": qty, "price_cents": price_cents}
Aggregate / cart / payment intent / cache gibi state-li domain için:
from hypothesis.stateful import RuleBasedStateMachine, rule, invariant
class CartStateMachine(RuleBasedStateMachine):
def __init__(self):
super().__init__()
self.cart = Cart()
@rule(line=order_lines())
def add_line(self, line):
self.cart.add(line)
@rule()
def checkout(self):
if not self.cart.is_empty():
self.cart.checkout()
@invariant()
def total_non_negative(self):
assert self.cart.total() >= 0
State machine binlerce komut dizilimini deneyip invariant'ı her adımda kontrol eder; tek başına example'la bulunması olanaksız edge case'leri yakalar.
# Lokal
schemathesis run \
--checks all \
--hypothesis-max-examples=200 \
--hypothesis-deadline=2000 \
https://api.acme.com/openapi.json
# CI gate (GitHub Actions)
schemathesis run \
--checks all,response_schema_conformance \
--report-junit=schemathesis-report.xml \
--stateful=links \
openapi.yaml
Schemathesis bulgu örnekleri:
application/json ile XML kabul ediyor.\r\n smuggling.Counter-example bulunduğunda kalıcılaştır:
# Hypothesis: .hypothesis/examples/ dir auto-managed (sqlite). Commit'le.
# .gitignore'a `.hypothesis/examples/` EKLEME — KALICI corpus.
# Manual seed:
@given(st.text())
@example("\x00") # bulunan counter-example
@example("café") # unicode normalisation bug
def test_normalize(s: str) -> None:
assert normalize(normalize(s)) == normalize(s)
fast-check:
// seed manuel ekleme
fc.assert(
fc.property(fc.string(), (s) => normalize(normalize(s)) === normalize(s)),
{
seed: 1234567, // failed run seed (kalıcı kayıt)
path: '0:0:1', // shrink path
numRuns: 0, // sadece seed/path replay
},
);
# .github/workflows/test.yml
- name: PBT (Hypothesis)
run: pytest --hypothesis-profile=ci -q
env:
HYPOTHESIS_PROFILE: ci
# Hypothesis profile (conftest.py)
# settings.register_profile("ci",
# max_examples=500, deadline=2000,
# suppress_health_check=[HealthCheck.too_slow],
# derandomize=False, print_blob=True)
- name: Schemathesis (API)
run: |
schemathesis run --checks all \
--hypothesis-max-examples=100 \
--report-junit=stx.xml \
openapi.yaml
# PBT Findings: <module/path>
## Critical
- [ ] decode/encode round-trip `\x00` byte için fails — `parser.py:42` —
counter-example: `b"\x00\x00"` shrink edilmiş
## High
- [ ] cart aggregate total negatif olabilir checkout retry sonrası —
`cart.py:118` — state machine 14 komut sonrası invariant violation
## Medium
- [ ] schemathesis: POST /orders content-type missing → 500 (422 olmalı) —
`routes/orders.py:23` — issue #ABC-123
## Low
- [ ] generator `text(min_size=0)` boş string filter rate %12 — health check
warning
assume() filter rate < %30?Filtered too many, Slow data)?assume() filter ile invariant kurtarma → generator'ı düzelt.derandomize=True yapıp seed flat → flake bulamazsın; tersine PBT'nin
amacı stochastic exploration..hypothesis/examples/ .gitignore'da → regression corpus her
CI run'da kaybolur.@example'a eklememek.rules/property-based-testing.md — discipline rule.rules/testing.md — piramit içinde PBT konumu (unit + integration arası).skills/qa-strategy/SKILL.md — quality gate matrisinde PBT katmanı.agents/test-engineer.md — PBT planlama + flake yönetimi sahiplik.agents/api-contract-guardian.md — Schemathesis bulgusu spec değişikliği
tetiklediğinde.commands/property-test-review.md — slash command entrypoint.