From argos
Veri migration disiplini — Expand-Contract pattern, dual-write, backfill (chunked+idempotent), shadow read, cutover, rollback, audit, compliance. Big-bang migration yasak; zero-downtime / minimal-downtime stratejileri.
npx claudepluginhub resultakak/argos --plugin argosThis skill uses the workspace's default tool permissions.
`agents/shared/severity-rubric.md` ve `agents/shared/escalation-matrix.md` default-load
Mandates invoking relevant skills via tools before any response in coding sessions. Covers access, priorities, and adaptations for Claude Code, Copilot CLI, Gemini CLI.
Share bugs, ideas, or general feedback.
agents/shared/severity-rubric.md ve agents/shared/escalation-matrix.md default-load
sayılır (agents/coordination.md §11). Bu skill'in çıktısı Critical / High / Medium /
Low + kanıt formatında olmak zorunda — spekülatif Critical yasak. Sahiplik dışı bulgu
ilgili agent'a delege; karar yetkisi eşiği aşılırsa kullanıcı onayı zorunlu.
pg_dump | pg_restore.Kullanım: dev/staging only, küçük lab. Prod'da downtime kabul edilemiyorsa yasak.
Phase 1: Expand
┌──────────────────────────────────────────┐
│ Eski sistem yazıyor + okuyor (canonical) │
│ Yeni sistem yazıyor (shadow, no traffic) │
└──────────────────────────────────────────┘
Phase 2: Migrate
┌──────────────────────────────────────────┐
│ Eski + Yeni dual-write (eşzamanlı) │
│ Backfill geçmiş veri (chunked) │
│ Shadow read: yeni'den oku, eski ile karş. │
│ Diff sıfır → next phase │
└──────────────────────────────────────────┘
Phase 3: Switch
┌──────────────────────────────────────────┐
│ Read traffic yeni'ye (5/25/50/100 canary) │
│ Write hâlâ dual │
│ Eski hot replica olarak (read fallback) │
└──────────────────────────────────────────┘
Phase 4: Contract
┌──────────────────────────────────────────┐
│ Write yeni only │
│ Eski 30 gün hot (rollback ihtimali) │
│ Sonra cold archive → eventual delete │
└──────────────────────────────────────────┘
| Kategori | Tool | Notlar |
|---|---|---|
| CDC managed | AWS DMS, Azure DMS, GCP Datastream | RDB → RDB; schema convert opsiyonel |
| CDC self-host | Debezium | Kafka Connect bazlı; flexible |
| ETL/ELT | dbt, Airbyte, Fivetran | Data warehouse migration |
| Postgres-spesifik | pg_logical_replication, pglogical | Postgres → Postgres |
| Bulk load | pg_dump/restore, COPY, pgloader | Initial load |
| Custom script | Python + asyncio + chunked SELECT | Karmaşık transform |
| Application-level | Servis kendi yazar | Schema farkı çok |
Expand phase:
-- Expand: yeni shape eklendi, eski hâlâ yaşıyor
ALTER TABLE users ADD COLUMN address_v2 JSONB; -- yeni
-- address_v1 (string) hâlâ kullanılıyor
App katmanında her write iki sisteme gider:
async def create_order(order: Order) -> Order:
# Eski sistem canonical (success kriteri)
saved = await old_db.insert(order)
# Yeni sistem shadow (failure tolere)
try:
await new_db.insert(order_to_new_shape(saved))
except Exception as e:
log.warn("dual_write_failed", error=str(e), order_id=saved.id)
metrics.dual_write_fail.inc()
# eski sistem zaten başardı → user bilgi alır
return saved
Kurallar:
async def backfill_chunk(start_id: int, end_id: int) -> int:
"""Chunked + idempotent + restartable."""
rows = await old_db.fetch(
"SELECT * FROM orders WHERE id >= $1 AND id < $2 ORDER BY id",
start_id, end_id,
)
transformed = [order_to_new_shape(r) for r in rows]
# UPSERT (idempotent — restart safe)
await new_db.execute_many(
"INSERT INTO orders_v2 (id, ...) VALUES ($1, ...) "
"ON CONFLICT (id) DO UPDATE SET ... ",
transformed,
)
# Checkpoint
await save_checkpoint("orders_backfill", end_id)
return len(rows)
Kurallar:
backfill_rows_total,
ETA hesabı.App katmanında okuma eski'den ama paralel yeni'den de — fark logla:
async def get_order(order_id: int) -> Order:
old_result = await old_db.get(order_id)
# Shadow read: paralel, response'u etkilemiyor
asyncio.create_task(_shadow_compare(order_id, old_result))
return old_result # canonical eski
async def _shadow_compare(order_id: int, old_result: Order) -> None:
try:
new_result = await new_db.get(order_id)
if not orders_match(old_result, new_result):
log.warn("shadow_mismatch", order_id=order_id,
diff=diff_orders(old_result, new_result))
metrics.shadow_mismatch.inc()
except Exception as e:
log.warn("shadow_read_failed", error=str(e))
Mismatch < %0.01 → switch hazır.
Read traffic switch (Phase 3):
read_from_new_db per-region/per-tenant.Write traffic switch (Phase 4'e geçiş):
| Phase | Rollback | Komut |
|---|---|---|
| Expand | Kolay — yeni schema kullanılmıyor; drop. | ALTER TABLE users DROP COLUMN address_v2; |
| Migrate (dual-write) | Kolay — yeni'yi kapat (feature flag). Veri eski'de canonical. | flag.disable("dual_write_new_db") |
| Switch (read) | Read flag toggle → eski'ye geri. | flag.disable("read_from_new_db") |
| Switch (write) | Point of no return yakın. 30 gün hot eski'den yeni'ye reverse-backfill yap. | reverse migration script |
| Contract | No rollback (eski silinmiş). | DR backup'tan restore. |
Drill: phase 3'te 1 saatlik chaos drill — yeni DB'yi kapat, fallback eski → recovery süresi ölç.
backfill_rows_total{table} (counter)dual_write_fail_total (counter)shadow_mismatch_total (counter)migration_lag_seconds (gauge)cutover_phase (gauge: 0/1/2/3/4)SELECT count(*) FROM old.orders vs SELECT count(*) FROM new.orders.MD5(json_agg(row)) GROUP BY chunk.pg_dump | pg_restore prod'da — downtime saatler.User: /data-migration postgres-to-cockroach
Agent (migration-planner):
1. Scope: orders + payments + users 3 tablo, ~280M satır, 420 GB, peak 850 TPS.
2. Strateji: Expand-Contract (zero-downtime hedef; PCI scope korunmalı).
3. Tooling: pglogical + custom Python backfill (CockroachDB pgwire compatible).
4. Phase 1 (Expand, 1 hafta): yeni cluster + schema + sealed-secrets connection.
5. Phase 2 (Migrate, 3 hafta): dual-write app + backfill chunked 100 row/sn,
shadow read ön validation.
6. Phase 3 (Switch read, 2 hafta canary): 5/25/50/100 region.
7. Phase 4 (Switch write + Contract, 1 hafta): write yeni only; eski 30 gün hot.
8. Validation: row count + MD5 checksum 1k chunk + 5k sample diff.
9. Rollback drill phase 3'te (chaos: yeni cluster kapat → eski fallback).
10. Compliance: KMS key swap (CockroachDB encryption); GDPR audit log.
11. Output: 47 day migration plan + 12 issue + maintenance window 2026-08-15
customer notify.
# Data Migration: <source> → <target>
## Strateji
- Expand-Contract / Lift-Shift / Strangler
## Scope
- Tablo / row count / GB / RPS / constraint
## Phase Plan
| Phase | Süre | Aksiyon | Rollback |
## Backfill
- Tool, chunk size, rate, idempotent
## Validation
- Row count, checksum, sample diff
## Rollback
- Phase başına komut + drill
## Audit + Compliance
- KMS, GDPR, PCI, audit log
## Action Items
| P | Aksiyon | Sahip | Bitiş | Issue |