From ultrapowers-dev
Use when designing task queues, async workers, or event-driven processing that runs outside the request-response cycle.
How this skill is triggered — by the user, by Claude, or both
Slash command
/ultrapowers-dev:background-jobsThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
Work that does not need to complete within a user request belongs in a background job. Decouple production from consumption, and design every job to be safely retried.
Work that does not need to complete within a user request belongs in a background job. Decouple production from consumption, and design every job to be safely retried.
| Semantic | Guarantee | Practical Reality |
|---|---|---|
| At-most-once | May lose messages, never duplicates | Fire-and-forget; acceptable for analytics pings |
| At-least-once | No message loss, may duplicate | Most queues default here; requires idempotent consumers |
| Exactly-once | No loss, no duplicates | Extremely hard; approximate with at-least-once + idempotency |
Default to at-least-once and make consumers idempotent.
| Principle | Detail |
|---|---|
| Idempotency | Processing a job twice must produce the same result; use idempotency keys |
| Small payload | Store a reference (ID/URL) in the job, not the full data |
| Timeout | Every job needs a max execution time; kill and retry if exceeded |
| Isolation | One job failure must not block or crash other jobs |
| Observability | Log job ID, type, duration, and outcome on every execution |
Producer --> Queue --> Worker(s) --> Success / Failure
|
Retry Queue (with backoff)
|
Dead Letter Queue (DLQ)
| Strategy | How It Works |
|---|---|
| Idempotency key | Caller provides a unique key; server deduplicates on that key |
| Database constraint | Use unique constraints to prevent duplicate writes |
| State check before action | Verify current state before applying change (check-then-act) |
| Tombstone/processed flag | Mark job ID as processed; skip if already marked |
| Mistake | Fix |
|---|---|
| No idempotency on consumers | Add idempotency keys or deduplication checks |
| Giant payloads in the queue | Store references; fetch data inside the worker |
| Ignoring the dead letter queue | Alert on DLQ depth; review and reprocess regularly |
| Unbounded retries | Cap retries (3-5) with exponential backoff, then DLQ |
| No job timeout | Set max execution time; terminate and re-queue if exceeded |
Original — Datatide, MIT licensed.
npx claudepluginhub ennio-datatide/ultrapowers-devImplements background job processing with Bull/BullMQ (Node.js), Celery (Python), Sidekiq (Ruby), and cron. Covers prioritization, retries, dead letter queues, monitoring, rate limits, and shutdown for offloading tasks and pipelines.
Guides message queue and job processing setup with Kafka, RabbitMQ, SQS, BullMQ, Celery, Sidekiq. Covers architecture, retries, DLQs, idempotency, priorities, backpressure, and scaling.
Python background job patterns: task queues, workers, event-driven architecture. Helps decouple long-running work from request/response cycles using Celery or alternatives.