Skill

kafka-deep

Kafka deep-dive: topic design (partitions, replication factor), KafkaJS producer with idempotent writes, consumer groups with partition assignment, consumer lag monitoring, exactly-once semantics, schema registry, compacted topics, DLQ patterns. Use when designing Kafka topics, implementing producers/consumers, monitoring consumer lag.

Install

npx claudepluginhub rnavarych/alpha-engineer --plugin billy-milligan

Tool Access

This skill is limited to using the following tools:

ReadGrepGlob

Preview

- Designing topic structure (partitions, replication)

Supporting Assets

references/consumer-patterns.mdreferences/exactly-once.mdreferences/topic-design.md

SKILL.md

Similar Skills

agent-harness-construction

Designs and optimizes AI agent action spaces, tool definitions, observation formats, error recovery, and context for higher task completion rates.

everything-claude-code

156.2k

accessibility

Designs, implements, and audits WCAG 2.2 AA accessible UIs for Web (ARIA/HTML5), iOS (SwiftUI traits), and Android (Compose semantics). Audits code for compliance gaps.

everything-claude-code

156.2k

agent-eval

Compares coding agents like Claude Code and Aider on custom YAML-defined codebase tasks using git worktrees, measuring pass rate, cost, time, and consistency.

everything-claude-code

156.2k

Stats

Parent Repo Stars9

Parent Repo Forks0

Last CommitFeb 23, 2026

Actions

View Source View Plugin View on GitHub View README

kafka-deep

Core Principles

Partition count determines parallelism ceiling — 6 partitions = max 6 consumers in a group; you cannot scale beyond partition count

Replication factor 3 for production — 1 node lost: still operational; 2 nodes lost: read-only; RF=1 = data loss risk

Consumer commits must happen after processing — committing before processing = data loss on crash

At-least-once is the default — design consumers to be idempotent; dedup with Redis

Consumer lag is the key operational metric — lag >10k messages at current throughput = alert

References available

references/topic-design.md — naming conventions, partition count guidelines, replication factor, retention, partition key strategy

references/consumer-patterns.md — KafkaJS producer with idempotent writes, consumer group with DLQ, lag monitoring, Kafka vs RabbitMQ vs SQS

references/exactly-once.md — exactly-once semantics, transactions, schema registry, compacted topics

Kafka Deep Dive

When to Use This Skill

Designing topic structure (partitions, replication)
Implementing reliable producers with idempotent writes
Building consumer groups with proper error handling
Monitoring consumer lag
Choosing between Kafka, RabbitMQ, and SQS

Core Principles

Partition count determines parallelism ceiling — 6 partitions = max 6 consumers in a group; you cannot scale beyond partition count
Replication factor 3 for production — 1 node lost: still operational; 2 nodes lost: read-only; RF=1 = data loss risk
Consumer commits must happen after processing — committing before processing = data loss on crash
At-least-once is the default — design consumers to be idempotent; dedup with Redis
Consumer lag is the key operational metric — lag >10k messages at current throughput = alert

References available

references/topic-design.md — naming conventions, partition count guidelines, replication factor, retention, partition key strategy
references/consumer-patterns.md — KafkaJS producer with idempotent writes, consumer group with DLQ, lag monitoring, Kafka vs RabbitMQ vs SQS
references/exactly-once.md — exactly-once semantics, transactions, schema registry, compacted topics