Design and implement a scalable data pipeline for: $ARGUMENTS
/plugin marketplace add lifangda/claude-plugins/plugin install accessibility-suite@claude-pluginssonnetintegrated-tools/# Data Pipeline Architecture Design and implement a scalable data pipeline for: $ARGUMENTS Create a production-ready data pipeline including: 1. **Data Ingestion**: - Multiple source connectors (APIs, databases, files, streams) - Schema evolution handling - Incremental/batch loading - Data quality checks at ingestion - Dead letter queue for failures 2. **Transformation Layer**: - ETL/ELT architecture decision - Apache Beam/Spark transformations - Data cleansing and normalization - Feature engineering pipeline - Business logic implementation 3. **Orchestrat...
/data-pipelineYou are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.