Airflow plugins

Build production-ready data pipelines with Apache Airflow and dbt, manage scalable data warehouses, and implement vector search and RAG systems using embedding models and vector databases.

Manage the full data engineering lifecycle — author, test, debug, and deploy Apache Airflow DAGs; query and profile warehouses; trace lineage; run dbt with Cosmos; migrate between Airflow versions; and troubleshoot production deployments via the Astro CLI.

Guides data engineering projects through a structured Spec-Driven Development workflow with 58 specialized agents for pipeline design, schema modeling, SQL optimization, data quality, lakehouse architecture, and AI/ML infrastructure. Generates visual diagrams, HTML documentation, code reviews, and git-aware status reports.

Build and orchestrate data pipelines on Google Cloud using natural language: profile BigQuery data with Dataplex/dbt, generate Dataform code, run Spark on Dataproc, manage Cloud Composer (Airflow) DAGs, and provision infrastructure—all from your coding agent.

Orchestrate full enterprise workflows by deploying 75 specialized AI agents across engineering, product, infrastructure, data, security, marketing, sales, finance, legal, and HR. Auto-routes tasks to domain experts, executes multi-agent pipelines with pre-built runbooks, handoff templates, QA gates, and escalations for end-to-end automation.