From astronomer-data
Guides authoring Apache Airflow DAGs via structured workflow: discover environment with af CLI, plan structure, implement patterns, validate syntax and test.
npx claudepluginhub astronomer/agents --plugin astronomer-dataThis skill uses the workspace's default tool permissions.
This skill guides you through creating and validating Airflow DAGs using best practices and `af` CLI commands.
Builds production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use for data pipelines, workflow orchestration, or batch jobs.
Builds production-ready Apache Airflow DAGs with patterns for operators, sensors, testing, and deployment. For data pipelines, workflow orchestration, and batch jobs.
Tests, debugs, and fixes Airflow DAGs in iterative cycles using af CLI trigger-wait and Astro dev parse/pytest. For requests like 'test dag and fix if fails'.
Share bugs, ideas, or general feedback.
This skill guides you through creating and validating Airflow DAGs using best practices and af CLI commands.
For testing and debugging DAGs, see the testing-dags skill which covers the full test -> debug -> fix -> retest workflow.
These commands assume af is on PATH. Run via astro otto to get it automatically, or install standalone with uv tool install astro-airflow-mcp.
+-----------------------------------------+
| 1. DISCOVER |
| Understand codebase & environment |
+-----------------------------------------+
|
+-----------------------------------------+
| 2. PLAN |
| Propose structure, get approval |
+-----------------------------------------+
|
+-----------------------------------------+
| 3. IMPLEMENT |
| Write DAG following patterns |
+-----------------------------------------+
|
+-----------------------------------------+
| 4. VALIDATE |
| Check import errors, warnings |
+-----------------------------------------+
|
+-----------------------------------------+
| 5. TEST (with user consent) |
| Trigger, monitor, check logs |
+-----------------------------------------+
|
+-----------------------------------------+
| 6. ITERATE |
| Fix issues, re-validate |
+-----------------------------------------+
Before writing code, understand the context.
Use file tools to find existing patterns:
Glob for **/dags/**/*.py to find existing DAGsRead similar DAGs to understand conventionsrequirements.txt for available packagesUse af CLI commands to understand what's available:
| Command | Purpose |
|---|---|
af config connections | What external systems are configured |
af config variables | What configuration values exist |
af config providers | What operator packages are installed |
af config version | Version constraints and features |
af dags list | Existing DAGs and naming conventions |
af config pools | Resource pools for concurrency |
Example discovery questions:
af config connectionsaf config versionaf config providersBased on discovery, propose:
Get user approval before implementing.
Write the DAG following best practices (see below). Key steps:
requirements.txt if neededUse af CLI as a feedback loop to validate your DAG.
After saving, check for parse errors (Airflow will have already parsed the file):
af dags errors
Common causes: missing imports, syntax errors, missing packages.
af dags get <dag_id>
Check: DAG exists, schedule correct, tags set, paused status.
af dags warnings
Look for deprecation warnings or configuration issues.
af dags explore <dag_id>
Returns in one call: metadata, tasks, dependencies, source code.
If you're running on Astro, you can also validate locally before deploying:
astro dev parse to catch import errors and DAG-level issues without starting a full Airflow environmentastro deploy --dags for fast DAG-only deploys that skip the Docker image build — ideal for iterating on DAG codeSee the testing-dags skill for comprehensive testing guidance.
Once validation passes, test the DAG using the workflow in the testing-dags skill:
af runs trigger-wait <dag_id> --timeout 300af runs diagnose <dag_id> <run_id> and af tasks logs <dag_id> <run_id> <task_id># Ask user first, then:
af runs trigger-wait <dag_id> --timeout 300
For the full test -> debug -> fix -> retest loop, see testing-dags.
If issues found:
af dags errors| Phase | Command | Purpose |
|---|---|---|
| Discover | af config connections | Available connections |
| Discover | af config variables | Configuration values |
| Discover | af config providers | Installed operators |
| Discover | af config version | Version info |
| Validate | af dags errors | Parse errors (check first!) |
| Validate | af dags get <dag_id> | Verify DAG config |
| Validate | af dags warnings | Configuration warnings |
| Validate | af dags explore <dag_id> | Full DAG inspection |
Testing commands -- See the testing-dags skill for
af runs trigger-wait,af runs diagnose,af tasks logs, etc.
For code patterns and anti-patterns, see reference/best-practices.md.
Read this reference when writing new DAGs or reviewing existing ones. It covers what patterns are correct (including Airflow 3-specific behavior) and what to avoid.