From zenml-io-skills
Migrate Argo Workflows, WorkflowTemplates, ClusterWorkflowTemplates, and CronWorkflows to idiomatic ZenML pipelines. Handles concept mapping, YAML-to-Python translation, scheduling, retries, Kubernetes-native pattern analysis, and flags unsupported patterns such as status-based depends logic, shared volumes, containerSet, sidecars, synchronization locks, and Argo Events for human review. Use this skill whenever the user mentions Argo migration, converting Argo YAML, replacing Argo with ZenML, mapping an Argo concept to ZenML, or provides workflow YAML using terms like WorkflowTemplate, CronWorkflow, when, withItems, withParam, containerSet, onExit, Sensor, or EventSource. For quick conceptual questions, answer from the concept map without running the full migration workflow.
npx claudepluginhub joshuarweaver/cascade-ai-ml-engineering --plugin zenml-io-skillsThis skill uses the workspace's default tool permissions.
This skill translates Argo Workflows into idiomatic ZenML pipelines. It handles the full migration workflow: analyzing Argo YAML and referenced scripts, classifying each pattern, translating what maps cleanly, flagging what needs redesign, and producing a working ZenML project.
Creates isolated Git worktrees for feature branches with prioritized directory selection, gitignore safety checks, auto project setup for Node/Python/Rust/Go, and baseline verification.
Executes implementation plans in current session by dispatching fresh subagents per independent task, with two-stage reviews: spec compliance then code quality.
Dispatches parallel agents to independently tackle 2+ tasks like separate test failures or subsystems without shared state or dependencies.
This skill translates Argo Workflows into idiomatic ZenML pipelines. It handles the full migration workflow: analyzing Argo YAML and referenced scripts, classifying each pattern, translating what maps cleanly, flagging what needs redesign, and producing a working ZenML project.
Argo and ZenML can both express DAG-shaped workflows, but they speak different native languages.
@step, @pipeline, typed artifacts, stack components, and orchestrator-managed execution.That means this migration is not a simple rename operation. Some Argo features map well, some only preserve intent, and some depend so heavily on controller or pod semantics that they must be redesigned explicitly.
Every Argo concept falls into one of these categories:
| Type | Meaning | Action |
|---|---|---|
| Direct | Clean 1:1 or near-1:1 mapping exists | Translate automatically |
| Approximate | Conceptual equivalent exists but semantics differ | Translate with caveats noted in the migration report |
| Absent | No ZenML equivalent exists | Flag for human review with redesign suggestions |
See references/concept-map.md for the full mapping tables.
Ask the user for all relevant migration inputs. At minimum, request:
Workflow, WorkflowTemplate, ClusterWorkflowTemplate, CronWorkflow)Read everything thoroughly before doing anything else. For each workflow, identify:
Workflow, WorkflowTemplate, ClusterWorkflowTemplate, CronWorkflow, EventSource, Sensorcontainer, script, dag, steps, resource, http, suspend, plugin, data, containerSetdependencies, enhanced depends, templateRef, workflowTemplateRef, workflow-of-workflows patternsoutputs.result, artifacts, globalNamewhen, withItems, withParam, withSequence, continueOn, onExit, lifecycle hooksemptyDir, sidecars, init containers, node selectors, affinity, tolerations, service accountsCronWorkflow schedules, concurrency policy, timezone, Argo Events integrationFor each component identified in Phase 1, classify it using the mapping type (direct / approximate / absent). Use the decision logic below and the full tables in references/concept-map.md.
Direct or near-direct translations (translate automatically):
dag success-path dependencies -> pipeline step wiringwithSequence -> Python range(...) in a dynamic pipelineStepRetryConfig(...)Approximate translations (translate with caveats):
Workflow / WorkflowTemplate -> @pipeline plus a pipeline run or reusable Python moduleCronWorkflow -> Schedule(...) on a supported orchestratorcontainer / script template -> @step with Python logic, DockerSettings, or subprocess.run(...)outputs.result -> explicit step return valuewhen on values -> dynamic pipeline branching or explicit soft-conditional logicwithItems / withParam -> dynamic fanout using real list artifacts and .map()onExit -> redesign using hooks, idempotent cleanup, execution modes, or external cleanup controlresource / http templates -> explicit SDK or API calls inside stepsAbsent / redesign-first patterns (flag for human review):
depends expressions based on task status (A.Failed, B.Errored, etc.)containerSetsuspend templates unless the target project is on a recent 0.94.x release with zenml.wait(...) support (preferably >=0.94.1), and even then only as an explicit redesignemptyDir filesystem contracts across stepsEventSource + Sensor + dependency logic)Before writing any code, present a summary to the user:
"Here's what I found in your Argo workflow:
- Direct translations (will migrate cleanly): [list]
- Approximate translations (will work but with noted caveats): [list]
- Needs redesign (cannot auto-migrate): [list with brief explanation]
Shall I proceed with the migration?"
If there are HIGH-severity flags, explain each one concretely: what the Argo workflow does, why ZenML cannot reproduce it directly, and what the redesign options look like.
Translate the Argo workflow into a ZenML project. Follow these conventions strictly.
Every migrated project MUST use this layout:
migrated_pipeline/
├── steps/ # One file per step
│ ├── extract.py
│ ├── transform.py
│ └── load.py
├── pipelines/
│ └── my_pipeline.py # Pipeline definition
├── materializers/ # Custom materializers (if needed)
├── configs/
│ ├── dev.yaml
│ └── prod.yaml
├── run.py # CLI entry point (argparse, not click)
├── README.md
└── pyproject.toml
This matches the zenml-pipeline-authoring skill's conventions. Key rules:
steps/run.py uses argparse (click conflicts with ZenML)pyproject.toml uses requires-python = ">=3.12" and zenml>=0.94.1 as the dependency floorconfigs/dev.yaml and configs/prod.yamlREADME.md explaining what was migrated and what still needs manual attentionzenml init at the project rootSee references/code-patterns.md for side-by-side examples covering DAGs, steps templates, loops, conditionals, artifacts, CronWorkflows, exit handlers, shared-volume redesigns, and Argo Events integration patterns.
The core translation rule: move business logic out of YAML templates and into typed Python functions. Treat the YAML graph as a description of orchestration, then rebuild that orchestration in Python using step calls and artifact wiring.
# Argo mental model
# template "extract" writes a value to a file or stdout
# ZenML translation
@step
def extract() -> list[int]:
return [1, 2, 3]
Parameters -> function arguments:
@pipeline
def training_pipeline(dataset: str, learning_rate: float = 0.1) -> None:
prepared = prepare_data(dataset=dataset)
train_model(data=prepared, learning_rate=learning_rate)
Output files / outputs.result -> step returns:
@step
def read_threshold() -> float:
# Migration note: Argo previously read this from `valueFrom.path`.
# ZenML treats the extracted value as a normal typed return.
return 0.8
File artifacts -> typed artifacts or explicit Path contracts:
from pathlib import Path
@step
def produce_manifest() -> Path:
output = Path("/tmp/manifest.json")
output.write_text("{\"ok\": true}")
return output
Use a file- or path-based contract only when the original workflow genuinely depends on file identity or layout. Otherwise prefer typed domain objects.
CronWorkflow -> Schedule:
from zenml.config.schedule import Schedule
schedule = Schedule(cron_expression="0 2 * * *")
my_pipeline.with_options(schedule=schedule)()
Always note that scheduling is orchestrator-dependent and that CronWorkflow-specific behavior like timezone and concurrency policy may need extra handling.
Suspend / pause-resume note:
If the source workflow uses suspend, do not present ZenML wait/resume as a generic drop-in replacement. Treat it as a redesign option that requires a recent 0.94.x release -- preferably zenml>=0.94.1, where zenml.wait(...) and pipeline run resume are documented in the official changelog.
Keep migration-related comments short and actionable:
# Migration note: for brief inline caveats# TODO(migration): for unsupported patterns or manual follow-upWhen translating approximate patterns, add a brief note explaining the semantic shift:
@step
def run_cli(command: list[str]) -> str:
# Migration note: Argo container templates can run any image + command.
# This ZenML step runs in a Python-capable environment and wraps the CLI
# with subprocess. Verify tooling is present in the step image.
import subprocess
completed = subprocess.run(command, check=True, text=True, capture_output=True)
return completed.stdout
For patterns that have no ZenML equivalent, do NOT silently approximate them. Instead:
# TODO(migration) comment in the generated code# TODO(migration): UNSUPPORTED -- Argo enhanced depends expression
# `cleanup` previously ran when `train.Failed || validate.Errored`.
# ZenML cannot branch on upstream failure states this way. Redesign with
# explicit status artifacts, hooks, or separate alerting/cleanup flows.
@step
def cleanup_after_failure(status: dict[str, bool]) -> None:
...
After generating the ZenML project, produce a MIGRATION_REPORT.md in the project root.
# Migration Report: [Argo Workflow] -> [ZenML Pipeline]
## Summary
- **Source**: Argo `[kind]` `[name]`
- **Target**: ZenML pipeline `[pipeline_name]`
- **Templates migrated**: X direct, Y approximate, Z flagged
## Direct Translations
| Argo Template / Concept | ZenML Equivalent | Notes |
|---|---|---|
| parameters | pipeline args | Clean translation |
## Approximate Translations
| Argo Template / Concept | ZenML Equivalent | What Changed |
|---|---|---|
| CronWorkflow | Schedule(...) | Schedule support depends on orchestrator; concurrency policy may need extra handling |
## Flagged for Review
| Argo Pattern | Severity | Issue | Suggested Redesign |
|---|---|---|---|
| containerSet | HIGH | No multi-container same-pod primitive in ZenML | Collapse into one step or externalize to Kubernetes-native service |
## Scheduling
- **Original**: CronWorkflow `0 2 * * *`, timezone `UTC`
- **Migrated**: `Schedule(cron_expression="0 2 * * *")`
- **Note**: Verify orchestrator scheduling support and any concurrency / timezone semantics
## Kubernetes-Native / Infrastructure Assumptions
| Original Argo Assumption | Migration Status | Notes |
|---|---|---|
| Shared PVC | Flagged | Redesign around artifacts or explicit external storage |
## Limitations and Key Differences
[Summarize the most important semantic shifts before listing benefits.]
## What's NOT Migrated
[List Argo features outside the portable ZenML scope: Sensors, EventSources, cluster policy, sidecars, etc.]
## What You Get for Free After Migration
- typed artifacts and artifact lineage
- step caching
- stack abstraction
- service connectors and secrets management
- Model Control Plane for ML workflows
## Recommended Next Steps
1. Run the `zenml-quick-wins` skill
2. Install the ZenML docs MCP server
3. Review every HIGH-severity redesign item
4. Use the `zenml-pipeline-authoring` skill for Docker settings, custom materializers, deployment, or deeper configuration
After migration is complete, always include a "Recommended Next Steps" section in the migration report AND communicate it to the user.
zenml-quick-wins skillAlways suggest this as the immediate next step:
"Now that the migration is done, I'd recommend running the
zenml-quick-winsskill to add metadata logging, experiment tracking, alerters, and other production-readiness features."
For every flagged pattern, include a link to the relevant ZenML documentation:
https://docs.zenml.io/how-to/steps-pipelines/schedule-a-pipelinehttps://docs.zenml.io/how-to/steps-pipelines/dynamic-pipelineshttps://docs.zenml.io/how-to/steps-pipelines/trigger-a-pipelinehttps://docs.zenml.io/stacks/stack-components/orchestratorshttps://docs.zenml.io/how-to/containerization/containerizationhttps://docs.zenml.io/how-to/project-setup-and-management/secret-managementhttps://docs.zenml.io/how-to/infrastructure-deployment/auth-management"For easier access to ZenML documentation while you work, you can install the ZenML docs MCP server:
claude mcp add zenmldocs --transport http https://docs.zenml.io/~gitbook/mcp"
When the migration has HIGH-severity flags, offer to help the user get support from the ZenML community.
When there are 2+ HIGH-severity flags, generate a pre-made Slack message for zenml.io/slack that includes:
**Argo -> ZenML Migration Help**
I'm migrating an Argo Workflow that uses [patterns]. The migration skill flagged these as needing redesign:
1. **[Pattern]**: [brief description + Argo snippet]
- Suggested workaround: [X]
2. **[Pattern]**: [brief description + Argo snippet]
- Suggested workaround: [Y]
I'm looking for advice on whether there is a better ZenML-native pattern or an upcoming feature that would fit this use case.
When the migration reveals a real missing feature in ZenML, offer to open a GitHub issue on zenml-io/zenml using gh issue create. Include the Argo pattern, why it matters, what workaround was attempted, and why the gap is broadly useful.
/simplify to clean up the migrated codeAlways suggest running /simplify after the migration:
"The migration is done. I'd recommend running
/simplifyon the generated code to clean up migration comments, reduce duplication, and make the result feel more like native ZenML code."
zenml-pipeline-authoringThe zenml-pipeline-authoring skill handles deeper customization:
These are the most common sources of confusion after migration. Always mention the relevant ones in the migration report.
Argo YAML is the execution program. ZenML Python is the execution program. That means a migration rewrites orchestration into code; it does not merely reformat manifests.
In Argo, file paths, output files, and artifact repository wiring are often part of the workflow contract. In ZenML, the typed value is the contract, and storage is delegated to the artifact store plus materializers.
Argo assumes Kubernetes everywhere. ZenML can run on Kubernetes, but it treats Kubernetes as one execution backend among several. Pod-level settings need to be isolated as infrastructure concerns, not mixed into portable pipeline logic.
Argo patterns built around shared PVCs, emptyDir, sidecars, init containers, or same-pod container coordination do not map cleanly to independent ZenML steps. These are redesign hotspots.
Argo can branch on task result states directly. ZenML control flow is much more naturally expressed as value-based logic in Python. If the old behavior depends on failure states, model that explicitly and document the semantic change.
| Anti-pattern | Why it's wrong | What to do instead |
|---|---|---|
| Translating YAML templates 1:1 into step files without redesigning data flow | Preserves Argo syntax shape but not ZenML semantics | Design step interfaces around typed args, returns, and artifacts |
| Treating output parameters as ZenML pipeline parameters | Upstream-produced values are artifacts in ZenML | Return values from steps and wire them downstream |
Returning /tmp/... paths from one step and assuming another step can read them | ZenML steps are isolated; local paths are not stable contracts | Return typed data, or explicitly model a file / URI contract |
| Reusing arbitrary non-Python images unchanged | ZenML step execution still expects a Python-capable runtime | Build a Python-capable image or keep that execution outside ZenML |
Translating depends: "A.Failed" into a naive Python if after A() | A failing ZenML step changes run behavior; it is not just a boolean value | Model failure as data, use hooks, or split flows |
Replacing shared-volume logic with /tmp across multiple steps | /tmp is container-local, not workflow-global | Collapse into one step or externalize state |
Rewriting withParam as JSON strings passed between steps | Keeps Argo's serialization hack and loses ZenML typing | Return real list[...] artifacts and fan out dynamically |
Mapping onExit to a success hook | Success hooks do not behave like workflow finalizers | Use try/finally, failure hooks, or external cleanup orchestration |
| Claiming Argo Events maps directly to native ZenML event graphs | Public ZenML docs do not describe Argo-Events-style parity | Use external event infrastructure to invoke ZenML runs |
For topics beyond migration (stack setup, experiment tracking, deployment, or orchestrator setup), query the ZenML docs at https://docs.zenml.io.