Workflow Compose

workflow.compose

Purpose

Allows declarative execution of Betty Framework workflows by reading a YAML definition and chaining skills like skill.create, skill.define, and registry.update.

Enables complex multi-step processes to be defined once and executed reliably with proper error handling and audit logging.

Usage

Basic Usage

python skills/workflow.compose/workflow_compose.py <path_to_workflow.yaml>

Arguments

Argument	Type	Required	Description
workflow_path	string	Yes	Path to the workflow YAML file to execute

Workflow YAML Structure

# workflows/create_and_register.yaml
name: "Create and Register Skill"
description: "Complete lifecycle: create, validate, and register a new skill"

steps:
  - skill: skill.create
    args: ["workflow.validate", "Validates workflow definitions"]
    required: true

  - skill: skill.define
    args: ["skills/workflow.validate/skill.yaml"]
    required: true

  - skill: registry.update
    args: ["skills/workflow.validate/skill.yaml"]
    required: false  # Continue even if this fails

Workflow Fields

Field	Required	Description	Example
name	No	Workflow name	"API Design Workflow"
description	No	What the workflow does	"Complete API lifecycle"
steps	Yes	Array of steps to execute	See below

Step Fields

Field	Required	Description	Example
skill	Yes	Skill name to execute	api.validate
args	No	Arguments to pass to skill	["specs/api.yaml", "zalando"]
required	No	Stop workflow if step fails	true (default: false)

Behavior

1. Load Workflow: Parses the workflow YAML file
2. Sequential Execution: Runs each step in order
3. Error Handling:
   • If required: true, workflow stops on failure
   • If required: false, workflow continues and logs error
4. Audit Logging: Calls audit.log skill (if available) for each step
5. History Tracking: Records execution history in /registry/workflow_history.json

Outputs

Success Response

{
  "ok": true,
  "status": "success",
  "errors": [],
  "path": "workflows/create_and_register.yaml",
  "details": {
    "workflow_name": "Create and Register Skill",
    "steps_executed": 3,
    "steps_succeeded": 3,
    "steps_failed": 0,
    "duration_ms": 1234,
    "history_file": "/registry/workflow_history.json"
  }
}

Partial Failure Response

{
  "ok": false,
  "status": "failed",
  "errors": [
    "Step 2 (skill.define) failed: Missing required fields: version"
  ],
  "path": "workflows/create_and_register.yaml",
  "details": {
    "workflow_name": "Create and Register Skill",
    "steps_executed": 2,
    "steps_succeeded": 1,
    "steps_failed": 1,
    "failed_step": "skill.define",
    "failed_step_index": 1
  }
}

Example Workflow Files

Example 1: Complete Skill Lifecycle

# workflows/create_and_register.yaml
name: "Create and Register Skill"
description: "Scaffold, validate, and register a new skill"

steps:
  - skill: skill.create
    args: ["workflow.validate", "Validates workflow definitions"]
    required: true

  - skill: skill.define
    args: ["skills/workflow.validate/skill.yaml"]
    required: true

  - skill: registry.update
    args: ["skills/workflow.validate/skill.yaml"]
    required: true

Execution:

$ python skills/workflow.compose/workflow_compose.py workflows/create_and_register.yaml
{
  "ok": true,
  "status": "success",
  "details": {
    "steps_executed": 3,
    "steps_succeeded": 3
  }
}

Example 2: API Design Workflow

# workflows/api_design.yaml
name: "API Design Workflow"
description: "Design, validate, and generate models for new API"

steps:
  - skill: api.define
    args: ["user-service", "openapi", "zalando", "specs", "1.0.0"]
    required: true

  - skill: api.validate
    args: ["specs/user-service.openapi.yaml", "zalando", "true"]
    required: true

  - skill: api.generate-models
    args: ["specs/user-service.openapi.yaml", "typescript", "src/models"]
    required: false  # Continue even if model generation fails

Example 3: Multi-Spec Validation

# workflows/validate_all_specs.yaml
name: "Validate All API Specs"
description: "Validate all OpenAPI specifications in specs directory"

steps:
  - skill: api.validate
    args: ["specs/users.openapi.yaml", "zalando"]
    required: false

  - skill: api.validate
    args: ["specs/orders.openapi.yaml", "zalando"]
    required: false

  - skill: api.validate
    args: ["specs/payments.openapi.yaml", "zalando"]
    required: false

Workflow History

Execution history is logged to /registry/workflow_history.json:

{
  "executions": [
    {
      "workflow_path": "workflows/create_and_register.yaml",
      "workflow_name": "Create and Register Skill",
      "timestamp": "2025-10-23T12:34:56Z",
      "status": "success",
      "steps_executed": 3,
      "steps_succeeded": 3,
      "duration_ms": 1234
    }
  ]
}

Audit Integration

If audit.log skill is available, each step execution is logged:

log_audit_entry(
    skill_name="api.validate",
    status="success",
    duration_ms=456,
    metadata={"workflow": "api_design.yaml", "step": 1}
)

Integration

With workflow.validate

Validate workflow syntax before execution:

# Validate first
python skills/workflow.validate/workflow_validate.py workflows/my-workflow.yaml

# Then execute
python skills/workflow.compose/workflow_compose.py workflows/my-workflow.yaml

With Hooks

Auto-validate workflows when saved:

python skills/hook.define/hook_define.py \
  --event on_file_save \
  --pattern "workflows/*.yaml" \
  --command "python skills/workflow.validate/workflow_validate.py {file_path}" \
  --blocking true

In CI/CD

# .github/workflows/test.yml
- name: Run workflow tests
  run: |
    python skills/workflow.compose/workflow_compose.py workflows/test_suite.yaml

Common Errors

Error	Cause	Solution
"Workflow file not found"	Path incorrect	Check workflow file path
"Invalid YAML in workflow"	Malformed YAML	Fix YAML syntax errors
"Skill handler not found"	Referenced skill doesn't exist	Ensure skill is registered or path is correct
"Step X failed"	Skill execution failed	Check skill's error output, fix issues
"Skill execution timed out"	Skill took >5 minutes	Optimize skill or increase timeout in code

Best Practices

1. Validate First: Run workflow.validate before executing workflows
2. Use Required Judiciously: Only mark critical steps as required: true
3. Small Workflows: Keep workflows focused on single logical task
4. Error Handling: Plan for partial failures in non-required steps
5. Test Workflows: Test workflows in development before using in production
6. Version Control: Keep workflow files in git

Files Modified

• History: /registry/workflow_history.json – Execution history
• Logs: Step execution logged to Betty's logging system

Exit Codes

• 0: Success (all required steps succeeded)
• 1: Failure (at least one required step failed)

Timeout

Each skill has a 5-minute (300 second) timeout by default. If a skill exceeds this, the workflow fails.

See Also

• workflow.validate – Validate workflow syntax (workflow.validate SKILL.md)
• Betty Architecture – Five-Layer Model for understanding workflows
• API-Driven Development – Example workflows

Status

Active – Production-ready, core orchestration skill

Version History

• 0.1.0 (Oct 2025) – Initial implementation with sequential execution, error handling, and audit logging