Advanced Orchestration Patterns Agent

Specialist agent implementing five core patterns for complex multi-agent workflows: Blackboard (collaborative knowledge space), Circuit Breaker (fault detection), Dynamic Replanning (adaptive strategy), Hierarchical Decomposition (recursive task breakdown), and Saga (distributed transaction consistency).

Pattern 1: Blackboard Pattern

Concept: Shared knowledge space where specialized agents contribute domain expertise to collaboratively solve complex problems.

State Structure: Blackboard with problem statement, knowledge entries (observations, hypotheses, solutions, constraints), synthesis summary, and solution candidates ranked by confidence.

Workflow:

1. Initialize blackboard with problem and identify 3-7 expert agents
2. Spawn parallel agents to contribute observations, hypotheses, constraints with confidence scores
3. Monitor convergence: knowledge saturation >0.8, consensus >0.6, solution confidence >0.75
4. Select solution with highest confidence + agent votes
5. Archive blackboard to Obsidian with extracted patterns

Key Metrics: Convergence time <10 min, solution confidence >0.75, agent consensus >60%, knowledge reuse >40%

---

Pattern 2: Circuit Breaker Pattern

Concept: Detect and isolate failing agents/phases to prevent cascading failures. Auto-disable problematic components and gracefully degrade functionality.

States:

• CLOSED: Normal operation, all requests pass through, monitor for failures
• OPEN: Agent disabled after threshold failures, fail fast, return cached results or fallback
• HALF_OPEN: Allow limited requests to test recovery, close if success threshold met, reopen on failure

Workflow:

1. Track all agent executions with success/failure classification (timeout, exception, validation)
2. Increment failure count on agent error; open circuit if threshold exceeded
3. When OPEN: use cached results, fallback agent, or degraded mode
4. After timeout, transition to HALF_OPEN to test recovery
5. Update Jira issue with degradation comment

Fallback Strategies: Define per-agent with fallback_agent, cache_duration, degraded_features

Key Metrics: Failure detection <60s, recovery success >80%, false positive <5%, degraded mode availability >95%

---

Pattern 3: Dynamic Replanning

Concept: Continuously evaluate plan execution and adaptively replan when blockers are encountered, strategies fail, or better paths are discovered.

Workflow:

1. Monitor phase execution every 5 minutes; compute metrics (progress, velocity, risk, blocker impact)
2. Detect blockers from agent failures, external dependencies, changed requirements, violated constraints
3. Generate alternative strategies with confidence, effort, risk, benefit analysis
4. Compare current plan vs best alternative considering switch costs
5. If replan: save version, create new plan, preserve completed work, update Jira, resume

Replan Triggers: Critical blocker, velocity <0.5, risk >0.75, critical health, deadline at risk

Key Metrics: Decision time <5 min, success rate >85%, alternative quality >0.7, unnecessary replans <10%

---

Pattern 4: Hierarchical Decomposition

Concept: Recursively break down complex tasks into smaller subtasks (max 5 levels deep), automatically parallelize independent tasks, and aggregate results bottom-up.

Decomposition Rules: Decompose if complexity >13, level <5, divisible, not atomic. Strategies: by_layer, by_user_journey, by_component, by_phase.

Workflow:

1. Recursively decompose root task using appropriate strategy; max depth 5 levels
2. Build dependency graph (DAG); validate no circular dependencies; compute critical path
3. Identify tasks ready to execute (dependencies met); group into parallel groups
4. Execute tasks level by level (deepest first) with parallel concurrency
5. Aggregate results bottom-up: concatenation, union, or synthesis

Aggregation Strategies:

• Sequential: Combine outputs sequentially
• Parallel: Merge outputs, take max effort
• Synthesis: Use blackboard to synthesize complementary results

Key Metrics: Parallelization efficiency >60%, decomposition overhead <20%, optimal granularity 5-8 pts, critical path optimization >40%

---

Pattern 5: Saga Pattern

Concept: Manage distributed transactions across multiple agents with compensating actions for rollback. Ensures state consistency when agents fail mid-workflow.

Saga Structure: Steps with forward action, compensating action, agent, status (completed/pending/failed/compensated). Compensation log tracks rollback actions.

Choreography:

• Forward Phase: Execute steps sequentially; if success proceed, if failure stop and begin compensation
• Compensation Phase: Execute compensating actions in reverse order; log compensation result; best effort (flag manual intervention if compensation fails)

Workflow:

1. Define saga steps with forward and compensating actions; validate each step has compensation
2. Execute forward phase: record results, stop on failure
3. Execute compensation in reverse if any step fails
4. Persist saga state after each step for crash recovery
5. Generate reconciliation report; create manual fix tasks if needed

Saga Patterns: Sequential (user registration), Parallel (multi-service deployment), Nested (e-commerce order with nested payment saga)

Key Metrics: Compensation success >95%, state consistency >99%, recovery time <2 min, throughput >10/min

---

Pattern Integration

Combining Patterns: Use Hierarchical Decomposition for large tasks, Blackboard for complex planning decisions, Circuit Breaker for external dependencies, Dynamic Replanning for handling blockers, Saga for distributed transactions.

Pattern Selection Matrix:

• Complex problem, unclear solution → Blackboard + Dynamic Replanning
• Large task, many subtasks → Hierarchical Decomposition
• Unreliable external services → Circuit Breaker
• Multi-agent data consistency → Saga
• High-stakes workflow → All patterns
• Simple linear workflow → Basic orchestration only

Graceful Degradation: Circuit Breaker detects failure → Dynamic Replanning evaluates alternatives → Replan if alternative exists, else degraded mode → Saga ensures consistency → Blackboard preserves knowledge.

---

Best Practices

Pattern Selection:

• Start simple; don't use patterns unless complexity warrants
• Combine thoughtfully; patterns complement each other
• Monitor overhead; pattern coordination has cost
• Document decisions; log why patterns were chosen

State Management:

• Persist frequently; save state after significant steps
• Idempotent operations; actions safely retryable
• Version tracking; track state schema versions
• Cleanup; archive old state to Obsidian

Performance:

• Lazy loading; load pattern state on-demand
• Caching; cache frequently accessed state
• Batch operations; group related updates
• Async execution; use Task tool for parallel work

Error Recovery:

• Graceful degradation; always have fallback
• Clear error messages; help users understand what happened
• Actionable logs; include remediation steps
• Human escalation; know when to involve humans

---

Integration with Jira Orchestrator

When to Activate:

• Story points >13 → hierarchical_decomposition
• Requires architecture decision → blackboard
• Has external dependencies → circuit_breaker
• Requires distributed transaction → saga
• Multiple patterns → dynamic_replanning

Status Reporting: Update Jira with pattern execution status, task counts, circuit breaker health, active blackboards, replanning events, active sagas.

---

Key Takeaways

1. Patterns solve specific coordination challenges
2. Combine patterns for maximum effectiveness
3. Monitor pattern performance and adjust
4. Document pattern usage for learning
5. Always have graceful degradation strategies

For pattern enhancement proposals, document in Obsidian: System/Orchestration-Patterns/