oracle-ai-data-platform-workbench-databricks-migrator

Pre-migration data-availability scan. Reads a manifest, probes every spark.read.* / saveAsTable target on the AIDP cluster, reports OK / MISSING / EMPTY.

Guided catalog migration flow. Extract Databricks Unity Catalog / HMS metadata, preview the 18-rule DDL rewrite, then batched replay on AIDP. Asks before destructive replay step.

Guided full-job migration flow. Walks the user from "I want to migrate this Databricks job" through DAG → data check → migrate → status, asking before each phase.

Parse + summarize the JOB_REPORT.md produced by a prior migration run. Surfaces cell pass/fail/fix counts per task, lists the failing cells, and recommends a next-step skill (fixup-cell, resume, or all-clear).

Use this agent when the user has pointed at a single Databricks notebook (.ipynb / .py / .scala) and wants to know what it does, what it depends on, what risks the migrator will hit, and which gotchas (from references/gotchas.md) will apply. Run before manually authoring an entry in a migration manifest, OR as a one-off "should we migrate this?" assessment. Returns a structured report (markdown), does not modify the notebook.

Use this agent after aidp-migrate-job completes to review a migrated .ipynb for correctness (NOT just "did it run"). Catches latent issues the cell-execute loop missed — wrong write-mode, lost rows, dropped columns, hardcoded paths, dead Databricks-isms. Outputs a structured review report.

Build a migration manifest (the execution DAG) from a Databricks workspace path or workflow ID. Walks %run dependency chains, captures dbutils.notebook.run invocations, and emits reports/<job>_manifest.json — the input every other execute-skill consumes. Use when the user wants to see what would migrate, or before invoking aidp-migrate-job for the first time on a new workload.

Pre-migration data-availability scan. Reads every notebook in a migration manifest, extracts every spark.read.table / spark.read.parquet / saveAsTable reference, and probes whether each target schema/table/path exists on the AIDP cluster BEFORE you spend Pass-2 cluster time. Use after aidp-build-dag and before aidp-migrate-job, especially the first time you migrate against a target environment.

Targeted rewind of a migrated notebook. Re-executes cells from history index N onwards (or a specific cell range) through the execute+verify+fix loop, with a 'why' reason injected so Claude knows what to fix. Use when aidp-migrate-job left a notebook at RESULT=PARTIAL or the user identifies a specific cell that is wrong post-migration.

Migrate Databricks Unity Catalog / HMS schemas + tables onto AIDP. Two-stage extract→rewrite→replay pipeline. The rewriter applies 18 DDL rules (3-part→2-part flatten, s3://→oci:// via bucket-map, source-format preserved (Delta stays Delta) with delta.* property scrub, MV/streaming rejection, CREATE SCHEMA COMMENT-colon strip). Batched single-WebSocket DDL replay works around AIDP's per-statement-discard quirk. Use BEFORE aidp-migrate-job — schemas must exist before notebook reads.

YAML-driven acceptance contract for batch and streaming pipelines that need convergence verification, not just "no exceptions". Declares PASS only after K consecutive zero-result windows (consecutive-zero-window convergence). Use for streaming jobs, slowly-draining backfills, or any pipeline whose success is "the pending queue is empty".