ID Unification Validation Command

Purpose

MANDATORY validation gate that checks ALL generated unification files against exact templates from agent prompts. This prevents deployment of incorrect configurations.

⚠️ CRITICAL: This command MUST complete successfully before td wf push or workflow execution.

What This Command Validates

1. File Existence Check

✅ unification/unif_runner.dig exists
✅ unification/dynmic_prep_creation.dig exists
✅ unification/id_unification.dig exists
✅ unification/enrich_runner.dig exists
✅ unification/config/environment.yml exists
✅ unification/config/src_prep_params.yml exists
✅ unification/config/unify.yml exists
✅ unification/config/stage_enrich.yml exists
✅ All SQL files in unification/queries/ exist
✅ All SQL files in unification/enrich/queries/ exist

2. Template Compliance Check

unif_runner.dig Validation:

✅ Uses require> operator (NOT call>)
✅ No echo> operators with subtasks
✅ Matches exact template from /plugins/cdp-unification/prompt.md lines 186-217
✅ Has _error: section with email_alert
✅ Includes both config/environment.yml and config/src_prep_params.yml

stage_enrich.yml Validation:

✅ RULE 1: unif_input table has column and key both using alias_as
✅ RULE 2: Staging tables have column using col.name and key using alias_as
✅ All key_columns match actual columns from src_prep_params.yml
✅ No template columns (like adobe_clickstream, loyalty_id_std)
✅ Table names match src_tbl (NO _prep suffix)

enrich_runner.dig Validation:

✅ Matches exact template from unification-staging-enricher.md lines 261-299
✅ Includes all 3 config files in _export
✅ Uses td_for_each> for dynamic execution
✅ Has Presto and Hive conditional execution

3. Database & Table Existence Check

✅ ${client_short_name}_${src} database exists
✅ ${client_short_name}_${stg} database exists
✅ ${client_short_name}_${gld} database exists (if used)
✅ ${client_short_name}_${lkup} database exists
✅ cdp_unification_${unif_name} database exists
✅ ${client_short_name}_${lkup}.exclusion_list table exists

4. Configuration Validation

✅ All variables in environment.yml are defined
✅ All tables in src_prep_params.yml exist in source database
✅ All columns in src_prep_params.yml exist in source tables
✅ unify.yml merge_by_keys match src_prep_params.yml alias_as columns
✅ No undefined variables (${...})

5. YAML Syntax Check

✅ All YAML files have valid syntax
✅ Proper indentation (2 spaces)
✅ No tabs in YAML files
✅ All strings properly quoted where needed

Validation Report Format

╔══════════════════════════════════════════════════════════════╗
║          ID UNIFICATION VALIDATION REPORT                    ║
╚══════════════════════════════════════════════════════════════╝

[1/5] File Existence Check
  ✅ unification/unif_runner.dig
  ✅ unification/dynmic_prep_creation.dig
  ✅ unification/id_unification.dig
  ✅ unification/enrich_runner.dig
  ✅ unification/config/environment.yml
  ✅ unification/config/src_prep_params.yml
  ✅ unification/config/unify.yml
  ✅ unification/config/stage_enrich.yml
  ✅ 3/3 SQL files in queries/
  ✅ 4/4 SQL files in enrich/queries/

[2/5] Template Compliance Check
  ✅ unif_runner.dig uses require> operator
  ✅ unif_runner.dig has no echo> conflicts
  ✅ stage_enrich.yml RULE 1 compliant (unif_input table)
  ✅ stage_enrich.yml RULE 2 compliant (staging tables)
  ❌ stage_enrich.yml has incorrect mapping on line 23
      Expected: column: email_address_std
      Found:    column: email
      FIX: Update line 23 to use col.name from src_prep_params.yml

[3/5] Database & Table Existence
  ✅ client_src exists
  ✅ client_stg exists
  ✅ client_gld exists
  ✅ client_config exists
  ❌ client_config.exclusion_list does NOT exist
      FIX: Run: td query -d client_config -t presto -w "CREATE TABLE IF NOT EXISTS exclusion_list (key_value VARCHAR, key_name VARCHAR, tbls ARRAY(VARCHAR), note VARCHAR)"

[4/5] Configuration Validation
  ✅ All variables defined in environment.yml
  ✅ Source table client_stg.snowflake_orders exists
  ✅ All columns exist in source table
  ✅ unify.yml keys match src_prep_params.yml

[5/5] YAML Syntax Check
  ✅ All YAML files have valid syntax
  ✅ Proper indentation
  ✅ No tabs found

╔══════════════════════════════════════════════════════════════╗
║                    VALIDATION SUMMARY                        ║
╚══════════════════════════════════════════════════════════════╝

Total Checks: 45
Passed: 43 ✅
Failed: 2 ❌

❌ VALIDATION FAILED - DO NOT DEPLOY

Required Actions:
1. Fix stage_enrich.yml line 23 mapping
2. Create client_config.exclusion_list table

Re-run validation after fixes: /cdp-unification:unify-validate

Error Codes

EXIT 0: All validations passed ✅
EXIT 1: File existence failures
EXIT 2: Template compliance failures
EXIT 3: Database/table missing
EXIT 4: Configuration errors
EXIT 5: YAML syntax errors

Usage

Standalone:

/cdp-unification:unify-validate

Auto-triggered in unify-setup (MANDATORY step before deployment)

Manual validation before deployment:

cd unification
/cdp-unification:unify-validate

If validation PASSES → Proceed with td wf push unification If validation FAILS → Fix errors and re-validate

Integration with unify-setup

The /unify-setup command will automatically:

Generate all unification files
RUN VALIDATION (this command)
BLOCK deployment if validation fails
Show fix instructions for each error
Auto-retry validation after fixes
Only proceed to deployment after 100% validation success

Success Criteria

✅ ALL checks must pass before deployment is allowed ✅ No exceptions - even 1 failure blocks deployment ✅ Detailed error messages with exact fix instructions ✅ Auto-remediation suggestions where possible

Let's validate your unification files!

/unify-validate