alive:system-upgrade

System Upgrade

Upgrade a world from any previous version of the ALIVE Context System to the current version. Handles structural renames, file migrations, terminology updates, legacy cleanup, and integrity verification.

This skill has been battle-tested on a 16GB world with 142 walnuts, 88 bundles, 81 people, and 6 distinct legacy patterns. Every edge case below comes from a real upgrade.

---

When It Fires

• The session-new hook detects a legacy structure (.walnut/, _core/, _capsules/, companion.md, bundles/, _kernel/_generated/, tasks.md, observations.md)
• The human explicitly invokes /alive:system-upgrade
• The human says "upgrade my world", "migrate to the new version", "update alive"

---

Version Reference

Component	v1	v2	v3
System dir	_core/	_kernel/_generated/	_kernel/ (flat)
Bundles	_core/_capsules/	bundles/	walnut root (flat)
Manifest	companion.md	context.manifest.yaml	context.manifest.yaml
Tasks	N/A	tasks.md	tasks.json (script-operated)
State	now.md	_kernel/_generated/now.json	_kernel/now.json (script-generated)
Observations	N/A	observations.md	removed (stash -> log)
Inbox domain	03_Inputs/	03_Inputs/	03_Inbox/

---

Process

Phase 1: Mine Existing System

Before touching anything, understand what's there. Dispatch a scout agent (or multiple in parallel) to map the full world structure.

Core structure scan:

• .walnut/ vs .alive/ -- which system folder exists? Do BOTH exist (merge needed)?
• _core/ vs _kernel/ -- which kernel structure is in use?
• _kernel/_generated/ present? (v2 intermediate directory -- needs flattening in v3)
• _capsules/ vs bundles/ vs flat-at-root -- which bundle structure is in use?
• companion.md vs context.manifest.yaml -- which manifest format exists?
• now.md vs _kernel/_generated/now.json vs _kernel/now.json -- which state format is in use?
• tasks.md present anywhere? (v2 -- needs conversion to tasks.json in v3)
• observations.md present in any bundles? (v2 -- removed in v3)
• Walnut count, people count, bundle/capsule count
• Squirrel entries and their format
• Custom skills, rules, hooks in the human's space
• .claude/ configuration referencing old paths

Legacy structure detection (critical -- older ALIVE versions used different layouts):

• Un-numbered domain folders at world root: archive/, life/, ventures/, experiments/, inbox/, docs/, product/, plugin/, _working/ -- these are from pre-ALIVE-framework eras when domains didn't have numbered prefixes
• _brain/ folders inside walnuts (v3 era state management)
• _state/ folders inside walnuts (v3 era)
• Flat walnut structures (key.md at walnut root, no _core/ or _kernel/)
• src/ or other code directories orphaned at world root from website development
• Any folder at world root that is NOT: 01_Archive/, 02_Life/, 03_Inbox/, 04_Ventures/, 05_Experiments/, .alive/, .claude/, or standard dotfiles

v2 structure detection (for v2->v3 upgrade):

• bundles/ directory inside any walnut root (v2 -- bundles flatten to walnut root in v3)
• _kernel/_generated/ directory structure (v2 -- _kernel/ is flat in v3)
• tasks.md files anywhere under any walnut (v2 -- converted to tasks.json in v3)
• observations.md files inside bundles (v2 -- removed in v3)

Duplicate walnut detection:

• Scan for person walnuts that appear in multiple locations (e.g., people/professional/jane/ AND people/jane-smith/)
• Compare key.md content: which is the stub, which has real data?
• Flag all duplicates for human review before upgrade

World size check:

• Run du -sh on the world root
• If >1GB: warn that git is impractical, recommend tarball backup
• If >5GB: flag specific heavy directories (likely code repos with node_modules)

Sync script audit:

• Read preferences.yaml context_sources: for configured sync scripts
• Check .claude/scripts/ for any scripts with hardcoded paths
• Flag any that reference _core/, _capsules/, .walnut/, companion.md, now.md, _kernel/_generated/, bundles/, tasks.md, observations.md, or un-numbered domain paths like inbox/ instead of 03_Inbox/

Root-level now.md duplicates:

• Some walnuts have now.md at BOTH the walnut root AND inside _core/
• Detect all duplicates, flag for merge (use _core/ version as canonical)

_core/ directories with only _capsules/ inside:

• Some _core/ dirs contain only _capsules/ (no key.md, now.md, etc.)
• These still need renaming -- include them in the kernel rename phase

╭─ 🐿️ system scan complete
│
│  Current version: [detected]
│  System folder: [.walnut/ | .alive/ | both]
│  Kernel: [_core/ | _kernel/_generated/ | _kernel/ (flat) | mixed]
│  Bundles: [_capsules/ | bundles/ | flat at root | mixed]
│  State: [now.md | _kernel/_generated/now.json | _kernel/now.json | mixed]
│  Tasks: [tasks.md | tasks.json | none | mixed]
│  Observations: [observations.md present | none]
│  Squirrels: [count] in [location]
│  Custom: [N skills, N rules, N hooks]
│
│  ! Legacy: [un-numbered domains at root | _brain/ folders | etc.]
│  ! Duplicates: [N person walnuts in multiple locations]
│  ! World size: [size] -- [git OK | tarball recommended]
│  ! Sync scripts: [N scripts reference old paths]
│
│  Upgrade path: [v1->v3 | v2->v3 | already v3 | specific operations needed]
╰─

Phase 2: Visualise Refactor Plan

Generate an interactive HTML visualisation showing what will change. Open it in the browser so the human can review before committing.

The visualisation shows:

• Every file/folder that will be renamed or moved (old path -> new path)
• Files that will be converted (companion.md -> context.manifest.yaml, now.md -> now.json, tasks.md -> tasks.json)
• Files that will be removed (observations.md)
• Files that won't be touched (log.md, key.md, raw/ contents)
• Risk assessment per change (safe rename / content conversion / potential conflict)
• Estimated scope (number of operations, affected walnuts)
• Legacy folders flagged for cleanup
• Duplicate walnuts flagged for merge
• Which upgrade path applies (v1->v2->v3, v2->v3, or verify-only)

Phase 3: Ask for Preferences and Permissions

Before executing, confirm with the human:

Preferences:

1. Squirrel name -- "What should your squirrel be called?" (current name carried over if set)
2. Backup strategy -- if world <1GB: "Create a git branch?" If >1GB: "Create a tarball backup of system files?" (always recommended)
3. Batch size -- "Upgrade all walnuts at once, or walnut-by-walnut?" (recommended: all at once)
4. Legacy folders -- "Clean up [N] legacy folders at world root?" (list them, let human decide)
5. Duplicates -- "Merge [N] duplicate person walnuts?" (show pairs, let human pick primary)

Permissions -- batched into one AskUserQuestion:

1. Approve all (recommended)
2. Review each change type
3. Do a dry run first
4. Cancel

Phase 4: Execute Upgrade

Generate a Python upgrade script and run it. The script MUST be generated fresh (not fetched from a URL) so the human can review it. Write it to .alive/_generated/upgrade.py.

The script must handle these known edge cases:

YAML Frontmatter Parsing

companion.md and now.md files in the wild contain YAML that breaks standard parsers:

• Em-dashes (long dashes) in unquoted description strings
• Wikilinks ([[walnut-name]]) that YAML interprets as nested flow sequences
• Colons in unquoted description strings (e.g., description: Schema spec for dev: block)
• Mixed date formats (2026-03-12 vs 2026-03-12T14:00:00 vs 2026-03-12T14:00:00Z)

The parser must:

1. Try yaml.safe_load() first
2. If that fails, sanitize the frontmatter (quote strings with em-dashes, escape wikilinks) and retry
3. If that still fails, fall back to a regex-based key-value extractor
4. Never crash on malformed YAML -- warn and use what was extracted

companion.md -> context.manifest.yaml Conversion (v1->v2, v1->v3)

Field mapping:

• type: capsule -> removed (inferred from location in bundles/)
• goal: -> preserved, also used to generate description: (first 120 chars)
• status: -> preserved (known values: draft, prototype, published, done, active)
• linked_capsules: -> linked_bundles:
• squirrel: (singular, rare) -> normalize to squirrels: (plural, as array)
• sources: -> preserved, but normalize bare strings to {path: str, description: "", type: "document"}
• All other frontmatter fields -> preserved as-is

Body section handling (body sections vary wildly across worlds):

• ## Context / ## Summary / ## Current State -> context: YAML field in manifest
• ## Tasks -> extract to tasks.md sibling file (if contains [ ], [x], [~] markers)
• ## Work Log -> extract to observations.md sibling file
• ## Changelog, ## Decisions, ## Open Questions, custom sections -> preserved in context: or dropped (these are historical, not operational)
• Files with no body -> fine, just write frontmatter-only manifest
• Files with no frontmatter -> warn, skip

After conversion, rename original to companion.md.bak (never delete source material during upgrade).

now.md -> now.json Conversion (v1->v2, v1->v3)

Field mapping:

• phase: -> preserved (10+ distinct values exist in the wild: active, building, launching, pre-launch, onboarding, waiting, legacy, planning, ready, complete, starting, retainer-pending)
• updated: -> preserved as string (normalize to ISO format if possible)
• capsule: -> rename to bundle: (can be null)
• next: -> preserved
• squirrel: -> preserved (can be a hex hash OR the word "migration")
• health: -> dropped (calculated at read time in v2+)
• links: -> dropped (lives in key.md)
• model: -> dropped (lives in squirrel entry)
• Body (## Context, ## Open, custom headings, or bare prose) -> context: JSON string

Create _kernel/_generated/ directory before writing now.json (for v2 target). For v3 target, write directly to _kernel/now.json. Delete now.md after successful conversion.

Credential scrubbing: Scan body text for patterns matching password, api[_-]?key, secret, token, Bearer. Scrub matching lines from the JSON context field. Warn the human about what was removed.

tasks.md -> tasks.json Conversion (v2->v3)

Parse markdown checkbox format into structured JSON:

Status mapping from checkboxes:

• [ ] -> status: "todo"
• [~] -> status: "active"
• [x] -> status: "done" -> route to completed.json instead of tasks.json

Field extraction:

• @session_id anywhere in the line -> session field (strip the @ prefix)
• Section headers map to priority:
  • ## Urgent or ### Urgent -> priority: "urgent"
  • ## Active or ### Active -> priority: "active"
  • ## To Do or ### To Do -> priority: "normal"
  • ## Blocked or ### Blocked -> priority: "blocked"
• MoSCoW tags in the line -> tags field (array):
  • [M] -> "must"
  • [S] -> "should"
  • [C] -> "could"
  • [W] -> "wont"
• The remaining text (after stripping checkbox, session, MoSCoW tags) -> text field

Output format (tasks.json):

{
  "tasks": [
    {
      "text": "Implement the parser",
      "status": "active",
      "priority": "urgent",
      "session": "abc123",
      "tags": ["must"]
    }
  ]
}

Output format (completed.json):

{
  "completed": [
    {
      "text": "Write the spec",
      "status": "done",
      "priority": "normal",
      "session": "def456",
      "tags": []
    }
  ]
}

Error handling:

• Lines that don't match any checkbox pattern -> flag to stderr for manual review
• Blank lines and section headers -> skip (headers are consumed for priority context)
• Lines with only whitespace after the checkbox -> skip with warning

After successful conversion, rename original to tasks.md.bak.

observations.md Handling (v2->v3)

• If file is empty or fewer than 3 non-blank lines -> remove (rename to observations.md.bak)
• If file has substantial content (3+ non-blank lines) -> migrate content into the bundle's context.manifest.yaml context: field, appending to any existing context
• After migration, rename original to observations.md.bak

bundles/ Flattening (v2->v3)

Each subdirectory under bundles/ moves to the walnut root:

• walnut-root/bundles/my-bundle/ -> walnut-root/my-bundle/
• Check for naming conflicts at walnut root before moving (e.g., a bundle named _kernel would collide)
• If a conflict exists, warn and skip that bundle (flag for manual resolution)
• After all bundles are moved, remove the empty bundles/ directory

_kernel/_generated/ Flattening (v2->v3)

• Move _kernel/_generated/now.json -> _kernel/now.json
• Move any other files in _kernel/_generated/ to _kernel/ (preserving names)
• Remove the empty _kernel/_generated/ directory
• If _kernel/_generated/ contains subdirectories, warn and skip (flag for manual review)

now.json Regeneration (v2->v3)

After structural migration is complete:

• Run project.py --walnut {path} to generate a fresh _kernel/now.json reflecting the new structure
• If project.py is not available, preserve the migrated now.json as-is and warn

_capsules/ Move Semantics (two patterns, v1->v2, v1->v3)

• _capsules/ inside _core/ (now _kernel/): These must be MOVED to walnut root bundles/ (v2) or walnut root directly (v3), not just renamed. The move changes relative path depth.
• _capsules/ at walnut root: Simple rename to bundles/ (v2) or leave in place with rename (v3).

Template File Exclusion

Skip files in templates/ directories -- these are plugin scaffolding templates, not walnut state:

• templates/walnut/now.md -- do not convert
• templates/companion/companion.md -- do not convert

Root-level now.md Duplicates

If both walnut-root/now.md AND _core/now.md (or _kernel/now.md) exist:

• Use _core/ / _kernel/ version as canonical (it's the system file)
• Delete the root-level duplicate
• If root version has unique content, merge it into the context field

Execution Order (matters for safety)

v1->v3 upgrade (full path -- runs v1->v2 steps then v2->v3 steps):

1. Create backup -- tarball of .walnut/ + all _core/ dirs + all companion.md + all now.md + all tasks.md + all observations.md. Write to .alive/_generated/pre-upgrade-backup.tar.gz
2. System folder merge -- .walnut/ into .alive/ (handle overlapping squirrels, skills, scripts)
3. Kernel renames -- _core/ -> _kernel/ (deepest first to avoid parent-before-child issues). Include dirs that only contain _capsules/.
4. Bundle moves -- _capsules/ -> walnut root (check both _kernel/_capsules/ and walnut-root _capsules/)
5. Manifest conversions -- companion.md -> context.manifest.yaml (with YAML sanitization)
6. State conversions -- now.md -> _kernel/now.json (with credential scrubbing, writing directly to flat _kernel/)
7. Task conversions -- tasks.md -> tasks.json + completed.json (with checkbox parsing)
8. Observations migration -- observations.md -> context field or removed
9. Post-upgrade fixups -- find-replace _core/ -> _kernel/, _capsules/ -> bundle names, _kernel/_generated/ -> _kernel/, bundles/ -> flat refs, etc. in all custom skills, scripts, CLAUDE.md files, hooks
10. Regenerate state -- run project.py --walnut {path} for fresh now.json
11. Log the upgrade -- write entry to .alive/log.md

v2->v3 upgrade:

1. Create backup -- tarball of all _kernel/_generated/ dirs + all bundles/ dirs + all tasks.md + all observations.md. Write to .alive/_generated/pre-upgrade-backup.tar.gz
2. Present summary -- show every change that will happen, grouped by walnut. Wait for confirmation.
3. Flatten _kernel/_generated/ -- move _kernel/_generated/now.json -> _kernel/now.json, remove empty _kernel/_generated/
4. Flatten bundles/ -- move each bundles/*/ to walnut root, remove empty bundles/
5. Convert tasks -- tasks.md -> tasks.json + completed.json (per bundle and per kernel)
6. Create completed.json -- _kernel/completed.json with migrated done tasks
7. Migrate observations -- observations.md -> manifest context field or removed
8. Rename inbox domain -- 03_Inputs/ -> 03_Inbox/ at world root (if exists)
9. Post-upgrade fixups -- find-replace _kernel/_generated/ -> _kernel/, bundles/X -> X, tasks.md -> tasks.json, observations.md references removed, 03_Inputs -> 03_Inbox, in all custom skills, scripts, CLAUDE.md files, hooks
10. Clean domain folders -- if 02_Life/_kernel/ exists, flag as legacy cruft (domain folders are NOT walnuts). If People/ exists at world root, flag for move to 02_Life/people/.
11. Regenerate all projections -- run project.py --walnut {path} for EVERY walnut to generate fresh _kernel/now.json. Then run generate-index.py on the world root to rebuild the world index with task counts, recent sessions, and unsigned stash data. This is the final step that makes /alive:world work.
12. Verify structure -- confirm v3 expectations met
13. Log the upgrade -- write entry to .alive/log.md

Safety rules for v2->v3 (and all paths):

• Back up every file before moving or converting
• Don't delete originals until migration is verified
• Present a summary of what will change BEFORE executing
• Ask for confirmation before each major step (flatten kernel, flatten bundles, convert tasks, migrate observations)

Log every operation to .alive/_generated/upgrade-log.yaml with type, source, target, timestamp, status.

Phase 5: Verify Everything Works

Structural verification:

1. .alive/ exists with correct structure (_squirrels/, scripts/, preferences.yaml)
2. No .walnut/ remains
3. No _core/ kernel directories remaining (find + verify)
4. No _capsules/ directories remaining
5. No unconverted companion.md inside any walnut (.bak files are OK)
6. No now.md remaining in _kernel/ directories
7. No _kernel/_generated/ directories remaining (v3 flattened)
8. No bundles/ directories remaining at walnut root (v3 flattened)
9. No tasks.md remaining (converted to tasks.json, .bak files OK)
10. No observations.md remaining (migrated or removed, .bak files OK)
11. All _kernel/ dirs contain now.json directly (not nested in _generated/)
12. All squirrel entries intact and readable
13. Custom skills reference correct paths (grep for _core/, _kernel/_generated/, bundles/ -- should find nothing)
14. Statusline renders without errors

Task conversion verification (v2->v3): 15. Every tasks.json is valid JSON and parseable 16. Every completed.json is valid JSON and parseable 17. Task count in tasks.json + completed.json matches original tasks.md checkbox count (minus flagged unparseable lines)

Root-level hygiene: 18. No unexpected folders at world root -- only 01_Archive/, 02_Life/, 03_Inbox/, 04_Ventures/, 05_Experiments/, .alive/, .claude/, and standard dotfiles should exist 19. No People/ at world root (should be 02_Life/people/) 20. No _kernel/ inside domain folders (02_Life/_kernel/ is legacy cruft) 21. 03_Inbox/ exists (not 03_Inputs/) 22. Flag any legacy folders that remain (un-numbered domains, _working, src, etc.)

Projection verification: 23. Every walnut has _kernel/now.json (generated by project.py) 24. .alive/_index.json exists and is current (generated by generate-index.py) 25. World index includes recent_sessions and unsigned_with_stash data

Sync script audit: 20. Check configured sync scripts for old path references -- flag any that will recreate legacy folders 21. Surface specific line numbers and suggested fixes

Report:

╭─ 🐿️ upgrade complete
│
│  Version: ALIVE Context System v3.0
│  System folder: .alive/
│  Walnuts upgraded: N/N
│  Bundles flattened: N/N
│  Tasks converted: N/N (N routed to completed.json)
│  Observations migrated: N/N
│  Custom capabilities updated: N/N
│  Verification: N/N checks passed
│
│  ! [any remaining issues]
│
│  Backup: .alive/_generated/pre-upgrade-backup.tar.gz
│  Log: .alive/_generated/upgrade-log.yaml
╰─

---

Upgrade Paths Supported

From	To	Operations
Un-numbered domains + _brain/ + _state/	Current v3	Full restructure -- route to numbered domains, convert state, flatten kernel + bundles, convert tasks
.alive/ + _core/ + _capsules/ + companion.md	v3	Kernel rename, bundle move + flatten, manifest convert, state convert, task convert
.walnut/ + _core/ + _capsules/ + companion.md	v3	System folder merge + all above
.walnut/ + _kernel/ + bundles/	v3	System folder merge + flatten kernel + flatten bundles + convert tasks + migrate observations
.alive/ + _kernel/_generated/ + bundles/ + tasks.md	v3	Flatten _kernel/_generated/ to _kernel/, flatten bundles/ to root, convert tasks.md to tasks.json, migrate observations.md
.alive/ + _kernel/ + bundles/ + context.manifest.yaml	v3	Flatten bundles + convert tasks + migrate observations (partial v2->v3)
.alive/ + _kernel/ (flat) + flat bundles + tasks.json	(current v3)	Already up to date -- verify only
Both .walnut/ AND .alive/ exist	Merge .walnut/ into .alive/ then v3	Handle overlapping squirrels, skills, scripts -- keep newer versions
No system folder	Fresh install	Redirect to alive:world for initial setup

The upgrade detects what's present and only performs the operations needed. It never forces a full rebuild when a partial upgrade suffices.

---

Rollback

If something goes wrong mid-upgrade:

1. Check upgrade-log.yaml for the last successful operation
2. Offer restore from tarball backup
3. Or offer manual fix for the specific failed operation

The tarball contains every file that was about to be renamed or converted -- full restore is always possible.

╭─ 🐿️ upgrade issue
│
│  Failed at: converting [path]/tasks.md
│  Error: [specific error]
│
│  ▸ Options:
│  1. Skip this one, continue (fix manually later)
│  2. Show me the file so I can fix it
│  3. Restore from backup
╰─

---

What This Skill Does NOT Touch

• Walnut content -- key.md, log.md, raw files are never modified (only moved within renames)
• Git history -- no force pushes, no history rewrites
• Plugin cache -- ~/.claude/plugins/ is managed by Claude Code, not this skill

What This Skill DOES Audit (but doesn't auto-fix)

• Sync scripts -- flags old paths, shows the fix, lets the human decide
• External integrations -- MCP servers, email, Slack sync scripts are surfaced if they reference old structure
• Legacy folders -- surfaced for human cleanup, never auto-deleted

---

What System Upgrade Is NOT

• Not alive:build-extensions -- extensions create new capabilities. Upgrade migrates existing structure.
• Not alive:system-cleanup -- cleanup fixes broken things in the current version. Upgrade moves between versions.
• Not a fresh install -- if no existing system is found, redirect to alive:world for initial setup.

Cleanup fixes. Upgrade transforms.