model-migrate-flagos

FL Plugin — Model Migration Skill

Usage

/model-migrate-flagos <model_name> [upstream_folder] [plugin_folder]

Argument	Required	Default
model_name	Yes	—
upstream_folder	No	/tmp/vllm-upstream-ref
plugin_folder	No	current working directory

Execution

Step 1: Parse arguments and validate paths

Extract from user input:

• {{model_name}} = first argument (required, snake_case)
• {{upstream_folder}} = second argument or /tmp/vllm-upstream-ref
• {{plugin_folder}} = third argument or current working directory

If {{upstream_folder}} doesn't exist, ask user whether to clone it. If {{plugin_folder}} doesn't exist, error out.

→ Tell user: Confirm parsed model name and paths.

Step 2: Load references and resolve placeholders

Read these files (relative to this SKILL.md):

• references/procedure.md — step-by-step migration procedure
• references/compatibility-patches.md — 0.13.0 patch catalog
• references/operational-rules.md — communication, TaskList, bash rules, resilience

The procedure references executable scripts in scripts/:

• scripts/validate_migration.py — automated code review (Step 6)
• scripts/benchmark.sh — benchmark verification (Step 9)
• scripts/serve.sh — serve model locally (Step 10.1, also used for E2E)
• scripts/request.sh — test request (Step 10.2)
• scripts/e2e_eval.py — E2E correctness verification (Step 11)
• scripts/e2e_test_prompts.json — test prompts for E2E (5 text + 5 multimodal)
• scripts/e2e_config.template.json — E2E config template (copy to e2e_config.json and fill in)
• scripts/e2e_remote_serve.sh — manage GT server on remote machine via SSH

Then investigate upstream source + HuggingFace to resolve all placeholders:

Placeholder	How to derive
{{model_name}}	Direct from argument
{{model_name_lower}}	Lowercase of model_name (usually identical, e.g. qwen3_5) — used in file paths
{{MODEL_DISPLAY_NAME}}	From upstream code or HF model card
{{ModelClassName}}	From upstream model class (PascalCase)
{{model_type}}	From HF config.json model_type field
{{ConfigClassName}}	From upstream or derive from model_type
{{skill_root}}	Absolute path to this skill's folder (the directory containing this SKILL.md)

Naming conventions vary per model — always verify from actual source, never guess.

→ Tell user: Present all resolved values. Use AskUserQuestion if anything is ambiguous.

Step 3: Execute procedure

With placeholders resolved, execute every step in procedure.md sequentially. Apply patches from compatibility-patches.md during the copy-then-patch step. Follow operational-rules.md throughout.

→ Tell user: Before starting, output a numbered plan. Report progress at each step boundary.

Scripts Reference

Script	Step	Description
validate_migration.py	6	Automated import/API/registration checks
benchmark.sh	9	vllm bench throughput with dummy weights
serve.sh	10, 11	Start local vLLM server (port 8122, VLLM_FL_PREFER_ENABLED=false)
request.sh	10	Quick smoke-test request
e2e_eval.py	11	Token-level comparison vs upstream GT server
e2e_test_prompts.json	11	5 text + 5 multimodal test prompts
e2e_config.template.json	11	Config template (GT machine, local port, eval params)
e2e_remote_serve.sh	11	SSH-based GT server lifecycle (start/stop/status/logs)

Examples

Example 1: Typical new model

User says: "/model-migrate-flagos kimi_k25"
Actions:
  1. Parse → model_name=kimi_k25, defaults for upstream/plugin paths
  2. Clone upstream, find vllm/model_executor/models/kimi_k25.py
  3. Discover it wraps DeepseekV2 → follow kimi_k25 (wrapper) pattern
  4. Copy file, apply P1+P2 patches, create config bridge
  5. Register, validate, test, benchmark, serve+request
  6. E2E verification against upstream GT
Result: kimi_k25 fully working in plugin, all 11 steps passed

Example 2: Re-run after upstream update

User says: "migrate qwen3_5 again, upstream updated"
Actions:
  1. Idempotent re-run — overwrite existing files with fresh upstream copy
  2. Re-apply patches, re-validate, re-test
  3. Re-run E2E to confirm no regression
Result: qwen3_5 updated to match latest upstream, no regressions

Troubleshooting

General principle: When any runtime error occurs, first compare vLLM upstream code against both the plugin adaptation and the installed 0.13.0 environment. The diff is the fastest path to root cause. See operational-rules.md § Debugging Priority: Upstream-First for the full protocol.

Problem	Typical Cause	Fix
ImportError after copy-then-patch	Missing P1 fix (relative→absolute imports)	Verify all from .xxx converted to from vllm.* or from vllm_fl.*
AttributeError: module 'vllm' has no attribute X	API doesn't exist in 0.13.0	Check P3 in compatibility-patches.md; stub or remove
Config not recognized by vLLM	model_type mismatch or config bridge missing	Verify _CONFIG_REGISTRY[model_type] matches HF config.json exactly
Registration has no effect	Class name or import path typo	Compare with existing registrations in __init__.py
Benchmark KeyError on config field	Config bridge missing a field	Compare upstream config class vs bridge; add missing fields with defaults
Benchmark/Serve fails with OOM or "insufficient memory"	GPUs occupied by other processes	Kill GPU processes: nvidia-smi --query-compute-apps=pid --format=csv,noheader | xargs -r kill -9 then retry. Never skip these steps.
Model outputs garbled/gibberish text	ColumnParallelLinear used for merged projections with different sub-dimensions (TP sharding mismatch)	Override __init__ to use MergedColumnParallelLinear(output_sizes=[...]). See P8 in compatibility-patches.md
AssertionError: Duplicate op name	Child class imports custom op from different module path than parent	Use same import path as parent module (e.g. vllm_fl.ops.fla not vllm_fl.models.fla_ops). See P11
AttributeError on fused_recurrent_* during CUDA graph warmup	__init__ override with nn.Module.__init__(self) missed attributes used by inherited _forward_core	Create ALL attributes from parent's __init__, especially custom ops. See P12
E2E: local server not reachable	serve.sh port doesn't match e2e_config.json local port	Ensure both use same port (default 8122)
E2E: GT server not reachable	GT machine down or docker/conda env wrong	Check e2e_remote_serve.sh status or SSH manually
E2E: early token divergence (first 5 tokens)	Weight loading bug, TP sharding error	Check load_weights, stacked_params_mapping, MergedColumnParallelLinear
E2E: late minor divergence (token #15+)	Numerical noise from different op implementations	Usually acceptable; document in report
resolve_op fails with VLLM_FL_PREFER_ENABLED=false	Op not registered in dispatch, no fallback	Add try/except fallback to flag_gems in op import code