MotionEyes
MotionEyes is an agent-first SwiftUI motion observability system.
Overview
MotionEyes combines:
package/: runtime tracing primitives that emit motion data over time (.motionTrace,Trace.value,Trace.geometry,Trace.scrollGeometry)skill/motioneyes-animation-debug: installs or integrates the package, adds focused traces, captures logs, and validates behavior against intentskill/motioneyes-visual-analysis: analyzes frame sequences with computer vision and produces annotated images plus JSON summaries
The goal is to let agents read real motion, geometry, and scroll values from logs instead of guessing from code.
Skill demo:
Visual Analysis Outputs
motioneyes-visual-analysis can generate keyframe sprites, grid overlays, and pixel diffs for motion analysis. The recommended interpretation flow is:
- Start with a clean keyframe sprite to understand overall motion progression.
- Use the matching grid sequence when you need coordinate-aware position checks.
- Use adjacent-frame diffs for precise pixel-change confirmation.
Example artifacts below were generated from a focused demo sequence:
Keyframe sprite (clean motion progression, 5 selected frames)
Grid sequence (same 5 frames with coordinate labels)
Adjacent diff sprite (4 selected-frame deltas)
GridGPT Dependency
Grid overlays require the GridGPT submodule at third_party/GridGPT.
git submodule update --init --recursive
test -f third_party/GridGPT/arial.ttf && echo "GridGPT font OK" || echo "GridGPT font missing"
pip3 install -r skill/motioneyes-visual-analysis/requirements.txt
Generate analysis artifacts like the examples above:
python3 skill/motioneyes-visual-analysis/scripts/analyze_sequence.py \
--video /path/to/capture.mp4 \
--fps 15 \
--duration 1.0 \
--trim \
--diff-grid \
--output /path/to/report
By default, diff outputs are rendered for keyframe-related pairs only. Add --all-pairs when you need diff/ and diff_grid/ for every consecutive frame pair.
How It Works
End-to-End Flow
- Define expected behavior in measurable terms.
- Locate the views and state values that drive the behavior.
- Ensure MotionEyes is available or integrate it if missing.
- Add temporary, focused traces to values, geometry, or scroll state.
- Reproduce the issue and capture MotionEyes logs.
- Compare observed traces against expected motion relationships and timing.
- Apply fixes, re-run validation, then remove only the temporary instrumentation added for that run.
Geometry Modes
Trace.geometry separates coordinate target from frame source:
- Layout-relative debugging:
space: .swiftUI(.global), source: .layout - Window-relative debugging:
space: .window, source: .layout - True on-screen movement:
space: .screen, source: .presentation
Tooling and Runtime Requirements
MotionEyes works best alongside XcodeBuildMCP for simulator control and log capture, but it is not strictly required.
If MCP is unavailable, agents can still capture and read simulator logs directly:
xcrun simctl spawn booted log stream \
--style compact \
--level debug \
--predicate 'subsystem == "MotionEyes"'
Limitations
.transitionvisual behavior is not directly observable when no measurable underlying value is exposed.
Skills
See skill/SKILLS.md for guidance on when to use each skill.
Repository Layout
package/: MotionEyes Swift packageskill/motioneyes-animation-debug/: MotionEyes agent skillskill/motioneyes-visual-analysis/: Frame-based visual analysis skillskill/motioneyes-animation-debug/SKILL.md: Skill workflow definition
Package Quick Start
From repo root:
swift build --package-path package
swift test --package-path package