moviebot

MovieBot — MARS Cinematography Skill Development

Use this skill when writing, modifying, or debugging MARS robot cinematography skills. It provides the complete API reference, code patterns, and constraints for the arm-as-camera architecture.

When to Use

• Writing new camera movement skills for the MARS robot
• Modifying existing cinematography skills
• Debugging arm positioning or camera framing issues
• Planning shot sequences for the Cinematographer agent
• Adding new director vocabulary to the agent prompt

Architecture: Arm as Camera

The MARS robot has a 2MP RGB camera (1080p, 160° FOV, 30 FPS) mounted on the gripper of a 6-joint arm (~40cm reach). The arm IS the camera crane/gimbal/jib. Every camera move is an arm move.

The camera streams via ROS2 /mars/arm/image_raw — it does NOT appear in RobotState.

API Reference

ManipulationInterface (arm = camera control)

from brain_client.skill_types import Skill, Interface, InterfaceType, SkillResult

class MySkill(Skill):
    manipulation = Interface(InterfaceType.MANIPULATION)
    
    def execute(self, ...):
        # Move camera to Cartesian pose (blocking)
        self.manipulation.move_to_cartesian_pose(x, y, z, roll, pitch, yaw, duration)
        
        # Read current camera position
        self.manipulation.get_current_end_effector_pose()
        # Returns: {'position': [x, y, z], 'orientation': [qx, qy, qz, qw]}
        
        # Read current orientation as Euler angles
        self.manipulation.get_current_orientation_rpy()
        # Returns: {'roll': r, 'pitch': p, 'yaw': y}
        
        # Move to joint configuration (6 values)
        self.manipulation.goto_joint_state(joints)
        
        # Validate a pose is reachable before committing
        self.manipulation.solve_ik(x, y, z, roll, pitch, yaw, timeout)
        # Returns: list[float] (6 joints) or None if unreachable

MobilityInterface (base = dolly track)

class MySkill(Skill):
    mobility = Interface(InterfaceType.MOBILITY)
    
    def execute(self, ...):
        self.mobility.rotate(angle_radians)                        # blocking
        self.mobility.send_cmd_vel(linear_x, angular_z, duration)  # non-blocking
        self.mobility.rotate_in_place(angular_speed, duration)     # non-blocking

Shared Pose Library

from arm_camera_poses import (
    CAMERA_FORWARD_NEUTRAL,   # (0.30, 0, 0.25, 0, 0, 0) — straight ahead
    CAMERA_FORWARD_LOW,       # (0.30, 0, 0.10, 0, 0.3, 0) — heroic low angle
    CAMERA_FORWARD_HIGH,      # (0.20, 0, 0.34, 0, -0.4, 0) — god shot
    CAMERA_STOWED,            # (0.10, 0, 0.15, 0, 0, 0) — safe transport
    CAMERA_BOOM_OUT,          # (0.28, 0, 0.27, 0, -0.2, 0) — max reach
    camera_pose_from_tilt,    # degrees (-25 to +15) → 6-tuple pose
    interpolate_camera_pose,  # lerp between two poses, t=0..1
)

All poses are 6-tuples: (x, y, z, roll, pitch, yaw) in meters/radians.

Reference Joint Configs

HOME_POSE_JOINTS = [0, -0.5, 1.5, -1.0, 0, 0]   # Safe stowed
READY_POSE_JOINTS = [0, -0.3, 1.0, -0.8, 0, 0]   # Extended ready

Skill Pattern Template

Every cinematography skill follows this pattern:

"""
Skill Name — Brief description of the camera move.

Canon: film references that use this technique.
"""

import time
from brain_client.skill_types import Skill, Interface, InterfaceType, SkillResult
from arm_camera_poses import camera_pose_from_tilt, interpolate_camera_pose

class MyShot(Skill):
    """One-line description."""

    manipulation = Interface(InterfaceType.MANIPULATION)
    mobility = Interface(InterfaceType.MOBILITY)  # only if base moves

    @property
    def name(self) -> str:
        return "my_shot"  # snake_case, matches agent's get_skills() list

    def guidelines(self) -> str:
        return (
            "Use when the director says '...'. "
            "Describe physical requirements and best conditions."
        )

    def execute(self, camera_tilt_degrees: float = 0.0, ...) -> tuple:
        """Docstring with Args."""
        # 1. Position camera
        pose = camera_pose_from_tilt(camera_tilt_degrees)
        self.manipulation.move_to_cartesian_pose(*pose, duration=1.0)
        time.sleep(1.0)

        # 2. Execute the move
        self._send_feedback("Starting shot...")
        # ... mobility commands, arm transitions, etc.

        # 3. Check cancellation in loops
        for i in range(steps):
            if self._cancelled:
                return ("Cancelled.", SkillResult.CANCELLED)
            # ... do work ...

        return ("Shot complete.", SkillResult.SUCCESS)

    def cancel(self):
        self._cancelled = True
        self.mobility.send_cmd_vel(linear_x=0.0, angular_z=0.0, duration=0.1)

Smooth Camera Transitions

For skills that need smooth arm movement over time (crane, reveal, drift):

start_pose = camera_pose_from_tilt(start_tilt)
end_pose = camera_pose_from_tilt(end_tilt)

steps = int(duration_seconds * 10)  # 10 Hz
step_time = duration_seconds / steps

for i in range(steps):
    if self._cancelled:
        return ("Cancelled.", SkillResult.CANCELLED)
    t = (i + 1) / steps
    pose = interpolate_camera_pose(start_pose, end_pose, t)
    self.manipulation.move_to_cartesian_pose(*pose, duration=step_time)
    time.sleep(step_time)

IK Validation (for extreme poses)

target_pose = (x, y, z, roll, pitch, yaw)
joints = self.manipulation.solve_ik(*target_pose, timeout=1.0)
if joints is None:
    return ("Target pose unreachable within arm workspace.", SkillResult.FAILURE)
self.manipulation.move_to_cartesian_pose(*target_pose, duration=duration)

Anti-Patterns (NEVER DO THESE)

• InterfaceType.HEAD — controls the nav camera, NOT the cinematography camera
• InterfaceType.ARM — does not exist
• self.head.set_position() — wrong camera entirely
• RobotStateType.LAST_MAIN_CAMERA_IMAGE_B64 — that's the nav camera
• 7 joint values — arm has exactly 6 joints
• head_tilt_degrees parameter name — use camera_tilt_degrees

Physical Constraints

• Arm reach: ~40cm (all poses must stay within this envelope)
• Arm joints: 6 (not 7)
• Camera FOV: 160° (very wide — framing is forgiving)
• Speed caps: 0.3 m/s forward, 0.2 m/s for Kubrick, 0.15 m/s for combined moves
• Base: differential drive, no true lateral strafe (use rotate-drive-rotate)
• No rear obstacle sensors (warn director before dolly_out)

Cinematographer Agent Integration

New skills must be registered in mars/agents/cinematographer.py:

1. Add the skill's name property value to get_skills() list
2. Add director vocabulary mapping in get_prompt()
3. Add composition defaults if the skill has unique framing needs

Existing Skills (15 total)

Skill Name	File	Type
orbit_subject	orbit_subject.py	Core
orbit_smooth_arc	orbit_subject.py	Core
dolly_in	dolly_in.py	Core
dolly_out	dolly_out.py	Core
static_frame	static_frame.py	Core
lateral_track	lateral_track.py	Core
kubrick_symmetrical	kubrick_symmetrical.py	Core
crane_up	crane_arm.py	Crane
crane_down	crane_arm.py	Crane
jib_swoop	crane_arm.py	Crane
whip_pan	whip_pan.py	Advanced
reveal_tilt_up	reveal_tilt_up.py	Advanced
dolly_zoom	dolly_zoom.py	Advanced
oner_walk_and_talk	oner_walk_and_talk.py	Advanced
handheld_simulated	handheld_simulated.py	Effects

Post-Processing

Skills that produce real motion (orbit, dolly, lateral, oner) should recommend video-to-video post. Static or vertical skills can use image-to-video. The dolly_zoom skill emits a JSON manifest with zoom ratios for the counter-zoom effect.

Recommended engines: Runway Gen-3, Google Veo, Seedance.