media-os

Designs end-to-end media pipelines before a single command runs. Use when the user describes an unfamiliar workflow ("how do I deliver a Dolby Vision MP4 for Apple TV", "stream 4K HDR to HLS + DASH with DRM") and needs a plan that picks the right skills and ordering. Produces a staged recipe with exact tools, not just prose.

Packages finished renders for streaming distribution and VOD — HLS/DASH packaging with Shaka/GPAC, MP4 fragmentation, DRM (Widevine/PlayReady/FairPlay) with cbcs, CDN upload (Cloudflare Stream, Mux, Bunny), and platform-spec delivery (YouTube, Vimeo, broadcast IMF/MXF). Use when the encode is done and the file needs to get to viewers.

Executes video/audio encodes with the right codec, container, and flags for the target. Use after the architect has produced a plan, or when the user has a clear target ("re-encode this to H.265 10-bit under 5 Mbps", "transcode to ProRes 422 HQ for Premiere"). Owns rate-control math, preset selection, pixel format, and container flags.

Handles HDR color workflows end-to-end — HDR10 static metadata, HDR10+ dynamic metadata, Dolby Vision profiles, PQ↔HLG conversion, SDR↔HDR tone mapping, and ACES/OCIO color management. Use when the user mentions Dolby Vision, HDR10, HDR10+, HLG, PQ, tone mapping, or anything about color pipelines that must survive delivery.

Runs live production — OBS control over obs-websocket, multi-bitrate streaming (RTMP/SRT/RIST/WHIP), NDI routing, DeckLink capture, PTZ camera control, and low-latency contribution feeds. Use when the user is LIVE — streaming now, about to go live, mid-show, or wiring up the broadcast rig.

Manipulate macOS audio at the Core Audio HAL layer: afinfo (audio file metadata), afplay (play), afconvert (convert format/rate/bit-depth/channel layout), say (TTS), SwitchAudioSource (change default input/output/system device — brew install switchaudio-osx), audiodevice CLIs (npm/Go variants), AppleScript via osascript to drive System Preferences Sound. HAL plugins at /Library/Audio/Plug-Ins/HAL/ load virtual drivers (BlackHole 2ch/16ch/64ch, Loopback, Background Music). Aggregate Device (drift-corrected multi-in) and Multi-Output Device via Audio MIDI Setup.app or AudioHardwareCreateAggregateDevice. Use when the user asks to change default macOS audio device, route audio between apps on Mac, install BlackHole, build an Aggregate Device, convert audio files with afconvert, or script Mac audio from the terminal.

Route professional low-latency audio with JACK Audio Connection Kit (jackaudio.org): jackd server (ALSA/CoreAudio/PortAudio/ASIO backends), jack_control (D-Bus JACK2 config), jack_lsp (list ports/connections), jack_connect/jack_disconnect (manage links), jack_cpu_load, jack_samplerate, jack_bufsize (runtime period size), jack_transport, jack_rec (multi-channel WAV recording), jack_iodelay (round-trip latency), jack_wait, jack_midi_dump. JACK1 C maintenance vs JACK2 C++ multicore. Modern Linux swaps jackd for PipeWire's libjack shim — same client API. JackTrip for uncompressed multichannel UDP over the Internet. Use when the user asks to run a JACK server, connect audio clients, measure roundtrip latency, record from JACK ports, set JACK period size, or bridge JACK over the network with JackTrip.

Route and introspect Linux audio/video/MIDI graphs with PipeWire: pw-cli (interactive object/module control), pw-dump (full graph JSON with --monitor), pw-link (list/create/destroy port links), pw-cat / pw-play / pw-record (playback+record), pw-top (real-time DSP load + xrun viewer), pw-metadata (shared metadata store), pw-loopback (source/sink bridges), pw-midiplay/pw-midirecord, pw-jack (JACK client compat shim), pw-mon, pw-profiler. WirePlumber session manager handles routing/defaults/profiles. Replaces PulseAudio (via pipewire-pulse) and JACK (via libjack.so shim) simultaneously. Use when the user asks to route Linux audio, link two apps, list audio devices on Linux, create a loopback, record system audio, monitor the audio graph, or set up a virtual audio device on modern Linux.

Manipulate Windows audio via WASAPI (learn.microsoft.com/coreaudio): SoundVolumeView + svcl.exe (NirSoft — set default device, per-app volumes, save/load profiles), AudioDeviceCmdlets PowerShell module (Get-AudioDevice, Set-AudioDevice, Get-AudioDevicePlaybackMute), nircmd setdefaultsounddevice. Shared mode (Windows engine resamples all streams to endpoint fixed format) vs Exclusive mode (bit-perfect, audiophile, ~sub-3ms latency with IAudioClient3 + event-driven callbacks). WASAPI then MMDevice then Audio Engine (audiodg.exe) then WDM/KS. ASIO is Steinberg's parallel stack bypassing Windows audio. Virtual audio devices: VB-Audio Virtual Cable, VoiceMeeter/Banana/Potato. Use when the user asks to change Windows default audio device, route app audio, use VoiceMeeter, install VB-Cable, set a specific device from PowerShell, or enable exclusive-mode low-latency output.

Google MediaPipe Tasks API (current supported, NOT legacy mediapipe.solutions): Vision Tasks (Face Detector, Face Landmarker, Face Stylizer, Hand Landmarker, Gesture Recognizer, Pose Landmarker, Holistic Landmarker, Object Detector, Image Classifier, Image Segmenter, Interactive Segmenter, Image Embedder, Image Generator), Audio Tasks (Audio Classifier), Text Tasks (Text Classifier, Text Embedder, Language Detector), GenAI Tasks (LLM Inference — on-device Gemma/Phi-2/Falcon/StableLM). Three run modes: IMAGE, VIDEO, LIVE_STREAM (async). Python: mediapipe.tasks.python.vision/audio/text/genai. Docs at ai.google.dev/edge/mediapipe/solutions. Model Maker for transfer learning. Use when the user asks to detect pose/face/hand landmarks, classify an image, run a body-pose model, segment people, recognize gestures, do on-device LLM inference via MediaPipe, or build a mobile/embedded CV pipeline.

5 hooks across 5 events

Probe SHAKA_KEY_SERVER_URL; warn on auth failures or extended downtime.

Poll nvidia-smi for temperature + sustained-load warnings so Claude knows before the GPU throttles.

Poll INCOMING_MEDIA_DIR for new stable media files; notify Claude with a probe suggestion.

Poll RENDER_QUEUE_URL (HTTP JSON) or RENDER_QUEUE_DIR (status sidecars); notify on job state transitions.

ffprobe LIVE_STREAM_URL every STREAM_POLL_SEC; notify on down/up/bitrate-drop transitions.