From membrane-framework
Provides guidance on building and debugging Membrane pipelines, creating custom elements, and understanding the architecture of the Elixir multimedia framework.
npx claudepluginhub membraneframework/membrane_coreThis skill uses the workspace's default tool permissions.
**Package**: `membrane_core` ~> 1.2 | **Docs**: https://hexdocs.pm/membrane_core/ | **Module index**: https://hexdocs.pm/membrane_core/llms.txt | **Demos**: https://github.com/membraneframework/membrane_demo | **All packages**: [packages_list.md](../../guides/llms/packages_list.md)
Provides FFmpeg 7.1/8.0 commands for RTMP streaming to Twitch/YouTube/Facebook, HLS/DASH ABR ladders, LL-HLS/LL-DASH, SRT, WebRTC/WHIP, protocol conversion, multi-destination, nginx-rtmp, Docker integration.
Guides Elixir developers in implementing GenServers, Supervisors, and OTP behaviors for fault-tolerant concurrent systems and supervision trees.
Implements OTP actor patterns in Gleam: processes, message passing, GenServer, supervisors, fault tolerance, state management for concurrent apps on Erlang VM.
Share bugs, ideas, or general feedback.
Package: membrane_core ~> 1.2 | Docs: https://hexdocs.pm/membrane_core/ | Module index: https://hexdocs.pm/membrane_core/llms.txt | Demos: https://github.com/membraneframework/membrane_demo | All packages: packages_list.md
handle_buffer/4 for filters/sinks, handle_demand/5 for manual-flow sources)mix membrane.gen.filter MyApp.MyFilter, mix membrane.gen.source, mix membrane.gen.sink, mix membrane.gen.endpoint, mix membrane.gen.bin instead of writing component skeletons by handFilter for transformations (has sensible defaults for stream_format forwarding); use Endpoint only when output is unrelated to input (e.g. a UDP Endpoint); use Source/Sink for pure producers/consumers:auto on all pads; only use :manual when you need fine-grained backpressure control; Almost the only use case of :push are output pads of Sources/Endpoints that cannot control when they produce data, e.g. UDP Source/Endpoint.child/2, get_child/1, via_in/2, via_out/2) (more info: Membrane.ChildrenSpec)spec: from handle_init/2 for static pipelines; return additional spec: actions from any callback (e.g. handle_child_notification/4) to grow the topology at runtime:source) for singletons, tuples ({:decoder, track_id}) for multi-instance children of the same typehandle_element_end_of_stream/4 in the pipeline to know when a sink's input pad received EOS; then return {[terminate: :normal], state} (doesn't work if sink is a Membrane.Bin - then expect a custom message from the bin in handle_child_notification callback instead, if the bin sends it){spec, group: <name>, crash_group_mode: :temporary}; handle recovery in handle_crash_group_down/3; see Crash Groups guidechild(:probe, %Membrane.Debug.Filter{handle_buffer: &IO.inspect(&1, label: :buffer)}) between any two elements to log buffers without changing pipeline logic. You can use different logging functions than IO.inspect/2. More info: Membrane.Debug.Filter.accepted_format compatibilityctx; key fields: ctx.children, ctx.pads, ctx.playback; crash callbacks also have ctx.crash_initiator, ctx.exit_reason, ctx.group_name; see Pipeline.CallbackContext, Bin.CallbackContext, Element.CallbackContextMembrane.Logger instead of Logger in Membrane components; it prepends component path and name to log messages. Requires require Membrane.Logger in the module before calling any logging functions.deps/mix hex.info <plugin name> when you need to check the newest version of a pluginREADME.md, in all-packages sectiondeps/ (use cat <filename> | grep def_input_pad and cat <filename> | grep def_output pad) to make sure output pad's accepted_stream_format is compatible with accepted_stream_format of the input pad which it is linked to.accepted_stream_format doesn't match, search for an element which can act as an adapterdeps/ directory - never do thatchild/2 for an already-spawned child — child/2 always spawns a new process; use get_child/1 to reference an existing one; duplicating a name raises an errorspec that spawns the component; linking them later raises a LinkErrorhandle_pad_added/3 — a dynamic bin input pad must be connected to an internal child within 5 seconds or a LinkError is raisedhandle_init/2 — handle_init is synchronous and blocks the parent; move file I/O, network connections, etc. to handle_setup/2handle_playing/2 — pads are not ready until :playing; don't send buffers from handle_setup/2:push flow control carelessly — whenever it is possible use :auto instead (eventually :manual). Source/Endpoint output pads are the exception.handle_init/2 — we recommend to return only :spec action from this callback.Pipeline
├── Element (Source/Filter/Sink/Endpoint) ← leaf, processes data
├── Bin ← dual role: parent (has children) + child (has pads)
│ ├── Element
│ └── Bin ← bins nest arbitrarily deep
└── ...
| Type | Parent | Child | Has Pads |
|---|---|---|---|
| Pipeline | yes | no | no |
| Bin | yes | yes | yes |
| Element | no | yes | yes |
Element subtypes: Source (output only) · Filter (in + out, output is transformed input) · Sink (input only) · Endpoint (in + out, but output might be not related to input)
Defined on Elements and Bins (not Pipelines) using def_input_pad/2 (Membrane.Element.WithInputPads) and def_output_pad/2 (Membrane.Element.WithOutputPads):
def_input_pad :input, accepted_format: _any
def_output_pad :output, accepted_format: Membrane.RawAudio, flow_control: :auto
:always (static, one instance, referenced by atom) or :on_request (dynamic, reference via Pad.ref(:name, id)):auto (framework manages demand — preferred), :manual (explicit via :demand/:redemand), :push (no demand, risk of overflow)accepted_format:input/:output allow omitting via_in/via_out in specsaccepted_format matching syntax: _any (accept anything) · Membrane.RawAudio (any struct of that type) · %Membrane.RawAudio{channels: 2} (match specific fields) · %Membrane.RemoteStream{} (unknown/unparsed stream). any_of(patter1, pattern2, ...) matches if any pattern matches.handle_init/2 sync, blocks parent — parse opts, return initial spec
handle_setup/2 async — heavy init (open files, connect services)
return {[setup: :incomplete], state} to delay :playing
handle_pad_added/3 fires for dynamic pads linked in the same spec
handle_playing/2 component is ready — start producing/consuming data
All components spawned in the same :spec action enter :playing together (they synchronize to the slowest setup). Elements and Bins wait for their parent before handle_playing/2.
Stream format and EOS rules (critical for filter authors):
{:stream_format, {pad, format}} before the first buffer on each output pad, or downstream elements crashhandle_stream_format/4 in filters forwards the format downstream — if you override it, you must forward manually or return the {:stream_format, ...} action yourselfhandle_end_of_stream/3 in filters forwards EOS downstream — overriding without forwarding will stall the pipelineFull lifecycle guide: Lifecycle of Membrane Components
# Linear chain — child/2 spawns a new named child
child(:source, %Membrane.File.Source{location: "input.mp4"})
|> child(:filter, MyFilter)
|> child(:sink, %Membrane.File.Sink{location: "out.raw"})
# Explicit pad names (required for non-default names or dynamic pads)
get_child(:demuxer)
|> via_out(Pad.ref(:output, track_id))
|> via_in(:video_input)
|> child(:decoder, Membrane.H264.FFmpeg.Decoder)
# Link to an already-existing child
get_child(:existing_filter) |> child(:new_sink, MySink)
# Inside a Bin — bin_input/bin_output connect the bin's own pads to internal children
bin_input(:input) |> child(:filter, MyFilter) |> bin_output(:output)
# Crash group — all children in the spec share the group; a crash in any terminates all
{child(:source, Source) |> child(:sink, Sink), group: :my_group, crash_group_mode: :temporary}
Bin pad wiring rules:
bin_input(pad_ref) / bin_output(pad_ref) are the interior side of the bin's own padshandle_pad_added/3 within 5 seconds or a LinkError is raisedThe standard approach for variable-track streams (e.g. MP4 demuxers):
# 1. Spawn source + demuxer; demuxer hasn't identified tracks yet
def handle_init(_ctx, state) do
{[spec: child(:source, Source) |> child(:demuxer, Demuxer)], state}
end
# 2. Demuxer notifies parent once tracks are known
def handle_child_notification({:new_tracks, tracks}, :demuxer, _ctx, state) do
spec = Enum.map(tracks, fn {id, _fmt} ->
get_child(:demuxer)
|> via_out(Pad.ref(:output, id))
|> child({:decoder, id}, Decoder)
|> child({:sink, id}, Sink)
end)
{[spec: spec], state}
end
| Module | Purpose |
|---|---|
Membrane.Funnel | Multiple inputs → one output |
Membrane.Tee | One input → multiple outputs |
Membrane.Connector | Connect dynamic pads with internal buffering |
Membrane.Testing.Source | Inject buffers into a pipeline in tests |
Membrane.Testing.Sink | Capture and assert on buffers in tests |
Membrane.Debug.Filter | Log/inspect buffers flowing through pipeline |
Membrane.Debug.Sink | Log/inspect buffers at pipeline end |
Membrane.FilterAggregator | It is deprecated, just don't use it |
import Membrane.ChildrenSpec
import Membrane.Testing.Assertions
alias Membrane.Testing
pipeline = Testing.Pipeline.start_link_supervised!(spec: [
child(:source, %Testing.Source{output: [<<1, 2, 3>>, <<4, 5, 6>>]})
|> child(:sink, Testing.Sink)
])
assert_sink_buffer(pipeline, :sink, %Membrane.Buffer{payload: <<1, 2, 3>>})
assert_start_of_stream(pipeline, :sink)
assert_end_of_stream(pipeline, :sink)
Testing.DynamicSource — like Testing.Source but with a dynamic output padAll timestamps are Membrane.Time.t(). It is integer nanoseconds under the hood, but don't use this information, because it is a part of the private API. However, keep in mind you can perform operations on Membrane.Time.t() using + or - operators. Helpers: Membrane.Time.seconds/1, Membrane.Time.milliseconds/1, Membrane.Time.microseconds/1, etc. Timers started with :start_timer action fire handle_tick/3.
More info: Membrane.Time, Timestamps guide.
Actions are returned from callbacks as {[action_list], state}. Full reference by component type:
| Module | Purpose |
|---|---|
Membrane.Pipeline | Pipeline behaviour & all callbacks |
Membrane.Pipeline.Action | Pipeline action type specs |
Membrane.Bin | Bin behaviour & all callbacks |
Membrane.Bin.Action | Bin action type specs |
Membrane.Element.Base | Shared element callbacks |
Membrane.Element.WithInputPads | handle_buffer/4, handle_stream_format/4, handle_end_of_stream/3 |
Membrane.Element.WithOutputPads | handle_demand/5 |
Membrane.Element.Action | Element action type specs |
Membrane.Pad | Pad definitions, Pad.ref/2 |
Membrane.Buffer | Buffer struct |
Membrane.ChildrenSpec | Topology DSL |
Callbacks are documented in the relevant behaviour modules:
handle_init, handle_setup, handle_playing, handle_call, handle_child_notification, handle_child_terminated, handle_crash_group_down, handle_element_end_of_stream, etc.handle_pad_added, handle_pad_removed, handle_parent_notificationhandle_init, handle_setup, handle_playing, handle_pad_added, handle_pad_removed, handle_parent_notification, handle_info, handle_tickhandle_buffer, handle_stream_format, handle_start_of_stream, handle_end_of_streamhandle_demand