MediaMolder

The modern media processing engine that gives you 100% of FFmpeg's power and performance — with a visual editor, bulletproof validation, live observability, pure-Go extensibility, and an intelligent real-time adaptive controller.

MediaMolder GUI — Adaptive Bitrate x264 encode with real-time controller active!Video Encoding Real Time Controller

• Import any ffmpeg command line instantly
• Drag filters, encoders, sources, and sinks onto a canvas
• Set parameter values in an inspector panel with extended help for each parameter
• Validate your job to catch problems before you run it

• Hover edges for full stream metadata
• Live performance metrics while running
• In real-time mode, the Real-time controller panel shows detailed statistics (threads, presets, frame drops)
• Export back to the equivalent FFmpeg command line

Real-Time Controller

Activate with --realtime (CLI) or global_options.realtime: true (JSON) and MediaMolder turns on an adaptive closed-loop control system that enables reliable live video encoding**.

• Adaptive control loop (500 ms ticks) continuously monitors the performance of every encoder
• Three-tier adaptation:

• Configurable encoder input buffer (~4 s) + rolling output buffer (~4 s) absorbs upstream and downstream jitter (TCP stalls, HLS segment hiccups, SRT bursts, etc.)
• Live status badges in GUI + mediamolder watch + HTTP/SSE API
• Perfect for live streaming, HLS/DASH playout, broadcast, and any long-running job that must stay on pace

Graphical User Interface / Visual editor

FFmpeg runs media processing graphs, but until now you would have to visualize those graphs in your head. MediaMolder can import your FFmpeg command-line, and the GUI enables you to view, edit, validate, and run your graph with detailed performance metrics. The MediaMolder Graphical User Interface (GUI) is a fluid, drag-and-drop graph editor that runs in your web browser. The GUI is launched from the mediamolder binary by the gui subcommand. For details, see gui.md

• Build encode graphs by dragging filters, encoders, sources, and sinks onto

• The Inspector displays typed forms for every node: encoder rate-control

• When you select an input file, it is probed to determine its technical parameters.
• Hover any edge (wire) to see every technical property MediaMolder can infer for

• FFmpeg -> parses any ffmpeg command line and drops the equivalent

• -> FFmpeg shows you the equivalent FFmpeg command-line for

• The Run panel shows live per-node metrics — packets, rate, error count,

• MediaMolder graphs are saved as JSON files that can be run by passing the

• MediaMolder saves the position of every node in your graph layout, and it

• The properties panel includes extended help for most parameters, explaining

Safe by default

MediaMolder validates your graph before the first frame is touched.

mediamolder validate (and the GUI's inline annotations) run a static + probe-assisted analysis pass that catches every class of problem that would cause FFmpeg to crash silently or produce unusable output hours into a job: graph topology errors, codec/container incompatibilities, pixel-format mismatches, hardware boundary violations, HDR without tone-mapping, interlaced sources without a deinterlacer, VFR streams without an fps filter, odd dimensions rejected by encoders, and more. Every issue is reported in a single pass with a human-readable message, an ERROR/WARNING/INFO severity, and the exact node and edge where the problem occurs.

Where the fix is unambiguous, the GUI offers one-click automated remediation — auto-insert yadif/bwdif for interlaced sources, tonemap/zscale for HDR→SDR conversions, fps/format/scale adapters at incompatible boundaries, hwupload/hwdownload at hardware device transitions. You see the problem and its fix before committing any compute time.

Observable at every level

MediaMolder was designed for long-running and production jobs where "check after it finishes" is not an option.

• Per-node performance tracking (NodePerfTracker) records each node's

• Prometheus metrics for every node and graph: 20+ gauges, counters,

• /perf and /perf/stream HTTP endpoints expose the per-node snapshot

• mediamolder perf renders a live colour-coded terminal table — green

• OpenTelemetry span wiring: every graph run and every handler goroutine

Extensible in pure Go

Custom processing logic — object detection, AI filters, scene detection, subtitle generation, business-specific metadata — slots into any graph as a first-class node, written as an ordinary Go struct that implements the processors.Processor interface. No C, no rebuilds, no filtergraph string hacks. The engine schedules, monitors, and error-handles custom nodes identically to built-in nodes. For more details, see go-processor-nodes.md.

Processors may also implement the optional FrameSource sub-interface to generate their own frames rather than processing inbound ones — the built-in sequence_editor timeline node is one example (see Video editing built in).

You can add a custom Yolo-v8 object-detection node to a graph and it will run directly inside your media graph. See Yolo-V8 Guide

For multimodal scene understanding — captions, temporal grounding, edit plans, QA — the built-in vidi_analyzer node connects any graph to a Vidi 2.5 inference service. See Vidi 2.5 Guide

For cloud-hosted video understanding — index, search, caption, and embed clips via the TwelveLabs Marengo and Pegasus models — the twelvelabs_indexer, twelvelabs_analyzer, twelvelabs_searcher, and twelvelabs_embedder nodes are built in. See TwelveLabs Guide

For local, offline speech-to-text, the built-in whisper_stt node transcribes an audio stream to timestamped subtitles (SRT/VTT/JSON/TXT) with whisper.cpp. See Whisper Speech-to-Text Guide

For true camera-RAW develop (NEF/CR2/CR3/ARW/RAF/ORF/RW2/PEF/SRW/DNG) to a full, demosaicked 8-bit sRGB image via LibRaw — not the camera's embedded JPEG preview, and not libav's black RAW render — use the mediamolder raw-decode command or the built-in raw_decode node inside a graph. Deterministic develop (camera white balance, sRGB, AHD); LibRaw is bundled from pinned source and linked statically. See Camera-RAW Decode Guide

For native face analysis — detect faces (YOLOv8-face), align each, and optionally embed them (SFace) for recognition/clustering — use the mediamolder face-detect command for images/video or the built-in face_detect node inside a graph. Reproducible embeddings; models loaded as data, never linked. See Face Detection Guide

For packet-level bitstream analysis — every H.264/H.265/AV1 NAL unit / OBU reported to JSON (or JSONL/CSV) with no decoding: parameter sets, SEI, typed coded-picture records with derived Picture Order Count, per-unit class (vcl/ps/sei) and timing for bit-rate plots, optionally every syntax element — use the mediamolder trace-headers command or the built-in bitstream_trace node. An improved, machine-readable version of FFmpeg's trace_headers, validated byte-for-byte against it. See Bitstream Trace

Video editing built in

Assemble clips into a finished video — cuts, trims, wipes, dissolves, layering, and audio crossfades — entirely inside a job graph, with no external NLE. The built-in sequence_editor node is a multi-track timeline that opens its own sources and emits a finished stream: place clips on tracks at explicit times, add transitions composited by a native Go engine (the full xfade set), and mix audio from the same clips with crossfades auto-coupled to each cut. Build it as a spreadsheet-style table in the GUI or as declarative JSON. See the Video Editing Guide.

Frame-accurate trimming without a full re-encode

Cut a clip to exact in/out points while re-encoding only the GOP(s) the cuts land in — every whole GOP between them is stream-copied byte-for-byte, so the interior is never re-degraded. Set codec_video: "smartcopy" on an output with an ss/t/to window. See the Smart-cut Trimming Guide.

Hardware acceleration — any platform, properly

MediaMolder makes hardware acceleration safe and understandable. See hardware-acceleration.md

• A Hardware Capabilities dialog probes all available backends at startup

• Per-input, per-stream hardware decode control with a live scope hint in the

• Automatic hardware filter mapping: assign a CUDA device to a scale

• Apple ProRes RAW hardware decode via VideoToolbox — including

Production-grade infrastructure

• Declarative, version-controlled graphs. JSON files are diffable,

• Full timing control. -ss/-t/-to at input and output scope, a

• Graph state machine with live pause/resume, graceful cancellation via

• Trivially embeddable. The CLI and GUI are thin consumers of a clean Go

Remote and distributed execution

MediaMolder is built from the ground up to run jobs anywhere — on your laptop, on a single remote GPU box, or across a horizontally-scaled cluster — without changing a line of your graph JSON.

Three deployment tiers, one binary, one job format:

| Tier | How to start | Best for | |------|-------------|----------| | Local | mediamolder run job.json | Development, single-machine encodes | | Tier 1 — remote server | mediamolder serve --mode=server | Run jobs on a more powerful machine; GUI/CLI stays on your laptop | | Tier 2 — distributed cluster | --mode=api + --mode=worker | Scene-parallel encoding, fault tolerance, cloud autoscaling |

Why this matters for production workloads:

• Zero job-format migration. The same JSON config file that runs locally

• Fault-tolerant task execution. In Tier 2, every encode task is retried

• Scene-parallel encoding. The fanout_dynamic strategy splits a source

• Capability-aware routing. Workers advertise their hardware (cuda,

• S3 presigning at the server boundary. Submit jobs with s3:// URIs;

• Local file upload. Enable --enable-uploads to let GUI users and CLI

• Pluggable state and queue backends. Run in-memory + SQLite for local

• Strong authentication built in. Choose a static bearer token

• Distributed tracing end-to-end. Tier 2 propagates OpenTelemetry span

• GUI-first remote workflow. Click Backend in the toolbar, enter a

For setup instructions see Remote Backend Guide.

Drop-in FFmpeg migration

mediamolder convert-cmd turns any FFmpeg command line into a validated JSON config in one step: rate-control flags, per-stream maps, stream-copy nodes, tee/HLS/DASH muxers, bitstream filters, hardware devices, cover-art and attachment handling, -map_metadata/-map_chapters, two-pass encoding, and more — all converted with high fidelity and covered by round-trip regression tests. The generated graph runs immediately; the Inspector shows every option the conversion inferred so you can review and adjust. See FFmpeg Migration Guide

MediaMolder gives you 100% of FFmpeg's media processing capabilities — every codec, filter, hardware backend, and container format — with a graph model that validates before it runs, shows you what's happening while it runs, and tells you exactly what went wrong when it doesn't.

Prerequisites

• Go 1.23+
• FFmpeg 8.1+ (libavcodec 62.x, libavformat 62.x, libavfilter 11.x, libavutil 60.x)

• pkg-config (if using system FFmpeg)
• Git LFS (for the media test corpus, when available): git lfs install

Build / Install

See Build & Packaging

For detailed instructions see MacOS, Windows and Linux

Documentation

Usage

• Using MediaMolder (CLI)
• Visual Editor (GUI)
• Remote Backend Guide — run your jobs on a single remote server or a distributed cluster with API + workers
• Concepts — Graph Model, Nodes, Edges, Lifecycle
• JSON Config Reference
• FFmpeg Migration Guide
• Export to FFmpeg CLI
• Validation
• Video Editing Guide — assemble clips into a finished video: multi-track timelines, cuts/trims, dissolves and the full transition set with audio crossfades (sequence_editor)
• Go Processor Nodes — Processor interface, FrameSource interface, built-in processors (sequence_editor, scene detectors, TwelveLabs, Vidi, …), writing custom nodes
• Scene Detection — seven detectors: go-scene-detect ports (content, adaptive, threshold, hash, histogram), FFmpeg scdet, and the motion-compensated scene_change_mc (frame-accurate dissolve/fade detection)
• Yolov8 object detection/classification
• Vidi 2.5 multimodal analysis
• TwelveLabs video understanding
• Whisper speech-to-text — local, offline transcription to SRT/VTT/JSON/TXT (whisper_stt)
• Bitstream Trace — report every H.264/H.265/AV1 NAL unit / OBU as JSON without decoding; an improved, machine-readable trace_headers (trace-headers CLI + bitstream_trace node)
• Camera-RAW Decode — develop NEF/CR2/CR3/ARW/RAF/ORF/RW2/PEF/SRW/DNG to 8-bit sRGB via bundled LibRaw (raw-decode CLI + raw_decode node)
• Face Detection — detect, align, and embed faces in images/video for recognition/clustering (face-detect CLI + face_detect node)
• Real-Time Controller — adaptive control loop, encoder preset stepping, output buffers, mediamolder watch, HTTP API

Code

• Architecture
• Graph State Machine
• Graph Instrumentation Roadmap
• Clock & Sync
• Event Bus
• Error Handling
• Hardware Acceleration
• Observability — Prometheus metrics, OpenTelemetry tracing, per-node performance monitoring, mediamolder perf CLI
• Graph Compilation
• Video Transitions — native Go transition engine (wipes, slides, fades, circles, …) that replaces libavfilter xfade
• Camera-RAW Decode — LibRaw develop boundary, the three decode intents, determinism scoping, static-link rationale

Project

• MediaMolder Project
• Contribution & Governance
• Project Specification
• Benchmarks — mediamolder hwbench user tool + Go graph CI benchmarks
• Licensing

Star History