AetherEngine
A media player engine for Apple platforms.
FFmpeg demuxes. VideoToolbox decodes. AVPlayer handles Dolby Atmos.
Video, live TV with DVR timeshift, or a lean audio-only path with system Now-Playing. You ship the UI.
What it is
A player engine that gets the hard parts right (HDR, Dolby Vision, Dolby Atmos, container coverage, codec coverage) and exposes a single AetherPlayerView (UIKit / AppKit) or AetherPlayerSurface (SwiftUI) plus a handful of async methods. No AVPlayerViewController. No opinionated controls. No analytics. Bind the view, call play(), read the published properties for state.
The view is polymorphic: under the hood the engine swaps the hosted CALayer (AVPlayerLayer for the native AVPlayer path, AVSampleBufferDisplayLayer for the SW dav1d fallback path) per session without the host having to know.
You provide the transport bar. You provide the dropdowns. You provide the pretty.
Used by
- Sodalite: native Jellyfin client for Apple TV.
- AetherPlayer: native macOS media player.
- NowSeen: IPTV / Manifest app for tvOS.
- KSKPix: KSKPix is a premium IPTV player for Live TV, Movies & Series.
- Syravo: Xtream Codes, Jellyfin and radio client for iPhone, iPad and Apple TV.
- KIPTV: Premium, cross-platform IPTV Player.
- Silo: native iOS, tvOS and macOS client for the Silo self-hosted media server.
- File Box: File Box is a simple and practical local file manager on iPhone and iPad.
- Moonfin: A multi-platform third party Jellyfin client.
- Vivid: open-source media app for iPhone, iPad and Apple TV.
- Snappier IPTV: IPTV App for iOS/iPadOS and tvOS.
- stashy: iOS/tvOS native player for stash.
- Gflix: A flawless media app for your media.
- Melon Video: Video Player for Vision Pro.
- WUW IPTV Player: Watch U Want — or simply WUW — your IPTV player for iPhone, iPad, Mac, Apple TV and Apple Vision Pro.
- GiVo Stream: media player for iPhone, iPad and Apple TV with bring-your-own playlists, live timeshift, DVR and VOD.
- LillyPlayer: The all-in-one IPTV and media player for Apple TV, iPhone and iPad (live TV, EPG, replay, Jellyfin, Plex, SMB...).
Shipping something on AetherEngine? Submit it to get listed here and on aetherengine.superuser404.de.
What it handles
A scannable summary; the depth for each row lives in docs/formats.md.
| Area | Summary |
| --- | --- |
| Containers | MKV, MP4, WebM, MPEG-TS, AVI, ASF / WMV, OGG, FLV |
| Disc | DVD-Video and Blu-ray ISO (decrypted): selectable titles and chapters, demuxed through the normal path |
| Video (HW) | H.264, HEVC, HEVC Main10 via VideoToolbox; AV1 Main profile where HW AV1 exists |
| Video (SW) | AV1 (dav1d) without HW, and AV1 High / Professional (4:4:4, 4:2:2, 12-bit) everywhere, VP9 / VP8, MPEG-4 Part 2 / MPEG-2 / VC-1, QuickTime RLE, the Flash tail (Sorenson Spark, On2 VP6) and anything else the FFmpeg build carries a decoder for (software is the default route; only HEVC, H.264 and HW-decodable AV1 go native), H.264 High 4:2:2 / 4:4:4 / 10 and HEVC Rext where VideoToolbox has no HW decoder (Intel Macs, older chips), interlaced H.264 (AVPlayer does not deinterlace; on VOD the declared field order is verified against decoded frames, so progressive-in-interlaced-carriage keeps hardware decode); GPU deinterlace (yadif_videotoolbox, Metal, field-rate by default) with a CPU bwdif fallback |
| HDR | HDR10, HDR10+ (per-frame ST 2094-40), Dolby Vision (P5, P7 on the native route as single-layer 8.1 (HDR10 base layer on a display without Dolby Vision), or composed dual-layer on Metal with per-frame DV metadata for FEL when preferDualLayerDolbyVision is set, the DualLayerDolbyVision trait is on and VideoToolbox decodes HEVC in hardware, P8.1, P8.4, AV1 P10.x), HLG, HDR Vivid (played on its HLG / PQ base, CUVA metadata reported by probe) |
| Audio | AAC, AC3, EAC3, FLAC, ALAC stream-copy lossless; TrueHD / MLP / DTS / DTS-HD MA / MP3 / MP2 / Opus / Vorbis / LPCM (incl. Blu-ray, G.711) / WMA (Standard, Pro, Lossless, Voice) / Nellymoser / ADPCM-SWF / Speex bridge to EAC3 5.1 (default) or lossless FLAC |
| Dolby Atmos | EAC3+JOC stream-copied on every route (HDMI MAT 2.0, AirPods spatial, BT downmix) except the opt-in Profile 7 FEL dual-layer route (preferDualLayerDolbyVision, DualLayerDolbyVision trait), which runs on the software host and plays an E-AC-3 JOC track as its 5.1 / 7.1 bed without the objects. No container reliably declares JOC pre-decode, so an honest TrackInfo.isAtmos needs a bounded decode: AetherEngine.probeDetectingAtmos(url:/source:) answers for a details screen without starting playback, and LoadOptions.confirmAtmos has the running session confirm its own tracks in the background and republish audioTracks. Both are opt-in and neither sits on the playback-start path. TrueHD Atmos keeps its heights too, opt-in through LoadOptions.objectAudioRendering in a build with the TrueHDObjectAudio trait (Optional components): the engine decodes the Atmos objects, renders them into a speaker bed and delivers Apple Positional Audio, which tvOS sends to an Atmos receiver as Dolby MAT and iOS plays as Spatial Audio (details). DTS:X takes the same route with the DTSXObjectAudio trait, its heights and objects unfolded from the compatible 7.1 (details) |
| Surround | 5.1 / 7.1 with correct AudioChannelLayout |
| Audio-only | LoadOptions.audioOnly: lean pipeline, no video machinery, system Now-Playing on tvOS / iOS |
| Background audio | Audio keeps playing when the app backgrounds on iOS: native AVPlayer stays alive, the software path drops video and keeps decoding audio (backgroundPlaybackEnabled / pictureInPictureActive); a paused session survives quick app switches for a grace window (backgroundTeardownGraceSeconds, default 15 s) before the wedge-safe teardown runs. tvOS tears down immediately (wedge-safe) unless a PiP window is active, which keeps the pipeline and loopback server alive; with PiP the software path keeps decoding video too (the window needs frames). Hosts gate corrective actions on the published isSessionReady |
| Picture in Picture | Native path: hosts build AVPictureInPictureController around currentAVPlayer, or around the session's own nativePlayerLayer when they render through bind(view:) rather than AVKit; while pictureInPictureActive a native->native load hands the AVPlayerItem over in place so the window survives next-episode transitions. Software path: published softwarePiPSource carries the AVSampleBufferDisplayLayer plus transport answers on the enqueued frames' PTS axis for sample-buffer PiP. While PiP is active the software path also composites active subtitle cues (text and PGS/DVB bitmaps) into the decoded frames, so the window shows subtitles the host overlay cannot reach. On the native path, a selected bitmap track's compositions are OCR-recognized into its WebVTT rendition, so PGS subtitles render in the PiP window there as well. iOS renders sample-buffer PiP; tvOS AVKit does not evaluate sample-buffer content sources (Apple FB9751461, verified through tvOS 26) |
| Subtitles | Text (SRT / ASS / SSA / VTT / mov_text) inline, bitmap (PGS / DVB / DVD) as CGImage, in-band CEA-608 closed captions (field-1, from an eia_608/c608 demuxable track or extracted from A53 cc_data embedded in the video bitstream: H.264/HEVC SEI on the native path, decoded-frame side data such as MPEG-2 on the software path, with the caption track surfacing lazily on first real caption data), DVB teletext decoded to text cues with broadcaster colour preserved and a selectable caption page (libzvbi, LoadOptions.teletextPage at load, setTeletextPage(_:) while the channel plays), external files as first-class tracks (registered, listed, selected like embedded streams), a live channel's own HLS SUBTITLES renditions surfaced as tracks and fetched only once one is selected, opt-in raw ASS markup + fonts; embedded-text cues harvested from the producer's own read (instant enable, no side-channel bandwidth); opt-in native WebVTT renditions (one per text track incl. load-declared external files, language-tagged) so subtitles survive PiP / AirPlay / external display (LoadOptions.prepareNativeSubtitles); bitmap tracks (PGS / DVB / DVD, embedded and external .sup) join as OCR-fed renditions: on-device Vision text recognition runs while the track is selected and fills a language-tagged text rendition, so bitmap subtitles survive PiP / AirPlay / external display on the native path too (lossy by design; fullscreen keeps the pixel-accurate overlay) |
| Frames | Off-playback FrameExtractor: thumbnail (scrub preview) + snapshot (frame-accurate) |
| Audio tap | Opt-in installAudioTap(): decoded playback audio as mono Float32 48 kHz PCM with source-PTS timestamps, off the render path (live transcription, ShazamKit); delivers on the loopback, remote-HLS (VOD + live), and software paths. The stream is bound to its session, so a session-preserving reload (audio / subtitle / disc-title switch, reloadAtCurrentPosition) finishes it and the host re-installs on stream end |
| Metadata | MediaMetadata (title / artist / album + cover) parsed on load; a container's album artist folds into artist as a fallback |
| Seek | VOD seeks into watched content are restart-free cache hits (byte-budgeted retention, 2 GiB cap); short forward scrubs ride the cached window; only never-produced targets restart the producer |
| Streaming | One long-lived forward-streaming connection, reconnect-on-drop; CDN-stutter resilient; optional caller-bounded open-time probe budget (LoadOptions.probesize / maxAnalyzeDuration) to cut first-frame latency on sparse remote remuxes; configurable forward-buffer window (LoadOptions.forwardBufferSegments), from the 40 s default up to an opt-in whole-source pre-buffer that is bounded in bytes by the session's disk budget rather than in segments |
| Prewarm | AetherEngine.prewarm(url:httpHeaders:byteBudget:) fetches a source's opening bytes before anything asks to play it, for a host whose UI knows what is next. The following load() of that URL serves its parse reads out of RAM, takes the size with the bytes instead of probing for it, and opens its data connection at the warm frontier rather than at byte zero, so nothing waits on a first byte. Static and off the main actor: no engine instance, no audio session, no layer. It never queues for the origin, the bytes live in memory only until they are adopted and are dropped under memory pressure, and the headers are part of the key |
| Live / DVR | Unbounded live + optional timeshift; direct HLS ingest with AES-128 clear-key and SSAI ad-pod handling |
| Custom input | Play any byte source via the IOReader protocol (load(source:)) |
| Network | SMB2/3 shares via the optional AetherEngineSMB product (NTLMv2 / guest, read-only) |
| Certificate trust | A media server behind a self-signed or private-CA certificate plays once the host answers for it: EngineTLS.serverTrustEvaluator is asked per challenge about the origin the challenge came from, so a LAN address behind a private certificate and a WAN address with a real one are decided separately. Covers every session the engine owns, and the native remote-HLS route too, where AVPlayer asks no delegate of its own: there the engine stands a loopback relay in front of the origin so the handshake runs where the evaluator is asked. The relay is mounted only for an origin the system actually refuses, and media is relayed as it arrives rather than read whole |
How it compares
On Apple platforms the real choice is between AVPlayer, with deep OS integration but only the formats Apple ships, and a VLC- or mpv-derived engine, which plays almost anything but renders its own frames and bypasses the system's Dolby Vision, Atmos, and HDR handling. AetherEngine is built to give you both: FFmpeg's format breadth layered on top of VideoToolbox and AVPlayer, so Dolby Vision, Atmos, and Match Content keep working. KSPlayer is the closest analog, it reaches the same outcome through the same AVPlayer route, but it ships as a full player with its own UI, and its free build is GPL while a paid LGPL tier unlocks AV1 hardware decoding, the full demuxer and decoder set, and a list of playback features its own feature matrix enumerates; AetherEngine is an embeddable engine you drive from your own SwiftUI, with that codec and HDR breadth in the open-source core.
| | AetherEngine | KSPlayer | AVPlayer | VLCKit | libmpv | | --- | --- | --- | --- | --- | --- | | Approach | Embeddable engine, Apple-only | Full player with bundled UI, FFmpeg + AVPlayer, Apple-only | Apple's built-in player | libVLC wrapped for Apple | libmpv, cross-platform | | Container & codec breadth | Wide, FFmpeg demux | Wide, FFmpeg demux | Narrow, Apple's set | Wide | Wide | | Hardware decode | VideoToolbox, dav1d SW fallback | VideoToolbox, FFmpeg SW fallback | VideoToolbox | VideoToolbox plus software | VideoToolbox plus software | | Dolby Vision | P5, P7 as 8.1 (or opt-in FEL dual-layer), P8.1, P8.4, AV1 P10.x, real display switch | P5, P8 via AVPlayer; their matrix lists P5 HDR display without overheating as paid LGPL | P5 and P8.1 only | Tone-maps, no DV display | Tone-maps, no DV display | | Dolby Atmos | EAC3+JOC stream-copied (HDMI MAT, spatial); TrueHD Atmos and DTS:X rendered to APAC, heights kept (optional components) | EAC3+JOC via AVPlayer | EAC3+JOC passthrough | Decodes to PCM, no object passthrough | No Atmos passthrough on Apple | | HDR on tvOS | Native Match Content switch | Native Match Content on AVPlayer path, else Metal tone-map | Native Match Content | Software tone-mapping | Software tone-mapping | | Rendering & UI | OS-native, you ship SwiftUI | Own Metal renderer, bundled controls | OS-native, you ship UI | Own renderer, bundled controls | Own renderer, bundled OSC | | Apple TV / App Store | Yes, LGPL plus store exception | Free build GPL, paid LGPL tier for AV1 hardware decode and the full decoder set | Yes | Yes, LGPL | Not practical, GPL, no tvOS |
The engine leans on the platform where the platform is best (hardware decode, Dolby Vision display, Atmos passthrough) and only falls back to its own software path (dav1d, libavcodec) for the formats VideoToolbox cannot handle.
Measured
The table above is qualitative. The numbers below are measured on 4K HDR HEVC in Matroska, the container media servers actually serve, at 38 Mbit/s and at 90 Mbit/s, the rate of a UHD remux. Every engine plays each file twice: from local disk, and over HTTP from a local range origin with a fixed latency and a shared link rate, because media servers deliver over HTTP and an engine's network reader is not its file reader. They are produced by aetherengine-bench, which is run by this project's author, so its method, its raw data and the cases where AetherEngine does not win are all in that repository. AVPlayer does not open Matroska at all; on MP4, the one container it plays, it uses less than every engine here, see that repository's README.
Measured on Apple-M1 (MacBookAir10,1), macOS 27.0, hevc-4k-hdr10.mkv (38 Mbit/s), hevc-4k-hdr10-90m.mkv (90 Mbit/s), each from local disk and over HTTP from a local range origin at 1000 Mbit/s shared, 20 ms latency per request, windowed, 3840x2160 px rendered, 60 s, median of 2. MacBookAir10,1 is fanless: sustained decode can reach thermal pressure, which is why the protocol has cooldowns between runs and discards throttled windows.
hevc-4k-hdr10.mkv (38 Mbit/s)
| | Source | GPU power | CPU load | RSS | Peak footprint | Plays | | --- | --- | --- | --- | --- | --- | --- | | AetherEngine | disk | 41 mW | 4.4% of a core | 342 MB | 323 MB | Yes | | | HTTP | 40 mW | 5.4% of a core | 336 MB | 350 MB | Yes | | KSPlayer | disk | 43 mW | 5.8% of a core | 333 MB | 772 MB | Yes | | | HTTP | 41 mW | 6.4% of a core | 357 MB | 496 MB | Yes | | AVPlayer | disk | | | | | n/a (refuses hevc-4k-hdr10.mkv: This media format is not supported.) | | | HTTP | | | | | n/a (refuses hevc-4k-hdr10.mkv: This media format is not supported.) | | VLCKit | disk | 266 mW | 15.0% of a core | 111 MB | 281 MB | Yes | | | HTTP | 244 mW | 17.0% of a core | 122 MB | 300 MB | Yes | | libmpv | disk | 887 mW | 20.7% of a core | 475 MB | 933 MB | Yes | | | HTTP | 892 mW | 21.5% of a core | 629 MB | 1191 MB | Yes |
Launch failures on hevc-4k-hdr10.mkv (crashes that were retried, not refusals, see the README): KSPlayer 1 (HTTP).
hevc-4k-hdr10-90m.mkv (90 Mbit/s)
| | Source | GPU power | CPU load | RSS | Peak footprint | Plays | | --- | --- | --- | --- | --- | --- | --- | | AetherEngine | disk | 40 mW | 6.6% of a core | 577 MB | 681 MB | Yes | | | HTTP | 40 mW | 9.3% of a core | 557 MB | 700 MB | Yes | | KSPlayer | disk | 41 mW | 6.7% of a core | 589 MB | 1040 MB | Yes | | | HTTP | 41 mW | 7.6% of a core | 578 MB | 703 MB | Yes | | AVPlayer | disk | | | | | n/a (refuses hevc-4k-hdr10-90m.mkv: This media format is not supported.) | | | HTTP | | | | | n/a (refuses hevc-4k-hdr10-90m.mkv: This media format is not supported.) | | VLCKit | disk | 235 mW | 16.2% of a core | 117 MB | 314 MB | Yes | | | HTTP | 235 mW | 18.4% of a core | 135 MB | 325 MB | Yes | | libmpv | disk | 953 mW | 21.5% of a core | 501 MB | 980 MB | Yes | | | HTTP | 962 mW | 22.3% of a core | 684 MB | 1188 MB | Yes |
Launch failures on hevc-4k-hdr10-90m.mkv (crashes that were retried, not refusals, see the README): KSPlayer 4 (disk).
Frame delivery and output (median; resolution, bit depth and HDR transfer are informational per engine only, shown as 'varies across repeats' when they disagree, see the README): hevc-4k-hdr10.mkv
- AetherEngine, disk: 1438/1438 delivered/expected, dropped 0, 3840x1714, 10-bit, hdr10, rendered into 3840x2160.
- AetherEngine, HTTP: 1441/1438 delivered/expected, dropped 0, 3840x1714, 10-bit, hdr10, rendered into 3840x2160.
- KSPlayer, disk: 1440/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, SMPTE_ST_2084_PQ, rendered into 3840x2160. Served via KSMEPlayer.
- KSPlayer, HTTP: 1440/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, SMPTE_ST_2084_PQ, rendered into 3840x2160. Served via KSMEPlayer.
- VLCKit, disk: 1451/1440 delivered/expected, dropped 0, 3840x1714, bit depth and color transfer not reported by this engine, rendered into 3840x2160.
- VLCKit, HTTP: 1454/1440 delivered/expected, dropped 0, 3840x1714, bit depth and color transfer not reported by this engine, rendered into 3840x2160.
- libmpv, disk: 1443/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, pq, rendered into 3840x2160.
- libmpv, HTTP: 1443/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, pq, rendered into 3840x2160.
- AetherEngine, disk: 1436/1438 delivered/expected, dropped 0, 3840x1714, 10-bit, hdr10, rendered into 3840x2160.
- AetherEngine, HTTP: 1439/1438 delivered/expected, dropped 0, 3840x1714, 10-bit, hdr10, rendered into 3840x2160.
- KSPlayer, disk: 1440/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, SMPTE_ST_2084_PQ, rendered into 3840x2160. Served via KSMEPlayer.
- KSPlayer, HTTP: 1440/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, SMPTE_ST_2084_PQ, rendered into 3840x2160. Served via KSMEPlayer.
- VLCKit, disk: 1454/1440 delivered/expected, dropped 0, 3840x1714, bit depth and color transfer not reported by this engine, rendered into 3840x2160.
- VLCKit, HTTP: 1452/1440 delivered/expected, dropped 0, 3840x1714, bit depth and color transfer not reported by this engine, rendered into 3840x2160.
- libmpv, disk: 1442/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, pq, rendered into 3840x2160.
- libmpv, HTTP: 1443/1440 delivered/expected, dropped 0, 3840x1714, 10-bit, pq, rendered into 3840x2160.
Power figures are package power with an idle baseline subtracted, so they are attributable to the run and not to the machine. CPU package power was measured for every run but is not published above: the idle baseline drifts with load enough that one engine's own repeats can disagree more than the column would be used to show between engines. For example, on hevc-4k-hdr10.mkv KSPlayer measured 76 to 267 mW of CPU package power across its own repeats, a 3.5x range. Recorded for every run in Results/; see "Known gaps" in the README for the full reasoning. libmpv is measured with --hwdec=auto-safe (hardware decode), not mpv's own software-decode default, see "Fairness decisions" in the README. Peak footprint is the kernel's lifetime maximum of the player process's physical footprint (what the system's memory limit acts on), load phase included; RSS is the mean over the measured window. Decoding that happens in a system service (VideoToolbox) is in neither. The hevc-4k-hdr10-90m.mkv rows come from a second run of the same protocol, build and origin on the same day and machine. A tvOS Simulator had been left running in the background for the first half of the main run, which raised idle package power to 188 to 214 mW (19 to 26 mW once it was shut down). Both hevc-4k-hdr10-90m.mkv baselines failed validation under it, so that fixture was measured again. The hevc-4k-hdr10.mkv baselines were taken under it too and passed; the GPU idle baseline stayed under 0.05 mW throughout, and CPU load, RSS and footprint are not baseline-subtracted, so none of the published columns moved with it. Method and raw results: https://github.com/superuser404notfound/aetherengine-bench
Quick start
import AetherEngine
import SwiftUI
let player = try AetherEngine()
// SwiftUI: drop AetherPlayerSurface anywhere in the view tree
var body: some View { AetherPlayerSurface(engine: player) }
// UIKit / AppKit: bind an AetherPlayerView directly
let surface = AetherPlayerView()
player.bind(view: surface)
try await player.load(url: videoURL) // or with a resume position
try await player.load(url: videoURL, startPosition: 347.5)
try await player.load(url: videoURL, options: .init(
httpHeaders: headers, // attached to every demux + segment fetch
matchContentEnabled: matchContent // tvOS Match Content master toggle
))
try await player.reloadAtCurrentPosition() // background reopen, preserves options + transport
try await player.load(url: trackURL, options: .init(audioOnly: true)) // lean audio path
// Transport
player.play()
player.pause()
player.togglePlayPause()
player.setRate(1.5) // clamped to player.maxSupportedRate (2x video, 3x audio-only)
await player.seek(to: 120)
player.stop()
// State (Combine @Published)
player.$state // .idle, .loading, .playing, .paused, .seeking, .ended, .error(String)
// .ended = played to completion (any backend); .idle = pre-load / stopped.
// .ended is TERMINAL: seek is rejected and play() does not revive it, so a
// replay is another load(). A VOD merely parked at its final frame is the
// other case and play() rewinds it. How every other failure arrives, and
// why a CancellationError out of load() is not one, is in docs/api.md.
player.$errorInfo // PlaybackErrorInfo?, the machine-readable half of .error: a stable
// PlaybackErrorKind plus the underlying NSError domain and code. Classify on
// this. The message cannot: on the native paths it is AVPlayerItem.error
// .localizedDescription forwarded verbatim, so it is in the device's
// language, and a substring rule over it buckets every non-English device as
// unknown. Set before .error is published, cleared when the state leaves it.
player.$duration
player.$videoFormat // .sdr, .hdr10, .hdr10Plus, .dolbyVision, .hlg
player.$isSeeking // true until a seek physically lands (programmatic + native scrubs, and
// seeks stashed before the session can take them)
player.$seekTarget // in-flight seek destination (source-PTS), nil otherwise
player.seekEvents // AnyPublisher<SeekEvent, Never>: .began / .landed(renderedTime:) /
// .stalled / .superseded / .rejected, each with the target it belongs to.
// Use this where the FALLING edge of $isSeeking matters: the level cannot
// say whether a seek landed, gave up, or was superseded, and a seek that
// gave up can still land minutes later on a stalled source.
player.$playbackPhase // unified: .idle/.loading/.playing/.paused/.seeking/.rebuffering/
// .stalled(reconnecting:)/.ended/.error. One source of truth for a status
// spinner; derived from state + isBuffering + isSeeking + source reconnect.
// Prefer this over stitching the raw signals or matching EngineLog text.
player.$videoRoute // pipeline actually serving the session: .remoteBypass (AVPlayer on the
// origin URL) / .loopback / .software / .audio / .none. LoadOptions
// .nativeRemoteHLS is only the request: the carriage watchdog, the
// remembered verdict and the HLS reroutes move a session between the
// bypass and the loopback, mid-session too. Branch on this where behaviour
// differs per pipeline, above all who draws subtitles: on .remoteBypass
// AVPlayer renders the origin's renditions, elsewhere the host renders.
player.$audioDelivery // how the audio reaches the renderer: .streamCopy / .bridged / .decoded /
// .noAudioInSource / .playerManaged (AVFoundation owns it) / .none, and
// .droppedNoPipeline: the source HAS audio and none of it could be
// delivered (no decoder in this build, or the bridge could not be built),
// so the session plays video-only and silently. That is the value a
// fallback ladder demotes on, the same way it demotes on the
// PlaybackErrorKind.audioBridgeProducedNoOutput at the other end of the
// cascade. Classify on this; $activeAudioDecoder is the label for a human
// and cannot separate a source without audio from one that lost it.
player.$hasFirstFrameReadyForDisplay
// the running path has a first frame ready for display, for the media THIS
// load opened: the edge a black cover comes off on. readyToPlay is not that
// edge (AVFoundation reaches it before the layer holds a picture, and it
// stays true across a seek), so a cover lifted on isSessionReady lifts onto
// black. Latched for the load, false again at the next load() / stop().
// For "has this seek reached the screen" use seekEvents .landed instead.
player.$startupProgress // StartupProgress?, for a determinate loading bar: .completed of .total
// checkpoints, .fraction, and .stage naming the work in flight. Every value
// is work some part of the load actually finished, never a timer and never an
// estimate, so a slow stretch holds and a skipped one jumps. It covers the
// two stretches a host cannot otherwise see: the source open (connection,
// container, stream analysis) and the display-criteria handshake. Scoped to
// .generation, which counts the startups a user waited through rather than
// teardowns, so an engine-initiated reroute continues the bar instead of
// resetting it. Monotonic and deduped; nil before the first load and after
// stop(); a load that fails simply never reaches the last checkpoint.
player.$currentAVPlayer // active AVPlayer, re-emitted on every reload (MPNowPlayingSession).
// nil on the .software route: that pipeline renders into its own display
// layer, and the property is cleared so AVKit cannot hold a player nothing
// feeds any more. A host that only ever hands currentAVPlayer to an
// AVPlayerViewController therefore gets audio and AVKit's own spinner over an
// empty video plane on that route (#298). Bind a surface for it, and pick the
// presentation off $videoRoute rather than off the source's codec.
// System Now-Playing on the native video path (tvOS / iOS). Off by default: an
// AVPlayerViewController host already gets this from AVKit and must NOT opt in.
// Custom-transport hosts set it before load(), then register commands on the
// session and stage identity through setVideoNowPlayingInfo.
player.ownsVideoNowPlayingSession = true
player.videoNowPlayingSession // MPNowPlayingSession?, nil unless opted in
player.setVideoNowPlayingInfo([ / MPMediaItemProperty… / ])
// Time lives on player.clock, a SEPARATE ObservableObject, so the ~10 Hz
// ticks never fire objectWillChange on the engine (track lists / state views
// don't re-render per tick; native tvOS Menu dropdowns stay stable).
player.clock.$currentTime // ~10 Hz playback clock (transport / scrub / resume)
player.clock.$sourceTime // source PTS of the displayed frame (render subtitles against this)
player.clock.$bufferedPosition // source-axis position buffered ahead; draw a buffer bar as bufferedPosition / duration
// Tracks
player.audioTracks // [TrackInfo]
player.selectAudioTrack(index: trackID)
player.subtitleTracks // [TrackInfo], text + bitmap + external, one list
player.selectSubtitleTrack(index: streamID)
player.clearSubtitle()
player.$subtitleCues // [SubtitleCue]: .text(String), .richText([SubtitleTextRun]), or .image(SubtitleImage)
// #233: styled text arrives as .richText. A run carries colour, bold, italic, underline,
// strikeout, font face and an ASS-relative font size; the cue carries the placement it asks
// for (numpad alignment plus an optional [0, 1] anchor). This covers SRT, WebVTT, teletext
// and ASS alike, because libavcodec converts them all to ASS event lines before the engine
// sees them. A cue with no styling still arrives as .text, so handling only that case keeps
// working. WebVTT cue settings are the exception: libavcodec does not convert them, so VTT
// positioning does not arrive (its inline bold/italic/underline does).
for cue in player.subtitleCues {
if case .richText(let runs) = cue.body { render(runs, at: cue.placement) }
}
// External subtitle files are first-class tracks (#88): they appear in subtitleTracks
// (isExternal == true, synthetic id) and select through the same call as embedded streams.
let track = player.addExternalSubtitleTrack(
ExternalSubtitleTrack(url: srtURL, name: "English", language: "en"))
player.selectSubtitleTrack(index: track.id)
// Declared at load instead, external tracks also join the native WebVTT renditions (PiP):
// LoadOptions(prepareNativeSubtitles: true, externalSubtitles: [ExternalSubtitleTrack(url: srtURL, language: "en")])
// When the URL is a container holding several subtitle streams, register one track per stream
// with its absolute AVStream index; tracks sharing a URL are decoded in one pass (#266).
ExternalSubtitleTrack(url: mkvURL, name: "Spanish", language: "es", sourceStreamIndex: 3)
// Native WebVTT subtitle renditions (subtitles in PiP / AirPlay / external display; opt-in
// via LoadOptions.prepareNativeSubtitles, details in docs/formats.md)
player.$nativeSubtitleTracks // [NativeSubtitleTrack]: ordinal, language, displayName
player.setNativeSubtitleSelected(track: 2) // engage a rendition (e.g. on PiP entry); nil deselects
// The SYSTEM turned captions on by itself: iOS 26 Settings > Accessibility > Subtitles & Captioning
// > Automatic Subtitles (show when muted, on skip back, on a language mismatch). Those toggles have
// no read API, so the selection is the only way to see the ask. The engine deselects the option (a
// rendition rendered in fullscreen draws a caption box over the host's own subtitles) and forwards
// the request; a host that wants the behaviour picks its own matching track. Subscribe per session.
player.systemCaptionRequest // PassthroughSubject<SystemCaptionRequest, Never>
// Second simultaneous subtitle track (bilingual / language learning)
player.selectSecondarySubtitleTrack(index: streamID)
player.selectSecondarySidecarSubtitle(url: srt2URL)
player.clearSecondarySubtitle()
player.$secondarySubtitleCues // [SubtitleCue] for the secondary track
player.$isSecondarySubtitleActive // Bool
// Disc titles + chapters (DVD-Video / Blu-ray ISO; empty for non-disc sources)
player.$discTitles // [TitleInfo]: id, name, durationSeconds, chapterCount (longest first, id 0 is the main feature)
player.$selectedDiscTitle // TitleInfo?
player.selectTitle(id: titleID) // switch title (rebuilds from the new title's head)
player.$discChapters // [ChapterInfo] for the selected title
player.selectChapter(id: chapterID) // seek to a chapter (disc chapters only)
// Container chapters (Matroska / MP4; empty for disc sources, which publish discChapters instead)
player.$mediaChapters // [ChapterInfo]; startSeconds are seek(to:) timestamps
// Info panel / Now Playing (iOS / tvOS)
player.setExternalMetadata([ AVMetadataItem(/ title, artwork, etc. /) ])
// Frame-accurate host rendering on the native path (#260). Cue times, chapters and sourceTime live on
// the SOURCE axis; AVPlayerItem.currentTime() and its timebase read the ITEM axis. They differ by the
// producer shift, which steps at every producer epoch (a seek restart, a live program boundary).
player.presentationAxisMap // conversion both ways, readable off the main actor
player.presentationAxisMap.itemSeconds(forSourceSeconds: cue.startTime) // stamp an overlay sample
player.presentationAxisMap.sourceSeconds(forItemSeconds: item.currentTime().seconds)
player.$currentAVPlayerItem // items swap in place; this is the signal for it
// Per muxed video frame, on both axes at once. Called on the producer's pump thread in DECODE order,
// so source is not monotonic under B-frames; sort before using it as a frame-boundary list.
// epoch rises strictly, process-wide: a restart and a load() both continue the sequence, so
// "a higher epoch retires my older entries" separates one item's frames from the next's.
player.setNativeVideoFrameTimeObserver { frame in
frame.source; frame.item; frame.segmentIndex; frame.isKeyframe; frame.epoch
}
// The same question on the software path (#311), where the engine decodes and enqueues the source
// timestamp unchanged: one axis, no segments, and the reports arrive past the reorder buffer in
// ASCENDING presentation order. generation moves on every renderer flush, i.e. on a seek, and
// rises across a load() the same way epoch does.
player.softwarePresentationTimebase // the master clock, on the source axis
player.setSoftwareVideoFrameTimeObserver { frame in
frame.presentation; frame.generation
}
// The rectangle those frames land in (#353): coded dimensions under the pixel aspect ratio the
// decoder attached, read off the format description the renderer enqueues. sourceVideoWidth and
// sourceVideoHeight are the CODED size, so anamorphic content laid out against them is off by the
// pixel aspect (720x576 at 64:45 presents as 1024x576). nil off the software path and before the
// first frame; it follows a mid-stream format change and is cleared with the session.
player.softwareDisplaySize // CGSize?, @Published
Subtitle cues land in raw source PTS; render the overlay against player.sourceTime (see docs/formats.md › Subtitles). A host compositing its own overlay onto the native path (libass and friends) needs the item axis too, since that is what the compositor pairs its samples against: presentationAxisMap converts arbitrary positions, setNativeVideoFrameTimeObserver reports the frames themselves. On the software path neither is needed: softwarePresentationTimebase hands out the clock the frames are presented against and setSoftwareVideoFrameTimeObserver reports them, both on the same axis as the cues, and softwareDisplaySize gives the rectangle to lay the overlay out in (the native path measures its own on AVPlayerLayer.videoRect). Both return nothing rather than a guess when no axis is established, because a defaulted shift is indistinguishable from a measured one at the call site. The 1 Hz diagnostics snapshot lives on player.diagnostics.liveTelemetry, off-the-engine for the same render-stability reason. Frame extraction and authored-ASS styling are documented in docs/formats.md; the full published surface, including the contracts that require the host to act rather than to read, is docs/api.md.
Install via Swift Package Manager:
.package(url: "https://github.com/superuser404notfound/AetherEngine", from: "7.39.0")Three samples ship in Examples/:
MinimalPlayer/: a single-file SwiftUI drop-in, transport bar included. Copy it into a new tvOS / iOS / macOS app, point at a URL, run.LiveHost/: the live-TV half, the four contracts a channel needs that no compiler asks about (theliveSourceResetretune and its guard,CancellationErrorout of a supersededload, the audio tap that ends with its session).DemoPlayerMac/: a standalone macOS app for testers. Drop a file on the window, it plays. A notarized universal.dmgis attached to every GitHub Release.
Custom input source
final class MyArchiveReader: IOReader {
func read(_ buffer: UnsafeMutablePointer<UInt8>?, size: Int32) -> Int32 { / ... / }
func seek(offset: Int64, whence: Int32) -> Int64 { / ... / } // AVSEEK_SIZE (65536) returns total size
func close() { / ... / }
// Optional requirements have defaults. Override to unlock extra features:
func cancel() { / unblock a blocked read at teardown, do NOT invalidate the reader / }
func makeIndependentReader() -> IOReader? { / a fresh cursor over the same source, or nil / }
// Optional, defaults to true. Return false when this is known to be an
// ordinary media file rather than a raw ISO/UDF disc image.
var discImageProbeEnabled: Bool { false }
}
let probe = try await engine.load(source: .custom(MyArchiveReader(), formatHint: "mp4"))
// load() returns the probe metadata it gathered (discardable). A one-shot
// AetherEngine.probe(source:) without starting playback works too.
Seekable readers support audio-track switching and background reload; embedded subtitles and scrub-preview thumbnails additionally need makeIndependentReader() (a second cursor). Forward-only readers support plain playback + seeking (VOD on the software path; live sessions stay native). On the native path a custom reader's bytes are re-muxed to cleartext fMP4 on the loopback cache, fine for encrypted-at-rest archives, a cleartext exposure for content-protected sources. Full contract in docs/formats.md.
SMB shares (optional AetherEngineSMB product)
Playing media off an SMB2/3 share is a ready-made IOReader, shipped as a separate product so the SMB dependency (SMBClient, MIT, pure Swift, NWConnection-based) only enters consumers that opt in. Add the AetherEngineSMB product alongside AetherEngine; hosts that do not need SMB link only the core. The transport is pure Swift over Network.framework rather than libsmb2, which fails with EPERM on tvOS/iOS.
import AetherEngineSMB
let smb = try await SMBConnection.connect(
server: URL(string: "smb://nas.local")!, share: "media",
path: "Movies/film.mkv", user: "alice", password: "s3cret"
)
try await engine.load(source: .custom(
SMBIOReader(source: smb),
formatHint: "matroska"
))
When the SMB path is known to be an ordinary media file, construct the reader with
discImageProbeEnabled: false to skip ISO/UDF signature reads. Keep the default for raw disc
images so DVD/Blu-ray recognition remains available.
Read-only, NTLMv2 / guest auth (no Kerberos). On tvOS the host must declare NSLocalNetworkUsageDescription + the local-network entitlement to reach a LAN share. See aetherctl smbtest to validate a share from macOS.
Known limitation: SMBClient negotiates only SMB 2.0.2 and 2.1, so there is no SMB3 transport encryption or AES-CMAC signing. Servers configured SMB3-only or with smb encrypt = required won't connect (libsmb2 spoke 3.1.1 here, but was itself unusable on tvOS/iOS, see above).
Live TV / DVR
// Live-only (seek() is a no-op), or live + timeshift:
try await player.load(url: streamURL, options: LoadOptions(isLive: true))
try await player.load(url: streamURL, options: LoadOptions(isLive: true, dvrWindowSeconds: 1800))
// IPTV channel zapping: join in ~3-6s instead of 10-18s on strict-realtime origins by letting
// TARGETDURATION (and the live-edge holdback derived from it) track the source keyframe cadence:
try await player.load(url: streamURL, options: LoadOptions(isLive: true, liveJoinProfile: .fastZap))
// Drive a scrubber from the live-edge fields (they tick, so they live on player.clock):
player.clock.$seekableLiveRange // ClosedRange<Double>?, session-relative; nil when DVR off
player.clock.$behindLiveSeconds // seconds behind the edge; 0 at the edge
player.clock.$liveEdgeTime
await player.seekToLiveEdge()
await player.seek(to: player.liveEdgeTime - 300) // 5 minutes back
// The retune contract: the engine parked the session and only a fresh URL revives it.
player.liveSourceReset // PassthroughSubject<Void, Never>; subscribe per session
// Ingest a live HLS upstream directly, no media server in the data path:
try await player.load(
source: .custom(HLSLiveIngestReader(playlistURL: upstreamM3U8), formatHint: "mpegts"),
options: LoadOptions(isLive: true, dvrWindowSeconds: 600)
)
// Record the channel WITHOUT opening a second connection to the origin:
try await player.startRecording(to: fileURL)
await player.stopRecording()
player.$recordingState // .idle / .recording / .ended / .failed
startRecording(to:) (AetherEngine#560) writes the live source to an MPEG-TS file fed from the connection the session already holds. That is the point of it rather than a detail: an IPTV plan commonly caps an account at 1 to 3 simultaneous connections, so a host that opens its own connection to record either fails outright or knocks the viewer off the channel. It is a stream copy of the source packets taken before any audio bridging, so a TrueHD or DTS channel records its original audio while playback listens to the bridged rendition, and a file cut short by a crash is still playable up to the cut. Recording follows the source rather than the playhead, so pausing or scrubbing back inside the DVR window does not interrupt it, and a liveSourceReset ends it cleanly rather than writing past a seam the codecs may not survive. The one route that cannot record is nativeRemoteHLS (.remoteBypass), where AVFoundation holds the source connection and the engine never sees a byte; startRecording throws .unsupportedRoute there instead of producing an empty file. Full contract in docs/api.md.
liveJoinProfile: .fastZap (AetherEngine#195/#208) cuts live segments at every keyframe past 0.5 s instead of the standard ~4 s, so the served TARGETDURATION collapses to 1.5 x the source GOP length (1 s GOPs serve 2, and so do segments up to 1.666 s; the headroom covers a broadcast's irregular GOPs, AetherEngine#670) and its live-edge holdback (HOLD-BACK = 3 x TARGETDURATION, the RFC 8216bis floor; AetherEngine#189) shrinks with it. The first manifest still prefers the full holdback. After two finalized segments, a strict-realtime source gets one observed-segment grace clamped to 0.5...2.0 s, then a shallow first window may be served so startup stays bounded. This can produce one early -16832 or a short rebuffer. .standard retains the full-holdback guarantee. The smaller TARGETDURATION also tightens AVPlayer's unchanged-playlist patience and live-edge buffer, so origins that stall or burst mid-stream rebuffer more readily; opt in for zapping UX, keep .standard for lean-back viewing.
liveSourceReset is live's counterpart to a terminal state = .error, and a host that plays live has to subscribe to it. It fires where the session cannot be revived from inside the engine and only a new URL can: a source that restarted from byte 0 (a Jellyfin transcode respawn), a playlist still frozen after the stall ladder's last reload rung (#65), or an in-engine reopen transport whose budget is spent (#199). Each of those halts production first, so a dead provider stops advertising blocking reloads behind the host's back. Answer it by negotiating a fresh URL and calling load again, and guard that answer (one retune in flight, a minimum spacing, a bounded count per session) or a permanently dead upstream turns into a retune loop. Spend that bound out loud: a retune ladder that ends on a silent return leaves the same dead channel behind a counter, so surface the exhaustion the way a terminal .error would be surfaced.
A host with no negotiation to do, an IPTV channel whose URL is fixed, answers by loading the same URL again, and that is the cheap path rather than a no-op. The #168 carriage verdict (a master advertising HEVC while delivering MPEG-TS) is remembered per exact absolute URL for six hours, 32 entries, so the retune routes straight onto the live ingest instead of re-paying the doomed native mount and its watchdog grace. A URL carrying a rotated per-session token misses that memory and re-pays the one-time discovery per retune, which is worth knowing where a first-frame budget is measured against the retune as well.
Left unsubscribed it costs a channel that stops while the engine still reports a session, which f
... (README truncated for length)