mp4-rb

Ruby MP4 parser + fMP4 segmenter + on-the-fly HLS/DASH origin server. A learning project — not production software.

This project originally started as a personal learning exercise and proof of concept to better understand the MP4 container format, including its internal structure, parsing, and segmentation.

Over time, it evolved into an end-to-end prototype that demonstrates the complete media delivery pipeline: parsing MP4 files, generating fragmented MP4 (fMP4) segments, and delivering adaptive HLS and DASH streams.

While this implementation is far from production-ready and doesn't reflect the architecture of my current streaming server, it showcases the core concepts behind MP4 parsing, segment generation, and adaptive streaming. It represents an important milestone in my journey toward building a fully featured on-the-fly streaming solution.

⚠️ Disclaimer

This project was built for fun and for learning purposes. It is not maintained as a product, has known rough edges, and should not be used in production. If you're here to understand how MP4 containers work under the hood — how boxes/atoms nest, how the sample table (stbl) drives timing, how fragmented MP4 initialization and media segments are laid out, or how HLS and DASH manifests reference those segments — this repository is a good hands-on reference. You can read the code, run the demo, break things, and see what happens.

If that sounds useful to you, welcome. If you need a battle-tested packager, use Shaka Packager or Bento4 instead.

What you can learn from this repo

  • How MP4 is parsed. Every atom is a BinData::Record under lib/mp4/binary/ — ftyp, mvhd, tkhd, stsd, stts, stsc, stsz, stco, co64, ctts, elst, saio, saiz, senc, etc. No hand-rolled unpack.
  • How the sample table drives time. lib/mp4/metadata/memoize/ walks the parsed tree and flattens it into a per-sample cache: chunk offset, sample size, decode delta, composition offset, keyframe flag, and PTS in milliseconds.
  • How fMP4 segments are cut. lib/mp4/nolan/ divides each track into segments at video keyframes, and lib/m4s/ emits the styp + moof + mdat bytes for each media segment plus the moov init segment.
  • How HLS + DASH reference the same segments. lib/m4s/dash/manifest.rb writes the MPD XML with Nokogiri; lib/m4s/m3u8/ writes the master + media playlists. Both point at the exact same .m4s bytes (CMAF-style shared init) — the server just relabels the URL suffix.
  • How to serve any of the above on the fly. lib/mp4/server/api.rb is a Sinatra origin that parses on the first request, memoises the result in an LRU cache, and serves subsequent segment requests in milliseconds. It reads sources from a local directory or from S3-compatible object storage (MinIO in the demo) using GetObject byte-range reads.

The just demo — one-command reviewer setup

The fastest way to see the whole pipeline working end-to-end is:

just demo
# open http://127.0.0.1:4567/demo

That single command runs docker compose up --build on a four-service stack. You need only Docker installed — no Ruby, no ffmpeg, no bundle install.

What happens when you run just demo

  1. Encoder container synthesises examples/adaptive_demo.mp4 — a 20-minute video generated from ffmpeg -f lavfi -i testsrc2 + -f lavfi -i sine with a drawtext HH:MM:SS clock overlay. Four video renditions (1080p / 720p / 480p / 360p) + AAC-LC audio, GOP=48. No external MP4 required. (If you already have the file, this step is skipped.)

  2. MinIO container boots an S3-compatible object store on :9000 (console on :9001).

  3. MinIO-init container waits for MinIO to become healthy, creates the videos bucket, and uploads the fixture with mc cp.

  4. Server container waits for the encoder AND the S3 upload to complete, then boots the Sinatra origin. Before it accepts a single request, it pre-warms both caches (local + S3) by running the full parse → memoise → segmentise pipeline. This takes roughly a minute, and is why the first user hit on manifest.mpd is ~200 ms instead of 30 seconds.

  5. Open http://127.0.0.1:4567/demo and you get a page with four shaka-player instances side-by-side:

    Card Endpoint
    HLS · local /local/adaptive_demo.mp4/master.m3u8
    DASH · local /local/adaptive_demo.mp4/manifest.mpd
    HLS · S3 (MinIO) /s3/videos/adaptive_demo.mp4/master.m3u8
    DASH · S3 (MinIO) /s3/videos/adaptive_demo.mp4/manifest.mpd

    Each card has a quality picker (1080p → 360p), an ABR toggle, and a live stats overlay (currentTime, buffered, active variant, estimated bandwidth, dropped frames). All four play the same segments concurrently.

Total time from git clone to first frame on screen: ~11 minutes on a fresh clone (dominated by the encoder synthesising the 20-minute fixture). On subsequent runs, the encoder skips its work and boot time drops to under a minute.

Tearing it down

docker compose down                   # stop containers, keep fixture
docker compose down -v                # also drop MinIO volume
rm -f examples/adaptive_demo.mp4      # also drop the encoded fixture

Running things without Docker

If you have Ruby 3.x + Bundler + ffmpeg locally, you can also run each piece by hand:

just install                              # bundle install
just spec                                 # run the RSpec suite
just fixture                              # encode examples/adaptive_demo.mp4
just segments examples/foo.mp4            # emit fMP4 + DASH + HLS to out/fmp4/
just remux examples/foo.mp4               # regenerate a full MP4 from cache
just xlsx examples/foo.mp4                # per-sample analysis workbook
just server                               # boot Sinatra on :4567
just console                              # IRB with lib/mp4 loaded

Repository layout at a glance

lib/mp4/binary/       — BinData readers, one file per atom
lib/mp4/layout/       — writers for full MP4 remux
lib/mp4/metadata/     — parsed tree → per-sample cache
lib/mp4/nolan/        — keyframe-aligned segmentation
lib/mp4/cache/        — memory + SQLite cache backends
lib/mp4/source/       — local, HTTP, S3 source adapters
lib/mp4/server/       — Sinatra origin + LRU source cache
lib/m4s/binary/       — BinData readers for fMP4 boxes
lib/m4s/layout/       — init + media segment writers
lib/m4s/dash/         — MPD XML generation
lib/m4s/m3u8/         — master + media playlist generation

Development

just install
just spec
just lint

bin/console (just console) drops you into an IRB with MP4 loaded. It's the most useful way to poke at a real file:

mp4 = MP4.new(path: './examples/some.mp4')
mp4.cache.tracks
mp4.cache.chunks(1).first
mp4.build_dash_manifest_xml(init_segment_path: nil, segment_path: nil)

Not in scope

  • DRM (Widevine / PlayReady / FairPlay) — parsing of senc/saio/saiz is implemented, but nothing is encrypted end-to-end.
  • Live streaming — this is a VOD-only prototype.
  • Production hardening — no authentication, no rate limiting, no CDN fronting. Do not expose this to the public internet.

License

MIT-ish personal project. Use whatever you learn here freely; blame nobody but yourself if you ship it to production.