lz4rip: Ractor-safe LZ4 for Ruby

CI Gem Version License: MIT Ruby

Ruby bindings for lz4rip, a pure-Rust LZ4 implementation. Built as an rb-sys native extension and declared Ractor-safe so you can compress from any Ractor without a global lock.

Features

  • Block codec with reusable compressor scratch table
  • Frame codec for standard .lz4 frames
  • Dictionary support for both block and frame codecs
  • COVER-based dictionary trainer (DictTrainer)
  • Ractor-safe: FrameCodec is shareable across Ractors, BlockCodec is per-Ractor (mutable scratch state)

Install

Requires Ruby >= 3.4 and a Rust toolchain (for building the native extension):

gem install lz4rip

Or in your Gemfile:

gem "lz4rip"

Usage

Frame codec (standard LZ4 frames)

require "lz4rip"

codec = Lz4rip::FrameCodec.new
compressed = codec.compress("hello world " * 1000)
original   = codec.decompress(compressed)
bounded    = codec.decompress(compressed, max_decompressed_size: 12_000)

Block codec (raw LZ4 blocks)

codec = Lz4rip::BlockCodec.new
compressed = codec.compress("hello world " * 1000)
original   = codec.decompress(compressed, decompressed_size: 12_000)

Block decompression requires the original size up front. This is by design: LZ4 block format does not store it, so the caller must track it.

Dictionary compression

dict = Lz4rip::Dictionary.new(bytes: "common log prefix: ")
codec = Lz4rip::FrameCodec.new(dict: dict)

compressed = codec.compress("common log prefix: event=login user=alice")
original   = codec.decompress(compressed)

Dictionary training

trainer = Lz4rip::DictTrainer.new(2048)
messages.each { |msg| trainer.add_sample(msg) }
dict_bytes = trainer.train

dict  = Lz4rip::Dictionary.new(bytes: dict_bytes)
codec = Lz4rip::BlockCodec.new(dict: dict)

Ractor safety

On Ruby VMs without Ractor support, the codecs still work normally; the Ractor-specific guarantees and examples do not apply.

codec = Lz4rip::FrameCodec.new

ractors = 4.times.map do |i|
  Ractor.new(codec) do |c|
    data = "ractor #{Ractor.current} payload " * 100
    ct   = c.compress(data)
    raise "mismatch" unless c.decompress(ct) == data
    :ok
  end
end

ractors.each { |r| p r.value }  # => :ok, :ok, :ok, :ok

Documentation

Reference: https://rubydoc.info/gems/lz4rip

License

MIT