Module: Sentiero::Stores::SQLite::PayloadCodec

Defined in:
lib/sentiero/stores/sqlite/payload_codec.rb

Overview

zlib compression for events.data payloads. FullSnapshots dominate the table's bytes (~93% in production samples) and deflate ~6-7x, so payloads at or above the threshold are stored as compressed BLOBs; smaller rows stay plain JSON TEXT (near-zero win, and skipping them keeps ~85% of rows out of the deflate path). Reads sniff the storage format from the payload itself: a zlib stream starts with 0x78 and a valid header checksum, which no JSON text can.

Constant Summary collapse

COMPRESSION_THRESHOLD = 
1024
ZLIB_FIRST_BYTE =

CMF byte for deflate with a 32K window — the only one zlib emits.

0x78

Class Method Summary collapse

Class Method Details

.compressed?(data) ⇒ Boolean

Returns:

  • (Boolean) 


34
35
36
37
38
 
# File 'lib/sentiero/stores/sqlite/payload_codec.rb', line 34

def compressed?(data)
  data.getbyte(0) == ZLIB_FIRST_BYTE &&
    (flg = data.getbyte(1)) &&
    (((ZLIB_FIRST_BYTE << 8) | flg) % 31).zero?
end

.decode(data) ⇒ Object 



28
29
30
31
32
 
# File 'lib/sentiero/stores/sqlite/payload_codec.rb', line 28

def decode(data)
  return data unless compressed?(data)

  Zlib::Inflate.inflate(data).force_encoding(Encoding::UTF_8)
end

.encode(json) ⇒ Object 



22
23
24
25
26
 
# File 'lib/sentiero/stores/sqlite/payload_codec.rb', line 22

def encode(json)
  return json if json.bytesize < COMPRESSION_THRESHOLD

  ::SQLite3::Blob.new(Zlib::Deflate.deflate(json))
end