Class: Omnizip::Algorithms::Zstandard::Decoder

Inherits:
Object
  • Object
show all
Includes:
Constants
Defined in:
lib/omnizip/algorithms/zstandard/decoder.rb

Overview

Pure Ruby Zstandard decoder (RFC 8878).

Pipeline:

  1. For each frame: parse the frame header and reset per-frame state (repeat offsets, previous Huffman table, previous FSE tables).
  2. For each block: parse the 3-byte block header and dispatch on the block type (raw copy, RLE expansion, or compressed = literals + sequences).
  3. Verify the optional content checksum: the low 32 bits of XXHash64 over the decoded frame content.

Constant Summary

Constants included from Constants

Constants::BLOCK_HEADER_SIZE, Constants::BLOCK_MAX_SIZE, Constants::BLOCK_TYPE_COMPRESSED, Constants::BLOCK_TYPE_RAW, Constants::BLOCK_TYPE_RESERVED, Constants::BLOCK_TYPE_RLE, Constants::BUFFER_SIZE, Constants::DEFAULT_LEVEL, Constants::DEFAULT_REPEAT_OFFSETS, Constants::FSE_DEFAULT_TABLELOG, Constants::FSE_MAX_ACCURACY_LOG, Constants::FSE_MIN_ACCURACY_LOG, Constants::HUFFMAN_MAX_BITS, Constants::HUFFMAN_MAX_CODE_LENGTH, Constants::HUFFMAN_MAX_LOG, Constants::HUFFMAN_STANDARD_TABLE_SIZE, Constants::HUF_SYMBOLVALUE_MAX, Constants::LDM_MIN_LEVEL, Constants::LITERALS_BLOCK_COMPRESSED, Constants::LITERALS_BLOCK_RAW, Constants::LITERALS_BLOCK_RLE, Constants::LITERALS_BLOCK_TREELESS, Constants::LITERALS_LENGTH_ACCURACY_LOG, Constants::LITERAL_LENGTH_TABLE, Constants::MAGIC_BYTES, Constants::MAGIC_NUMBER, Constants::MATCH_LENGTH_ACCURACY_LOG, Constants::MATCH_LENGTH_TABLE, Constants::MAX_LEVEL, Constants::MIN_LEVEL, Constants::MODE_FSE, Constants::MODE_PREDEFINED, Constants::MODE_REPEAT, Constants::MODE_RLE, Constants::OFFSET_ACCURACY_LOG, Constants::OF_BASE, Constants::OF_BITS, Constants::PREDEFINED_LL_DISTRIBUTION, Constants::PREDEFINED_ML_DISTRIBUTION, Constants::PREDEFINED_OFFSET_DISTRIBUTION, Constants::REPEAT_OFFSET_1, Constants::REPEAT_OFFSET_2, Constants::REPEAT_OFFSET_3, Constants::SKIPPABLE_MAGIC_BASE, Constants::SKIPPABLE_MAGIC_MASK, Constants::WINDOW_LOG_MAX, Constants::WINDOW_LOG_MIN

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from Constants

highbit32

Constructor Details

#initialize(input_stream) ⇒ Decoder

Returns a new instance of Decoder.



44
45
46
47
48
49
 
# File 'lib/omnizip/algorithms/zstandard/decoder.rb', line 44

def initialize(input_stream)
  @input_stream = input_stream
  @executor = SequenceExecutor.new
  @previous_huffman_table = nil
  @previous_fse_tables = {}
end

Instance Attribute Details

#input_stream ⇒ IO (readonly)

Returns input stream.

Returns:

  • (IO) —

    input stream



42
43
44
 
# File 'lib/omnizip/algorithms/zstandard/decoder.rb', line 42

def input_stream
  @input_stream
end

Instance Method Details

#decode_stream ⇒ String

Decode a complete stream (one or more concatenated frames).

Returns:

  • (String) —

    decompressed data (binary)



54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
 
# File 'lib/omnizip/algorithms/zstandard/decoder.rb', line 54

def decode_stream
  data = read_all
  output = String.new(encoding: Encoding::BINARY)
  pos = 0

  loop do
    remaining = data.bytesize - pos
    break if remaining.zero?
    raise Omnizip::DecompressionError, "trailing bytes are not a frame" if remaining < 4

    magic = data.byteslice(pos, 4).unpack1("V")

    if (magic & SKIPPABLE_MAGIC_MASK) == SKIPPABLE_MAGIC_BASE
      pos = skip_skippable_frame(data, pos)
      next
    end
    unless magic == MAGIC_NUMBER
      raise Omnizip::DecompressionError,
            "invalid Zstandard magic: 0x#{magic.to_s(16)}"
    end

    frame_output, pos = decode_frame(data, pos + 4)
    output << frame_output
  end

  output
end

#decode_stream_with_dict(dict) ⇒ String

Decode a stream produced by a dictionary-primed encoder: the dictionary's content primes the reconstruction window of frames carrying its ID (verified; mismatch raises).

Parameters:

Returns:

  • (String) —

    decompressed data (binary)



88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
 
# File 'lib/omnizip/algorithms/zstandard/decoder.rb', line 88

def decode_stream_with_dict(dict)
  data = read_all
  output = String.new(encoding: Encoding::BINARY)
  pos = 0

  loop do
    remaining = data.bytesize - pos
    break if remaining.zero?
    raise Omnizip::DecompressionError, "trailing bytes are not a frame" if remaining < 4

    magic = data.byteslice(pos, 4).unpack1("V")
    unless magic == MAGIC_NUMBER
      raise Omnizip::DecompressionError,
            "invalid Zstandard magic: 0x#{magic.to_s(16)}"
    end

    frame_output, pos = decode_frame(data, pos + 4, dict)
    output << frame_output
  end

  output
end