Class: Omnizip::Algorithms::Zstandard::Frame::Header

Inherits:
Object
  • Object
show all
Includes:
Constants
Defined in:
lib/omnizip/algorithms/zstandard/frame/header.rb

Overview

Zstandard frame header parser (RFC 8878 Section 3.1.1.1)

Frame_Header structure:

  • Frame_Header_Descriptor: 1 byte
  • Window_Descriptor: 0-1 byte (optional)
  • Dictionary_ID: 0-4 bytes (optional)
  • Frame_Content_Size: 0-8 bytes (optional)

Constant Summary

Constants included from Constants

Constants::BLOCK_HEADER_SIZE, Constants::BLOCK_MAX_SIZE, Constants::BLOCK_TYPE_COMPRESSED, Constants::BLOCK_TYPE_RAW, Constants::BLOCK_TYPE_RESERVED, Constants::BLOCK_TYPE_RLE, Constants::BUFFER_SIZE, Constants::DEFAULT_LEVEL, Constants::DEFAULT_REPEAT_OFFSETS, Constants::FSE_DEFAULT_TABLELOG, Constants::FSE_MAX_ACCURACY_LOG, Constants::FSE_MIN_ACCURACY_LOG, Constants::HUFFMAN_MAX_BITS, Constants::HUFFMAN_MAX_CODE_LENGTH, Constants::HUFFMAN_MAX_LOG, Constants::HUFFMAN_STANDARD_TABLE_SIZE, Constants::HUF_SYMBOLVALUE_MAX, Constants::LDM_MIN_LEVEL, Constants::LITERALS_BLOCK_COMPRESSED, Constants::LITERALS_BLOCK_RAW, Constants::LITERALS_BLOCK_RLE, Constants::LITERALS_BLOCK_TREELESS, Constants::LITERALS_LENGTH_ACCURACY_LOG, Constants::LITERAL_LENGTH_TABLE, Constants::MAGIC_BYTES, Constants::MAGIC_NUMBER, Constants::MATCH_LENGTH_ACCURACY_LOG, Constants::MATCH_LENGTH_TABLE, Constants::MAX_LEVEL, Constants::MIN_LEVEL, Constants::MODE_FSE, Constants::MODE_PREDEFINED, Constants::MODE_REPEAT, Constants::MODE_RLE, Constants::OFFSET_ACCURACY_LOG, Constants::OF_BASE, Constants::OF_BITS, Constants::PREDEFINED_LL_DISTRIBUTION, Constants::PREDEFINED_ML_DISTRIBUTION, Constants::PREDEFINED_OFFSET_DISTRIBUTION, Constants::REPEAT_OFFSET_1, Constants::REPEAT_OFFSET_2, Constants::REPEAT_OFFSET_3, Constants::SKIPPABLE_MAGIC_BASE, Constants::SKIPPABLE_MAGIC_MASK, Constants::WINDOW_LOG_MAX, Constants::WINDOW_LOG_MIN

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Constants

highbit32

Constructor Details

#initialize(descriptor) ⇒ Header

Initialize with descriptor byte

Parameters:

  • descriptor (Integer) —

    Frame header descriptor byte



106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 106

def initialize(descriptor)
  @descriptor = descriptor

  # Extract flags from descriptor byte
  @content_size_flag = (descriptor >> 6) & 0x03
  @single_segment = (descriptor >> 5).allbits?(0x01)
  @checksum_flag = (descriptor >> 2) & 0x01
  @dictionary_id_flag = descriptor & 0x03

  @window_log = nil
  @window_mantissa = 0
  @dictionary_id = nil
  @content_size = nil
  @header_size = 1
end

Instance Attribute Details

#checksum_flag ⇒ Integer (readonly)

Returns Content checksum flag (bit 2).

Returns:

  • (Integer) —

    Content checksum flag (bit 2)



44
45
46
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 44

def checksum_flag
  @checksum_flag
end

#content_size ⇒ Integer? (readonly)

Returns Frame content size.

Returns:

  • (Integer, nil) —

    Frame content size



56
57
58
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 56

def content_size
  @content_size
end

#content_size_flag ⇒ Integer (readonly)

Returns Frame content size flag (bits 6-7).

Returns:

  • (Integer) —

    Frame content size flag (bits 6-7)



38
39
40
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 38

def content_size_flag
  @content_size_flag
end

#dictionary_id ⇒ Integer? (readonly)

Returns Dictionary ID.

Returns:

  • (Integer, nil) —

    Dictionary ID



53
54
55
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 53

def dictionary_id
  @dictionary_id
end

#dictionary_id_flag ⇒ Integer (readonly)

Returns Dictionary ID flag (bits 0-1).

Returns:

  • (Integer) —

    Dictionary ID flag (bits 0-1)



47
48
49
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 47

def dictionary_id_flag
  @dictionary_id_flag
end

#header_size ⇒ Integer (readonly)

Returns Total header size in bytes.

Returns:

  • (Integer) —

    Total header size in bytes



59
60
61
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 59

def header_size
  @header_size
end

#single_segment ⇒ Boolean (readonly)

Returns Single segment flag (bit 5).

Returns:

  • (Boolean) —

    Single segment flag (bit 5)



41
42
43
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 41

def single_segment
  @single_segment
end

#window_log ⇒ Integer? (readonly)

Returns Window log value.

Returns:

  • (Integer, nil) —

    Window log value



50
51
52
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 50

def window_log
  @window_log
end

Class Method Details

.parse(input) ⇒ Header

Parse frame header from input

Parameters:

  • input (IO) —

    Input stream positioned at frame header

Returns:



65
66
67
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 65

def self.parse(input)
  parse_from(input.read, 0).first
end

.parse_from(data, pos) ⇒ Array(Header, Integer)

Parse frame header from a byte string.

Parameters:

  • data (String)
  • pos (Integer) —

    offset of the descriptor byte

Returns:

  • (Array(Header, Integer)) —

    header and position after the header



75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 75

def self.parse_from(data, pos)
  descriptor = data.getbyte(pos)

  header = new(descriptor)
  offset = pos + 1

  if header.window_descriptor?
    header.parse_window_descriptor_byte(data.getbyte(offset))
    offset += 1
  end

  if header.dictionary_id?
    size = header.dictionary_id_size
    bytes = data.byteslice(offset, size)
    offset += size
    header.assign_dictionary_id(bytes)
  end

  if header.content_size?
    size = header.content_size_size
    bytes = data.byteslice(offset, size)
    offset += size
    header.assign_content_size(bytes)
  end

  [header, offset]
end

Instance Method Details

#assign_content_size(bytes) ⇒ Object

Assign the content size from its wire bytes.



217
218
219
220
221
222
223
224
225
226
227
228
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 217

def assign_content_size(bytes)
  @content_size = case bytes.bytesize
                  when 1 then bytes.ord
                  when 2 then bytes.unpack1("v") + 256
                  when 4 then bytes.unpack1("V")
                  when 8
                    low, high = bytes.unpack("VV")
                    low + (high << 32)
                  end

  @header_size += bytes.bytesize
end

#assign_dictionary_id(bytes) ⇒ Object

Assign the dictionary ID from its wire bytes.



206
207
208
209
210
211
212
213
214
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 206

def assign_dictionary_id(bytes)
  @dictionary_id = case bytes.bytesize
                   when 1 then bytes.ord
                   when 2 then bytes.unpack1("v")
                   when 4 then bytes.unpack1("V")
                   end

  @header_size += bytes.bytesize
end

#content_checksum? ⇒ Boolean

Check if content checksum is present

Returns:

  • (Boolean) 


146
147
148
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 146

def content_checksum?
  @checksum_flag == 1
end

#content_size? ⇒ Boolean

Check if content size is present

Returns:

  • (Boolean) 


139
140
141
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 139

def content_size?
  @content_size_flag != 0 || @single_segment
end

#content_size_size ⇒ Integer

Get the size of content size field

Returns:

  • (Integer) 


165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 165

def content_size_size
  if @single_segment
    # For single segment, FCS size depends on content_size_flag
    case @content_size_flag
    when 0 then 1
    when 1 then 2
    when 2 then 4
    when 3 then 8
    end
  else
    case @content_size_flag
    when 0 then 0
    when 1 then 2
    when 2 then 4
    when 3 then 8
    end
  end
end

#dictionary_id? ⇒ Boolean

Check if dictionary ID is present

Returns:

  • (Boolean) 


132
133
134
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 132

def dictionary_id?
  @dictionary_id_flag != 0
end

#dictionary_id_size ⇒ Integer

Get the size of dictionary ID field

Returns:

  • (Integer) 


153
154
155
156
157
158
159
160
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 153

def dictionary_id_size
  case @dictionary_id_flag
  when 0 then 0
  when 1 then 1
  when 2 then 2
  when 3 then 4
  end
end

#parse_window_descriptor_byte(byte) ⇒ Object

Parse window descriptor byte



199
200
201
202
203
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 199

def parse_window_descriptor_byte(byte)
  @window_mantissa = byte & 0x07
  @window_log = 10 + ((byte >> 3) & 0x1F)
  @header_size += 1
end

#window_descriptor? ⇒ Boolean

Check if window descriptor is present

Returns:

  • (Boolean) 


125
126
127
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 125

def window_descriptor?
  !@single_segment
end

#window_size ⇒ Integer?

Get window size

windowAdd = (windowBase / 8) * Mantissa (RFC 8878 ยง3.1.1.1.2); the mantissa occupies descriptor bits 0-2 and the exponent bits 3-7.

Returns:

  • (Integer, nil) —

    Window size or nil if not applicable



191
192
193
194
195
196
 
# File 'lib/omnizip/algorithms/zstandard/frame/header.rb', line 191

def window_size
  return nil unless @window_log

  window_base = 1 << @window_log
  window_base + ((window_base / 8) * @window_mantissa)
end