Class: Omnizip::Algorithms::Zstandard::Encoder

Inherits:
Object
  • Object
show all
Includes:
Constants
Defined in:
lib/omnizip/algorithms/zstandard/encoder.rb

Overview

Pure Ruby Zstandard encoder (RFC 8878).

Emits a single-segment frame. Each 128 KiB chunk becomes one block, choosing whichever is smallest: RLE, a Compressed_Block (LZ77 match finding + Huffman literals + FSE-coded sequences), or a Raw_Block fallback for incompressible data. The match finder works over absolute positions, so blocks can reference earlier blocks; repeat-offset state is carried between blocks exactly as the decoder tracks it.

Constant Summary

Constants included from Constants

Constants::BLOCK_HEADER_SIZE, Constants::BLOCK_MAX_SIZE, Constants::BLOCK_TYPE_COMPRESSED, Constants::BLOCK_TYPE_RAW, Constants::BLOCK_TYPE_RESERVED, Constants::BLOCK_TYPE_RLE, Constants::BUFFER_SIZE, Constants::DEFAULT_LEVEL, Constants::DEFAULT_REPEAT_OFFSETS, Constants::FSE_DEFAULT_TABLELOG, Constants::FSE_MAX_ACCURACY_LOG, Constants::FSE_MIN_ACCURACY_LOG, Constants::HUFFMAN_MAX_BITS, Constants::HUFFMAN_MAX_CODE_LENGTH, Constants::HUFFMAN_MAX_LOG, Constants::HUFFMAN_STANDARD_TABLE_SIZE, Constants::HUF_SYMBOLVALUE_MAX, Constants::LDM_MIN_LEVEL, Constants::LITERALS_BLOCK_COMPRESSED, Constants::LITERALS_BLOCK_RAW, Constants::LITERALS_BLOCK_RLE, Constants::LITERALS_BLOCK_TREELESS, Constants::LITERALS_LENGTH_ACCURACY_LOG, Constants::LITERAL_LENGTH_TABLE, Constants::MAGIC_BYTES, Constants::MAGIC_NUMBER, Constants::MATCH_LENGTH_ACCURACY_LOG, Constants::MATCH_LENGTH_TABLE, Constants::MAX_LEVEL, Constants::MIN_LEVEL, Constants::MODE_FSE, Constants::MODE_PREDEFINED, Constants::MODE_REPEAT, Constants::MODE_RLE, Constants::OFFSET_ACCURACY_LOG, Constants::OF_BASE, Constants::OF_BITS, Constants::PREDEFINED_LL_DISTRIBUTION, Constants::PREDEFINED_ML_DISTRIBUTION, Constants::PREDEFINED_OFFSET_DISTRIBUTION, Constants::REPEAT_OFFSET_1, Constants::REPEAT_OFFSET_2, Constants::REPEAT_OFFSET_3, Constants::SKIPPABLE_MAGIC_BASE, Constants::SKIPPABLE_MAGIC_MASK, Constants::WINDOW_LOG_MAX, Constants::WINDOW_LOG_MIN

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from Constants

highbit32

Constructor Details

#initialize(output_stream, options = {}) ⇒ Encoder

Initialize encoder

Parameters:

  • output_stream (IO) —

    output for compressed data

  • options (Hash) (defaults to: {})

Options Hash (options):

  • :level (Integer) —

    compression level (1-22)

  • :checksum (Boolean) —

    write the frame checksum



46
47
48
49
50
51
 
# File 'lib/omnizip/algorithms/zstandard/encoder.rb', line 46

def initialize(output_stream, options = {})
  @output_stream = output_stream
  @options = options
  @level = options[:level] || DEFAULT_LEVEL
  @checksum = options.fetch(:checksum, false)
end

Instance Attribute Details

#options ⇒ Object (readonly)

Returns the value of attribute options.



38
39
40
 
# File 'lib/omnizip/algorithms/zstandard/encoder.rb', line 38

def options
  @options
end

#output_stream ⇒ Object (readonly)

Returns the value of attribute output_stream.



38
39
40
 
# File 'lib/omnizip/algorithms/zstandard/encoder.rb', line 38

def output_stream
  @output_stream
end

Instance Method Details

#encode_stream(data) ⇒ Object

Encode a data stream

Parameters:

  • data (String) 


56
57
58
59
 
# File 'lib/omnizip/algorithms/zstandard/encoder.rb', line 56

def encode_stream(data)
  data = data.dup.force_encoding(Encoding::BINARY)
  write_frame(data)
end

#encode_stream_with_dict(data, dict) ⇒ Object

Encode a data stream primed with a dictionary prefix (port of the Rust encode_frame_with_dict): the dictionary content is prepended as virtual history, the match finder is seeded with its positions, and only the plaintext is emitted. Frame_Content_Size and the checksum cover the plaintext only; the frame header carries the dictionary's ID and requires the dict-aware decoder.

Parameters:



71
72
73
74
75
76
77
78
79
80
 
# File 'lib/omnizip/algorithms/zstandard/encoder.rb', line 71

def encode_stream_with_dict(data, dict)
  data = data.dup.force_encoding(Encoding::BINARY)
  virtual = dict.content + data
  prefix_len = dict.content.bytesize

  write_u32le(MAGIC_NUMBER)
  write_frame_header(data, dict.id)
  write_blocks(virtual, prefix_len)
  write_u32le(XXHash64.frame_checksum(data)) if @checksum
end