Class: Omnizip::Implementations::XZUtils::LZMA2::Encoder

Inherits:
Base::LZMA2EncoderBase show all
Includes:
Algorithms::LZMA::Constants
Defined in:
lib/omnizip/implementations/xz_utils/lzma2/encoder.rb

Overview

XZ Utils LZMA2 encoder.

Constant Summary collapse

UNCOMPRESSED_MAX =

XZ Utils limits (from lzma2_encoder.h) Maximum UNCOMPRESSED size per chunk: 2MB

1 << 21
COMPRESSED_MAX =

Maximum COMPRESSED size per chunk: 64KB

1 << 16
UNCOMPRESSED_CHUNK_MAX =

Uncompressed chunks store their size in a 16-bit field, so each uncompressed chunk is also limited to 64KB.

1 << 16

Constants included from Algorithms::LZMA::Constants

Algorithms::LZMA::Constants::BIT_MODEL_TOTAL, Algorithms::LZMA::Constants::COMPRESSION_LEVEL_DEFAULT, Algorithms::LZMA::Constants::COMPRESSION_LEVEL_MAX, Algorithms::LZMA::Constants::COMPRESSION_LEVEL_MIN, Algorithms::LZMA::Constants::DICT_SIZE_MAX, Algorithms::LZMA::Constants::DICT_SIZE_MIN, Algorithms::LZMA::Constants::DIST_ALIGN_BITS, Algorithms::LZMA::Constants::DIST_ALIGN_SIZE, Algorithms::LZMA::Constants::DIST_SLOT_FAST_LIMIT, Algorithms::LZMA::Constants::END_POS_MODEL_INDEX, Algorithms::LZMA::Constants::EOS_MARKER, Algorithms::LZMA::Constants::INIT_PROBS, Algorithms::LZMA::Constants::LEN_HIGH_SYMBOLS, Algorithms::LZMA::Constants::LEN_LOW_SYMBOLS, Algorithms::LZMA::Constants::LEN_MID_SYMBOLS, Algorithms::LZMA::Constants::LIT_SIZE_MAX, Algorithms::LZMA::Constants::MATCH_LEN_MAX, Algorithms::LZMA::Constants::MATCH_LEN_MIN, Algorithms::LZMA::Constants::MOVE_BITS, Algorithms::LZMA::Constants::NUM_DIRECT_BITS, Algorithms::LZMA::Constants::NUM_DIST_SLOTS, Algorithms::LZMA::Constants::NUM_DIST_SLOT_BITS, Algorithms::LZMA::Constants::NUM_FULL_DISTANCES, Algorithms::LZMA::Constants::NUM_LEN_HIGH_BITS, Algorithms::LZMA::Constants::NUM_LEN_LOW_BITS, Algorithms::LZMA::Constants::NUM_LEN_MID_BITS, Algorithms::LZMA::Constants::NUM_LEN_TO_POS_STATES, Algorithms::LZMA::Constants::NUM_LIT_CONTEXT_BITS_MAX, Algorithms::LZMA::Constants::NUM_LIT_POS_BITS_MAX, Algorithms::LZMA::Constants::NUM_POS_BITS_MAX, Algorithms::LZMA::Constants::NUM_STATES, Algorithms::LZMA::Constants::POS_STATES_MAX, Algorithms::LZMA::Constants::START_POS_MODEL_INDEX, Algorithms::LZMA::Constants::TOP

Instance Attribute Summary

Attributes inherited from Base::LZMA2EncoderBase

#dict_size, #lc, #lp, #pb

Instance Method Summary collapse

Methods inherited from Base::LZMA2EncoderBase

#standalone?

Constructor Details

#initialize(options = {}) ⇒ Encoder

Initialize the encoder

Parameters:

  • options (Hash) (defaults to: {}) —

    Encoding options

Options Hash (options):

  • :dict_size (Integer) —

    Dictionary size (default: 8MB)

  • :lc (Integer) —

    Literal context bits (default: 3)

  • :lp (Integer) —

    Literal position bits (default: 0)

  • :pb (Integer) —

    Position bits (default: 2)

  • :standalone (Boolean) —

    If true, write property byte at start (default: true)



60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/omnizip/implementations/xz_utils/lzma2/encoder.rb', line 60

def initialize(options = {})
  dict_size = options.fetch(:dict_size, 8 * 1024 * 1024)
  lc = options.fetch(:lc, 3)
  lp = options.fetch(:lp, 0)
  pb = options.fetch(:pb, 2)
  standalone = options.fetch(:standalone, true)

  super(
    dict_size: dict_size,
    lc: lc,
    lp: lp,
    pb: pb,
    standalone: standalone
  )

  # Shared state across all chunks
  @dictionary = Omnizip::Algorithms::LZMA::Dictionary.new(dict_size)
  @state = Omnizip::Algorithms::LZMA::LZMAState.new(0)
  @models = Omnizip::Algorithms::LZMA::XzProbabilityModels.new(lc,
                                                               lp, pb)
  @match_finder = Omnizip::Algorithms::LZMA::MatchFinder.new(@dictionary)
  @optimal = Omnizip::Algorithms::LZMA::OptimalEncoder.new(mode: :fast)

  # Track previous byte for literal context
  @prev_byte = 0

  # CRITICAL: For XZ Utils compatibility, first chunk MUST reset the dictionary
  # (matches XZ Utils behavior - see lzma2_encoder.c:334-336)
  # need_dictionary_reset is set to true for the first compressed chunk
  @need_state_reset = false
  @need_dictionary_reset = true # Always reset dictionary for first chunk (XZ Utils compatibility)
end

Instance Method Details

#encode(input_data) ⇒ Object 



93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
 
# File 'lib/omnizip/implementations/xz_utils/lzma2/encoder.rb', line 93

def encode(input_data)
  # CRITICAL: Reset match finder state for each encoding session
  # This ensures hash table and hash chain start fresh for each Xz.create call
  @match_finder.reset

  output = StringIO.new
  output.set_encoding(Encoding::BINARY)

  # Write property byte if standalone mode (for .lz2 files)
  # The property byte encodes dictionary size
  if @standalone
    output.putc(encode_dict_size(@dict_size))
  end

  input = StringIO.new(input_data)
  input.set_encoding(Encoding::BINARY)

  # Process in chunks (UNCOMPRESSED_MAX = 2MB per chunk)
  while !input.eof?
    chunk_data = input.read(UNCOMPRESSED_MAX)
    break if chunk_data.nil? || chunk_data.empty?

    consume_chunk(chunk_data, output)
  end

  # End marker (0x00) is REQUIRED for all LZMA2 streams
  # The @standalone flag only controls whether a property byte is written
  # at the START (for raw LZMA2 format like .lz2), not the end marker.
  # XZ format requires the end marker to properly terminate the LZMA2 stream.
  output.write([0x00].pack("C"))

  output.string
end

#implementation_name ⇒ Symbol

Get implementation identifier.

Returns:

  • (Symbol) —

    :xz_utils



130
131
132
 
# File 'lib/omnizip/implementations/xz_utils/lzma2/encoder.rb', line 130

def implementation_name
  :xz_utils
end