Class: Omnizip::Algorithms::LZMA::LzipDecoder

Inherits:
Object
  • Object
show all
Defined in:
lib/omnizip/algorithms/lzma/lzip_decoder.rb

Overview

Decoder for .lz (lzip) format

This is the lzip format, a DIFFERENT container format from both XZ and .lzma (LZMA_Alone). Lzip was created as an alternative to the legacy .lzma format with better integrity checking.

File format:

  • Magic bytes: "LZIP" (0x4C 0x5A 0x49 0x50)
  • Version (1 byte): 0 or 1
  • Dictionary size (1 byte): encoded format
  • LZMA1 compressed stream (with fixed LC=3, LP=0, PB=2)
  • Footer:
    • Version 0 (12 bytes): CRC32 (4) + Uncompressed size (8)
    • Version 1 (20 bytes): CRC32 (4) + Uncompressed size (8) + Member size (8)

Reference: /Users/mulgogi/src/external/xz/src/liblzma/common/lzip_decoder.c

This decoder uses the same LZMA1 decoding engine as XZ format, but with the lzip container format and CRC32 integrity checking.

Examples:

Decode .lz file

data = File.binread("file.lz")
decoder = Omnizip::Algorithms::LZMA::LzipDecoder.new(StringIO.new(data))
result = decoder.decode_stream

Defined Under Namespace

Classes: TrackingInputStream

Constant Summary collapse

MAGIC =

Lzip magic bytes: "LZIP" in ASCII Reference: lzip_decoder.c:106

[0x4C, 0x5A, 0x49, 0x50].freeze
LZIP_LC =

Fixed LC/LP/PB values for lzip format Reference: lzip_decoder.c:23-26

3
LZIP_LP = 
0
LZIP_PB = 
2
12
20
LZIP_V1_FOOTER_SIZE
MIN_DICT_SIZE =

Minimum and maximum dictionary sizes (in bytes) Reference: lzip_decoder.c:197-198

4096
MAX_DICT_SIZE =

4 KiB

(512 << 20)

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(input, options = {}) ⇒ LzipDecoder

Initialize the decoder with .lz format input

Parameters:

  • input (IO) —

    Input stream of .lz compressed data

  • options (Hash) (defaults to: {}) —

    Decoding options

Options Hash (options):

  • :ignore_check (Boolean) —

    If true, skip CRC32 verification (default: false)

  • :concatenated (Boolean) —

    If true, decode concatenated .lz members (default: false)

Raises:



85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
 
# File 'lib/omnizip/algorithms/lzma/lzip_decoder.rb', line 85

def initialize(input, options = {})
  @input = input
  @ignore_check = options.fetch(:ignore_check, false)
  # Concatenated mode is enabled by default (lzip natively supports multiple members)
  @concatenated = options.fetch(:concatenated, true)

  # Parse .lz header
  parse_header

  # Track member size (including header and footer)
  # We start with the 6 bytes we've already read (magic + version + dict_size)
  @member_size = 6

  # For concatenated mode, track if this is the first member
  @first_member = true

  # Initialize CRC32 calculator
  @crc32 = 0
  @uncompressed_size = 0
end

Instance Attribute Details

#dict_size ⇒ Object (readonly)

Returns the value of attribute dict_size.



55
56
57
 
# File 'lib/omnizip/algorithms/lzma/lzip_decoder.rb', line 55

def dict_size
  @dict_size
end

#member_size ⇒ Object (readonly)

Returns the value of attribute member_size.



55
56
57
 
# File 'lib/omnizip/algorithms/lzma/lzip_decoder.rb', line 55

def member_size
  @member_size
end

#version ⇒ Object (readonly)

Returns the value of attribute version.



55
56
57
 
# File 'lib/omnizip/algorithms/lzma/lzip_decoder.rb', line 55

def version
  @version
end

Instance Method Details

#decode_stream(output = nil) ⇒ String, Integer

Decode the .lz stream

Parameters:

  • output (IO, nil) (defaults to: nil) —

    Optional output stream

Returns:

  • (String, Integer) —

    Decompressed data or bytes written



110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
 
# File 'lib/omnizip/algorithms/lzma/lzip_decoder.rb', line 110

def decode_stream(output = nil)
  # For concatenated mode, accumulate all decoded data
  all_decoded_data = String.new(encoding: Encoding::BINARY)
  bytes_written = 0
  result = nil # Initialize result variable

  loop do
    # Track the starting position of compressed data
    start_pos = @input.pos

    # Initialize the XZ Utils LZMA decoder with fixed lzip parameters
    decoder = XzUtilsDecoder.new(@input,
                                 lzma2_mode: true,
                                 lc: LZIP_LC,
                                 lp: LZIP_LP,
                                 pb: LZIP_PB,
                                 dict_size: @dict_size,
                                 uncompressed_size: 0xFFFFFFFFFFFFFFFF) # Unknown size, allow EOPM

    # Decode the LZMA stream (allow EOPM for .lz format)
    # Get decoded data as string (no output stream)
    decoded_data = decoder.decode_stream(nil, check_rc_finished: false)

    # If caller provided output stream, write to it
    if output
      output.write(decoded_data)
      bytes_written += decoded_data.bytesize
      result = bytes_written
    else
      all_decoded_data << decoded_data
      result = all_decoded_data
    end

    # Calculate member size (header + compressed data + footer)
    # We calculate it here (compressed data + header), then add footer size below
    @member_size = @input.pos - start_pos + 6 # +6 for header bytes

    # Calculate and verify CRC32
    if @ignore_check
      # Skip footer
      footer_size = @version.zero? ? LZIP_V0_FOOTER_SIZE : LZIP_V1_FOOTER_SIZE
      @input.read(footer_size)
      @member_size += footer_size
    else
      data_to_crc = decoded_data || +""
      calculated_crc = Omnizip::Checksums::Crc32.calculate(data_to_crc)
      @uncompressed_size = data_to_crc.bytesize

      # Read and verify footer (also updates @member_size to include footer)
      verify_footer(calculated_crc)
    end

    # Check for concatenated members
    break unless @concatenated

    # Peek ahead to check if there's another LZIP member
    break unless has_next_member?

    # Parse header for next member
    parse_header
  end

  # Return decoded data or bytes written
  result
end