Class: Omnizip::Parallel::ParallelExtractor

Inherits:
Object
  • Object
show all
Defined in:
lib/omnizip/parallel/parallel_extractor.rb

Overview

Parallel extraction coordinator using Fractor

Manages parallel extraction of files from an archive. Distributes extraction work across multiple workers and writes files to disk in a thread-safe manner.

Examples:

Extract archive in parallel

extractor = Omnizip::Parallel::ParallelExtractor.new(threads: 4)
extractor.extract('backup.zip', 'output/')

With options

options = Omnizip::Models::ParallelOptions.new
options.threads = 8
extractor = Omnizip::Parallel::ParallelExtractor.new(options)
extractor.extract('backup.zip', 'output/')

Defined Under Namespace

Classes: ExtractionWork, ExtractionWorker

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(options = nil, threads: nil) ⇒ ParallelExtractor

Initialize parallel extractor

Parameters:

  • options (Omnizip::Models::ParallelOptions, Hash) (defaults to: nil) —

    parallel options

  • threads (Integer) (defaults to: nil) —

    number of threads (overrides options)



129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
 
# File 'lib/omnizip/parallel/parallel_extractor.rb', line 129

def initialize(options = nil, threads: nil)
  @options = case options
             when Omnizip::Models::ParallelOptions
               options.dup
             when Hash
               Omnizip::Models::ParallelOptions.new.apply(options)
             else
               Omnizip::Models::ParallelOptions.new
             end

  @options.threads = threads if threads
  @options.validate!

  @stats = {
    files_extracted: 0,
    bytes_extracted: 0,
    start_time: nil,
    end_time: nil,
  }

  @write_mutex = Mutex.new
end

Instance Attribute Details

#options ⇒ Omnizip::Models::ParallelOptions (readonly)

Returns parallel options.

Returns:



120
121
122
 
# File 'lib/omnizip/parallel/parallel_extractor.rb', line 120

def options
  @options
end

#stats ⇒ Hash (readonly)

Returns extraction statistics.

Returns:

  • (Hash) —

    extraction statistics



123
124
125
 
# File 'lib/omnizip/parallel/parallel_extractor.rb', line 123

def stats
  @stats
end

Instance Method Details

#extract(archive, dest, **options) ⇒ Array<String>

Extract archive to directory in parallel

Parameters:

  • archive (String) —

    archive path

  • dest (String) —

    destination directory

  • options (Hash) —

    extraction options

Options Hash (**options):

  • :overwrite (Boolean) —

    overwrite existing files

  • :progress (Proc) —

    progress callback

Returns:

  • (Array<String>) —

    extracted file paths



160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
 
# File 'lib/omnizip/parallel/parallel_extractor.rb', line 160

def extract(archive, dest, **options)
  unless ::File.exist?(archive)
    raise Errno::ENOENT,
          "Archive not found: #{archive}"
  end

  overwrite = options.fetch(:overwrite, false)
  options[:progress]

  @stats[:start_time] = Time.now

  # Read archive to get entries
  entries = read_archive_entries(archive)

  # Create destination directory
  FileUtils.mkdir_p(dest)

  # Create job queue
  job_queue = JobQueue.new(max_size: @options.queue_size)

  # Schedule jobs
  entries.each do |entry|
    file_size = safe_entry_size(entry)

    job_queue.push_with_size(
      file: entry.name,
      size: file_size,
      data: { entry: entry },
    )
  end

  # Create work items from jobs
  work_items = []
  until job_queue.empty?
    job = job_queue.pop(timeout: 0.1)
    break unless job

    work_items << ExtractionWork.new(
      entry: job.data[:entry],
      archive_path: archive,
      dest_dir: dest,
    )
  end

  # Create worker pool
  pool = WorkerPool.new(
    worker_class: ExtractionWorker,
    num_workers: @options.threads,
    continuous: false,
  )

  pool.start
  pool.submit_batch(work_items)
  pool.run

  # Collect results
  results = pool.successful_results
  errors = pool.failed_results

  # Handle errors
  unless errors.empty?
    error_msgs = errors.map do |e|
      "#{e.work&.entry&.name}: #{e.error}"
    end.join("\n")
    raise Omnizip::ExtractionError, "Extraction errors:\n#{error_msgs}"
  end

  # Write files to disk (thread-safe)
  extracted_paths = write_extracted_files(results, overwrite: overwrite)

  pool.shutdown

  @stats[:end_time] = Time.now
  @stats[:files_extracted] = results.size

  extracted_paths
end

#statistics ⇒ Hash

Get extraction statistics

Returns:

  • (Hash) —

    statistics



241
242
243
244
245
246
247
248
249
250
251
252
 
# File 'lib/omnizip/parallel/parallel_extractor.rb', line 241

def statistics
  duration = if @stats[:start_time] && @stats[:end_time]
               @stats[:end_time] - @stats[:start_time]
             else
               0
             end

  @stats.merge(
    duration: duration,
    throughput_mbps: calculate_throughput(duration),
  )
end