Module: Philiprehberger::Batch

Defined in:
lib/philiprehberger/batch.rb,
lib/philiprehberger/batch/chunk.rb,
lib/philiprehberger/batch/result.rb,
lib/philiprehberger/batch/version.rb,
lib/philiprehberger/batch/processor.rb

Defined Under Namespace

Classes: Chunk, Error, Processor, Result, TimeoutError

Constant Summary collapse

VERSION = 
'0.7.0'

Class Method Summary collapse

Class Method Details

.process(collection, size: 100, concurrency: 1, retries: 0, timeout_per_chunk: nil, on_progress: nil) {|chunk| ... } ⇒ Result

Process a collection in batches with chunking, progress, and error collection.

Parameters:

  • collection (Array, Enumerable) —

    items to process

  • size (Integer) (defaults to: 100) —

    chunk size

  • concurrency (Integer) (defaults to: 1) —

    number of concurrent workers

  • retries (Integer) (defaults to: 0) —

    max retries per failed item (default: 0)

  • timeout_per_chunk (Numeric, nil) (defaults to: nil) —

    optional per-chunk timeout in seconds; when a chunk exceeds it, a TimeoutError is appended to Result#errors and processing continues with the next chunk

  • on_progress (Proc, nil) (defaults to: nil) —

    optional callback invoked after each chunk with a progress info hash

Yields:

  • (chunk) —

    block that receives a Chunk for processing

Returns:

  • (Result) —

    processing result



29
30
31
32
33
34
35
36
37
38
 
# File 'lib/philiprehberger/batch.rb', line 29

def self.process(collection, size: 100, concurrency: 1, retries: 0,
                 timeout_per_chunk: nil, on_progress: nil, &block)
  processor = Processor.new(
    size: size,
    concurrency: concurrency,
    retries: retries,
    timeout_per_chunk: timeout_per_chunk
  )
  processor.call(collection, on_progress: on_progress, &block)
end