Class: Omnizip::Chunked::Reader

Inherits:
Object
  • Object
show all
Defined in:
lib/omnizip/chunked/reader.rb

Overview

Read large files in chunks for memory-efficient processing

Constant Summary collapse

DEFAULT_CHUNK_SIZE =

64MB

64 * 1024 * 1024

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(file_path, chunk_size: DEFAULT_CHUNK_SIZE) ⇒ Reader

Initialize a chunked reader

Parameters:

  • file_path (String) —

    Path to file to read

  • options (Hash) —

    Reader options



15
16
17
18
19
20
 
# File 'lib/omnizip/chunked/reader.rb', line 15

def initialize(file_path, chunk_size: DEFAULT_CHUNK_SIZE)
  @file_path = file_path
  @chunk_size = chunk_size
  @total_size = File.size(file_path)
  @position = 0
end

Instance Attribute Details

#chunk_size ⇒ Object (readonly)

Returns the value of attribute chunk_size.



9
10
11
 
# File 'lib/omnizip/chunked/reader.rb', line 9

def chunk_size
  @chunk_size
end

#file_path ⇒ Object (readonly)

Returns the value of attribute file_path.



9
10
11
 
# File 'lib/omnizip/chunked/reader.rb', line 9

def file_path
  @file_path
end

#total_size ⇒ Object (readonly)

Returns the value of attribute total_size.



9
10
11
 
# File 'lib/omnizip/chunked/reader.rb', line 9

def total_size
  @total_size
end

Instance Method Details

#chunk_count ⇒ Integer

Get number of chunks needed for file

Returns:

  • (Integer) —

    Number of chunks



67
68
69
 
# File 'lib/omnizip/chunked/reader.rb', line 67

def chunk_count
  (@total_size.to_f / @chunk_size).ceil
end

#each_chunk {|chunk, position, total| ... } ⇒ Object

Iterate through all chunks

Yields:

  • (chunk, position, total) —

    Block called for each chunk



37
38
39
40
41
42
 
# File 'lib/omnizip/chunked/reader.rb', line 37

def each_chunk
  reset
  while (chunk = read_chunk)
    yield chunk, @position - chunk.bytesize, @total_size
  end
end

#eof? ⇒ Boolean

Check if at end of file

Returns:

  • (Boolean) —

    True if at EOF



61
62
63
 
# File 'lib/omnizip/chunked/reader.rb', line 61

def eof?
  @position >= @total_size
end

#progress ⇒ Float

Get current progress as percentage

Returns:

  • (Float) —

    Progress from 0.0 to 1.0



46
47
48
49
50
 
# File 'lib/omnizip/chunked/reader.rb', line 46

def progress
  return 1.0 if @total_size.zero?

  @position.to_f / @total_size
end

#read_chunk ⇒ String?

Read next chunk from file

Returns:

  • (String, nil) —

    Chunk data or nil if EOF



24
25
26
27
28
29
30
31
32
33
 
# File 'lib/omnizip/chunked/reader.rb', line 24

def read_chunk
  return nil if @position >= @total_size

  File.open(@file_path, "rb") do |f|
    f.seek(@position)
    chunk = f.read(@chunk_size)
    @position += chunk.bytesize if chunk
    chunk
  end
end

#remaining ⇒ Integer

Get remaining bytes to read

Returns:

  • (Integer) —

    Remaining bytes



73
74
75
 
# File 'lib/omnizip/chunked/reader.rb', line 73

def remaining
  @total_size - @position
end

#reset ⇒ self

Reset reader to beginning of file

Returns:

  • (self) 


54
55
56
57
 
# File 'lib/omnizip/chunked/reader.rb', line 54

def reset
  @position = 0
  self
end