Class: Sfdown::Downloader

Inherits:
Object
  • Object
show all
Defined in:
lib/sfdown.rb

Overview

Stage 2: create the local tree and download files. Counters are mutex-guarded so a ticker thread can snapshot progress while workers advance it. Sequential for now (concurrency arrives in a later milestone).

Constant Summary collapse

ILLEGAL =

Windows-illegal chars (per path segment)

/[<>:"\\|?*\x00-\x1f]/.freeze

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(project, http, dest, log: ->(m) { warn m }) ⇒ Downloader

dest is the project root directory (/).



348
349
350
351
352
353
354
355
356
357
358
359
360
361
 
# File 'lib/sfdown.rb', line 348

def initialize(project, http, dest, log: ->(m) { warn m })
  @project = project
  @http = http
  @dest = dest
  @log = log
  @mutex = Mutex.new
  @files = []
  @total_files = 0
  @total_bytes = 0
  @files_done = 0
  @bytes_done = 0
  @failures = 0
  @current = ""
end

Instance Attribute Details

#total_bytes ⇒ Object (readonly)

Returns the value of attribute total_bytes.



345
346
347
 
# File 'lib/sfdown.rb', line 345

def total_bytes
  @total_bytes
end

#total_files ⇒ Object (readonly)

Returns the value of attribute total_files.



345
346
347
 
# File 'lib/sfdown.rb', line 345

def total_files
  @total_files
end

Instance Method Details

#apply_folder_times(root) ⇒ Object

Apply folder timestamps last, deepest-first — writing files into a directory bumps its mtime, so folders must be stamped after their contents are final.



385
386
387
388
389
390
391
392
 
# File 'lib/sfdown.rb', line 385

def apply_folder_times(root)
  dirs = []
  each_node(root) { |n| dirs << n if n.dir? }
  dirs.sort_by { |n| -n.path.count("/") }.each do |n|
    path = local_path(n)
    File.utime(n.timestamp, n.timestamp, path) if n.timestamp && File.directory?(path)
  end
end

#bytes_done ⇒ Object 



364
 
# File 'lib/sfdown.rb', line 364

def bytes_done = @mutex.synchronize { @bytes_done }

#current ⇒ Object 



366
 
# File 'lib/sfdown.rb', line 366

def current = @mutex.synchronize { @current }

#download_all ⇒ Object 



379
380
381
 
# File 'lib/sfdown.rb', line 379

def download_all
  @files.each { |f| download_one(f) }
end

#failures ⇒ Object 



365
 
# File 'lib/sfdown.rb', line 365

def failures = @mutex.synchronize { @failures }

#files_done ⇒ Object 



363
 
# File 'lib/sfdown.rb', line 363

def files_done = @mutex.synchronize { @files_done }

#prepare(root) ⇒ Object

Create every directory up front and collect the flat file list + totals.



369
370
371
372
373
374
375
376
377
 
# File 'lib/sfdown.rb', line 369

def prepare(root)
  FileUtils.mkdir_p(@dest)
  each_node(root) do |n|
    FileUtils.mkdir_p(local_path(n)) if n.dir? && !n.path.empty?
    @files << n if n.file?
  end
  @total_files = @files.size
  @total_bytes = @files.sum(&:size)
end