Class: Factorix::Transfer::Downloader

Inherits:
Object
  • Object
show all
Defined in:
lib/factorix/transfer/downloader.rb

Overview

File downloader with caching and progress tracking

Downloads files from HTTPS URLs with automatic caching. Uses file locking to prevent concurrent downloads of the same file. HTTP redirects are handled automatically by the HTTP layer. Publishes progress events during download.

Instance Method Summary collapse

Instance Method Details

#download(url, output, expected_sha1: nil) ⇒ void

This method returns an undefined value.

Download a file from the given URL with caching support.

If the file exists in cache, it will be copied from cache instead of downloading. If the cached file fails SHA1 verification, it will be invalidated and re-downloaded. If multiple processes attempt to download the same file, only one will download while others wait for the download to complete. HTTP redirects are followed automatically by the HTTP layer.

Parameters:

  • url (URI::HTTPS) —

    URL to download from

  • output (Pathname) —

    path to save the downloaded file

  • expected_sha1 (String, nil) (defaults to: nil) —

    expected SHA1 digest for verification (optional)

Raises:



49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
 
# File 'lib/factorix/transfer/downloader.rb', line 49

def download(url, output, expected_sha1: nil)
  unless url.is_a?(URI::HTTPS)
    logger.error "Invalid URL: must be HTTPS"
    raise URLError, "URL must be HTTPS"
  end

  logger.info("Starting download", output: output.to_s)
  cache_key = strip_query(url)

  case try_cache_hit(cache_key, output, expected_sha1:)
  when :hit
    return
  when :miss
    logger.debug("Cache miss, downloading", output: output.to_s)
    publish("cache.miss", output: output.to_s)
  when :corrupted
    logger.debug("Re-downloading after cache invalidation", output: output.to_s)
    publish("cache.miss", output: output.to_s)
  else
    raise RuntimeError, "Unexpected cache state"
  end

  cache.with_lock(cache_key) do
    return if try_cache_hit(cache_key, output, expected_sha1:) == :hit

    with_temporary_file do |temp_file|
      download_file_with_progress(url, temp_file)
      verify_sha1(temp_file, expected_sha1) if expected_sha1
      cache.store(cache_key, temp_file)
      cache.write_to(cache_key, output)
    end
  end
end