Class: Beni::Vendor::Downloader

Inherits:
Object
  • Object
show all
Defined in:
lib/beni/vendor/downloader.rb,
sig/beni/vendor/downloader.rbs

Overview

Tarball download with exponential-backoff retry on transient network failures. One instance per (url, dest) pair; reuse is not supported and not needed by Beni::Tasks.

Public contract is the single #download entry point; TRANSIENT_ERRORS and MAX_RETRIES are exposed as tunable knobs but the retry mechanics themselves are internal.

Constant Summary collapse

MAX_RETRIES =

Retry attempts wait 1 << attempt seconds (2 + 4 + 8 = 14s total) — enough to ride out a GitHub archive 502 / TCP read timeout.

Returns:

  • (Integer) 
3
TRANSIENT_ERRORS =

Transient network errors retried by the internal with_retry wrapper. OpenURI::HTTPError is narrowed to 5xx; 4xx (URL typo, deleted repo) bypasses the retry path.

Returns:

  • (Array[singleton(StandardError)]) 
[
  OpenURI::HTTPError, Net::ReadTimeout, Net::OpenTimeout,
  Errno::ECONNRESET, SocketError
].freeze

Instance Method Summary collapse

Constructor Details

#initialize(url, dest) ⇒ Downloader

Returns a new instance of Downloader.

Parameters:

  • url (String)
  • dest (String) 


29
30
31
32
 
# File 'lib/beni/vendor/downloader.rb', line 29

def initialize(url, dest)
  @url = url
  @dest = dest
end

Instance Method Details

#download ⇒ void

This method returns an undefined value.

Fetch url into dest atomically via a .part sidecar, retrying transient failures with exponential backoff. Permanent failures (4xx, DNS resolution failure on non-network condition) surface immediately. Raises whatever the underlying URI#open raises after the retry budget is exhausted.



39
40
41
42
43
44
 
# File 'lib/beni/vendor/downloader.rb', line 39

def download
  FileUtils.mkdir_p(File.dirname(@dest))
  tmp = "#{@dest}.part"
  with_retry { fetch(tmp) }
  File.rename(tmp, @dest)
end

#fetch(tmp) ⇒ void

This method returns an undefined value.

Stream the URL body into tmp — the seam between the retry policy and the network; tests override this to script outcomes per attempt.

Parameters:

  • tmp (String) 


51
52
53
 
# File 'lib/beni/vendor/downloader.rb', line 51

def fetch(tmp)
  URI.parse(@url).open("rb") { |io| File.open(tmp, "wb") { |f| IO.copy_stream(io, f) } }
end

#permanent?(error) ⇒ Boolean

Parameters:

  • error (StandardError)

Returns:

  • (Boolean) 


67
68
69
 
# File 'lib/beni/vendor/downloader.rb', line 67

def permanent?(error)
  error.is_a?(OpenURI::HTTPError) && !error.message.match?(/\A5\d\d\b/)
end

#warn_and_sleep(error, attempt) ⇒ void

This method returns an undefined value.

Parameters:

  • error (StandardError)
  • attempt (Integer) 


71
72
73
74
75
 
# File 'lib/beni/vendor/downloader.rb', line 71

def warn_and_sleep(error, attempt)
  warn "[beni] retry #{attempt}/#{MAX_RETRIES} after #{error.class}: " \
       "#{error.message.lines.first&.strip}"
  sleep(1 << attempt)
end

#with_retry ⇒ void

This method returns an undefined value.



55
56
57
58
59
60
61
62
63
64
65
 
# File 'lib/beni/vendor/downloader.rb', line 55

def with_retry
  attempts = 0
  begin
    yield
  rescue *TRANSIENT_ERRORS => e
    raise if permanent?(e) || (attempts += 1) > MAX_RETRIES

    warn_and_sleep(e, attempts)
    retry
  end
end