Module: Ucode::Fetch::Http

Defined in:
lib/ucode/fetch/http.rb

Overview

Shared HTTP wrapper. Single network boundary for the whole project.

Streaming download with retries and exponential backoff. Raises Ucode::NetworkError on final failure (after ‘http_retries` attempts).

Class Method Summary collapse

Class Method Details

.get(url, dest:, retries: nil, timeout: nil) ⇒ Pathname

Stream ‘url` to `dest` (a Pathname or String path).

Parameters:

  • url (String, URI) —

    full URL.

  • dest (Pathname, String) —

    destination file path. Parent directory is created if absent.

  • retries (Integer, nil) (defaults to: nil) —

    override Config.http_retries.

  • timeout (Integer, nil) (defaults to: nil) —

    override Config.http_timeout.

Returns:

  • (Pathname) —

    destination path on success.

Raises:



28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
 
# File 'lib/ucode/fetch/http.rb', line 28

def get(url, dest:, retries: nil, timeout: nil)
  uri = url.is_a?(URI) ? url : URI(url)
  destination = Pathname.new(dest)
  destination.dirname.mkpath

  attempts = retries || Ucode.configuration.http_retries
  read_timeout = timeout || Ucode.configuration.http_timeout
  backoff_sequence = DEFAULT_BACKOFF.take(attempts + 1)

  last_error = nil
  (attempts + 1).times do |attempt|
    return stream_to(uri, destination, read_timeout)
  rescue StandardError => e
    last_error = e
    sleep_for = backoff_sequence[attempt] || backoff_sequence.last
    Ucode.configuration.logger&.warn do
      "Http GET #{uri} failed (attempt #{attempt + 1}/#{attempts + 1}): " \
        "#{e.class}: #{e.message}; retrying in #{sleep_for}s"
    end
    sleep(sleep_for)
  end

  raise Ucode::NetworkError.new(
    "GET #{uri} failed after #{attempts + 1} attempts",
    context: { url: uri.to_s, last_error: last_error&.message },
  )
end