Class: Api2Convert::Http::Transport

Inherits:
Object
  • Object
show all
Defined in:
lib/api2convert/http/transport.rb

Overview

The HTTP layer: authenticated requests, transient-failure retries with exponential backoff, error-response mapping to typed exceptions, and JSON decoding.

Resources talk to the API through #request; the uploader and the downloader use #send_request / #interpret directly because they need non-JSON bodies and per-job auth. Internal.

Constant Summary collapse

RETRYABLE_STATUSES = 
[429, 500, 502, 503, 504].freeze
IDEMPOTENT_METHODS = 
%w[GET HEAD PUT DELETE OPTIONS TRACE].freeze
MAX_BACKOFF_SECONDS = 
8.0
MAX_RETRY_AFTER_SECONDS =

Upper bound for an honored Retry-After — a hostile/misconfigured value asking for an absurd delay can never stall a worker for hours.

120.0
TRANSPORT_ERRORS =

Transport-level failures worth wrapping as NetworkError (and retrying, for idempotent requests). SystemCallError covers the Errno::* family.

[
  SocketError, SystemCallError, IOError, EOFError,
  Timeout::Error, OpenSSL::SSL::SSLError, URI::Error,
  Net::OpenTimeout, Net::ReadTimeout, Net::HTTPBadResponse, Net::ProtocolError
].freeze

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(sender, config, sleeper: nil, rng: nil) ⇒ Transport

Returns a new instance of Transport.



38
39
40
41
42
43
 
# File 'lib/api2convert/http/transport.rb', line 38

def initialize(sender, config, sleeper: nil, rng: nil)
  @sender = sender
  @config = config
  @sleeper = sleeper || ->(seconds) { Kernel.sleep(seconds) }
  @rng = rng || -> { Kernel.rand }
end

Instance Attribute Details

#config ⇒ Object (readonly)

Returns the value of attribute config.



36
37
38
 
# File 'lib/api2convert/http/transport.rb', line 36

def config
  @config
end

Instance Method Details

#build_request(method, url, headers: {}, body_stream: nil, content_length: nil) ⇒ Object

Build a request object without sending it (used by the uploader, which supplies a streamed multipart body).



207
208
209
210
211
212
 
# File 'lib/api2convert/http/transport.rb', line 207

def build_request(method, url, headers: {}, body_stream: nil, content_length: nil)
  Request.new(
    method: method, url: url, headers: headers,
    body_stream: body_stream, content_length: content_length
  )
end

#download(uri, headers = {}, follow_redirects: true, sink: nil) ⇒ Object

Download a (self-contained) URL. Used for output downloads — these URLs need no API key.

When sink is an IO-like object (responds to write), the body is streamed to it chunk-by-chunk and nil is returned, so a large file never has to be buffered whole in memory. With no sink the body is buffered and returned (the in-memory contents path).

follow_redirects is true for a passwordless download (storage URLs legitimately redirect, and no secret is carried) and false when a download-password header is present (so it can never leak on a redirect).



186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
 
# File 'lib/api2convert/http/transport.rb', line 186

def download(uri, headers = {}, follow_redirects: true, sink: nil)
  request_headers = headers.nil? ? {} : headers
  build = lambda do
    Request.new(method: "GET", url: uri, headers: request_headers.dup, response_sink: sink)
  end
  response = send_request(build, replayable: true, follow_redirects: follow_redirects)
  ensure_successful(response)
  # A no-follow download that resolved to a 3xx (e.g. a password-protected
  # download whose redirect the policy refused to follow) must never be
  # written as the file: the body is the redirect page, not the content.
  # Guard the success path to 2xx so a 3xx can never silently corrupt output.
  unless (200..299).cover?(response.status)
    raise NetworkError,
          "Unexpected HTTP #{response.status} on download: refusing to write a " \
          "non-2xx (redirect) response as file content."
  end
  sink.nil? ? response.body : nil
end

#ensure_successful(response) ⇒ Object

Raise the appropriate typed exception when response is an HTTP error.



142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
 
# File 'lib/api2convert/http/transport.rb', line 142

def ensure_successful(response)
  status = response.status
  return if status < 400

  body = decode_safe(response)
  api_message = body["message"]
  message = api_message.is_a?(String) ? api_message : fallback_message(response)
  request_id = response.header("x-request-id")
  request_id = nil if request_id.nil? || request_id.empty?

  case status
  when 401, 403
    raise AuthenticationError.new(message, status_code: status, request_id: request_id, body: body)
  when 402
    raise PaymentRequiredError.new(message, status_code: status, request_id: request_id, body: body)
  when 404
    raise NotFoundError.new(message, status_code: status, request_id: request_id, body: body)
  when 429
    raise RateLimitError.new(
      message, status_code: status, request_id: request_id, body: body,
               retry_after: parse_retry_after(response.header("retry-after"))
    )
  when 400, 422
    raise ValidationError.new(message, status_code: status, request_id: request_id, body: body)
  else
    if status >= 500
      raise ServerError.new(message, status_code: status, request_id: request_id, body: body)
    end

    raise ApiError.new(message, status_code: status, request_id: request_id, body: body)
  end
end

#inspect ⇒ Object

Redacted representation — the default #inspect would dump the config's API key in cleartext, so it is overridden to mask it.



47
48
49
50
 
# File 'lib/api2convert/http/transport.rb', line 47

def inspect
  "#<#{self.class.name} base_url=#{@config.base_url.inspect} " \
    "api_key=#{Support::Secret.mask(@config.api_key)}>"
end

#interpret(response) ⇒ Object

Raise a typed exception for error responses; otherwise decode JSON.



125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
 
# File 'lib/api2convert/http/transport.rb', line 125

def interpret(response)
  ensure_successful(response)

  raw = response.body
  return {} if raw.nil? || raw.empty?

  begin
    decoded = JSON.parse(raw)
  rescue JSON::ParserError => e
    # A 2xx carrying a non-JSON body (e.g. an intermediary HTML page) must
    # surface as an SDK exception, not a raw parser error escaping the tree.
    raise NetworkError, "API2Convert returned a non-JSON success response: #{e.message}"
  end
  decoded.is_a?(Hash) || decoded.is_a?(Array) ? decoded : {}
end

#pause(seconds) ⇒ Object

Sleep for (at least) seconds with a small upward jitter. Used by job polling; the jitter keeps a fleet from polling in lockstep.



58
59
60
 
# File 'lib/api2convert/http/transport.rb', line 58

def pause(seconds)
  @sleeper.call(jitter(seconds))
end

#request(method, path, body = nil, query = nil, headers = nil) ⇒ Object

Perform an authenticated JSON request and return the decoded body.



63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
 
# File 'lib/api2convert/http/transport.rb', line 63

def request(method, path, body = nil, query = nil, headers = nil)
  request_headers = { "X-Oc-Api-Key" => @config.api_key }
  request_headers.merge!(headers) unless headers.nil?
  content = nil
  unless body.nil?
    content = JSON.generate(body)
    request_headers["Content-Type"] = "application/json"
  end
  url = build_url(path, query)

  build = lambda do
    Request.new(method: method, url: url, headers: request_headers.dup, body: content)
  end
  interpret(send_request(build))
end

#send_request(build, replayable: true, follow_redirects: false) ⇒ Object

Send a request (rebuilt fresh each attempt) with retry/backoff. build returns a fresh Request — a retry re-invokes it so a seekable body is replayed from the start. Adds the common Accept/User-Agent headers but no auth (callers add the header they need). replayable must be false for a non-seekable body so it is sent once.

follow_redirects defaults to false: authenticated requests carry a secret in a custom X-Oc-* header, which a redirect could leak to another host. Only the self-contained download path (no account key) opts in.



88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
 
# File 'lib/api2convert/http/transport.rb', line 88

def send_request(build, replayable: true, follow_redirects: false)
  attempt = 0
  loop do
    request = build.call
    request.headers["Accept"] = "application/json"
    request.headers["User-Agent"] = user_agent
    request.follow_redirects = follow_redirects
    idempotent = idempotent?(request)

    begin
      response = @sender.call(request)
    rescue *TRANSPORT_ERRORS => e
      # A non-idempotent request must not be replayed on a network error:
      # the backend may already have acted, so a blind retry could create a
      # duplicate job (and a duplicate charge).
      if replayable && idempotent && attempt < @config.max_retries
        backoff(attempt)
        attempt += 1
        next
      end
      raise NetworkError, "Request to API2Convert failed: #{e.message}"
    end

    status = response.status
    may_retry = RETRYABLE_STATUSES.include?(status) && replayable &&
                attempt < @config.max_retries && (status == 429 || idempotent)
    if may_retry
      backoff(attempt, response.header("retry-after"))
      attempt += 1
      next
    end

    return response
  end
end

#to_s ⇒ Object 



52
53
54
 
# File 'lib/api2convert/http/transport.rb', line 52

def to_s
  inspect
end