Class: FetchUtil::SearchTransport::HttpClient

Inherits:
Object
  • Object
show all
Defined in:
lib/fetch_util/search_transport.rb

Overview

Separate from the regulatory HTTP client: this enforces the SERP host and deadline contract.

Constant Summary collapse

REDIRECT_LIMIT = 
4
MAX_RESPONSE_BYTES = 
2 * 1024 * 1024
INFLATE_CHUNK_BYTES = 
16 * 1024

Instance Method Summary collapse

Constructor Details

#initialize(clock: -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) }, max_response_bytes: MAX_RESPONSE_BYTES, net_http: nil) ⇒ HttpClient

Returns a new instance of HttpClient.



345
346
347
348
349
350
 
# File 'lib/fetch_util/search_transport.rb', line 345

def initialize(clock: -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) }, max_response_bytes: MAX_RESPONSE_BYTES,
               net_http: nil)
  @clock = clock
  @max_response_bytes = Integer(max_response_bytes)
  @net_http = net_http || ->(uri) { Net::HTTP.new(uri.host, uri.port) }
end

Instance Method Details

#get(url, deadline:, allowed_hosts:) ⇒ Object 



352
353
354
355
356
 
# File 'lib/fetch_util/search_transport.rb', line 352

def get(url, deadline:, allowed_hosts:)
  fetch(URI.parse(url), deadline, allowed_hosts.map(&:downcase), REDIRECT_LIMIT)
rescue URI::InvalidURIError
  HttpFailure.new(reason: "host", final_url: url)
end