Class: PatientHttp::ResponseReader

Inherits:
Object
  • Object
show all
Defined in:
lib/patient_http/response_reader.rb

Overview

Reads and decodes HTTP response bodies.

Reading happens on the reactor thread and collects the raw (possibly compressed) body chunks with size validation. Decoding — joining the chunks, inflating compressed content, and applying the charset — is a separate step so it can run on a completion worker thread instead of blocking the event loop.

Defined Under Namespace

Classes: ReadAbortedError

Constant Summary collapse

INFLATE_WINDOW_BITS =

Content encodings that are inflated during decoding, mapped to the window bits of each wire format the encoding may arrive in. Formats are tried in order until one inflates the body.

A "deflate" body should carry a zlib header (RFC 9110 specifies the zlib format), but some servers send a bare deflate stream instead, so the raw format is kept as a fallback.

{
  "gzip" => [Zlib::MAX_WBITS | 16].freeze,
  "deflate" => [Zlib::MAX_WBITS, -Zlib::MAX_WBITS].freeze
}.freeze
IDENTITY_ENCODING =

Content encoding that means the body was not encoded at all. It needs no work to decode, but it still has to be recognized so it does not stop the decode of the encodings applied before it.

"identity"

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(processor, config: nil) ⇒ ResponseReader

Initialize the reader.

Reading needs a processor so it can abort once the processor is past its shutdown deadline. Decoding needs only the configuration, so a caller that does its own reading can supply the configuration on its own.

Parameters:

  • processor (Processor, nil) —

    the processor object

  • config (Configuration, nil) (defaults to: nil) —

    the configuration; defaults to the processor's configuration



102
103
104
105
 
# File 'lib/patient_http/response_reader.rb', line 102

def initialize(processor, config: nil)
  @processor = processor
  @config = config || processor.config
end

Class Method Details

.content_encodings(headers_hash) ⇒ Array<String>

Parse the content-encoding header into encoding names.

Parameters:

  • headers_hash (Hash) —

    the response headers

Returns:

  • (Array<String>) —

    the lowercased encoding names in applied order



60
61
62
63
64
65
 
# File 'lib/patient_http/response_reader.rb', line 60

def content_encodings(headers_hash)
  headers_hash["content-encoding"].to_s.split(",").filter_map do |name|
    name = name.strip.downcase
    name unless name.empty?
  end
end

.decodable?(name) ⇒ Boolean

Returns true if the reader can remove this encoding.

Parameters:

  • name (String) —

    a lowercased content encoding name

Returns:

  • (Boolean) —

    true if the reader can remove this encoding



69
70
71
 
# File 'lib/patient_http/response_reader.rb', line 69

def decodable?(name)
  name == IDENTITY_ENCODING || INFLATE_WINDOW_BITS.key?(name)
end

.rewrite_content_encoding(headers_hash) ⇒ Hash

Restate the content-encoding header for a decoded body. The header is removed when nothing is left applied, and narrowed to the encodings the reader could not remove otherwise, so the header always describes the body delivered with it.

Parameters:

  • headers_hash (Hash) —

    the response headers

Returns:

  • (Hash) —

    the headers with content-encoding updated or removed



80
81
82
83
84
85
86
87
88
89
90
 
# File 'lib/patient_http/response_reader.rb', line 80

def rewrite_content_encoding(headers_hash)
  return headers_hash unless headers_hash.key?("content-encoding")

  remaining, _decodable = split_encodings(headers_hash)

  if remaining.empty?
    headers_hash.except("content-encoding")
  else
    headers_hash.merge("content-encoding" => remaining.join(", "))
  end
end

.split_encodings(headers_hash) ⇒ Array(Array<String>, Array<String>)

Split the encodings named in the content-encoding header into the ones that stay applied to the body and the ones that can be decoded.

A body can carry more than one encoding. They are listed in the order they were applied, so decoding runs from the last name backwards and stops at the first name it does not recognize. Everything before that point stays applied to the body.

Parameters:

  • headers_hash (Hash) —

    the response headers

Returns:

  • (Array(Array<String>, Array<String>)) —

    the encodings that remain applied and the encodings that can be decoded, both in applied order



48
49
50
51
52
53
54
 
# File 'lib/patient_http/response_reader.rb', line 48

def split_encodings(headers_hash)
  encodings = content_encodings(headers_hash)
  boundary = encodings.rindex { |name| !decodable?(name) }
  return [[], encodings] if boundary.nil?

  [encodings[0..boundary], encodings[(boundary + 1)..]]
end

Instance Method Details

#decode_body(chunks, headers_hash) ⇒ String?

Decode raw body chunks into the final body string.

Joins the chunks, inflates gzip/deflate content (enforcing max_response_size on the inflated bytes), and applies the charset from the Content-Type header. This is CPU-bound work intended to run on a completion worker thread.

An encoding the reader does not support leaves the body encoded, and a body that is still encoded keeps its binary encoding because the charset does not describe it. Use split_encodings to find what stays applied so the content-encoding header delivered with the response describes the body it carries.

Parameters:

  • chunks (Array<String>, nil) —

    the raw body chunks

  • headers_hash (Hash) —

    the response headers

Returns:

  • (String, nil) —

    the decoded body or nil if there was no body

Raises:



145
146
147
148
149
150
151
152
153
154
155
156
157
158
 
# File 'lib/patient_http/response_reader.rb', line 145

def decode_body(chunks, headers_hash)
  return nil if chunks.nil?

  remaining, decodable = self.class.split_encodings(headers_hash)
  warn_undecodable(remaining) unless remaining.empty?

  body = inflate_encodings(chunks, decodable).join
  body.force_encoding(Encoding::ASCII_8BIT)
  # A body that is still encoded is not text yet, so the charset does not
  # describe its bytes. Leave it binary for the caller to decode.
  return body unless remaining.empty?

  apply_charset_encoding(body, headers_hash)
end

#read_raw_body(async_response, headers_hash) ⇒ Array<String>?

Read the raw response body chunks with size validation.

Reads the async HTTP response body asynchronously to completion, which allows the connection to be reused. The async-http client handles connection pooling and keep-alive internally. Using iteration instead of read() ensures non-blocking I/O that yields to the reactor. The chunks are the wire bytes: when the response is compressed, the size check here applies to the compressed bytes and #decode_body applies the same limit to the inflated bytes.

Parameters:

  • async_response (Async::HTTP::Protocol::Response) —

    the async HTTP response

  • headers_hash (Hash) —

    the response headers

Returns:

  • (Array<String>, nil) —

    the raw body chunks or nil if no body present

Raises:



121
122
123
124
125
126
 
# File 'lib/patient_http/response_reader.rb', line 121

def read_raw_body(async_response, headers_hash)
  return nil unless async_response.body

  validate_content_length(headers_hash)
  read_body_chunks(async_response)
end