Class: OpenAI::NetHTTPClient

Inherits:

HTTPClient

• Object
• OpenAI::NetHTTPClient

Defined in:

lib/openai/net_http_client.rb,
sig/openai/net_http_client.rbs

Overview

The SDK's pooled Net::HTTP implementation.

Network operations from a non-blocking fiber cooperate with its active Ruby Fiber scheduler, allowing concurrent requests and streams without occupying one thread per request.

Pass a block to configure each SDK-created connection before it is pooled and started.

Constant Summary

KEEP_ALIVE_TIMEOUT =

from the golang stdlib https://github.com/golang/go/blob/c8eced8580028328fde7c03cbfcb720ce15b2358/src/net/http/transport.go#L49

Returns:

• (30)

30

DEFAULT_MAX_CONNECTIONS =

Returns:

• (Integer)

[Etc.nprocessors, 99].max

URI =

Returns:

• (:Generic url,)

Instance Method Summary

• #close ⇒ void

  Closes current pooled connections.

• #execute(request) ⇒ OpenAI::HTTPClient::Response

  Executes a request using a pooled Net::HTTP connection.

• #initialize(size: self.class::DEFAULT_MAX_CONNECTIONS, &connection_configurator) ⇒ NetHTTPClient constructor

  A new instance of NetHTTPClient.

Constructor Details

#initialize(size: self.class::DEFAULT_MAX_CONNECTIONS, &connection_configurator) ⇒ NetHTTPClient

Returns a new instance of NetHTTPClient.

Parameters:

• size (Integer) (defaults to: self.class::DEFAULT_MAX_CONNECTIONS)
• connection_configurator (#call, nil) —

  A block that configures every SDK-created Net::HTTP connection before it is pooled and started.

• size: (Integer) (defaults to: self.class::DEFAULT_MAX_CONNECTIONS)

# File 'lib/openai/net_http_client.rb', line 286

def initialize(size: self.class::DEFAULT_MAX_CONNECTIONS, &connection_configurator)
  super()
  @mutex = Mutex.new
  @size = size
  @cert_store = OpenSSL::X509::Store.new.tap(&:set_default_paths)
  @connection_configurator = connection_configurator
  @pools = {}
end

Instance Method Details

#close ⇒ void

This method returns an undefined value.

Closes current pooled connections. The client remains reusable and will create fresh pools on subsequent requests.

In-flight requests are allowed to finish before their connection closes.

# File 'lib/openai/net_http_client.rb', line 198

def close
  pools =
    @mutex.synchronize do
      current_pools = @pools
      @pools = {}
      current_pools
    end
  pools.each_value do |pool|
    pool.shutdown { |connection| connection.finish if connection.started? }
  end
  nil
end

#execute(request) ⇒ OpenAI::HTTPClient::Response

Executes a request using a pooled Net::HTTP connection.

Parameters:

• request (OpenAI::HTTPClient::Request)

Returns:

• (OpenAI::HTTPClient::Response)

# File 'lib/openai/net_http_client.rb', line 215

def execute(request)
  url = request.url
  deadline = request.timeout&.then { OpenAI::Internal::Util.monotonic_secs + _1 }

  req = nil
  finished = false

  # rubocop:disable Metrics/BlockLength
  enum = Enumerator.new do |y|
    next if finished

    with_pool(url, deadline: deadline) do |conn|
      eof = false
      closing = nil
      ::Thread.handle_interrupt(Object => :never) do
        ::Thread.handle_interrupt(Object => :immediate) do
          req, closing = build_request(request) do
            calibrate_socket_timeout(conn, deadline)
          end

          calibrate_socket_timeout(conn, deadline)
          conn.start unless conn.started?

          calibrate_socket_timeout(conn, deadline)
          ::Kernel.catch(:jump) do
            conn.request(req) do |rsp|
              y << [req, rsp]
              ::Kernel.throw(:jump) if finished

              rsp.read_body do |bytes|
                y << bytes.force_encoding(Encoding::BINARY)
                ::Kernel.throw(:jump) if finished

                calibrate_socket_timeout(conn, deadline)
              end
              eof = true
            end
          end
        end
      ensure
        begin
          conn.finish if !eof && conn&.started?
        ensure
          closing&.call
        end
      end
    end
  rescue ConnectionConfigurationError => e
    raise e.original, cause: e.original.cause
  rescue Timeout::Error
    raise OpenAI::Errors::APITimeoutError.new(url: url, request: req)
  rescue *NETWORK_ERRORS
    raise OpenAI::Errors::APIConnectionError.new(url: url, request: req)
  end
  # rubocop:enable Metrics/BlockLength

  _, response = enum.next
  body = OpenAI::Internal::Util.fused_enum(enum, external: true) do
    finished = true
    loop { enum.next }
  end
  OpenAI::HTTPClient::Response.new(
    status: Integer(response.code),
    headers: response.each_header.to_h,
    body: body
  )
end