Class: LLM::Transport::PersistentHTTP Private

Inherits:
LLM::Transport show all
Includes:
NetHTTPAdapter
Defined in:
lib/llm/transport/persistent_http.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

The LLM::Transport::PersistentHTTP transport is the built-in adapter for [Net::HTTP::Persistent](github.com/drbrain/net-http-persistent). It manages pooled HTTP connections, tracks active requests by owner, and interrupts in-flight requests when needed.

Defined Under Namespace

Classes: ActiveRequest

Constant Summary collapse

INTERRUPT_ERRORS =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future. 

[::IOError, ::EOFError, Errno::EBADF].freeze

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from LLM::Transport

curb, net_http, net_http_persistent, #set_body_stream

Constructor Details

#initialize(host:, port:, timeout:, ssl:) ⇒ LLM::Transport::PersistentHTTP

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Parameters:

  • host (String)
  • port (Integer)
  • timeout (Integer)
  • ssl (Boolean) 


37
38
39
40
41
42
43
44
 
# File 'lib/llm/transport/persistent_http.rb', line 37

def initialize(host:, port:, timeout:, ssl:)
  @host = host
  @port = port
  @timeout = timeout
  @ssl = ssl
  @base_uri = URI("#{ssl ? "https" : "http"}://#{host}:#{port}/")
  @monitor = Monitor.new
end

Class Method Details

.lock ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



27
28
29
 
# File 'lib/llm/transport/persistent_http.rb', line 27

def self.lock(&)
  @monitor.synchronize(&)
end

.registry ⇒ Hash

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the process-wide connection pool registry.

Returns:

  • (Hash) 


23
24
25
 
# File 'lib/llm/transport/persistent_http.rb', line 23

def self.registry
  @registry
end

Instance Method Details

#interrupt!(owner) ⇒ nil

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Interrupt an active request, if any.

Parameters:

  • owner (Fiber)

Returns:

  • (nil) 


64
65
66
67
68
69
70
71
72
 
# File 'lib/llm/transport/persistent_http.rb', line 64

def interrupt!(owner)
  req = request_for(owner) or return
  lock { (@interrupts ||= {})[owner] = true }
  close_socket(req.connection&.http)
  req.client.finish(req.connection)
  owner.stop if owner.respond_to?(:stop)
rescue *interrupt_errors
  nil
end

#interrupt_errors ⇒ Array<Class<Exception>>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns:

  • (Array<Class<Exception>>) 


56
57
58
 
# File 'lib/llm/transport/persistent_http.rb', line 56

def interrupt_errors
  [*INTERRUPT_ERRORS, *optional_interrupt_errors]
end

#interrupted?(owner) ⇒ Boolean?

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns whether an execution owner was interrupted.

Parameters:

  • owner (Fiber)

Returns:

  • (Boolean, nil) 


78
79
80
 
# File 'lib/llm/transport/persistent_http.rb', line 78

def interrupted?(owner)
  lock { @interrupts&.delete(owner) }
end

#request(request, owner:, stream: nil) {|response| ... } ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Performs a request on the current HTTP transport. Accepts both Net::HTTPRequest and Request.

Parameters:

Yield Parameters:

Returns:



91
92
93
94
95
96
97
98
99
 
# File 'lib/llm/transport/persistent_http.rb', line 91

def request(request, owner:, stream: nil, &b)
  http_req = resolve_request(request)
  client.connection_for(URI.join(base_uri, http_req.path)) do |connection|
    set_request(ActiveRequest.new(client:, connection:), owner)
    perform_request(connection.http, http_req, stream, &b)
  end
ensure
  clear_request(owner)
end

#request_owner ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the current request owner.

Returns:



49
50
51
52
 
# File 'lib/llm/transport/persistent_http.rb', line 49

def request_owner
  return Fiber.current unless defined?(::Async)
  Async::Task.current? ? Async::Task.current : Fiber.current
end