Class: LLM::Transport::PersistentHTTP Private

Inherits:
LLM::Transport show all
Defined in:
lib/llm/transport/persistent_http.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

The LLM::Transport::PersistentHTTP transport is the built-in adapter for [Net::HTTP::Persistent](github.com/drbrain/net-http-persistent). It manages pooled HTTP connections, tracks active requests by owner, and interrupts in-flight requests when needed.

Defined Under Namespace

Classes: Request

Constant Summary collapse

INTERRUPT_ERRORS =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future. 

[::IOError, ::EOFError, Errno::EBADF].freeze

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from LLM::Transport

net_http, net_http_persistent, #set_body_stream

Constructor Details

#initialize(host:, port:, timeout:, ssl:) ⇒ LLM::Transport::PersistentHTTP

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Parameters:

  • host (String)
  • port (Integer)
  • timeout (Integer)
  • ssl (Boolean) 


35
36
37
38
39
40
41
42
 
# File 'lib/llm/transport/persistent_http.rb', line 35

def initialize(host:, port:, timeout:, ssl:)
  @host = host
  @port = port
  @timeout = timeout
  @ssl = ssl
  @base_uri = URI("#{ssl ? "https" : "http"}://#{host}:#{port}/")
  @monitor = Monitor.new
end

Class Method Details

.lock ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



25
26
27
 
# File 'lib/llm/transport/persistent_http.rb', line 25

def self.lock(&)
  @monitor.synchronize(&)
end

.registry ⇒ Hash

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the process-wide connection pool registry.

Returns:

  • (Hash) 


21
22
23
 
# File 'lib/llm/transport/persistent_http.rb', line 21

def self.registry
  @registry
end

Instance Method Details

#interrupt!(owner) ⇒ nil

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Interrupt an active request, if any.

Parameters:

  • owner (Fiber)

Returns:

  • (nil) 


62
63
64
65
66
67
68
69
70
 
# File 'lib/llm/transport/persistent_http.rb', line 62

def interrupt!(owner)
  req = request_for(owner) or return
  lock { (@interrupts ||= {})[owner] = true }
  close_socket(req.connection&.http)
  req.client.finish(req.connection)
  owner.stop if owner.respond_to?(:stop)
rescue *interrupt_errors
  nil
end

#interrupt_errors ⇒ Array<Class<Exception>>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns:

  • (Array<Class<Exception>>) 


54
55
56
 
# File 'lib/llm/transport/persistent_http.rb', line 54

def interrupt_errors
  [*INTERRUPT_ERRORS, *optional_interrupt_errors]
end

#interrupted?(owner) ⇒ Boolean?

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns whether an execution owner was interrupted.

Parameters:

  • owner (Fiber)

Returns:

  • (Boolean, nil) 


76
77
78
 
# File 'lib/llm/transport/persistent_http.rb', line 76

def interrupted?(owner)
  lock { @interrupts&.delete(owner) }
end

#request(request, owner:, stream: nil) {|response| ... } ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Performs a request on the current HTTP transport.

Parameters:

  • request (Net::HTTPRequest)
  • owner (Fiber)
  • stream (LLM::Object, nil) (defaults to: nil)

Yield Parameters:

Returns:



87
88
89
90
91
92
93
94
 
# File 'lib/llm/transport/persistent_http.rb', line 87

def request(request, owner:, stream: nil, &b)
  client.connection_for(URI.join(base_uri, request.path)) do |connection|
    set_request(Request.new(client:, connection:), owner)
    perform_request(connection.http, request, stream, &b)
  end
ensure
  clear_request(owner)
end

#request_owner ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the current request owner.

Returns:



47
48
49
50
 
# File 'lib/llm/transport/persistent_http.rb', line 47

def request_owner
  return Fiber.current unless defined?(::Async)
  Async::Task.current? ? Async::Task.current : Fiber.current
end