Class: Phronomy::LLMAdapter::Base

Inherits:
Object
  • Object
show all
Defined in:
lib/phronomy/llm_adapter/base.rb

Overview

Beta extension SPI for LLM call adapters.

External adapters implement #complete and #stream. The adapter or the underlying provider client owns transport timeout, retry, backoff, and rate-limit behavior. Phronomy owns cooperative cancellation and isolates synchronous provider calls in OffloadPool.

The Agent pipeline calls the framework-owned #complete_async and #stream_async wrappers. Adapter implementers do not need to depend on EventLoop, FSMSession, OffloadPool, AgentInvocation, or ExecutionCoordinator.

The current input to this SPI is the configured/materialized chat runtime object. Formalizing this SPI therefore does not imply a provider-neutral replacement for RubyLLMMaterializer.

Direct Known Subclasses

RubyLLM

Instance Method Summary collapse

Instance Method Details

#complete(chat, message, config: {}) ⇒ Object

Performs a blocking (non-streaming) LLM completion.

Implementors call the configured chat/runtime client and return its response object. Transport/retry policy remains adapter-owned.

Parameters:

  • chat (Object) —

    the configured/materialized chat runtime object

  • message (String, nil) —

    user message, or nil to continue without adding a new user turn

  • config (Hash) (defaults to: {}) —

    invocation config (e.g. :cancellation_token)

Returns:

  • (Object) —

    LLM response object

Raises:

  • (NotImplementedError) 


33
34
35
 
# File 'lib/phronomy/llm_adapter/base.rb', line 33

def complete(chat, message, config: {})
  raise NotImplementedError, "#{self.class}#complete is not implemented"
end

#complete_async(chat, message, config: {}, pool: default_pool) ⇒ Phronomy::Task

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Submits a non-streaming LLM call to OffloadPool.

Transport timeout and retry remain the responsibility of the adapter or provider client; Phronomy does not attach an additional operation timeout.

Returns:



57
58
59
60
61
62
 
# File 'lib/phronomy/llm_adapter/base.rb', line 57

def complete_async(chat, message, config: {}, pool: default_pool)
  token = config[:cancellation_token]
  pool.submit(cancellation_token: token, on_full: :raise) do
    complete(chat, message, config: config)
  end
end

#stream(chat, message, config: {}) {|chunk| ... } ⇒ Object

Performs a blocking streaming LLM completion.

Parameters:

  • chat (Object) —

    the configured/materialized chat runtime object

  • message (String, nil) —

    user message, or nil to continue without adding a new user turn

  • config (Hash) (defaults to: {}) —

    invocation config

Yields:

  • (chunk) —

    streaming chunk from the LLM

Returns:

  • (Object) —

    LLM response object

Raises:

  • (NotImplementedError) 


46
47
48
 
# File 'lib/phronomy/llm_adapter/base.rb', line 46

def stream(chat, message, config: {}, &block)
  raise NotImplementedError, "#{self.class}#stream is not implemented"
end

#stream_async(chat, message, config: {}, pool: default_pool) {|chunk| ... } ⇒ Phronomy::Task

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Submits a streaming LLM call to OffloadPool.

The block is invoked on an OffloadPool worker thread. Agent code must pass only a lightweight internal sink that posts a value to EventLoop; application callbacks must never be passed directly to this method.

Yields:

  • (chunk) —

    streaming chunk on the worker thread

Returns:

Raises:

  • (ArgumentError) 


73
74
75
76
77
78
79
80
81
82
83
 
# File 'lib/phronomy/llm_adapter/base.rb', line 73

def stream_async(chat, message, config: {}, pool: default_pool, &block)
  raise ArgumentError, "stream_async requires a block" unless block

  token = config[:cancellation_token]
  pool.submit(cancellation_token: token, on_full: :raise) do
    stream(chat, message, config: config) do |chunk|
      token&.raise_if_cancelled!("invocation cancelled during streaming")
      block.call(chunk)
    end
  end
end