Class: Phronomy::LLMAdapter::Base

Inherits:
Object
  • Object
show all
Defined in:
lib/phronomy/llm_adapter/base.rb

Overview

Abstract base class for LLM adapters.

Subclasses must implement #complete and #stream. The agent pipeline calls #complete_async / #stream_async which wrap those methods in a BlockingAdapterPool submission.

Direct Known Subclasses

RubyLLM

Instance Method Summary collapse

Instance Method Details

#complete(chat, message, config: {}) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Performs a blocking (non-streaming) LLM completion. Implementors must call chat.ask(message) (or equivalent) and return the response object.

Parameters:

  • chat (Object) —

    the configured chat session object

  • message (String) —

    the user message

  • config (Hash) (defaults to: {}) —

    the invocation config (e.g. :cancellation_token)

Returns:

  • (Object) —

    LLM response object

Raises:

  • (NotImplementedError) 


21
22
23
 
# File 'lib/phronomy/llm_adapter/base.rb', line 21

def complete(chat, message, config: {})
  raise NotImplementedError, "#{self.class}#complete is not implemented"
end

#complete_async(chat, message, config: {}, pool: default_pool) ⇒ BlockingAdapterPool::PendingOperation

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Submits a non-streaming LLM call to BlockingAdapterPool and returns a BlockingAdapterPool::PendingOperation.

Parameters:

  • chat (Object) —

    configured chat session

  • message (String) —

    user message

  • config (Hash) (defaults to: {}) —

    invocation config

  • pool (BlockingAdapterPool) (defaults to: default_pool) —

    pool to submit to

Returns:

  • (BlockingAdapterPool::PendingOperation) 


49
50
51
52
53
54
55
 
# File 'lib/phronomy/llm_adapter/base.rb', line 49

def complete_async(chat, message, config: {}, pool: default_pool)
  token = config[:cancellation_token]
  timeout = config[:llm_timeout]
  pool.submit(timeout: timeout, cancellation_token: token) do
    complete(chat, message, config: config)
  end
end

#stream(chat, message, config: {}) {|chunk| ... } ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Performs a blocking streaming LLM completion. Implementors must call chat.ask(message) { |chunk| block.call(chunk) } (or equivalent) and return the response object.

Parameters:

  • chat (Object) —

    the configured chat session object

  • message (String) —

    the user message

  • config (Hash) (defaults to: {}) —

    the invocation config

Yields:

  • (chunk) —

    streaming chunk from the LLM

Returns:

  • (Object) —

    LLM response object

Raises:

  • (NotImplementedError) 


36
37
38
 
# File 'lib/phronomy/llm_adapter/base.rb', line 36

def stream(chat, message, config: {}, &block)
  raise NotImplementedError, "#{self.class}#stream is not implemented"
end

#stream_async(chat, message, config: {}, pool: default_pool, enqueue_to: nil) {|chunk| ... } ⇒ BlockingAdapterPool::PendingOperation

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Submits a streaming LLM call to BlockingAdapterPool and returns a BlockingAdapterPool::PendingOperation.

When enqueue_to: is given, streaming chunks are pushed into that AsyncQueue from the worker thread instead of being passed directly to the caller's block. The queue is closed (via ensure) after the LLM call finishes so the consumer's drain loop terminates naturally. This keeps user-supplied blocks off the blocking-pool worker thread.

When enqueue_to: is nil and a block is given, the block is invoked directly from the worker thread (legacy behaviour, preserved for backward compatibility).

Parameters:

  • chat (Object) —

    configured chat session

  • message (String) —

    user message

  • config (Hash) (defaults to: {}) —

    invocation config

  • pool (BlockingAdapterPool) (defaults to: default_pool) —

    pool to submit to

  • enqueue_to (AsyncQueue, nil) (defaults to: nil) —

    when set, push chunks here instead of calling the block on the worker thread

Yields:

  • (chunk) —

    streaming chunk — only used when enqueue_to: is nil

Returns:

  • (BlockingAdapterPool::PendingOperation) 


79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
 
# File 'lib/phronomy/llm_adapter/base.rb', line 79

def stream_async(chat, message, config: {}, pool: default_pool, enqueue_to: nil, &block)
  token = config[:cancellation_token]
  timeout = config[:llm_timeout]
  if enqueue_to
    pool.submit(timeout: timeout, cancellation_token: token) do
      stream(chat, message, config: config) do |chunk|
        enqueue_to.push(chunk)
      end
    ensure
      enqueue_to.close
    end
  else
    pool.submit(timeout: timeout, cancellation_token: token) do
      stream(chat, message, config: config, &block)
    end
  end
end