Class: Phronomy::LLMAdapter::Base

Inherits:
Object
  • Object
show all
Defined in:
lib/phronomy/llm_adapter/base.rb

Overview

Abstract base class for LLM adapters.

Subclasses must implement #complete and #stream. The agent pipeline calls #complete_async / #stream_async which wrap those methods in a BlockingAdapterPool submission.

Direct Known Subclasses

RubyLLM

Instance Method Summary collapse

Instance Method Details

#complete(chat, message, config: {}) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Performs a blocking (non-streaming) LLM completion. Implementors must call +chat.ask(message)+ (or equivalent) and return the response object.

Parameters:

  • chat (Object) —

    the configured chat session object

  • message (String) —

    the user message

  • config (Hash) (defaults to: {}) —

    the invocation config (e.g. +:cancellation_token+)

Returns:

  • (Object) —

    LLM response object

Raises:

  • (NotImplementedError) 


21
22
23
 
# File 'lib/phronomy/llm_adapter/base.rb', line 21

def complete(chat, message, config: {})
  raise NotImplementedError, "#{self.class}#complete is not implemented"
end

#complete_async(chat, message, config: {}, pool: default_pool) ⇒ BlockingAdapterPool::PendingOperation

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Submits a non-streaming LLM call to BlockingAdapterPool and returns a BlockingAdapterPool::PendingOperation.

Parameters:

  • chat (Object) —

    configured chat session

  • message (String) —

    user message

  • config (Hash) (defaults to: {}) —

    invocation config

  • pool (BlockingAdapterPool) (defaults to: default_pool) —

    pool to submit to

Returns:



49
50
51
52
53
54
55
 
# File 'lib/phronomy/llm_adapter/base.rb', line 49

def complete_async(chat, message, config: {}, pool: default_pool)
  token = config[:cancellation_token]
  timeout = config[:llm_timeout]
  pool.submit(timeout: timeout, cancellation_token: token) do
    complete(chat, message, config: config)
  end
end

#stream(chat, message, config: {}) {|chunk| ... } ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Performs a blocking streaming LLM completion. Implementors must call +chat.ask(message) { |chunk| block.call(chunk) }+ (or equivalent) and return the response object.

Parameters:

  • chat (Object) —

    the configured chat session object

  • message (String) —

    the user message

  • config (Hash) (defaults to: {}) —

    the invocation config

Yields:

  • (chunk) —

    streaming chunk from the LLM

Returns:

  • (Object) —

    LLM response object

Raises:

  • (NotImplementedError) 


36
37
38
 
# File 'lib/phronomy/llm_adapter/base.rb', line 36

def stream(chat, message, config: {}, &block)
  raise NotImplementedError, "#{self.class}#stream is not implemented"
end

#stream_async(chat, message, config: {}, pool: default_pool, enqueue_to: nil) {|chunk| ... } ⇒ BlockingAdapterPool::PendingOperation

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Submits a streaming LLM call to BlockingAdapterPool and returns a BlockingAdapterPool::PendingOperation.

When +enqueue_to:+ is given, streaming chunks are pushed into that AsyncQueue from the worker thread instead of being passed directly to the caller's block. The queue is closed (via +ensure+) after the LLM call finishes so the consumer's drain loop terminates naturally. This keeps user-supplied blocks off the blocking-pool worker thread.

When +enqueue_to:+ is nil and a block is given, the block is invoked directly from the worker thread (legacy behaviour, preserved for backward compatibility).

Parameters:

  • chat (Object) —

    configured chat session

  • message (String) —

    user message

  • config (Hash) (defaults to: {}) —

    invocation config

  • pool (BlockingAdapterPool) (defaults to: default_pool) —

    pool to submit to

  • enqueue_to (AsyncQueue, nil) (defaults to: nil) —

    when set, push chunks here instead of calling the block on the worker thread

Yields:

  • (chunk) —

    streaming chunk — only used when +enqueue_to:+ is nil

Returns:



79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
 
# File 'lib/phronomy/llm_adapter/base.rb', line 79

def stream_async(chat, message, config: {}, pool: default_pool, enqueue_to: nil, &block)
  token = config[:cancellation_token]
  timeout = config[:llm_timeout]
  if enqueue_to
    pool.submit(timeout: timeout, cancellation_token: token) do
      stream(chat, message, config: config) do |chunk|
        enqueue_to.push(chunk)
      end
    ensure
      enqueue_to.close
    end
  else
    pool.submit(timeout: timeout, cancellation_token: token) do
      stream(chat, message, config: config, &block)
    end
  end
end