Class: RubyPi::LLM::BaseProvider

Inherits:
Object
  • Object
show all
Defined in:
lib/ruby_pi/llm/base_provider.rb

Overview

Abstract base class that defines the contract every LLM provider must fulfill. Provides built-in retry logic with exponential backoff for transient errors and a unified #complete interface for both synchronous and streaming completions.

Subclasses MUST implement:

  • #perform_complete(messages:, tools:, stream:, &block)
  • #model_name
  • #provider_name

Examples:

Subclass implementation

class MyProvider < RubyPi::LLM::BaseProvider
  def model_name = "my-model"
  def provider_name = :my_provider

  private
  def perform_complete(messages:, tools:, stream:, &block)
    # Make HTTP request and return RubyPi::LLM::Response
  end
end

Direct Known Subclasses

Anthropic, Fallback, Gemini, OpenAI

Constant Summary collapse

MAX_STREAM_OUTPUT_BYTES = 
16 * 1024 * 1024
MAX_STREAM_TOOL_ARGUMENT_BYTES = 
1024 * 1024
MAX_STREAM_TOOL_CALLS = 
128
MAX_ERROR_BODY_BYTES = 
64 * 1024
RETRY_AFTER_CEILING = 
60.0

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(config: nil, max_retries: nil, retry_base_delay: nil, retry_max_delay: nil) ⇒ BaseProvider

Initializes the base provider with retry configuration.

Parameters:

  • config (RubyPi::Configuration, nil) (defaults to: nil) —

    optional per-agent config override. When provided, the provider uses this config instead of the global RubyPi.configuration singleton. This enables per-agent API keys, timeouts, and retry settings.

  • max_retries (Integer, nil) (defaults to: nil) —

    override max retries (defaults to config)

  • retry_base_delay (Float, nil) (defaults to: nil) —

    override base delay (defaults to config)

  • retry_max_delay (Float, nil) (defaults to: nil) —

    override max delay (defaults to config)



59
60
61
62
63
64
65
66
 
# File 'lib/ruby_pi/llm/base_provider.rb', line 59

def initialize(config: nil, max_retries: nil, retry_base_delay: nil, retry_max_delay: nil)
  @config = config || RubyPi.configuration
  @max_retries = max_retries || @config.max_retries
  @retry_base_delay = retry_base_delay || @config.retry_base_delay
  @retry_max_delay = retry_max_delay || @config.retry_max_delay

  validate_retry_configuration!
end

Instance Attribute Details

#max_retries ⇒ Integer (readonly)

Returns maximum number of retry attempts.

Returns:

  • (Integer) —

    maximum number of retry attempts



42
43
44
 
# File 'lib/ruby_pi/llm/base_provider.rb', line 42

def max_retries
  @max_retries
end

#retry_base_delay ⇒ Float (readonly)

Returns base delay in seconds for exponential backoff.

Returns:

  • (Float) —

    base delay in seconds for exponential backoff



45
46
47
 
# File 'lib/ruby_pi/llm/base_provider.rb', line 45

def retry_base_delay
  @retry_base_delay
end

#retry_max_delay ⇒ Float (readonly)

Returns maximum delay in seconds between retries.

Returns:

  • (Float) —

    maximum delay in seconds between retries



48
49
50
 
# File 'lib/ruby_pi/llm/base_provider.rb', line 48

def retry_max_delay
  @retry_max_delay
end

Instance Method Details

#complete(messages:, tools: [], stream: false) {|event| ... } ⇒ RubyPi::LLM::Response

Sends a completion request to the LLM provider with automatic retry logic for transient errors. When stream is true and a block is given, yields StreamEvent objects incrementally as they arrive.

Parameters:

  • messages (Array<Hash>) —

    conversation messages, each with :role and :content

  • tools (Array<Hash>) (defaults to: []) —

    tool/function definitions for the model

  • stream (Boolean) (defaults to: false) —

    whether to enable streaming mode

Yields:

  • (event) —

    yields StreamEvent objects when streaming

Yield Parameters:

Returns:

Raises:



82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
 
# File 'lib/ruby_pi/llm/base_provider.rb', line 82

def complete(messages:, tools: [], stream: false, &block)
  attempt = 0
  partial_chars = 0
  partial_tool_calls = false

  attempt_block = if stream && block
                    proc do |event|
                      partial_chars += event.data.to_s.length if event.text_delta?
                      partial_tool_calls = true if event.tool_call_delta?
                      block.call(event)
                    end
                  else
                    block
                  end

  begin
    attempt += 1
    perform_complete(messages: messages, tools: tools, stream: stream, &attempt_block)
  rescue RubyPi::AuthenticationError
    # Authentication errors are not retryable — raise immediately
    raise
  rescue RubyPi::RateLimitError, RubyPi::ApiError, RubyPi::TimeoutError => e
    # NOTE: RubyPi::ProviderError is intentionally NOT retried. Provider
    # errors are overwhelmingly deterministic request-construction
    # failures (missing tool_call_id, invalid tool-argument JSON, missing
    # tool name) raised by build_request_body BEFORE any HTTP call. They
    # produce the identical error on every attempt, so retrying only
    # burns the backoff schedule before surfacing the same failure.
    # Fallback wrappers still rescue RubyPi::Error (the ProviderError
    # superclass), so provider failover is unaffected.
    #
    # Retry up to max_retries times AFTER the initial attempt.
    # With max_retries: 3, attempt goes 1 (initial), 2, 3, 4 — the condition
    # `attempt <= @max_retries` allows retries on attempts 1..3, so we get
    # 3 retries + 1 initial = 4 total attempts. Previously used `< @max_retries`
    # which was off-by-one (only 2 retries with max_retries: 3).
    if retryable_error?(e) && attempt <= @max_retries
      delay = retry_delay_for(e, attempt)
      log_retry(attempt, delay, e)

      if stream && block
        block.call(StreamEvent.new(type: :retry_start, data: {
          provider: provider_name,
          error: e.message,
          attempt: attempt + 1,
          partial_output: partial_chars.positive? || partial_tool_calls,
          partial_chars: partial_chars,
          partial_tool_calls: partial_tool_calls
        }))
        partial_chars = 0
        partial_tool_calls = false
      end

      sleep(delay)
      retry
    else
      raise
    end
  end
end

#model_name ⇒ String

Returns the model name used by this provider instance. Subclasses MUST override this method.

Returns:

  • (String) —

    the model identifier

Raises:



148
149
150
 
# File 'lib/ruby_pi/llm/base_provider.rb', line 148

def model_name
  raise RubyPi::AbstractMethodError, :model_name
end

#provider_name ⇒ Symbol

Returns the provider identifier. Subclasses MUST override this method.

Returns:

  • (Symbol) —

    the provider identifier (e.g., :gemini, :anthropic, :openai)

Raises:



157
158
159
 
# File 'lib/ruby_pi/llm/base_provider.rb', line 157

def provider_name
  raise RubyPi::AbstractMethodError, :provider_name
end