Class: RubyPi::LLM::Fallback

Inherits:
BaseProvider show all
Defined in:
lib/ruby_pi/llm/fallback.rb

Overview

A resilient provider wrapper that tries a primary provider first and automatically falls back to an alternative provider on failure. Both providers must conform to the BaseProvider interface.

Authentication errors are NOT retried with the fallback since they indicate a configuration problem rather than a transient failure.

Issue #23: When streaming, the Fallback now buffers deltas from the primary provider. If the primary fails mid-stream, the buffered deltas are discarded and the fallback provider streams fresh from the start. This prevents the consumer from seeing partial output from the primary concatenated with the complete output from the fallback.

Examples:

Setting up a fallback chain

primary  = RubyPi::LLM.model(:gemini, "gemini-2.0-flash")
backup   = RubyPi::LLM.model(:openai, "gpt-4o")
provider = RubyPi::LLM::Fallback.new(primary: primary, fallback: backup)

# If Gemini fails, automatically retries with OpenAI
response = provider.complete(messages: messages)

Instance Attribute Summary collapse

Attributes inherited from BaseProvider

#max_retries, #retry_base_delay, #retry_max_delay

Instance Method Summary collapse

Constructor Details

#initialize(primary:, fallback:, **options) ⇒ Fallback

Creates a new Fallback wrapper with a primary and fallback provider.

Parameters:



44
45
46
47
48
 
# File 'lib/ruby_pi/llm/fallback.rb', line 44

def initialize(primary:, fallback:, **options)
  super(**options)
  @primary = primary
  @fallback = fallback
end

Instance Attribute Details

#fallback ⇒ RubyPi::LLM::BaseProvider (readonly)

Returns the fallback provider.

Returns:



37
38
39
 
# File 'lib/ruby_pi/llm/fallback.rb', line 37

def fallback
  @fallback
end

#primary ⇒ RubyPi::LLM::BaseProvider (readonly)

Returns the primary provider.

Returns:



34
35
36
 
# File 'lib/ruby_pi/llm/fallback.rb', line 34

def primary
  @primary
end

Instance Method Details

#complete(messages:, tools: [], stream: false) {|event| ... } ⇒ RubyPi::LLM::Response

Overrides BaseProvider#complete to skip the outer retry wrapper.

Without this override, Fallback inherits BaseProvider#complete which wraps perform_complete in a retry loop. Since perform_complete calls (also with retries), the retry layers compose multiplicatively:

outer_retries x (primary_retries + fallback_retries)

With default max_retries=3, that’s 4 x (4 + 4) = 32 total attempts instead of the expected 4 + 4 = 8.

This override calls perform_complete directly — no outer retry loop. Each inner provider handles its own retries independently.

Parameters:

  • messages (Array<Hash>) —

    conversation messages

  • tools (Array<Hash>) (defaults to: []) —

    tool/function definitions

  • stream (Boolean) (defaults to: false) —

    whether to enable streaming mode

Yields:

  • (event) —

    yields StreamEvent objects when streaming

Returns:



82
83
84
 
# File 'lib/ruby_pi/llm/fallback.rb', line 82

def complete(messages:, tools: [], stream: false, &block)
  perform_complete(messages: messages, tools: tools, stream: stream, &block)
end

#model_name ⇒ String

Returns the model name of the primary provider.

Returns:

  • (String) 


53
54
55
 
# File 'lib/ruby_pi/llm/fallback.rb', line 53

def model_name
  @primary.model_name
end

#provider_name ⇒ Symbol

Returns :fallback as the provider identifier.

Returns:

  • (Symbol) 


60
61
62
 
# File 'lib/ruby_pi/llm/fallback.rb', line 60

def provider_name
  :fallback
end