Class: RubyPi::LLM::Fallback

Inherits:
BaseProvider show all
Defined in:
lib/ruby_pi/llm/fallback.rb

Overview

A resilient provider wrapper that tries a primary provider first and automatically falls back to an alternative provider on failure. Both providers must conform to the BaseProvider interface.

Authentication errors are NOT retried with the fallback since they indicate a configuration problem rather than a transient failure.

Issue #23 + Issue #12: When streaming, events flow from the primary provider directly to the consumer in real time (no buffering), preserving the streaming UX on the happy path. If the primary fails mid-stream, a :fallback_start StreamEvent is emitted before the fallback takes over, so the consumer can discard any partial output already rendered from the failed primary. (The agent loop translates :fallback_start into a :provider_fallback event; raw Fallback consumers should handle :fallback_start themselves.)

Examples:

Setting up a fallback chain

primary  = RubyPi::LLM.model(:gemini, "gemini-2.0-flash")
backup   = RubyPi::LLM.model(:openai, "gpt-4o")
provider = RubyPi::LLM::Fallback.new(primary: primary, fallback: backup)

# If Gemini fails, automatically retries with OpenAI
response = provider.complete(messages: messages)

Instance Attribute Summary collapse

Attributes inherited from BaseProvider

#max_retries, #retry_base_delay, #retry_max_delay

Instance Method Summary collapse

Constructor Details

#initialize(primary:, fallback:, **options) ⇒ Fallback

Creates a new Fallback wrapper with a primary and fallback provider.

Parameters:



47
48
49
50
51
 
# File 'lib/ruby_pi/llm/fallback.rb', line 47

def initialize(primary:, fallback:, **options)
  super(**options)
  @primary = primary
  @fallback = fallback
end

Instance Attribute Details

#fallback ⇒ RubyPi::LLM::BaseProvider (readonly)

Returns the fallback provider.

Returns:



40
41
42
 
# File 'lib/ruby_pi/llm/fallback.rb', line 40

def fallback
  @fallback
end

#primary ⇒ RubyPi::LLM::BaseProvider (readonly)

Returns the primary provider.

Returns:



37
38
39
 
# File 'lib/ruby_pi/llm/fallback.rb', line 37

def primary
  @primary
end

Instance Method Details

#complete(messages:, tools: [], stream: false) {|event| ... } ⇒ RubyPi::LLM::Response

Overrides BaseProvider#complete to skip the outer retry wrapper.

Without this override, Fallback inherits BaseProvider#complete which wraps perform_complete in a retry loop. Since perform_complete calls (also with retries), the retry layers compose multiplicatively:

outer_retries x (primary_retries + fallback_retries)

With default max_retries=3, that’s 4 x (4 + 4) = 32 total attempts instead of the expected 4 + 4 = 8.

This override calls perform_complete directly — no outer retry loop. Each inner provider handles its own retries independently.

Parameters:

  • messages (Array<Hash>) —

    conversation messages

  • tools (Array<Hash>) (defaults to: []) —

    tool/function definitions

  • stream (Boolean) (defaults to: false) —

    whether to enable streaming mode

Yields:

  • (event) —

    yields StreamEvent objects when streaming

Returns:



85
86
87
 
# File 'lib/ruby_pi/llm/fallback.rb', line 85

def complete(messages:, tools: [], stream: false, &block)
  perform_complete(messages: messages, tools: tools, stream: stream, &block)
end

#model_name ⇒ String

Returns the model name of the primary provider.

Returns:

  • (String) 


56
57
58
 
# File 'lib/ruby_pi/llm/fallback.rb', line 56

def model_name
  @primary.model_name
end

#provider_name ⇒ Symbol

Returns :fallback as the provider identifier.

Returns:

  • (Symbol) 


63
64
65
 
# File 'lib/ruby_pi/llm/fallback.rb', line 63

def provider_name
  :fallback
end