Class: RubyLLM::Agents::Pipeline::Middleware::Reliability

Inherits:
Base
  • Object
show all
Defined in:
lib/ruby_llm/agents/pipeline/middleware/reliability.rb

Overview

Handles retries, fallbacks, and circuit breakers.

This middleware provides reliability features for agent executions:

  • Retries with configurable backoff (constant or exponential)

  • Model fallbacks when primary model fails

  • Circuit breaker protection per model

  • Total timeout across all attempts

Reliability is enabled via the agent’s reliability DSL:

class MyAgent < ApplicationAgent
  reliability do
    retries max: 3, backoff: :exponential
    fallback_models "gpt-4o-mini"
    total_timeout 30
    circuit_breaker errors: 5, within: 60
  end
end

Examples:

Simple retry

class MyEmbedder < RubyLLM::Agents::Embedder
  model "text-embedding-3-small"
  reliability do
    retries max: 2
  end
end

Constant Summary

Constants inherited from Base

Base::LOG_TAG

Instance Method Summary collapse

Methods inherited from Base

#initialize

Constructor Details

This class inherits a constructor from RubyLLM::Agents::Pipeline::Middleware::Base

Instance Method Details

#call(context) ⇒ Context

Process with reliability features

Parameters:

  • context (Context) —

    The execution context

Returns:

  • (Context) —

    The context after execution

Raises:

  • (AllModelsFailedError) —

    If all models fail

  • (TotalTimeoutError) —

    If total timeout exceeded

  • (CircuitOpenError) —

    If circuit breaker is open for all models



41
42
43
44
45
46
47
48
49
50
51
 
# File 'lib/ruby_llm/agents/pipeline/middleware/reliability.rb', line 41

def call(context)
  return @app.call(context) unless reliability_enabled?

  trace(context) do
    config = reliability_config
    models_to_try = build_models_list(context, config)
    total_deadline = calculate_deadline(config)

    execute_with_reliability(context, models_to_try, config, total_deadline)
  end
end