Class: OllamaAgent::Providers::CredentialRouter

Inherits:
Object
  • Object
show all
Defined in:
lib/ollama_agent/providers/credential_router.rb

Overview

Quota-aware failover dispatcher.

Picks credentials from a CredentialPool, builds the matching provider client via a provider_builder lambda, executes the chat call, and handles typed errors by marking the credential and retrying with the next one.

This is where reactive failover lives:

1. Pick next available credential (pool decides, weighted RR)
2. Build a transient provider client for that credential
3. Execute #chat
4a. On success → mark credential healthy, record usage, return response
4b. On AuthenticationError → permanently disable credential, STOP (don't retry)
4c. On quota/rate/temp error → mark cooldown, try next credential
5. Raise NoAvailableCredentialError when all attempts exhausted

Composes cleanly with the existing Providers::Router — the Router chooses between provider types (OpenAI vs Groq vs Ollama), while the CredentialRouter handles multiple keys for a single provider type.

Examples:

pool    = CredentialPool.new(credentials: [cred_a, cred_b])
builder = ->(cred) { Providers::OpenAI.new(api_key: cred.api_key) }
router  = CredentialRouter.new(pool: pool, provider_builder: builder)
response = router.chat(messages: [...], model: "gpt-4o")

Constant Summary collapse

MAX_ATTEMPTS = 
5

Instance Method Summary collapse

Constructor Details

#initialize(pool:, provider_builder:, health_monitor: nil) ⇒ CredentialRouter

Returns a new instance of CredentialRouter.



36
37
38
39
40
 
# File 'lib/ollama_agent/providers/credential_router.rb', line 36

def initialize(pool:, provider_builder:, health_monitor: nil)
  @pool             = pool
  @provider_builder = provider_builder
  @health_monitor   = health_monitor || HealthMonitor.new
end

Instance Method Details

#aggregate_usage ⇒ Hash

Aggregate usage across all credentials.

Returns:

  • (Hash) 


105
106
107
 
# File 'lib/ollama_agent/providers/credential_router.rb', line 105

def aggregate_usage
  @pool.aggregate_usage
end

#available? ⇒ Boolean

Returns:

  • (Boolean) 


93
94
95
 
# File 'lib/ollama_agent/providers/credential_router.rb', line 93

def available?
  @pool.any_available?
end

#chat(**args) ⇒ Base::Response

Execute a chat request with automatic failover across credentials.

rubocop:disable Metrics/MethodLength – failover loop requires full visibility

Parameters:

  • args (Hash) —

    forwarded to provider#chat (messages:, model:, …)

Returns:

Raises:



49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
 
# File 'lib/ollama_agent/providers/credential_router.rb', line 49

def chat(**args)
  attempts  = 0
  last_cred = nil

  loop do
    if attempts >= MAX_ATTEMPTS
      raise OllamaAgent::NoAvailableCredentialError,
            "All #{MAX_ATTEMPTS} credential attempts failed"
    end

    credential = @pool.next_credential
    provider   = @provider_builder.call(credential)
    started_at = Time.now

    begin
      response = provider.chat(**args)
      latency  = ((Time.now - started_at) * 1000).round

      credential.mark_success!(usage: response.usage)
      @health_monitor.record_success(credential, latency_ms: latency)

      return response
    rescue OllamaAgent::Error, StandardError => e
      typed = ErrorClassifier.classify(e)
      credential.mark_failure!(typed)
      @health_monitor.record_failure(credential, typed)

      # AuthenticationError → permanent disable, bubble up immediately
      raise typed if typed.is_a?(OllamaAgent::AuthenticationError)

      @health_monitor.record_switch(last_cred, credential, typed.class.name) if last_cred && last_cred != credential

      # If not retryable with another credential, bubble up
      raise typed unless ErrorClassifier.retryable_with_other_credential?(typed)

      last_cred = credential
      attempts += 1
      # Loop continues — picks next credential from pool
    end
  end
end

#first_available_key(provider) ⇒ String?

Find the first available API key for a given provider in the pool.

Parameters:

  • provider (String)

Returns:

  • (String, nil) 


125
126
127
 
# File 'lib/ollama_agent/providers/credential_router.rb', line 125

def first_available_key(provider)
  @pool.respond_to?(:first_available_key) ? @pool.first_available_key(provider) : nil
end

#near_exhaustion_warnings ⇒ Array<String>

Ids of near-exhaustion credentials for warnings.

Returns:

  • (Array<String>) 


118
119
120
 
# File 'lib/ollama_agent/providers/credential_router.rb', line 118

def near_exhaustion_warnings
  @pool.near_exhaustion_ids
end

#pool_status ⇒ Array<Hash>

Full pool status snapshot for TUI.

Returns:

  • (Array<Hash>) 


99
100
101
 
# File 'lib/ollama_agent/providers/credential_router.rb', line 99

def pool_status
  @pool.all_status
end

#routing_decisions(n = 10) ⇒ Array<String>

Recent routing decisions for the TUI decisions panel.

Parameters:

  • n (Integer) (defaults to: 10)

Returns:

  • (Array<String>) 


112
113
114
 
# File 'lib/ollama_agent/providers/credential_router.rb', line 112

def routing_decisions(n = 10)
  @health_monitor.routing_decisions(n)
end