Module: LlmConductor::Clients::Concerns::Retryable

Included in:: BaseClient

Defined in:: lib/llm_conductor/clients/concerns/retryable.rb

Overview

Shared retry policy for provider API calls.

LlmConductor::Configuration has always declared max_retries and retry_delay; this concern is what makes them mean something. It is mixed into BaseClient, so every provider goes through the same policy and the two configuration knobs behave identically everywhere.

Only failures that a second identical request can plausibly fix are retried: the rate-limit and overload statuses in RETRYABLE_STATUSES, plus the connection/read timeout classes in TRANSPORT_ERRORS. Everything else — notably 400/401/403/404 and any other 4xx — surfaces on the first attempt, because retrying a malformed request or a bad credential only wastes time.

Backoff is exponential and seeded from retry_delay:

delay = min(retry_delay * 2**(attempt - 1), MAX_BACKOFF_SECONDS)
delay += delay * JITTER_RATIO * rand

The jitter is additive, so a retry never fires sooner than configured, but concurrent workers hitting the same shared quota pool stop retrying in lockstep. A Retry-After header, when the API sends one, replaces the computed backoff (still jittered, still capped).

Constant Summary collapse

RETRYABLE_STATUSES = Statuses worth a second attempt: rate limiting and transient overload.

[408, 429, 500, 502, 503, 504].freeze

TRANSPORT_ERRORS = Transport-level failures, which carry no HTTP status at all.

[
  Faraday::ConnectionFailed,
  Faraday::TimeoutError,
  Faraday::NilStatusError,
  Net::OpenTimeout,
  Net::ReadTimeout,
  Errno::ECONNRESET,
  Errno::ECONNREFUSED
].freeze

MAX_BACKOFF_SECONDS = Ceiling for the computed exponential backoff, so a generous retry_delay cannot park a worker for minutes.

30.0

MAX_RETRY_AFTER_SECONDS = Ceiling for a server-supplied Retry-After. A library should not block its caller for longer than this; past it, failing fast is the better answer.

60.0

JITTER_RATIO = Additive jitter, as a fraction of the delay: 0.25 spreads retries over [delay, delay * 1.25).

0.25

Class Method Summary collapse

.sleep_for(seconds) ⇒ Object
All backoff waiting funnels through here.

Class Method Details

.sleep_for(seconds) ⇒ `Object`

All backoff waiting funnels through here. Keeping it a single module method gives tests one seam to observe instead of stubbing Kernel#sleep on every client, and leaves one place to change should a caller ever need a non-blocking scheduler.



63
64
65