Class: LittleGhost::Providers::OpenAICompatible

Inherits:
Base
  • Object
show all
Defined in:
lib/little_ghost/providers/openai_compatible.rb

Overview

OpenAICompatible brings OpenAI-style Responses or Chat Completions endpoints into LittleGhost. Agents receive the same streaming events whether the endpoint is OpenAI, a hosted model service, or an application gateway.

provider = LittleGhost::Providers::OpenAICompatible.new(
api_key: ENV.fetch("MODEL_API_KEY"),
model: "example-model",
base_url: "https://models.example.test/v1/"
)

The client translates ModelRequest values to the selected wire API and translates responses back to StreamEvent objects.

Retries and streaming output

Transient HTTP and stream failures retry with limited exponential backoff. A :model_retry event reports each retry and whether text had already been emitted. Partial text may repeat after a retry, so consumers that assemble streams must use that event to discard or replace superseded output.

Direct Known Subclasses

OpenAI, OpenRouter

Defined Under Namespace

Classes: ChatNormalizer, Normalizer, ResponsesNormalizer, StreamError

Constant Summary collapse

DEFAULT_BASE_URL =

The OpenAI API endpoint used when base_url is omitted.

"https://api.openai.com/v1/"
INITIAL_RETRY_DELAY =

:nodoc:

1
MAX_RETRY_DELAY =

:nodoc:

16
TRANSIENT_STREAM_ERROR_TYPES = 
%w[
  provider_overloaded provider_unavailable rate_limit_exceeded server timeout
].freeze
CONTEXT_OVERFLOW_MARKERS =

:nodoc:

[
  "context_length_exceeded", "context window", "maximum context length",
  "max context length", "input is too long", "too many input tokens"
].freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Base

#prepare_request

Constructor Details

#initialize(api_key:, model:, base_url: DEFAULT_BASE_URL, api: :responses, headers: {}, open_timeout: 10, read_timeout: 120, allow_insecure_http: false, max_response_bytes: Support::HTTPClient::DEFAULT_MAX_RESPONSE_BYTES, max_retries: 2, max_retry_delay: MAX_RETRY_DELAY, transport: nil, sleeper: nil, on_retry: ->(*) {}) ⇒ OpenAICompatible

Configures an OpenAI-compatible client.

api is :responses or :chat_completions. headers adds trusted endpoint-specific headers. max_retries controls retries before the original error is raised, and on_retry receives the attempt, error, and delay. Pass a custom transport for alternate HTTP execution.

Raises:



88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
 
# File 'lib/little_ghost/providers/openai_compatible.rb', line 88

def initialize(
  api_key:,
  model:,
  base_url: DEFAULT_BASE_URL,
  api: :responses,
  headers: {},
  open_timeout: 10,
  read_timeout: 120,
  allow_insecure_http: false,
  max_response_bytes: Support::HTTPClient::DEFAULT_MAX_RESPONSE_BYTES,
  max_retries: 2,
  max_retry_delay: MAX_RETRY_DELAY,
  transport: nil,
  sleeper: nil,
  on_retry: ->(*) {}
)
  @api_key = api_key
  @model = model
  @api = api.to_sym
  raise ConfigurationError, "api must be :responses or :chat_completions" unless %i[responses chat_completions].include?(@api)

  @headers = headers.transform_keys(&:to_s).freeze
  @max_retries = Integer(max_retries)
  @max_retry_delay = Integer(max_retry_delay)
  @transport = transport || Support::HTTPClient.new(
    base_url:,
    open_timeout:,
    read_timeout:,
    allow_insecure_http:,
    max_response_bytes:
  )
  @sleeper = sleeper
  @on_retry = on_retry
end

Instance Attribute Details

#api ⇒ Object (readonly)

Provider model identifier and selected OpenAI-compatible wire API.



80
81
82
 
# File 'lib/little_ghost/providers/openai_compatible.rb', line 80

def api
  @api
end

#model ⇒ Object (readonly)

Provider model identifier and selected OpenAI-compatible wire API.



80
81
82
 
# File 'lib/little_ghost/providers/openai_compatible.rb', line 80

def model
  @model
end

Class Method Details

.request_options ⇒ Object

Request policy supported by OpenAI-compatible HTTP clients.



30
31
32
 
# File 'lib/little_ghost/providers/openai_compatible.rb', line 30

def self.request_options
  %i[max_response_bytes max_retries max_retry_delay open_timeout read_timeout].freeze
end

Instance Method Details

#capabilities(metadata: {}) ⇒ Object

Returns the permissive capability contract expected from compatible APIs. Subclasses can override this when the endpoint advertises precise support.



165
166
167
 
# File 'lib/little_ghost/providers/openai_compatible.rb', line 165

def capabilities(metadata: {})
  ModelCapabilities.permissive
end

#stream(request) ⇒ Object

Streams LittleGhost StreamEvent objects for request.

Without a block, returns an Enumerator. Context-window errors normalize to ContextWindowOverflowError, and malformed tool calls normalize to MalformedToolCallError.



128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
 
# File 'lib/little_ghost/providers/openai_compatible.rb', line 128

def stream(request)
  return enum_for(__method__, request) unless block_given?

  attempts = 0

  begin
    partial_text = false
    request.cancellation_token.raise_if_cancelled!
    stream_once(request) do |event|
      partial_text ||= event.type == :text_delta && !event.data[:text].to_s.empty?
      yield event
    end
  rescue HTTPError, StreamError => error
    if context_window_overflow?(error)
      raise ContextWindowOverflowError, "The model context window was exceeded"
    end
    raise if !error.retryable? || attempts >= @max_retries

    attempts += 1
    request.cancellation_token.raise_if_cancelled!
    delay = capped_retry_delay(request, retry_delay(attempts))
    @on_retry.call(attempts, error, delay)
    yield StreamEvent.build(
      :model_retry,
      attempt: attempts,
      delay:,
      error_class: error.class.name,
      partial_text:,
      **retry_error_metadata(error)
    )
    wait_before_retry(request, delay)
    retry
  end
end