Class: Legion::Extensions::Llm::Openai::OpenaiCallable

Inherits:
Object
  • Object
show all
Defined in:
lib/legion/extensions/llm/openai/openai_callable.rb

Overview

Callable wrapper for an OpenAI provider instance. Implements the fleet dispatch operations by delegating to the per-instance Openai::Provider (errors propagate so normalize_dispatch_error can classify them), plus the disconnect and normalize_dispatch_error (error:) contracts required by Inventory::CallableHandle and Routing::ProviderOutcome.

Defined in its own file so the actor runtime guard in discovery_refresh.rb does not prevent specs from loading it.

Instance Method Summary collapse

Constructor Details

#initialize(instance_cfg:, logger:, provider: nil) ⇒ OpenaiCallable

Returns a new instance of OpenaiCallable.



20
21
22
23
24
25
 
# File 'lib/legion/extensions/llm/openai/openai_callable.rb', line 20

def initialize(instance_cfg:, logger:, provider: nil)
  @instance_cfg = instance_cfg
  @logger = logger
  @injected_provider = provider
  @disconnected = false
end

Instance Method Details

#chat(messages:, model:, **rest) ⇒ Object

-- Fleet dispatch operations ---------------------------------------- The fleet passes model: as a raw string (the offering's model id). chat/stream_chat render paths call model.id (maybe_normalize_ temperature, render_payload), so a Model::Info is required there; embed/count_tokens/image/moderate accept the value verbatim (the image and moderation render paths embed model directly in the wire payload, so wrapping those would corrupt the request body).



49
50
51
 
# File 'lib/legion/extensions/llm/openai/openai_callable.rb', line 49

def chat(messages:, model:, **rest)
  provider.chat(messages: messages, model: to_model_info(model), **rest)
end

#count_tokens(messages:, model:, **rest) ⇒ Object 



61
62
63
 
# File 'lib/legion/extensions/llm/openai/openai_callable.rb', line 61

def count_tokens(messages:, model:, **rest)
  provider.count_tokens(messages: messages, model: model, **rest)
end

#disconnect ⇒ Object 



31
32
33
34
35
 
# File 'lib/legion/extensions/llm/openai/openai_callable.rb', line 31

def disconnect
  @disconnected = true
  @provider&.disconnect
  @logger.debug { '[openai][callable] disconnected' }
end

#disconnected? ⇒ Boolean

Returns:

  • (Boolean) 


27
28
29
 
# File 'lib/legion/extensions/llm/openai/openai_callable.rb', line 27

def disconnected?
  @disconnected
end

#embed(text:, model:, **rest) ⇒ Object 



57
58
59
 
# File 'lib/legion/extensions/llm/openai/openai_callable.rb', line 57

def embed(text:, model:, **rest)
  provider.embed(text: text, model: model, **rest)
end

#image(prompt:, model:, **rest) ⇒ Object 



65
66
67
 
# File 'lib/legion/extensions/llm/openai/openai_callable.rb', line 65

def image(prompt:, model:, **rest)
  provider.image(prompt: prompt, model: model, **rest)
end

#moderate(input, model:, **rest) ⇒ Object 



69
70
71
 
# File 'lib/legion/extensions/llm/openai/openai_callable.rb', line 69

def moderate(input, model:, **rest)
  provider.moderate(input, model: model, **rest)
end

#normalize_dispatch_error(error:) ⇒ Object 



73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
 
# File 'lib/legion/extensions/llm/openai/openai_callable.rb', line 73

def normalize_dispatch_error(error:)
  reason = error.message.to_s[0, 512]

  kind = case error
         when Faraday::ConnectionFailed
           :connection_failure
         when Faraday::TimeoutError
           :timeout
         when Faraday::ClientError
           classify_client_error(error: error)
         when Faraday::ServerError
           classify_server_error(error: error)
         when Legion::Extensions::Llm::OverloadedError
           :overloaded
         else
           # ServiceUnavailableError and all other error types map to
           # :provider_error. OpenAI does not produce a distinct flat
           # instance-down signal separate from overload; only an
           # authoritative explicit instance_unavailable condition
           # (which OpenAI does not emit via normal dispatch) would
           # map to :instance_unavailable.
           :provider_error
         end

  Legion::Extensions::Llm::Routing::ProviderOutcome.new(
    kind: kind,
    reason: reason.empty? ? 'unknown dispatch error' : reason
  )
end

#provider ⇒ Object 



37
38
39
 
# File 'lib/legion/extensions/llm/openai/openai_callable.rb', line 37

def provider
  @provider ||= @injected_provider || Legion::Extensions::Llm::Openai::Provider.new(@instance_cfg)
end

#stream_chat(messages:, model:, **rest) ⇒ Object 



53
54
55
 
# File 'lib/legion/extensions/llm/openai/openai_callable.rb', line 53

def stream_chat(messages:, model:, **rest, &)
  provider.stream_chat(messages: messages, model: to_model_info(model), **rest, &)
end