Class: Legion::Extensions::Llm::Provider

Inherits:
Object
  • Object
show all
Includes:
Cache::Helper, Streaming, Logging::Helper
Defined in:
lib/legion/extensions/llm/provider.rb,
lib/legion/extensions/llm/provider/open_ai_compatible.rb

Overview

Base class for LLM providers.

Defined Under Namespace

Modules: OpenAICompatible

Constant Summary collapse

MODEL_DETAIL_CACHE_SCHEMA_VERSION = 
2
CAPABILITY_CONFIG_KEYS = 
%i[
  capabilities
  enable_completion
  enable_embedding
  enable_embeddings
  enable_streaming
  enable_tools
  enable_functions
  enable_function_calling
  enable_thinking
  enable_reasoning
  enable_vision
  enable_structured_output
  enable_moderation
  enable_image
  enable_images
  enable_image_generation
  enable_audio_transcription
  enable_audio_speech
  enable_audio_generation
  completion_flag
  embedding_flag
  embeddings_flag
  streaming_flag
  tool_flag
  tools_flag
  functions_flag
  function_calling_flag
  thinking_flag
  reasoning_flag
  vision_flag
  structured_output_flag
  moderation_flag
  image_flag
  images_flag
  image_generation_flag
  audio_transcription_flag
  audio_speech_flag
  audio_generation_flag
].freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Streaming

build_on_data_handler, build_stream_callback, build_stream_error_response, error_chunk?, faraday_1?, handle_data, handle_error_chunk, handle_error_event, handle_failed_response, handle_json_error_chunk, handle_parsed_error, handle_sse, handle_stream, json_error_payload?, parse_error_from_json, parse_streaming_error, persist_failed_response_body, persist_failed_response_custom_body?, persist_failed_response_env_body?, process_stream_chunk, raise_partial_streaming_error, raise_streaming_status_error, stream_response

Constructor Details

#initialize(config) ⇒ Provider

Returns a new instance of Provider.



81
82
83
84
85
 
# File 'lib/legion/extensions/llm/provider.rb', line 81

def initialize(config)
  @config = config.is_a?(Hash) ? HashConfig.new(config) : config
  ensure_configured!
  @connection = Connection.new(self, @config)
end

Instance Attribute Details

#config ⇒ Object (readonly)

Returns the value of attribute config.



79
80
81
 
# File 'lib/legion/extensions/llm/provider.rb', line 79

def config
  @config
end

#connection ⇒ Object (readonly)

Returns the value of attribute connection.



79
80
81
 
# File 'lib/legion/extensions/llm/provider.rb', line 79

def connection
  @connection
end

Class Method Details

.assume_models_exist? ⇒ Boolean

Returns:

  • (Boolean) 


772
773
774
 
# File 'lib/legion/extensions/llm/provider.rb', line 772

def assume_models_exist?
  false
end

.capabilities ⇒ Object 



744
745
746
 
# File 'lib/legion/extensions/llm/provider.rb', line 744

def capabilities
  nil
end

.configuration_options ⇒ Object 



752
753
754
 
# File 'lib/legion/extensions/llm/provider.rb', line 752

def configuration_options
  []
end

.configuration_requirements ⇒ Object 



748
749
750
 
# File 'lib/legion/extensions/llm/provider.rb', line 748

def configuration_requirements
  []
end

.configured?(config) ⇒ Boolean

Returns:

  • (Boolean) 


780
781
782
 
# File 'lib/legion/extensions/llm/provider.rb', line 780

def configured?(config)
  configuration_requirements.all? { |req| config.send(req) }
end

.default_tier ⇒ Object 



760
761
762
 
# File 'lib/legion/extensions/llm/provider.rb', line 760

def default_tier
  :frontier
end

.default_transport ⇒ Object 



756
757
758
 
# File 'lib/legion/extensions/llm/provider.rb', line 756

def default_transport
  :http
end

.local? ⇒ Boolean

Returns:

  • (Boolean) 


764
765
766
 
# File 'lib/legion/extensions/llm/provider.rb', line 764

def local?
  false
end

.model_policy(config, provider_family) ⇒ Object

Effective whitelist/blacklist for an instance config at build time (before provider instance exists). Same specificity cascade:

  1. Per-instance (config hash — extensions.llm..instances..model_whitelist)
  2. Provider-level (extensions.llm..model_whitelist)
  3. Global (extensions.llm.model_whitelist)


570
571
572
573
574
575
576
577
578
579
580
581
 
# File 'lib/legion/extensions/llm/provider.rb', line 570

def self.model_policy(config, provider_family)
  cfg = config.is_a?(Hash) ? config : {}
  provider_conf = CredentialSources.setting(:extensions, :llm, provider_family)
  provider_conf = {} unless provider_conf.is_a?(Hash)
  global_conf = (::Legion::Settings.dig(:extensions, :llm) if defined?(::Legion::Settings))
  global_conf = {} unless global_conf.is_a?(Hash)

  {
    whitelist: resolve_policy_value(cfg, provider_conf, global_conf, :model_whitelist),
    blacklist: resolve_policy_value(cfg, provider_conf, global_conf, :model_blacklist)
  }
end

.name ⇒ Object 



736
737
738
 
# File 'lib/legion/extensions/llm/provider.rb', line 736

def name
  to_s.split('::').last
end

.policy_allows?(model_name, whitelist: [], blacklist: []) ⇒ Boolean

Single source of truth for model-policy matching, usable both at runtime (instance #model_allowed?) and at instance-config build time (provider extensions choosing a default_model that does not violate the policy). Substring, case-insensitive: a whitelist permits models containing any pattern; a blacklist denies models containing any pattern; whitelist is applied before blacklist. Empty list = no restriction from that side.

Returns:

  • (Boolean) 


554
555
556
557
558
559
560
561
562
563
 
# File 'lib/legion/extensions/llm/provider.rb', line 554

def self.policy_allows?(model_name, whitelist: [], blacklist: [])
  name = model_name.to_s.downcase
  wl = Array(whitelist).map { |p| p.to_s.downcase }
  bl = Array(blacklist).map { |p| p.to_s.downcase }

  return false if wl.any? && wl.none? { |p| name.include?(p) }
  return false if bl.any? && bl.any? { |p| name.include?(p) }

  true
end

.policy_safe_default_model(configured:, fallback:, whitelist: [], blacklist: []) ⇒ Object

Choose a default_model that never violates the model policy: prefer an explicitly-configured default when permitted; else a provider fallback when permitted; else nil, so routing resolves an allowed discovered model rather than forcing a policy-forbidden default. Keeps a whitelist/blacklist authoritative over any hardcoded provider default.



600
601
602
603
604
605
606
607
 
# File 'lib/legion/extensions/llm/provider.rb', line 600

def self.policy_safe_default_model(configured:, fallback:, whitelist: [], blacklist: [])
  return configured if configured && !configured.to_s.empty? &&
                       policy_allows?(configured, whitelist:, blacklist:)
  return fallback if fallback && !fallback.to_s.empty? &&
                     policy_allows?(fallback, whitelist:, blacklist:)

  nil
end

.remote? ⇒ Boolean

Returns:

  • (Boolean) 


768
769
770
 
# File 'lib/legion/extensions/llm/provider.rb', line 768

def remote?
  !local?
end

.resolve_model_id(model_id, config: nil) ⇒ Object

rubocop:disable Lint/UnusedMethodArgument



776
777
778
 
# File 'lib/legion/extensions/llm/provider.rb', line 776

def resolve_model_id(model_id, config: nil) # rubocop:disable Lint/UnusedMethodArgument
  model_id
end

.resolve_policy_value(cfg, provider_conf, global_conf, key) ⇒ Object

Resolve a single policy value with the shared cascade (SettingsCascade: instance > provider, model leg skipped — no model exists at policy-build time) plus the legacy global extensions.llm. leg. Empty values fall through.



587
588
589
590
591
592
593
 
# File 'lib/legion/extensions/llm/provider.rb', line 587

def self.resolve_policy_value(cfg, provider_conf, global_conf, key)
  value = SettingsCascade.resolve_value(provider_conf: provider_conf, instance_cfg: cfg, key: key)
  return value unless value.nil?

  global_conf = {} unless global_conf.is_a?(::Hash)
  global_conf[key] || global_conf[key.to_s]
end

.slug ⇒ Object 



740
741
742
 
# File 'lib/legion/extensions/llm/provider.rb', line 740

def slug
  name.downcase
end

Instance Method Details

#api_base ⇒ Object

Raises:

  • (NotImplementedError) 


92
93
94
 
# File 'lib/legion/extensions/llm/provider.rb', line 92

def api_base
  raise NotImplementedError
end

#assume_models_exist? ⇒ Boolean

Returns:

  • (Boolean) 


386
387
388
 
# File 'lib/legion/extensions/llm/provider.rb', line 386

def assume_models_exist?
  self.class.assume_models_exist?
end

#cache_control_prefix_tokens ⇒ Object 



370
371
372
373
374
375
376
 
# File 'lib/legion/extensions/llm/provider.rb', line 370

def cache_control_prefix_tokens
  if config.respond_to?(:cache_control_prefix_tokens) && config.cache_control_prefix_tokens
    config.cache_control_prefix_tokens
  else
    4
  end
end

#cache_enabled? ⇒ Boolean

Returns:

  • (Boolean) 


357
358
359
360
361
362
363
364
365
366
367
368
 
# File 'lib/legion/extensions/llm/provider.rb', line 357

def cache_enabled?
  explicit = config.llm_cache_enabled if config.respond_to?(:llm_cache_enabled)

  unless explicit.nil?
    log.debug { "[#{slug}] cache_enabled? source=per_provider value=#{explicit}" }
    return explicit == true
  end

  global = global_prompt_caching_enabled?
  log.debug { "[#{slug}] cache_enabled? source=global value=#{global}" }
  global
end

#cache_instance_key ⇒ Object 



719
720
721
722
723
724
725
726
727
 
# File 'lib/legion/extensions/llm/provider.rb', line 719

def cache_instance_key
  if cache_local_instance?
    (respond_to?(:instance_id) ? instance_id : :default).to_s
  else
    require 'digest'
    urls = Array(config_base_url).map { |u| strip_scheme(u).downcase.chomp('/') }.sort
    Digest::SHA256.hexdigest(urls.join('|'))[0, 12]
  end
end

#cache_local_instance? ⇒ Boolean

── Cache helpers with local/shared tier selection ────────────────

Returns:

  • (Boolean) 


683
684
685
686
687
688
 
# File 'lib/legion/extensions/llm/provider.rb', line 683

def cache_local_instance?
  Array(config_base_url).any? do |url|
    host = url.to_s.downcase
    host.include?('localhost') || host.include?('127.0.0.1') || host.include?('::1')
  end
end

#capabilities ⇒ Object 



127
128
129
 
# File 'lib/legion/extensions/llm/provider.rb', line 127

def capabilities
  self.class.capabilities
end

#chat(messages:, model:, tools: [], temperature: nil, params: {}, headers: {}, schema: nil, thinking: nil, tool_prefs: nil) ⇒ Object

rubocop:disable Metrics/ParameterLists



136
137
138
139
 
# File 'lib/legion/extensions/llm/provider.rb', line 136

def chat(messages:, model:, tools: [], temperature: nil, params: {}, headers: {}, schema: nil, thinking: nil,
         tool_prefs: nil)
  complete(messages, tools:, temperature:, model:, params:, headers:, schema:, thinking:, tool_prefs:)
end

#complete(messages, tools:, temperature:, model:, params: {}, headers: {}, schema: nil, thinking: nil, tool_prefs: nil) ⇒ Object 



146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
 
# File 'lib/legion/extensions/llm/provider.rb', line 146

def complete(messages, tools:, temperature:, model:, params: {}, headers: {}, schema: nil, thinking: nil,
             tool_prefs: nil, &)
  enforce_model_allowed!(model)
  normalized_temperature = maybe_normalize_temperature(temperature, model)
  log_provider_request(
    messages: messages,
    tools: tools,
    temperature: temperature,
    normalized_temperature: normalized_temperature,
    model: model,
    params: params,
    headers: headers,
    schema: schema,
    thinking: thinking,
    tool_prefs: tool_prefs,
    streaming: block_given?
  )

  payload = Utils.deep_merge(
    render_payload(
      messages,
      tools: tools,
      tool_prefs: tool_prefs,
      temperature: normalized_temperature,
      model: model,
      stream: block_given?,
      schema: schema,
      thinking: thinking
    ),
    params
  )

  if block_given?
    stream_response @connection, payload, headers, &
  else
    sync_response @connection, payload, headers
  end
end

#config_base_url ⇒ Object 



641
642
643
 
# File 'lib/legion/extensions/llm/provider.rb', line 641

def config_base_url
  respond_to?(:settings) ? settings[:base_url] : nil
end

#configuration_requirements ⇒ Object 



131
132
133
 
# File 'lib/legion/extensions/llm/provider.rb', line 131

def configuration_requirements
  self.class.configuration_requirements
end

#configured? ⇒ Boolean

Returns:

  • (Boolean) 


353
354
355
 
# File 'lib/legion/extensions/llm/provider.rb', line 353

def configured?
  configuration_requirements.all? { |req| @config.send(req) }
end

#count_tokens(messages:, model:, params: {}) ⇒ Object 



295
296
297
298
299
300
301
 
# File 'lib/legion/extensions/llm/provider.rb', line 295

def count_tokens(messages:, model:, params: {})
  _ = [model, params]
  Array(messages).sum do |message|
    content = message.respond_to?(:content) ? message.content : message[:content] || message['content']
    estimate_text_tokens(content)
  end
end

#disconnect ⇒ Object 



87
88
89
90
 
# File 'lib/legion/extensions/llm/provider.rb', line 87

def disconnect
  @connection&.close
  @connection = nil
end

#discover_offerings(live: false, raise_on_unreachable: false, **filters) ⇒ Object 



192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
 
# File 'lib/legion/extensions/llm/provider.rb', line 192

def discover_offerings(live: false, raise_on_unreachable: false, **filters)
  return filter_cached_offerings(Array(@cached_offerings), filters) unless live

  provider_health = health(live:)
  @cached_offerings = Array(list_models(live:, **filters)).filter_map do |model|
    publish_discovered_model_to_registry(model, provider_health:, live:)
    next unless model_matches_filters?(model, filters)
    next unless model_allowed?(model.id)

    log.debug("[#{slug}] instance=#{provider_instance_id} action=model_discovered model=#{model.id} family=#{model.family}")
    offering_from_model(model, health: provider_health)
  end
  log.info("[#{slug}] instance=#{provider_instance_id} action=discover_complete model_count=#{Array(@cached_offerings).size}")
  @cached_offerings
rescue Faraday::ConnectionFailed, Faraday::TimeoutError => e
  log.warn("[#{slug}] instance=#{provider_instance_id} unreachable: #{e.message}")
  raise if raise_on_unreachable

  []
end

#discovery_registry_publisher ⇒ Object 



222
223
224
225
226
227
228
 
# File 'lib/legion/extensions/llm/provider.rb', line 222

def discovery_registry_publisher
  return unless self.class.respond_to?(:registry_publisher)

  self.class.registry_publisher
rescue StandardError
  nil
end

#discovery_registry_readiness(provider_health, live:) ⇒ Object 



230
231
232
233
234
235
236
237
238
 
# File 'lib/legion/extensions/llm/provider.rb', line 230

def discovery_registry_readiness(provider_health, live:)
  {
    provider: slug.to_sym,
    configured: configured?,
    ready: provider_health[:ready] == true,
    live: live,
    health: provider_health
  }
end

#embed(text:, model:, dimensions: nil, params: {}, headers: {}) ⇒ Object 



267
268
269
270
271
272
273
274
 
# File 'lib/legion/extensions/llm/provider.rb', line 267

def embed(text:, model:, dimensions: nil, params: {}, headers: {})
  enforce_model_allowed!(model)
  payload = Utils.deep_merge(render_embedding_payload(text, model:, dimensions:), params)
  response = @connection.post(embedding_url(model:), payload) do |req|
    req.headers = headers.merge(req.headers) unless headers.empty?
  end
  parse_embedding_response(response, model:, text:)
end

#endpoint_manifest ⇒ Object 



412
413
414
415
416
417
418
419
420
421
 
# File 'lib/legion/extensions/llm/provider.rb', line 412

def endpoint_manifest
  endpoint_methods.each_with_object({}) do |(key, method_name), result|
    next unless respond_to?(method_name)

    value = public_send(method_name)
    result[key] = value unless value.nil?
  rescue ArgumentError, NotImplementedError
    next
  end
end

#enforce_model_allowed!(model_name) ⇒ Object

Compliance guard: refuse to dispatch any request for a model excluded by the configured model_whitelist / model_blacklist. Invoked at every dispatch entry point (the last line before the model API call) so a denied model can never reach a provider API, regardless of caller. Fail closed — raises rather than silently routing elsewhere.

Raises:



614
615
616
617
618
619
620
 
# File 'lib/legion/extensions/llm/provider.rb', line 614

def enforce_model_allowed!(model_name)
  return if model_allowed?(model_name)

  log.warn("[#{slug}] action=model_denied model=#{model_name} instance=#{provider_instance_id} " \
           'reason=model_whitelist_or_blacklist')
  raise ModelNotAllowedError.new(model: model_name, provider: slug)
end

#fetch_model_detail(_model_name) ⇒ Object

Override in subclasses to make a live API call for model detail. Must return a Hash with symbol keys (e.g. { context_window: 128000 }).



715
716
717
 
# File 'lib/legion/extensions/llm/provider.rb', line 715

def fetch_model_detail(_model_name)
  nil
end

#find_reachable_url(urls) ⇒ Object 



653
654
655
656
657
658
659
 
# File 'lib/legion/extensions/llm/provider.rb', line 653

def find_reachable_url(urls)
  urls.each do |url|
    full = normalize_url(url)
    return full if url_reachable?(full)
  end
  nil
end

#format_messages(messages) ⇒ Object 



445
446
447
448
449
450
451
452
 
# File 'lib/legion/extensions/llm/provider.rb', line 445

def format_messages(messages)
  messages.map do |msg|
    {
      role: msg.role.to_s,
      content: msg.content
    }
  end
end

#format_tool_calls(_tool_calls) ⇒ Object 



454
455
456
 
# File 'lib/legion/extensions/llm/provider.rb', line 454

def format_tool_calls(_tool_calls)
  nil
end

#global_llm_setting(key) ⇒ Object

Global LLM setting: extensions.llm. (lowest specificity)



518
519
520
521
522
523
524
525
 
# File 'lib/legion/extensions/llm/provider.rb', line 518

def global_llm_setting(key)
  return nil unless defined?(Legion::Settings)

  llm_conf = Legion::Settings.dig(:extensions, :llm)
  llm_conf.is_a?(Hash) ? llm_conf[key] : nil
rescue StandardError
  nil
end

#headers ⇒ Object 



96
97
98
 
# File 'lib/legion/extensions/llm/provider.rb', line 96

def headers
  identity_headers
end

#health(live: false) ⇒ Object 



240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
 
# File 'lib/legion/extensions/llm/provider.rb', line 240

def health(live: false)
  readiness_data = readiness(live:)
  raw_health = readiness_data[:health] || readiness_data['health'] || {}
  status = health_status(readiness_data, raw_health)
  latency_ms = (raw_health[:latency_ms] || raw_health['latency_ms'] if raw_health.is_a?(Hash))
  {
    provider: slug.to_sym,
    instance_id: provider_instance_id,
    status:,
    ready: readiness_data[:ready] == true || readiness_data['ready'] == true,
    circuit_state: status == 'healthy' ? 'closed' : 'open',
    latency_ms: latency_ms,
    raw: raw_health
  }.compact
rescue StandardError => e
  handle_exception(e, level: :warn, handled: true, operation: 'llm.provider.health')
  {
    provider: slug.to_sym,
    instance_id: provider_instance_id,
    status: 'unhealthy',
    ready: false,
    circuit_state: 'open',
    error: e.class.name,
    message: e.message
  }
end

#identity_headers ⇒ Object 



100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/legion/extensions/llm/provider.rb', line 100

def identity_headers
  return {} unless defined?(Legion::Identity::Process) && Legion::Identity::Process.respond_to?(:identity_hash)

  id = Legion::Identity::Process.identity_hash
  hdrs = {
    'x-legion-identity-canonical-name' => id[:canonical_name].to_s,
    'x-legion-identity-trust' => id[:trust].to_s,
    'x-legion-identity-id' => id[:id].to_s,
    'x-legion-identity-kind' => id[:kind].to_s,
    'x-legion-identity-mode' => id[:mode].to_s,
    'x-legion-identity-source' => id[:source].to_s
  }
  hdrs['x-legion-identity-db-principal-id'] = id[:db_principal_id].to_s if id[:db_principal_id]
  hdrs['x-legion-identity-db-identity-id']  = id[:db_identity_id].to_s if id[:db_identity_id]
  hdrs
rescue StandardError
  {}
end

#image(prompt:, model:, size:, with: nil, mask: nil, params: {}) ⇒ Object

rubocop:disable Metrics/ParameterLists



291
292
293
 
# File 'lib/legion/extensions/llm/provider.rb', line 291

def image(prompt:, model:, size:, with: nil, mask: nil, params: {}) # rubocop:disable Metrics/ParameterLists
  paint(prompt, model:, size:, with:, mask:, params:)
end

#instance_setting(key) ⇒ Object

Pull a setting from the instance-level settings hash (if available), distinct from the config object which is a HashConfig wrapper.



488
489
490
491
492
493
494
495
496
497
498
499
 
# File 'lib/legion/extensions/llm/provider.rb', line 488

def instance_setting(key)
  config_hash =
    if instance_variable_defined?(:@settings)
      @settings
    elsif respond_to?(:settings)
      settings
    else
      config
    end
  config_hash = config_hash.to_h if config_hash.respond_to?(:to_h)
  config_hash.is_a?(Hash) ? (config_hash[key] || config_hash[key.to_s]) : nil
end

#list_models(live: false, **filters) ⇒ Object

rubocop:enable Metrics/ParameterLists



186
187
188
189
190
 
# File 'lib/legion/extensions/llm/provider.rb', line 186

def list_models(live: false, **filters)
  _ = [live, filters]
  response = @connection.get models_url
  parse_list_models_response response, slug, capabilities
end

#local? ⇒ Boolean

Returns:

  • (Boolean) 


378
379
380
 
# File 'lib/legion/extensions/llm/provider.rb', line 378

def local?
  self.class.local?
end

#model_allowed?(model_name) ⇒ Boolean

Returns:

  • (Boolean) 


527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
 
# File 'lib/legion/extensions/llm/provider.rb', line 527

def model_allowed?(model_name)
  wl = model_whitelist
  bl = model_blacklist
  allowed = self.class.policy_allows?(model_name, whitelist: wl, blacklist: bl)

  unless allowed
    reason_parts = []
    reason_parts << 'whitelist' if wl.any?
    reason_parts << 'blacklist' if bl.any?
    reason_str = reason_parts.empty? ? 'policy' : reason_parts.join(',')
    policy_src = if wl.any?
                   "wl=[#{wl.first(5).join(',')}#{',...' if wl.size > 5}]"
                 else
                   'no-whitelist'
                 end
    log.debug("[#{self.class.slug}] action=model_rejected name=#{model_name} reason=#{reason_str} #{policy_src}")
  end

  allowed
end

#model_blacklist ⇒ Object

Resolve model_blacklist with the same specificity cascade as model_whitelist.



478
479
480
481
482
483
484
 
# File 'lib/legion/extensions/llm/provider.rb', line 478

def model_blacklist
  bl = config.model_blacklist if config.respond_to?(:model_blacklist)
  bl ||= instance_setting(:model_blacklist)
  bl ||= runtime_provider_setting(:model_blacklist)
  bl ||= global_llm_setting(:model_blacklist)
  Array(bl).map { |p| p.to_s.downcase }
end

#model_cache_get(key) ⇒ Object 



690
691
692
693
694
695
696
697
 
# File 'lib/legion/extensions/llm/provider.rb', line 690

def model_cache_get(key)
  return nil unless defined?(Legion::Cache)

  cache_local_instance? ? local_cache_get(key) : cache_get(key)
rescue StandardError => e
  handle_exception(e, level: :warn, handled: true, operation: 'llm.provider.model_cache_get', key:)
  nil
end

#model_detail(model_name) ⇒ Object 



699
700
701
702
703
704
705
706
707
708
709
710
711
 
# File 'lib/legion/extensions/llm/provider.rb', line 699

def model_detail(model_name)
  key = model_detail_cache_key(model_name)
  cached = cache_get(key)
  return cached if cached

  result = fetch_model_detail(model_name)
  cache_set(key, result, ttl: 86_400) if result
  result
rescue StandardError => e
  handle_exception(e, level: :warn, handled: true, operation: 'llm.provider.model_detail',
                      model: model_name)
  nil
end

#model_whitelist ⇒ Object

Resolve model_whitelist with specificity cascade:

  1. Instance-level (config.model_whitelist — extensions.llm..instances..model_whitelist)
  2. Provider-level (extensions.llm..model_whitelist)
  3. Global (extensions.llm.model_whitelist) Returns the first non-nil, non-empty value found.


469
470
471
472
473
474
475
 
# File 'lib/legion/extensions/llm/provider.rb', line 469

def model_whitelist
  wl = config.model_whitelist if config.respond_to?(:model_whitelist)
  wl ||= instance_setting(:model_whitelist)
  wl ||= runtime_provider_setting(:model_whitelist)
  wl ||= global_llm_setting(:model_whitelist)
  Array(wl).map { |p| p.to_s.downcase }
end

#moderate(input, model:) ⇒ Object 



276
277
278
279
280
281
 
# File 'lib/legion/extensions/llm/provider.rb', line 276

def moderate(input, model:)
  enforce_model_allowed!(model)
  payload = render_moderation_payload(input, model:)
  response = @connection.post moderation_url, payload
  parse_moderation_response(response, model:)
end

#name ⇒ Object 



123
124
125
 
# File 'lib/legion/extensions/llm/provider.rb', line 123

def name
  self.class.name
end

#normalize_dispatch_error(error:) ⇒ Object

Runtime error-to-outcome normalization consumed by the common classifier. The inherited base is deliberately conservative: raw 503, Anthropic-style 529, ServiceUnavailableError, ServerError, and every unrecognized error return :provider_error, never :instance_unavailable. Provider PRs override only when their wire semantics supply stronger evidence. The fallback reason is the bounded exception class name — never a response body, credential, endpoint, or exception object. It is a base method, not a REQUIRED_SIGNATURES reflection entry. See Phase 2 §4.5.2.



333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
 
# File 'lib/legion/extensions/llm/provider.rb', line 333

def normalize_dispatch_error(error:)
  kind = case error
         when OverloadedError then :overloaded
         when RateLimitError then :rate_limited
         when UnauthorizedError then :authentication
         when PaymentRequiredError then :billing
         when ForbiddenError then :authorization
         when ContextLengthExceededError then :context_rejected
         when BadRequestError then :invalid_request
         when ModelNotFoundError then :model_missing
         when ModelNotAllowedError then :policy
         when Faraday::TimeoutError, Timeout::Error then :timeout
         when Faraday::ConnectionFailed, Errno::ECONNREFUSED, Errno::ECONNRESET, SocketError then :connection_failure
         else :provider_error
         end
  reason = error.class.name
  reason = 'UnknownError' if reason.nil? || reason.empty?
  Legion::Extensions::Llm::Routing::ProviderOutcome.new(kind: kind, reason: reason)
end

#normalize_url(url) ⇒ Object 



645
646
647
648
649
650
651
 
# File 'lib/legion/extensions/llm/provider.rb', line 645

def normalize_url(url)
  raw = url.to_s.strip
  return raw if raw.match?(%r{^https?://})

  scheme = tls_enabled? ? 'https' : 'http'
  "#{scheme}://#{raw}"
end

#offering_tier ⇒ Object 



628
629
630
 
# File 'lib/legion/extensions/llm/provider.rb', line 628

def offering_tier
  config.respond_to?(:tier) ? config.tier : self.class.default_tier
end

#offering_transport ⇒ Object

── Offering defaults ─────────────────────────────────────────────



624
625
626
 
# File 'lib/legion/extensions/llm/provider.rb', line 624

def offering_transport
  config.respond_to?(:transport) ? config.transport : self.class.default_transport
end

#paint(prompt, model:, size:, with: nil, mask: nil, params: {}) ⇒ Object

rubocop:disable Metrics/ParameterLists



283
284
285
286
287
288
289
 
# File 'lib/legion/extensions/llm/provider.rb', line 283

def paint(prompt, model:, size:, with: nil, mask: nil, params: {}) # rubocop:disable Metrics/ParameterLists
  enforce_model_allowed!(model)
  validate_paint_inputs!(with:, mask:)
  payload = render_image_payload(prompt, model:, size:, with:, mask:, params:)
  response = @connection.post images_url(with:, mask:), payload
  parse_image_response(response, model:)
end

#parse_error(response) ⇒ Object 



423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
 
# File 'lib/legion/extensions/llm/provider.rb', line 423

def parse_error(response)
  return if response.body.empty?

  body = try_parse_json(response.body)
  case body
  when Hash
    error = body['error']
    return error if error.is_a?(String)

    body.dig('error', 'message')
  when Array
    body.map do |part|
      error = part['error']
      error.is_a?(String) ? error : part.dig('error', 'message')
    end.join('. ')
  when String
    body[/"message"\s*:\s*"([^"]{1,500})/, 1] || body
  else
    body
  end
end

#parse_tool_calls(_tool_calls) ⇒ Object 



458
459
460
 
# File 'lib/legion/extensions/llm/provider.rb', line 458

def parse_tool_calls(_tool_calls)
  nil
end

#provider_instance_id ⇒ Object 



729
730
731
732
733
 
# File 'lib/legion/extensions/llm/provider.rb', line 729

def provider_instance_id
  return config.instance_id.to_sym if config.respond_to?(:instance_id) && config.instance_id

  :default
end

#publish_discovered_model_to_registry(model, provider_health:, live:) ⇒ Object 



213
214
215
216
217
218
219
220
 
# File 'lib/legion/extensions/llm/provider.rb', line 213

def publish_discovered_model_to_registry(model, provider_health:, live:)
  publisher = discovery_registry_publisher
  return unless publisher.respond_to?(:publish_models_async)

  publisher.publish_models_async([model], readiness: discovery_registry_readiness(provider_health, live:))
rescue StandardError => e
  handle_exception(e, level: :warn, handled: true, operation: 'llm.provider.publish_discovered_model')
end

#readiness(live: false) ⇒ Object 



390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
 
# File 'lib/legion/extensions/llm/provider.rb', line 390

def readiness(live: false)
  metadata = {
    provider: slug.to_sym,
    name: name,
    configured: configured?,
    ready: configured?,
    local: local?,
    remote: remote?,
    api_base: api_base,
    endpoints: endpoint_manifest,
    live: live
  }

  return metadata.merge(health: { checked: false }) unless live && metadata[:endpoints][:health]

  response = @connection.get(metadata[:endpoints][:health])
  metadata.merge(ready: configured? && health_ready?(response.body), health: response.body)
rescue StandardError => e
  handle_exception(e, level: :warn, handled: true, operation: 'llm.provider.readiness')
  metadata.merge(ready: false, health: { error: e.class.name, message: e.message })
end

#remote? ⇒ Boolean

Returns:

  • (Boolean) 


382
383
384
 
# File 'lib/legion/extensions/llm/provider.rb', line 382

def remote?
  self.class.remote?
end

#resolve_base_url ⇒ Object

── Multi-host base_url resolution ────────────────────────────────



634
635
636
637
638
639
 
# File 'lib/legion/extensions/llm/provider.rb', line 634

def resolve_base_url
  urls = Array(config_base_url)
  return nil if urls.empty?

  @resolve_base_url ||= find_reachable_url(urls) || normalize_url(urls.first)
end

#runtime_provider_setting(key) ⇒ Object

Provider-level setting: extensions.llm..



502
503
504
505
506
507
508
509
510
511
512
513
514
515
 
# File 'lib/legion/extensions/llm/provider.rb', line 502

def runtime_provider_setting(key)
  return nil unless defined?(Legion::Settings)

  ext = Legion::Settings[:extensions]
  return nil unless ext.is_a?(Hash) && ext[:llm].is_a?(Hash)

  provider_key = self.class.respond_to?(:slug) ? self.class.slug.to_sym : nil
  return nil unless provider_key

  provider_conf = ext[:llm][provider_key]
  provider_conf.is_a?(Hash) ? provider_conf[key] : nil
rescue StandardError
  nil
end

#slug ⇒ Object 



119
120
121
 
# File 'lib/legion/extensions/llm/provider.rb', line 119

def slug
  self.class.slug
end

#speak(text, model:, voice: nil, **provider_options) ⇒ Object

Raises:

  • (NotImplementedError) 


320
321
322
323
 
# File 'lib/legion/extensions/llm/provider.rb', line 320

def speak(text, model:, voice: nil, **provider_options)
  _ = [text, model, voice, provider_options]
  raise NotImplementedError, "#{self.class} does not implement speak"
end

#stream_chat(messages:, model:, tools: [], temperature: nil, params: {}, headers: {}, schema: nil, thinking: nil, tool_prefs: nil) ⇒ Object 



141
142
143
144
 
# File 'lib/legion/extensions/llm/provider.rb', line 141

def stream_chat(messages:, model:, tools: [], temperature: nil, params: {}, headers: {}, schema: nil,
                thinking: nil, tool_prefs: nil, &)
  complete(messages, tools:, temperature:, model:, params:, headers:, schema:, thinking:, tool_prefs:, &)
end

#strip_scheme(url) ⇒ Object 



661
662
663
 
# File 'lib/legion/extensions/llm/provider.rb', line 661

def strip_scheme(url)
  url.to_s.sub(%r{^https?://}, '')
end

#tls_enabled? ⇒ Boolean

Returns:

  • (Boolean) 


676
677
678
679
 
# File 'lib/legion/extensions/llm/provider.rb', line 676

def tls_enabled?
  tls = respond_to?(:settings) ? settings[:tls] : nil
  tls.is_a?(Hash) && tls[:enabled] == true
end

#transcribe(audio_file, model:, language:) ⇒ Object 



303
304
305
306
307
308
 
# File 'lib/legion/extensions/llm/provider.rb', line 303

def transcribe(audio_file, model:, language:, **)
  file_part = build_audio_file_part(audio_file)
  payload = render_transcription_payload(file_part, model:, language:, **)
  response = @connection.post transcription_url, payload
  parse_transcription_response(response, model:)
end

#translate(audio_file, model:, language:, **provider_options) ⇒ Object

Fail-loud base audio operations. Unsupported providers inherit these and publish OperationEvidence(status: :unsupported) or :unknown; a provider may publish :supported only when its Phase 2 conformance spec exercises the actual callable path. Neither method reads configuration or infers a model. See phase-1-lex-llm-additive.md section 14.1.

Raises:

  • (NotImplementedError) 


315
316
317
318
 
# File 'lib/legion/extensions/llm/provider.rb', line 315

def translate(audio_file, model:, language:, **provider_options)
  _ = [audio_file, model, language, provider_options]
  raise NotImplementedError, "#{self.class} does not implement translate"
end

#url_reachable?(url) ⇒ Boolean

Returns:

  • (Boolean) 


665
666
667
668
669
670
671
672
673
674
 
# File 'lib/legion/extensions/llm/provider.rb', line 665

def url_reachable?(url)
  require 'uri'
  require 'socket'
  uri = URI.parse(url)
  Socket.tcp(uri.host, uri.port, connect_timeout: 1).close
  true
rescue StandardError => e
  handle_exception(e, level: :warn, handled: true, operation: 'llm.provider.url_reachable', url:)
  false
end