Class: Legion::Extensions::Llm::Provider

Inherits:
Object
  • Object
show all
Includes:
Cache::Helper, Streaming, Logging::Helper
Defined in:
lib/legion/extensions/llm/provider.rb,
lib/legion/extensions/llm/provider/open_ai_compatible.rb

Overview

Base class for LLM providers.

Defined Under Namespace

Modules: OpenAICompatible

Constant Summary collapse

MODEL_DETAIL_CACHE_SCHEMA_VERSION = 
2
CAPABILITY_CONFIG_KEYS = 
%i[
  capabilities
  enable_completion
  enable_embedding
  enable_embeddings
  enable_streaming
  enable_tools
  enable_functions
  enable_function_calling
  enable_thinking
  enable_reasoning
  enable_vision
  enable_structured_output
  enable_moderation
  enable_image
  enable_images
  enable_image_generation
  enable_audio_transcription
  enable_audio_speech
  enable_audio_generation
  completion_flag
  embedding_flag
  embeddings_flag
  streaming_flag
  tool_flag
  tools_flag
  functions_flag
  function_calling_flag
  thinking_flag
  reasoning_flag
  vision_flag
  structured_output_flag
  moderation_flag
  image_flag
  images_flag
  image_generation_flag
  audio_transcription_flag
  audio_speech_flag
  audio_generation_flag
].freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Streaming

build_on_data_handler, build_stream_callback, build_stream_error_response, error_chunk?, faraday_1?, handle_data, handle_error_chunk, handle_error_event, handle_failed_response, handle_json_error_chunk, handle_parsed_error, handle_sse, handle_stream, json_error_payload?, parse_error_from_json, parse_streaming_error, persist_failed_response_body, persist_failed_response_custom_body?, persist_failed_response_env_body?, process_stream_chunk, raise_partial_streaming_error, raise_streaming_status_error, stream_response

Constructor Details

#initialize(config) ⇒ Provider

Returns a new instance of Provider.



81
82
83
84
85
 
# File 'lib/legion/extensions/llm/provider.rb', line 81

def initialize(config)
  @config = config.is_a?(Hash) ? HashConfig.new(config) : config
  ensure_configured!
  @connection = Connection.new(self, @config)
end

Instance Attribute Details

#config ⇒ Object (readonly)

Returns the value of attribute config.



79
80
81
 
# File 'lib/legion/extensions/llm/provider.rb', line 79

def config
  @config
end

#connection ⇒ Object (readonly)

Returns the value of attribute connection.



79
80
81
 
# File 'lib/legion/extensions/llm/provider.rb', line 79

def connection
  @connection
end

Class Method Details

.assume_models_exist? ⇒ Boolean

Returns:

  • (Boolean) 


774
775
776
 
# File 'lib/legion/extensions/llm/provider.rb', line 774

def assume_models_exist?
  false
end

.capabilities ⇒ Object 



746
747
748
 
# File 'lib/legion/extensions/llm/provider.rb', line 746

def capabilities
  nil
end

.configuration_options ⇒ Object 



754
755
756
 
# File 'lib/legion/extensions/llm/provider.rb', line 754

def configuration_options
  []
end

.configuration_requirements ⇒ Object 



750
751
752
 
# File 'lib/legion/extensions/llm/provider.rb', line 750

def configuration_requirements
  []
end

.configured?(config) ⇒ Boolean

Returns:

  • (Boolean) 


782
783
784
 
# File 'lib/legion/extensions/llm/provider.rb', line 782

def configured?(config)
  configuration_requirements.all? { |req| config.send(req) }
end

.default_tier ⇒ Object 



762
763
764
 
# File 'lib/legion/extensions/llm/provider.rb', line 762

def default_tier
  :frontier
end

.default_transport ⇒ Object 



758
759
760
 
# File 'lib/legion/extensions/llm/provider.rb', line 758

def default_transport
  :http
end

.local? ⇒ Boolean

Returns:

  • (Boolean) 


766
767
768
 
# File 'lib/legion/extensions/llm/provider.rb', line 766

def local?
  false
end

.model_policy(config, provider_family) ⇒ Object

Effective whitelist/blacklist for an instance config at build time (before provider instance exists). Same specificity cascade:

  1. Per-instance (config hash — extensions.llm..instances..model_whitelist)
  2. Provider-level (extensions.llm..model_whitelist)
  3. Global (extensions.llm.model_whitelist)


570
571
572
573
574
575
576
577
578
579
580
581
 
# File 'lib/legion/extensions/llm/provider.rb', line 570

def self.model_policy(config, provider_family)
  cfg = config.is_a?(Hash) ? config : {}
  provider_conf = CredentialSources.setting(:extensions, :llm, provider_family)
  provider_conf = {} unless provider_conf.is_a?(Hash)
  global_conf = (::Legion::Settings.dig(:extensions, :llm) if defined?(::Legion::Settings))
  global_conf = {} unless global_conf.is_a?(Hash)

  {
    whitelist: resolve_policy_value(cfg, provider_conf, global_conf, :model_whitelist),
    blacklist: resolve_policy_value(cfg, provider_conf, global_conf, :model_blacklist)
  }
end

.name ⇒ Object 



738
739
740
 
# File 'lib/legion/extensions/llm/provider.rb', line 738

def name
  to_s.split('::').last
end

.policy_allows?(model_name, whitelist: [], blacklist: []) ⇒ Boolean

Single source of truth for model-policy matching, usable both at runtime (instance #model_allowed?) and at instance-config build time (provider extensions choosing a default_model that does not violate the policy). Substring, case-insensitive: a whitelist permits models containing any pattern; a blacklist denies models containing any pattern; whitelist is applied before blacklist. Empty list = no restriction from that side.

Returns:

  • (Boolean) 


554
555
556
557
558
559
560
561
562
563
 
# File 'lib/legion/extensions/llm/provider.rb', line 554

def self.policy_allows?(model_name, whitelist: [], blacklist: [])
  name = model_name.to_s.downcase
  wl = Array(whitelist).map { |p| p.to_s.downcase }
  bl = Array(blacklist).map { |p| p.to_s.downcase }

  return false if wl.any? && wl.none? { |p| name.include?(p) }
  return false if bl.any? && bl.any? { |p| name.include?(p) }

  true
end

.policy_safe_default_model(configured:, fallback:, whitelist: [], blacklist: []) ⇒ Object

Choose a default_model that never violates the model policy: prefer an explicitly-configured default when permitted; else a provider fallback when permitted; else nil, so routing resolves an allowed discovered model rather than forcing a policy-forbidden default. Keeps a whitelist/blacklist authoritative over any hardcoded provider default.



602
603
604
605
606
607
608
609
 
# File 'lib/legion/extensions/llm/provider.rb', line 602

def self.policy_safe_default_model(configured:, fallback:, whitelist: [], blacklist: [])
  return configured if configured && !configured.to_s.empty? &&
                       policy_allows?(configured, whitelist:, blacklist:)
  return fallback if fallback && !fallback.to_s.empty? &&
                     policy_allows?(fallback, whitelist:, blacklist:)

  nil
end

.remote? ⇒ Boolean

Returns:

  • (Boolean) 


770
771
772
 
# File 'lib/legion/extensions/llm/provider.rb', line 770

def remote?
  !local?
end

.resolve_model_id(model_id, config: nil) ⇒ Object

rubocop:disable Lint/UnusedMethodArgument



778
779
780
 
# File 'lib/legion/extensions/llm/provider.rb', line 778

def resolve_model_id(model_id, config: nil) # rubocop:disable Lint/UnusedMethodArgument
  model_id
end

.resolve_policy_value(cfg, provider_conf, global_conf, key) ⇒ Object

Resolve a single policy value with instance > provider > global precedence.



584
585
586
587
588
589
590
591
592
593
594
595
 
# File 'lib/legion/extensions/llm/provider.rb', line 584

def self.resolve_policy_value(cfg, provider_conf, global_conf, key)
  # Instance-level
  val = cfg[key] || cfg[key.to_s]
  return val if val && !val.to_s.empty? && (val.is_a?(Array) ? val.any? : true)

  # Provider-level
  val = provider_conf[key] || provider_conf[key.to_s]
  return val if val && !val.to_s.empty? && (val.is_a?(Array) ? val.any? : true)

  # Global
  global_conf[key] || global_conf[key.to_s]
end

.slug ⇒ Object 



742
743
744
 
# File 'lib/legion/extensions/llm/provider.rb', line 742

def slug
  name.downcase
end

Instance Method Details

#api_base ⇒ Object

Raises:

  • (NotImplementedError) 


92
93
94
 
# File 'lib/legion/extensions/llm/provider.rb', line 92

def api_base
  raise NotImplementedError
end

#assume_models_exist? ⇒ Boolean

Returns:

  • (Boolean) 


386
387
388
 
# File 'lib/legion/extensions/llm/provider.rb', line 386

def assume_models_exist?
  self.class.assume_models_exist?
end

#cache_control_prefix_tokens ⇒ Object 



370
371
372
373
374
375
376
 
# File 'lib/legion/extensions/llm/provider.rb', line 370

def cache_control_prefix_tokens
  if config.respond_to?(:cache_control_prefix_tokens) && config.cache_control_prefix_tokens
    config.cache_control_prefix_tokens
  else
    4
  end
end

#cache_enabled? ⇒ Boolean

Returns:

  • (Boolean) 


357
358
359
360
361
362
363
364
365
366
367
368
 
# File 'lib/legion/extensions/llm/provider.rb', line 357

def cache_enabled?
  explicit = config.llm_cache_enabled if config.respond_to?(:llm_cache_enabled)

  unless explicit.nil?
    log.debug { "[#{slug}] cache_enabled? source=per_provider value=#{explicit}" }
    return explicit == true
  end

  global = global_prompt_caching_enabled?
  log.debug { "[#{slug}] cache_enabled? source=global value=#{global}" }
  global
end

#cache_instance_key ⇒ Object 



721
722
723
724
725
726
727
728
729
 
# File 'lib/legion/extensions/llm/provider.rb', line 721

def cache_instance_key
  if cache_local_instance?
    (respond_to?(:instance_id) ? instance_id : :default).to_s
  else
    require 'digest'
    urls = Array(config_base_url).map { |u| strip_scheme(u).downcase.chomp('/') }.sort
    Digest::SHA256.hexdigest(urls.join('|'))[0, 12]
  end
end

#cache_local_instance? ⇒ Boolean

── Cache helpers with local/shared tier selection ────────────────

Returns:

  • (Boolean) 


685
686
687
688
689
690
 
# File 'lib/legion/extensions/llm/provider.rb', line 685

def cache_local_instance?
  Array(config_base_url).any? do |url|
    host = url.to_s.downcase
    host.include?('localhost') || host.include?('127.0.0.1') || host.include?('::1')
  end
end

#capabilities ⇒ Object 



127
128
129
 
# File 'lib/legion/extensions/llm/provider.rb', line 127

def capabilities
  self.class.capabilities
end

#chat(messages:, model:, tools: [], temperature: nil, params: {}, headers: {}, schema: nil, thinking: nil, tool_prefs: nil) ⇒ Object

rubocop:disable Metrics/ParameterLists



136
137
138
139
 
# File 'lib/legion/extensions/llm/provider.rb', line 136

def chat(messages:, model:, tools: [], temperature: nil, params: {}, headers: {}, schema: nil, thinking: nil,
         tool_prefs: nil)
  complete(messages, tools:, temperature:, model:, params:, headers:, schema:, thinking:, tool_prefs:)
end

#complete(messages, tools:, temperature:, model:, params: {}, headers: {}, schema: nil, thinking: nil, tool_prefs: nil) ⇒ Object 



146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
 
# File 'lib/legion/extensions/llm/provider.rb', line 146

def complete(messages, tools:, temperature:, model:, params: {}, headers: {}, schema: nil, thinking: nil,
             tool_prefs: nil, &)
  enforce_model_allowed!(model)
  normalized_temperature = maybe_normalize_temperature(temperature, model)
  log_provider_request(
    messages: messages,
    tools: tools,
    temperature: temperature,
    normalized_temperature: normalized_temperature,
    model: model,
    params: params,
    headers: headers,
    schema: schema,
    thinking: thinking,
    tool_prefs: tool_prefs,
    streaming: block_given?
  )

  payload = Utils.deep_merge(
    render_payload(
      messages,
      tools: tools,
      tool_prefs: tool_prefs,
      temperature: normalized_temperature,
      model: model,
      stream: block_given?,
      schema: schema,
      thinking: thinking
    ),
    params
  )

  if block_given?
    stream_response @connection, payload, headers, &
  else
    sync_response @connection, payload, headers
  end
end

#config_base_url ⇒ Object 



643
644
645
 
# File 'lib/legion/extensions/llm/provider.rb', line 643

def config_base_url
  respond_to?(:settings) ? settings[:base_url] : nil
end

#configuration_requirements ⇒ Object 



131
132
133
 
# File 'lib/legion/extensions/llm/provider.rb', line 131

def configuration_requirements
  self.class.configuration_requirements
end

#configured? ⇒ Boolean

Returns:

  • (Boolean) 


353
354
355
 
# File 'lib/legion/extensions/llm/provider.rb', line 353

def configured?
  configuration_requirements.all? { |req| @config.send(req) }
end

#count_tokens(messages:, model:, params: {}) ⇒ Object 



295
296
297
298
299
300
301
 
# File 'lib/legion/extensions/llm/provider.rb', line 295

def count_tokens(messages:, model:, params: {})
  _ = [model, params]
  Array(messages).sum do |message|
    content = message.respond_to?(:content) ? message.content : message[:content] || message['content']
    estimate_text_tokens(content)
  end
end

#disconnect ⇒ Object 



87
88
89
90
 
# File 'lib/legion/extensions/llm/provider.rb', line 87

def disconnect
  @connection&.close
  @connection = nil
end

#discover_offerings(live: false, raise_on_unreachable: false, **filters) ⇒ Object 



192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
 
# File 'lib/legion/extensions/llm/provider.rb', line 192

def discover_offerings(live: false, raise_on_unreachable: false, **filters)
  return filter_cached_offerings(Array(@cached_offerings), filters) unless live

  provider_health = health(live:)
  @cached_offerings = Array(list_models(live:, **filters)).filter_map do |model|
    publish_discovered_model_to_registry(model, provider_health:, live:)
    next unless model_matches_filters?(model, filters)
    next unless model_allowed?(model.id)

    log.debug("[#{slug}] instance=#{provider_instance_id} action=model_discovered model=#{model.id} family=#{model.family}")
    offering_from_model(model, health: provider_health)
  end
  log.info("[#{slug}] instance=#{provider_instance_id} action=discover_complete model_count=#{Array(@cached_offerings).size}")
  @cached_offerings
rescue Faraday::ConnectionFailed, Faraday::TimeoutError => e
  log.warn("[#{slug}] instance=#{provider_instance_id} unreachable: #{e.message}")
  raise if raise_on_unreachable

  []
end

#discovery_registry_publisher ⇒ Object 



222
223
224
225
226
227
228
 
# File 'lib/legion/extensions/llm/provider.rb', line 222

def discovery_registry_publisher
  return unless self.class.respond_to?(:registry_publisher)

  self.class.registry_publisher
rescue StandardError
  nil
end

#discovery_registry_readiness(provider_health, live:) ⇒ Object 



230
231
232
233
234
235
236
237
238
 
# File 'lib/legion/extensions/llm/provider.rb', line 230

def discovery_registry_readiness(provider_health, live:)
  {
    provider: slug.to_sym,
    configured: configured?,
    ready: provider_health[:ready] == true,
    live: live,
    health: provider_health
  }
end

#embed(text:, model:, dimensions: nil, params: {}, headers: {}) ⇒ Object 



267
268
269
270
271
272
273
274
 
# File 'lib/legion/extensions/llm/provider.rb', line 267

def embed(text:, model:, dimensions: nil, params: {}, headers: {})
  enforce_model_allowed!(model)
  payload = Utils.deep_merge(render_embedding_payload(text, model:, dimensions:), params)
  response = @connection.post(embedding_url(model:), payload) do |req|
    req.headers = headers.merge(req.headers) unless headers.empty?
  end
  parse_embedding_response(response, model:, text:)
end

#endpoint_manifest ⇒ Object 



412
413
414
415
416
417
418
419
420
421
 
# File 'lib/legion/extensions/llm/provider.rb', line 412

def endpoint_manifest
  endpoint_methods.each_with_object({}) do |(key, method_name), result|
    next unless respond_to?(method_name)

    value = public_send(method_name)
    result[key] = value unless value.nil?
  rescue ArgumentError, NotImplementedError
    next
  end
end

#enforce_model_allowed!(model_name) ⇒ Object

Compliance guard: refuse to dispatch any request for a model excluded by the configured model_whitelist / model_blacklist. Invoked at every dispatch entry point (the last line before the model API call) so a denied model can never reach a provider API, regardless of caller. Fail closed — raises rather than silently routing elsewhere.

Raises:



616
617
618
619
620
621
622
 
# File 'lib/legion/extensions/llm/provider.rb', line 616

def enforce_model_allowed!(model_name)
  return if model_allowed?(model_name)

  log.warn("[#{slug}] action=model_denied model=#{model_name} instance=#{provider_instance_id} " \
           'reason=model_whitelist_or_blacklist')
  raise ModelNotAllowedError.new(model: model_name, provider: slug)
end

#fetch_model_detail(_model_name) ⇒ Object

Override in subclasses to make a live API call for model detail. Must return a Hash with symbol keys (e.g. { context_window: 128000 }).



717
718
719
 
# File 'lib/legion/extensions/llm/provider.rb', line 717

def fetch_model_detail(_model_name)
  nil
end

#find_reachable_url(urls) ⇒ Object 



655
656
657
658
659
660
661
 
# File 'lib/legion/extensions/llm/provider.rb', line 655

def find_reachable_url(urls)
  urls.each do |url|
    full = normalize_url(url)
    return full if url_reachable?(full)
  end
  nil
end

#format_messages(messages) ⇒ Object 



445
446
447
448
449
450
451
452
 
# File 'lib/legion/extensions/llm/provider.rb', line 445

def format_messages(messages)
  messages.map do |msg|
    {
      role: msg.role.to_s,
      content: msg.content
    }
  end
end

#format_tool_calls(_tool_calls) ⇒ Object 



454
455
456
 
# File 'lib/legion/extensions/llm/provider.rb', line 454

def format_tool_calls(_tool_calls)
  nil
end

#global_llm_setting(key) ⇒ Object

Global LLM setting: extensions.llm. (lowest specificity)



518
519
520
521
522
523
524
525
 
# File 'lib/legion/extensions/llm/provider.rb', line 518

def global_llm_setting(key)
  return nil unless defined?(Legion::Settings)

  llm_conf = Legion::Settings.dig(:extensions, :llm)
  llm_conf.is_a?(Hash) ? llm_conf[key] : nil
rescue StandardError
  nil
end

#headers ⇒ Object 



96
97
98
 
# File 'lib/legion/extensions/llm/provider.rb', line 96

def headers
  identity_headers
end

#health(live: false) ⇒ Object 



240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
 
# File 'lib/legion/extensions/llm/provider.rb', line 240

def health(live: false)
  readiness_data = readiness(live:)
  raw_health = readiness_data[:health] || readiness_data['health'] || {}
  status = health_status(readiness_data, raw_health)
  latency_ms = (raw_health[:latency_ms] || raw_health['latency_ms'] if raw_health.is_a?(Hash))
  {
    provider: slug.to_sym,
    instance_id: provider_instance_id,
    status:,
    ready: readiness_data[:ready] == true || readiness_data['ready'] == true,
    circuit_state: status == 'healthy' ? 'closed' : 'open',
    latency_ms: latency_ms,
    raw: raw_health
  }.compact
rescue StandardError => e
  handle_exception(e, level: :warn, handled: true, operation: 'llm.provider.health')
  {
    provider: slug.to_sym,
    instance_id: provider_instance_id,
    status: 'unhealthy',
    ready: false,
    circuit_state: 'open',
    error: e.class.name,
    message: e.message
  }
end

#identity_headers ⇒ Object 



100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/legion/extensions/llm/provider.rb', line 100

def identity_headers
  return {} unless defined?(Legion::Identity::Process) && Legion::Identity::Process.respond_to?(:identity_hash)

  id = Legion::Identity::Process.identity_hash
  hdrs = {
    'x-legion-identity-canonical-name' => id[:canonical_name].to_s,
    'x-legion-identity-trust' => id[:trust].to_s,
    'x-legion-identity-id' => id[:id].to_s,
    'x-legion-identity-kind' => id[:kind].to_s,
    'x-legion-identity-mode' => id[:mode].to_s,
    'x-legion-identity-source' => id[:source].to_s
  }
  hdrs['x-legion-identity-db-principal-id'] = id[:db_principal_id].to_s if id[:db_principal_id]
  hdrs['x-legion-identity-db-identity-id']  = id[:db_identity_id].to_s if id[:db_identity_id]
  hdrs
rescue StandardError
  {}
end

#image(prompt:, model:, size:, with: nil, mask: nil, params: {}) ⇒ Object

rubocop:disable Metrics/ParameterLists



291
292
293
 
# File 'lib/legion/extensions/llm/provider.rb', line 291

def image(prompt:, model:, size:, with: nil, mask: nil, params: {}) # rubocop:disable Metrics/ParameterLists
  paint(prompt, model:, size:, with:, mask:, params:)
end

#instance_setting(key) ⇒ Object

Pull a setting from the instance-level settings hash (if available), distinct from the config object which is a HashConfig wrapper.



488
489
490
491
492
493
494
495
496
497
498
499
 
# File 'lib/legion/extensions/llm/provider.rb', line 488

def instance_setting(key)
  config_hash =
    if instance_variable_defined?(:@settings)
      @settings
    elsif respond_to?(:settings)
      settings
    else
      config
    end
  config_hash = config_hash.to_h if config_hash.respond_to?(:to_h)
  config_hash.is_a?(Hash) ? (config_hash[key] || config_hash[key.to_s]) : nil
end

#list_models(live: false, **filters) ⇒ Object

rubocop:enable Metrics/ParameterLists



186
187
188
189
190
 
# File 'lib/legion/extensions/llm/provider.rb', line 186

def list_models(live: false, **filters)
  _ = [live, filters]
  response = @connection.get models_url
  parse_list_models_response response, slug, capabilities
end

#local? ⇒ Boolean

Returns:

  • (Boolean) 


378
379
380
 
# File 'lib/legion/extensions/llm/provider.rb', line 378

def local?
  self.class.local?
end

#model_allowed?(model_name) ⇒ Boolean

Returns:

  • (Boolean) 


527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
 
# File 'lib/legion/extensions/llm/provider.rb', line 527

def model_allowed?(model_name)
  wl = model_whitelist
  bl = model_blacklist
  allowed = self.class.policy_allows?(model_name, whitelist: wl, blacklist: bl)

  unless allowed
    reason_parts = []
    reason_parts << 'whitelist' if wl.any?
    reason_parts << 'blacklist' if bl.any?
    reason_str = reason_parts.empty? ? 'policy' : reason_parts.join(',')
    policy_src = if wl.any?
                   "wl=[#{wl.first(5).join(',')}#{',...' if wl.size > 5}]"
                 else
                   'no-whitelist'
                 end
    log.debug("[#{self.class.slug}] action=model_rejected name=#{model_name} reason=#{reason_str} #{policy_src}")
  end

  allowed
end

#model_blacklist ⇒ Object

Resolve model_blacklist with the same specificity cascade as model_whitelist.



478
479
480
481
482
483
484
 
# File 'lib/legion/extensions/llm/provider.rb', line 478

def model_blacklist
  bl = config.model_blacklist if config.respond_to?(:model_blacklist)
  bl ||= instance_setting(:model_blacklist)
  bl ||= runtime_provider_setting(:model_blacklist)
  bl ||= global_llm_setting(:model_blacklist)
  Array(bl).map { |p| p.to_s.downcase }
end

#model_cache_get(key) ⇒ Object 



692
693
694
695
696
697
698
699
 
# File 'lib/legion/extensions/llm/provider.rb', line 692

def model_cache_get(key)
  return nil unless defined?(Legion::Cache)

  cache_local_instance? ? local_cache_get(key) : cache_get(key)
rescue StandardError => e
  handle_exception(e, level: :warn, handled: true, operation: 'llm.provider.model_cache_get', key:)
  nil
end

#model_detail(model_name) ⇒ Object 



701
702
703
704
705
706
707
708
709
710
711
712
713
 
# File 'lib/legion/extensions/llm/provider.rb', line 701

def model_detail(model_name)
  key = model_detail_cache_key(model_name)
  cached = cache_get(key)
  return cached if cached

  result = fetch_model_detail(model_name)
  cache_set(key, result, ttl: 86_400) if result
  result
rescue StandardError => e
  handle_exception(e, level: :warn, handled: true, operation: 'llm.provider.model_detail',
                      model: model_name)
  nil
end

#model_whitelist ⇒ Object

Resolve model_whitelist with specificity cascade:

  1. Instance-level (config.model_whitelist — extensions.llm..instances..model_whitelist)
  2. Provider-level (extensions.llm..model_whitelist)
  3. Global (extensions.llm.model_whitelist) Returns the first non-nil, non-empty value found.


469
470
471
472
473
474
475
 
# File 'lib/legion/extensions/llm/provider.rb', line 469

def model_whitelist
  wl = config.model_whitelist if config.respond_to?(:model_whitelist)
  wl ||= instance_setting(:model_whitelist)
  wl ||= runtime_provider_setting(:model_whitelist)
  wl ||= global_llm_setting(:model_whitelist)
  Array(wl).map { |p| p.to_s.downcase }
end

#moderate(input, model:) ⇒ Object 



276
277
278
279
280
281
 
# File 'lib/legion/extensions/llm/provider.rb', line 276

def moderate(input, model:)
  enforce_model_allowed!(model)
  payload = render_moderation_payload(input, model:)
  response = @connection.post moderation_url, payload
  parse_moderation_response(response, model:)
end

#name ⇒ Object 



123
124
125
 
# File 'lib/legion/extensions/llm/provider.rb', line 123

def name
  self.class.name
end

#normalize_dispatch_error(error:) ⇒ Object

Runtime error-to-outcome normalization consumed by the common classifier. The inherited base is deliberately conservative: raw 503, Anthropic-style 529, ServiceUnavailableError, ServerError, and every unrecognized error return :provider_error, never :instance_unavailable. Provider PRs override only when their wire semantics supply stronger evidence. The fallback reason is the bounded exception class name — never a response body, credential, endpoint, or exception object. It is a base method, not a REQUIRED_SIGNATURES reflection entry. See Phase 2 §4.5.2.



333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
 
# File 'lib/legion/extensions/llm/provider.rb', line 333

def normalize_dispatch_error(error:)
  kind = case error
         when OverloadedError then :overloaded
         when RateLimitError then :rate_limited
         when UnauthorizedError then :authentication
         when PaymentRequiredError then :billing
         when ForbiddenError then :authorization
         when ContextLengthExceededError then :context_rejected
         when BadRequestError then :invalid_request
         when ModelNotFoundError then :model_missing
         when ModelNotAllowedError then :policy
         when Faraday::TimeoutError, Timeout::Error then :timeout
         when Faraday::ConnectionFailed, Errno::ECONNREFUSED, Errno::ECONNRESET, SocketError then :connection_failure
         else :provider_error
         end
  reason = error.class.name
  reason = 'UnknownError' if reason.nil? || reason.empty?
  Legion::Extensions::Llm::Routing::ProviderOutcome.new(kind: kind, reason: reason)
end

#normalize_url(url) ⇒ Object 



647
648
649
650
651
652
653
 
# File 'lib/legion/extensions/llm/provider.rb', line 647

def normalize_url(url)
  raw = url.to_s.strip
  return raw if raw.match?(%r{^https?://})

  scheme = tls_enabled? ? 'https' : 'http'
  "#{scheme}://#{raw}"
end

#offering_tier ⇒ Object 



630
631
632
 
# File 'lib/legion/extensions/llm/provider.rb', line 630

def offering_tier
  config.respond_to?(:tier) ? config.tier : self.class.default_tier
end

#offering_transport ⇒ Object

── Offering defaults ─────────────────────────────────────────────



626
627
628
 
# File 'lib/legion/extensions/llm/provider.rb', line 626

def offering_transport
  config.respond_to?(:transport) ? config.transport : self.class.default_transport
end

#paint(prompt, model:, size:, with: nil, mask: nil, params: {}) ⇒ Object

rubocop:disable Metrics/ParameterLists



283
284
285
286
287
288
289
 
# File 'lib/legion/extensions/llm/provider.rb', line 283

def paint(prompt, model:, size:, with: nil, mask: nil, params: {}) # rubocop:disable Metrics/ParameterLists
  enforce_model_allowed!(model)
  validate_paint_inputs!(with:, mask:)
  payload = render_image_payload(prompt, model:, size:, with:, mask:, params:)
  response = @connection.post images_url(with:, mask:), payload
  parse_image_response(response, model:)
end

#parse_error(response) ⇒ Object 



423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
 
# File 'lib/legion/extensions/llm/provider.rb', line 423

def parse_error(response)
  return if response.body.empty?

  body = try_parse_json(response.body)
  case body
  when Hash
    error = body['error']
    return error if error.is_a?(String)

    body.dig('error', 'message')
  when Array
    body.map do |part|
      error = part['error']
      error.is_a?(String) ? error : part.dig('error', 'message')
    end.join('. ')
  when String
    body[/"message"\s*:\s*"([^"]{1,500})/, 1] || body
  else
    body
  end
end

#parse_tool_calls(_tool_calls) ⇒ Object 



458
459
460
 
# File 'lib/legion/extensions/llm/provider.rb', line 458

def parse_tool_calls(_tool_calls)
  nil
end

#provider_instance_id ⇒ Object 



731
732
733
734
735
 
# File 'lib/legion/extensions/llm/provider.rb', line 731

def provider_instance_id
  return config.instance_id.to_sym if config.respond_to?(:instance_id) && config.instance_id

  :default
end

#publish_discovered_model_to_registry(model, provider_health:, live:) ⇒ Object 



213
214
215
216
217
218
219
220
 
# File 'lib/legion/extensions/llm/provider.rb', line 213

def publish_discovered_model_to_registry(model, provider_health:, live:)
  publisher = discovery_registry_publisher
  return unless publisher.respond_to?(:publish_models_async)

  publisher.publish_models_async([model], readiness: discovery_registry_readiness(provider_health, live:))
rescue StandardError => e
  handle_exception(e, level: :warn, handled: true, operation: 'llm.provider.publish_discovered_model')
end

#readiness(live: false) ⇒ Object 



390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
 
# File 'lib/legion/extensions/llm/provider.rb', line 390

def readiness(live: false)
  metadata = {
    provider: slug.to_sym,
    name: name,
    configured: configured?,
    ready: configured?,
    local: local?,
    remote: remote?,
    api_base: api_base,
    endpoints: endpoint_manifest,
    live: live
  }

  return metadata.merge(health: { checked: false }) unless live && metadata[:endpoints][:health]

  response = @connection.get(metadata[:endpoints][:health])
  metadata.merge(ready: configured? && health_ready?(response.body), health: response.body)
rescue StandardError => e
  handle_exception(e, level: :warn, handled: true, operation: 'llm.provider.readiness')
  metadata.merge(ready: false, health: { error: e.class.name, message: e.message })
end

#remote? ⇒ Boolean

Returns:

  • (Boolean) 


382
383
384
 
# File 'lib/legion/extensions/llm/provider.rb', line 382

def remote?
  self.class.remote?
end

#resolve_base_url ⇒ Object

── Multi-host base_url resolution ────────────────────────────────



636
637
638
639
640
641
 
# File 'lib/legion/extensions/llm/provider.rb', line 636

def resolve_base_url
  urls = Array(config_base_url)
  return nil if urls.empty?

  @resolve_base_url ||= find_reachable_url(urls) || normalize_url(urls.first)
end

#runtime_provider_setting(key) ⇒ Object

Provider-level setting: extensions.llm..



502
503
504
505
506
507
508
509
510
511
512
513
514
515
 
# File 'lib/legion/extensions/llm/provider.rb', line 502

def runtime_provider_setting(key)
  return nil unless defined?(Legion::Settings)

  ext = Legion::Settings[:extensions]
  return nil unless ext.is_a?(Hash) && ext[:llm].is_a?(Hash)

  provider_key = self.class.respond_to?(:slug) ? self.class.slug.to_sym : nil
  return nil unless provider_key

  provider_conf = ext[:llm][provider_key]
  provider_conf.is_a?(Hash) ? provider_conf[key] : nil
rescue StandardError
  nil
end

#slug ⇒ Object 



119
120
121
 
# File 'lib/legion/extensions/llm/provider.rb', line 119

def slug
  self.class.slug
end

#speak(text, model:, voice: nil, **provider_options) ⇒ Object

Raises:

  • (NotImplementedError) 


320
321
322
323
 
# File 'lib/legion/extensions/llm/provider.rb', line 320

def speak(text, model:, voice: nil, **provider_options)
  _ = [text, model, voice, provider_options]
  raise NotImplementedError, "#{self.class} does not implement speak"
end

#stream_chat(messages:, model:, tools: [], temperature: nil, params: {}, headers: {}, schema: nil, thinking: nil, tool_prefs: nil) ⇒ Object 



141
142
143
144
 
# File 'lib/legion/extensions/llm/provider.rb', line 141

def stream_chat(messages:, model:, tools: [], temperature: nil, params: {}, headers: {}, schema: nil,
                thinking: nil, tool_prefs: nil, &)
  complete(messages, tools:, temperature:, model:, params:, headers:, schema:, thinking:, tool_prefs:, &)
end

#strip_scheme(url) ⇒ Object 



663
664
665
 
# File 'lib/legion/extensions/llm/provider.rb', line 663

def strip_scheme(url)
  url.to_s.sub(%r{^https?://}, '')
end

#tls_enabled? ⇒ Boolean

Returns:

  • (Boolean) 


678
679
680
681
 
# File 'lib/legion/extensions/llm/provider.rb', line 678

def tls_enabled?
  tls = respond_to?(:settings) ? settings[:tls] : nil
  tls.is_a?(Hash) && tls[:enabled] == true
end

#transcribe(audio_file, model:, language:) ⇒ Object 



303
304
305
306
307
308
 
# File 'lib/legion/extensions/llm/provider.rb', line 303

def transcribe(audio_file, model:, language:, **)
  file_part = build_audio_file_part(audio_file)
  payload = render_transcription_payload(file_part, model:, language:, **)
  response = @connection.post transcription_url, payload
  parse_transcription_response(response, model:)
end

#translate(audio_file, model:, language:, **provider_options) ⇒ Object

Fail-loud base audio operations. Unsupported providers inherit these and publish OperationEvidence(status: :unsupported) or :unknown; a provider may publish :supported only when its Phase 2 conformance spec exercises the actual callable path. Neither method reads configuration or infers a model. See phase-1-lex-llm-additive.md section 14.1.

Raises:

  • (NotImplementedError) 


315
316
317
318
 
# File 'lib/legion/extensions/llm/provider.rb', line 315

def translate(audio_file, model:, language:, **provider_options)
  _ = [audio_file, model, language, provider_options]
  raise NotImplementedError, "#{self.class} does not implement translate"
end

#url_reachable?(url) ⇒ Boolean

Returns:

  • (Boolean) 


667
668
669
670
671
672
673
674
675
676
 
# File 'lib/legion/extensions/llm/provider.rb', line 667

def url_reachable?(url)
  require 'uri'
  require 'socket'
  uri = URI.parse(url)
  Socket.tcp(uri.host, uri.port, connect_timeout: 1).close
  true
rescue StandardError => e
  handle_exception(e, level: :warn, handled: true, operation: 'llm.provider.url_reachable', url:)
  false
end