Class: Legion::Extensions::Llm::Vllm::Provider

Inherits:
Provider
  • Object
show all
Includes:
Provider::OpenAICompatible, ProviderCanonicalRequestBridge, ProviderManagementMethods, ProviderRenderParseHelpers, Logging::Helper
Defined in:
lib/legion/extensions/llm/vllm/provider.rb

Overview

vLLM provider implementation for the Legion::Extensions::Llm base provider contract.

Defined Under Namespace

Modules: Capabilities

Class Method Summary collapse

Instance Method Summary collapse

Methods included from ProviderManagementMethods

#reset_mm_cache, #reset_prefix_cache, #sleep, #version, #wake_up

Class Method Details

.capabilities ⇒ Object 



199
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 199

def capabilities = Capabilities

.configuration_options ⇒ Object 



193
194
195
196
197
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 193

def configuration_options = %i[vllm_api_base vllm_api_key]
# V6: an explicit endpoint is what makes a vLLM instance
# executable — construction fails without it (the base
# ensure_configured! / configured? contract), and there is no
# localhost or sibling-endpoint fallback anywhere.

.configuration_requirements ⇒ Object

V6: an explicit endpoint is what makes a vLLM instance executable — construction fails without it (the base ensure_configured! / configured? contract), and there is no localhost or sibling-endpoint fallback anywhere.



198
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 198

def configuration_requirements = %i[vllm_api_base]

.default_tier ⇒ Object 



192
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 192

def default_tier = :direct

.default_transport ⇒ Object 



191
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 191

def default_transport = :http

.local? ⇒ Boolean

Returns:

  • (Boolean) 


190
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 190

def local? = false

.slug ⇒ Object 



189
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 189

def slug = 'vllm'

Instance Method Details

#api_base ⇒ Object

V6: the instance's OWN explicit endpoint only. The gem-template instances.default fallback is deleted — an instance with no resolvable endpoint is unexecutable, exactly as the discovery path already treats it (skip; never claim at localhost or a sibling instance's endpoint).



249
250
251
252
253
254
255
256
257
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 249

def api_base
  base = config.vllm_api_base
  if base.nil? || base.to_s.strip.empty?
    raise Legion::Extensions::Llm::ConfigurationError,
          'vllm instance has no vllm_api_base — an instance without an explicit endpoint is unexecutable'
  end

  normalize_url(base)
end

#headers ⇒ Object 



259
260
261
262
263
264
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 259

def headers
  hdrs = identity_headers
  token = config.vllm_api_key
  hdrs['Authorization'] = "Bearer #{token}" unless token.nil? || token.to_s.empty?
  hdrs
end

#health(live: false) ⇒ Object 



273
274
275
276
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 273

def health(live: false)
  log.info { "checking health live=#{live} at #{api_base}#{health_url}" }
  super
end

#health_url ⇒ Object 



266
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 266

def health_url = '/health'

#reset_mm_cache_url ⇒ Object 



269
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 269

def reset_mm_cache_url = '/reset_mm_cache'

#reset_prefix_cache_url ⇒ Object 



268
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 268

def reset_prefix_cache_url = '/reset_prefix_cache'

#sleep_url ⇒ Object 



270
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 270

def sleep_url = '/sleep'

#stream_usage_supported? ⇒ Boolean

Returns:

  • (Boolean) 


238
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 238

def stream_usage_supported? = true

#translator ⇒ Object 



240
241
242
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 240

def translator
  @translator ||= Translator.new(config: config)
end

#version_url ⇒ Object 



267
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 267

def version_url = '/version'

#wake_up_url ⇒ Object 



271
 
# File 'lib/legion/extensions/llm/vllm/provider.rb', line 271

def wake_up_url = '/wake_up'