Module: Legion::Extensions::Llm::Vllm::Runners::DiscoveryRefresh::Http

Included in:
Legion::Extensions::Llm::Vllm::Runners::DiscoveryRefresh
Defined in:
lib/legion/extensions/llm/vllm/runners/discovery_refresh/http.rb

Overview

Offering discovery, health probing, and instance-identity helpers for the vLLM discovery runner. Mixed into DiscoveryRefresh.

Instance Method Summary collapse

Instance Method Details

#check_health(instance_cfg:) ⇒ Object

── Health check (non-inference readiness) ─────────────────────



48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
 
# File 'lib/legion/extensions/llm/vllm/runners/discovery_refresh/http.rb', line 48

def check_health(instance_cfg:)
  base_url = normalize_api_base(instance_cfg[:vllm_api_base] || instance_cfg[:endpoint])
  conn = build_health_connection(base_url: base_url, instance_cfg: instance_cfg)
  response = conn.get('/health')
  Legion::Extensions::Llm::Inventory::ReadinessResult.new(
    ready: response.status == 200,
    reason: "vLLM /health returned #{response.status}",
    metadata: { status: response.status, base_url: base_url }
  )
rescue Faraday::ConnectionFailed => e
  handle_exception(e, level: :warn, handled: true, operation: 'vllm.runner.discovery.health',
                      base_url: base_url)
  readiness_failure(reason: "vLLM /health connection failed: #{e.message}", error: e)
rescue StandardError => e
  raise e if discovery_programming_error?(e)

  handle_exception(e, level: :warn, handled: true, operation: 'vllm.runner.discovery.health',
                      base_url: base_url)
  readiness_failure(reason: "vLLM /health error: #{e.message}", error: e)
end

#derive_physical_id(instance_cfg:) ⇒ Object

The derived host:port(/ak:) string is the SECONDARY physical identity — dedup/diagnostics only. The instance identity is the operator's config name (InstanceKey.instance_id); two config names sharing an endpoint stay distinct instances.



90
91
92
93
94
95
96
97
98
99
 
# File 'lib/legion/extensions/llm/vllm/runners/discovery_refresh/http.rb', line 90

def derive_physical_id(instance_cfg:)
  base_url = instance_cfg[:vllm_api_base] || instance_cfg[:endpoint]
  host_port = extract_host_port(url: normalize_api_base(base_url))
  api_key = instance_cfg[:vllm_api_key]
  if api_key.is_a?(String) && !api_key.strip.empty?
    return "#{host_port}/ak:#{::Digest::SHA256.hexdigest(api_key)[0, 6]}"
  end

  host_port
end

#discovery_programming_error?(error) ⇒ Boolean

D16: a programming bug in the discovery path must fail loud — swallowing it publishes ZERO offerings (or a false "instance down") and makes an activated instance invisible to the coordinator. NoMethodError is a NameError subclass.

Returns:

  • (Boolean) 


73
74
75
 
# File 'lib/legion/extensions/llm/vllm/runners/discovery_refresh/http.rb', line 73

def discovery_programming_error?(error)
  error.is_a?(NameError) || error.is_a?(ArgumentError)
end

#extract_host_port(url:) ⇒ Object 



101
102
103
104
105
106
107
108
 
# File 'lib/legion/extensions/llm/vllm/runners/discovery_refresh/http.rb', line 101

def extract_host_port(url:)
  uri = URI.parse(url.to_s)
  "#{uri.host || 'localhost'}:#{uri.port}"
rescue URI::InvalidURIError => e
  handle_exception(e, level: :warn, handled: true,
                      operation: 'vllm.runner.discovery.extract_host_port', url: url.to_s)
  raise
end

#fetch_models(instance_cfg:) ⇒ Object 



40
41
42
43
44
 
# File 'lib/legion/extensions/llm/vllm/runners/discovery_refresh/http.rb', line 40

def fetch_models(instance_cfg:)
  base_url = normalize_api_base(instance_cfg[:vllm_api_base] || instance_cfg[:endpoint])
  conn = build_catalog_connection(base_url: base_url, instance_cfg: instance_cfg)
  Legion::JSON.load(conn.get('/v1/models').body).fetch(:data, [])
end

#fetch_offerings(instance_cfg:, instance_key:) ⇒ Object

── Offering discovery ─────────────────────────────────────────



19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
 
# File 'lib/legion/extensions/llm/vllm/runners/discovery_refresh/http.rb', line 19

def fetch_offerings(instance_cfg:, instance_key:)
  models = fetch_models(instance_cfg: instance_cfg)
  builder = Legion::Extensions::Llm::Vllm::Helpers::OfferingBuilder.new(
    instance_cfg: instance_cfg, instance_key: instance_key
  )
  models.filter_map do |model_data|
    model_id = model_data[:id].to_s
    next if model_id.empty?

    builder.build(model_id: model_id, model_data: model_data)
  end
rescue StandardError => e
  raise e if discovery_programming_error?(e)

  # Network/runtime/parse errors (Faraday, timeout, JSON,
  # malformed model data) mean the instance has no catalog right
  # now — an empty set is the correct availability fact.
  handle_exception(e, level: :warn, operation: 'vllm.runner.discovery.discover_offerings')
  []
end

#normalize_api_base(url) ⇒ Object

Strip a trailing /v1 from an API base. Deliberately has NO localhost fallback — an instance with no resolvable endpoint is skipped by discovery rather than silently claimed at localhost.



113
114
115
 
# File 'lib/legion/extensions/llm/vllm/runners/discovery_refresh/http.rb', line 113

def normalize_api_base(url)
  url.to_s.sub(%r{/v1/?\z}, '')
end

#readiness_failure(reason:, error:) ⇒ Object 



77
78
79
80
81
82
 
# File 'lib/legion/extensions/llm/vllm/runners/discovery_refresh/http.rb', line 77

def readiness_failure(reason:, error:)
  Legion::Extensions::Llm::Inventory::ReadinessResult.new(
    ready: false, reason: reason,
    metadata: { error_class: error.class.name }
  )
end