Class: Legion::Extensions::Llm::Vllm::Helpers::Callable

Inherits:
Object
  • Object
show all
Defined in:
lib/legion/extensions/llm/vllm/helpers/callable.rb

Overview

Callable wrapper for a vLLM provider instance. Implements the disconnect and normalize_dispatch_error(error:) contracts required by Inventory::CallableHandle and Routing::ProviderOutcome, plus the fleet dispatch operations the coordinator invokes (chat, stream_chat, embed, count_tokens). Each dispatch delegates to a per-instance Vllm::Provider built lazily from the instance config; provider and Faraday errors are NOT rescued here so the coordinator's normalize_dispatch_error can classify them.

Constant Summary collapse

COMPLETION_NAMED_KEYS =

Keys the base Provider exposes as named kwargs for the completion operations. Anything else the fleet passes is folded into the payload params hash (temperature is a Canonical::Params member, never a named kwarg — 05 O4).

%i[tools schema thinking tool_prefs headers].freeze
EMBED_NAMED_KEYS = 
%i[dimensions headers].freeze

Instance Method Summary collapse

Constructor Details

#initialize(instance_cfg:, logger:) ⇒ Callable

Returns a new instance of Callable.



27
28
29
30
31
32
 
# File 'lib/legion/extensions/llm/vllm/helpers/callable.rb', line 27

def initialize(instance_cfg:, logger:)
  @instance_cfg = instance_cfg
  @logger = logger
  @disconnected = false
  @inference_calls = 0
end

Instance Method Details

#call_count ⇒ Object 



34
35
36
 
# File 'lib/legion/extensions/llm/vllm/helpers/callable.rb', line 34

def call_count
  @inference_calls
end

#chat(messages, model:, **rest) ⇒ Object

0.8.0 callable contract: chat/stream_chat take the canonical messages positionally (the fleet dispatch and the conformance kit both call callable.chat(messages, model:, ...)); count_tokens and embed keep their keyword forms.



54
55
56
57
58
59
60
61
62
 
# File 'lib/legion/extensions/llm/vllm/helpers/callable.rb', line 54

def chat(messages, model:, **rest)
  record_inference
  # Canonical boundary (N x N law): pipeline dispatch delivers
  # Canonical::Message objects only. Hash/legacy shapes are the
  # bypass class — reject loudly, never coerce.
  provider.enforce_canonical_messages!(messages)
  named, params = split_fleet_kwargs(rest, COMPLETION_NAMED_KEYS)
  provider.chat(messages, model: model, params: canonical_params(params), **named)
end

#count_tokens(messages:, model:, **rest) ⇒ Object 



77
78
79
80
81
82
 
# File 'lib/legion/extensions/llm/vllm/helpers/callable.rb', line 77

def count_tokens(messages:, model:, **rest)
  record_inference
  provider.enforce_canonical_messages!(messages)
  _named, params = split_fleet_kwargs(rest, [])
  provider.count_tokens(messages: messages, model: model, params: params)
end

#disconnect ⇒ Object 



42
43
44
45
46
 
# File 'lib/legion/extensions/llm/vllm/helpers/callable.rb', line 42

def disconnect
  @disconnected = true
  @provider&.disconnect
  @logger.debug { '[vllm][callable] disconnected' }
end

#disconnected? ⇒ Boolean

Returns:

  • (Boolean) 


38
39
40
 
# File 'lib/legion/extensions/llm/vllm/helpers/callable.rb', line 38

def disconnected?
  @disconnected
end

#embed(text:, model:, **rest) ⇒ Object 



71
72
73
74
75
 
# File 'lib/legion/extensions/llm/vllm/helpers/callable.rb', line 71

def embed(text:, model:, **rest)
  record_inference
  named, params = split_fleet_kwargs(rest, EMBED_NAMED_KEYS)
  provider.embed(text: text, model: model, params: params, **named)
end

#normalize_dispatch_error(error:) ⇒ Object

V5: the reason is the bounded exception CLASS name (the base contract) — never the message, which embeds endpoint URLs and provider error bodies that must not reach shared state. The kind override stays: vLLM's explicit offline body phrases are stronger evidence than a raw status (the base law that raw 503 alone never manufactures unavailability is preserved).



90
91
92
93
94
95
 
# File 'lib/legion/extensions/llm/vllm/helpers/callable.rb', line 90

def normalize_dispatch_error(error:)
  kind = classify_dispatch_error(error: error)
  reason = error.class.name
  reason = 'UnknownError' if reason.nil? || reason.empty?
  Legion::Extensions::Llm::Routing::ProviderOutcome.new(kind: kind, reason: reason)
end

#stream_chat(messages, model:, **rest) ⇒ Object 



64
65
66
67
68
69
 
# File 'lib/legion/extensions/llm/vllm/helpers/callable.rb', line 64

def stream_chat(messages, model:, **rest, &)
  record_inference
  provider.enforce_canonical_messages!(messages)
  named, params = split_fleet_kwargs(rest, COMPLETION_NAMED_KEYS)
  provider.stream_chat(messages, model: model, params: canonical_params(params), **named, &)
end