Class: Legion::Extensions::Llm::Bedrock::Helpers::Callable

Inherits:
Object
  • Object
show all
Defined in:
lib/legion/extensions/llm/bedrock/helpers/callable.rb

Overview

Per-instance dispatch callable for a Bedrock provider instance.

Wraps a per-instance Bedrock::Provider built from the instance config and implements the 0.8.0 fleet dispatch operations the coordinator invokes (chat, stream_chat, embed, count_tokens) plus the disconnect / normalize_dispatch_error(error:) contracts required by Inventory::CallableHandle and Routing::ProviderOutcome.

0.8.0 callable boundary (WorkerExecution.dispatch_operation): chat/stream_chat take messages POSITIONALLY (base Provider#chat form); count_tokens takes the messages: kwarg. temperature and max_tokens are Canonical::Params members (05 O4), never named completion keys — the folded wire params become a Canonical::Params at this boundary, because the 0.8.0 renderer reads params.temperature / params.max_tokens (a raw Hash would NoMethodError). Provider and AWS SDK errors propagate unchanged so normalize_dispatch_error can classify them.

Constant Summary collapse

COMPLETION_NAMED_KEYS =

Keys the base Provider exposes as named kwargs for the completion operations. Anything else the fleet passes (sampling scalars, temperature — a Canonical::Params member, 05 O4) is folded into Canonical::Params at the dispatch boundary.

%i[tools tool_prefs thinking].freeze
COUNT_TOKENS_NAMED_KEYS = 
%i[system].freeze
EMBED_NAMED_KEYS = 
%i[dimensions].freeze

Instance Method Summary collapse

Constructor Details

#initialize(instance_cfg:, logger:) ⇒ Callable

Returns a new instance of Callable.



40
41
42
43
44
45
46
47
 
# File 'lib/legion/extensions/llm/bedrock/helpers/callable.rb', line 40

def initialize(instance_cfg:, logger:)
  @instance_cfg = instance_cfg
  @logger = logger
  @provider = nil
  @disconnected = false
  @dispatch_mutex = Mutex.new
  @dispatch_count = 0
end

Instance Method Details

#chat(messages, model:, **rest) ⇒ Object 



64
65
66
67
68
69
70
71
72
73
 
# File 'lib/legion/extensions/llm/bedrock/helpers/callable.rb', line 64

def chat(messages, model:, **rest)
  dispatch! do
    # Canonical boundary (N x N law): pipeline dispatch delivers
    # Canonical::Message objects only. Hash/legacy shapes are the
    # bypass class — reject loudly, never coerce.
    provider.enforce_canonical_messages!(messages)
    named, params = split_fleet_kwargs(rest, COMPLETION_NAMED_KEYS)
    provider.chat(messages: messages, model: model, params: canonical_params(params), **named)
  end
end

#count_tokens(messages:, model:, **rest) ⇒ Object

The fleet WorkerExecution calls count_tokens with the messages: KWARG (worker_execution.rb) — only chat/stream_chat are positional.



96
97
98
99
100
101
102
 
# File 'lib/legion/extensions/llm/bedrock/helpers/callable.rb', line 96

def count_tokens(messages:, model:, **rest)
  dispatch! do
    provider.enforce_canonical_messages!(messages)
    named, _params = split_fleet_kwargs(rest, COUNT_TOKENS_NAMED_KEYS)
    provider.count_tokens(messages: messages, model: model, **named)
  end
end

#disconnect ⇒ Object 



57
58
59
60
61
62
 
# File 'lib/legion/extensions/llm/bedrock/helpers/callable.rb', line 57

def disconnect
  @disconnected = true
  @provider&.disconnect
  @provider = nil
  @logger.debug { '[bedrock][callable] disconnected' }
end

#disconnected? ⇒ Boolean

Returns:

  • (Boolean) 


49
50
51
 
# File 'lib/legion/extensions/llm/bedrock/helpers/callable.rb', line 49

def disconnected?
  @disconnected
end

#dispatch_count ⇒ Object 



53
54
55
 
# File 'lib/legion/extensions/llm/bedrock/helpers/callable.rb', line 53

def dispatch_count
  @dispatch_mutex.synchronize { @dispatch_count }
end

#embed(text:, model:, **rest) ⇒ Object

B19: unowned fleet params are not forwarded — the exact execution binding carries no payload the operation does not own (the old params passthrough reached the InvokeModel body silently). Only the operation's named keys cross the boundary.



87
88
89
90
91
92
 
# File 'lib/legion/extensions/llm/bedrock/helpers/callable.rb', line 87

def embed(text:, model:, **rest)
  dispatch! do
    named, _params = split_fleet_kwargs(rest, EMBED_NAMED_KEYS)
    provider.embed(text: text, model: model, **named)
  end
end

#normalize_dispatch_error(error:) ⇒ Object 



104
105
106
107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/legion/extensions/llm/bedrock/helpers/callable.rb', line 104

def normalize_dispatch_error(error:)
  # B9: the base reason policy (10 §1E) — the bounded exception
  # CLASS NAME; never a response body, credential, endpoint, or
  # exception message (AWS SDK messages embed request context).
  reason = error.class.name
  reason = 'UnknownError' if reason.nil? || reason.empty?

  kind = classify_error(error: error)

  Legion::Extensions::Llm::Routing::ProviderOutcome.new(
    kind: kind,
    reason:
  )
end

#stream_chat(messages, model:, **rest) ⇒ Object 



75
76
77
78
79
80
81
 
# File 'lib/legion/extensions/llm/bedrock/helpers/callable.rb', line 75

def stream_chat(messages, model:, **rest, &)
  dispatch! do
    provider.enforce_canonical_messages!(messages)
    named, params = split_fleet_kwargs(rest, COMPLETION_NAMED_KEYS)
    provider.stream(messages: messages, model: model, params: canonical_params(params), **named, &)
  end
end