Class: Legion::Extensions::Llm::Bedrock::Actor::BedrockCallable

Inherits:
Object
  • Object
show all
Defined in:
lib/legion/extensions/llm/bedrock/callable.rb

Overview

Per-instance dispatch callable for a Bedrock provider instance.

Wraps a per-instance Bedrock::Provider built from the instance config and implements the fleet dispatch operations the coordinator invokes (chat, stream_chat, embed, count_tokens) with ** passthrough, plus the disconnect / normalize_dispatch_error(error:) contracts required by Inventory::CallableHandle and Routing::ProviderOutcome. Provider and AWS SDK errors propagate unchanged so normalize_dispatch_error can classify them.

Instance Method Summary collapse

Constructor Details

#initialize(instance_cfg:, logger:) ⇒ BedrockCallable

Returns a new instance of BedrockCallable.



24
25
26
27
28
29
30
31
 
# File 'lib/legion/extensions/llm/bedrock/callable.rb', line 24

def initialize(instance_cfg:, logger:)
  @instance_cfg = instance_cfg
  @logger = logger
  @provider = nil
  @disconnected = false
  @dispatch_mutex = Mutex.new
  @dispatch_count = 0
end

Instance Method Details

#chat(messages:, model:, temperature: nil, max_tokens: nil, tools: {}, tool_prefs: nil, thinking: nil, params: {}, **opts) ⇒ Object 



48
49
50
51
52
53
54
 
# File 'lib/legion/extensions/llm/bedrock/callable.rb', line 48

def chat(messages:, model:, temperature: nil, max_tokens: nil, tools: {}, tool_prefs: nil,
         thinking: nil, params: {}, **opts)
  dispatch! do
    provider.chat(messages: messages, model: model, temperature: temperature, max_tokens: max_tokens,
                  tools: tools, tool_prefs: tool_prefs, thinking: thinking, params: params.merge(opts))
  end
end

#count_tokens(messages:, model:, system: nil, params: {}, **opts) ⇒ Object 



69
70
71
72
73
 
# File 'lib/legion/extensions/llm/bedrock/callable.rb', line 69

def count_tokens(messages:, model:, system: nil, params: {}, **opts)
  dispatch! do
    provider.count_tokens(messages: messages, model: model, system: system, params: params.merge(opts))
  end
end

#disconnect ⇒ Object 



41
42
43
44
45
46
 
# File 'lib/legion/extensions/llm/bedrock/callable.rb', line 41

def disconnect
  @disconnected = true
  @provider&.disconnect
  @provider = nil
  @logger.debug { '[bedrock][callable] disconnected' }
end

#disconnected? ⇒ Boolean

Returns:

  • (Boolean) 


33
34
35
 
# File 'lib/legion/extensions/llm/bedrock/callable.rb', line 33

def disconnected?
  @disconnected
end

#dispatch_count ⇒ Object 



37
38
39
 
# File 'lib/legion/extensions/llm/bedrock/callable.rb', line 37

def dispatch_count
  @dispatch_mutex.synchronize { @dispatch_count }
end

#embed(text:, model:, dimensions: nil, params: {}, **opts) ⇒ Object 



65
66
67
 
# File 'lib/legion/extensions/llm/bedrock/callable.rb', line 65

def embed(text:, model:, dimensions: nil, params: {}, **opts)
  dispatch! { provider.embed(text: text, model: model, dimensions: dimensions, params: params.merge(opts)) }
end

#normalize_dispatch_error(error:) ⇒ Object 



75
76
77
78
79
80
81
82
83
84
 
# File 'lib/legion/extensions/llm/bedrock/callable.rb', line 75

def normalize_dispatch_error(error:)
  reason = error.message.to_s[0, 512]

  kind = classify_error(error: error)

  Legion::Extensions::Llm::Routing::ProviderOutcome.new(
    kind: kind,
    reason: reason.empty? ? 'unknown dispatch error' : reason
  )
end

#stream_chat(messages:, model:, temperature: nil, max_tokens: nil, tools: {}, tool_prefs: nil, thinking: nil, params: {}, **opts) ⇒ Object 



56
57
58
59
60
61
62
63
 
# File 'lib/legion/extensions/llm/bedrock/callable.rb', line 56

def stream_chat(messages:, model:, temperature: nil, max_tokens: nil, tools: {}, tool_prefs: nil,
                thinking: nil, params: {}, **opts, &)
  dispatch! do
    provider.stream(messages: messages, model: model, temperature: temperature, max_tokens: max_tokens,
                    tools: tools, tool_prefs: tool_prefs, thinking: thinking,
                    params: params.merge(opts), &)
  end
end