Class: Riffer::Providers::AmazonBedrock

Inherits:
Base
  • Object
show all
Defined in:
lib/riffer/providers/amazon_bedrock.rb,
sig/_private/riffer/providers/amazon_bedrock.rbs,
sig/generated/riffer/providers/amazon_bedrock.rbs

Overview

Amazon Bedrock provider for Claude and other foundation models. Requires the aws-sdk-bedrockruntime gem.

Constant Summary collapse

ANTHROPIC_MODEL_PATTERN =

Matches Anthropic models on Bedrock — bare (+anthropic.claude-...+) and cross-region (+us.anthropic.claude-...+) ids.

Returns:

  • (Regexp) 
/(?:^|\.)anthropic\./
FINISH_REASONS =

Returns:

  • (Hash[String, Symbol]) 
{
  "end_turn" => :stop,
  "stop_sequence" => :stop,
  "max_tokens" => :length,
  "tool_use" => :tool_calls,
  "guardrail_intervened" => :content_filter,
  "content_filtered" => :content_filter
}.freeze
BEDROCK_FORMAT_MAP =

Returns:

  • (Hash[String, String]) 
{
  "image/jpeg" => "jpeg",
  "image/png" => "png",
  "image/gif" => "gif",
  "image/webp" => "webp",
  "application/pdf" => "pdf",
  "text/plain" => "txt",
  "text/csv" => "csv",
  "text/html" => "html"
}.freeze

Constants inherited from Base

Base::REQUEST_PARAM_ATTRIBUTES, Base::WIRE_SEPARATOR

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Base

#apply_pricing, #capture_input, #capture_messages?, #capture_output, #chat_span_attributes, #decode_tool_name, #depends_on, #encode_tool_name, #generate_text, #in_chat_span, #merge_consecutive_messages, #normalize_messages, #parse_structured_output, #parse_tool_arguments, #pricing_rates, #record_finish_reason, #record_stream_outcome, #stream_text, #tag_attributes, #validate_input!, #validate_normalized_messages!, #yield_finish_reason

Constructor Details

#initialize(api_token: nil, region: nil, **options) ⇒ AmazonBedrock

-- : (?api_token: String?, ?region: String?, **untyped) -> void

Parameters:

  • api_token: (String, nil) (defaults to: nil)
  • region: (String, nil) (defaults to: nil)
  • (Object) 


40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 40

def initialize(api_token: nil, region: nil, **options)
  depends_on "aws-sdk-bedrockruntime"

  api_token ||= Riffer.config.amazon_bedrock.api_token
  region ||= Riffer.config.amazon_bedrock.region

  @client = if api_token && !api_token.empty?
    Aws::BedrockRuntime::Client.new(
      region: region,
      token_provider: Aws::StaticTokenProvider.new(api_token),
      auth_scheme_preference: ["httpBearerAuth"],
      **options
    )
  else
    Aws::BedrockRuntime::Client.new(region: region, **options)
  end
end

Class Method Details

.semconv_provider_name ⇒ String

The GenAI semconv well-known provider name.

: () -> String

Returns:

  • (String) 


34
35
36
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 34

def self.semconv_provider_name
  "aws.bedrock"
end

.skills_adapter(model = nil) ⇒ singleton(Riffer::Skills::Adapter)

Returns the skill adapter for the Bedrock model — XML for Anthropic models (which Bedrock hosts alongside other vendors'), else Markdown.

: (?String?) -> singleton(Riffer::Skills::Adapter)

Parameters:

  • (String, nil)

Returns:



26
27
28
29
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 26

def self.skills_adapter(model = nil)
  return Riffer::Skills::XmlAdapter if model && ANTHROPIC_MODEL_PATTERN.match?(model)
  Riffer::Skills::MarkdownAdapter
end

Instance Method Details

#append_tool_result(conversation_messages, message) ⇒ void

This method returns an undefined value.

-- : (Array[Hash[Symbol, untyped]], Riffer::Messages::Tool) -> void

Parameters:



374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 374

def append_tool_result(conversation_messages, message)
  tool_result = {
    tool_result: {
      tool_use_id: message.tool_call_id,
      content: [{text: message.content}]
    }
  }

  prev = conversation_messages.last
  if prev && prev[:role] == "user" && prev[:content]&.first&.key?(:tool_result)
    prev[:content] << tool_result
  else
    conversation_messages << {role: "user", content: [tool_result]}
  end
end

#apply_cache_point(params, cache_control) ⇒ void

This method returns an undefined value.

Converse chains tools -> system -> messages, so a single cachePoint at the end of the system array (or the tools array, when there is no system prompt) also caches the preceding sections.

: (Hash[Symbol, untyped], untyped) -> void

Parameters:

  • (Hash[Symbol, untyped])
  • (Object) 


114
115
116
117
118
119
120
121
122
123
124
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 114

def apply_cache_point(params, cache_control)
  cache_point = {cache_point: build_cache_point(cache_control)}
  system = params[:system]
  tools = params.dig(:tool_config, :tools)

  if system && !system.empty?
    system << cache_point
  elsif tools && !tools.empty?
    tools << cache_point
  end
end

#bedrock_format(media_type) ⇒ String

-- : (String) -> String

Parameters:

  • (String)

Returns:

  • (String) 


423
424
425
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 423

def bedrock_format(media_type)
  BEDROCK_FORMAT_MAP.fetch(media_type)
end

#build_cache_point(cache_control) ⇒ Hash[Symbol, untyped]

-- : (untyped) -> Hash[Symbol, untyped]

Parameters:

  • (Object)

Returns:

  • (Hash[Symbol, untyped]) 


128
129
130
131
132
133
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 128

def build_cache_point(cache_control)
  point = {type: "default"} #: Hash[Symbol, untyped]
  ttl = cache_control.is_a?(Hash) ? cache_control[:ttl] : nil
  point[:ttl] = ttl if ttl
  point
end

#build_finish_reason(stop_reason) ⇒ Riffer::Providers::FinishReason?

-- : (untyped) -> Riffer::Providers::FinishReason?

Parameters:

  • (Object)

Returns:



173
174
175
176
177
178
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 173

def build_finish_reason(stop_reason)
  return nil unless stop_reason

  raw = stop_reason.to_s
  Riffer::Providers::FinishReason.new(reason: FINISH_REASONS.fetch(raw, :other), raw: raw)
end

#build_request_params(messages, model, options) ⇒ Hash[Symbol, untyped]

-- : (Array, String?, Hash[Symbol, untyped]) -> Hash[Symbol, untyped]

Parameters:

Returns:

  • (Hash[Symbol, untyped]) 


62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 62

def build_request_params(messages, model, options)
  partitioned_messages = partition_messages(messages)
  tools = options[:tools]
  structured_output = options[:structured_output]
  cache_control = options[:cache_control]
  tags = options[:tags] || {}

  params = {
    model_id: model,
    system: partitioned_messages[:system],
    messages: partitioned_messages[:conversation],
    **options.except(:tools, :structured_output, :cache_control, :tags)
  } #: Hash[Symbol, untyped]

  # requestMetadata is a flat String=>String map used to filter invocation
  # logs; every tag (including the reserved user_id) rides along, since
  # Converse has no dedicated end-user field.
  params[:request_metadata] = tags unless tags.empty?

  if tools && !tools.empty?
    params[:tool_config] = {
      tools: tools.map { |t| convert_tool_to_bedrock_format(t) }
    }
  end

  if structured_output
    # Use strict schema to make optional fields nullable. Without this,
    # Bedrock may return string literals like ": null," instead of actual
    # null values for optional fields that the model has no value for.
    params[:output_config] = {
      text_format: {
        type: "json_schema",
        structure: {
          json_schema: {
            schema: structured_output.json_schema(strict: true).to_json,
            name: "response"
          }
        }
      }
    }
  end

  apply_cache_point(params, cache_control) if cache_control

  params
end

#build_token_usage(usage) ⇒ Riffer::Providers::TokenUsage

Converse's input_tokens excludes the cache buckets; TokenUsage's input includes them.

: (untyped) -> Riffer::Providers::TokenUsage

Parameters:

  • (Object)

Returns:



152
153
154
155
156
157
158
159
160
161
162
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 152

def build_token_usage(usage)
  cache_write = usage.cache_write_input_tokens
  cache_read = usage.cache_read_input_tokens

  apply_pricing(Riffer::Providers::TokenUsage.new(
    input_tokens: usage.input_tokens + (cache_write || 0) + (cache_read || 0),
    output_tokens: usage.output_tokens,
    cache_write_tokens: cache_write,
    cache_read_tokens: cache_read
  ))
end

#convert_assistant_to_bedrock_format(message) ⇒ Hash[Symbol, untyped]

-- : (Riffer::Messages::Assistant) -> Hash[Symbol, untyped]

Parameters:

Returns:

  • (Hash[Symbol, untyped]) 


355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 355

def convert_assistant_to_bedrock_format(message)
  content = [] #: Array[Hash[Symbol, untyped]]
  content << {text: message.content} if message.content && !message.content.empty?

  message.tool_calls.each do |tc|
    content << {
      tool_use: {
        tool_use_id: tc.call_id,
        name: encode_tool_name(tc.name),
        input: parse_tool_arguments(tc.arguments)
      }
    }
  end

  {role: "assistant", content: content}
end

#convert_file_part_to_bedrock_format(file) ⇒ Hash[Symbol, untyped]

-- : (Riffer::Messages::FilePart) -> Hash[Symbol, untyped]

Parameters:

Returns:

  • (Hash[Symbol, untyped]) 


392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 392

def convert_file_part_to_bedrock_format(file)
  format = bedrock_format(file.media_type)

  source = if file.data
    {bytes: Base64.decode64(file.data)}
  elsif file.url&.start_with?("s3://")
    {s3_location: {uri: file.url}}
  else
    raise Riffer::ArgumentError, "Amazon Bedrock only supports S3 URI or base64 data file sources"
  end

  if file.image?
    {image: {format: format, source: source}}
  else
    {document: {format: format, name: file.filename, source: source}}
  end
end

#convert_tool_to_bedrock_format(tool) ⇒ Hash[Symbol, untyped]

-- : (singleton(Riffer::Tool)) -> Hash[Symbol, untyped]

Parameters:

Returns:

  • (Hash[Symbol, untyped]) 


429
430
431
432
433
434
435
436
437
438
439
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 429

def convert_tool_to_bedrock_format(tool)
  {
    tool_spec: {
      name: encode_tool_name(tool.name),
      description: tool.description,
      input_schema: {
        json: tool.parameters_schema(strict: true)
      }
    }
  }
end

#execute_generate(params) ⇒ Object

-- : (Hash[Symbol, untyped]) -> untyped

Parameters:

  • (Hash[Symbol, untyped])

Returns:

  • (Object) 


137
138
139
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 137

def execute_generate(params)
  @client.converse(**params)
end

#execute_stream(params, yielder) ⇒ void

This method returns an undefined value.

-- : (Hash[Symbol, untyped], Riffer::Providers::_EventSink) -> void

Parameters:



220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 220

def execute_stream(params, yielder)
  current_state = {
    text: nil,
    tool_call: nil
  } #: Hash[Symbol, untyped]

  @client.converse_stream(**params) do |stream|
    stream.on_event do |event|
      case event
      when Aws::BedrockRuntime::Types::ContentBlockStartEvent
        handle_content_block_start_tool_use(event, state: current_state, yielder: yielder) if event.start&.tool_use
      when Aws::BedrockRuntime::Types::ContentBlockDeltaEvent
        handle_content_block_delta_text_delta(event, state: current_state, yielder: yielder) if event.delta&.text
        handle_content_block_delta_tool_use(event, state: current_state, yielder: yielder) if event.delta&.tool_use
      when Aws::BedrockRuntime::Types::ContentBlockStopEvent
        handle_content_block_stop_text_delta(event, state: current_state, yielder: yielder) if current_state[:text]
        handle_content_block_stop_tool_use(event, state: current_state, yielder: yielder) if current_state[:tool_call]
      when Aws::BedrockRuntime::Types::MessageStopEvent
        yield_finish_reason(yielder, build_finish_reason(event.stop_reason))
      when Aws::BedrockRuntime::Types::ConverseStreamMetadataEvent
        handle_metadata_usage(event, state: current_state, yielder: yielder) if event.usage
      else
        raise_if_stream_exception!(event)
      end
    end
  end
end

#extract_content(response) ⇒ String

-- : (untyped) -> String

Parameters:

  • (Object)

Returns:

  • (String) 


182
183
184
185
186
187
188
189
190
191
192
193
194
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 182

def extract_content(response)
  typed_response = response #: Aws::BedrockRuntime::Client::_ConverseResponseSuccess
  content_blocks = typed_response.output&.message&.content
  return "" if content_blocks.nil? || content_blocks.empty?

  text_content = ""

  content_blocks.each do |block|
    text_content += block.text if block.respond_to?(:text) && block.text
  end

  text_content
end

#extract_finish_reason(response) ⇒ Riffer::Providers::FinishReason?

-- : (untyped) -> Riffer::Providers::FinishReason?

Parameters:

  • (Object)

Returns:



166
167
168
169
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 166

def extract_finish_reason(response)
  typed_response = response #: Aws::BedrockRuntime::Client::_ConverseResponseSuccess
  build_finish_reason(typed_response.stop_reason)
end

#extract_token_usage(response) ⇒ Riffer::Providers::TokenUsage?

-- : (untyped) -> Riffer::Providers::TokenUsage?

Parameters:

  • (Object)

Returns:



143
144
145
146
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 143

def extract_token_usage(response)
  typed_response = response #: Aws::BedrockRuntime::Client::_ConverseResponseSuccess
  build_token_usage(typed_response.usage)
end

#extract_tool_calls(response) ⇒ Array[Riffer::Messages::Assistant::ToolCall]

-- : (untyped) -> Array

Parameters:

  • (Object)

Returns:



198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 198

def extract_tool_calls(response)
  typed_response = response #: Aws::BedrockRuntime::Client::_ConverseResponseSuccess
  content_blocks = typed_response.output&.message&.content
  return [] if content_blocks.nil? || content_blocks.empty?

  tool_calls = [] #: Array[Riffer::Messages::Assistant::ToolCall]

  content_blocks.each do |block|
    if block.respond_to?(:tool_use) && block.tool_use
      tool_calls << Riffer::Messages::Assistant::ToolCall.new(
        call_id: block.tool_use.tool_use_id,
        name: decode_tool_name(block.tool_use.name, tools: @current_tools),
        arguments: block.tool_use.input.to_json
      )
    end
  end

  tool_calls
end

#handle_content_block_delta_text_delta(event, state:, yielder:) ⇒ void

This method returns an undefined value.

-- : (untyped, state: Hash[Symbol, untyped], yielder: Riffer::Providers::_EventSink) -> void

Parameters:



276
277
278
279
280
281
282
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 276

def handle_content_block_delta_text_delta(event, state:, yielder:)
  typed_event = event #: Aws::BedrockRuntime::Types::ContentBlockDeltaEvent
  delta_text = typed_event.delta.text
  state[:text] ||= ""
  state[:text] += delta_text
  yielder << Riffer::StreamEvents::TextDelta.new(delta_text)
end

#handle_content_block_delta_tool_use(event, state:, yielder:) ⇒ void

This method returns an undefined value.

-- : (untyped, state: Hash[Symbol, untyped], yielder: Riffer::Providers::_EventSink) -> void

Parameters:



286
287
288
289
290
291
292
293
294
295
296
297
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 286

def handle_content_block_delta_tool_use(event, state:, yielder:)
  typed_event = event #: Aws::BedrockRuntime::Types::ContentBlockDeltaEvent
  input_delta = typed_event.delta.tool_use.input

  state[:tool_call][:arguments] += input_delta

  yielder << Riffer::StreamEvents::ToolCallDelta.new(
    item_id: state[:tool_call][:id],
    name: state[:tool_call][:name],
    arguments_delta: input_delta
  )
end

#handle_content_block_start_tool_use(event, state:, yielder:) ⇒ void

This method returns an undefined value.

-- : (untyped, state: Hash[Symbol, untyped], yielder: Riffer::Providers::_EventSink) -> void

Parameters:



265
266
267
268
269
270
271
272
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 265

def handle_content_block_start_tool_use(event, state:, yielder:)
  typed_event = event #: Aws::BedrockRuntime::Types::ContentBlockStartEvent
  state[:tool_call] = {
    id: typed_event.start.tool_use.tool_use_id,
    name: decode_tool_name(typed_event.start.tool_use.name, tools: @current_tools),
    arguments: ""
  }
end

#handle_content_block_stop_text_delta(_event, state:, yielder:) ⇒ void

This method returns an undefined value.

-- : (untyped, state: Hash[Symbol, untyped], yielder: Riffer::Providers::_EventSink) -> void

Parameters:



301
302
303
304
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 301

def handle_content_block_stop_text_delta(_event, state:, yielder:)
  yielder << Riffer::StreamEvents::TextDone.new(state[:text])
  state[:text] = nil
end

#handle_content_block_stop_tool_use(_event, state:, yielder:) ⇒ void

This method returns an undefined value.

-- : (untyped, state: Hash[Symbol, untyped], yielder: Riffer::Providers::_EventSink) -> void

Parameters:



308
309
310
311
312
313
314
315
316
317
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 308

def handle_content_block_stop_tool_use(_event, state:, yielder:)
  tool_call = state[:tool_call]
  yielder << Riffer::StreamEvents::ToolCallDone.new(
    item_id: tool_call[:id],
    call_id: tool_call[:id],
    name: tool_call[:name],
    arguments: tool_call[:arguments]
  )
  state[:tool_call] = nil
end

#handle_metadata_usage(event, state:, yielder:) ⇒ void

This method returns an undefined value.

-- : (untyped, state: Hash[Symbol, untyped], yielder: Riffer::Providers::_EventSink) -> void

Parameters:



321
322
323
324
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 321

def handle_metadata_usage(event, state:, yielder:)
  typed_event = event #: Aws::BedrockRuntime::Types::ConverseStreamMetadataEvent
  yielder << Riffer::StreamEvents::TokenUsageDone.new(token_usage: build_token_usage(typed_event.usage))
end

#partition_messages(messages) ⇒ Hash[Symbol, untyped]

-- : (Array) -> Hash[Symbol, untyped]

Parameters:

Returns:

  • (Hash[Symbol, untyped]) 


328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 328

def partition_messages(messages)
  system_prompts = [] #: Array[Hash[Symbol, untyped]]
  conversation_messages = [] #: Array[Hash[Symbol, untyped]]

  messages.each do |message|
    case message
    when Riffer::Messages::System
      system_prompts << {text: message.content}
    when Riffer::Messages::User
      content = [{text: message.content}]
      message.files.each { |file| content << convert_file_part_to_bedrock_format(file) }
      conversation_messages << {role: "user", content: content}
    when Riffer::Messages::Assistant
      conversation_messages << convert_assistant_to_bedrock_format(message)
    when Riffer::Messages::Tool
      append_tool_result(conversation_messages, message)
    end
  end

  {
    system: system_prompts,
    conversation: conversation_messages
  }
end

#raise_if_stream_exception!(event) ⇒ void

This method returns an undefined value.

Re-raises a Bedrock stream-exception event as the matching Aws::BedrockRuntime::Errors class. ConverseStream delivers API errors on the same channel as content, so without this a mid-stream failure would silently end the stream with no content.

: (untyped) -> void

Parameters:

  • (Object) 


254
255
256
257
258
259
260
261
 
# File 'lib/riffer/providers/amazon_bedrock.rb', line 254

def raise_if_stream_exception!(event)
  klass_name = event.class.name&.split("::")&.last
  return unless klass_name&.end_with?("Exception")

  error_klass = Aws::BedrockRuntime::Errors.const_get(klass_name)
  context = Seahorse::Client::RequestContext.new(operation_name: :converse_stream)
  raise error_klass.new(context, event.message, event)
end