Class: LLM::Bedrock

Inherits:
Provider show all
Includes:
RequestAdapter
Defined in:
lib/llm/providers/bedrock.rb,
lib/llm/providers/bedrock/models.rb,
lib/llm/providers/bedrock/signature.rb,
lib/llm/providers/bedrock/error_handler.rb,
lib/llm/providers/bedrock/stream_parser.rb,
lib/llm/providers/bedrock/stream_decoder.rb,
lib/llm/providers/bedrock/request_adapter.rb,
lib/llm/providers/bedrock/response_adapter.rb

Overview

The Bedrock class implements a provider for [Amazon Bedrock](aws.amazon.com/bedrock/).

Bedrock provides access to foundation models from Anthropic, Meta, Mistral, AI21 Labs, Cohere, and more through the AWS infrastructure. This provider uses the Bedrock Converse API for chat completions, and the Converse Stream API for streaming.

Unlike other llm.rb providers which use API key authentication, Bedrock uses AWS Signature V4 (SigV4) for request signing. You must provide AWS credentials (access key, secret key, and region) instead of a single API key.

Streaming uses the AWS Event Stream binary protocol instead of standard SSE. The binary framing is decoded inline using only Ruby’s stdlib.

Examples:

require "llm"

llm = LLM.bedrock(
  access_key_id: ENV["AWS_ACCESS_KEY_ID"],
  secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"],
  region: "us-east-1"
)
ctx = LLM::Context.new(llm)
ctx.talk "Hello, how are you?"
ctx.messages.select(&:assistant?).each { puts _1.content }

Defined Under Namespace

Modules: RequestAdapter, ResponseAdapter Classes: ErrorHandler, Models, Signature, StreamDecoder, StreamParser

Constant Summary collapse

HOST_PATTERN = 
"bedrock-runtime.%s.amazonaws.com"

Instance Method Summary collapse

Methods included from RequestAdapter

#adapt

Methods inherited from Provider

#chat, #developer_role, #inspect, #interrupt!, #persist!, #request_owner, #respond, #schema, #server_tool, #server_tools, #streamable?, #system_role, #tracer, #tracer=, #user_role, #web_search, #with, #with_tracer

Constructor Details

#initialize(access_key_id: nil, secret_access_key: nil, region: nil, session_token: nil, host: nil, port: 443, ssl: true, timeout: 60) ⇒ Bedrock

Returns a new instance of Bedrock.

Parameters:

  • access_key_id (String) (defaults to: nil) —

    AWS access key ID

  • secret_access_key (String) (defaults to: nil) —

    AWS secret access key

  • region (String) (defaults to: nil) —

    AWS region (e.g. “us-east-1”)

  • session_token (String, nil) (defaults to: nil) —

    AWS session token for temporary credentials

  • host (String, nil) (defaults to: nil) —

    Override the Bedrock API host

  • port (Integer) (defaults to: 443) —

    Connection port

  • ssl (Boolean) (defaults to: true) —

    Whether to use SSL

  • timeout (Integer) (defaults to: 60) —

    Request timeout in seconds



58
59
60
61
62
63
64
65
66
67
68
69
70
 
# File 'lib/llm/providers/bedrock.rb', line 58

def initialize(access_key_id: nil, secret_access_key: nil,
               region: nil, session_token: nil,
               host: nil, port: 443, ssl: true, timeout: 60,
               **)
  region ||= "us-east-1"
  @access_key_id = access_key_id
  @secret_access_key = secret_access_key
  @aws_region = region
  @session_token = session_token
  host ||= HOST_PATTERN % region
  @aws_host = host
  super(key: @access_key_id, host:, port:, ssl:, timeout:, persistent: false)
end

Instance Method Details

#assistant_role ⇒ String

Returns:

  • (String) 


156
157
158
 
# File 'lib/llm/providers/bedrock.rb', line 156

def assistant_role
  "assistant"
end

#audio ⇒ Object

Raises:

  • (NotImplementedError) 


126
127
128
 
# File 'lib/llm/providers/bedrock.rb', line 126

def audio
  raise NotImplementedError
end

#complete(prompt, params = {}) ⇒ LLM::Response

Provides an interface to the Bedrock Converse API

Parameters:

  • prompt (String) —

    The input prompt to be completed

  • params (Hash) (defaults to: {}) —

    The parameters to maintain throughout the conversation. Any parameter the provider supports can be included and not only those listed here.

Returns:

See Also:



86
87
88
89
90
91
92
93
94
95
96
97
 
# File 'lib/llm/providers/bedrock.rb', line 86

def complete(prompt, params = {})
  params, stream, tools, role = normalize_complete_params(params)
  req, messages, body = build_complete_request(prompt, params, role, stream:)
  tracer.set_request_metadata(user_input: extract_user_input(messages, fallback: prompt))
  sign!(req, body)
  model_id = model_id_for(req.path)
  res, span, tracer = execute(request: req, stream:, operation: "chat", stream_parser:, model: model_id)
  res = ResponseAdapter.adapt(res, type: :completion)
    .extend(Module.new { define_method(:__tools__) { tools } })
  tracer.on_request_finish(operation: "chat", model: model_id, res:, span:)
  res
end

#default_model ⇒ String

Returns:

  • (String) 


170
171
172
 
# File 'lib/llm/providers/bedrock.rb', line 170

def default_model
  "deepseek.v3.2"
end

#embed(input, model: nil, **params) ⇒ Object

Raises:

  • (NotImplementedError) 


150
151
152
 
# File 'lib/llm/providers/bedrock.rb', line 150

def embed(input, model: nil, **params)
  raise NotImplementedError
end

#files ⇒ Object

Raises:

  • (NotImplementedError) 


114
115
116
 
# File 'lib/llm/providers/bedrock.rb', line 114

def files
  raise NotImplementedError
end

#images ⇒ Object

Raises:

  • (NotImplementedError) 


120
121
122
 
# File 'lib/llm/providers/bedrock.rb', line 120

def images
  raise NotImplementedError
end

#models ⇒ LLM::Bedrock::Models

Note:

Unlike the Converse API (bedrock-runtime), this endpoint lives on the control plane (bedrock.<region>.amazonaws.com).

Provides an interface to Bedrock’s ListFoundationModels API.

Returns:

See Also:



108
109
110
 
# File 'lib/llm/providers/bedrock.rb', line 108

def models
  LLM::Bedrock::Models.new(self)
end

#moderations ⇒ Object

Raises:

  • (NotImplementedError) 


132
133
134
 
# File 'lib/llm/providers/bedrock.rb', line 132

def moderations
  raise NotImplementedError
end

#name ⇒ Symbol

Returns the provider’s name

Returns:

  • (Symbol) —

    Returns the provider’s name



74
75
76
 
# File 'lib/llm/providers/bedrock.rb', line 74

def name
  :bedrock
end

#responses ⇒ Object

Raises:

  • (NotImplementedError) 


138
139
140
 
# File 'lib/llm/providers/bedrock.rb', line 138

def responses
  raise NotImplementedError
end

#tool_role ⇒ Symbol

Bedrock expects tool results as user messages containing ‘toolResult` content blocks rather than a distinct `tool` role.

Returns:

  • (Symbol) 


164
165
166
 
# File 'lib/llm/providers/bedrock.rb', line 164

def tool_role
  :user
end

#vector_stores ⇒ Object

Raises:

  • (NotImplementedError) 


144
145
146
 
# File 'lib/llm/providers/bedrock.rb', line 144

def vector_stores
  raise NotImplementedError
end