Class: LittleGhost::Providers::Bedrock

Inherits:
Base
  • Object
show all
Defined in:
lib/little_ghost/providers/bedrock.rb,
lib/little_ghost/providers/bedrock/http_client.rb,
lib/little_ghost/providers/bedrock/aws_protocol.rb,
lib/little_ghost/providers/bedrock/catalog_source.rb,
lib/little_ghost/providers/bedrock/credential_resolver.rb

Overview

Bedrock lets LittleGhost agents use models available through Amazon Bedrock Converse. Its output follows the same streaming events as every other LittleGhost provider.

provider = LittleGhost::Providers::Bedrock.new(
model: ENV.fetch("BEDROCK_MODEL_ID"),
region: ENV.fetch("AWS_REGION")
)

The default client uses LittleGhost's standard-library SigV4 and AWS EventStream implementations. Applications may inject client instead.

Transient service and stream failures retry with exponential backoff. Each retry emits :model_retry and reports whether partial text was already emitted, allowing stream consumers to handle repeated output deliberately.

Defined Under Namespace

Classes: AwsSigV4, CatalogSource, CredentialResolver, Credentials, EventStreamDecoder, HTTPClient, StreamError, StreamNormalizer

Constant Summary collapse

INITIAL_RETRY_DELAY =

:nodoc:

1
MAX_RETRY_DELAY =

:nodoc:

16
TRANSIENT_STREAM_ERRORS = 
%w[
  internal_server_exception model_stream_error_exception service_unavailable_exception throttling_exception
].freeze
CONTEXT_OVERFLOW_MARKERS =

:nodoc:

[
  "context window", "maximum context length", "max context length",
  "input is too long", "too many input tokens"
].freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Base

#prepare_request

Constructor Details

#initialize(model:, region: nil, client: nil, max_retries: 2, sleeper: nil, on_retry: ->(*) {}, **client_options) ⇒ Bedrock

Configures Bedrock for model.

region and remaining client_options configure the built-in HTTP client. max_retries, sleeper, and on_retry control retry behavior. Injecting client bypasses creation of the built-in HTTP client.



68
69
70
71
72
73
74
75
 
# File 'lib/little_ghost/providers/bedrock.rb', line 68

def initialize(model:, region: nil, client: nil, max_retries: 2, sleeper: nil,
  on_retry: ->(*) {}, **client_options)
  @model = model
  @client = client || build_client(region:, **client_options)
  @max_retries = Integer(max_retries)
  @sleeper = sleeper
  @on_retry = on_retry
end

Instance Attribute Details

#model ⇒ Object (readonly)

Bedrock model identifier used for requests.



61
62
63
 
# File 'lib/little_ghost/providers/bedrock.rb', line 61

def model
  @model
end

Class Method Details

.request_options ⇒ Object

Request policy supported by Bedrock retries and its built-in HTTP client.



31
32
33
 
# File 'lib/little_ghost/providers/bedrock.rb', line 31

def self.request_options
  %i[max_response_bytes max_retries open_timeout read_timeout].freeze
end

Instance Method Details

#capabilities(metadata: {}) ⇒ Object

Reads capabilities from Bedrock supported_parameters metadata. Missing metadata produces ModelCapabilities.unknown.



137
138
139
140
141
142
143
144
145
146
147
148
 
# File 'lib/little_ghost/providers/bedrock.rb', line 137

def capabilities(metadata: {})
  parameters = metadata[:supported_parameters] || metadata["supported_parameters"]
  return ModelCapabilities.unknown unless parameters.is_a?(Array)

  values = parameters.map(&:to_s)
  ModelCapabilities.new(
    native_structured_output: values.include?("structured_outputs"),
    tools: values.include?("tools"),
    tool_choice: values.include?("tool_choice"),
    supported_parameters: values
  )
end

#stream(request) ⇒ Object

Streams LittleGhost StreamEvent objects for request.

Without a block, returns an Enumerator. Context-window failures normalize to ContextWindowOverflowError and malformed tool calls normalize to MalformedToolCallError.



82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
 
# File 'lib/little_ghost/providers/bedrock.rb', line 82

def stream(request)
  return enum_for(__method__, request) unless block_given?

  attempts = 0

  begin
    partial_text = false
    request.cancellation_token.raise_if_cancelled!
    normalizer = StreamNormalizer.new(model:)
    stream = Support::InterruptibleStream.new(
      cancellation_token: request.cancellation_token,
      deadline: request.deadline
    ) do |emit|
      response = @client.converse_stream(**request_parameters(request))
      response.stream.each { |event| emit.call(event) }
    end
    stream.each do |event|
      normalizer.consume(event_hash(event)).each do |normalized|
        partial_text ||= normalized.type == :text_delta && !normalized.data[:text].to_s.empty?
        yield normalized
      end
    end
    normalizer.finish.each do |event|
      partial_text ||= event.type == :text_delta && !event.data[:text].to_s.empty?
      yield event
    end
  rescue CancelledError, DeadlineExceededError, CleanupError
    raise
  rescue => error
    raise if error.is_a?(Error) && !error.is_a?(StreamError)

    if context_window_overflow?(error)
      raise ContextWindowOverflowError, "The model context window was exceeded"
    end
    raise provider_error(error) if !retryable?(error) || attempts >= @max_retries

    attempts += 1
    request.cancellation_token.raise_if_cancelled!
    delay = capped_retry_delay(request, retry_delay(attempts))
    @on_retry.call(attempts, error, delay)
    wait_before_retry(request, delay)
    yield StreamEvent.build(
      :model_retry,
      attempt: attempts,
      delay:,
      error_class: error.class.name,
      error_code: (error.event_type if error.is_a?(StreamError)),
      partial_text:
    )
    retry
  end
end