Class: LittleGhost::Providers::Bedrock

Inherits:
Object
  • Object
show all
Defined in:
lib/little_ghost/providers/bedrock.rb

Overview

Bedrock lets LittleGhost agents use models available through Amazon Bedrock Converse. Its output follows the same streaming events as every other LittleGhost provider.

provider = LittleGhost::Providers::Bedrock.new(
model: ENV.fetch("BEDROCK_MODEL_ID"),
region: ENV.fetch("AWS_REGION")
)

The default client requires the optional aws-sdk-bedrockruntime gem and uses the AWS SDK credential chain. Applications may inject client instead.

Transient service and stream failures retry with exponential backoff. Each retry emits :model_retry and reports whether partial text was already emitted, allowing stream consumers to handle repeated output deliberately.

Defined Under Namespace

Classes: StreamError, StreamNormalizer

Constant Summary collapse

INITIAL_RETRY_DELAY =

:nodoc:

1
MAX_RETRY_DELAY =

:nodoc:

16
TRANSIENT_STREAM_ERRORS = 
%w[
  internal_server_exception model_stream_error_exception service_unavailable_exception throttling_exception
].freeze
CONTEXT_OVERFLOW_MARKERS =

:nodoc:

[
  "context window", "maximum context length", "max context length",
  "input is too long", "too many input tokens"
].freeze

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(model:, region: nil, client: nil, max_retries: 2, sleeper: nil, on_retry: ->(*) {}, **client_options) ⇒ Bedrock

Configures Bedrock for model.

region and remaining client_options configure the default AWS client. max_retries, sleeper, and on_retry control retry behavior. Injecting client bypasses creation of the optional SDK client.



60
61
62
63
64
65
66
67
 
# File 'lib/little_ghost/providers/bedrock.rb', line 60

def initialize(model:, region: nil, client: nil, max_retries: 2, sleeper: nil,
  on_retry: ->(*) {}, **client_options)
  @model = model
  @client = client || build_client(region:, **client_options)
  @max_retries = Integer(max_retries)
  @sleeper = sleeper
  @on_retry = on_retry
end

Instance Attribute Details

#model ⇒ Object (readonly)

Bedrock model identifier used for requests.



53
54
55
 
# File 'lib/little_ghost/providers/bedrock.rb', line 53

def model
  @model
end

Instance Method Details

#capabilities(metadata: {}) ⇒ Object

Reads capabilities from Bedrock supported_parameters metadata. Missing metadata produces ModelCapabilities.unknown.



129
130
131
132
133
134
135
136
137
138
139
140
 
# File 'lib/little_ghost/providers/bedrock.rb', line 129

def capabilities(metadata: {})
  parameters = metadata[:supported_parameters] || metadata["supported_parameters"]
  return ModelCapabilities.unknown unless parameters.is_a?(Array)

  values = parameters.map(&:to_s)
  ModelCapabilities.new(
    native_structured_output: values.include?("structured_outputs"),
    tools: values.include?("tools"),
    tool_choice: values.include?("tool_choice"),
    supported_parameters: values
  )
end

#stream(request) ⇒ Object

Streams LittleGhost StreamEvent objects for request.

Without a block, returns an Enumerator. Context-window failures normalize to ContextWindowOverflowError and malformed tool calls normalize to MalformedToolCallError.



74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
 
# File 'lib/little_ghost/providers/bedrock.rb', line 74

def stream(request)
  return enum_for(__method__, request) unless block_given?

  attempts = 0

  begin
    partial_text = false
    request.cancellation_token.raise_if_cancelled!
    normalizer = StreamNormalizer.new(model:)
    stream = Support::InterruptibleStream.new(
      cancellation_token: request.cancellation_token,
      deadline: request.deadline
    ) do |emit|
      response = @client.converse_stream(**request_parameters(request))
      response.stream.each { |event| emit.call(event) }
    end
    stream.each do |event|
      normalizer.consume(event_hash(event)).each do |normalized|
        partial_text ||= normalized.type == :text_delta && !normalized.data[:text].to_s.empty?
        yield normalized
      end
    end
    normalizer.finish.each do |event|
      partial_text ||= event.type == :text_delta && !event.data[:text].to_s.empty?
      yield event
    end
  rescue CancelledError, DeadlineExceededError, CleanupError
    raise
  rescue => error
    raise if error.is_a?(Error) && !error.is_a?(StreamError)

    if context_window_overflow?(error)
      raise ContextWindowOverflowError, "The model context window was exceeded"
    end
    raise provider_error(error) if !retryable?(error) || attempts >= @max_retries

    attempts += 1
    request.cancellation_token.raise_if_cancelled!
    delay = capped_retry_delay(request, retry_delay(attempts))
    @on_retry.call(attempts, error, delay)
    wait_before_retry(request, delay)
    yield StreamEvent.build(
      :model_retry,
      attempt: attempts,
      delay:,
      error_class: error.class.name,
      error_code: (error.event_type if error.is_a?(StreamError)),
      partial_text:
    )
    retry
  end
end