Class: Riffer::Providers::Base

Inherits:
Object
  • Object
show all
Defined in:
lib/riffer/providers/base.rb,
sig/generated/riffer/providers/base.rbs

Overview

Base class for all LLM providers. A template-method flow: subclasses implement the hooks (+build_request_params+, execute_generate, execute_stream, extract_token_usage, extract_content, extract_tool_calls) and the base class orchestrates them.

Direct Known Subclasses

AmazonBedrock, Anthropic, Gemini, Mock, OpenAI, OpenRouter

Constant Summary collapse

WIRE_SEPARATOR =

RBS:

  • @current_tools: Array[singleton(Riffer::Tool)]

  • @current_model: String?

  • @client: untyped

Returns:

  • (String) 
"__"
REQUEST_PARAM_ATTRIBUTES =

A deliberate whitelist — caller options outside it stay off spans.

Returns:

  • (Hash[Symbol, String]) 
{
  temperature: "gen_ai.request.temperature",
  max_tokens: "gen_ai.request.max_tokens",
  max_output_tokens: "gen_ai.request.max_tokens",
  top_p: "gen_ai.request.top_p",
  top_k: "gen_ai.request.top_k",
  frequency_penalty: "gen_ai.request.frequency_penalty",
  presence_penalty: "gen_ai.request.presence_penalty",
  seed: "gen_ai.request.seed",
  stop_sequences: "gen_ai.request.stop_sequences",
}.freeze

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.semconv_provider_name ⇒ String

Returns the provider name stamped as gen_ai.provider.name on trace spans, ideally a GenAI semconv well-known value. Defaults to the snake_cased class name rather than raising like the abstract provider methods, so enabling tracing never breaks an otherwise-working custom provider.

: () -> String

Returns:

  • (String) 


31
32
33
34
35
36
 
# File 'lib/riffer/providers/base.rb', line 31

def self.semconv_provider_name
  class_name = name
  return "unknown" unless class_name

  Riffer::Helpers::ClassNameConverter.convert(class_name.split("::").last.to_s)
end

.skills_adapter(_model = nil) ⇒ singleton(Riffer::Skills::Adapter)

Returns the preferred skill adapter for this provider; override in subclasses (optionally introspecting model) for provider-specific formats.

: (?String?) -> singleton(Riffer::Skills::Adapter)

Parameters:

  • (String, nil)

Returns:



21
22
23
 
# File 'lib/riffer/providers/base.rb', line 21

def self.skills_adapter(_model = nil)
  Riffer::Skills::MarkdownAdapter
end

Instance Method Details

#apply_pricing(usage) ⇒ Riffer::Providers::TokenUsage

: (Riffer::Providers::TokenUsage) -> Riffer::Providers::TokenUsage

Parameters:

Returns:



174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
 
# File 'lib/riffer/providers/base.rb', line 174

def apply_pricing(usage)
  rates = pricing_rates
  return usage unless rates

  cost = rates.cost_for(
    input_tokens: usage.input_tokens,
    output_tokens: usage.output_tokens,
    cache_read_tokens: usage.cache_read_tokens,
    cache_write_tokens: usage.cache_write_tokens,
  )
  Riffer::Providers::TokenUsage.new(
    input_tokens: usage.input_tokens,
    output_tokens: usage.output_tokens,
    cache_write_tokens: usage.cache_write_tokens,
    cache_read_tokens: usage.cache_read_tokens,
    cost: cost,
  )
end

#build_client ⇒ Object

-- : () -> untyped

Returns:

  • (Object) 


132
133
134
 
# File 'lib/riffer/providers/base.rb', line 132

def build_client
  raise NotImplementedError, "Subclasses must implement #build_client"
end

#build_request_params(messages, model, options) ⇒ Hash[Symbol, untyped]

-- : (Array, String?, Hash[Symbol, untyped]) -> Hash[Symbol, untyped]

Parameters:

Returns:

  • (Hash[Symbol, untyped]) 


151
152
153
 
# File 'lib/riffer/providers/base.rb', line 151

def build_request_params(messages, model, options)
  raise NotImplementedError, "Subclasses must implement #build_request_params"
end

#capture_input(span, messages) ⇒ void

This method returns an undefined value.

-- : ((Riffer::Tracing::Otel::Span | Riffer::Tracing::NoOp::Span), Array) -> void

Parameters:



311
312
313
314
315
316
317
 
# File 'lib/riffer/providers/base.rb', line 311

def capture_input(span, messages)
  return unless capture_messages?(span)

  span.set_attribute("gen_ai.input.messages", Riffer::Tracing::Capture.input_messages(messages))
  system_instructions = Riffer::Tracing::Capture.system_instructions(messages)
  span.set_attribute("gen_ai.system_instructions", system_instructions) if system_instructions
end

#capture_messages?(span) ⇒ Boolean

-- : ((Riffer::Tracing::Otel::Span | Riffer::Tracing::NoOp::Span)) -> bool

Parameters:

Returns:

  • (Boolean) 


336
337
338
 
# File 'lib/riffer/providers/base.rb', line 336

def capture_messages?(span)
  Riffer.config.tracing.capture_messages && span.recording?
end

#capture_output(span, content:, tool_calls:, finish_reason:) ⇒ void

This method returns an undefined value.

-- : ((Riffer::Tracing::Otel::Span | Riffer::Tracing::NoOp::Span), content: String?, tool_calls: Array, finish_reason: Symbol?) -> void

Parameters:



321
322
323
324
325
326
327
328
329
330
331
332
 
# File 'lib/riffer/providers/base.rb', line 321

def capture_output(span, content:, tool_calls:, finish_reason:)
  return unless capture_messages?(span)

  span.set_attribute(
    "gen_ai.output.messages",
    Riffer::Tracing::Capture.output_messages(
      content: content,
      tool_calls: tool_calls,
      finish_reason: finish_reason,
    ),
  )
end

#chat_span_attributes(model, options) ⇒ Hash[String, untyped]

-- : (String?, Hash[Symbol, untyped]) -> Hash[String, untyped]

Parameters:

  • (String, nil)
  • (Hash[Symbol, untyped])

Returns:

  • (Hash[String, untyped]) 


262
263
264
265
266
267
268
269
270
271
272
273
274
275
 
# File 'lib/riffer/providers/base.rb', line 262

def chat_span_attributes(model, options)
  attributes = {
    "gen_ai.operation.name" => "chat",
    "gen_ai.provider.name" => self.class.semconv_provider_name,
  } #: Hash[String, untyped]
  attributes["gen_ai.request.model"] = model if model

  REQUEST_PARAM_ATTRIBUTES.each do |key, attribute|
    value = options[key]
    attributes[attribute] = value if value
  end

  attributes.merge(tag_attributes(options[:tags] || {}))
end

#client ⇒ Object

Returns the client for the current LLM call. A configured client wins, resolved on every call so a Proc can vary the client by process or credential lifetime; otherwise the provider builds one from the configured credentials, memoized for the life of the provider.

: () -> untyped

Returns:

  • (Object) 


115
116
117
118
119
120
 
# File 'lib/riffer/providers/base.rb', line 115

def client
  configured = global_client
  return Riffer::Helpers::CallOrValue.resolve(configured) if configured

  @client ||= build_client
end

#decode_tool_name(wire_name, tools:) ⇒ String

-- : (String, tools: Array) -> String

Parameters:

Returns:

  • (String) 


144
145
146
147
 
# File 'lib/riffer/providers/base.rb', line 144

def decode_tool_name(wire_name, tools:)
  tool = tools.find { |t| encode_tool_name(t.name) == wire_name }
  tool ? tool.name : wire_name
end

#depends_on(gem_name) ⇒ true

: (String) -> true

Parameters:

  • (String)

Returns:

  • (true) 


105
106
107
 
# File 'lib/riffer/providers/base.rb', line 105

def depends_on(gem_name)
  Riffer::Helpers::Dependencies.depends_on(gem_name)
end

#encode_tool_name(name) ⇒ String

-- : (String) -> String

Parameters:

  • (String)

Returns:

  • (String) 


138
139
140
 
# File 'lib/riffer/providers/base.rb', line 138

def encode_tool_name(name)
  name.gsub("/", WIRE_SEPARATOR)
end

#execute_generate(params) ⇒ Object

-- : (Hash[Symbol, untyped]) -> untyped

Parameters:

  • (Hash[Symbol, untyped])

Returns:

  • (Object) 


157
158
159
 
# File 'lib/riffer/providers/base.rb', line 157

def execute_generate(params)
  raise NotImplementedError, "Subclasses must implement #execute_generate"
end

#execute_stream(params, yielder) ⇒ void

This method returns an undefined value.

-- : (Hash[Symbol, untyped], Riffer::Providers::_EventSink) -> void

Parameters:



163
164
165
 
# File 'lib/riffer/providers/base.rb', line 163

def execute_stream(params, yielder)
  raise NotImplementedError, "Subclasses must implement #execute_stream"
end

#extract_content(response) ⇒ String

-- : (untyped) -> String

Parameters:

  • (Object)

Returns:

  • (String) 


218
219
220
 
# File 'lib/riffer/providers/base.rb', line 218

def extract_content(response)
  raise NotImplementedError, "Subclasses must implement #extract_content"
end

#extract_finish_reason(_response) ⇒ Riffer::Providers::FinishReason?

Defaults to nil rather than raising — finish reasons are optional, so providers that don't report one stay valid.

: (untyped) -> Riffer::Providers::FinishReason?

Parameters:

  • (Object)

Returns:



212
213
214
 
# File 'lib/riffer/providers/base.rb', line 212

def extract_finish_reason(_response)
  nil
end

#extract_token_usage(response) ⇒ Riffer::Providers::TokenUsage?

-- : (untyped) -> Riffer::Providers::TokenUsage?

Parameters:

  • (Object)

Returns:



169
170
171
 
# File 'lib/riffer/providers/base.rb', line 169

def extract_token_usage(response)
  raise NotImplementedError, "Subclasses must implement #extract_token_usage"
end

#extract_tool_calls(response) ⇒ Array[Riffer::Messages::Assistant::ToolCall]

-- : (untyped) -> Array

Parameters:

  • (Object)

Returns:



224
225
226
 
# File 'lib/riffer/providers/base.rb', line 224

def extract_tool_calls(response)
  raise NotImplementedError, "Subclasses must implement #extract_tool_calls"
end

#generate_text(prompt: nil, system: nil, messages: nil, model: nil, files: nil, **options) ⇒ Riffer::Messages::Assistant

Generates text using the provider.

-- : (?prompt: String?, ?system: String?, ?messages: Array[Hash[Symbol, untyped] | Riffer::Messages::Base]?, ?model: String?, ?files: Array[Hash[Symbol, untyped] | Riffer::Messages::FilePart]?, **untyped) -> Riffer::Messages::Assistant

Parameters:

  • prompt: (String, nil) (defaults to: nil)
  • system: (String, nil) (defaults to: nil)
  • messages: (Array[Hash[Symbol, untyped] | Riffer::Messages::Base], nil) (defaults to: nil)
  • model: (String, nil) (defaults to: nil)
  • files: (Array[Hash[Symbol, untyped] | Riffer::Messages::FilePart], nil) (defaults to: nil)
  • (Object)

Returns:



42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
 
# File 'lib/riffer/providers/base.rb', line 42

def generate_text(prompt: nil, system: nil, messages: nil, model: nil, files: nil, **options)
  validate_input!(prompt: prompt, system: system, messages: messages)
  @current_tools = options[:tools] || [] #: Array[singleton(Riffer::Tool)]
  @current_model = model
  messages = normalize_messages(prompt: prompt, system: system, messages: messages, files: files)
  validate_normalized_messages!(messages)
  messages = merge_consecutive_messages(messages)
  params = build_request_params(messages, model, options)

  in_chat_span(model, messages, options) do |span|
    response = execute_generate(params)

    content = extract_content(response)
    tool_calls = extract_tool_calls(response)
    token_usage = extract_token_usage(response)
    finish_reason = extract_finish_reason(response)
    structured_output = parse_structured_output(content) if options[:structured_output] && tool_calls.empty?

    Riffer::Tracing.record_usage(span, token_usage)
    record_finish_reason(span, finish_reason&.reason, finish_reason&.raw)
    capture_output(span, content: content, tool_calls: tool_calls, finish_reason: finish_reason&.reason)

    Riffer::Messages::Assistant.new(
      content,
      tool_calls: tool_calls,
      token_usage: token_usage,
      structured_output: structured_output,
      finish_reason: finish_reason&.reason,
    )
  end
end

#global_client ⇒ Object

Returns the consumer-configured client for this provider; nil when none is configured, and for providers that take no configuration at all.

: () -> untyped

Returns:

  • (Object) 


126
127
128
 
# File 'lib/riffer/providers/base.rb', line 126

def global_client
  nil
end

#in_chat_span(model, messages, options) ⇒ void

This method returns an undefined value.

-- : [R] (String?, Array, Hash[Symbol, untyped]) { (Riffer::Tracing::Otel::Span | Riffer::Tracing::NoOp::Span) -> R } -> R



244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
 
# File 'lib/riffer/providers/base.rb', line 244

def in_chat_span(model, messages, options)
  Riffer::Tracing.in_span(
    model ? "chat #{model}" : "chat",
    attributes: chat_span_attributes(model, options),
    kind: :client,
  ) do |span|
    capture_input(span, messages)
    yield span
  rescue StandardError => e
    # The backend records the exception and error status on the re-raise;
    # error.type is the one semconv attribute it doesn't set.
    span.set_attribute("error.type", e.class.name)
    raise
  end
end

#merge_consecutive_messages(messages) ⇒ Array[Riffer::Messages::Base]

-- : (Array) -> Array

Parameters:

Returns:



369
370
371
372
373
374
375
 
# File 'lib/riffer/providers/base.rb', line 369

def merge_consecutive_messages(messages)
  messages.chunk(&:role).flat_map do |role, group|
    next group if role == :tool || group.size == 1

    group.inject { |merged, msg| merged + msg }
  end
end

#normalize_messages(prompt:, system:, messages:, files: nil) ⇒ Array[Riffer::Messages::Base]

-- : (prompt: String?, system: String?, messages: Array[Hash[Symbol, untyped] | Riffer::Messages::Base]?, ?files: Array[Hash[Symbol, untyped] | Riffer::Messages::FilePart]?) -> Array

Parameters:

Returns:



390
391
392
393
394
395
396
397
398
399
400
401
402
403
 
# File 'lib/riffer/providers/base.rb', line 390

def normalize_messages(prompt:, system:, messages:, files: nil)
  if messages && files && !files.empty?
    raise Riffer::ArgumentError, "cannot provide both files and messages; attach files to individual messages instead"
  end

  return messages.map { |msg| Riffer::Messages::Base.from_hash(msg) } if messages

  result = [] #: Array[Riffer::Messages::Base]
  result << Riffer::Messages::System.new(system) if system
  file_parts = (files || []).map { |f| Riffer::Messages::FilePart.from_hash(f) }
  prompt_text = prompt #: String
  result << Riffer::Messages::User.new(prompt_text, files: file_parts)
  result
end

#parse_structured_output(content) ⇒ Hash[Symbol, untyped]?

-- : (String) -> Hash[Symbol, untyped]?

Parameters:

  • (String)

Returns:

  • (Hash[Symbol, untyped], nil) 


353
354
355
356
357
 
# File 'lib/riffer/providers/base.rb', line 353

def parse_structured_output(content)
  JSON.parse(content, symbolize_names: true)
rescue JSON::ParserError
  nil
end

#parse_tool_arguments(arguments) ⇒ Hash[String, untyped]

-- : ((String | Hash[String, untyped])?) -> Hash[String, untyped]

Parameters:

  • (String, Hash[String, untyped], nil)

Returns:

  • (Hash[String, untyped]) 


361
362
363
364
365
 
# File 'lib/riffer/providers/base.rb', line 361

def parse_tool_arguments(arguments)
  return {} if arguments.nil? || arguments.empty?

  arguments.is_a?(String) ? JSON.parse(arguments) : arguments
end

#pricing_rates ⇒ Riffer::Config::Pricing::Rates?

-- : () -> Riffer::Config::Pricing::Rates?

Returns:



195
196
197
198
199
200
201
202
203
204
205
206
 
# File 'lib/riffer/providers/base.rb', line 195

def pricing_rates
  model = @current_model
  return nil unless model

  pricing = Riffer.config.pricing
  return nil if pricing.empty?

  key = Riffer::Providers::Repository.key_for(self.class)
  return nil unless key

  pricing.rates_for("#{key}/#{model}")
end

#record_finish_reason(span, reason, raw) ⇒ void

This method returns an undefined value.

-- : ((Riffer::Tracing::Otel::Span | Riffer::Tracing::NoOp::Span), Symbol?, String?) -> void

Parameters:



287
288
289
290
291
292
 
# File 'lib/riffer/providers/base.rb', line 287

def record_finish_reason(span, reason, raw)
  return unless reason

  span.set_attribute("gen_ai.response.finish_reasons", [reason.to_s])
  span.set_attribute("riffer.finish_reason.raw", raw) if raw && raw != reason.to_s
end

#record_stream_outcome(span, recorder) ⇒ void

This method returns an undefined value.

-- : ((Riffer::Tracing::Otel::Span | Riffer::Tracing::NoOp::Span), Riffer::Tracing::StreamRecorder) -> void

Parameters:



296
297
298
299
300
301
302
303
304
305
306
307
 
# File 'lib/riffer/providers/base.rb', line 296

def record_stream_outcome(span, recorder)
  Riffer::Tracing.record_usage(span, recorder.token_usage)
  record_finish_reason(span, recorder.finish_reason, recorder.raw_finish_reason)
  time_to_first_chunk = recorder.time_to_first_chunk
  span.set_attribute("gen_ai.response.time_to_first_chunk", time_to_first_chunk) if time_to_first_chunk
  capture_output(
    span,
    content: recorder.content,
    tool_calls: recorder.tool_calls,
    finish_reason: recorder.finish_reason,
  )
end

#stream_text(prompt: nil, system: nil, messages: nil, model: nil, files: nil, **options) ⇒ Enumerator[Riffer::StreamEvents::Base, void]

Streams text from the provider.

-- : (?prompt: String?, ?system: String?, ?messages: Array[Hash[Symbol, untyped] | Riffer::Messages::Base]?, ?model: String?, ?files: Array[Hash[Symbol, untyped] | Riffer::Messages::FilePart]?, **untyped) -> Enumerator[Riffer::StreamEvents::Base, void]

Parameters:

  • prompt: (String, nil) (defaults to: nil)
  • system: (String, nil) (defaults to: nil)
  • messages: (Array[Hash[Symbol, untyped] | Riffer::Messages::Base], nil) (defaults to: nil)
  • model: (String, nil) (defaults to: nil)
  • files: (Array[Hash[Symbol, untyped] | Riffer::Messages::FilePart], nil) (defaults to: nil)
  • (Object)

Returns:



78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
 
# File 'lib/riffer/providers/base.rb', line 78

def stream_text(prompt: nil, system: nil, messages: nil, model: nil, files: nil, **options)
  validate_input!(prompt: prompt, system: system, messages: messages)
  @current_tools = options[:tools] || [] #: Array[singleton(Riffer::Tool)]
  @current_model = model
  messages = normalize_messages(prompt: prompt, system: system, messages: messages, files: files)
  validate_normalized_messages!(messages)
  messages = merge_consecutive_messages(messages)
  params = build_request_params(messages, model, options)

  # The enumerator body runs in its own fiber, where the fiber-local OTEL
  # context is empty — capture here so the chat span parents to the caller's
  # trace.
  trace_context = Riffer::Tracing.current_context
  Enumerator.new do |yielder|
    Riffer::Tracing.with_context(trace_context) do
      in_chat_span(model, messages, options) do |span|
        sink = span.recording? ? Riffer::Tracing::StreamRecorder.new(yielder) : yielder
        execute_stream(params, sink)
        record_stream_outcome(span, sink) if sink.is_a?(Riffer::Tracing::StreamRecorder)
      end
    end
  end
end

#tag_attributes(tags) ⇒ Hash[String, String]

Maps normalized tags to their namespaced span attribute form. An empty map yields an empty hash, so merging it is a no-op.

: (Hash[String, String]) -> Hash[String, String]

Parameters:

  • (Hash[String, String])

Returns:

  • (Hash[String, String]) 


281
282
283
 
# File 'lib/riffer/providers/base.rb', line 281

def tag_attributes(tags)
  tags.transform_keys { |key| "riffer.tag.#{key}" }
end

#validate_input!(prompt:, system:, messages:) ⇒ void

This method returns an undefined value.

-- : (prompt: String?, system: String?, messages: Array[Hash[Symbol | String, untyped] | Riffer::Messages::Base]?) -> void

Parameters:

  • prompt: (String, nil)
  • system: (String, nil)
  • messages: (Array[Hash[Symbol | String, untyped] | Riffer::Messages::Base], nil) 


379
380
381
382
383
384
385
386
 
# File 'lib/riffer/providers/base.rb', line 379

def validate_input!(prompt:, system:, messages:)
  if messages.nil?
    raise Riffer::ArgumentError, "prompt is required when messages is not provided" if prompt.nil?
  else
    raise Riffer::ArgumentError, "cannot provide both prompt and messages" if prompt
    raise Riffer::ArgumentError, "cannot provide both system and messages" if system
  end
end

#validate_normalized_messages!(messages) ⇒ void

This method returns an undefined value.

-- : (Array) -> void

Parameters:



407
408
409
410
 
# File 'lib/riffer/providers/base.rb', line 407

def validate_normalized_messages!(messages)
  has_user = messages.any?(Riffer::Messages::User)
  raise Riffer::ArgumentError, "messages must include at least one user message" unless has_user
end

#yield_finish_reason(yielder, finish_reason) ⇒ void

This method returns an undefined value.

-- : (Riffer::Providers::_EventSink, Riffer::Providers::FinishReason?) -> void

Parameters:



342
343
344
345
346
347
348
349
 
# File 'lib/riffer/providers/base.rb', line 342

def yield_finish_reason(yielder, finish_reason)
  return unless finish_reason

  yielder << Riffer::StreamEvents::FinishReasonDone.new(
    finish_reason: finish_reason.reason,
    raw_finish_reason: finish_reason.raw,
  )
end