Class: LLM::Tracer

Inherits:
Object
  • Object
show all
Defined in:
lib/llm/tracer.rb

Overview

The LLM::Tracer is the superclass of all LLM tracers. It can be helpful for implementing instrumentation and hooking into the lifecycle of an LLM request. See LLM::Tracer::Telemetry, and LLM::Tracer::Logger for example tracer implementations.

Defined Under Namespace

Classes: Logger, Null, Telemetry

Constant Summary collapse

FINISH_METADATA_PROC_KEY = 
:"llm.tracer.finish_metadata_proc"

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(provider, options = {}) ⇒ Tracer

Returns a new instance of Tracer.

Parameters:

  • provider (LLM::Provider) —

    A provider

  • options (Hash) (defaults to: {}) —

    A hash of options



25
26
27
28
 
# File 'lib/llm/tracer.rb', line 25

def initialize(provider, options = {})
  @llm = provider
  @options = {}
end

Instance Attribute Details

#llm ⇒ LLM::Provider (readonly)

Returns:



18
19
20
 
# File 'lib/llm/tracer.rb', line 18

def llm
  @llm
end

Instance Method Details

#consume_extra_inputs ⇒ Hash

Returns and clears extra inputs for the next span. Called by the telemetry tracer when starting a span. Subclasses can override to return stored inputs; default returns {}.

Returns:

  • (Hash) —

    Attribute key => value to set on the span at start



188
189
190
 
# File 'lib/llm/tracer.rb', line 188

def consume_extra_inputs
  {}
end

#consume_extra_outputs ⇒ Hash

Returns and clears extra outputs for the current span. Called by the telemetry tracer when finishing a span. Subclasses override to return fiber-local outputs; default returns {}.

Returns:

  • (Hash) —

    Attribute key => value to set on the span at finish



198
199
200
 
# File 'lib/llm/tracer.rb', line 198

def consume_extra_outputs
  {}
end

#consume_request_metadata ⇒ Hash

Consume and clear per-request metadata. Called by the telemetry tracer at span start.

Returns:

  • (Hash) 


220
221
222
223
224
225
 
# File 'lib/llm/tracer.rb', line 220

def consume_request_metadata
  key = thread_request_metadata_key
  data = thread[key] || {}
  thread[key] = nil
  data
end

#flush! ⇒ nil

Note:

This method is only implemented by the Telemetry tracer. It is a noop for other tracers.

Flush the tracer

Returns:

  • (nil) 


144
145
146
 
# File 'lib/llm/tracer.rb', line 144

def flush!
  nil
end

#inspect ⇒ String

Returns:

  • (String) 


128
129
130
 
# File 'lib/llm/tracer.rb', line 128

def inspect
  "#<#{LLM::Utils.object_id(self)} @provider=#{@llm.class} @tracer=#{@tracer.inspect}>"
end

#merge_extra(metadata: nil, inputs: nil, outputs: nil) ⇒ self

Merges extra attributes for the current trace/span. Used by applications (e.g. chatbot) to add metadata, span inputs, or span outputs to the next span or to the trace. No-op by default.

Parameters:

  • metadata (Hash, nil) (defaults to: nil) —

    Key-value pairs merged into trace/span metadata.

  • inputs (Hash, nil) (defaults to: nil) —

    Key-value pairs set on the next span at start (e.g. gen_ai.input.messages). Consumed when the span is created.

  • outputs (Hash, nil) (defaults to: nil) —

    Key-value pairs set on the current span at finish (e.g. gen_ai.output.messages). Must be set before the request finishes (e.g. in a block passed to the provider).

Returns:

  • (self) 


162
163
164
 
# File 'lib/llm/tracer.rb', line 162

def merge_extra(metadata: nil, inputs: nil, outputs: nil)
  self
end

#on_request_error(ex:, span:) ⇒ void

This method returns an undefined value.

Called when an LLM provider request fails.

Parameters:

Raises:

  • (NotImplementedError) 


58
59
60
 
# File 'lib/llm/tracer.rb', line 58

def on_request_error(ex:, span:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_request_finish(operation:, res:, model: nil, span: nil, outputs: nil, metadata: nil) ⇒ void

This method returns an undefined value.

Called after an LLM provider request succeeds.

Parameters:

  • operation (String)
  • res (LLM::Response)
  • span (Object, nil) (defaults to: nil)
  • model (String) (defaults to: nil)
  • outputs (Hash, nil) (defaults to: nil) —

    Optional span attributes (e.g. gen_ai.output.messages) from llm.rb or caller.

  • metadata (Hash, nil) (defaults to: nil) —

    Optional metadata from llm.rb or caller.

Raises:

  • (NotImplementedError) 


49
50
51
 
# File 'lib/llm/tracer.rb', line 49

def on_request_finish(operation:, res:, model: nil, span: nil, outputs: nil, metadata: nil)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_request_start(operation:, model: nil, inputs: nil) ⇒ void

This method returns an undefined value.

Called before an LLM provider request is executed.

Parameters:

  • operation (String)
  • model (String) (defaults to: nil)
  • inputs (Hash, nil) (defaults to: nil) —

    Optional span attributes (e.g. gen_ai.input.messages) from llm.rb or caller.

Raises:

  • (NotImplementedError) 


36
37
38
 
# File 'lib/llm/tracer.rb', line 36

def on_request_start(operation:, model: nil, inputs: nil)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_tool_error(ex:, span:) ⇒ void

This method returns an undefined value.

Called when a local tool/function raises.

Parameters:

  • ex (Exception) —

    The raised error.

  • span (Object, nil) —

    The span/context object returned by #on_tool_start.

Raises:

  • (NotImplementedError) 


95
96
97
 
# File 'lib/llm/tracer.rb', line 95

def on_tool_error(ex:, span:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_tool_finish(result:, span:) ⇒ void

This method returns an undefined value.

Called after a local tool/function succeeds.

Parameters:

Raises:

  • (NotImplementedError) 


84
85
86
 
# File 'lib/llm/tracer.rb', line 84

def on_tool_finish(result:, span:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#on_tool_start(id:, name:, arguments:, model:) ⇒ void

This method returns an undefined value.

Called before a local tool/function executes.

Parameters:

  • id (String) —

    The tool call ID assigned by the model/provider

  • name (String) —

    The tool (function) name.

  • arguments (Hash) —

    The parsed tool arguments.

  • model (String) —

    The model name

Raises:

  • (NotImplementedError) 


73
74
75
 
# File 'lib/llm/tracer.rb', line 73

def on_tool_start(id:, name:, arguments:, model:)
  raise NotImplementedError, "#{self.class} does not implement '#{__method__}'"
end

#set_finish_metadata_proc(proc) ⇒ self

Optional: set a proc to supply metadata when the next chat span finishes. The proc is called with the response (res) and should return a Hash of metadata (e.g. { intent: "...", confidence: 1.0 }) to merge onto the span. Cleared after use. Used by apps to attach routing/intent that is only known after the response.

Parameters:

  • proc (Proc, nil) —

    (res) -> Hash or nil

Returns:

  • (self) 


175
176
177
178
 
# File 'lib/llm/tracer.rb', line 175

def set_finish_metadata_proc(proc)
  thread[FINISH_METADATA_PROC_KEY] = proc
  self
end

#set_request_metadata(metadata) ⇒ nil

Store per-request metadata (e.g. user_input) to be consumed by tracers when starting the next span. Used for plain-text input.value / output.value.

Parameters:

  • metadata (Hash) —

    e.g. { user_input: "the user question" }

Returns:

  • (nil) 


208
209
210
211
212
213
214
 
# File 'lib/llm/tracer.rb', line 208

def set_request_metadata(metadata)
  return nil unless metadata && !metadata.empty?
  key = thread_request_metadata_key
  current = thread[key] || {}
  thread[key] = current.merge(metadata.compact)
  nil
end

#spans ⇒ Array

Returns:

  • (Array) 


134
135
136
 
# File 'lib/llm/tracer.rb', line 134

def spans
  []
end

#start_trace(trace_group_id: nil, name: "llm", attributes: {}, metadata: nil) ⇒ self

Opens a trace group so subsequent LLM spans share the same OpenTelemetry trace_id (and appear as one trace in backends like Langfuse). When trace_group_id is a string, it is used to derive the trace_id.

Parameters:

  • trace_group_id (String, nil) (defaults to: nil) —

    Optional. When present, converted to a 16-byte trace_id so all spans created until #stop_trace are grouped in one trace.

  • name (String) (defaults to: "llm") —

    Name for the root span (e.g. "chatbot.turn").

  • attributes (Hash) (defaults to: {}) —

    OpenTelemetry attributes to set on the root span.

  • metadata (Hash, nil) (defaults to: nil) —

    Optional. Trace-level metadata merged into the trace by tracers that support it.

Returns:

  • (self) 


114
115
116
 
# File 'lib/llm/tracer.rb', line 114

def start_trace(trace_group_id: nil, name: "llm", attributes: {}, metadata: nil)
  self
end

#stop_trace ⇒ self

Finishes the trace group started by #start_trace. Safe to call even if no trace is active.

Returns:

  • (self) 


122
123
124
 
# File 'lib/llm/tracer.rb', line 122

def stop_trace
  self
end