Class: LLM::Tracer::Telemetry

Inherits:
Tracer
  • Object
show all
Defined in:
lib/llm/tracer/telemetry.rb

Overview

The LLM::Tracer::Telemetry tracer provides telemetry support through the opentelemetry-ruby RubyGem. The gem should be installed separately since this feature is opt-in and disabled by default.

Examples:

InMemory export

#!/usr/bin/env ruby
require "llm"
require "pp"

llm = LLM.openai(key: ENV["KEY"])
llm.tracer = LLM::Tracer::Telemetry.new(llm)

ctx = LLM::Context.new(llm)
ctx.talk "hello"
ctx.talk "how are you?"
ctx.tracer.spans.each { |span| pp span }

OTLP export

#!/usr/bin/env ruby
require "llm"
require "opentelemetry-exporter-otlp"

exporter = OpenTelemetry::Exporter::OTLP::Exporter.new

llm = LLM.openai(key: ENV["KEY"])
llm.tracer = LLM::Tracer::Telemetry.new(llm, exporter:)

ctx = LLM::Context.new(llm)
ctx.talk "hello"
ctx.talk "how are you?"

See Also:

Instance Method Summary collapse

Constructor Details

#initialize(provider, options = {}) ⇒ LLM::Tracer::Telemetry

param [LLM::Provider] provider An LLM provider



46
47
48
49
50
 
# File 'lib/llm/tracer/telemetry.rb', line 46

def initialize(provider, options = {})
  super
  @exporter = options.delete(:exporter)
  setup!
end

Instance Method Details

#flush! ⇒ nil

Note:

Exports are batched in the background by default. Long-lived processes usually do not need to call this method. Short-lived scripts should call #flush! before exit to reduce the risk of losing spans that are still buffered.

Flushes queued telemetry to the configured exporter.

Returns:

  • (nil) 


186
187
188
189
 
# File 'lib/llm/tracer/telemetry.rb', line 186

def flush!
  @tracer_provider.force_flush
  nil
end

#on_request_error(ex:, span:) ⇒ Object

Parameters:



110
111
112
113
114
115
116
117
 
# File 'lib/llm/tracer/telemetry.rb', line 110

def on_request_error(ex:, span:)
  return nil unless span
  attributes = {"error.type" => ex.class.to_s}.compact
  attributes.each { span.set_attribute(_1, _2) }
  span.add_event("gen_ai.request.finish")
  span.status = ::OpenTelemetry::Trace::Status.error(ex.message)
  span.tap(&:finish)
end

#on_request_finish(operation:, res:, model: nil, span: nil, outputs: nil, metadata: nil) ⇒ Object

Parameters:

  • operation (String)
  • res (LLM::Response)
  • span (Object, nil) (defaults to: nil)
  • model (String) (defaults to: nil)
  • outputs (Hash, nil) (defaults to: nil) —

    Optional span attributes (e.g. gen_ai.output.messages) from llm.rb or caller.

  • metadata (Hash, nil) (defaults to: nil) —

    Optional metadata from llm.rb or caller.



99
100
101
102
103
104
105
106
 
# File 'lib/llm/tracer/telemetry.rb', line 99

def on_request_finish(operation:, res:, model: nil, span: nil, outputs: nil, metadata: nil)
  return nil unless span
  case operation
  when "chat" then finish_chat(operation:, model:, res:, span:, outputs:, metadata:)
  when "retrieval" then finish_retrieval(operation:, res:, span:)
  else nil
  end
end

#on_request_start(operation:, model: nil, inputs: nil) ⇒ Object

Parameters:

  • operation (String)
  • model (String) (defaults to: nil)
  • inputs (Hash, nil) (defaults to: nil) —

    Optional span attributes (e.g. gen_ai.input.messages) from llm.rb or caller.



89
90
91
92
93
94
95
 
# File 'lib/llm/tracer/telemetry.rb', line 89

def on_request_start(operation:, model: nil, inputs: nil)
  case operation
  when "chat" then start_chat(operation:, model:, inputs:)
  when "retrieval" then start_retrieval(operation:)
  else nil
  end
end

#on_tool_error(ex:, span:) ⇒ void

This method returns an undefined value.

Parameters:

  • ex (Exception) —

    The raised error.

  • span (Object, nil) —

    The span/context object returned by #on_tool_start.



157
158
159
160
161
162
163
164
 
# File 'lib/llm/tracer/telemetry.rb', line 157

def on_tool_error(ex:, span:)
  return nil unless span
  attributes = {"error.type" => ex.class.to_s}.compact
  attributes.each { span.set_attribute(_1, _2) }
  span.add_event("gen_ai.tool.finish")
  span.status = ::OpenTelemetry::Trace::Status.error(ex.message)
  span.tap(&:finish)
end

#on_tool_finish(result:, span:) ⇒ void

This method returns an undefined value.

Parameters:



142
143
144
145
146
147
148
149
150
151
152
 
# File 'lib/llm/tracer/telemetry.rb', line 142

def on_tool_finish(result:, span:)
  return nil unless span
  attributes = {
    "gen_ai.tool.call.id" => result.id,
    "gen_ai.tool.name" => result.name&.to_s,
    "gen_ai.tool.call.result" => LLM.json.dump(result.value)
  }.compact
  attributes.each { span.set_attribute(_1, _2) }
  span.add_event("gen_ai.tool.finish")
  span.tap(&:finish)
end

#on_tool_start(id:, name:, arguments:, model:) ⇒ void

This method returns an undefined value.

Parameters:

  • id (String) —

    The tool call ID assigned by the model/provider

  • name (String) —

    The tool (function) name.

  • arguments (Hash) —

    The parsed tool arguments.

  • model (String) —

    The model name



122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
 
# File 'lib/llm/tracer/telemetry.rb', line 122

def on_tool_start(id:, name:, arguments:, model:)
  attributes = {
    "gen_ai.operation.name" => "execute_tool",
    "gen_ai.request.model" => model,
    "gen_ai.tool.call.id" => id,
    "gen_ai.tool.name" => name&.to_s,
    "gen_ai.tool.call.arguments" => LLM.json.dump(arguments),
    "gen_ai.provider.name" => provider_name,
    "server.address" => provider_host,
    "server.port" => provider_port
  }.merge!(trace_attributes(span_kind: "tool")).compact
  span_name = ["execute_tool", name].compact.join(" ")
  span = create_span(span_name.empty? ? "gen_ai.tool" : span_name, attributes:)
  span.add_event("gen_ai.tool.start")
  span
end

#spans ⇒ Array<OpenTelemetry::SDK::Trace::SpanData>

Note:

This method returns an empty array for exporters that do not implement 'finished_spans' such as the OTLP exporter

Returns:

  • (Array<OpenTelemetry::SDK::Trace::SpanData>) 


172
173
174
175
176
 
# File 'lib/llm/tracer/telemetry.rb', line 172

def spans
  return [] unless @exporter.respond_to?(:finished_spans)
  flush!
  @exporter.finished_spans
end

#start_trace(trace_group_id: nil, name: "llm", attributes: {}, metadata: nil) ⇒ self

When trace_group_id is provided, it is converted to an OpenTelemetry trace_id (via a deterministic 16-byte hash) so all spans until #stop_trace share that trace_id and appear as one trace in OTLP/Langfuse.

Parameters:

  • trace_group_id (String, nil) (defaults to: nil) —

    Optional. When present, converted to a 16-byte trace_id so all spans created until #stop_trace are grouped in one trace.

  • name (String) (defaults to: "llm") —

    Name for the root span (e.g. "chatbot.turn").

  • attributes (Hash) (defaults to: {}) —

    OpenTelemetry attributes to set on the root span.

  • metadata (Hash, nil) (defaults to: nil) —

    Optional. Trace-level metadata merged into the trace by tracers that support it.

Returns:

  • (self) 


59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
 
# File 'lib/llm/tracer/telemetry.rb', line 59

def start_trace(trace_group_id: nil, name: "llm", attributes: {}, metadata: nil)
  return self if trace_group_id.to_s.empty?
  span_context = span_context_from_trace_group_id(trace_group_id.to_s)
  parent_ctx = ::OpenTelemetry::Trace.context_with_span(
    ::OpenTelemetry::Trace.non_recording_span(span_context)
  )
  attrs = attributes.compact
  attrs["llm.trace_group_id"] = trace_group_id.to_s
  root_span = @tracer.start_span(
    name,
    kind: :server,
    attributes: attrs,
    with_parent: parent_ctx
  )
  @root_span = root_span
  @root_context = ::OpenTelemetry::Trace.context_with_span(root_span)
  self
end

#stop_trace ⇒ self

Returns:

  • (self) 


80
81
82
83
84
85
 
# File 'lib/llm/tracer/telemetry.rb', line 80

def stop_trace
  @root_span&.finish
  @root_span = nil
  @root_context = nil
  self
end