Module: ActiveAgents::Telemetry::RubyLLM

Defined in:
lib/activeagents/telemetry/ruby_llm.rb,
lib/activeagents/telemetry/ruby_llm/version.rb

Overview

Reports RubyLLM chats to an ActiveAgents-compatible trace endpoint.

Requires RubyLLM.config.instrumenter = ActiveSupport::Notifications (RubyLLM 1.x; 2.x wires this up under Rails).

One trace per conversation turn: a root span, an llm span covering the whole provider loop, and a tool span per tool_call.ruby_llm event.

RubyLLM emits a chat.ruby_llm event per provider round, and the two generations of the gem arrange those rounds differently: through 1.x a tool round recurses inside the enclosing event, while 2.x drives a flat step until complete? loop whose rounds are siblings with tool calls firing between them. Rounds are therefore accumulated and flushed on the round that ends the turn — the one that errors or answers without requesting tools — which yields the same trace under both arrangements. Tokens are summed per round from the assistant messages that round added, so a repeated event-level count is never double counted.

Tool arguments and results are never sent; error messages are truncated.

Defined Under Namespace

Classes: ChatSubscriber, State, ToolCallSubscriber

Constant Summary collapse

AGENT_KEY = 
:activeagents_telemetry_ruby_llm_agent
STATE_KEY = 
:activeagents_telemetry_ruby_llm_state
TOOL_STARTED_AT_KEY = 
:_activeagents_telemetry_started_at
SDK_NAME = 
"activeagents-telemetry-ruby_llm"
MAX_TURN_SECONDS =

A turn that never reaches a final round (a halted tool call, or an app driving RubyLLM 2.x's step by hand) would otherwise accumulate forever.

600
DEFAULT_AGENT = 
{ name: "RubyLLM::Chat", action: "chat" }.freeze
VERSION = 
"0.1.0"

Class Attribute Summary collapse

Class Method Summary collapse

Class Attribute Details

.reporter ⇒ Object 



84
85
86
 
# File 'lib/activeagents/telemetry/ruby_llm.rb', line 84

def reporter
  @reporter ||= Reporter.new(configuration, sdk_name: SDK_NAME, sdk_version: VERSION)
end

Class Method Details

.begin_round(payload) ⇒ Object 



118
119
120
121
122
123
124
125
126
127
128
 
# File 'lib/activeagents/telemetry/ruby_llm.rb', line 118

def begin_round(payload)
  turn = state
  if turn.depth.zero?
    chat_key = payload[:chat].object_id
    flush! if turn.rounds.positive? && (turn.chat_key != chat_key || turn_expired?(turn))
    turn = state
    turn.chat_key = chat_key
    turn.started_at ||= Time.now
  end
  turn.depth += 1
end

.build_tool_span(payload, started_at, finished_at) ⇒ Object 



140
141
142
143
144
145
146
147
148
149
150
 
# File 'lib/activeagents/telemetry/ruby_llm.rb', line 140

def build_tool_span(payload, started_at, finished_at)
  error = payload[:exception_object]
  span = Span.new(
    "tool.#{payload[:tool_name]}",
    type: "tool",
    start_time: started_at,
    attributes: { "tool.name" => payload[:tool_name].to_s, "tool.call_id" => payload[:tool_call_id].to_s }
  )
  span.record_error(error, message_limit: configuration.error_message_limit) if error
  span.finish(at: finished_at)
end

.clear_state ⇒ Object 



103
104
105
 
# File 'lib/activeagents/telemetry/ruby_llm.rb', line 103

def clear_state
  Thread.current[STATE_KEY] = nil
end

.configuration ⇒ Object 



80
81
82
 
# File 'lib/activeagents/telemetry/ruby_llm.rb', line 80

def configuration
  @configuration ||= Telemetry.configuration
end

.finish_round(payload) ⇒ Object 



130
131
132
133
134
135
136
137
138
 
# File 'lib/activeagents/telemetry/ruby_llm.rb', line 130

def finish_round(payload)
  turn = state
  turn.depth -= 1
  return unless turn.depth.zero?

  turn.rounds += 1
  accumulate_tokens(turn, payload)
  flush!(payload) if payload[:exception_object] || !payload[:tool_call]
end

.flush!(payload = {}) ⇒ Object

Reports whatever the current turn has accumulated. Apps that drive RubyLLM 2.x's step/run_tools themselves can call this to close a turn that ends while tool calls are still pending.



110
111
112
113
114
115
116
 
# File 'lib/activeagents/telemetry/ruby_llm.rb', line 110

def flush!(payload = {})
  turn = Thread.current[STATE_KEY]
  return if turn.nil? || turn.rounds.zero?

  clear_state
  report_turn(payload, turn)
end

.state ⇒ Object 



99
100
101
 
# File 'lib/activeagents/telemetry/ruby_llm.rb', line 99

def state
  Thread.current[STATE_KEY] ||= State.new(0, nil, [], 0, Span::ZERO_TOKENS.dup, nil)
end

.subscribe!(api_key: nil, endpoint: nil, service_name: nil, environment: nil, agent_resolver: nil, async: nil, configuration: nil) ⇒ Object

Subscribes to RubyLLM's instrumentation.

Destination settings fall back to ActiveAgents::Telemetry.configuration, so an app that already called ActiveAgents::Telemetry.configure can call this with no arguments at all.

agent_resolver: optional callable receiving the chat event payload and returning { name:, action: }, so traffic can be attributed from an initializer alone; an enclosing with_agent block still wins. RubyLLM carries no application identity on the payload — neither RubyLLM::Agent nor an acts_as_chat record reaches the instrumenter — so unattributed traffic reports as RubyLLM::Chat.



57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
 
# File 'lib/activeagents/telemetry/ruby_llm.rb', line 57

def subscribe!(api_key: nil, endpoint: nil, service_name: nil, environment: nil,
               agent_resolver: nil, async: nil, configuration: nil)
  @configuration = configuration || Telemetry.configuration.dup
  @configuration.api_key = api_key unless api_key.nil?
  @configuration.endpoint = endpoint unless endpoint.nil?
  @configuration.service_name = service_name unless service_name.nil?
  @configuration.environment = environment unless environment.nil?
  @configuration.async = async unless async.nil?

  @agent_resolver = agent_resolver
  @reporter = Reporter.new(@configuration, sdk_name: SDK_NAME, sdk_version: VERSION)

  @subscriptions ||= [
    ActiveSupport::Notifications.subscribe("chat.ruby_llm", ChatSubscriber.new),
    ActiveSupport::Notifications.subscribe("tool_call.ruby_llm", ToolCallSubscriber.new)
  ]
end

.unsubscribe! ⇒ Object 



75
76
77
78
 
# File 'lib/activeagents/telemetry/ruby_llm.rb', line 75

def unsubscribe!
  Array(@subscriptions).each { |subscription| ActiveSupport::Notifications.unsubscribe(subscription) }
  @subscriptions = nil
end

.with_agent(name, action: "chat") ⇒ Object

Attributes traces inside the block to a named agent/action.



91
92
93
94
95
96
97
 
# File 'lib/activeagents/telemetry/ruby_llm.rb', line 91

def with_agent(name, action: "chat")
  previous = Thread.current[AGENT_KEY]
  Thread.current[AGENT_KEY] = { name: name, action: action }
  yield
ensure
  Thread.current[AGENT_KEY] = previous
end