Class: ActionAgent::TelemetryTrace

Inherits:
ActiveRecord::Base
  • Object
show all
Includes:
AdapterAware
Defined in:
app/models/action_agent/telemetry_trace.rb

Overview

Stores telemetry traces from ActiveAgent clients.

Each trace represents a complete generation lifecycle, including prompt preparation, LLM calls, tool invocations, and error handling.

This model supports two modes:

  • Local mode: No account association (single-tenant, self-hosted)
  • Multi-tenant mode: With account association (for activeagents.ai platform)

Examples:

Creating a trace from ingested data (local mode)

ActionAgent::TelemetryTrace.create_from_payload(trace_payload, sdk_info)

Creating a trace with account (multi-tenant mode)

ActionAgent::TelemetryTrace.create_from_payload(trace_payload, sdk_info, account: )

Constant Summary collapse

STATUS_OK =

Status values for traces

"OK"
STATUS_ERROR = 
"ERROR"
STATUS_UNSET = 
"UNSET"

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.create_from_payload(trace, sdk_info = {}, account: nil) ⇒ Object 



84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
 
# File 'app/models/action_agent/telemetry_trace.rb', line 84

def self.create_from_payload(trace, sdk_info = {}, account: nil)
  spans = trace["spans"] || []
  root_span = spans.find { |s| s["parent_span_id"].nil? } || spans.first || {}

  total_duration = root_span["duration_ms"]

  # Instrumentation mirrors LLM token usage onto the root span for
  # display, so summing every span double-counts. When child spans
  # carry token data, they are the source of truth; the root span only
  # counts for single-span traces.
  counted_spans = spans.reject { |s| s["parent_span_id"].nil? }
  counted_spans = spans if counted_spans.none? { |s| span_token_sum(s).positive? }

  total_input = 0
  total_output = 0
  total_thinking = 0

  counted_spans.each do |span|
    tokens = span["tokens"] || {}
    total_input += (tokens["input"] || 0)
    total_output += (tokens["output"] || 0)
    total_thinking += (tokens["thinking"] || 0)
  end

  # Extract agent info from root span attributes
  attributes = root_span["attributes"] || {}
  agent_class = attributes["agent.class"]
  agent_action = attributes["agent.action"]

  # Find any error message
  error_span = spans.find { |s| s["status"] == STATUS_ERROR }
  error_message = error_span&.dig("attributes", "error.message")

  attrs = {
    trace_id: trace["trace_id"],
    service_name: trace["service_name"],
    environment: trace["environment"],
    timestamp: Time.parse(trace["timestamp"]),
    spans: spans,
    resource_attributes: trace["resource_attributes"],
    sdk_info: sdk_info,
    total_duration_ms: total_duration,
    total_input_tokens: total_input,
    total_output_tokens: total_output,
    total_thinking_tokens: total_thinking,
    status: root_span["status"] || STATUS_UNSET,
    agent_class: agent_class,
    agent_action: agent_action,
    error_message: error_message
  }

  # Add account if in multi-tenant mode
  attrs[:account] =  if ActionAgent.multi_tenant? && 

  create!(attrs)
end

.pluck_with_llm_model(scope, *columns) ⇒ TelemetryTrace

Creates a TelemetryTrace from an ingested trace payload.

Extracts relevant data from the trace payload and stores it in a normalized format for querying and analysis.

Plucks [llm_model, *columns] per trace, where llm_model comes from the first llm span. PostgreSQL digs into the spans jsonb in SQL so span payloads never reach Ruby; other adapters read the column back and dig in Ruby, which costs more but keeps the dashboard adapter-agnostic.

Parameters:

  • trace (Hash) —

    The trace payload from ActiveAgent::Telemetry

  • sdk_info (Hash) —

    SDK metadata

  • account (Object, nil) —

    Optional account for multi-tenant mode

Returns:



62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
 
# File 'app/models/action_agent/telemetry_trace.rb', line 62

def self.pluck_with_llm_model(scope, *columns)
  if postgres?
    scope.pluck(
      Arel.sql(
        # spans is cast rather than assumed to be jsonb: the column is
        # json on every install created before the migration template
        # started picking jsonb per adapter, and jsonb_array_elements
        # rejects a json argument outright.
        "(SELECT s.value -> 'attributes' ->> 'llm.model' " \
        "FROM jsonb_array_elements(spans::jsonb) AS s " \
        "WHERE s.value ->> 'type' = 'llm' LIMIT 1)"
      ),
      *columns
    )
  else
    scope.pluck(:spans, *columns).map do |spans, *rest|
      llm = Array(spans).find { |span| span.is_a?(Hash) && span["type"].to_s == "llm" }
      [ llm&.dig("attributes", "llm.model"), *rest ]
    end
  end
end

.span_token_sum(span) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Sums a span's token counts (used to decide which spans carry the authoritative token data during ingestion).



145
146
147
148
 
# File 'app/models/action_agent/telemetry_trace.rb', line 145

def self.span_token_sum(span)
  tokens = span["tokens"] || {}
  tokens.fetch("input", 0).to_i + tokens.fetch("output", 0).to_i + tokens.fetch("thinking", 0).to_i
end

Instance Method Details

#declared_tools ⇒ Array<Hash>

Returns the tools this trace's generation request OFFERED the provider, whether or not the model went on to call any of them.

Instrumentation records the roster on the prompt span as prompt.input.tools (name, description, parameter keys), which is the agent's declared tool surface for that generation. Reading it here is what lets a dashboard show a tool that exists but has never been invoked — a state that tool spans alone can't express.

Returns:

  • (Array<Hash>) —

    entries with :name, :description, :parameters, :origin and :mcp_server



213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
 
# File 'app/models/action_agent/telemetry_trace.rb', line 213

def declared_tools
  Array(tool_roster).filter_map do |tool|
    next unless tool.is_a?(Hash)

    name = (tool["name"] || tool[:name]).to_s
    next if name.empty?

    classification = ActiveAgent::Telemetry::ToolOrigin.classify(name)
    {
      name: name,
      description: (tool["description"] || tool[:description]).presence,
      parameters: normalize_parameters(tool["parameters"] || tool[:parameters]),
      origin: classification[:origin],
      mcp_server: classification[:server]
    }
  end
end

#display_name ⇒ String

Returns display name for the trace.

Returns:

  • (String) —

    Display name (e.g., "WeatherAgent.forecast")



257
258
259
260
261
262
263
264
265
 
# File 'app/models/action_agent/telemetry_trace.rb', line 257

def display_name
  if agent_class && agent_action
    "#{agent_class}.#{agent_action}"
  elsif agent_class
    agent_class
  else
    trace_id&.first(8)
  end
end

#error? ⇒ Boolean

Returns whether this trace had an error.

Returns:

  • (Boolean) 


250
251
252
 
# File 'app/models/action_agent/telemetry_trace.rb', line 250

def error?
  status == STATUS_ERROR
end

#formatted_duration ⇒ String

Returns formatted duration.

Returns:

  • (String) —

    Duration in ms or s



270
271
272
273
274
275
276
277
278
 
# File 'app/models/action_agent/telemetry_trace.rb', line 270

def formatted_duration
  return "—" unless total_duration_ms

  if total_duration_ms >= 1000
    "#{(total_duration_ms / 1000.0).round(2)}s"
  else
    "#{total_duration_ms.round(0)}ms"
  end
end

#formatted_tokens ⇒ String

Returns formatted token count.

Returns:

  • (String) —

    Token count with K suffix for large numbers



283
284
285
286
287
288
289
290
291
292
 
# File 'app/models/action_agent/telemetry_trace.rb', line 283

def formatted_tokens
  count = total_tokens
  return "0" if count.zero?

  if count >= 1000
    "#{(count / 1000.0).round(1)}K"
  else
    count.to_s
  end
end

#llm_spans ⇒ Array<Hash>

Returns all LLM spans in this trace.

Returns:

  • (Array<Hash>) —

    LLM spans



160
161
162
 
# File 'app/models/action_agent/telemetry_trace.rb', line 160

def llm_spans
  spans&.select { |s| s["type"] == "llm" } || []
end

#mcp_servers ⇒ Array<String>

Returns the distinct MCP servers this trace touched — both the ones it called and the ones it was merely offered.

Returns:

  • (Array<String>) —

    server names, in first-seen order



235
236
237
238
 
# File 'app/models/action_agent/telemetry_trace.rb', line 235

def mcp_servers
  (tool_usage.filter_map { |tool| tool[:mcp_server] } +
    declared_tools.filter_map { |tool| tool[:mcp_server] }).uniq
end

#model ⇒ String?

Returns the model used (from LLM spans).

Returns:

  • (String, nil) —

    Model name



307
308
309
310
311
312
 
# File 'app/models/action_agent/telemetry_trace.rb', line 307

def model
  llm_span = llm_spans.first
  return nil unless llm_span

  llm_span.dig("attributes", "llm.model")
end

#provider ⇒ String?

Returns the provider used (from LLM spans).

Returns:

  • (String, nil) —

    Provider name



297
298
299
300
301
302
 
# File 'app/models/action_agent/telemetry_trace.rb', line 297

def provider
  llm_span = llm_spans.first
  return nil unless llm_span

  llm_span.dig("attributes", "llm.provider")
end

#root_span ⇒ Hash?

Returns the root span of this trace.

Returns:

  • (Hash, nil) —

    The root span or nil



153
154
155
 
# File 'app/models/action_agent/telemetry_trace.rb', line 153

def root_span
  spans&.find { |s| s["parent_span_id"].nil? }
end

#tool_spans ⇒ Array<Hash>

Returns all tool call spans in this trace.

Returns:

  • (Array<Hash>) —

    Tool spans



167
168
169
 
# File 'app/models/action_agent/telemetry_trace.rb', line 167

def tool_spans
  spans&.select { |s| s["type"] == "tool" } || []
end

#tool_usage ⇒ Array<Hash>

Returns each tool call in this trace, normalized for display.

Tool spans are tagged with their origin at instrumentation time (ActiveAgent::Telemetry::ToolOrigin), but traces ingested before that shipped — or sent by another SDK — only carry tool.name. Those are classified on read from the same naming convention, so a dashboard sees consistent attribution across old and new traces.

Returns:

  • (Array<Hash>) —

    one entry per tool span with :name, :base_name, :origin, :mcp_server, :duration_ms, :status, :error, :arguments and :result



182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
 
# File 'app/models/action_agent/telemetry_trace.rb', line 182

def tool_usage
  tool_spans.map do |span|
    attributes = span["attributes"] || {}
    name = attributes["tool.name"] || span["name"].to_s.delete_prefix("tool.")
    classification = classify_tool(name, attributes)

    {
      name: name,
      base_name: attributes["tool.base_name"] || classification[:tool],
      origin: attributes["tool.origin"] || classification[:origin],
      mcp_server: attributes["tool.mcp_server"] || classification[:server],
      duration_ms: span["duration_ms"],
      status: span["status"],
      error: attributes["error.message"],
      arguments: attributes["tool.input.args"],
      result: attributes["tool.output.result"]
    }
  end
end

#total_tokens ⇒ Integer

Returns total token count.

Returns:

  • (Integer) —

    Total tokens used



243
244
245
 
# File 'app/models/action_agent/telemetry_trace.rb', line 243

def total_tokens
  (total_input_tokens || 0) + (total_output_tokens || 0) + (total_thinking_tokens || 0)
end