Class: RubyLLM::Agents::AttemptTracker Private

Inherits:
Object
  • Object
show all
Defined in:
lib/ruby_llm/agents/infrastructure/attempt_tracker.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

Tracks attempts during agent execution with reliability features

Records details about each attempt (retries and fallbacks) during an execution, including timing, token usage, and errors. This data is stored in the execution record’s ‘attempts` JSONB array.

Examples:

Tracking attempts

tracker = AttemptTracker.new
attempt = tracker.start_attempt("gpt-4o")
# ... execute LLM call ...
tracker.complete_attempt(attempt, success: true, response: response)

See Also:

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ AttemptTracker

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of AttemptTracker.



22
23
24
25
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 22

def initialize
  @attempts = []
  @current_attempt = nil
end

Instance Attribute Details

#attempts ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



20
21
22
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 20

def attempts
  @attempts
end

Instance Method Details

#attempts_count ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the number of attempts made

Returns:

  • (Integer) —

    Number of attempts



204
205
206
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 204

def attempts_count
  @attempts.length
end

#chosen_model_id ⇒ String?

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the chosen model (from successful attempt)

Returns:

  • (String, nil) —

    The model ID that succeeded



160
161
162
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 160

def chosen_model_id
  successful_attempt&.dig(:model_id)
end

#complete_attempt(attempt, success:, response: nil, error: nil) ⇒ Hash

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Completes the current attempt with results

Parameters:

  • attempt (Hash) —

    The attempt hash from start_attempt

  • success (Boolean) —

    Whether the attempt succeeded

  • response (Object, nil) (defaults to: nil) —

    The LLM response (if successful)

  • error (Exception, nil) (defaults to: nil) —

    The error (if failed)

Returns:

  • (Hash) —

    The completed attempt



58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 58

def complete_attempt(attempt, success:, response: nil, error: nil)
  started_at = Time.parse(attempt[:started_at])
  completed_at = Time.current

  attempt[:completed_at] = completed_at.iso8601
  attempt[:duration_ms] = ((completed_at - started_at) * 1000).round

  if response
    attempt[:input_tokens] = safe_value(response, :input_tokens)
    attempt[:output_tokens] = safe_value(response, :output_tokens)
    attempt[:cached_tokens] = safe_value(response, :cached_tokens, 0)
    attempt[:cache_creation_tokens] = safe_value(response, :cache_creation_tokens, 0)
    attempt[:model_id] = safe_value(response, :model_id) || attempt[:model_id]
  end

  if error
    attempt[:error_class] = error.class.name
    attempt[:error_message] = error.message.to_s.truncate(1000)
    attempt[:error_backtrace] = error.backtrace&.first(20)
  end

  @attempts << attempt
  @current_attempt = nil

  emit_finish_notification(attempt, success)

  attempt
end

#failed_attempts ⇒ Array<Hash>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns all failed attempts

Returns:

  • (Array<Hash>) —

    Failed attempt data



134
135
136
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 134

def failed_attempts
  @attempts.select { |a| a[:error_class].present? }
end

#failed_attempts_count ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the number of failed attempts

Useful for debugging in rails console.

Returns:

  • (Integer) —

    Number of failed attempts



213
214
215
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 213

def failed_attempts_count
  @attempts.count { |a| a[:error_class].present? }
end

#last_failed_attempt ⇒ Hash?

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Finds the last failed attempt

Useful for debugging in rails console.

Returns:

  • (Hash, nil) —

    The last failed attempt or nil



127
128
129
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 127

def last_failed_attempt
  @attempts.reverse.find { |a| a[:error_class].present? }
end

#record_short_circuit(model_id) ⇒ Hash

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Records a short-circuited attempt (circuit breaker open)

Parameters:

  • model_id (String) —

    The model identifier

Returns:

  • (Hash) —

    The recorded attempt



91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 91

def record_short_circuit(model_id)
  now = Time.current

  attempt = {
    model_id: model_id,
    started_at: now.iso8601,
    completed_at: now.iso8601,
    duration_ms: 0,
    input_tokens: nil,
    output_tokens: nil,
    cached_tokens: 0,
    cache_creation_tokens: 0,
    error_class: "RubyLLM::Agents::Reliability::CircuitBreakerOpenError",
    error_message: "Circuit breaker is open",
    short_circuited: true
  }

  @attempts << attempt

  emit_short_circuit_notification(attempt)

  attempt
end

#short_circuited_count ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the number of short-circuited attempts

Useful for debugging in rails console.

Returns:

  • (Integer) —

    Number of short-circuited attempts



222
223
224
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 222

def short_circuited_count
  @attempts.count { |a| a[:short_circuited] }
end

#start_attempt(model_id) ⇒ Hash

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Starts tracking a new attempt

Parameters:

  • model_id (String) —

    The model identifier being used

Returns:

  • (Hash) —

    The attempt hash (pass to complete_attempt)



31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 31

def start_attempt(model_id)
  @current_attempt = {
    model_id: model_id,
    started_at: Time.current.iso8601,
    completed_at: nil,
    duration_ms: nil,
    input_tokens: nil,
    output_tokens: nil,
    cached_tokens: 0,
    cache_creation_tokens: 0,
    error_class: nil,
    error_message: nil,
    short_circuited: false
  }

  emit_start_notification(@current_attempt)

  @current_attempt
end

#successful_attempt ⇒ Hash?

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Finds the successful attempt (if any)

Returns:

  • (Hash, nil) —

    The successful attempt or nil



118
119
120
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 118

def successful_attempt
  @attempts.find { |a| a[:error_class].nil? && !a[:short_circuited] }
end

#to_json_array ⇒ Array<Hash>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns attempts as JSON-compatible array

Returns:

  • (Array<Hash>) —

    Attempts data for persistence



229
230
231
232
233
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 229

def to_json_array
  @attempts.map do |attempt|
    attempt.transform_keys(&:to_s)
  end
end

#total_cached_tokens ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Calculates total cached tokens across all attempts

Returns:

  • (Integer) —

    Total cached tokens



188
189
190
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 188

def total_cached_tokens
  @attempts.sum { |a| a[:cached_tokens] || 0 }
end

#total_duration_ms ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Calculates total duration across all attempts

Useful for debugging in rails console.

Returns:

  • (Integer) —

    Total duration in milliseconds



197
198
199
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 197

def total_duration_ms
  @attempts.sum { |a| a[:duration_ms] || 0 }
end

#total_input_tokens ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Calculates total input tokens across all attempts

Returns:

  • (Integer) —

    Total input tokens



174
175
176
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 174

def total_input_tokens
  @attempts.sum { |a| a[:input_tokens] || 0 }
end

#total_output_tokens ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Calculates total output tokens across all attempts

Returns:

  • (Integer) —

    Total output tokens



181
182
183
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 181

def total_output_tokens
  @attempts.sum { |a| a[:output_tokens] || 0 }
end

#total_tokens ⇒ Integer

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Calculates total tokens across all attempts

Returns:

  • (Integer) —

    Total input + output tokens



167
168
169
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 167

def total_tokens
  @attempts.sum { |a| (a[:input_tokens] || 0) + (a[:output_tokens] || 0) }
end

#used_fallback? ⇒ Boolean

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Checks if a fallback model was used

A fallback was used if multiple models were tried (more than one unique model) or if the successful model is different from the first attempted model.

Returns:

  • (Boolean) —

    true if a fallback model was used



144
145
146
147
148
149
150
151
152
153
154
155
 
# File 'lib/ruby_llm/agents/infrastructure/attempt_tracker.rb', line 144

def used_fallback?
  return false if @attempts.empty?

  first_model = @attempts.first[:model_id]
  successful = successful_attempt

  # Fallback used if successful model differs from first attempted
  return true if successful && successful[:model_id] != first_model

  # Or if we tried multiple models (even if all failed)
  @attempts.map { |a| a[:model_id] }.uniq.length > 1
end