Class: Brute::Middleware::ToolErrorTracking

Inherits:
Base
  • Object
show all
Defined in:
lib/brute/middleware/tool_error_tracking.rb

Overview

Tracks per-tool error counts and total tool call count across LLM calls, and signals when the error ceiling is reached.

This middleware doesn’t execute tools itself — it inspects the tool results that were sent as input to the LLM call (env) and counts failures and totals.

When any tool exceeds max_failures, it sets env[:tool_error_limit_reached] so the agent loop can decide to stop.

Also stores env[:tool_calls] with the cumulative number of tool invocations in the current session.

Constant Summary collapse

DEFAULT_MAX_FAILURES = 
3

Instance Method Summary collapse

Constructor Details

#initialize(app, max_failures: DEFAULT_MAX_FAILURES) ⇒ ToolErrorTracking

Returns a new instance of ToolErrorTracking.



24
25
26
27
28
29
 
# File 'lib/brute/middleware/tool_error_tracking.rb', line 24

def initialize(app, max_failures: DEFAULT_MAX_FAILURES)
  super(app)
  @max_failures = max_failures
  @errors = Hash.new(0) # tool_name → count
  @total_tool_calls = 0
end

Instance Method Details

#call(env) ⇒ Object 



31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
 
# File 'lib/brute/middleware/tool_error_tracking.rb', line 31

def call(env)
  # PRE: count errors and totals from tool results that are about to be sent
  if (results = env[:tool_results])
    @total_tool_calls += results.size

    results.each do |name, result|
      @errors[name] += 1 if result.is_a?(Hash) && result[:error]
    end
  end

  env[:metadata][:tool_calls] = @total_tool_calls
  env[:metadata][:tool_errors] = @errors.dup
  env[:metadata][:tool_error_limit_reached] = @errors.any? { |_, c| c >= @max_failures }

  if env[:metadata][:tool_error_limit_reached]
    failed_tool, fail_count = @errors.max_by { |_, c| c }
    env[:should_exit] ||= {
      reason:  "tool_error_limit_reached",
      message: "Tool '#{failed_tool}' has failed #{fail_count} times (limit: #{@max_failures}). Stopping.",
      source:  "ToolErrorTracking",
    }
  end

  @app.call(env)
end

#reset! ⇒ Object

Reset counts (e.g., between user turns).



58
59
60
61
 
# File 'lib/brute/middleware/tool_error_tracking.rb', line 58

def reset!
  @errors.clear
  @total_tool_calls = 0
end