Class: Brute::Compaction::Middleware::ToolResults

Inherits:
Strategy show all
Defined in:
lib/brute/compaction/middleware/tool_results.rb

Overview

The cheapest thing to give up first: what the tools said.

Tool output dominates a long run, and most of it stops being useful the moment the model has acted on it. So the results are rewritten in place rather than removed -- every call keeps the result that answers it, the model can still see what it ran, and it can run it again if it turns out it still needed the answer.

Oldest first, and it stops the moment the transcript is under target, so the most recent output survives the longest.

Constant Summary collapse

PLACEHOLDER = 
"Result dropped to free context. Call %s again if you still need it."

Constants included from Hooks

Hooks::AFTER_LLM_EVENT, Hooks::AFTER_TOOL_EVENT, Hooks::APPROVE_TOOL_EVENT, Hooks::BEFORE_LLM_EVENT, Hooks::BEFORE_TOOL_EVENT, Hooks::COMPACTED_EVENT, Hooks::COMPACT_DURATION_EVENT, Hooks::DURATION_EVENT, Hooks::ENTER_EVENT, Hooks::EXIT_EVENT, Hooks::FARADAY_ERROR_EVENT, Hooks::LLM_DURATION_EVENT, Hooks::LLM_FAILURE_EVENT, Hooks::MIDDLEWARE_ADDED_EVENT, Hooks::OPEN_ROUTER_SERVER_ERROR_EVENT, Hooks::STANDARD_ERROR_EVENT, Hooks::TOOL_DURATION_EVENT, Hooks::TURN_DURATION_EVENT, Hooks::TURN_END_EVENT, Hooks::TURN_START_EVENT

Instance Method Summary collapse

Methods inherited from Strategy

#call

Methods inherited from Middleware::Base

#call

Methods included from Hooks

new

Constructor Details

#initialize(app = nil, keep_steps: 1, min_tokens: 200) ⇒ ToolResults

Returns a new instance of ToolResults.



22
23
24
25
26
27
28
29
30
 
# File 'lib/brute/compaction/middleware/tool_results.rb', line 22

def initialize(app = nil, keep_steps: 1, min_tokens: 200)
  if keep_steps < 1
    raise ArgumentError, "keep_steps must be at least 1: the model has not acted on the newest results yet"
  end

  @app = app
  @keep_steps = keep_steps
  @min_tokens = min_tokens
end

Instance Method Details

#rewrite(messages, target:) ⇒ Object 



34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
 
# File 'lib/brute/compaction/middleware/tool_results.rb', line 34

def rewrite(messages, target:)
  guarded = guarded_indices(messages)
  names   = tool_names(messages)
  pruned  = messages.dup
  running = tokens(messages)
  changed = false

  messages.each_with_index do |message, index|
    if running <= target
      break
    end

    replacement = prune(message, names[message.tool_call_id])

    unless guarded.include?(index) || replacement.nil?
      running -= tokens([message]) - tokens([replacement])
      pruned[index] = replacement
      changed = true
    end
  end

  if changed
    pruned
  end
end

#strategy ⇒ Object 



32
 
# File 'lib/brute/compaction/middleware/tool_results.rb', line 32

def strategy = "tool_results"