Class: Brute::Compaction::Middleware::SlidingWindow

Inherits:
Strategy show all
Defined in:
lib/brute/compaction/middleware/sliding_window.rb

Overview

When rewriting the tool output was not enough, whole stretches of the conversation go.

The instructions and the message the current task hangs off are never given up. Past that, the oldest complete turns go first; only once every turn is gone does the current task start losing its own oldest steps, and never the newest.

A note stands where the removed messages were, so the model can see that something was there rather than quietly reasoning from a gap.

Constant Summary collapse

NOTE = 
"%d earlier messages were dropped to free context. They cannot be recovered."

Constants included from Hooks

Hooks::AFTER_LLM_EVENT, Hooks::AFTER_TOOL_EVENT, Hooks::APPROVE_TOOL_EVENT, Hooks::BEFORE_LLM_EVENT, Hooks::BEFORE_TOOL_EVENT, Hooks::COMPACTED_EVENT, Hooks::COMPACT_DURATION_EVENT, Hooks::DURATION_EVENT, Hooks::ENTER_EVENT, Hooks::EXIT_EVENT, Hooks::FARADAY_ERROR_EVENT, Hooks::LLM_DURATION_EVENT, Hooks::LLM_FAILURE_EVENT, Hooks::MIDDLEWARE_ADDED_EVENT, Hooks::OPEN_ROUTER_SERVER_ERROR_EVENT, Hooks::STANDARD_ERROR_EVENT, Hooks::TOOL_DURATION_EVENT, Hooks::TURN_DURATION_EVENT, Hooks::TURN_END_EVENT, Hooks::TURN_START_EVENT

Instance Method Summary collapse

Methods inherited from Strategy

#call

Methods inherited from Middleware::Base

#call

Methods included from Hooks

new

Constructor Details

#initialize(app = nil, keep_steps: 1) ⇒ SlidingWindow

Returns a new instance of SlidingWindow.



22
23
24
25
 
# File 'lib/brute/compaction/middleware/sliding_window.rb', line 22

def initialize(app = nil, keep_steps: 1)
  @app = app
  @keep_steps = keep_steps
end

Instance Method Details

#rewrite(messages, target:) ⇒ Object 



29
30
31
32
33
34
35
36
37
38
39
40
41
 
# File 'lib/brute/compaction/middleware/sliding_window.rb', line 29

def rewrite(messages, target:)
  kept, note_index, dropped = split(messages, target)

  # Swapping one note for another frees nothing. The base would refuse
  # it anyway; declining here says why.
  if dropped.any? && !dropped.all? { |index| Brute::Compaction::Transcript.marked?(messages[index], strategy) }
    note = Brute::Message.new(
      role: :user,
      content: Brute::Compaction::Transcript.mark(strategy, format(NOTE, dropped.length)),
    )
    [*kept[...note_index], note, *kept[note_index..]]
  end
end

#strategy ⇒ Object 



27
 
# File 'lib/brute/compaction/middleware/sliding_window.rb', line 27

def strategy = "sliding_window"