Class: Brute::Middleware::DefaultCompactionPipeline

Inherits:
Base
  • Object
show all
Defined in:
lib/brute/middleware/040_default_compaction_pipeline.rb

Overview

Compacts the conversation once it fills too much of the model's window.

This layer owns the when: before each call it estimates the size of the conversation, and once it reaches compact_at of the window it asks a compactor to bring it down to compact_to. What is given up, and in what order, is the compactor's business.

use Brute::Middleware::Loop::ToolResult
use Brute::Middleware::DefaultCompactionPipeline,
window:     200_000,
summariser: Brute::Completion::OpenRouter.new(config: { access_token: key })
use Brute::Middleware::DefaultToolPipeline, tools: tools

The compactor it builds is the usual ladder, cheapest first:

use Brute::Compaction::Middleware::ToolResults     rewrite older tool output
use Brute::Compaction::Middleware::SlidingWindow   drop the oldest turns, then steps
run Brute::Compaction::Summarize                   summarise what is left, until it fits

Pass compactor: to say something else. A Brute::Turn::CompactionPipeline is the usual thing to pass, but anything answering #compact(messages, target:) will do. Passing no summariser leaves the terminal doing nothing, which is how an agent says it would rather live with a full context than pay to shrink it.

It belongs inside the tool loop rather than around it, so it runs before every call rather than once a turn -- a long run of tool results can fill the window without the turn ever ending.

Sizing anchors on what the provider itself counted for the last call and measures locally only what has been appended since, so the estimate is exact for the bulk of the conversation and approximate only for its tail.

Tool schemas ride in every request and are part of what fills a window, so they are counted too -- against the trigger when nothing has been reported yet, and off the target the compactor is given, because what the schemas occupy is not room the conversation may have. This layer sits above the tool pipeline, so on the very first call of a turn env is not set yet: pass tools: to have them counted then.

Compaction is lossy, and this layer keeps no record of what it gave up: it rewrites env and says so with :compacted. An application that keeps a transcript preserves it from there.

agent.on(:compacted) { |env, payload| archive(env, payload) }

Defined Under Namespace

Classes: Declines

Constant Summary

Constants included from Hooks

Hooks::AFTER_LLM_EVENT, Hooks::AFTER_TOOL_EVENT, Hooks::APPROVE_TOOL_EVENT, Hooks::BEFORE_LLM_EVENT, Hooks::BEFORE_TOOL_EVENT, Hooks::COMPACTED_EVENT, Hooks::COMPACT_DURATION_EVENT, Hooks::DURATION_EVENT, Hooks::ENTER_EVENT, Hooks::EXIT_EVENT, Hooks::FARADAY_ERROR_EVENT, Hooks::LLM_DURATION_EVENT, Hooks::LLM_FAILURE_EVENT, Hooks::MIDDLEWARE_ADDED_EVENT, Hooks::OPEN_ROUTER_SERVER_ERROR_EVENT, Hooks::STANDARD_ERROR_EVENT, Hooks::TOOL_DURATION_EVENT, Hooks::TURN_DURATION_EVENT, Hooks::TURN_END_EVENT, Hooks::TURN_START_EVENT

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Hooks

new

Constructor Details

#initialize(app, window:, summariser: nil, compactor: nil, keep_steps: 2, compact_at: 0.7, compact_to: 0.4, token_counter: nil, tools: nil) ⇒ DefaultCompactionPipeline

Returns a new instance of DefaultCompactionPipeline.



55
56
57
58
59
60
61
62
63
64
65
66
67
68
 
# File 'lib/brute/middleware/040_default_compaction_pipeline.rb', line 55

def initialize(app, window:, summariser: nil, compactor: nil, keep_steps: 2,
               compact_at: 0.7, compact_to: 0.4, token_counter: nil, tools: nil)
  unless compact_to.positive? && compact_to < compact_at && compact_at <= 1
    raise ArgumentError, "expected 0 < compact_to < compact_at <= 1, got #{compact_to} and #{compact_at}"
  end

  @app = app
  @compactor = compactor || self.class.compactor(summariser: summariser, keep_steps: keep_steps)
  @window = window
  @compact_at = compact_at
  @compact_to = compact_to
  @token_counter = token_counter || Brute::TokenCounter.default
  @tools = tools
end

Class Method Details

.compactor(summariser: nil, keep_steps: 2) ⇒ Object

The ladder this layer wires when it is not given one.



78
79
80
81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/brute/middleware/040_default_compaction_pipeline.rb', line 78

def self.compactor(summariser: nil, keep_steps: 2)
  terminal = Declines.new

  unless summariser.nil?
    terminal = Brute::Compaction::Summarize.new(summariser, keep_steps: keep_steps)
  end

  Brute::Turn::CompactionPipeline.new do
    use Brute::Compaction::Middleware::ToolResults, keep_steps: 1
    use Brute::Compaction::Middleware::SlidingWindow, keep_steps: keep_steps

    run terminal
  end
end

Instance Method Details

#call(env) ⇒ Object 



70
71
72
73
74
75
 
# File 'lib/brute/middleware/040_default_compaction_pipeline.rb', line 70

def call(env)
  env[:compactor] = @compactor
  compact(env)
  @app.call(env)
  env
end