Class: Brute::Middleware::CompactionCheck

Inherits:
Base
  • Object
show all
Defined in:
lib/brute/middleware/compaction_check.rb

Overview

Checks context size after each LLM call and triggers compaction when thresholds are exceeded.

Runs POST-call: inspects message count and token usage. If compaction is needed, summarizes older messages and replaces env with the summary so the next LLM call starts with a compact history.

Instance Method Summary collapse

Constructor Details

#initialize(app, compactor:, system_prompt:) ⇒ CompactionCheck

Returns a new instance of CompactionCheck.



16
17
18
19
20
 
# File 'lib/brute/middleware/compaction_check.rb', line 16

def initialize(app, compactor:, system_prompt:)
  super(app)
  @compactor = compactor
  @system_prompt = system_prompt
end

Instance Method Details

#call(env) ⇒ Object 



22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
 
# File 'lib/brute/middleware/compaction_check.rb', line 22

def call(env)
  response = @app.call(env)

  messages = env[:messages]
  usage = env[:metadata].dig(:tokens, :last_call)

  if @compactor.should_compact?(messages, usage: usage)
    result = @compactor.compact(messages)
    if result
      summary_text, _recent = result
      env[:metadata][:compaction] = {
        messages_before: messages.size,
        timestamp: Time.now.iso8601,
      }
      # Replace the message history with the summary
      env[:messages] = [
        LLM::Message.new(:system, @system_prompt),
        LLM::Message.new(:user, "[Previous conversation summary]\n\n#{summary_text}"),
      ]
    end
  end

  response
end