Class: Crimson::Compactor

Inherits:
Object
  • Object
show all
Defined in:
lib/crimson/compactor.rb

Overview

Compacts conversation history by summarizing older messages when context limits are approached.

Constant Summary collapse

DEFAULT_MAX_CONTEXT_TOKENS =

Default max context tokens before compaction is triggered.

100_000
KEEP_RECENT_MESSAGES =

Number of most recent messages to preserve verbatim during compaction.

4

Instance Method Summary collapse

Constructor Details

#initialize(client:, max_context_tokens: DEFAULT_MAX_CONTEXT_TOKENS, model: nil, provider: nil) ⇒ Compactor

Returns a new instance of Compactor.

Parameters:

  • client (Client::Base) —

    the API client used for summarization

  • max_context_tokens (Integer) (defaults to: DEFAULT_MAX_CONTEXT_TOKENS) —

    threshold for triggering compaction

  • model (String, nil) (defaults to: nil) —

    model name for token counting

  • provider (String, nil) (defaults to: nil) —

    provider name



15
16
17
18
19
 
# File 'lib/crimson/compactor.rb', line 15

def initialize(client:, max_context_tokens: DEFAULT_MAX_CONTEXT_TOKENS, model: nil, provider: nil)
  @client = client
  @max_context_tokens = max_context_tokens
  @token_counter = TokenCounter.new(model: model, provider: provider)
end

Instance Method Details

#compact(history, system_prompt:) ⇒ Array<Message::Base>

Compact history by summarizing older entries and keeping recent ones verbatim.

Parameters:

Returns:



32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
 
# File 'lib/crimson/compactor.rb', line 32

def compact(history, system_prompt:)
  return history if history.length <= KEEP_RECENT_MESSAGES + 1

  older = history[0...-KEEP_RECENT_MESSAGES]
  recent = history[-KEEP_RECENT_MESSAGES..]

  file_ops = extract_file_operations(history)
  summary = summarize(older, system_prompt, file_ops)

  compacted = []
  compacted << Message::User.new("[Previous conversation summary]\n#{summary}")
  compacted << Message::Assistant.new(content: "Understood. I have the context from the previous conversation.")
  compacted.concat(recent)
  compacted
end

#needs_compaction?(history) ⇒ Boolean

Check whether the history exceeds 80% of the max context token budget.

Parameters:

Returns:

  • (Boolean) 


24
25
26
 
# File 'lib/crimson/compactor.rb', line 24

def needs_compaction?(history)
  estimated_tokens(history) > @max_context_tokens * 0.8
end