Class: Ask::Agent::Compactor

Inherits:
Object
  • Object
show all
Defined in:
lib/ask/agent/compactor.rb

Overview

Context compaction for long sessions. When conversation tokens approach the model's context window, older messages are summarized and replaced with a structured summary, preserving a recent tail verbatim.

Trigger modes:

  1. Threshold — tokens exceed (context_window - reserve). Compact, no retry.
  2. Overflow — LLM returned context overflow. Compact, then auto-retry.

Reserve is model-aware: it defaults to the model's declared max output tokens (capped at DEFAULT_RESERVE_TOKENS), so compaction leaves exactly the headroom a single turn can consume. For models without declared limits, a static default applies. A safety floor clamps the reserve for tiny-window models so threshold compaction can fire usefully instead of triggering on every turn.

The recent tail is also token-aware: #keep_recent_tokens preserves the last N tokens of conversation verbatim (recent-context fidelity depends on the active work, not on message counts) and summarizes only what's older. When not configured, the legacy fixed message-count behavior is used for backward compatibility.

Constant Summary collapse

CONTEXT_WINDOWS = 
{
  "gpt-4o" => 128_000,
  "gpt-4o-mini" => 128_000,
  "gpt-4-turbo" => 128_000,
  "claude-sonnet-4" => 200_000,
  "claude-4" => 200_000,
  "gemini-2.0-flash" => 1_048_576,
  "gemini-2.5-pro" => 1_048_576,
  "deepseek-v4-flash" => 1_000_000,
  "deepseek-v4-pro" => 1_000_000,
}.tap { |h| h.default = 128_000 }
DEFAULT_RESERVE_TOKENS =

Default headroom reserved for a single model turn when the model's max output tokens are unknown.

20_000
DEFAULT_KEEP_RECENT_TOKENS =

Default verbatim recent-context tail preserved during compaction.

8_000
MIN_RESERVE_TOKENS =

Minimum reserve for tiny-window models (safety floor).

1_024
DEFAULT_KEEP_COUNT =

Legacy fixed message-count tail (backward compatibility).

8
MIN_MESSAGES =

Conversations smaller than this are never compacted.

6

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(threshold: nil, strategy: :proactive, llm: nil, reserve_tokens: nil, keep_recent_tokens: nil, keep_count: DEFAULT_KEEP_COUNT, min_messages: MIN_MESSAGES) ⇒ Compactor

Returns a new instance of Compactor.

Parameters:

  • threshold (Float, nil) (defaults to: nil) —

    Compact when tokens exceed context_window * threshold. When nil (default), compaction triggers at context_window - reserve_tokens (model-aware).

  • strategy (Symbol) (defaults to: :proactive) —

    Reserved; :proactive is the only strategy.

  • llm (Object, String, nil) (defaults to: nil) —

    LLM used for summarization. When nil, a heuristic summary is generated instead.

  • reserve_tokens (Integer, nil) (defaults to: nil) —

    Explicit headroom for one turn. When nil, derived from the model's max output tokens (see #derive_reserve).

  • keep_recent_tokens (Integer, nil) (defaults to: nil) —

    Explicit verbatim recent-tail budget. When nil, the legacy fixed message-count tail is preserved.

  • keep_count (Integer) (defaults to: DEFAULT_KEEP_COUNT) —

    Legacy fixed tail in messages when keep_recent_tokens is nil.

  • min_messages (Integer) (defaults to: MIN_MESSAGES) —

    Conversations with fewer messages are never compacted.



71
72
73
74
75
76
77
78
79
80
81
82
83
 
# File 'lib/ask/agent/compactor.rb', line 71

def initialize(threshold: nil, strategy: :proactive, llm: nil,
               reserve_tokens: nil, keep_recent_tokens: nil,
               keep_count: DEFAULT_KEEP_COUNT, min_messages: MIN_MESSAGES)
  @threshold = threshold
  @strategy = strategy
  @llm = llm
  @reserve_tokens = reserve_tokens
  @keep_recent_tokens = keep_recent_tokens
  @keep_count = keep_count
  @min_messages = min_messages
  @already_compacted = false
  @overflow_recovered = false
end

Instance Attribute Details

#chat ⇒ Object

Returns the value of attribute chat.



54
55
56
 
# File 'lib/ask/agent/compactor.rb', line 54

def chat
  @chat
end

#llm ⇒ Object

Returns the value of attribute llm.



54
55
56
 
# File 'lib/ask/agent/compactor.rb', line 54

def llm
  @llm
end

Instance Method Details

#compact! ⇒ Object

Summarize older messages and replace them with a summary, preserving a recent tail. The tail is either token-based (#keep_recent_tokens) or the legacy fixed message count.



122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
 
# File 'lib/ask/agent/compactor.rb', line 122

def compact!
  return unless @chat
  messages = @chat.messages.dup
  return if messages.size < @min_messages

  split_index = find_split_index(messages)
  older = messages.first(split_index)
  return if older.empty?

  summary = if @llm
    generate_llm_summary(older) || generate_summary(older)
  else
    generate_summary(older)
  end

  older.size.times { @chat.messages.delete_at(0) }
  @chat.add_message(role: :system, content: "[Previous conversation summary]: #{summary}")
end

#compact_threshold_tokens ⇒ Integer

The token count at which compaction triggers.

Returns:

  • (Integer) 


98
99
100
101
102
103
 
# File 'lib/ask/agent/compactor.rb', line 98

def compact_threshold_tokens
  window = context_window
  return (window * @threshold).round if @threshold

  window - reserve_tokens
end

#context_window ⇒ Integer

The model's context window. Consulted from the model catalog first, then the bundled table, then the default.

Returns:

  • (Integer) 


187
188
189
190
191
192
 
# File 'lib/ask/agent/compactor.rb', line 187

def context_window
  info = model_info
  return info.context_window if info&.context_window

  CONTEXT_WINDOWS[@chat.model.to_s] || CONTEXT_WINDOWS.default
end

#estimate_tokens(text) ⇒ Integer

Rough token estimate: ~4 characters per token.

Parameters:

  • text (String)

Returns:

  • (Integer) 


170
171
172
 
# File 'lib/ask/agent/compactor.rb', line 170

def estimate_tokens(text)
  (text.to_s.length / 4.0).ceil
end

#estimate_total_tokens ⇒ Integer

Total estimated tokens across all messages, including tool-call payloads.

Returns:

  • (Integer) 


178
179
180
181
 
# File 'lib/ask/agent/compactor.rb', line 178

def estimate_total_tokens
  return 0 unless @chat
  @chat.messages.sum { |msg| estimate_message_tokens(msg) }
end

#extract_summary ⇒ String

The text of the most recent injected summary, or "" if none exists.

Returns:

  • (String) 


212
213
214
215
 
# File 'lib/ask/agent/compactor.rb', line 212

def extract_summary
  @chat.messages.each { |msg| return msg.content.to_s if msg.content.to_s.start_with?("[Previous conversation summary]") }
  ""
end

#keep_recent_tokens ⇒ Integer

Verbatim recent-tail budget in tokens.

Returns:

  • (Integer) 


205
206
207
 
# File 'lib/ask/agent/compactor.rb', line 205

def keep_recent_tokens
  @keep_recent_tokens || DEFAULT_KEEP_RECENT_TOKENS
end

#microcompact! ⇒ Object

Aggressive overflow fallback: clears oversized tool results in place, keeping the conversation structure intact.



143
144
145
146
147
148
149
150
151
152
153
154
155
156
 
# File 'lib/ask/agent/compactor.rb', line 143

def microcompact!
  return unless @chat
  @chat.messages.map! do |msg|
    next msg unless msg.role == :tool
    next msg unless msg.content.to_s.length > 200

    Ask::Message.new(
      role: :tool,
      content: "[Tool result cleared by compaction]",
      tool_call_id: msg.tool_call_id,
      metadata: msg.metadata
    )
  end
end

#overflow_recovered? ⇒ Boolean

Returns:

  • (Boolean) 


85
 
# File 'lib/ask/agent/compactor.rb', line 85

def overflow_recovered? = @overflow_recovered

#recover_from_overflow ⇒ Object

Recover from a context-overflow error: compact once, then fall back to micro-compaction on subsequent overflows in the same session.



160
161
162
163
164
 
# File 'lib/ask/agent/compactor.rb', line 160

def recover_from_overflow
  if @already_compacted then microcompact! else compact! end
  @already_compacted = true
  @overflow_recovered = true
end

#reserve_tokens ⇒ Integer

Headroom reserved for one model turn. Explicit value wins; otherwise derived from the model's max output tokens with a safety floor.

Returns:

  • (Integer) 


198
199
200
 
# File 'lib/ask/agent/compactor.rb', line 198

def reserve_tokens
  @reserve_tokens || derive_reserve
end

#run(event_emitter: nil) ⇒ Object

Run compaction, emitting start/end events.

Parameters:

  • event_emitter (#emit, nil) (defaults to: nil) 


108
109
110
111
112
113
114
115
116
117
 
# File 'lib/ask/agent/compactor.rb', line 108

def run(event_emitter: nil)
  return unless @chat

  tokens_before = estimate_total_tokens
  event_emitter&.emit(Events::CompactionStart.new(tokens_before: tokens_before, reason: :threshold))
  compact!
  tokens_after = estimate_total_tokens
  @already_compacted = true
  event_emitter&.emit(Events::CompactionEnd.new(tokens_before: tokens_before, tokens_after: tokens_after, summary: extract_summary))
end

#should_compact? ⇒ Boolean

Whether the conversation is close enough to the model's window that compaction should run. Triggers at either threshold mode (window * threshold) or reserve mode (window - reserve).

Returns:

  • (Boolean) 


90
91
92
93
 
# File 'lib/ask/agent/compactor.rb', line 90

def should_compact?
  return false unless @chat
  estimate_total_tokens >= compact_threshold_tokens
end