Class: Mistri::Compaction

Inherits:
Object
  • Object
show all
Defined in:
lib/mistri/compaction.rb

Overview

When a session compacts, and how much of it survives. Compaction is client-side and provider-agnostic: the session's own provider writes a visible summary, so a host can always show the user exactly what the model still remembers.

The trigger measures real token accounting, not guesses: the last healthy turn's reported usage plus a character heuristic for whatever came after it.

Constant Summary collapse

DEFAULT_RESERVE = 
16_384
DEFAULT_KEEP_RECENT = 
20_000
OUTPUT_SAFETY = 
4_096
IMAGE_CHARS = 
4_800
SUMMARY_PREFACE = 
"The earlier conversation was compacted. This summary replaces it:"

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(reserve: nil, keep_recent: DEFAULT_KEEP_RECENT, window: nil, instructions: nil) ⇒ Compaction

window overrides the model catalog's context window (required for models the catalog does not know). A nil reserve selects automatic headroom; a number is explicit host policy. instructions add a host-specific focus to the summary prompt.



28
29
30
31
32
33
34
35
 
# File 'lib/mistri/compaction.rb', line 28

def initialize(reserve: nil, keep_recent: DEFAULT_KEEP_RECENT,
               window: nil, instructions: nil)
  @automatic_reserve = reserve.nil?
  @reserve = reserve || DEFAULT_RESERVE
  @keep_recent = keep_recent
  @window = window
  @instructions = instructions
end

Instance Attribute Details

#instructions ⇒ Object (readonly)

Returns the value of attribute instructions.



22
23
24
 
# File 'lib/mistri/compaction.rb', line 22

def instructions
  @instructions
end

#keep_recent ⇒ Object (readonly)

Returns the value of attribute keep_recent.



22
23
24
 
# File 'lib/mistri/compaction.rb', line 22

def keep_recent
  @keep_recent
end

#reserve ⇒ Object (readonly)

Returns the value of attribute reserve.



22
23
24
 
# File 'lib/mistri/compaction.rb', line 22

def reserve
  @reserve
end

#window ⇒ Object (readonly)

Returns the value of attribute window.



22
23
24
 
# File 'lib/mistri/compaction.rb', line 22

def window
  @window
end

Class Method Details

.context_tokens(messages, usage_from: 0) ⇒ Object

Context size for a replay: the last healthy turn's reported tokens (prompt, cache, and output all sit in context next turn) plus an estimate of every message after it.



58
59
60
61
62
63
64
 
# File 'lib/mistri/compaction.rb', line 58

def context_tokens(messages, usage_from: 0)
  index = (usage_from...messages.length).reverse_each.find do |position|
    reported(messages[position])
  end
  base = index ? reported(messages[index]) : 0
  messages.drop(index ? index + 1 : 0).sum(base) { |message| estimate(message) }
end

.estimate(message) ⇒ Object 



66
67
68
 
# File 'lib/mistri/compaction.rb', line 66

def estimate(message)
  (chars(message) / 4.0).ceil
end

Instance Method Details

#automatic_reserve? ⇒ Boolean

Returns:

  • (Boolean) 


37
 
# File 'lib/mistri/compaction.rb', line 37

def automatic_reserve? = @automatic_reserve

#needed?(tokens, window, max_output: nil) ⇒ Boolean

Automatic headroom fits output that shares the context window, plus estimation and framing slack. An explicit reserve remains host policy. An unknown window never triggers.

Returns:

  • (Boolean) 


42
43
44
 
# File 'lib/mistri/compaction.rb', line 42

def needed?(tokens, window, max_output: nil)
  window ? tokens > window - effective_reserve(max_output) : false
end