Class: Mistri::Compaction

Inherits:
Object
  • Object
show all
Defined in:
lib/mistri/compaction.rb

Overview

When a session compacts, and how much of it survives. Compaction is client-side and provider-agnostic: the session's own provider writes a visible summary, so a host can always show the user exactly what the model still remembers.

The trigger measures real token accounting, not guesses: the last healthy turn's reported usage plus a character heuristic for whatever came after it.

Constant Summary collapse

DEFAULT_RESERVE = 
16_384
DEFAULT_KEEP_RECENT = 
20_000
IMAGE_CHARS = 
4_800
SUMMARY_PREFACE = 
"The earlier conversation was compacted. This summary replaces it:"

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(reserve: DEFAULT_RESERVE, keep_recent: DEFAULT_KEEP_RECENT, window: nil, instructions: nil) ⇒ Compaction

window overrides the model catalog's context window (required for models the catalog does not know). instructions add a host-specific focus to the summary prompt.



26
27
28
29
30
31
32
 
# File 'lib/mistri/compaction.rb', line 26

def initialize(reserve: DEFAULT_RESERVE, keep_recent: DEFAULT_KEEP_RECENT,
               window: nil, instructions: nil)
  @reserve = reserve
  @keep_recent = keep_recent
  @window = window
  @instructions = instructions
end

Instance Attribute Details

#instructions ⇒ Object (readonly)

Returns the value of attribute instructions.



21
22
23
 
# File 'lib/mistri/compaction.rb', line 21

def instructions
  @instructions
end

#keep_recent ⇒ Object (readonly)

Returns the value of attribute keep_recent.



21
22
23
 
# File 'lib/mistri/compaction.rb', line 21

def keep_recent
  @keep_recent
end

#reserve ⇒ Object (readonly)

Returns the value of attribute reserve.



21
22
23
 
# File 'lib/mistri/compaction.rb', line 21

def reserve
  @reserve
end

#window ⇒ Object (readonly)

Returns the value of attribute window.



21
22
23
 
# File 'lib/mistri/compaction.rb', line 21

def window
  @window
end

Class Method Details

.context_tokens(messages) ⇒ Object

Context size for a replay: the last healthy turn's reported tokens (prompt, cache, and output all sit in context next turn) plus an estimate of every message after it.



44
45
46
47
48
 
# File 'lib/mistri/compaction.rb', line 44

def context_tokens(messages)
  index = messages.rindex { |message| reported(message) }
  base = index ? reported(messages[index]) : 0
  messages.drop(index ? index + 1 : 0).sum(base) { |message| estimate(message) }
end

.estimate(message) ⇒ Object 



50
51
52
 
# File 'lib/mistri/compaction.rb', line 50

def estimate(message)
  (chars(message) / 4.0).ceil
end

Instance Method Details

#needed?(tokens, window) ⇒ Boolean

Compact when the context has grown into the reserve headroom. An unknown window never triggers.

Returns:

  • (Boolean) 


36
37
38
 
# File 'lib/mistri/compaction.rb', line 36

def needed?(tokens, window)
  window ? tokens > window - reserve : false
end