Class: SwarmSDK::V3::Memory::Compressor

Inherits:
Object
  • Object
show all
Defined in:
lib/swarm_sdk/v3/memory/compressor.rb

Overview

Compression ladder for memory cards

Demotes low-exposure cards through compression levels:

  • L0: Raw (original text)

  • L1: Faithful summary

  • L2: Sparse bullets

  • L3: Schema-only (entities + type)

  • L4: Embedding + title only

Also supports promotion: when a compressed card (L1-L4) is accessed frequently, it gets promoted one level up by rebuilding a richer summary from its graph neighbors and cluster context.

Uses LLM for L0→L1 and L1→L2 transitions. L3 and L4 are mechanical transformations that don’t require LLM calls.

Examples:

Compress

compressor = Compressor.new(adapter: adapter, chat: llm_chat)
compressor.compress_card(card)  # Advances one level

Promote

compressor.promote_card(card)   # Moves one level up if eligible

Constant Summary collapse

BATCH_SIZE =

Maximum cards per LLM call to stay within token limits

20

Instance Method Summary collapse

Constructor Details

#initialize(adapter:, chat: nil) ⇒ Compressor

Returns a new instance of Compressor.

Parameters:

  • adapter (Adapters::Base) —

    Storage adapter

  • chat (RubyLLM::Chat, nil) (defaults to: nil) —

    LLM chat for summarization (nil = skip LLM steps)



34
35
36
37
 
# File 'lib/swarm_sdk/v3/memory/compressor.rb', line 34

def initialize(adapter:, chat: nil)
  @adapter = adapter
  @chat = chat
end

Instance Method Details

#compress_card(card) ⇒ Card

Compress a card one level down the ladder

Parameters:

  • card (Card) —

    Card to compress

Returns:

  • (Card) —

    Updated card (also persisted to adapter)



43
44
45
46
47
48
49
50
51
 
# File 'lib/swarm_sdk/v3/memory/compressor.rb', line 43

def compress_card(card)
  case card.compression_level
  when 0 then compress_l0_to_l1(card)
  when 1 then compress_l1_to_l2(card)
  when 2 then compress_l2_to_l3(card)
  when 3 then compress_l3_to_l4(card)
  else card # Already at max compression
  end
end

#compress_low_exposure(exposure_tracker:, threshold: 1.0) ⇒ Integer

Compress all low-exposure cards using batched LLM calls

Groups cards by compression level transition and batches LLM calls for L0→L1 and L1→L2. L2→L3 and L3→L4 are mechanical (no LLM).

Parameters:

  • exposure_tracker (ExposureTracker) —

    For scoring cards

  • threshold (Float) (defaults to: 1.0) —

    Exposure score threshold

Returns:

  • (Integer) —

    Number of cards compressed



61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
 
# File 'lib/swarm_sdk/v3/memory/compressor.rb', line 61

def compress_low_exposure(exposure_tracker:, threshold: 1.0)
  candidates = exposure_tracker.low_exposure_cards(threshold: threshold)
  eligible = candidates.reject { |c| c.compression_level >= 4 }
  DebugLog.log("compressor", "compress_low_exposure: #{eligible.size}/#{candidates.size} eligible cards")
  return 0 if eligible.empty?

  # Group by current compression level
  l0_cards = eligible.select { |c| c.compression_level == 0 }
  l1_cards = eligible.select { |c| c.compression_level == 1 }
  l2_cards = eligible.select { |c| c.compression_level == 2 }
  l3_cards = eligible.select { |c| c.compression_level == 3 }

  count = 0

  # Batch LLM transitions (L0→L1, L1→L2)
  count += batch_compress_l0_to_l1(l0_cards) if l0_cards.any?
  count += batch_compress_l1_to_l2(l1_cards) if l1_cards.any?

  # Mechanical transitions (no LLM needed)
  l2_cards.each { |c| compress_l2_to_l3(c) }
  count += l2_cards.size

  l3_cards.each { |c| compress_l3_to_l4(c) }
  count += l3_cards.size

  DebugLog.log("compressor", "compressed #{count} cards")
  count
end

#promote_card(card) ⇒ Card

Promote a compressed card one level up

When a compressed card is accessed frequently, it should be promoted to recover detail. L4→L3 and L3→L2 are mechanical (rebuild from entities/type). L2→L1 and L1→L0 use LLM to rebuild richer text from graph neighbors and cluster context.

Parameters:

  • card (Card) —

    Compressed card (compression_level > 0)

Returns:

  • (Card) —

    Updated card at one level higher, or unchanged if L0



99
100
101
102
103
104
105
106
107
 
# File 'lib/swarm_sdk/v3/memory/compressor.rb', line 99

def promote_card(card)
  case card.compression_level
  when 4 then promote_l4_to_l3(card)
  when 3 then promote_l3_to_l2(card)
  when 2 then promote_l2_to_l1(card)
  when 1 then promote_l1_to_l0(card)
  else card # Already at L0
  end
end

#promote_eligible(access_threshold: 5) ⇒ Integer

Promote all eligible compressed cards that have high access

Cards at L1+ with access_count above the threshold are promoted one level toward L0. Uses batched LLM calls for L2→L1 and L1→L0 promotions.

Parameters:

  • access_threshold (Integer) (defaults to: 5) —

    Minimum access count for promotion

Returns:

  • (Integer) —

    Number of cards promoted



117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
 
# File 'lib/swarm_sdk/v3/memory/compressor.rb', line 117

def promote_eligible(access_threshold: 5)
  eligible = @adapter.list_cards.select do |card|
    card.compression_level >= 1 && card.access_count >= access_threshold
  end
  return 0 if eligible.empty?

  # Group by current compression level
  l4_cards = eligible.select { |c| c.compression_level == 4 }
  l3_cards = eligible.select { |c| c.compression_level == 3 }
  l2_cards = eligible.select { |c| c.compression_level == 2 }
  l1_cards = eligible.select { |c| c.compression_level == 1 }

  count = 0

  # Mechanical promotions (L4→L3, L3→L2)
  l4_cards.each { |c| promote_l4_to_l3(c) }
  count += l4_cards.size

  l3_cards.each { |c| promote_l3_to_l2(c) }
  count += l3_cards.size

  # Batch LLM promotions (L2→L1, L1→L0)
  count += batch_promote_l2_to_l1(l2_cards) if l2_cards.any?
  count += batch_promote_l1_to_l0(l1_cards) if l1_cards.any?

  count
end