Class: SmartPrompt::SummaryBasedStrategy

Inherits:
Object
  • Object
show all
Includes:
ContextStrategy
Defined in:
lib/smart_prompt/summary_based_strategy.rb

Overview

SummaryBasedStrategy implements a context selection strategy that automatically compresses older messages through summarization

This strategy:

  • Monitors message count and triggers summarization at threshold

  • Keeps recent messages uncompressed for context continuity

  • Generates summaries of older messages to reduce token usage

  • Falls back to truncation when summarization fails

  • Maintains conversation coherence while reducing token costs

Instance Method Summary collapse

Constructor Details

#initialize(config = {}) ⇒ SummaryBasedStrategy

Initialize the summary-based strategy

Parameters:

  • config (Hash) (defaults to: {}) —

    Configuration options

Options Hash (config):

  • :summary_threshold (Integer) — default: 20 —

    Message count that triggers summarization

  • :keep_recent (Integer) — default: 5 —

    Number of recent messages to keep uncompressed

  • :compression_engine (CompressionEngine) —

    Engine for generating summaries

  • :preserve_system (Boolean) — default: true —

    Whether to always keep system messages



22
23
24
25
26
27
28
29
30
 
# File 'lib/smart_prompt/summary_based_strategy.rb', line 22

def initialize(config = {})
  @summary_threshold = config[:summary_threshold] || 20
  @keep_recent = config[:keep_recent] || 5
  @compression_engine = config[:compression_engine]
  @preserve_system = config[:preserve_system] != false

  # Create a default compression engine if none provided
  @compression_engine ||= CompressionEngine.new(config[:compression] || {})
end

Instance Method Details

#select_messages(messages, max_tokens, current_message = nil) ⇒ Array<Message>

Select messages using summary-based approach Automatically summarizes older messages when threshold is exceeded

Parameters:

  • messages (Array<Message>) —

    All messages in the session

  • max_tokens (Integer, nil) —

    Maximum token limit for selected messages

  • current_message (Message, nil) (defaults to: nil) —

    Not used in this strategy

Returns:

  • (Array<Message>) —

    Selected messages with summaries



38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
 
# File 'lib/smart_prompt/summary_based_strategy.rb', line 38

def select_messages(messages, max_tokens, current_message = nil)
  return [] if messages.nil? || messages.empty?

  # If below threshold, return messages (filtering system messages if needed)
  if messages.count <= @summary_threshold
    filtered = @preserve_system ? messages : messages.reject(&:system_message?)
    return max_tokens ? trim_to_token_limit(filtered, max_tokens) : filtered
  end

  # Separate system messages, summaries, and regular messages
  system_messages = @preserve_system ? messages.select(&:system_message?) : []
  existing_summaries = messages.select { |msg| msg.is_summary }
  regular_messages = messages.reject { |msg| msg.system_message? || msg.is_summary }

  # Keep the most recent messages
  recent_messages = regular_messages.last(@keep_recent)

  # Get older messages that need summarization
  old_messages = regular_messages[0...-@keep_recent]

  # Generate summary if we have old messages and no existing summary
  if !old_messages.empty? && existing_summaries.empty?
    begin
      summary = @compression_engine.summarize(old_messages)
      if summary
        # Combine: system messages + summary + recent messages
        selected = system_messages + [summary] + recent_messages
      else
        # If summarization failed, fall back to keeping more recent messages
        SmartPrompt.logger.warn "Summarization failed, falling back to recent messages only"
        fallback_count = [@summary_threshold / 2, regular_messages.count].min
        selected = system_messages + regular_messages.last(fallback_count)
      end
    rescue => e
      SmartPrompt.logger.error "Error during summarization: #{e.message}, using fallback"
      # Fallback: keep system messages and recent messages
      selected = system_messages + recent_messages
    end
  else
    # Use existing summaries if available
    selected = system_messages + existing_summaries + recent_messages
  end

  # Trim to token limit if specified
  max_tokens ? trim_to_token_limit(selected, max_tokens) : selected
end

#should_compress?(session) ⇒ Boolean

Determine if compression should be triggered Recommends compression when message count exceeds threshold

Parameters:

  • session (Session) —

    The session to evaluate

Returns:

  • (Boolean) —

    true if message count > summary_threshold



89
90
91
92
 
# File 'lib/smart_prompt/summary_based_strategy.rb', line 89

def should_compress?(session)
  return false if session.nil?
  session.message_count > @summary_threshold
end