Module: Rubino::Memory::SqliteExtraction

Included in:
Backends::Sqlite
Defined in:
lib/rubino/memory/sqlite_extraction.rb

Overview

Per-session extraction windowing for the Sqlite backend (#249).

A thin mixin that bounds each post-turn extraction to the messages a turn actually added, using a persisted per-session watermark (‘sessions.memory_extracted_msg_id`) instead of re-reading an overlapping recency window every turn. Without it, the extractor re-fed (and the aux model re-processed) earlier messages it had already mined, so the per-turn cost grew with session length and a turn that added nothing still spent a redundant extraction pass.

Cross-session recall is untouched: facts are not session-scoped, and the cursor only governs what each turn FEEDS the extractor — never what recall reads. A nil/unset cursor (never-extracted or pre-migration session) yields the whole short session, so the first extraction behaves exactly as before.

Instance Method Summary collapse

Instance Method Details

#advance_extraction_cursor(session_id, processed_messages) ⇒ Object

Move the watermark to the newest message we just fed the extractor, so the next turn starts strictly after it. Best-effort: a failure here only costs one redundant re-extraction next turn, never correctness.



60
61
62
63
64
65
66
67
 
# File 'lib/rubino/memory/sqlite_extraction.rb', line 60

def advance_extraction_cursor(session_id, processed_messages)
  newest = processed_messages.last&.id
  return unless newest

  @db[:sessions].where(id: session_id).update(memory_extracted_msg_id: newest)
rescue StandardError
  nil
end

#live_facts_for_prompt ⇒ Object

The live fact set rendered for the extractor prompt — newest 60, id truncated for compactness. (Uses the backend’s #live_dataset.)



30
31
32
33
34
 
# File 'lib/rubino/memory/sqlite_extraction.rb', line 30

def live_facts_for_prompt
  live_dataset.order(Sequel.desc(:created_at)).limit(60).all.map do |r|
    { id: r[:id][0, 8], kind: r[:kind], text: r[:text] }
  end
end

#parse_json(content) ⇒ Object

The aux model may wrap JSON in prose or a fenced block; extract the outermost object and parse leniently.



38
39
40
41
42
43
44
45
 
# File 'lib/rubino/memory/sqlite_extraction.rb', line 38

def parse_json(content)
  return nil if content.to_s.strip.empty?

  str = content[/\{.*\}/m] || content
  JSON.parse(str)
rescue JSON::ParserError
  nil
end

#turn_text(messages) ⇒ Object

Render the user/assistant turn transcript fed to the aux extractor.



48
49
50
51
52
53
54
55
 
# File 'lib/rubino/memory/sqlite_extraction.rb', line 48

def turn_text(messages)
  messages.filter_map do |m|
    next if m.content.nil? || m.content.to_s.empty?
    next unless %w[user assistant].include?(m.role)

    "#{m.role.upcase}: #{m.content}"
  end.join("\n")
end

#unextracted_messages(session_id) ⇒ Object

Messages added since this session’s extraction watermark.



21
22
23
24
25
26
 
# File 'lib/rubino/memory/sqlite_extraction.rb', line 21

def unextracted_messages(session_id)
  cursor = @db[:sessions].where(id: session_id).get(:memory_extracted_msg_id)
  Session::Store.new(db: @db).since(session_id, after_id: cursor)
rescue StandardError
  []
end