Class: HTM::LongTermMemory

Inherits:
Object
  • Object
show all
Includes:
FulltextSearch, HybridSearch, NodeOperations, RelevanceScorer, RobotOperations, TagOperations, VectorSearch
Defined in:
lib/htm/long_term_memory.rb,
lib/htm/long_term_memory/hybrid_search.rb,
lib/htm/long_term_memory/vector_search.rb,
lib/htm/long_term_memory/tag_operations.rb,
lib/htm/long_term_memory/fulltext_search.rb,
lib/htm/long_term_memory/node_operations.rb,
lib/htm/long_term_memory/relevance_scorer.rb,
lib/htm/long_term_memory/robot_operations.rb

Overview

Long-term Memory - PostgreSQL-backed permanent storage

LongTermMemory provides durable storage for all memory nodes with:

  • Vector similarity search (RAG)

  • Full-text search

  • Time-range queries

  • Relationship graphs

  • Tag system

  • Sequel ORM for data access

  • Query result caching for efficiency

This class uses standalone utility classes and modules:

Standalone classes (used via class methods or instances):

  • HTM::SqlBuilder: SQL condition building helpers (class methods)

  • HTM::QueryCache: Query result caching (instantiated as @cache)

Included modules:

  • RelevanceScorer: Dynamic relevance scoring

  • NodeOperations: Node CRUD operations

  • RobotOperations: Robot registration and activity

  • TagOperations: Tag management

  • VectorSearch: Vector similarity search

  • FulltextSearch: Full-text search

  • HybridSearch: Combined search strategies

Defined Under Namespace

Modules: FulltextSearch, HybridSearch, NodeOperations, RelevanceScorer, RobotOperations, TagOperations, VectorSearch

Constant Summary collapse

DEFAULT_QUERY_TIMEOUT =

milliseconds (30 seconds)

30_000
MAX_VECTOR_DIMENSION =

Maximum supported dimension with HNSW index (pgvector limitation)

2000
DEFAULT_CACHE_SIZE =

Number of queries to cache

1000
DEFAULT_CACHE_TTL =

Cache lifetime in seconds (5 minutes)

300

Constants included from RelevanceScorer

RelevanceScorer::ACCESS_SCORE_NORMALIZER, RelevanceScorer::DEFAULT_NEUTRAL_SCORE, RelevanceScorer::RELEVANCE_MAX, RelevanceScorer::RELEVANCE_MIN, RelevanceScorer::RELEVANCE_SCALE

Constants included from HybridSearch

HybridSearch::CANDIDATE_MULTIPLIER, HybridSearch::MAX_HYBRID_LIMIT, HybridSearch::RRF_K

Constants included from FulltextSearch

FulltextSearch::MAX_FULLTEXT_LIMIT, FulltextSearch::TRIGRAM_SIMILARITY_THRESHOLD, FulltextSearch::TSVECTOR_SCORE_BOOST

Constants included from VectorSearch

VectorSearch::MAX_VECTOR_LIMIT

Constants included from TagOperations

TagOperations::DEFAULT_TAG_SIMILARITY_THRESHOLD, TagOperations::MAX_TAG_QUERY_LIMIT, TagOperations::MAX_TAG_SAMPLE_SIZE, TagOperations::POPULAR_TAGS_CACHE_TTL

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from RelevanceScorer

#calculate_relevance, #fetch_candidates_by_timeframe, #recency_half_life_hours, #search_by_tags, #search_with_relevance, #weight_access, #weight_recency, #weight_semantic, #weight_tag

Methods included from HybridSearch

#search_hybrid

Methods included from FulltextSearch

#search_fulltext

Methods included from VectorSearch

#search

Methods included from NodeOperations

#add, #delete, #exists?, #link_robot_to_node, #mark_evicted, #retrieve, #track_access, #update_last_accessed

Methods included from RobotOperations

#register_robot, #update_robot_activity

Methods included from TagOperations

#add_tag, #batch_load_node_tags, #find_query_matching_tags, #get_node_tags, #node_topics, #nodes_by_topic, #ontology_structure, #popular_tags, #search_tags, #topic_relationships

Constructor Details

#initialize(config, pool_size: nil, query_timeout: DEFAULT_QUERY_TIMEOUT, cache_size: DEFAULT_CACHE_SIZE, cache_ttl: DEFAULT_CACHE_TTL) ⇒ LongTermMemory

Initialize long-term memory storage

Examples:

Initialize with defaults

ltm = LongTermMemory.new(HTM::Database.default_config)

Initialize with custom cache settings

ltm = LongTermMemory.new(config, cache_size: 500, cache_ttl: 600)

Disable caching

ltm = LongTermMemory.new(config, cache_size: 0)

Parameters:

  • config (Hash) —

    Database configuration (host, port, dbname, user, password)

  • pool_size (Integer, nil) (defaults to: nil) —

    Connection pool size (uses Sequel default if nil)

  • query_timeout (Integer) (defaults to: DEFAULT_QUERY_TIMEOUT) —

    Query timeout in milliseconds (default: 30000)

  • cache_size (Integer) (defaults to: DEFAULT_CACHE_SIZE) —

    Number of query results to cache (default: 1000, use 0 to disable)

  • cache_ttl (Integer) (defaults to: DEFAULT_CACHE_TTL) —

    Cache time-to-live in seconds (default: 300)



89
90
91
92
93
94
95
96
97
98
99
 
# File 'lib/htm/long_term_memory.rb', line 89

def initialize(config, pool_size: nil, query_timeout: DEFAULT_QUERY_TIMEOUT, cache_size: DEFAULT_CACHE_SIZE,
               cache_ttl: DEFAULT_CACHE_TTL)
  @config = config
  @query_timeout = query_timeout  # in milliseconds

  # Set statement timeout for Sequel queries
  HTM.db.run("SET statement_timeout = #{@query_timeout}")

  # Initialize query result cache (disable with cache_size: 0)
  @cache = HTM::QueryCache.new(size: cache_size, ttl: cache_ttl)
end

Instance Attribute Details

#query_timeout ⇒ Object (readonly)

Returns the value of attribute query_timeout.



70
71
72
 
# File 'lib/htm/long_term_memory.rb', line 70

def query_timeout
  @query_timeout
end

Instance Method Details

#clear_cache! ⇒ void

This method returns an undefined value.

Clear the query result cache



135
136
137
 
# File 'lib/htm/long_term_memory.rb', line 135

def clear_cache!
  @cache&.clear!
end

#pool_size ⇒ Object

For backwards compatibility with tests/code that expect pool_size



140
141
142
 
# File 'lib/htm/long_term_memory.rb', line 140

def pool_size
  HTM.db.pool.size
end

#shutdown ⇒ Object

Shutdown - no-op with Sequel (connection pool managed by Sequel)



126
127
128
129
 
# File 'lib/htm/long_term_memory.rb', line 126

def shutdown
  # Sequel handles connection pool shutdown
  # This method kept for API compatibility
end

#stats ⇒ Hash

Get memory statistics

Returns:

  • (Hash) —

    Statistics



105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
 
# File 'lib/htm/long_term_memory.rb', line 105

def stats
  base_stats = {
    total_nodes: HTM::Models::Node.count,
    nodes_by_robot: HTM::Models::RobotNode.group_and_count(:robot_id).as_hash(:robot_id, :count),
    total_tags: HTM::Models::Tag.count,
    oldest_memory: HTM::Models::Node.min(:created_at),
    newest_memory: HTM::Models::Node.max(:created_at),
    active_robots: HTM::Models::Robot.count,
    robot_activity: HTM::Models::Robot.select(:id, :name, :last_active).all.map(&:values),
    database_size: HTM.db.get(Sequel.function(:pg_database_size, Sequel.function(:current_database))).to_i
  }

  # Include cache statistics if cache is enabled
  if @cache&.enabled?
    base_stats[:cache] = @cache.stats
  end

  base_stats
end