Module: HTM::LongTermMemory::HybridSearch

Included in:
HTM::LongTermMemory
Defined in:
lib/htm/long_term_memory/hybrid_search.rb

Overview

Hybrid search using Reciprocal Rank Fusion (RRF)

Performs three independent searches and merges results:

  1. Vector similarity search for semantic matching

  2. Full-text search for keyword matching

  3. Tag-based search for hierarchical category matching

Results are merged using RRF scoring. Nodes appearing in multiple searches receive boosted scores, making them rank higher.

Tag scoring uses hierarchical depth matching - the more levels of a tag hierarchy that match, the higher the score contribution.

RRF Formula: score = Σ 1/(k + rank) for each search where node appears

Results are cached for performance.

Security: All queries use parameterized placeholders to prevent SQL injection.

Constant Summary collapse

MAX_HYBRID_LIMIT =

Maximum results to prevent DoS via unbounded queries

1000
RRF_K =

RRF constant - higher values reduce the impact of rank differences 60 is the standard value from the original RRF paper

60
CANDIDATE_MULTIPLIER =

Multiplier for candidates from each search We fetch more candidates than requested to ensure good fusion

3

Instance Method Summary collapse

Instance Method Details

#search_hybrid(timeframe:, query:, limit:, embedding_service:, prefilter_limit: 100, metadata: {}) ⇒ Array<Hash>

Hybrid search using Reciprocal Rank Fusion

Parameters:

  • timeframe (Range) —

    Time range to search

  • query (String) —

    Search query

  • limit (Integer) —

    Maximum results (capped at MAX_HYBRID_LIMIT)

  • embedding_service (Object) —

    Service to generate embeddings

  • prefilter_limit (Integer) (defaults to: 100) —

    Candidates per search (default: 100)

  • metadata (Hash) (defaults to: {}) —

    Filter by metadata fields (default: {})

Returns:

  • (Array<Hash>) —

    Matching nodes



46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
 
# File 'lib/htm/long_term_memory/hybrid_search.rb', line 46

def search_hybrid(timeframe:, query:, limit:, embedding_service:, prefilter_limit: 100, metadata: {})
  # Enforce limits to prevent DoS
  safe_limit = limit.to_i.clamp(1, MAX_HYBRID_LIMIT)
  safe_prefilter = [prefilter_limit.to_i, 1].max

  start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  result = @cache.fetch(:hybrid, timeframe, query, safe_limit, safe_prefilter, metadata) do
    search_hybrid_rrf(
      timeframe: timeframe,
      query: query,
      limit: safe_limit,
      embedding_service: embedding_service,
      candidate_limit: safe_prefilter * CANDIDATE_MULTIPLIER,
      metadata: metadata
    )
  end
  elapsed_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - start_time) * 1000).round
  HTM::Telemetry.search_latency.record(elapsed_ms, attributes: { 'strategy' => 'hybrid' })
  result
end