Class: Phronomy::Agent::Context::Knowledge::VectorStore::RedisSearch

Inherits:
Base
  • Object
show all
Defined in:
lib/phronomy/agent/context/knowledge/vector_store/redis_search.rb

Overview

Redis-backed vector store using the RediSearch module (FT.* commands).

Requires:

  • The +redis+ gem (add to your Gemfile)
  • A Redis server with the RediSearch (RedisSearch) module enabled (or Redis Stack which bundles RediSearch)

Vectors are stored as FLOAT32 binary blobs in Redis Hash fields and searched using the KNN approximate-nearest-neighbour algorithm.

Examples:

Usage

redis = Redis.new(url: "redis://localhost:6379")
store = Phronomy::Agent::Context::Knowledge::VectorStore::RedisSearch.new(redis: redis, dimension: 1536)
store.add(id: "doc1", embedding: [0.1, 0.9], metadata: {text: "hello"})
results = store.search(query_embedding: [0.1, 0.8], k: 5)

Instance Method Summary collapse

Methods included from AsyncBackend

#add_async, #clear_async, #remove_async, #search_async

Constructor Details

#initialize(redis:, index_name: "phronomy_vectors", dimension: nil) ⇒ RedisSearch

Returns a new instance of RedisSearch.

Parameters:

  • redis (Redis) —

    configured Redis client

  • index_name (String) (defaults to: "phronomy_vectors") —

    RediSearch index name

  • dimension (Integer, nil) (defaults to: nil) —

    vector dimension; auto-detected on first add. When connecting to an existing RediSearch index, you MUST pass dimension: explicitly. Without it, a freshly constructed instance treats the index as uninitialized until #add is called, and #search silently returns [] in the meantime.



37
38
39
40
41
42
43
44
45
46
47
48
49
50
 
# File 'lib/phronomy/agent/context/knowledge/vector_store/redis_search.rb', line 37

def initialize(redis:, index_name: "phronomy_vectors", dimension: nil)
  begin
    require "redis"
  rescue LoadError
    raise LoadError,
      "redis gem is required for Phronomy::Agent::Context::Knowledge::VectorStore::RedisSearch. " \
      "Add `gem 'redis'` to your Gemfile."
  end
  @redis = redis
  @index_name = index_name
  @dimension = dimension
  @index_created = false
  @mutex = Mutex.new
end

Instance Method Details

#add(id:, embedding:, metadata: {}, cancellation_token: nil) ⇒ Object

Parameters:



57
58
59
60
61
62
63
64
65
66
67
68
69
70
 
# File 'lib/phronomy/agent/context/knowledge/vector_store/redis_search.rb', line 57

def add(id:, embedding:, metadata: {}, cancellation_token: nil)
  cancellation_token&.raise_if_cancelled!
  # Establish expected dimension on first add (not race-free for concurrent
  # first adds), then validate, then create/reuse the index.
  @dimension ||= embedding.size
  validate_embedding_dimension!(embedding, @dimension)
  ensure_index!(@dimension)
  @redis.call(
    "HSET", "#{DOC_PREFIX}#{id}",
    "embedding", pack_vector(embedding),
    "metadata", metadata.to_json
  )
  self
end

#clear ⇒ Object 



119
120
121
122
123
124
125
126
127
128
129
 
# File 'lib/phronomy/agent/context/knowledge/vector_store/redis_search.rb', line 119

def clear
  @mutex.synchronize do
    begin
      @redis.call("FT.DROPINDEX", @index_name, "DD")
    rescue => e
      raise unless e.message.to_s.include?("Unknown Index name")
    end
    @index_created = false
  end
  self
end

#remove(id:) ⇒ Object 



100
101
102
103
 
# File 'lib/phronomy/agent/context/knowledge/vector_store/redis_search.rb', line 100

def remove(id:)
  @redis.call("DEL", "#{DOC_PREFIX}#{id}")
  self
end

#search(query_embedding:, k: 5, cancellation_token: nil) ⇒ Array<Hash>

Returns sorted by descending similarity score.

Parameters:

Returns:

  • (Array<Hash>) —

    sorted by descending similarity score



77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
 
# File 'lib/phronomy/agent/context/knowledge/vector_store/redis_search.rb', line 77

def search(query_embedding:, k: 5, cancellation_token: nil)
  cancellation_token&.raise_if_cancelled!
  # search never establishes dimension.  If dimension is unknown and the
  # index has not been created yet, there are no documents to return.
  return [] if @dimension.nil? && !@index_created

  validate_embedding_dimension!(query_embedding, @dimension)
  ensure_index!(@dimension)
  k_safe = validate_k!(k)
  blob = pack_vector(query_embedding)

  raw = @redis.call(
    "FT.SEARCH", @index_name,
    "*=>[KNN #{k_safe} @embedding $BLOB AS score]",
    "PARAMS", 2, "BLOB", blob,
    "SORTBY", "score",
    "RETURN", 2, "score", "metadata",
    "DIALECT", 2
  )

  parse_results(raw)
end

#size ⇒ Object

Returns the number of documents indexed. Queries FT.INFO when the index has been created; returns 0 otherwise.



107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/phronomy/agent/context/knowledge/vector_store/redis_search.rb', line 107

def size
  return 0 unless @index_created

  raw = @redis.call("FT.INFO", @index_name)
  return 0 unless raw.is_a?(Array)

  idx = raw.index("num_docs")
  idx ? raw[idx + 1].to_i : 0
rescue
  0
end