Class: SwarmSDK::V3::Memory::Adapters::Base

Inherits:
Object
  • Object
show all
Defined in:
lib/swarm_sdk/v3/memory/adapters/base.rb

Overview

Abstract adapter interface for memory storage

Defines the contract that all storage backends must satisfy. SDK users can implement custom adapters (e.g., Postgres + pgvector) by subclassing this and implementing all methods.

## Why adapters own vector operations

The SDK code (ContextBuilder, IngestionPipeline, Retriever, etc.) never computes similarity directly. All vector math goes through the adapter so that each backend can use its native capabilities:

  • FilesystemAdapter: FAISS for indexed search, Ruby cosine similarity (via VectorUtils) for pairwise comparison

  • PgvectorAdapter: ‘<=>` operator for both indexed and pairwise

  • Qdrant/Pinecone: Client API for all similarity operations

There are two distinct vector operations:

### ‘vector_search(embedding, top_k:, threshold:)` Searches an index of card embeddings. Used by the Retriever for semantic search and by the Consolidator for dedup/conflict detection. The filesystem adapter uses FAISS for this. A pgvector adapter would use `ORDER BY embedding <=> query LIMIT k`.

### ‘similarity(embedding_a, embedding_b)` Computes pairwise similarity between **two arbitrary vectors**. This is needed when comparing vectors that aren’t both in the card index — for example:

  • Card embedding vs. cluster centroid (clusters aren’t indexed)

  • Two cards already in hand during dedup in the ContextBuilder

  • Query embedding vs. candidate cards in the exploration sprinkle

FAISS cannot do this — it only supports ‘index.search(, k)`, which searches vectors that have been `add`’d to the index.

For in-memory adapters, include VectorUtils to get a default Ruby implementation. For database adapters, implement it using your backend’s native operator.

## Implementing a custom adapter

Subclass Base and implement every method. Methods that raise NotImplementedError are required. For a starting point, see FilesystemAdapter which implements the full interface.

If your adapter doesn’t have native vector math, include VectorUtils to get a default #similarity implementation:

class MyAdapter < Base
  include VectorUtils  # provides similarity()
  # ... implement the rest
end

Examples:

Postgres + pgvector adapter

class PgvectorAdapter < SwarmSDK::V3::Memory::Adapters::Base
  def initialize(connection_string)
    @db = PG.connect(connection_string)
  end

  def write_card(card)
    @db.exec_params(
      "INSERT INTO cards (id, text, type, embedding, ...) VALUES ($1, $2, $3, $4, ...)",
      [card.id, card.text, card.type.to_s, card.embedding.to_s]
    )
  end

  def similarity(embedding_a, embedding_b)
    result = @db.exec_params(
      "SELECT 1 - ($1::vector <=> $2::vector) AS sim",
      [embedding_a.to_s, embedding_b.to_s]
    )
    result[0]["sim"].to_f
  end

  def vector_search(embedding, top_k:, threshold: 0.0)
    rows = @db.exec_params(
      "SELECT id, 1 - (embedding <=> $1::vector) AS similarity " \
      "FROM cards WHERE 1 - (embedding <=> $1::vector) >= $3 " \
      "ORDER BY embedding <=> $1::vector LIMIT $2",
      [embedding.to_s, top_k, threshold]
    )
    rows.map { |r| { id: r["id"], similarity: r["similarity"].to_f } }
  end

  # ... implement remaining methods
end

Direct Known Subclasses

FilesystemAdapter, SqliteAdapter

Instance Method Summary collapse

Instance Method Details

#delete_card(id) ⇒ void

This method returns an undefined value.

Delete a card by ID

Parameters:

  • id (String) —

    Card ID

Raises:

  • (NotImplementedError) 


117
118
119
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 117

def delete_card(id)
  raise NotImplementedError, "#{self.class}#delete_card not implemented"
end

#delete_edges_for(card_id) ⇒ void

This method returns an undefined value.

Delete all edges involving a card (as source or target)

Parameters:

  • card_id (String) —

    Card ID

Raises:

  • (NotImplementedError) 


169
170
171
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 169

def delete_edges_for(card_id)
  raise NotImplementedError, "#{self.class}#delete_edges_for not implemented"
end

#edges_for(card_id, type: nil) ⇒ Array<Edge>

Get edges for a card (as source or target)

Parameters:

  • card_id (String) —

    Card ID

  • type (Symbol, nil) (defaults to: nil) —

    Filter by edge type

Returns:

Raises:

  • (NotImplementedError) 


161
162
163
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 161

def edges_for(card_id, type: nil)
  raise NotImplementedError, "#{self.class}#edges_for not implemented"
end

#list_cards(prefix: nil) ⇒ Array<Card>

List cards, optionally filtered by ID prefix

Parameters:

  • prefix (String, nil) (defaults to: nil) —

    ID prefix filter

Returns:

Raises:

  • (NotImplementedError) 


125
126
127
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 125

def list_cards(prefix: nil)
  raise NotImplementedError, "#{self.class}#list_cards not implemented"
end

#list_cards_for_compression(max_level: 3) ⇒ Array<Card>

List cards eligible for compression

Default implementation loads all cards and filters in Ruby. Database adapters should override with a server-side query.

Examples:

Override in a Postgres adapter

def list_cards_for_compression(max_level: 3)
  @db.exec_params("SELECT * FROM cards WHERE compression_level <= $1", [max_level])
    .map { |row| card_from_row(row) }
end

Parameters:

  • max_level (Integer) (defaults to: 3) —

    Maximum compression level to include (default: 3)

Returns:

  • (Array<Card>) —

    Cards eligible for compression



142
143
144
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 142

def list_cards_for_compression(max_level: 3)
  list_cards.select { |c| c.compression_level <= max_level }
end

#list_clusters ⇒ Array<Cluster>

List all clusters

Returns:

Raises:

  • (NotImplementedError) 


194
195
196
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 194

def list_clusters
  raise NotImplementedError, "#{self.class}#list_clusters not implemented"
end

#load ⇒ void

This method returns an undefined value.

Load state from durable storage into memory

Raises:

  • (NotImplementedError) 


290
291
292
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 290

def load
  raise NotImplementedError, "#{self.class}#load not implemented"
end

#read_card(id) ⇒ Card?

Read a card by ID

Parameters:

  • id (String) —

    Card ID

Returns:

  • (Card, nil) —

    Card or nil if not found

Raises:

  • (NotImplementedError) 


109
110
111
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 109

def read_card(id)
  raise NotImplementedError, "#{self.class}#read_card not implemented"
end

#read_cluster(id) ⇒ Cluster?

Read a cluster by ID

Parameters:

  • id (String) —

    Cluster ID

Returns:

Raises:

  • (NotImplementedError) 


187
188
189
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 187

def read_cluster(id)
  raise NotImplementedError, "#{self.class}#read_cluster not implemented"
end

#rebuild_index ⇒ void

This method returns an undefined value.

Rebuild the vector index from all stored cards

Called when the index needs to be reconstructed, e.g., after bulk imports or if the index file is missing/corrupted.

Raises:

  • (NotImplementedError) 


254
255
256
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 254

def rebuild_index
  raise NotImplementedError, "#{self.class}#rebuild_index not implemented"
end

#save ⇒ void

This method returns an undefined value.

Save all in-memory state to durable storage

Raises:

  • (NotImplementedError) 


283
284
285
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 283

def save
  raise NotImplementedError, "#{self.class}#save not implemented"
end

#similarity(embedding_a, embedding_b) ⇒ Float

Compute similarity between two arbitrary embedding vectors

Used by the SDK for pairwise comparisons where an indexed search doesn’t apply:

  • Card vs. cluster centroid (cluster assignment in IngestionPipeline)

  • Card vs. card (deduplication in ContextBuilder)

  • Query vs. card (exploration sprinkle in ContextBuilder)

For in-memory adapters, include VectorUtils to get a default cosine similarity implementation. For database adapters, implement using your backend’s native operator.

Parameters:

  • embedding_a (Array<Float>) —

    First embedding vector

  • embedding_b (Array<Float>) —

    Second embedding vector

Returns:

  • (Float) —

    Similarity score (-1.0 to 1.0 for cosine)

Raises:

  • (NotImplementedError)

See Also:



227
228
229
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 227

def similarity(embedding_a, embedding_b)
  raise NotImplementedError, "#{self.class}#similarity not implemented"
end

#transaction { ... } ⇒ Object

Execute a block within a transaction

The default implementation is a no-op pass-through. Adapters with transactional backends (e.g., SQLite, Postgres) should override this to wrap the block in a real transaction.

Examples:

adapter.transaction do
  adapter.write_card(card)
  adapter.write_edge(edge)
end

Yields:

  • Block to execute within the transaction

Returns:

  • (Object) —

    Return value of the block



274
275
276
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 274

def transaction
  yield
end

#vector_search(embedding, top_k:, threshold: 0.0) ⇒ Array<Hash>

Search the card embedding index for similar vectors

Used by the SDK for top-k retrieval:

  • Retriever: semantic search for relevant cards

  • Consolidator: finding near-duplicates and conflicts

This searches an index of card embeddings. The adapter decides how to implement the index (FAISS, pgvector, etc.).

Parameters:

  • embedding (Array<Float>) —

    Query embedding

  • top_k (Integer) —

    Maximum number of results

  • threshold (Float) (defaults to: 0.0) —

    Minimum similarity to include

Returns:

  • (Array<Hash>) —

    Array of ‘{ id: String, similarity: Float }`

Raises:

  • (NotImplementedError) 


244
245
246
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 244

def vector_search(embedding, top_k:, threshold: 0.0)
  raise NotImplementedError, "#{self.class}#vector_search not implemented"
end

#write_card(card) ⇒ void

This method returns an undefined value.

Write a card to storage (insert or update)

Parameters:

  • card (Card) —

    Card to store

Raises:

  • (NotImplementedError) 


101
102
103
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 101

def write_card(card)
  raise NotImplementedError, "#{self.class}#write_card not implemented"
end

#write_cluster(cluster) ⇒ void

This method returns an undefined value.

Write a cluster to storage (insert or update)

Parameters:

  • cluster (Cluster) —

    Cluster to store

Raises:

  • (NotImplementedError) 


179
180
181
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 179

def write_cluster(cluster)
  raise NotImplementedError, "#{self.class}#write_cluster not implemented"
end

#write_edge(edge) ⇒ void

This method returns an undefined value.

Write an edge to storage

Parameters:

  • edge (Edge) —

    Edge to store

Raises:

  • (NotImplementedError) 


152
153
154
 
# File 'lib/swarm_sdk/v3/memory/adapters/base.rb', line 152

def write_edge(edge)
  raise NotImplementedError, "#{self.class}#write_edge not implemented"
end