Module: Parse::Embeddings::Cache

Defined in:
lib/parse/embeddings/cache.rb

Overview

Process-local embedding cache keyed by (provider, model, input_type, input_hash).

Query-side embedding is the hot repeat path: the same natural- language query (an agent retrying a tool call, a user paging through results, a dashboard refreshing) re-embeds identical text on every call, paying provider latency and per-token cost each time. The cache short-circuits those repeats. Write-side managed embeds (embed / embed_image save callbacks) already have their own digest-tracked elision and do not use this cache.

== Disabled by default

With the cache disabled Cache.fetch_vector is a pass-through. Opt in:

Parse::Embeddings::Cache.enable!(max_entries: 2048, ttl: 600)

The default store is an in-process LRU with per-entry TTL. A custom store (e.g. Redis-backed) can be supplied via enable!(store: my_store) — it must respond to get(key) (returning Array<Float> or nil) and set(key, vector); TTL management is then the store's responsibility.

== Key derivation

provider.class.name | model_name | input_type | SHA-256(input). The full input text never becomes part of the key, so a shared external store does not accumulate plaintext queries.

== Observability

A cache hit emits the same parse.embeddings.embed AS::N event a real provider call would, with cached: true — existing spend-tracking subscribers see hits and misses on one stream.

Defined Under Namespace

Classes: MonetaStore

Class Method Summary collapse

Class Method Details

.clear!

This method returns an undefined value.

Clear cached entries (default store) and reset hit/miss counters.



204
205
206
207
208
209
210
211
 
# File 'lib/parse/embeddings/cache.rb', line 204

def clear!
  MONITOR.synchronize do
    @store.clear if @store.respond_to?(:clear)
    @hits = 0
    @misses = 0
  end
  nil
end

.disable!

This method returns an undefined value.

Disable and drop the store.



189
190
191
192
193
194
195
 
# File 'lib/parse/embeddings/cache.rb', line 189

def disable!
  MONITOR.synchronize do
    @enabled = false
    @store = nil
  end
  nil
end

.enable!(max_entries: 2048, ttl: 600, store: nil)

This method returns an undefined value.

Enable the cache.

Parameters:

  • max_entries (Integer) (defaults to: 2048) —

    LRU capacity (default store only).

  • ttl (Numeric, nil) (defaults to: 600) —

    per-entry lifetime in seconds; nil disables expiry (default store only). Default 600.

  • store (#get, #set, nil) (defaults to: nil) —

    custom backing store; overrides the built-in LRU when given.

Raises:

  • (ArgumentError) 


171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
 
# File 'lib/parse/embeddings/cache.rb', line 171

def enable!(max_entries: 2048, ttl: 600, store: nil)
  if store && !(store.respond_to?(:get) && store.respond_to?(:set))
    raise ArgumentError,
          "Parse::Embeddings::Cache.enable!: store must respond to #get and #set."
  end
  me = Integer(max_entries)
  raise ArgumentError, "max_entries must be positive" if me <= 0
  MONITOR.synchronize do
    @store = store || LRUStore.new(max_entries: me, ttl: ttl && Float(ttl))
    @enabled = true
    @hits = 0
    @misses = 0
  end
  nil
end

.enabled? ⇒ Boolean

Returns:

  • (Boolean) 


198
199
200
 
# File 'lib/parse/embeddings/cache.rb', line 198

def enabled?
  MONITOR.synchronize { !!@enabled }
end

.fetch_vector(provider, input, input_type: :search_query) ⇒ Array<Float>

Embed a single input through provider, serving repeats from the cache. Pass-through (no caching, no instrumentation changes) when the cache is disabled.

Parameters:

  • provider (Provider) —

    the embedding provider.

  • input (String) —

    the text to embed.

  • input_type (Symbol) (defaults to: :search_query) —

    forwarded to embed_text.

Returns:

  • (Array<Float>) —

    the embedding vector.



234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
 
# File 'lib/parse/embeddings/cache.rb', line 234

def fetch_vector(provider, input, input_type: :search_query)
  unless enabled?
    return embed_single!(provider, input, input_type)
  end
  key = key_for(provider, input, input_type)
  cached = MONITOR.synchronize { @store && @store.get(key) }
  if cached
    MONITOR.synchronize { @hits = @hits.to_i + 1 }
    instrument_hit(provider, input_type)
    return cached
  end
  vector = embed_single!(provider, input, input_type)
  MONITOR.synchronize do
    @misses = @misses.to_i + 1
    @store.set(key, vector) if @store
  end
  vector
end

.stats ⇒ Hash

Returns { enabled:, hits:, misses:, size: }. size is nil for custom stores that don't expose one.

Returns:

  • (Hash) —

    { enabled:, hits:, misses:, size: }. size is nil for custom stores that don't expose one.



215
216
217
218
219
220
221
222
223
224
 
# File 'lib/parse/embeddings/cache.rb', line 215

def stats
  MONITOR.synchronize do
    {
      enabled: !!@enabled,
      hits: @hits.to_i,
      misses: @misses.to_i,
      size: (@store.respond_to?(:size) ? @store.size : nil),
    }
  end
end