Class: Langfuse::RailsCacheAdapter

Inherits:
Object
  • Object
show all
Includes:
StaleWhileRevalidate
Defined in:
lib/langfuse/rails_cache_adapter.rb

Overview

Rails.cache adapter for distributed caching with Redis

Wraps Rails.cache to provide distributed caching for prompts across multiple processes and servers. Requires Rails with Redis cache store.

Examples:

adapter = Langfuse::RailsCacheAdapter.new(ttl: 60)
adapter.set("greeting:1", prompt_data)
adapter.get("greeting:1") # => prompt_data

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from StaleWhileRevalidate

#fetch_with_stale_while_revalidate, #initialize_swr, #shutdown, #swr_enabled?

Constructor Details

#initialize(ttl: 60, namespace: "langfuse", lock_timeout: 10, stale_ttl: 0, refresh_threads: 5, logger: default_logger) ⇒ RailsCacheAdapter

Initialize a new Rails.cache adapter

Parameters:

  • ttl (Integer) (defaults to: 60) —

    Time-to-live in seconds (default: 60)

  • namespace (String) (defaults to: "langfuse") —

    Cache key namespace (default: “langfuse”)

  • lock_timeout (Integer) (defaults to: 10) —

    Lock timeout in seconds for stampede protection (default: 10)

  • stale_ttl (Integer) (defaults to: 0) —

    Stale TTL for SWR in seconds (default: 0, SWR disabled). Note: :indefinite is normalized to 1000 years by Config before being passed here.

  • refresh_threads (Integer) (defaults to: 5) —

    Number of background refresh threads (default: 5)

  • logger (Logger, nil) (defaults to: default_logger) —

    Logger instance for error reporting (default: nil, creates new logger)

Raises:



48
49
50
51
52
53
54
55
56
57
58
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 48

def initialize(ttl: 60, namespace: "langfuse", lock_timeout: 10, stale_ttl: 0, refresh_threads: 5,
               logger: default_logger)
  validate_rails_cache!

  @ttl = ttl
  @namespace = namespace
  @lock_timeout = lock_timeout
  @stale_ttl = stale_ttl
  @logger = logger
  initialize_swr(refresh_threads: refresh_threads) if swr_enabled?
end

Instance Attribute Details

#lock_timeout ⇒ Integer (readonly)

Returns Lock timeout in seconds for stampede protection.

Returns:

  • (Integer) —

    Lock timeout in seconds for stampede protection



27
28
29
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 27

def lock_timeout
  @lock_timeout
end

#logger ⇒ Logger (readonly)

Returns Logger instance for error reporting.

Returns:

  • (Logger) —

    Logger instance for error reporting



36
37
38
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 36

def logger
  @logger
end

#namespace ⇒ String (readonly)

Returns Cache key namespace.

Returns:

  • (String) —

    Cache key namespace



24
25
26
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 24

def namespace
  @namespace
end

#stale_ttl ⇒ Integer (readonly)

Returns Stale TTL for SWR in seconds.

Returns:

  • (Integer) —

    Stale TTL for SWR in seconds



30
31
32
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 30

def stale_ttl
  @stale_ttl
end

#thread_pool ⇒ Concurrent::CachedThreadPool? (readonly)

Returns Thread pool for background refreshes.

Returns:

  • (Concurrent::CachedThreadPool, nil) —

    Thread pool for background refreshes



33
34
35
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 33

def thread_pool
  @thread_pool
end

#ttl ⇒ Integer (readonly)

Returns Time-to-live in seconds.

Returns:

  • (Integer) —

    Time-to-live in seconds



21
22
23
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 21

def ttl
  @ttl
end

Class Method Details

.build_key(name, version: nil, label: nil) ⇒ String

Build a cache key from prompt name and options

Parameters:

  • name (String) —

    Prompt name

  • version (Integer, nil) (defaults to: nil) —

    Optional version

  • label (String, nil) (defaults to: nil) —

    Optional label

Returns:

  • (String) —

    Cache key



117
118
119
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 117

def self.build_key(name, version: nil, label: nil)
  PromptCache.build_key(name, version: version, label: label)
end

Instance Method Details

#clear ⇒ void

This method returns an undefined value.

Clear the entire Langfuse cache namespace

Note: This uses delete_matched which may not be available on all cache stores. Works with Redis, Memcached, and memory stores. File store support varies.



86
87
88
89
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 86

def clear
  # Delete all keys matching the namespace pattern
  Rails.cache.delete_matched("#{namespace}:*")
end

#empty? ⇒ Boolean

Check if cache is empty

Note: Rails.cache doesn’t provide an efficient way to check if empty, so we return false to indicate this operation is not supported.

Returns:

  • (Boolean) —

    Always returns false (unsupported operation)



107
108
109
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 107

def empty?
  false
end

#fetch_with_lock(key) { ... } ⇒ Object

Fetch a value from cache with lock for stampede protection

This method prevents cache stampedes (thundering herd) by ensuring only one process/thread fetches from the source when the cache is empty. Others wait for the first one to populate the cache.

Uses exponential backoff: 50ms, 100ms, 200ms (3 retries max, ~350ms total). If cache is still empty after waiting, falls back to fetching from source.

Examples:

cache.fetch_with_lock("greeting:v1") do
  api_client.get_prompt("greeting")
end

Parameters:

  • key (String) —

    Cache key

Yields:

  • Block to execute if cache miss (should fetch fresh data)

Returns:

  • (Object) —

    Cached or freshly fetched value



138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 138

def fetch_with_lock(key)
  # 1. Check cache first (fast path - no lock needed)
  cached = get(key)
  return cached if cached

  # 2. Cache miss - try to acquire lock
  lock_key = build_lock_key(key)

  if acquire_lock(lock_key)
    begin
      # We got the lock - fetch from source and populate cache
      value = yield
      set(key, value)
      value
    ensure
      # Always release lock, even if block raises
      release_lock(lock_key)
    end
  else
    # Someone else has the lock - wait for them to populate cache
    cached = wait_for_cache(key)
    return cached if cached

    # Cache still empty after waiting - fall back to fetching ourselves
    # (This handles cases where lock holder crashed or took too long)
    yield
  end
end

#get(key) ⇒ Object?

Get a value from the cache

Parameters:

  • key (String) —

    Cache key

Returns:

  • (Object, nil) —

    Cached value or nil if not found/expired



64
65
66
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 64

def get(key)
  Rails.cache.read(namespaced_key(key))
end

#set(key, value) ⇒ Object

Set a value in the cache

Parameters:

  • key (String) —

    Cache key

  • value (Object) —

    Value to cache

Returns:

  • (Object) —

    The cached value



73
74
75
76
77
78
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 73

def set(key, value)
  # Calculate expiration: use total_ttl if SWR enabled, otherwise just ttl
  expires_in = swr_enabled? ? total_ttl : ttl
  Rails.cache.write(namespaced_key(key), value, expires_in:)
  value
end

#size ⇒ nil

Get current cache size

Note: Rails.cache doesn’t provide a size method, so we return nil to indicate this operation is not supported.

Returns:

  • (nil) 


97
98
99
 
# File 'lib/langfuse/rails_cache_adapter.rb', line 97

def size
  nil
end