Module: Langfuse::StaleWhileRevalidate

Included in:
PromptCache, RailsCacheAdapter
Defined in:
lib/langfuse/stale_while_revalidate.rb

Overview

Stale-While-Revalidate caching pattern module

Provides SWR functionality for cache implementations. When included, allows serving stale data immediately while refreshing in the background.

Including classes must implement:

  • cache_get(key) - Read from cache

  • cache_set(key, value) - Write to cache

  • acquire_lock(lock_key) - Acquire lock for background refresh

  • release_lock(lock_key) - Release refresh lock

Examples:

class MyCache
  include Langfuse::StaleWhileRevalidate

  def initialize(ttl: 60, stale_ttl: 0)
    @ttl = ttl
    @stale_ttl = stale_ttl
    @logger = Logger.new($stdout)
    initialize_swr if stale_ttl.positive?
  end

  def cache_get(key)
    @storage[key]
  end

  def cache_set(key, value)
    @storage[key] = value
  end

  def acquire_lock(lock_key)
    # Implementation-specific lock acquisition
  end

  def release_lock(lock_key)
    # Implementation-specific lock release
  end
end

Instance Method Summary collapse

Instance Method Details

#fetch_with_stale_while_revalidate(key) { ... } ⇒ Object

Fetch a value from cache with Stale-While-Revalidate support

This method implements SWR caching: serves stale data immediately while refreshing in the background. Requires SWR to be enabled (stale_ttl must be positive).

Three cache states:

  • FRESH: Return immediately, no action needed

  • STALE: Return stale data + trigger background refresh

  • EXPIRED: Must fetch fresh data synchronously

Examples:

cache.fetch_with_stale_while_revalidate("greeting:v1") do
  api_client.get_prompt("greeting")
end

Parameters:

  • key (String) —

    Cache key

Yields:

  • Block to execute to fetch fresh data

Returns:

  • (Object) —

    Cached, stale, or freshly fetched value

Raises:



75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
 
# File 'lib/langfuse/stale_while_revalidate.rb', line 75

def fetch_with_stale_while_revalidate(key, &)
  raise ConfigurationError, "fetch_with_stale_while_revalidate requires a positive stale_ttl" unless swr_enabled?

  entry = cache_get(key)

  if entry&.fresh?
    # FRESH - return immediately
    logger.debug("CACHE HIT!")
    entry.data
  elsif entry&.stale?
    # REVALIDATE - return stale + refresh in background
    logger.debug("CACHE STALE!")
    schedule_refresh(key, &)
    entry.data # Instant response!
  else
    # MISS - must fetch synchronously
    logger.debug("CACHE MISS!")
    fetch_and_cache(key, &)
  end
end

#initialize_swr(refresh_threads: 5) ⇒ void

This method returns an undefined value.

Initialize SWR infrastructure

Must be called by including class after setting @stale_ttl, @ttl, and @logger. Typically called in the class’s initialize method when stale_ttl is provided.

Parameters:

  • refresh_threads (Integer) (defaults to: 5) —

    Number of background refresh threads (default: 5)



52
53
54
 
# File 'lib/langfuse/stale_while_revalidate.rb', line 52

def initialize_swr(refresh_threads: 5)
  @thread_pool = initialize_thread_pool(refresh_threads)
end

#shutdown ⇒ void

This method returns an undefined value.

Shutdown the cache refresh thread pool gracefully



109
110
111
112
113
114
 
# File 'lib/langfuse/stale_while_revalidate.rb', line 109

def shutdown
  return unless @thread_pool

  @thread_pool.shutdown
  @thread_pool.wait_for_termination(5) # Wait up to 5 seconds
end

#swr_enabled? ⇒ Boolean

Check if SWR is enabled

SWR is enabled when stale_ttl is positive, meaning there’s a grace period where stale data can be served while revalidating in the background.

Returns:

  • (Boolean) —

    true if stale_ttl is positive



102
103
104
 
# File 'lib/langfuse/stale_while_revalidate.rb', line 102

def swr_enabled?
  stale_ttl.positive?
end