Module: Langfuse::StaleWhileRevalidate

Included in:
PromptCache, RailsCacheAdapter
Defined in:
lib/langfuse/stale_while_revalidate.rb

Overview

Stale-While-Revalidate caching pattern module

Provides SWR functionality for cache implementations. When included, allows serving stale data immediately while refreshing in the background.

Including classes must implement:

  • cache_get(key) - Read from cache

  • cache_set(key, value) - Write to cache

  • acquire_lock(lock_key) - Acquire lock for background refresh

  • release_lock(lock_key) - Release refresh lock

rubocop:disable Metrics/ModuleLength

Examples:

class MyCache
  include Langfuse::StaleWhileRevalidate

  def initialize(ttl: 60, stale_ttl: 0)
    @ttl = ttl
    @stale_ttl = stale_ttl
    @logger = Logger.new($stdout)
    initialize_swr if stale_ttl.positive?
  end

  def cache_get(key)
    @storage[key]
  end

  def cache_set(key, value)
    @storage[key] = value
  end

  def acquire_lock(lock_key)
    # Implementation-specific lock acquisition
  end

  def release_lock(lock_key)
    # Implementation-specific lock release
  end
end

Instance Method Summary collapse

Instance Method Details

#fetch_with_stale_while_revalidate(key, ttl: nil, stale_ttl: nil) { ... } ⇒ Object

Fetch a value from cache with Stale-While-Revalidate support

This method implements SWR caching: serves stale data immediately while refreshing in the background. Requires SWR to be enabled (stale_ttl must be positive).

Three cache states:

  • FRESH: Return immediately, no action needed

  • STALE: Return stale data + trigger background refresh

  • EXPIRED: Must fetch fresh data synchronously

Examples:

cache.fetch_with_stale_while_revalidate("greeting:v1") do
  api_client.get_prompt("greeting")
end

Parameters:

  • key (String) —

    Cache key

Yields:

  • Block to execute to fetch fresh data

Returns:

  • (Object) —

    Cached, stale, or freshly fetched value

Raises:



76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
 
# File 'lib/langfuse/stale_while_revalidate.rb', line 76

def fetch_with_stale_while_revalidate(key, ttl: nil, stale_ttl: nil, &)
  raise ConfigurationError, "fetch_with_stale_while_revalidate requires a positive stale_ttl" unless swr_enabled?

  entry = cache_get(key)

  if entry&.fresh?
    # FRESH - return immediately
    logger.debug("CACHE HIT!")
    entry.data
  elsif entry&.stale?
    # REVALIDATE - return stale + refresh in background
    logger.debug("CACHE STALE!")
    schedule_refresh(key, ttl: ttl, stale_ttl: stale_ttl, &)
    entry.data # Instant response!
  else
    # MISS - must fetch synchronously
    logger.debug("CACHE MISS!")
    fetch_and_cache(key, ttl: ttl, stale_ttl: stale_ttl, &)
  end
end

#initialize_swr(refresh_threads: 5) ⇒ void

This method returns an undefined value.

Initialize SWR infrastructure

Must be called by including class after setting @stale_ttl, @ttl, and @logger. Typically called in the class’s initialize method when stale_ttl is provided.

Parameters:

  • refresh_threads (Integer) (defaults to: 5) —

    Number of background refresh threads (default: 5)



53
54
55
 
# File 'lib/langfuse/stale_while_revalidate.rb', line 53

def initialize_swr(refresh_threads: 5)
  @thread_pool = initialize_thread_pool(refresh_threads)
end

#refresh_async(key, ttl: nil, stale_ttl: nil, on_success: nil, on_failure: nil) { ... } ⇒ Boolean

Schedule a cache refresh without performing a read.

Parameters:

  • key (String) —

    Cache key

  • ttl (Integer, nil) (defaults to: nil) —

    Optional fresh TTL override

  • stale_ttl (Integer, nil) (defaults to: nil) —

    Optional stale TTL override

  • on_success (#call, nil) (defaults to: nil) —

    Callback invoked after a successful write

  • on_failure (#call, nil) (defaults to: nil) —

    Callback invoked when refresh raises

Yields:

  • Block to execute to fetch fresh data

Returns:

  • (Boolean) —

    true if a refresh was scheduled

Raises:



106
107
108
109
110
111
112
113
114
115
116
117
 
# File 'lib/langfuse/stale_while_revalidate.rb', line 106

def refresh_async(key, ttl: nil, stale_ttl: nil, on_success: nil, on_failure: nil, &)
  raise ConfigurationError, "refresh_async requires a positive stale_ttl" unless swr_enabled?

  schedule_refresh(
    key,
    ttl: ttl,
    stale_ttl: stale_ttl,
    on_success: on_success,
    on_failure: on_failure,
    &
  )
end

#shutdown ⇒ void

This method returns an undefined value.

Shutdown the cache refresh thread pool gracefully



145
146
147
148
149
150
 
# File 'lib/langfuse/stale_while_revalidate.rb', line 145

def shutdown
  return unless @thread_pool

  @thread_pool.shutdown
  @thread_pool.wait_for_termination(5) # Wait up to 5 seconds
end

#swr_enabled? ⇒ Boolean

Check if SWR is enabled

SWR is enabled when stale_ttl is positive, meaning there’s a grace period where stale data can be served while revalidating in the background.

Returns:

  • (Boolean) —

    true if stale_ttl is positive



138
139
140
 
# File 'lib/langfuse/stale_while_revalidate.rb', line 138

def swr_enabled?
  stale_ttl.positive?
end

#write_with_stale_while_revalidate(key, value, ttl: nil, stale_ttl: nil) ⇒ Object

Write a value with stale-while-revalidate metadata.

Parameters:

  • key (String) —

    Cache key

  • value (Object) —

    Value to cache

  • ttl (Integer, nil) (defaults to: nil) —

    Optional fresh TTL override

  • stale_ttl (Integer, nil) (defaults to: nil) —

    Optional stale TTL override

Returns:

  • (Object) —

    The cached value

Raises:



126
127
128
129
130
 
# File 'lib/langfuse/stale_while_revalidate.rb', line 126

def write_with_stale_while_revalidate(key, value, ttl: nil, stale_ttl: nil)
  raise ConfigurationError, "write_with_stale_while_revalidate requires a positive stale_ttl" unless swr_enabled?

  set_cache_entry(key, value, ttl: ttl, stale_ttl: stale_ttl)
end