Class: Langfuse::CacheWarmer

Inherits:
Object
  • Object
show all
Defined in:
lib/langfuse/cache_warmer.rb

Overview

Cache warming utility for pre-loading prompts into cache

Useful for deployment scenarios where you want to warm the cache before serving traffic, preventing cold-start API calls.

Examples:

Warm cache with specific prompts

warmer = Langfuse::CacheWarmer.new
results = warmer.warm(['greeting', 'conversation', 'rag-pipeline'])
puts "Cached #{results[:success].size} prompts"

Warm cache with error handling

warmer = Langfuse::CacheWarmer.new(client: my_client)
results = warmer.warm(['greeting', 'conversation'])

results[:failed].each do |failure|
  logger.warn "Failed to cache #{failure[:name]}: #{failure[:error]}"
end

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(client: nil) ⇒ CacheWarmer

Initialize a new cache warmer

Parameters:

  • client (Client, nil) (defaults to: nil) —

    Optional Langfuse client (defaults to global client)



29
30
31
 
# File 'lib/langfuse/cache_warmer.rb', line 29

def initialize(client: nil)
  @client = client || Langfuse.client
end

Instance Attribute Details

#client ⇒ Client (readonly)

Returns Langfuse client used for fetching prompts.

Returns:

  • (Client) —

    Langfuse client used for fetching prompts



24
25
26
 
# File 'lib/langfuse/cache_warmer.rb', line 24

def client
  @client
end

Instance Method Details

#cache_enabled? ⇒ Boolean

Check if cache warming is enabled

Returns false if caching is disabled (cache_ttl = 0)

Returns:

  • (Boolean) 


150
151
152
 
# File 'lib/langfuse/cache_warmer.rb', line 150

def cache_enabled?
  client.prompt_cache_stats[:enabled] == true
end

#cache_stats ⇒ Hash?

Get cache statistics (if supported by backend)

Returns:

  • (Hash, nil) —

    Cache stats or nil if not supported



157
158
159
160
161
162
163
164
165
166
167
 
# File 'lib/langfuse/cache_warmer.rb', line 157

def cache_stats
  stats = client.prompt_cache_stats
  return nil unless stats[:enabled]

  {
    backend: public_backend_name(stats[:backend]),
    ttl: stats[:ttl],
    size: stats[:size],
    max_size: stats[:max_size]
  }
end

#warm(prompt_names, versions: {}, labels: {}) ⇒ Hash

Warm the cache with specified prompts

Fetches each prompt and populates the cache. This is idempotent - safe to call multiple times.

Examples:

Basic warming

results = warmer.warm(['greeting', 'conversation'])
# => { success: ['greeting', 'conversation'], failed: [] }

With specific versions

results = warmer.warm(
  ['greeting', 'conversation'],
  versions: { 'greeting' => 2, 'conversation' => 1 }
)

With labels

results = warmer.warm(
  ['greeting', 'conversation'],
  labels: { 'greeting' => 'production' }
)

Parameters:

  • prompt_names (Array<String>) —

    List of prompt names to cache

  • versions (Hash<String, Integer>) (defaults to: {}) —

    Optional version numbers per prompt

  • labels (Hash<String, String>) (defaults to: {}) —

    Optional labels per prompt

Returns:

  • (Hash) —

    Results with :success and :failed arrays



58
59
60
61
62
63
64
65
66
 
# File 'lib/langfuse/cache_warmer.rb', line 58

def warm(prompt_names, versions: {}, labels: {})
  results = { success: [], failed: [] }

  prompt_names.each do |name|
    warm_single_prompt(name, results, versions, labels)
  end

  results
end

#warm!(prompt_names, versions: {}, labels: {}) ⇒ Hash

Warm the cache and raise on any failures

Same as #warm but raises an error if any prompts fail to cache. Useful when you want to abort deployment if cache warming fails.

Examples:

begin
  warmer.warm!(['greeting', 'conversation'])
rescue Langfuse::CacheWarmingError => e
  abort "Cache warming failed: #{e.message}"
end

Parameters:

  • prompt_names (Array<String>) —

    List of prompt names to cache

  • versions (Hash<String, Integer>) (defaults to: {}) —

    Optional version numbers per prompt

  • labels (Hash<String, String>) (defaults to: {}) —

    Optional labels per prompt

Returns:

  • (Hash) —

    Results with :success array

Raises:



134
135
136
137
138
139
140
141
142
143
 
# File 'lib/langfuse/cache_warmer.rb', line 134

def warm!(prompt_names, versions: {}, labels: {})
  results = warm(prompt_names, versions: versions, labels: labels)

  if results[:failed].any?
    failed_names = results[:failed].map { |f| f[:name] }.join(", ")
    raise CacheWarmingError, "Failed to cache prompts: #{failed_names}"
  end

  results
end

#warm_all(default_label: "production", versions: {}, labels: {}) ⇒ Hash

Warm the cache with all prompts (auto-discovery)

Automatically discovers all prompts in your Langfuse project via the list_prompts API and warms the cache with all of them. By default, fetches prompts with the “production” label. Useful for deployment scenarios where you want to ensure all prompts are cached without manually specifying them.

Examples:

Auto-discover and warm all prompts with “production” label

results = warmer.warm_all
puts "Cached #{results[:success].size} prompts"

Warm with a different default label

results = warmer.warm_all(default_label: "staging")

Warm without any label (API-determined selection)

results = warmer.warm_all(default_label: nil)

With specific versions for some prompts

results = warmer.warm_all(versions: { 'greeting' => 2 })

Override label for specific prompts

results = warmer.warm_all(
  default_label: "production",
  labels: { 'greeting' => 'staging' }  # Use staging for this one
)

Parameters:

  • default_label (String, nil) (defaults to: "production") —

    Label to use for all prompts (default: “production”)

  • versions (Hash<String, Integer>) (defaults to: {}) —

    Optional version numbers per prompt

  • labels (Hash<String, String>) (defaults to: {}) —

    Optional labels per specific prompts (overrides default_label)

Returns:

  • (Hash) —

    Results with :success and :failed arrays



99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
 
# File 'lib/langfuse/cache_warmer.rb', line 99

def warm_all(default_label: "production", versions: {}, labels: {})
  prompt_list = client.list_prompts
  prompt_names = prompt_list.map { |p| p["name"] }.uniq

  # Build labels hash: apply default_label to all prompts, then merge overrides
  # BUT: if a version is specified for a prompt, don't apply a label (version takes precedence)
  final_labels = {}
  if default_label
    prompt_names.each do |name|
      # Only apply default label if no version specified for this prompt
      final_labels[name] = default_label unless versions[name]
    end
  end
  final_labels.merge!(labels) # Specific label overrides win

  warm(prompt_names, versions: versions, labels: final_labels)
end