Class: RubyLLM::Agents::Pipeline::Middleware::Cache

Inherits:
Base
  • Object
show all
Defined in:
lib/ruby_llm/agents/pipeline/middleware/cache.rb

Overview

Caches results to avoid redundant API calls.

This middleware provides caching for agent executions:

  • Checks cache before execution

  • Stores successful results in cache

  • Respects TTL configuration from agent DSL

Caching is skipped if:

  • Caching is not enabled on the agent class (no cache_for DSL)

  • The cache store is not configured

Examples:

Enable caching on an agent

class MyEmbedder < RubyLLM::Agents::Embedder
  model "text-embedding-3-small"
  cache_for 1.hour
end

Constant Summary

Constants inherited from Base

Base::LOG_TAG

Instance Method Summary collapse

Methods inherited from Base

#initialize

Constructor Details

This class inherits a constructor from RubyLLM::Agents::Pipeline::Middleware::Base

Instance Method Details

#call(context) ⇒ Context

Process caching

Parameters:

  • context (Context) —

    The execution context

Returns:

  • (Context) —

    The context (possibly from cache)



31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
 
# File 'lib/ruby_llm/agents/pipeline/middleware/cache.rb', line 31

def call(context)
  return @app.call(context) unless cache_enabled?

  cache_action = nil
  result = trace(context, action: "cache") do
    cache_key = generate_cache_key(context)

    # Skip cache read if skip_cache is true
    unless context.skip_cache
      # Try to read from cache
      if (cached = cache_read(cache_key))
        context.output = cached
        context.cached = true
        context[:cache_key] = cache_key
        cache_action = "hit"
        debug("Cache hit for #{cache_key}", context)
        emit_cache_notification("ruby_llm_agents.cache.hit", cache_key)
        next context
      end
    end

    cache_action = "miss"
    emit_cache_notification("ruby_llm_agents.cache.miss", cache_key)

    # Execute the chain
    @app.call(context)

    # Cache successful results
    if context.success?
      cache_write(cache_key, context.output)
      debug("Cache write for #{cache_key}", context)
      emit_cache_notification("ruby_llm_agents.cache.write", cache_key)
    end

    context
  end

  # Update the last trace entry with the specific cache action
  if context.trace_enabled? && cache_action && context.trace.last
    context.trace.last[:action] = cache_action
  end

  result
end