Class: Ragnar::LLMManager

Inherits:
Object
  • Object
show all
Includes:
Singleton
Defined in:
lib/ragnar/llm_manager.rb

Overview

Singleton manager for RubyLLM chat instances to avoid reloading models. Supports any RubyLLM provider (red_candle for local, openai, anthropic, etc.)

Instance Method Summary collapse

Constructor Details

#initialize ⇒ LLMManager

Returns a new instance of LLMManager.



7
8
9
10
 
# File 'lib/ragnar/llm_manager.rb', line 7

def initialize
  @chats = {}
  @mutex = Mutex.new
end

Instance Method Details

#clear_cache ⇒ Object

Clear all cached chat instances (useful for memory management)



32
33
34
35
36
 
# File 'lib/ragnar/llm_manager.rb', line 32

def clear_cache
  @mutex.synchronize do
    @chats.clear
  end
end

#default_chat ⇒ Object Also known as: default_llm

Get the default chat instance for the application



39
40
41
 
# File 'lib/ragnar/llm_manager.rb', line 39

def default_chat
  get_chat
end

#get_chat(provider: nil, model: nil) ⇒ RubyLLM::Chat Also known as: get_llm

Get or create a RubyLLM chat instance

Parameters:

  • provider (String, Symbol) (defaults to: nil) —

    The RubyLLM provider (default from config)

  • model (String) (defaults to: nil) —

    The model identifier (default from config)

Returns:

  • (RubyLLM::Chat) —

    A cached chat instance



16
17
18
19
20
21
22
23
24
25
26
27
28
29
 
# File 'lib/ragnar/llm_manager.rb', line 16

def get_chat(provider: nil, model: nil)
  config = Config.instance
  provider ||= config.llm_provider
  model ||= config.llm_model

  cache_key = "#{provider}:#{model}"

  @mutex.synchronize do
    @chats[cache_key] ||= begin
      puts "Loading LLM: #{model} (#{provider})..." if ENV['DEBUG']
      Config.instance.create_chat
    end
  end
end