Module: LLM

Defined in:
lib/llm.rb,
lib/llm/pipe.rb,
lib/llm/repl.rb,
lib/llm/agent.rb,
lib/llm/error.rb,
lib/llm/skill.rb,
lib/llm/utils.rb,
lib/llm/buffer.rb,
lib/llm/stream.rb,
lib/llm/tracer.rb,
lib/llm/context.rb,
lib/llm/message.rb,
lib/llm/session.rb,
lib/llm/uridata.rb,
lib/llm/version.rb,
lib/llm/contract.rb,
lib/llm/response.rb,
lib/llm/compactor.rb,
lib/llm/transport.rb,
lib/llm/tracer/null.rb,
lib/llm/eventhandler.rb,
lib/llm/json_adapter.rb,
lib/llm/providers/xai.rb,
lib/llm/providers/zai.rb,
lib/llm/tracer/logger.rb,
lib/llm/providers/google.rb,
lib/llm/providers/ollama.rb,
lib/llm/providers/openai.rb,
lib/llm/tracer/telemetry.rb,
lib/llm/providers/bedrock.rb,
lib/llm/providers/mistral.rb,
lib/llm/providers/deepseek.rb,
lib/llm/providers/llamacpp.rb,
lib/llm/providers/anthropic.rb,
lib/llm/providers/deepinfra.rb

Overview

llm.rb is a zero-dependency AI runtime for Ruby. Twelve providers, six concurrency strategies, MCP and A2A, streaming tool calls with cancellation, context compaction, and ORM persistence.

Examples:

The three-step workflow

require "llm"
llm = LLM.deepseek(key: ENV["KEY"])           # 1. pick a provider
agent = LLM::Agent.new(llm, stream: $stdout)   # 2. create an agent
agent.talk "Hello world"                       # 3. talk to it

See Also:

Defined Under Namespace

Modules: ActiveRecord, Contract, EventStream, Sequel, Utils Classes: A2A, Agent, Anthropic, Bedrock, Buffer, Compactor, Context, Cost, DeepInfra, DeepSeek, Error, EventHandler, File, Function, Google, JSONAdapter, LlamaCpp, LoopGuard, MCP, Message, Mime, Mistral, Model, Multipart, Object, Ollama, OpenAI, Pipe, Prompt, Provider, Registry, Repl, Response, Schema, ServerTool, Skill, Stream, Tool, Tracer, Transport, URIData, Usage, XAI, ZAI

Constant Summary collapse

UnauthorizedError =

HTTPUnauthorized

Class.new(Error)
RateLimitError =

HTTPTooManyRequests

Class.new(Error)
ServerError =

HTTPServerError

Class.new(Error)
NotFoundError =

HTTPNotFound

Class.new(Error)
FormatError =

When an given an input object that is not understood

Class.new(Error)
PromptError =

When given a prompt object that is not understood

Class.new(FormatError)
InvalidRequestError =

When given an invalid request

Class.new(Error)
ContextWindowError =

When the context window is exceeded

Class.new(InvalidRequestError)
ToolLoopError =

When stuck in a tool call loop

Class.new(Error)
GuardError =

When a guard blocks pending tool execution

Class.new(Error)
Interrupt =

When a request is interrupted

Class.new(Error)
RactorError =

When a concurrency strategy cannot execute a given tool

Class.new(Error)
NoSuchToolError =

When a tool call cannot be mapped to a local tool

Class.new(Error)
NoSuchModelError =

When Registry can't map a model

Class.new(Error)
NoSuchRegistryError =

When Registry can't map a registry

Class.new(Error)
LoadError =

When an optional runtime dependency cannot be required

Class.new(Error)
Session =
Deprecated.

Use Context instead. Scheduled for removal in v6.0.

Backward-compatible alias for LLM::Context

Context
VERSION = 
"13.0.0"
Command =

Convenience constant

LLM::Repl::Command

Class Method Summary collapse

Class Method Details

.a2a(http:, binding: :rest) ⇒ LLM::A2A

Creates a new A2A client connected to a remote agent.

Parameters:

  • http (Hash, nil)
  • binding (Symbol) (defaults to: :rest) —

    The protocol binding to use. One of :rest or :jsonrpc

Options Hash (http:):

  • :url (String) —

    The base URL of the A2A agent (e.g., "https://agent.example.com")

  • :headers (Hash<String, String>) —

    Extra HTTP headers (e.g., Authorization)

  • :timeout (Integer, nil) —

    Request timeout in seconds

  • :transport (LLM::Transport, Class, nil) —

    Optional transport override

Returns:



252
253
254
 
# File 'lib/llm.rb', line 252

def a2a(http:, binding: :rest)
  LLM::A2A.http(**http, binding:)
end

.anthropic ⇒ Object 



133
134
135
136
 
# File 'lib/llm.rb', line 133

def anthropic(**)
  lock(:require) { require_relative "llm/providers/anthropic" unless defined?(LLM::Anthropic) }
  LLM::Anthropic.new(**)
end

.bedrock ⇒ Object 



189
190
191
192
 
# File 'lib/llm.rb', line 189

def bedrock(**)
  lock(:require) { require_relative "llm/providers/bedrock" unless defined?(LLM::Bedrock) }
  LLM::Bedrock.new(**)
end

.deepinfra ⇒ Object 



181
182
183
184
 
# File 'lib/llm.rb', line 181

def deepinfra(**)
  lock(:require) { require_relative "llm/providers/deepinfra" unless defined?(LLM::DeepInfra) }
  LLM::DeepInfra.new(**)
end

.deepseek ⇒ Object 



165
166
167
168
 
# File 'lib/llm.rb', line 165

def deepseek(**)
  lock(:require) { require_relative "llm/providers/deepseek" unless defined?(LLM::DeepSeek) }
  LLM::DeepSeek.new(**)
end

.File(obj) ⇒ LLM::File

Parameters:

  • obj (String, File, LLM::Response) —

    The path to the file, or an existing file reference

Returns:



89
90
91
92
93
94
95
96
97
98
 
# File 'lib/llm/file.rb', line 89

def LLM.File(obj)
  case obj
  when File
    obj.close unless obj.closed?
    LLM.File(obj.path)
  when LLM::File, LLM::Response then obj
  when String then LLM::File.new(obj)
  else raise TypeError, "don't know how to handle #{obj.class} objects"
  end
end

.function(key, &b) ⇒ LLM::Function

Define a function

Examples:

LLM.function(:system) do |fn|
  fn.description "Run system command"
  fn.params do |schema|
    schema.object(command: schema.string.required)
  end
  fn.define do |command:|
    system(command)
  end
end

Parameters:

  • key (Symbol) —

    The function name / key

  • b (Proc) —

    The block to define the function

Returns:



271
272
273
 
# File 'lib/llm.rb', line 271

def function(key, &b)
  LLM::Function.new(key, &b)
end

.google ⇒ Object 



141
142
143
144
 
# File 'lib/llm.rb', line 141

def google(**)
  lock(:require) { require_relative "llm/providers/google" unless defined?(LLM::Google) }
  LLM::Google.new(**)
end

.json ⇒ Class

Returns the JSON adapter used by the library

Returns:

  • (Class) —

    Returns a class that responds to dump and load



102
103
104
 
# File 'lib/llm.rb', line 102

def json
  @json ||= JSONAdapter::JSON
end

.json=(adapter) ⇒ void

Note:

This should be set once from the main thread when your program starts. Defaults to LLM::JSONAdapter::JSON.

This method returns an undefined value.

Sets the JSON adapter used by the library

Parameters:

  • adapter (Class, String, Symbol) —

    A JSON adapter class or its name



114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
 
# File 'lib/llm.rb', line 114

def json=(adapter)
  @json = case adapter.to_s
  when "JSON", "json" then JSONAdapter::JSON
  when "Oj", "oj" then JSONAdapter::Oj
  when "Yajl", "yajl" then JSONAdapter::Yajl
  else
    is_class = Class === adapter
    is_subclass = is_class && adapter.ancestors.include?(LLM::JSONAdapter)
    if is_subclass
      adapter
    else
      raise TypeError, "Adapter must be a subclass of LLM::JSONAdapter"
    end
  end
end

.llamacpp(key: nil) ⇒ Object 



157
158
159
160
 
# File 'lib/llm.rb', line 157

def llamacpp(key: nil, **)
  lock(:require) { require_relative "llm/providers/llamacpp" unless defined?(LLM::LlamaCpp) }
  LLM::LlamaCpp.new(key:, **)
end

.lock(name, &block) ⇒ void

This method returns an undefined value.

Provides a thread-safe lock

Parameters:

  • name (Symbol) —

    The name of the lock

  • block (Proc) —

    The block to execute within the lock



290
 
# File 'lib/llm.rb', line 290

def lock(name, &block) = @monitors[name].synchronize(&block)

.logger(llm, **params) ⇒ LLM::Tracer::Logger

Parameters:

  • llm (LLM::Provider) —

    A provider

  • params (Hash) —

    Forwarded to LLM::Tracer::Logger

Returns:



281
282
283
 
# File 'lib/llm.rb', line 281

def logger(llm, **params)
  LLM::Tracer::Logger.new(llm, params)
end

.mcp(**opts) ⇒ LLM::MCP

Parameters:

  • opts (Hash) —

    MCP client options

Options Hash (**opts):

  • :stdio (Hash, nil) —

    Standard I/O transport options

  • :stdio/:argv (Array<String>) —

    The command to run for the MCP process

  • :stdio/:env (Hash) —

    The environment variables to set for the MCP process

  • :stdio/:cwd (String, nil) —

    The working directory for the MCP process

Returns:



233
234
235
 
# File 'lib/llm.rb', line 233

def mcp(**opts)
  LLM::MCP.new(**opts)
end

.mistral ⇒ Object 



207
208
209
210
 
# File 'lib/llm.rb', line 207

def mistral(**)
  lock(:require) { require_relative "llm/providers/mistral" unless defined?(LLM::Mistral) }
  LLM::Mistral.new(**)
end

.ollama(key: nil) ⇒ Object 



149
150
151
152
 
# File 'lib/llm.rb', line 149

def ollama(key: nil, **)
  lock(:require) { require_relative "llm/providers/ollama" unless defined?(LLM::Ollama) }
  LLM::Ollama.new(key:, **)
end

.openai ⇒ Object 



173
174
175
176
 
# File 'lib/llm.rb', line 173

def openai(**)
  lock(:require) { require_relative "llm/providers/openai" unless defined?(LLM::OpenAI) }
  LLM::OpenAI.new(**)
end

.registry_for(llm) ⇒ LLM::Object

Parameters:

  • llm (Symbol, LLM::Provider) —

    The name of a provider, or an instance of LLM::Provider

Returns:



89
90
91
92
93
94
 
# File 'lib/llm.rb', line 89

def self.registry_for(llm)
  lock(:registry) do
    name = Symbol === llm ? llm : llm.name
    @registry[name] ||= Registry.for(name)
  end
end

.require(name, version = nil) ⇒ Object

Requires an optional runtime dependency

Parameters:

  • name (String) —

    The name of a gem

  • version (String, nil) (defaults to: nil) —

    Optional gem version

Raises:



72
73
74
75
76
77
78
79
80
81
82
83
 
# File 'lib/llm.rb', line 72

def self.require(name, version = nil)
  names = {"xchan" => "xchan.rb", "net/http/persistent" => "net-http-persistent"}
  gem(names[name] || name, version) if version
  super(name)
rescue ::LoadError
  name = names[name] || name
  raise LLM::LoadError,
    "#{name}#{version ? " #{version}" : ""} is an optional " \
    "runtime dependency but it does not appear to be installed. " \
    "Consider 'gem install #{name}', adding '#{name}' to your Gemfile or " \
    "opting out of the functionality provided by '#{name}'"
end

.xai ⇒ Object 



198
199
200
201
 
# File 'lib/llm.rb', line 198

def xai(**)
  lock(:require) { require_relative "llm/providers/xai" unless defined?(LLM::XAI) }
  LLM::XAI.new(**)
end

.zai ⇒ Object 



216
217
218
219
 
# File 'lib/llm.rb', line 216

def zai(**)
  lock(:require) { require_relative "llm/providers/zai" unless defined?(LLM::ZAI) }
  LLM::ZAI.new(**)
end