Class: LLM::Context

Inherits:
Object
  • Object
show all
Includes:
Deserializer, Serializer
Defined in:
lib/llm/context.rb,
lib/llm/context/serializer.rb,
lib/llm/context/deserializer.rb

Overview

LLM::Context is the low-level stateful execution boundary in llm.rb. Most users should start with Agent, which wraps Context and manages tool loops automatically. Use Context directly when you need manual control over tool execution.

It holds the evolving runtime state for an LLM workflow: conversation history, tool calls and returns, schema and streaming configuration, accumulated usage, and request ownership for interruption.

This is broader than prompt context alone. A context is the object that lets one-off prompts, streaming turns, tool execution, persistence, retries, and serialized long-lived workflows all run through the same model.

A context can drive the chat completions API that all providers support or the Responses API on providers that expose it.

Examples:

#!/usr/bin/env ruby
require "llm"

llm = LLM.deepseek(key: ENV["KEY"])
ctx = LLM::Context.new(llm, stream: $stdout)
ctx.talk "If a train goes 60 mph for 1.5 hours, how far does it travel?"
ctx.messages.each { |m| puts "[#{m.role}] #{m.content}" }

See Also:

Defined Under Namespace

Modules: Deserializer, Serializer

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Deserializer

#deserialize

Constructor Details

#initialize(llm, params = {}) ⇒ Context

Returns a new instance of Context.

Parameters:

  • llm (LLM::Provider) —

    A provider

  • params (Hash) (defaults to: {}) —

    The parameters to maintain throughout the conversation. Any parameter the provider supports can be included and not only those listed here.

Options Hash (params):

  • :mode (Symbol) —

    Defaults to :responses for OpenAI, otherwise it defaults to :completions.

  • :model (String) —

    Defaults to the provider's default model

  • :compactor (Class<LLM::Compactor>, nil) —

    A compactor class to use for context compaction. Defaults to LLM::Compactor::Null.

  • :compactor_options (Hash) —

    Options passed to the compactor's call method. Defaults to {}.

  • :transformer (Class<LLM::Transformer>, nil) —

    A transformer class to use for message transformation. Defaults to Transformer::Null.

  • :transformer_options (Hash) —

    Options passed to the transformer's call method. Defaults to {}.

  • :guard (Class<LLM::Guard>, nil) —

    A guard class to supervise agentic tool execution. Defaults to Guard::Null.

  • :guard_options (Hash) —

    Options passed to the guard's call method. Defaults to {}.

  • :tools (Array<LLM::Function>, nil) —

    Defaults to nil

  • :skills (Array<String>, nil) —

    Defaults to nil



100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
 
# File 'lib/llm/context.rb', line 100

def initialize(llm, params = {})
  params = {}.merge!(params)
  @llm = llm
  @record = params.delete(:record)
  @mode = params.delete(:mode) || (llm.name == :openai ? :responses : :completions)
  tools = [*params.delete(:tools), *load_skills(params.delete(:skills))]
  @params = {model: llm.default_model, schema: nil}.compact.merge!(params)
  @params[:tools] = tools unless tools.empty?
  @params[:store] ||= false if @mode == :responses
  @messages = LLM::Buffer.new(llm)
  extra = @params.slice(:model, :tools).merge!(ctx: self, tracer:)
  @params[:stream] = LLM::Stream.try(@params[:stream], extra:)
  @compactor = {
    klass: params.delete(:compactor) || LLM::Compactor::Null,
    options: params.delete(:compactor_options) || {}
  }
  @transformer = {
    klass: params.delete(:transformer) || LLM::Transformer::Null,
    options: params.delete(:transformer_options) || {}
  }
  @guard = {
    klass: params.delete(:guard) || LLM::Guard::Null,
    options: params.delete(:guard_options) || {}
  }
  @retry_budget = params.delete(:retry_budget) || 0
end

Instance Attribute Details

#compacted ⇒ Boolean Also known as: compacted?

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns whether the context has been compacted and no later model response has cleared that state.

Returns:

  • (Boolean) 


153
154
155
 
# File 'lib/llm/context.rb', line 153

def compacted
  @compacted
end

#llm ⇒ LLM::Provider (readonly)

Returns a provider

Returns:



60
61
62
 
# File 'lib/llm/context.rb', line 60

def llm
  @llm
end

#messages ⇒ LLM::Buffer<LLM::Message> (readonly)

Returns the accumulated message history for this context

Returns:



55
56
57
 
# File 'lib/llm/context.rb', line 55

def messages
  @messages
end

#mode ⇒ Symbol (readonly)

Returns the context mode

Returns:

  • (Symbol) 


65
66
67
 
# File 'lib/llm/context.rb', line 65

def mode
  @mode
end

#record ⇒ Object? (readonly)

Returns the ORM record this context is bound to, or nil.

Returns:



70
71
72
 
# File 'lib/llm/context.rb', line 70

def record
  @record
end

Class Method Details

.params ⇒ Array<Symbol>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the set of runtime parameters that configure this context and must never be forwarded to a provider request body.

Returns:

  • (Array<Symbol>) 


48
49
50
 
# File 'lib/llm/context.rb', line 48

def self.params
  %w[guard retry_budget concurrency transformer compactor record]
end

Instance Method Details

#ask(prompt, options = {}) {|String| ... } ⇒ LLM::Response

Ask a question and return the content string directly. Accepts with: for file attachments and a block for streaming. This interface is compatible with RubyLLM's ask method.

Parameters:

  • prompt (String)
  • options (Hash) (defaults to: {})

Options Hash (options):

  • :with (String, Array<String>, nil) —

    File path(s) to attach

  • :stream (#<<, LLM::Stream, nil) —

    A stream target

Yields:

  • (String) —

    content chunks when streaming

Returns:



228
229
230
231
232
233
234
235
236
237
238
239
240
 
# File 'lib/llm/context.rb', line 228

def ask(prompt, options = {}, &block)
  options = {with: nil, stream: nil}.merge!(options || {})
  with, stream = options.values_at(:with, :stream)
  prompt = with ? [prompt, [*with].map { local_file(_1) }] : prompt
  target = if block
    blk = block.dup
    blk.singleton_class.alias_method(:<<, :call)
    blk
  else
    stream
  end
  target ? talk(prompt, stream: target) : talk(prompt)
end

#compactor ⇒ LLM::Compactor

Returns a context compactor

Returns:



144
145
146
 
# File 'lib/llm/context.rb', line 144

def compactor
  @compactor[:klass]
end

#context_usage ⇒ Rational?

Returns the fraction of the context window currently used. For example: Rational(100, 10_000), or nil when unknown

Returns:

  • (Rational, nil) —

    Returns the fraction of the context window currently used. For example: Rational(100, 10_000), or nil when unknown



377
378
379
380
381
382
383
384
385
 
# File 'lib/llm/context.rb', line 377

def context_usage
  return nil if @messages.size < 2
  used = context_used
  return nil if used.nil?
  total = context_window
  total.nil? || total <= 0 ? nil : Rational(used, total)
rescue LLM::NoSuchModelError, LLM::NoSuchRegistryError
  nil
end

#context_used ⇒ Integer?

Returns the live context size (in tokens) of the most recent assistant message.

Returns:

  • (Integer, nil) —

    Returns the live context size (in tokens) of the most recent assistant message.



366
367
368
369
370
371
 
# File 'lib/llm/context.rb', line 366

def context_used
  @messages
    .find(&:assistant?)
    &.token_usage
    &.total_tokens
end

#context_window ⇒ Integer?

Note:

This method returns nil when the context window size is not known to the runtime

Returns the model's context window. The context window is the maximum amount of input and output tokens a model can consider in a single request.

Returns:

  • (Integer, nil) 


395
396
397
398
399
400
401
 
# File 'lib/llm/context.rb', line 395

def context_window
  registry
    .limit(model:)
    .context
rescue LLM::NoSuchModelError, LLM::NoSuchRegistryError
  nil
end

#cost ⇒ LLM::Cost

Returns an approximate cost for a given context based on both the provider, and model

Returns:

  • (LLM::Cost) —

    Returns an approximate cost for a given context based on both the provider, and model



518
519
520
 
# File 'lib/llm/context.rb', line 518

def cost
  LLM::Cost.from(self)
end

#guard ⇒ Class<LLM::Guard>

Returns the configured guard class.

Guards are context-level supervisors for agentic execution. A guard can inspect the runtime state and decide whether pending tool work should be blocked before the context keeps looping.

The guard is stamped onto the functions the context binds, so it runs whenever a task is spawned — including tool calls queued from a stream via Stream#on_tool_call. A blocked call yields its in-band guard_error return without executing.

The built-in implementation is LLM::Guard::Loop, which detects repeated tool-call patterns and turns them into in-band guard_error tool returns.

Returns:



173
174
175
 
# File 'lib/llm/context.rb', line 173

def guard
  @guard[:klass]
end

#image_url(url) ⇒ LLM::Object

Recongize an object as a URL to an image

Parameters:

  • url (String) —

    The URL

Returns:



429
430
431
 
# File 'lib/llm/context.rb', line 429

def image_url(url)
  LLM::Object.from(value: url, kind: :image_url)
end

#inspect ⇒ String

Returns:

  • (String) 


244
245
246
247
248
 
# File 'lib/llm/context.rb', line 244

def inspect
  "#<#{LLM::Utils.object_id(self)} " \
  "@llm=#{@llm.class}, @mode=#{@mode.inspect}, @params=#{@params.inspect}, " \
  "@messages=#{@messages.inspect}>"
end

#interrupt! ⇒ nil Also known as: cancel!

Interrupt the active request, if any. This is inspired by Go's context cancellation model.

Returns:

  • (nil) 


340
341
342
343
344
345
346
347
 
# File 'lib/llm/context.rb', line 340

def interrupt!
  llm.interrupt!(@owner)
  queue&.interrupt!
  pending_functions.each(&:interrupt!)
  @queue = nil
  @owner = nil
  nil
end

#local_file(path) ⇒ LLM::Object

Recongize an object as a local file

Parameters:

  • path (String) —

    The path

Returns:



439
440
441
 
# File 'lib/llm/context.rb', line 439

def local_file(path)
  LLM::Object.from(value: LLM.File(path), kind: :local_file)
end

#model ⇒ String

Returns the model a Context is actively using

Returns:

  • (String) 


478
479
480
 
# File 'lib/llm/context.rb', line 478

def model
  messages.find(&:assistant?)&.model || @params[:model]
end

#params ⇒ Hash

Returns the default params for this context

Returns:

  • (Hash) 


137
138
139
 
# File 'lib/llm/context.rb', line 137

def params
  @params.dup
end

#pending_functions ⇒ Array<LLM::Function>

Returns an array of functions that can be called

Returns:



253
254
255
256
257
258
259
260
261
262
263
264
265
266
 
# File 'lib/llm/context.rb', line 253

def pending_functions
  return_ids = returns.map(&:id)
  guard = @guard[:klass].new(self)
  @messages
    .select(&:assistant?)
    .flat_map do |msg|
      fns = msg.functions.select { _1.pending? && !return_ids.include?(_1.id) }
      fns.each do |fn|
        fn.tracer = tracer
        fn.model  = msg.model
        fn.guard  = guard
      end
    end.extend(LLM::Function::Array)
end

#pending_functions? ⇒ Boolean

Returns whether there is pending tool work in this context. This prefers queued streamed tool work when present, and otherwise falls back to unresolved functions derived from the message history.

Returns:

  • (Boolean) 


273
274
275
276
 
# File 'lib/llm/context.rb', line 273

def pending_functions?
  pending = queue
  (pending && !pending.empty?) || pending_functions.any?
end

#prompt(&b) ⇒ LLM::Prompt Also known as: build_prompt

Build a role-aware prompt for a single request.

Prefer this method over #build_prompt. The older method name is kept for backward compatibility.

Examples:

prompt = ctx.prompt do
  system "Your task is to assist the user"
  user "Hello, can you assist me?"
end
ctx.talk(prompt)

Parameters:

  • b (Proc) —

    A block that composes messages. If it takes one argument, it receives the prompt object. Otherwise it runs in prompt context.

Returns:



418
419
420
 
# File 'lib/llm/context.rb', line 418

def prompt(&b)
  LLM::Prompt.new(@llm, &b)
end

#registry ⇒ LLM::Registry

Returns:

See Also:



525
526
527
 
# File 'lib/llm/context.rb', line 525

def registry
  llm.registry
end

#remote_file(res) ⇒ LLM::Object

Reconginize an object as a remote file

Parameters:

Returns:



449
450
451
 
# File 'lib/llm/context.rb', line 449

def remote_file(res)
  LLM::Object.from(value: res, kind: :remote_file)
end

#retry_budget ⇒ Integer

Returns the retry budget for rate-limited requests.

Returns:

  • (Integer) 


130
131
132
 
# File 'lib/llm/context.rb', line 130

def retry_budget
  @retry_budget
end

#returns ⇒ Array<LLM::Function::Return>

Returns tool returns accumulated in this context

Returns:



291
292
293
294
295
296
297
298
299
 
# File 'lib/llm/context.rb', line 291

def returns
  @messages
    .select(&:tool_return?)
    .flat_map do |msg|
      LLM::Function::Return === msg.content ?
        [msg.content] :
        [*msg.content].grep(LLM::Function::Return)
    end
end

#serialize(path:) ⇒ void Also known as: save

This method returns an undefined value.

Save the current context state

Examples:

llm = LLM.openai(key: ENV["KEY"])
ctx = LLM::Context.new(llm)
ctx.talk "Hello"
ctx.save(path: "context.json")

Raises:

  • (SystemCallError) —

    Might raise a number of SystemCallError subclasses



509
510
511
 
# File 'lib/llm/context.rb', line 509

def serialize(path:)
  ::File.binwrite path, LLM.json.dump(to_h)
end

#spawn(function, strategy) ⇒ LLM::Function::Task

Spawns a function through the context.

Parameters:

Returns:



284
285
286
 
# File 'lib/llm/context.rb', line 284

def spawn(function, strategy)
  function.task(strategy)
end

#stream ⇒ LLM::Stream, ...

Returns a stream object, or nil

Returns:

  • (LLM::Stream, #<<, nil) —

    Returns a stream object, or nil



471
472
473
 
# File 'lib/llm/context.rb', line 471

def stream
  @stream || @params[:stream]
end

#talk(prompt, params = {}) ⇒ LLM::Response

Interact with the context via the chat completions API. This method immediately sends a request to the LLM and returns the response.

Examples:

llm = LLM.openai(key: ENV["KEY"])
ctx = LLM::Context.new(llm)
res = ctx.talk("Hello, what is your name?")
puts res.messages[0].content

Parameters:

  • params (defaults to: {}) —

    The params, including optional :role (defaults to :user), :stream, :tools, :schema etc.

  • prompt (String) —

    The input prompt to be completed

Returns:



199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
 
# File 'lib/llm/context.rb', line 199

def talk(prompt, params = {})
  @owner = @llm.request_owner
  @compactor[:klass].new(self).call(**@compactor[:options])
  repair!(@messages, prompt)
  prompt, params, res = try { mode == :responses ? respond(prompt, params) : complete(prompt, params) }
  self.compacted = false
  if prompt.all?(&:tool_return?)
    @messages.concat prompt.map { LLM::Message.new(@llm.tool_role, _1.content, _1.extra) }
  else
    @messages.concat(prompt)
  end
  @messages.concat([res.choices[-1]].compact)
  res
ensure
  @owner = nil
end

#to_h ⇒ Hash

Returns:

  • (Hash) 


484
485
486
487
488
489
490
491
 
# File 'lib/llm/context.rb', line 484

def to_h
  {
    schema_version: 1,
    model:,
    compacted:,
    messages: @messages.map { serialize_message(_1) }
  }
end

#to_json ⇒ String

Returns:

  • (String) 


495
496
497
 
# File 'lib/llm/context.rb', line 495

def to_json(...)
  LLM.json.dump(to_h, ...)
end

#token_usage ⇒ LLM::Usage Also known as: usage

Returns token usage accumulated in this context

Returns:



353
354
355
356
357
358
359
 
# File 'lib/llm/context.rb', line 353

def token_usage
  @messages
    .select(&:assistant?)
    .map(&:token_usage)
    .compact
    .reduce(LLM::Usage.zero, :+)
end

#tracer ⇒ LLM::Tracer

Returns an LLM tracer

Returns:



456
457
458
 
# File 'lib/llm/context.rb', line 456

def tracer
  @llm.tracer
end

#tracer=(other) ⇒ void

This method returns an undefined value.

Parameters:



464
465
466
 
# File 'lib/llm/context.rb', line 464

def tracer=(other)
  @llm.tracer = other || LLM::Tracer::Null.new(@llm)
end

#transformer ⇒ Class<LLM::Transformer>

Returns the configured transformer class.

Transformers rewrite the most recent message before it is sent to the provider.

Returns:



184
185
186
 
# File 'lib/llm/context.rb', line 184

def transformer
  @transformer[:klass]
end

#wait(strategy, except: []) ⇒ Array<LLM::Function::Return>

Waits for queued tool work to finish.

This prefers queued streamed tool work when the configured stream exposes a non-empty queue. Otherwise it falls back to waiting on the context's pending functions directly.

Parameters:

  • strategy (Symbol, Array<Symbol>) —

    If the stream queue already has tool work, wait will drain it without using this argument. Otherwise, this controls how pending functions are resolved directly. Use :sequential for sequential execution without spawning.

  • except (Array<LLM::Function>) (defaults to: []) —

    A list of functions to exclude from the wait

Returns:



316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
 
# File 'lib/llm/context.rb', line 316

def wait(strategy, except: [])
  if stream.queue.empty?
    ##
    # Every pending function is spawned as a task that checks its own
    # guard (stamped on the function) before running. Blocked tasks
    # yield their guard's return, so all pending calls still close.
    tools = except.empty? ? pending_functions : pending_functions - except
    @queue = tools.task(strategy)
    returns = @queue.wait
    emit_tool_returns(tools, returns)
    returns
  else
    @queue = stream.queue
    @queue.wait
  end
ensure
  @queue = nil
  @stream = nil
end