Class: LLM::Context

Inherits:
Object
  • Object
show all
Includes:
Deserializer, Serializer
Defined in:
lib/llm/context.rb,
lib/llm/context/serializer.rb,
lib/llm/context/deserializer.rb

Overview

LLM::Context is the low-level stateful execution boundary in llm.rb. Most users should start with Agent, which wraps Context and manages tool loops automatically. Use Context directly when you need manual control over tool execution.

It holds the evolving runtime state for an LLM workflow: conversation history, tool calls and returns, schema and streaming configuration, accumulated usage, and request ownership for interruption.

This is broader than prompt context alone. A context is the object that lets one-off prompts, streaming turns, tool execution, persistence, retries, and serialized long-lived workflows all run through the same model.

A context can drive the chat completions API that all providers support or the Responses API on providers that expose it.

Examples:

#!/usr/bin/env ruby
require "llm"

llm = LLM.deepseek(key: ENV["KEY"])
ctx = LLM::Context.new(llm, stream: $stdout)
ctx.talk "If a train goes 60 mph for 1.5 hours, how far does it travel?"
ctx.messages.each { |m| puts "[#{m.role}] #{m.content}" }

See Also:

Defined Under Namespace

Modules: Deserializer, Serializer

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from Deserializer

#deserialize

Constructor Details

#initialize(llm, params = {}) ⇒ Context

Returns a new instance of Context.

Parameters:

  • llm (LLM::Provider) —

    A provider

  • params (Hash) (defaults to: {}) —

    The parameters to maintain throughout the conversation. Any parameter the provider supports can be included and not only those listed here.

Options Hash (params):

  • :mode (Symbol) —

    Defaults to :responses for OpenAI, otherwise it defaults to :completions.

  • :model (String) —

    Defaults to the provider's default model

  • :compactor (Class<LLM::Compactor>, nil) —

    A compactor class to use for context compaction. Defaults to LLM::Compactor::Null.

  • :compactor_options (Hash) —

    Options passed to the compactor's call method. Defaults to {}.

  • :tools (Array<LLM::Function>, nil) —

    Defaults to nil

  • :skills (Array<String>, nil) —

    Defaults to nil



88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
 
# File 'lib/llm/context.rb', line 88

def initialize(llm, params = {})
  @llm = llm
  @mode = params.delete(:mode) || (llm.name == :openai ? :responses : :completions)
  @guard = params.delete(:guard)
  @transformer = params.delete(:transformer)
  tools = [*params.delete(:tools), *load_skills(params.delete(:skills))]
  @params = {model: llm.default_model, schema: nil}.compact.merge!(params)
  @params[:tools] = tools unless tools.empty?
  @params[:store] ||= false if @mode == :responses
  @messages = LLM::Buffer.new(llm)
  extra = @params.slice(:model, :tools).merge!(ctx: self, tracer:)
  @params[:stream] = LLM::Stream.try(@params[:stream], extra:)
  @compactor = {
    klass: params.delete(:compactor) || LLM::Compactor::Null,
    options: params.delete(:compactor_options) || {}
  }
end

Instance Attribute Details

#compacted ⇒ Boolean Also known as: compacted?

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns whether the context has been compacted and no later model response has cleared that state.

Returns:

  • (Boolean) 


125
126
127
 
# File 'lib/llm/context.rb', line 125

def compacted
  @compacted
end

#llm ⇒ LLM::Provider (readonly)

Returns a provider

Returns:



63
64
65
 
# File 'lib/llm/context.rb', line 63

def llm
  @llm
end

#messages ⇒ LLM::Buffer<LLM::Message> (readonly)

Returns the accumulated message history for this context

Returns:



58
59
60
 
# File 'lib/llm/context.rb', line 58

def messages
  @messages
end

#mode ⇒ Symbol (readonly)

Returns the context mode

Returns:

  • (Symbol) 


68
69
70
 
# File 'lib/llm/context.rb', line 68

def mode
  @mode
end

Instance Method Details

#ask(prompt, options = {}) {|String| ... } ⇒ LLM::Response

Ask a question and return the content string directly. Accepts with: for file attachments and a block for streaming. This interface is compatible with RubyLLM's ask method.

Parameters:

  • prompt (String)
  • options (Hash) (defaults to: {})

Options Hash (options):

  • :with (String, Array<String>, nil) —

    File path(s) to attach

  • :stream (#<<, LLM::Stream, nil) —

    A stream target

Yields:

  • (String) —

    content chunks when streaming

Returns:



221
222
223
224
225
226
227
228
229
230
231
232
233
 
# File 'lib/llm/context.rb', line 221

def ask(prompt, options = {}, &block)
  options = {with: nil, stream: nil}.merge!(options || {})
  with, stream = options.values_at(:with, :stream)
  prompt = with ? [prompt, [*with].map { local_file(_1) }] : prompt
  target = if block
    blk = block.dup
    blk.singleton_class.alias_method(:<<, :call)
    blk
  else
    stream
  end
  target ? talk(prompt, stream: target) : talk(prompt)
end

#compactor ⇒ LLM::Compactor

Returns a context compactor

Returns:



116
117
118
 
# File 'lib/llm/context.rb', line 116

def compactor
  @compactor[:klass]
end

#context_window ⇒ Integer

Note:

This method returns 0 when the provider or model can't be found within Registry.

Returns the model's context window. The context window is the maximum amount of input and output tokens a model can consider in a single request.

Returns:

  • (Integer) 


491
492
493
494
495
496
497
498
 
# File 'lib/llm/context.rb', line 491

def context_window
  LLM
    .registry_for(llm)
    .limit(model:)
    .context
rescue LLM::NoSuchModelError, LLM::NoSuchRegistryError
  0
end

#cost ⇒ LLM::Cost

Returns an approximate cost for a given context based on both the provider, and model

Returns:

  • (LLM::Cost) —

    Returns an approximate cost for a given context based on both the provider, and model



479
480
481
 
# File 'lib/llm/context.rb', line 479

def cost
  LLM::Cost.from(self)
end

#guard ⇒ #call?

Returns a guard, if configured.

Guards are context-level supervisors for agentic execution. A guard can inspect the runtime state and decide whether pending tool work should be blocked before the context keeps looping.

The built-in implementation is LLM::LoopGuard, which detects repeated tool-call patterns and turns them into in-band LLM::GuardError tool returns.

Returns:

  • (#call, nil) 


140
141
142
143
144
145
 
# File 'lib/llm/context.rb', line 140

def guard
  return if @guard.nil? || @guard == false
  @guard = LLM::LoopGuard.new if @guard == true
  @guard = LLM::LoopGuard.new(@guard) if Hash === @guard
  @guard
end

#guard=(guard) ⇒ #call, ...

Sets a guard or guard config.

Guards must implement call(ctx) and return either nil or a warning string. Returning a warning tells the context to block pending tool work with guarded tool errors instead of continuing the loop.

Parameters:

  • guard (#call, Hash, Boolean, nil)

Returns:

  • (#call, Hash, Boolean, nil) 


156
157
158
 
# File 'lib/llm/context.rb', line 156

def guard=(guard)
  @guard = guard
end

#image_url(url) ⇒ LLM::Object

Recongize an object as a URL to an image

Parameters:

  • url (String) —

    The URL

Returns:



390
391
392
 
# File 'lib/llm/context.rb', line 390

def image_url(url)
  LLM::Object.from(value: url, kind: :image_url)
end

#inspect ⇒ String

Returns:

  • (String) 


237
238
239
240
241
 
# File 'lib/llm/context.rb', line 237

def inspect
  "#<#{LLM::Utils.object_id(self)} " \
  "@llm=#{@llm.class}, @mode=#{@mode.inspect}, @params=#{@params.inspect}, " \
  "@messages=#{@messages.inspect}>"
end

#interrupt! ⇒ nil Also known as: cancel!

Interrupt the active request, if any. This is inspired by Go's context cancellation model.

Returns:

  • (nil) 


333
334
335
336
337
338
339
340
 
# File 'lib/llm/context.rb', line 333

def interrupt!
  llm.interrupt!(@owner)
  queue&.interrupt!
  pending_functions.each(&:interrupt!)
  @queue = nil
  @owner = nil
  nil
end

#local_file(path) ⇒ LLM::Object

Recongize an object as a local file

Parameters:

  • path (String) —

    The path

Returns:



400
401
402
 
# File 'lib/llm/context.rb', line 400

def local_file(path)
  LLM::Object.from(value: LLM.File(path), kind: :local_file)
end

#model ⇒ String

Returns the model a Context is actively using

Returns:

  • (String) 


439
440
441
 
# File 'lib/llm/context.rb', line 439

def model
  messages.find(&:assistant?)&.model || @params[:model]
end

#params ⇒ Hash

Returns the default params for this context

Returns:

  • (Hash) 


109
110
111
 
# File 'lib/llm/context.rb', line 109

def params
  @params.dup
end

#pending_functions ⇒ Array<LLM::Function>

Returns an array of functions that can be called

Returns:



246
247
248
249
250
251
252
253
254
255
256
257
 
# File 'lib/llm/context.rb', line 246

def pending_functions
  return_ids = returns.map(&:id)
  @messages
    .select(&:assistant?)
    .flat_map do |msg|
      fns = msg.functions.select { _1.pending? && !return_ids.include?(_1.id) }
      fns.each do |fn|
        fn.tracer = tracer
        fn.model  = msg.model
      end
    end.extend(LLM::Function::Array)
end

#pending_functions? ⇒ Boolean

Returns whether there is pending tool work in this context. This prefers queued streamed tool work when present, and otherwise falls back to unresolved functions derived from the message history.

Returns:

  • (Boolean) 


263
264
265
266
 
# File 'lib/llm/context.rb', line 263

def pending_functions?
  pending = queue
  (pending && !pending.empty?) || pending_functions.any?
end

#prompt(&b) ⇒ LLM::Prompt Also known as: build_prompt

Build a role-aware prompt for a single request.

Prefer this method over #build_prompt. The older method name is kept for backward compatibility.

Examples:

prompt = ctx.prompt do
  system "Your task is to assist the user"
  user "Hello, can you assist me?"
end
ctx.talk(prompt)

Parameters:

  • b (Proc) —

    A block that composes messages. If it takes one argument, it receives the prompt object. Otherwise it runs in prompt context.

Returns:



379
380
381
 
# File 'lib/llm/context.rb', line 379

def prompt(&b)
  LLM::Prompt.new(@llm, &b)
end

#remote_file(res) ⇒ LLM::Object

Reconginize an object as a remote file

Parameters:

Returns:



410
411
412
 
# File 'lib/llm/context.rb', line 410

def remote_file(res)
  LLM::Object.from(value: res, kind: :remote_file)
end

#returns ⇒ Array<LLM::Function::Return>

Returns tool returns accumulated in this context

Returns:



286
287
288
289
290
291
292
293
294
 
# File 'lib/llm/context.rb', line 286

def returns
  @messages
    .select(&:tool_return?)
    .flat_map do |msg|
      LLM::Function::Return === msg.content ?
        [msg.content] :
        [*msg.content].grep(LLM::Function::Return)
    end
end

#serialize(path:) ⇒ void Also known as: save

This method returns an undefined value.

Save the current context state

Examples:

llm = LLM.openai(key: ENV["KEY"])
ctx = LLM::Context.new(llm)
ctx.talk "Hello"
ctx.save(path: "context.json")

Raises:

  • (SystemCallError) —

    Might raise a number of SystemCallError subclasses



470
471
472
 
# File 'lib/llm/context.rb', line 470

def serialize(path:)
  ::File.binwrite path, LLM.json.dump(to_h)
end

#spawn(function, strategy) ⇒ LLM::Function::Return, LLM::Function::Task

Spawns a function through the context.

When a guard is configured, this method can return an in-band guarded tool error instead of spawning work.

Parameters:

Returns:



277
278
279
280
281
 
# File 'lib/llm/context.rb', line 277

def spawn(function, strategy)
  warning = guard&.call(self)
  return guarded_return_for(function, warning) if warning
  function.task(strategy)
end

#stream ⇒ LLM::Stream, ...

Returns a stream object, or nil

Returns:

  • (LLM::Stream, #<<, nil) —

    Returns a stream object, or nil



432
433
434
 
# File 'lib/llm/context.rb', line 432

def stream
  @stream || @params[:stream]
end

#talk(prompt, params = {}) ⇒ LLM::Response

Interact with the context via the chat completions API. This method immediately sends a request to the LLM and returns the response.

Examples:

llm = LLM.openai(key: ENV["KEY"])
ctx = LLM::Context.new(llm)
res = ctx.talk("Hello, what is your name?")
puts res.messages[0].content

Parameters:

  • params (defaults to: {}) —

    The params, including optional :role (defaults to :user), :stream, :tools, :schema etc.

  • prompt (String) —

    The input prompt to be completed

Returns:



194
195
196
197
198
199
200
201
202
203
204
205
206
207
 
# File 'lib/llm/context.rb', line 194

def talk(prompt, params = {})
  @owner = @llm.request_owner
  @compactor[:klass].new(self).call(**@compactor[:options])
  repair!(@messages, prompt)
  prompt, params, res = mode == :responses ? respond(prompt, params) : complete(prompt, params)
  self.compacted = false
  role = params[:role] || @llm.user_role
  role = @llm.tool_role if params[:role].nil? && [*prompt].grep(LLM::Function::Return).any?
  @messages.concat LLM::Prompt === prompt ? prompt.to_a : [LLM::Message.new(role, prompt)]
  @messages.concat [res.choices[-1]].compact
  res
ensure
  @owner = nil
end

#to_h ⇒ Hash

Returns:

  • (Hash) 


445
446
447
448
449
450
451
452
 
# File 'lib/llm/context.rb', line 445

def to_h
  {
    schema_version: 1,
    model:,
    compacted:,
    messages: @messages.map { serialize_message(_1) }
  }
end

#to_json ⇒ String

Returns:

  • (String) 


456
457
458
 
# File 'lib/llm/context.rb', line 456

def to_json(...)
  LLM.json.dump(to_h, ...)
end

#tracer ⇒ LLM::Tracer

Returns an LLM tracer

Returns:



417
418
419
 
# File 'lib/llm/context.rb', line 417

def tracer
  @llm.tracer
end

#tracer=(other) ⇒ void

This method returns an undefined value.

Parameters:



425
426
427
 
# File 'lib/llm/context.rb', line 425

def tracer=(other)
  @llm.tracer = other || LLM::Tracer::Null.new(@llm)
end

#transformer ⇒ #call?

Returns a transformer, if configured.

Transformers can rewrite outgoing prompts and params before a request is sent to the provider.

Returns:

  • (#call, nil) 


167
168
169
 
# File 'lib/llm/context.rb', line 167

def transformer
  @transformer
end

#transformer=(transformer) ⇒ #call?

Sets a transformer.

Transformers must implement call(ctx, prompt, params) and return a two-element array of [prompt, params].

Parameters:

  • transformer (#call, nil)

Returns:

  • (#call, nil) 


179
180
181
 
# File 'lib/llm/context.rb', line 179

def transformer=(transformer)
  @transformer = transformer
end

#usage ⇒ LLM::Object

Returns token usage accumulated in this context

Returns:



346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
 
# File 'lib/llm/context.rb', line 346

def usage
  if usage = @messages.find(&:assistant?)&.usage
    LLM::Object.from(
      input_tokens: usage.input_tokens || 0,
      output_tokens: usage.output_tokens || 0,
      reasoning_tokens: usage.reasoning_tokens || 0,
      input_audio_tokens: usage.input_audio_tokens || 0,
      output_audio_tokens: usage.output_audio_tokens || 0,
      input_image_tokens: usage.input_image_tokens || 0,
      cache_read_tokens: usage.cache_read_tokens || 0,
      cache_write_tokens: usage.cache_write_tokens || 0,
      total_tokens: usage.total_tokens || 0
    )
  else
    ZERO_USAGE
  end
end

#wait(strategy, except: []) ⇒ Array<LLM::Function::Return>

Waits for queued tool work to finish.

This prefers queued streamed tool work when the configured stream exposes a non-empty queue. Otherwise it falls back to waiting on the context's pending functions directly.

Parameters:

  • strategy (Symbol, Array<Symbol>) —

    If the stream queue already has tool work, wait will drain it without using this argument. Otherwise, this controls how pending functions are resolved directly. Use :sequential for sequential execution without spawning.

  • except (Array<LLM::Function>) (defaults to: []) —

    A list of functions to exclude from the wait

Returns:



311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'lib/llm/context.rb', line 311

def wait(strategy, except: [])
  if stream.queue.empty?
    tools  = except.empty? ? pending_functions : pending_functions - except
    guards = guarded_returns(tools:)
    return guards if guards
    @queue = tools.task(strategy)
    returns = @queue.wait
    emit_tool_returns(tools, returns)
    returns
  else
    @queue = stream.queue
    @queue.wait
  end
ensure
  @queue = nil
  @stream = nil
end