Class: LLM::Provider Abstract

Inherits:
Object
  • Object
show all
Includes:
Transport::Execution
Defined in:
lib/llm/provider.rb

Overview

This class is abstract.

The Provider class is the abstract base for LLM service integrations. Most users interact with providers through Agent or Context rather than calling #complete directly.

Direct Known Subclasses

Anthropic, Bedrock, Google, Ollama, OpenAI

Instance Method Summary collapse

Constructor Details

#initialize(key:, host:, port: 443, timeout: 900, ssl: true, base_path: "", persistent: false, transport: nil) ⇒ Provider

Returns a new instance of Provider.

Parameters:

  • key (String, nil) —

    The secret key for authentication

  • host (String) —

    The host address of the LLM provider

  • port (Integer) (defaults to: 443) —

    The port number

  • timeout (Integer) (defaults to: 900) —

    The number of seconds to wait for a response

  • ssl (Boolean) (defaults to: true) —

    Whether to use SSL for the connection

  • base_path (String) (defaults to: "") —

    Optional base path prefix for HTTP API routes.

  • persistent (Boolean) (defaults to: false) —

    Whether to use a persistent connection. Requires the net-http-persistent gem.

  • transport (LLM::Transport, Class, nil) (defaults to: nil) —

    Optional override with any Transport instance or subclass.



30
31
32
33
34
35
36
37
38
39
40
41
 
# File 'lib/llm/provider.rb', line 30

def initialize(key:, host:, port: 443, timeout: 900, ssl: true, base_path: "", persistent: false, transport: nil)
  @key = key
  @host = host
  @port = port
  @timeout = timeout
  @ssl = ssl
  @base_path = LLM::Utils.normalize_base_path(base_path)
  @base_uri = URI("#{ssl ? "https" : "http"}://#{host}:#{port}/")
  @headers = {"User-Agent" => "llm.rb v#{LLM::VERSION}"}
  @transport = LLM::Transport::Utils.resolve_transport(host:, port:, timeout:, ssl:, transport:, persistent:)
  @monitor = Monitor.new
end

Instance Method Details

#adapt_function(fn) ⇒ Hash

This method is abstract.

Adapt a Function to the provider-specific tool schema.

Parameters:

Returns:

  • (Hash)

Raises:

  • (NotImplementedError) 


404
405
406
 
# File 'lib/llm/provider.rb', line 404

def adapt_function(fn)
  raise NotImplementedError
end

#assistant_role ⇒ String

Returns the role of the assistant in the conversation. Usually "assistant" or "model"

Returns:

  • (String) —

    Returns the role of the assistant in the conversation. Usually "assistant" or "model"

Raises:

  • (NotImplementedError) 


232
233
234
 
# File 'lib/llm/provider.rb', line 232

def assistant_role
  raise NotImplementedError
end

#audio ⇒ LLM::OpenAI::Audio

Returns an interface to the audio API

Returns:

Raises:

  • (NotImplementedError) 


196
197
198
 
# File 'lib/llm/provider.rb', line 196

def audio
  raise NotImplementedError
end

#build_messages(prompt, params, role, key: :messages) ⇒ Array<LLM::Message>

Builds the outgoing message array for a turn. Normalizes the prompt into one or more Message objects and prepends the existing history.

The method is idempotent. If the prompt is already an Message or an array of Messages (ie it was built by a previous call and possibly transformed), it is returned as-is without rebuilding.

Parameters:

  • prompt (String, Array, LLM::Message, LLM::Prompt)
  • params (Hash) —

    Turn params. The history is taken from params[:messages].

  • role (Symbol) —

    The role to assign to a raw prompt

  • key (Symbol) (defaults to: :messages) —

    The params key that holds the history (:messages for chat completions, :input for the responses API).

Returns:



62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
 
# File 'lib/llm/provider.rb', line 62

def build_messages(prompt, params, role, key: :messages)
  case prompt
  when LLM::Message
    [prompt]
  when Array
    if prompt.all? { LLM::Message === _1 }
      prompt
    else
      [*(params.delete(key) || []), LLM::Message.new(role, prompt)]
    end
  when LLM::Prompt
    [*(params.delete(key) || []), *prompt.to_a]
  else
    [*(params.delete(key) || []), LLM::Message.new(role, prompt)]
  end
end

#chat(prompt, params = {}) ⇒ LLM::Context

Starts a new chat powered by the chat completions API

Parameters:

  • prompt (String) —

    The input prompt to be completed

  • params (Hash) (defaults to: {}) —

    The parameters to maintain throughout the conversation. Any parameter the provider supports can be included and not only those listed here.

Returns:



159
160
161
162
 
# File 'lib/llm/provider.rb', line 159

def chat(prompt, params = {})
  role = params.delete(:role)
  LLM::Context.new(self, params).talk(prompt, role:)
end

#complete(prompt, params = {}) ⇒ LLM::Response

Provides an interface to the chat completions API. Most users should use Context#talk or Agent#talk instead.

Examples:

llm = LLM.openai(key: ENV["KEY"])
messages = [{role: "system", content: "Your task is to answer all of my questions"}]
res = llm.complete("5 + 2 ?", messages:)
print "[#{res.messages[0].role}]", res.messages[0].content, "\n"

Parameters:

  • prompt (String) —

    The input prompt to be completed

  • params (Hash) (defaults to: {}) —

    The parameters to maintain throughout the conversation. Any parameter the provider supports can be included and not only those listed here.

Options Hash (params):

  • :role (Symbol) —

    Defaults to the provider's default role

  • :model (String) —

    Defaults to the provider's default model

  • :schema (#to_json, nil) —

    Defaults to nil

  • :tools (Array<LLM::Function>, nil) —

    Defaults to nil

Returns:

Raises:

  • (NotImplementedError) —

    When the method is not implemented by a subclass



150
151
152
 
# File 'lib/llm/provider.rb', line 150

def complete(prompt, params = {})
  raise NotImplementedError
end

#default_model ⇒ String

Returns the default model for chat completions

Returns:

  • (String) —

    Returns the default model for chat completions

Raises:

  • (NotImplementedError) 


239
240
241
 
# File 'lib/llm/provider.rb', line 239

def default_model
  raise NotImplementedError
end

#developer_role ⇒ Symbol

Returns:

  • (Symbol) 


317
318
319
 
# File 'lib/llm/provider.rb', line 317

def developer_role
  :developer
end

#embed(input, model: nil, **params) ⇒ LLM::Response

Provides an embedding

Parameters:

  • input (String, Array<String>) —

    The input to embed

  • model (String) (defaults to: nil) —

    The embedding model to use

  • params (Hash) —

    Other embedding parameters

Returns:

Raises:

  • (NotImplementedError) —

    When the method is not implemented by a subclass



114
115
116
 
# File 'lib/llm/provider.rb', line 114

def embed(input, model: nil, **params)
  raise NotImplementedError
end

#files ⇒ LLM::OpenAI::Files

Returns an interface to the files API

Returns:

Raises:

  • (NotImplementedError) 


203
204
205
 
# File 'lib/llm/provider.rb', line 203

def files
  raise NotImplementedError
end

#images ⇒ LLM::OpenAI::Images, LLM::Google::Images

Returns an interface to the images API

Returns:

Raises:

  • (NotImplementedError) 


189
190
191
 
# File 'lib/llm/provider.rb', line 189

def images
  raise NotImplementedError
end

#inspect ⇒ String

Note:

The secret key is redacted in inspect for security reasons

Returns an inspection of the provider object

Returns:

  • (String) 


83
84
85
 
# File 'lib/llm/provider.rb', line 83

def inspect
  "#<#{LLM::Utils.object_id(self)} @key=[REDACTED] @transport=#{transport.inspect} @tracer=#{tracer.inspect}>"
end

#interrupt!(owner) ⇒ nil Also known as: cancel!

Interrupt the active request, if any.

Parameters:

  • owner (Fiber)

Returns:

  • (nil) 


378
379
380
 
# File 'lib/llm/provider.rb', line 378

def interrupt!(owner)
  transport.interrupt!(owner)
end

#key? ⇒ Boolean

Returns true when an API key is configured

Returns:

  • (Boolean) —

    Returns true when an API key is configured



394
395
396
 
# File 'lib/llm/provider.rb', line 394

def key?
  @key != nil && @key.to_s.strip.size > 0
end

#models ⇒ LLM::OpenAI::Models

Returns an interface to the models API

Returns:

Raises:

  • (NotImplementedError) 


210
211
212
 
# File 'lib/llm/provider.rb', line 210

def models
  raise NotImplementedError
end

#moderations ⇒ LLM::OpenAI::Moderations

Returns an interface to the moderations API

Returns:

Raises:

  • (NotImplementedError) 


217
218
219
 
# File 'lib/llm/provider.rb', line 217

def moderations
  raise NotImplementedError
end

#name ⇒ Symbol

Returns the provider's name

Returns:

  • (Symbol) —

    Returns the provider's name

Raises:

  • (NotImplementedError) —

    When the method is not implemented by a subclass



92
93
94
 
# File 'lib/llm/provider.rb', line 92

def name
  raise NotImplementedError
end

#ocr ⇒ LLM::Response

Note:

This feature is not implemented by all providers, and it will raise NotImplementedError for providers that do not support it.

Returns:

Raises:

  • (NotImplementedError) 


124
125
126
 
# File 'lib/llm/provider.rb', line 124

def ocr(...)
  raise NotImplementedError
end

#registry ⇒ LLM::Registry

Returns the provider's model registry.

Returns:



99
100
101
 
# File 'lib/llm/provider.rb', line 99

def registry
  LLM.registry_for(self)
end

#request_owner ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the current request owner used by the transport.

Returns:



387
388
389
 
# File 'lib/llm/provider.rb', line 387

def request_owner
  transport.request_owner
end

#respond(prompt, params = {}) ⇒ LLM::Context

Starts a new chat powered by the responses API

Parameters:

  • prompt (String) —

    The input prompt to be completed

  • params (Hash) (defaults to: {}) —

    The parameters to maintain throughout the conversation. Any parameter the provider supports can be included and not only those listed here.

Returns:

Raises:

  • (NotImplementedError) —

    When the method is not implemented by a subclass



170
171
172
173
 
# File 'lib/llm/provider.rb', line 170

def respond(prompt, params = {})
  role = params.delete(:role)
  LLM::Context.new(self, params).respond(prompt, role:)
end

#responses ⇒ LLM::OpenAI::Responses

Note:

Compared to the chat completions API, the responses API can require less bandwidth on each turn, maintain state server-side, and produce faster responses.

Returns:

Raises:

  • (NotImplementedError) 


182
183
184
 
# File 'lib/llm/provider.rb', line 182

def responses
  raise NotImplementedError
end

#schema ⇒ LLM::Schema

Returns an object that can generate a JSON schema

Returns:



246
247
248
 
# File 'lib/llm/provider.rb', line 246

def schema
  LLM::Schema.new
end

#server_tool(name, options = {}) ⇒ LLM::ServerTool

Note:

OpenAI, Anthropic, and Gemini provide platform-tools for things like web search, and more.

Returns a tool provided by a provider.

Examples:

llm   = LLM.openai(key: ENV["KEY"])
tools = [llm.server_tool(:web_search)]
res   = llm.responses.create("Summarize today's news", tools:)
print res.output_text, "\n"

Parameters:

  • name (String, Symbol) —

    The name of the tool

  • options (Hash) (defaults to: {}) —

    Configuration options for the tool

Returns:



289
290
291
 
# File 'lib/llm/provider.rb', line 289

def server_tool(name, options = {})
  LLM::ServerTool.new(name, options, self)
end

#server_tools ⇒ String => LLM::ServerTool

Note:

This method might be outdated, and the LLM::Provider#server_tool method can be used if a tool is not found here.

Returns all known tools provided by a provider.

Returns:



272
273
274
 
# File 'lib/llm/provider.rb', line 272

def server_tools
  {}
end

#system_role ⇒ Symbol

Returns:

  • (Symbol) 


311
312
313
 
# File 'lib/llm/provider.rb', line 311

def system_role
  :system
end

#tool_role ⇒ Symbol

Returns:

  • (Symbol) 


323
324
325
 
# File 'lib/llm/provider.rb', line 323

def tool_role
  :tool
end

#tracer ⇒ LLM::Tracer

Returns the current scoped tracer override or provider default tracer

Returns:

  • (LLM::Tracer) —

    Returns the current scoped tracer override or provider default tracer



330
331
332
 
# File 'lib/llm/provider.rb', line 330

def tracer
  weakmap[self] || @tracer || LLM::Tracer::Null.new(self)
end

#tracer=(tracer) ⇒ void

This method returns an undefined value.

Set the provider's default tracer This tracer is shared by the provider instance and becomes the fallback whenever no scoped override is active.

Examples:

llm = LLM.openai(key: ENV["KEY"])
llm.tracer = LLM::Tracer::Logger.new(llm, path: "/path/to/log.txt")

Parameters:



344
345
346
 
# File 'lib/llm/provider.rb', line 344

def tracer=(tracer)
  @tracer = tracer || LLM::Tracer::Null.new(self)
end

#user_role ⇒ Symbol

Returns:

  • (Symbol) 


305
306
307
 
# File 'lib/llm/provider.rb', line 305

def user_role
  :user
end

#vector_stores ⇒ LLM::OpenAI::VectorStore

Returns an interface to the vector stores API

Returns:

  • (LLM::OpenAI::VectorStore) —

    Returns an interface to the vector stores API

Raises:

  • (NotImplementedError) 


224
225
226
 
# File 'lib/llm/provider.rb', line 224

def vector_stores
  raise NotImplementedError
end

#web_search(query:) ⇒ LLM::Response

Provides a web search capability

Parameters:

  • query (String) —

    The search query

Returns:

Raises:

  • (NotImplementedError) —

    When the method is not implemented by a subclass



299
300
301
 
# File 'lib/llm/provider.rb', line 299

def web_search(query:)
  raise NotImplementedError
end

#with(headers:) ⇒ LLM::Provider

Add one or more headers to all requests

Examples:

llm = LLM.openai(key: ENV["KEY"])
llm.with(headers: {"OpenAI-Organization" => ENV["ORG"]})
llm.with(headers: {"OpenAI-Project" => ENV["PROJECT"]})

Parameters:

  • headers (Hash<String,String>) —

    One or more headers

Returns:



260
261
262
263
264
 
# File 'lib/llm/provider.rb', line 260

def with(headers:)
  lock do
    tap { @headers.merge!(headers) }
  end
end

#with_tracer(tracer) { ... } ⇒ Object

Override the tracer for the current fiber while the block runs. This is useful when you want per-request or per-turn tracing without replacing the provider's default tracer.

Examples:

llm.with_tracer(LLM::Tracer::Logger.new(llm, io: $stdout)) do
  llm.complete("hello", model: "gpt-5.4-mini")
end

Parameters:

Yields:

Returns:



359
360
361
362
363
364
365
366
367
368
369
370
371
372
 
# File 'lib/llm/provider.rb', line 359

def with_tracer(tracer)
  had_override = weakmap.key?(self)
  previous = weakmap[self]
  weakmap[self] = tracer || LLM::Tracer::Null.new(self)
  yield
ensure
  if had_override
    weakmap[self] = previous
  elsif weakmap.respond_to?(:delete)
    weakmap.delete(self)
  else
    weakmap[self] = nil
  end
end