Class: LLM::Provider Abstract

Inherits:
Object
  • Object
show all
Includes:
Transport::HTTP::Execution
Defined in:
lib/llm/provider.rb,
lib/llm/provider/transport/http.rb,
lib/llm/provider/transport/http/interruptible.rb

Overview

This class is abstract.

The Provider class represents an abstract class for LLM (Language Model) providers.

Direct Known Subclasses

Anthropic, Google, Ollama, OpenAI

Defined Under Namespace

Modules: Transport

Instance Method Summary collapse

Constructor Details

#initialize(key:, host:, port: 443, timeout: 60, ssl: true, base_path: "", persistent: false) ⇒ Provider

Returns a new instance of Provider.

Parameters:

  • key (String, nil) —

    The secret key for authentication

  • host (String) —

    The host address of the LLM provider

  • port (Integer) (defaults to: 443) —

    The port number

  • timeout (Integer) (defaults to: 60) —

    The number of seconds to wait for a response

  • ssl (Boolean) (defaults to: true) —

    Whether to use SSL for the connection

  • base_path (String) (defaults to: "") —

    Optional base path prefix for HTTP API routes.

  • persistent (Boolean) (defaults to: false) —

    Whether to use a persistent connection. Requires the net-http-persistent gem.



30
31
32
33
34
35
36
37
38
39
40
41
 
# File 'lib/llm/provider.rb', line 30

def initialize(key:, host:, port: 443, timeout: 60, ssl: true, base_path: "", persistent: false)
  @key = key
  @host = host
  @port = port
  @timeout = timeout
  @ssl = ssl
  @base_path = normalize_base_path(base_path)
  @base_uri = URI("#{ssl ? "https" : "http"}://#{host}:#{port}/")
  @headers = {"User-Agent" => "llm.rb v#{LLM::VERSION}"}
  @transport = Transport::HTTP.new(host:, port:, timeout:, ssl:, persistent:)
  @monitor = Monitor.new
end

Instance Method Details

#assistant_role ⇒ String

Returns the role of the assistant in the conversation. Usually “assistant” or “model”

Returns:

  • (String) —

    Returns the role of the assistant in the conversation. Usually “assistant” or “model”

Raises:

  • (NotImplementedError) 


177
178
179
 
# File 'lib/llm/provider.rb', line 177

def assistant_role
  raise NotImplementedError
end

#audio ⇒ LLM::OpenAI::Audio

Returns an interface to the audio API

Returns:

Raises:

  • (NotImplementedError) 


141
142
143
 
# File 'lib/llm/provider.rb', line 141

def audio
  raise NotImplementedError
end

#chat(prompt, params = {}) ⇒ LLM::Context

Starts a new chat powered by the chat completions API

Parameters:

  • prompt (String) —

    The input prompt to be completed

  • params (Hash) (defaults to: {}) —

    The parameters to maintain throughout the conversation. Any parameter the provider supports can be included and not only those listed here.

Returns:



104
105
106
107
 
# File 'lib/llm/provider.rb', line 104

def chat(prompt, params = {})
  role = params.delete(:role)
  LLM::Context.new(self, params).talk(prompt, role:)
end

#complete(prompt, params = {}) ⇒ LLM::Response

Provides an interface to the chat completions API

Examples:

llm = LLM.openai(key: ENV["KEY"])
messages = [{role: "system", content: "Your task is to answer all of my questions"}]
res = llm.complete("5 + 2 ?", messages:)
print "[#{res.messages[0].role}]", res.messages[0].content, "\n"

Parameters:

  • prompt (String) —

    The input prompt to be completed

  • params (Hash) (defaults to: {}) —

    The parameters to maintain throughout the conversation. Any parameter the provider supports can be included and not only those listed here.

Options Hash (params):

  • :role (Symbol) —

    Defaults to the provider’s default role

  • :model (String) —

    Defaults to the provider’s default model

  • :schema (#to_json, nil) —

    Defaults to nil

  • :tools (Array<LLM::Function>, nil) —

    Defaults to nil

Returns:

Raises:

  • (NotImplementedError) —

    When the method is not implemented by a subclass



95
96
97
 
# File 'lib/llm/provider.rb', line 95

def complete(prompt, params = {})
  raise NotImplementedError
end

#default_model ⇒ String

Returns the default model for chat completions

Returns:

  • (String) —

    Returns the default model for chat completions

Raises:

  • (NotImplementedError) 


184
185
186
 
# File 'lib/llm/provider.rb', line 184

def default_model
  raise NotImplementedError
end

#developer_role ⇒ Symbol

Returns:

  • (Symbol) 


262
263
264
 
# File 'lib/llm/provider.rb', line 262

def developer_role
  :developer
end

#embed(input, model: nil, **params) ⇒ LLM::Response

Provides an embedding

Parameters:

  • input (String, Array<String>) —

    The input to embed

  • model (String) (defaults to: nil) —

    The embedding model to use

  • params (Hash) —

    Other embedding parameters

Returns:

Raises:

  • (NotImplementedError) —

    When the method is not implemented by a subclass



71
72
73
 
# File 'lib/llm/provider.rb', line 71

def embed(input, model: nil, **params)
  raise NotImplementedError
end

#files ⇒ LLM::OpenAI::Files

Returns an interface to the files API

Returns:

Raises:

  • (NotImplementedError) 


148
149
150
 
# File 'lib/llm/provider.rb', line 148

def files
  raise NotImplementedError
end

#images ⇒ LLM::OpenAI::Images, LLM::Google::Images

Returns an interface to the images API

Returns:

Raises:

  • (NotImplementedError) 


134
135
136
 
# File 'lib/llm/provider.rb', line 134

def images
  raise NotImplementedError
end

#inspect ⇒ String

Note:

The secret key is redacted in inspect for security reasons

Returns an inspection of the provider object

Returns:

  • (String) 


47
48
49
 
# File 'lib/llm/provider.rb', line 47

def inspect
  "#<#{self.class.name}:0x#{object_id.to_s(16)} @key=[REDACTED] @transport=#{transport.inspect} @tracer=#{tracer.inspect}>"
end

#interrupt!(owner) ⇒ nil Also known as: cancel!

Interrupt the active request, if any.

Parameters:

  • owner (Fiber)

Returns:

  • (nil) 


322
323
324
 
# File 'lib/llm/provider.rb', line 322

def interrupt!(owner)
  transport.interrupt!(owner)
end

#models ⇒ LLM::OpenAI::Models

Returns an interface to the models API

Returns:

Raises:

  • (NotImplementedError) 


155
156
157
 
# File 'lib/llm/provider.rb', line 155

def models
  raise NotImplementedError
end

#moderations ⇒ LLM::OpenAI::Moderations

Returns an interface to the moderations API

Returns:

Raises:

  • (NotImplementedError) 


162
163
164
 
# File 'lib/llm/provider.rb', line 162

def moderations
  raise NotImplementedError
end

#name ⇒ Symbol

Returns the provider’s name

Returns:

  • (Symbol) —

    Returns the provider’s name

Raises:

  • (NotImplementedError) —

    When the method is not implemented by a subclass



56
57
58
 
# File 'lib/llm/provider.rb', line 56

def name
  raise NotImplementedError
end

#persist! ⇒ LLM::Provider Also known as: persistent

This method configures a provider to use a persistent connection pool via the optional dependency [Net::HTTP::Persistent](github.com/drbrain/net-http-persistent)

Examples:

llm = LLM.openai(key: ENV["KEY"]).persistent
# do something with 'llm'

Returns:



312
313
314
315
 
# File 'lib/llm/provider.rb', line 312

def persist!
  transport.persist!
  self
end

#respond(prompt, params = {}) ⇒ LLM::Context

Starts a new chat powered by the responses API

Parameters:

  • prompt (String) —

    The input prompt to be completed

  • params (Hash) (defaults to: {}) —

    The parameters to maintain throughout the conversation. Any parameter the provider supports can be included and not only those listed here.

Returns:

Raises:

  • (NotImplementedError) —

    When the method is not implemented by a subclass



115
116
117
118
 
# File 'lib/llm/provider.rb', line 115

def respond(prompt, params = {})
  role = params.delete(:role)
  LLM::Context.new(self, params).respond(prompt, role:)
end

#responses ⇒ LLM::OpenAI::Responses

Note:

Compared to the chat completions API, the responses API can require less bandwidth on each turn, maintain state server-side, and produce faster responses.

Returns:

Raises:

  • (NotImplementedError) 


127
128
129
 
# File 'lib/llm/provider.rb', line 127

def responses
  raise NotImplementedError
end

#schema ⇒ LLM::Schema

Returns an object that can generate a JSON schema

Returns:



191
192
193
 
# File 'lib/llm/provider.rb', line 191

def schema
  LLM::Schema.new
end

#server_tool(name, options = {}) ⇒ LLM::ServerTool

Note:

OpenAI, Anthropic, and Gemini provide platform-tools for things like web search, and more.

Returns a tool provided by a provider.

Examples:

llm   = LLM.openai(key: ENV["KEY"])
tools = [llm.server_tool(:web_search)]
res   = llm.responses.create("Summarize today's news", tools:)
print res.output_text, "\n"

Parameters:

  • name (String, Symbol) —

    The name of the tool

  • options (Hash) (defaults to: {}) —

    Configuration options for the tool

Returns:



234
235
236
 
# File 'lib/llm/provider.rb', line 234

def server_tool(name, options = {})
  LLM::ServerTool.new(name, options, self)
end

#server_tools ⇒ String => LLM::ServerTool

Note:

This method might be outdated, and the LLM::Provider#server_tool method can be used if a tool is not found here.

Returns all known tools provided by a provider.

Returns:



217
218
219
 
# File 'lib/llm/provider.rb', line 217

def server_tools
  {}
end

#streamable?(stream) ⇒ Boolean

Parameters:

Returns:

  • (Boolean) 


330
331
332
 
# File 'lib/llm/provider.rb', line 330

def streamable?(stream)
  LLM::Stream === stream || stream.respond_to?(:<<)
end

#system_role ⇒ Symbol

Returns:

  • (Symbol) 


256
257
258
 
# File 'lib/llm/provider.rb', line 256

def system_role
  :system
end

#tool_role ⇒ Symbol

Returns:

  • (Symbol) 


268
269
270
 
# File 'lib/llm/provider.rb', line 268

def tool_role
  :tool
end

#tracer ⇒ LLM::Tracer

Returns a fiber-local tracer

Returns:



275
276
277
 
# File 'lib/llm/provider.rb', line 275

def tracer
  weakmap[self] || LLM::Tracer::Null.new(self)
end

#tracer=(tracer) ⇒ void

This method returns an undefined value.

Set a fiber-local tracer

Examples:

llm = LLM.openai(key: ENV["KEY"])
Thread.new do
  llm.tracer = LLM::Tracer::Logger.new(llm, path: "/path/to/log/1.txt")
end
Thread.new do
  llm.tracer = LLM::Tracer::Logger.new(llm, path: "/path/to/log/2.txt")
end
# ...

Parameters:



293
294
295
296
297
298
299
300
301
302
303
 
# File 'lib/llm/provider.rb', line 293

def tracer=(tracer)
  if tracer.nil?
    if weakmap.respond_to?(:delete)
      weakmap.delete(self)
    else
      weakmap[self] = nil
    end
  else
    weakmap[self] = tracer
  end
end

#user_role ⇒ Symbol

Returns:

  • (Symbol) 


250
251
252
 
# File 'lib/llm/provider.rb', line 250

def user_role
  :user
end

#vector_stores ⇒ LLM::OpenAI::VectorStore

Returns an interface to the vector stores API

Returns:

  • (LLM::OpenAI::VectorStore) —

    Returns an interface to the vector stores API

Raises:

  • (NotImplementedError) 


169
170
171
 
# File 'lib/llm/provider.rb', line 169

def vector_stores
  raise NotImplementedError
end

#web_search(query:) ⇒ LLM::Response

Provides a web search capability

Parameters:

  • query (String) —

    The search query

Returns:

Raises:

  • (NotImplementedError) —

    When the method is not implemented by a subclass



244
245
246
 
# File 'lib/llm/provider.rb', line 244

def web_search(query:)
  raise NotImplementedError
end

#with(headers:) ⇒ LLM::Provider

Add one or more headers to all requests

Examples:

llm = LLM.openai(key: ENV["KEY"])
llm.with(headers: {"OpenAI-Organization" => ENV["ORG"]})
llm.with(headers: {"OpenAI-Project" => ENV["PROJECT"]})

Parameters:

  • headers (Hash<String,String>) —

    One or more headers

Returns:



205
206
207
208
209
 
# File 'lib/llm/provider.rb', line 205

def with(headers:)
  lock do
    tap { @headers.merge!(headers) }
  end
end