Class: Legate::LLM::Ollama

Inherits:
Adapter
  • Object
show all
Defined in:
lib/legate/llm/ollama.rb

Overview

LLM adapter backed by a local Ollama server (ollama.com).

Talks to Ollama’s /api/generate HTTP endpoint — no API key, no cost, fully local. Configure the host via the :host option or the OLLAMA_HOST env var (default localhost:11434). Wire it up globally with:

Legate::LLM.default_adapter_factory = lambda do |model:, **|
  Legate::LLM::Ollama.new(model: model)
end

Constant Summary collapse

DEFAULT_HOST = 
'http://localhost:11434'

Instance Method Summary collapse

Methods inherited from Adapter

#generate_with_tools, #supports_function_calling?, #supports_structured_output?

Constructor Details

#initialize(model:, host: nil, logger: nil, read_timeout: 120, **_ignored) ⇒ Ollama

Returns a new instance of Ollama.

Parameters:

  • model (String) —

    the Ollama model tag, e.g. ‘llama3’ or ‘qwen2.5’

  • host (String, nil) (defaults to: nil) —

    base URL of the Ollama server

  • logger (Logger, nil) (defaults to: nil)
  • read_timeout (Integer) (defaults to: 120) —

    seconds to wait for a completion (default 120)



27
28
29
30
31
32
33
 
# File 'lib/legate/llm/ollama.rb', line 27

def initialize(model:, host: nil, logger: nil, read_timeout: 120, **_ignored)
  super()
  @model = model
  @host = (host || ENV['OLLAMA_HOST'] || DEFAULT_HOST).to_s.chomp('/')
  @logger = logger || Legate.logger
  @read_timeout = read_timeout
end

Instance Method Details

#available? ⇒ Boolean

Ollama is a local server; assume it’s reachable rather than pinging it on every planner init. A real failure surfaces from #generate with a clear message.

Returns:

  • (Boolean) 


38
39
40
 
# File 'lib/legate/llm/ollama.rb', line 38

def available?
  true
end

#generate(prompt, json: false, schema: nil) ⇒ Object

‘schema:` is accepted for interface parity but ignored (Ollama’s ‘format: json` is the only structured constraint used here).

See Also:



49
50
51
52
53
54
55
56
57
58
 
# File 'lib/legate/llm/ollama.rb', line 49

def generate(prompt, json: false, schema: nil) # rubocop:disable Lint/UnusedMethodArgument
  body = { model: @model, prompt: prompt, stream: false }
  # Ollama supports constrained JSON output via the "format" field.
  body[:format] = 'json' if json

  post_json('/api/generate', body)['response']
rescue StandardError => e
  @logger.error("Ollama generate failed (#{@host}, model '#{@model}'): #{e.class}: #{e.message}")
  raise
end

#model_name ⇒ Object 



42
43
44
 
# File 'lib/legate/llm/ollama.rb', line 42

def model_name
  @model
end