RubyCanUseLLM

A unified Ruby client for multiple LLM providers with generators. One interface, every LLM.

The Problem

Every time a Ruby developer wants to add LLMs to their app, they start from scratch: pick a provider gem, learn its API, write a service object, handle errors, parse responses. Switch providers? Rewrite everything.

The Solution

RubyCanUseLLM gives you two things:

1. Unified client — One interface that works the same across OpenAI, Anthropic, and more. Switch providers by changing a string, not your code.
2. Generators — Commands that scaffold ready-to-use boilerplate. You don't start from zero, you start with something that works.

Installation

gem install rubycanusellm

Or add to your Gemfile:

gem "rubycanusellm"

Quick Start

1. Generate configuration

rubycanusellm generate:config

This creates a config file with your provider and API key. In Rails it goes to config/initializers/rubycanusellm.rb, otherwise to config/llm.rb.

2. Generate a completion service

rubycanusellm generate:completion

This creates a ready-to-use service object. In Rails it goes to app/services/, otherwise to lib/.

3. Use it

RubyCanUseLLM.configure do |config|
  config.provider = :openai
  config.api_key = ENV["LLM_API_KEY"]
end

response = RubyCanUseLLM.chat([
  { role: :user, content: "What is Ruby?" }
])

puts response.content
puts "Tokens: #{response.total_tokens}"

Switch providers in one line

config.provider = :anthropic

That's it. Same code, different provider.

Supported Providers

Provider	Models	Status	Notes
OpenAI	gpt-4o-mini, gpt-4o, etc.	✅	Chat + Embeddings
Anthropic	claude-sonnet-4-20250514, etc.	✅	Chat only
Mistral	mistral-small-latest, mistral-large-latest, etc.	✅	Chat + Embeddings
Ollama	llama3.2, mistral, etc.	✅	Chat + Embeddings (local)
Voyage AI	voyage-3.5, voyage-4, etc.	✅	Embeddings only

API Reference

Configuration

RubyCanUseLLM.configure do |config|
  config.provider = :openai          # :openai, :anthropic, :mistral, or :ollama
  config.api_key = "your-key"        # required (not needed for Ollama)
  config.model = "gpt-4o-mini"       # optional, has sensible defaults
  config.timeout = 30                # optional, default 30s
  config.base_url = "http://localhost:11434"  # optional, for Ollama (default shown)
  config.embedding_provider = :voyage  # optional, for separate embedding provider
  config.embedding_api_key = "key"     # required when embedding_provider is set
end

Ollama (local, no API key needed):

RubyCanUseLLM.configure do |config|
  config.provider = :ollama
  # config.base_url = "http://localhost:11434"  # default, change if needed
end

Chat

response = RubyCanUseLLM.chat(messages, **options)

messages — Array of hashes with :role and :content:

messages = [
  { role: :system, content: "You are helpful." },
  { role: :user, content: "Hello" }
]

options — Override config per request:

RubyCanUseLLM.chat(messages, model: "gpt-4o", temperature: 0.5)

Streaming

Pass stream: true with a block to receive tokens as they arrive:

RubyCanUseLLM.chat(messages, stream: true) do |chunk|
  print chunk.content
end

Each chunk is a RubyCanUseLLM::Chunk with content (the token text) and role ("assistant"). Works with OpenAI, Anthropic, Mistral, and Ollama.

Response

response.content       # "Hello! How can I help?"
response.model         # "gpt-4o-mini"
response.input_tokens  # 10
response.output_tokens # 5
response.total_tokens  # 15
response.raw           # original provider response

Embeddings

response = RubyCanUseLLM.embed("Hello world")
response.embedding  # [0.1, 0.2, ...]
response.tokens     # 3
response.model      # "text-embedding-3-small"

OpenAI users — embeddings work out of the box, no extra config needed:

RubyCanUseLLM.configure do |config|
  config.provider = :openai
  config.api_key = ENV["OPENAI_API_KEY"]
end

RubyCanUseLLM.embed("Hello world")

Anthropic users with Voyage AI (recommended by Anthropic):

RubyCanUseLLM.configure do |config|
  config.provider = :anthropic
  config.api_key = ENV["ANTHROPIC_API_KEY"]
  config.embedding_provider = :voyage
  config.embedding_api_key = ENV["VOYAGE_API_KEY"]
end

RubyCanUseLLM.embed("Hello world")

Anthropic users with OpenAI for embeddings:

RubyCanUseLLM.configure do |config|
  config.provider = :anthropic
  config.api_key = ENV["ANTHROPIC_API_KEY"]
  config.embedding_provider = :openai
  config.embedding_api_key = ENV["OPENAI_API_KEY"]
end

RubyCanUseLLM.embed("Hello world")

Cosine similarity:

a = RubyCanUseLLM.embed("cat")
b = RubyCanUseLLM.embed("dog")
a.cosine_similarity(b.embedding)  # 0.87

Error Handling

begin
  RubyCanUseLLM.chat(messages)
rescue RubyCanUseLLM::AuthenticationError
  # invalid API key
rescue RubyCanUseLLM::RateLimitError
  # too many requests
rescue RubyCanUseLLM::TimeoutError
  # request timed out
rescue RubyCanUseLLM::ProviderError => e
  # other provider error
end

Tool Calling

Define tools once, use them with any provider:

tools = [
  {
    name: "get_weather",
    description: "Get current weather for a city",
    parameters: {
      type: "object",
      properties: {
        location: { type: "string", description: "City name" }
      },
      required: ["location"]
    }
  }
]

messages = [{ role: :user, content: "What's the weather in Paris?" }]
response = RubyCanUseLLM.chat(messages, tools: tools)

if response.tool_call?
  tc = response.tool_calls.first
  # tc.id        => "call_abc123"
  # tc.name      => "get_weather"
  # tc.arguments => { "location" => "Paris" }

  # Execute the tool and continue the conversation
  weather = fetch_weather(tc.arguments["location"])

  messages << { role: :assistant, content: response.content, tool_calls: response.tool_calls }
  messages << { role: :tool, tool_call_id: tc.id, name: tc.name, content: weather }

  final_response = RubyCanUseLLM.chat(messages, tools: tools)
  puts final_response.content
end

Works the same across providers — OpenAI, Anthropic, Mistral, and Ollama. Format differences (Anthropic uses input_schema and tool_result messages) are handled internally.

Prompt Templates

Keep prompts out of your Ruby code. Define them in YAML files with ERB for dynamic content:

# prompts/commodity_analysis.yml
system: |
  You are an expert in <%= domain %> classification.
user: |
  Analyze this item: <%= description %>
  <% if references.any? %>
  Similar references:
  <% references.each do |ref| %>
  - <%= ref %>
  <% end %>
  <% end %>

messages = RubyCanUseLLM::Prompt.load("prompts/commodity_analysis.yml",
  domain: "electronics",
  description: "capacitor 10uF",
  references: ["ceramic", "electrolytic"]
)
RubyCanUseLLM.chat(messages)

For inline prompts:

prompt = RubyCanUseLLM::Prompt.new(
  system: "You are a <%= role %> assistant.",
  user: "Help me with: <%= task %>"
)
messages = prompt.render(role: "coding", task: "fix this bug")
RubyCanUseLLM.chat(messages)

ERB is supported in both cases — loops, conditionals, any Ruby expression.

Generators

Command	Description
rubycanusellm generate:config	Configuration file with provider setup
rubycanusellm generate:completion	Completion service object
rubycanusellm generate:embedding	Embedding service object

Roadmap

• [x] Project setup
• [x] Configuration module
• [x] OpenAI provider
• [x] Anthropic provider
• [x] generate:config command
• [x] generate:completion command
• [x] v0.1.0 release
• [x] Streaming support
• [x] Embeddings + configurable embedding provider
• [x] Voyage AI provider (embeddings)
• [x] Mistral provider (chat + embeddings)
• [x] Ollama provider (chat + embeddings, local)
• [x] generate:embedding command
• [x] Prompt templates (ERB + YAML file-based)
• [x] Tool calling

Development

git clone https://github.com/mgznv/rubycanusellm.git
cd rubycanusellm
bin/setup
bundle exec rspec

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/mgznv/rubycanusellm.

License

The gem is available as open source under the terms of the MIT License.