RubyLLM::Test

This gem provides testing utilities for RubyLLM, a Ruby library for working with large language models (LLMs). It enables calls to LLMs to be stubbed so that the surrounding application logic can be tested without making actual calls to the LLM. This is particularly useful for testing code that interacts with LLMs, as it allows developers to simulate responses from the LLM without incurring the cost, latency, or randomness of real API calls.

RubyLLM::Test.stub_response("Outlook good")

chat = RubyLLM.chat
response = chat.ask "What are the odds this works?"

assert_equal "Outlook good", response.content

Installation

Add this line to your application's Gemfile in the test group:

gem 'ruby_llm-test'

And then execute:

$ bundle

Or install it yourself as:

$ gem install ruby_llm-test

Usage

Add the following lines to your spec/spec_helper.rb or test/test_helper.rb:

require 'ruby_llm/test'

RubyLLM::Models.singleton_class.prepend(RubyLLM::Test::ResolveWithTestProvider)

Then, in your tests, you can use the stub_response method to stub responses from the LLM. For example:

it 'returns a stubbed response' do
  RubyLLM::Test.stub_response('Hello, world!')

  response = MyLLMClient.call('Hello?')
  expect(response).to eq('Hello, world!')
end

If you make multiple calls to the LLM, you can call stub_response more than once or use method stub_responses to stub multiple responses at once. For example:

it 'returns multiple stubbed responses' do
  RubyLLM::Test.stub_responses('Hello, world!', 'How are you?')
  response1 = MyLLMClient.call('Hello?')
  response2 = MyLLMClient.call('How are you?')

  expect(response1).to eq('Hello, world!')
  expect(response2).to eq('How are you?')
end

Stubbing with a Message

If you stub with a string, it will be wrapped in a RubyLLM::Message with the role of :assistant. If you want more control over the message, you can stub with a RubyLLM::Message directly. For example:

it 'returns a stubbed message' do
  message = RubyLLM::Message.new(role: :assistant, content: 'Hello, world!')
  RubyLLM::Test.stub_response(message)

  response = MyLLMClient.call('Hello?')
  expect(response).to eq(message)
end

Stubbing with a Hash Returns JSON

If you stub with a hash, it will be converted to a RubyLLM::Message with the content set to the JSON representation of the hash. For example:

it 'returns a stubbed JSON message' do
  hash = { key: 'value' }
  RubyLLM::Test.stub_response(hash)

  response = MyLLMClient.call('Hello?')
  expect(response.content).to eq(hash.to_json)
end

Resetting Stubs

Make sure to reset stubs after each test to avoid interference before or between tests.

RubyLLM::Test.reset

Stubbing with a Block

You can also stub responses in a block, which handles the setup and teardown of stubs automatically. For example:

RubyLLM::Test.with_responses('Hello, world!') do
  response = MyLLMClient.call('Hello?')
  expect(response).to eq('Hello, world!')
end

Testing Arguments

You can verify arguments passed to the LLM by checking the requests received by the test provider with methods requests and last_request.

RubyLLM::Test.stub_response('Hello, world!')
chat = RubyLLM.chat(model: 'gpt-5-nano')
chat.with_tools(GreeterTool)
chat.ask('Hello?')
request = RubyLLM::Test.last_request

assert_equal 'gpt-5-nano', request.model
assert_includes request.tool_classes, GreeterTool

Any parameters passed to the Provider's complete method are available on the request object. The tool_classes method is a helper that returns the classes of any tools included in the request.

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake test to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/RockSolt/ruby_llm-test.

License

The gem is available as open source under the terms of the MIT License.