Smartest

Smartest is a small Ruby test runner with a keyword-fixture-first design.

It lets you write tests like this:

test("factorial") do
  expect(1 * 2 * 3).to eq(6)
end

and fixture-driven tests like this:

test("GET /me") do |logged_in_client:|
  response = logged_in_client.get("/me")

  expect(response.status).to eq(200)
end

Smartest is designed around three ideas:

1. Tests should be readable at the top level.
2. Fixture dependencies should be explicit.
3. Teardown should be written only when it is needed.

Installation

Add this line to your application's Gemfile:

gem "smartest"

Then run:

bundle install

Or install it directly:

gem install smartest

Quick start

Initialize a test scaffold:

bundle exec smartest --init

This creates:

smartest/test_helper.rb
smartest/fixtures/
smartest/matchers/
smartest/matchers/predicate_matcher.rb
smartest/example_test.rb

The generated example looks like this:

# smartest/example_test.rb
require "test_helper"

test("example") do
  expect(1 + 1).to eq(2)
end

Run the suite:

bundle exec smartest

By default, Smartest loads smartest/**/*_test.rb, so a separate test/ directory can remain available for Minitest.

You can also pass explicit paths:

bundle exec smartest smartest/**/*_test.rb

To run tests by line number, append :line or :start-end to the file path. Smartest runs tests whose test blocks contain or intersect the selected lines:

bundle exec smartest smartest/user_test.rb:12
bundle exec smartest smartest/user_test.rb:3-12

CLI help and version output are available with:

bundle exec smartest --help
bundle exec smartest --version

Expected output:

Running 1 test

✓ example

1 test, 1 passed, 0 failed

Defining tests

Use test at the top level:

test("adds numbers") do
  expect(1 + 2).to eq(3)
end

A test can request fixtures using required keyword arguments:

test("uses a user") do |user:|
  expect(user.name).to eq("Alice")
end

Smartest intentionally favors keyword arguments for fixture injection:

test("GET /me") do |logged_in_client:|
  # ...
end

This makes fixture usage explicit and avoids relying on positional argument order.

Skipping and pending tests

Use skip at the start of a test when the rest of the body should not run:

test("PDF export") do |browser:|
  skip "firefox is not supported" if browser.firefox?

  export_pdf(browser)
end

Use pending when the test should continue running but is expected to fail. If the test passes after pending, Smartest fails it so the stale pending marker is removed.

test("PDF export") do |browser:|
  pending "Not supported by WebDriver BiDi yet" if browser.bidi?

  export_pdf(browser)
end

skip and pending are available in test bodies and around_test hooks, but not as test metadata or fixture APIs. See Skipping Tests.

Expectations

Smartest uses an expectation style:

expect(actual).to eq(expected)
expect(actual).not_to eq(expected)
expect { action }.to raise_error(ErrorClass)
expect { action }.to raise_error(/message/)
expect { action }.to raise_error(ErrorClass, /message/)
expect { action }.to change { value }

Examples:

test("string") do
  expect("hello").to eq("hello")
end

test("array") do
  expect([1, 2, 3]).to include(2)
end

test("URL") do
  expect("about:blank").to start_with("about:")
end

test("download") do
  expect("screenshot.png").to end_with(".png")
end

test("type") do
  expect("smartest").to be_a(String)
end

test("URL pattern") do
  expect("https://example.test").to match(%r{\Ahttps://})
end

test("events") do
  expect(%i[request close open]).to contain_exactly(:open, :request, :close)
end

Supported matchers include:

eq(expected)
include(expected)
start_with(prefix, ...)
end_with(suffix, ...)
be_a(ClassOrModule)
be_an(ClassOrModule)
be_nil
match(regexp)
contain_exactly(item, ...)
match_array(items)
raise_error(ErrorClass)
raise_error(/message/)
raise_error(ErrorClass, /message/)
change { value }
change { value }.from(before).to(after)
change { value }.by(delta)

raise_error accepts an error class, a message regexp, or both. Use an error class to check the raised exception class, a regexp to check the raised exception message, or both to require both conditions. No-argument and exact string message forms are not supported.

contain_exactly and match_array compare collections without requiring a specific order, preserve duplicate counts, and can use matcher objects such as match(/foo/) or eq(42) as expected items.

change is only supported with expect { ... } block expectations and must be written with a value block.

Custom matcher modules can be registered from around_suite or around_test with use_matcher. The generated scaffold includes a PredicateMatcher custom matcher for be_<predicate> calls. See Matchers.

Fixtures

Fixtures are defined in classes.

class AppFixture < Smartest::Fixture
  fixture :user do
    User.create!(
      name: "Alice",
      email: "alice@example.com"
    )
  end
end

Register fixture classes from around_suite in smartest/test_helper.rb:

around_suite do |suite|
  use_fixture AppFixture
  suite.run
end

Tests request fixtures by keyword:

test("user") do |user:|
  expect(user.name).to eq("Alice")
end

A fixture is requested by name from a test block keyword argument.

test("user") do |user:|
  # Smartest resolves the `user` fixture
end

Fixture dependencies

Fixtures can depend on other fixtures using required keyword arguments.

class AppFixture < Smartest::Fixture
  suite_fixture :server do
    TestServer.start
  end

  fixture :client do |server:|
    Client.new(base_url: server.url)
  end
end

The dependency is explicit:

fixture :client do |server:|
  Client.new(base_url: server.url)
end

When a test requests client, Smartest resolves server first.

test("GET /health") do |client:|
  response = client.get("/health")

  expect(response.status).to eq(200)
end

Fixture scopes

Regular fixture definitions are test-scoped. Smartest creates a fresh value for each test that requests the fixture.

Use suite_fixture for expensive resources that should be created once and released after the full suite finishes:

class BrowserFixture < Smartest::Fixture
  suite_fixture :browser do
    browser = Browser.launch
    cleanup { browser.close }
    browser
  end

  fixture :page do |browser:|
    browser.new_page
  end
end

Suite fixtures are lazy: setup runs the first time a test requests the fixture, and cleanup runs once after all tests finish. Test-scoped fixtures can depend on suite fixtures, but suite fixtures cannot depend on test-scoped fixtures.

Suite hooks

Use around_suite when the full test run must execute inside another block:

around_suite do |suite|
  Async do
    suite.run
  end
end

The hook receives a run target and must call suite.run exactly once. The block wraps every test, test-scoped fixture setup and cleanup, suite fixture setup, and suite fixture cleanup.

Fixture and matcher registrations made before suite.run are applied to that run:

around_suite do |suite|
  use_fixture GlobalFixture
  suite.run
end

Multiple around_suite hooks run in registration order. The first hook is the outermost wrapper:

around_suite do |suite|
  with_outer_resource { suite.run }
end

around_suite do |suite|
  with_inner_resource { suite.run }
end

If an around_suite hook raises or does not call suite.run, Smartest reports a suite failure and exits with status 1.

Test hooks

Use around_test when each test needs to run inside another block:

around_test do |test|
  SomeAutoCloseResource.new do
    test.run
  end
end

The hook receives a run target and must call test.run exactly once. It wraps fixture setup, the test body, and fixture cleanup.

around_test is file-scoped when it is written directly in a test file. Smartest copies the current file's around_test hooks when each test is registered, so hooks apply to tests defined later in the same file.

Define around_test inside around_suite when the hook should apply to the whole run:

around_suite do |suite|
  around_test do |test|
    with_some_resource do
      test.run
    end
  end

  suite.run
end

around_test can also register fixture classes or matcher modules for that test run:

around_test do |test|
  use_fixture LocalFixture
  use_matcher LocalMatcher
  test.run
end

Fixture classes registered from around_test must define only test-scoped fixtures. If a class defines suite_fixture, register it from around_suite instead so its cache and cleanup belong to the suite lifecycle.

use_fixture and use_matcher are only available inside around_suite or around_test blocks. They are not top-level DSL methods.

Fixtures with teardown

Not every fixture needs teardown. For fixtures that do, use cleanup.

class WebFixture < Smartest::Fixture
  fixture :server do
    server = TestServer.start
    cleanup { server.stop }

    server.wait_until_ready!
    server
  end

  fixture :client do |server:|
    Client.new(base_url: server.url)
  end
end

cleanup blocks run after the fixture's scope finishes. For regular fixtures that means after the test. For suite_fixture, cleanup runs after the full suite.

They are executed in reverse order of registration.

fixture :temp_dir do
  dir = Dir.mktmpdir
  cleanup { FileUtils.rm_rf(dir) }

  dir
end

Recommended pattern:

fixture :server do
  server = TestServer.start
  cleanup { server.stop }

  server.wait_until_ready!
  server
end

Register cleanup immediately after acquiring the resource, before later setup steps that may fail.

Logged-in client example

class WebFixture < Smartest::Fixture
  fixture :server do
    server = TestServer.start
    cleanup { server.stop }

    server.wait_until_ready!
    server
  end

  fixture :client do |server:|
    Client.new(base_url: server.url)
  end

  fixture :user do
    User.create!(
      name: "Alice",
      email: "alice@example.com"
    )
  end

  fixture :logged_in_client do |client:, user:|
    client.login(user)
    client
  end
end

# smartest/test_helper.rb
around_suite do |suite|
  use_fixture WebFixture
  suite.run
end

# smartest/web_test.rb
require "test_helper"

test("GET /me") do |logged_in_client:|
  response = logged_in_client.get("/me")

  expect(response.status).to eq(200)
end

Dependency graph:

logged_in_client
  ├── client
  │   └── server
  └── user

Execution flow:

server setup
client setup
user setup
logged_in_client setup
test body
server cleanup

Registering fixture classes

Use use_fixture inside around_suite from smartest/test_helper.rb:

around_suite do |suite|
  use_fixture AppFixture
  suite.run
end

Multiple fixture classes can be registered:

around_suite do |suite|
  use_fixture UserFixture
  use_fixture WebFixture
  use_fixture ApiFixture
  suite.run
end

Fixture names must be unique across registered fixture classes.

If two fixture classes define the same fixture name, Smartest raises an error.

Suite hooks and fixture cleanup

Suite hooks are separate from fixture cleanup. Use fixture cleanup for resource-specific teardown:

fixture :server do
  server = TestServer.start
  cleanup { server.stop }
  server
end

Use around_suite for broad suite-level execution context:

around_suite do |suite|
  Async { suite.run }
end

Recommended file structure

smartest/
  test_helper.rb
  fixtures/
    app_fixture.rb
    web_fixture.rb
  matchers/
    predicate_matcher.rb
    have_status_matcher.rb
  example_test.rb

# smartest/test_helper.rb
require "smartest/autorun"

Dir[File.join(__dir__, "fixtures", "**", "*.rb")].sort.each do |fixture_file|
  require fixture_file
end

Dir[File.join(__dir__, "matchers", "**", "*.rb")].sort.each do |matcher_file|
  require matcher_file
end

around_suite do |suite|
  use_fixture WebFixture
  use_matcher PredicateMatcher
  suite.run
end

The generated helper loads Ruby files under smartest/fixtures/ and smartest/matchers/ in sorted order. Register suite-wide fixture classes and matcher modules from around_suite with use_fixture and use_matcher.

Example:

# smartest/fixtures/web_fixture.rb
class WebFixture < Smartest::Fixture
  fixture :server do
    server = TestServer.start
    cleanup { server.stop }
    server
  end

  fixture :client do |server:|
    Client.new(base_url: server.url)
  end
end

# smartest/example_test.rb
require "test_helper"

test("GET /health") do |client:|
  expect(client.get("/health").status).to eq(200)
end

Design choices

Smartest intentionally does not use this style as the primary API:

test("GET /me") do |logged_in_client|
end

Instead, Smartest prefers:

test("GET /me") do |logged_in_client:|
end

Keyword arguments make fixture injection explicit.

Smartest also avoids this fixture dependency style:

fixture :client, with: [:server] do |server|
  Client.new(base_url: server.url)
end

Instead, it prefers:

fixture :client do |server:|
  Client.new(base_url: server.url)
end

The dependency declaration and usage are in one place.

Status

Smartest is currently a design-stage test runner.

The intended MVP includes:

• top-level test
• class-based fixtures
• keyword-argument fixture injection
• fixture dependencies through keyword arguments
• fixture cleanup
• suite hooks with around_suite
• test hooks with around_test
• skipped and pending tests through skip and pending
• expect(...).to eq(...)
• console reporter
• CLI runner
• circular fixture dependency detection
• duplicate fixture detection