The simplecov-rspec Gem

Gem Version Documentation Change Log Build Status Conventional Commits Slack

simplecov-rspec is a Ruby gem that integrates SimpleCov with RSpec. SimpleCov (>= 1.0) already enforces minimum_coverage for line, branch, and method coverage and fails the build when a threshold is missed. This gem layers four things on top that SimpleCov doesn't do on its own:

  1. Suppresses coverage failures when RSpec is run in dry-run mode (e.g. from an IDE).
  2. Lists (or summarizes) the individual uncovered lines, branches, and methods.
  3. Scopes that listing to the files you name, or to the code the run described.
  4. Lets all of the above be overridden from the environment, for CI.

When simplecov-rspec is used, RSpec will report an error if the percent of test coverage falls below a defined threshold:

Coverage report generated for RSpec to coverage/index.html
Line coverage: 284 / 286 (99.30%)
Line coverage (99.30%) is below the expected minimum coverage (100.00%).
  Lowest-coverage files (line):
     99.30%  lib/example_project.rb
SimpleCov failed with exit 2 due to a coverage related error

All of that comes from SimpleCov itself. If configured to list the items that were not covered by tests, this gem adds its own listing between SimpleCov's summary and its failure message:

Coverage report generated for RSpec to coverage/index.html
Line coverage: 284 / 286 (99.30%)

2 lines are not covered by tests:
  ./lib/example_project.rb:74
  ./lib/example_project.rb:75
Line coverage (99.30%) is below the expected minimum coverage (100.00%).

Scoping the listing to particular files changes its shape again. It is marked off with a header saying how much of the result it covers, and each criterion reports what the scoped files cover directly above what they miss — see Scoping the listing to specific files:

Coverage report generated for RSpec to coverage/index.html
Line coverage: 284 / 286 (99.30%)
Branch coverage: 138 / 150 (92.00%)

-- Reporting uncovered lines and branches for 1 of 12 files --

Scoped line coverage: 73 / 74 (98.64%)
1 line is not covered by tests:
  ./lib/example_project/parser.rb:74

Scoped branch coverage: 11 / 12 (91.66%)
1 branch is not covered by tests:
  ./lib/example_project/parser.rb:82 (then branch)

Installation

To install the gem, add to the following line to your application's gemspec OR Gemfile:

gemspec:

  spec.add_development_dependency "simplecov-rspec", '~> 1.0'

Gemfile:

gem "simplecov-rspec", "~> 1.0", groups: [:development, :test]

and then run bundle install

If bundler is not being used to manage dependencies, install the gem by executing:

gem install simplecov-rspec

Getting started

To use simplecov-rspec, follow these steps:

  1. Add require 'simplecov-rspec' to your spec_helper.rb.
  2. Replace SimpleCov.start with SimpleCov::RSpec.start in your spec_helper.rb, ensuring this line appears before requiring your project files.

Here is an example spec_helper.rb. Your spec helper may include other code in addition to these:

require 'simplecov-rspec'

SimpleCov::RSpec.start

require 'my_project'

This will configure RSpec to fail when test coverage falls below 100%.

That is it!

Basic setup

To initialize simplecov-rspec with defaults, add the following to your spec_helper.rb:

require 'simplecov-rspec'

SimpleCov::RSpec.start

This is equivalent to starting with the following options:

SimpleCov::RSpec.start(
    minimum_coverage: { line: 100 },
    fail_on_low_coverage: true,
    list_uncovered: false,
    list_uncovered_detail: true,
    list_uncovered_files: nil
)

minimum_coverage is the minimum percent of lines (and, optionally, branches and methods) covered by tests, enforced by SimpleCov itself.

To require less than 100% line coverage:

SimpleCov::RSpec.start(minimum_coverage: 90)

To also require branch (and/or method) coverage, pass a Hash. Any criterion named here is automatically enabled via SimpleCov.enable_coverage:

SimpleCov::RSpec.start(minimum_coverage: { line: 100, branch: 90 })

Listing uncovered items

To list the individual lines, branches, and/or methods that are not covered, set list_uncovered. It accepts :all, a single criterion, or an Array of criteria — independent of what minimum_coverage enforces:

SimpleCov::RSpec.start(minimum_coverage: { line: 100, branch: 90 }, list_uncovered: :all)

2 lines are not covered by tests:
  ./lib/example_project.rb:74
  ./lib/example_project.rb:75

1 branch is not covered by tests:
  ./lib/example_project.rb:82 (else branch)

1 method is not covered by tests:
  ./lib/example_project.rb:96 ExampleProject#unused

A criterion with nothing uncovered is left out entirely, so :all prints fewer sections than this when there is less to say.

For a quieter CI log, set list_uncovered_detail: false to print only the count per criterion, along with a hint on how to see the details:

SimpleCov::RSpec.start(list_uncovered: :all, list_uncovered_detail: false)

2 lines are not covered by tests.
1 branch is not covered by tests.
1 method is not covered by tests.

Run with LIST_UNCOVERED_DETAIL=true to see the uncovered lines, branches and methods.

Scoping the listing to specific files

By default the uncovered listing covers every file SimpleCov tracked. On a focused run — one spec file, or one directory — that listing is mostly noise: spec_helper requires the whole project, so nearly all of it is legitimately unexercised.

list_uncovered_files (available since version 1.1) narrows the listing to the files you care about, given as Dir.glob patterns resolved against SimpleCov.root. An absolute path is used as given:

SimpleCov::RSpec.start(list_uncovered: :all, list_uncovered_files: 'lib/example_project/parser.rb')

This option scopes the listing that list_uncovered asks for; it does not ask for one. list_uncovered defaults to false, which lists nothing, so setting only list_uncovered_files produces no output at all. Set both.

-- Reporting uncovered lines, branches and methods for 1 of 218 files --

Scoped line coverage: 73 / 74 (98.64%)
1 line is not covered by tests:
  ./lib/example_project/parser.rb:74

Scoped branch coverage: 12 / 12 (100.00%)
Scoped method coverage: 8 / 8 (100.00%)

Each criterion reports what the scoped files cover directly above what they miss, so a count like "1 line is not covered" arrives with the denominator that makes it readable, and a blank line always means "next criterion". These are deliberately labelled differently from SimpleCov's own project-wide summary, and the report is marked off with a header, because SimpleCov prints that summary a few lines earlier on the same stream while counting different things.

This narrows only the listing. Coverage is still measured, enforced, and formatted for the whole project, so the reported percentage and the HTML report mean the same thing whether or not this option is set. There is one definition of "the coverage number", and this option does not change it.

A scoped report always prints something, and always says how much of the result it covered, so it can never be mistaken for a clean run of the whole suite:

-- Reporting uncovered lines, branches and methods for 1 of 218 files --

Scoped line coverage: 74 / 74 (100.00%)
Scoped branch coverage: 12 / 12 (100.00%)
Scoped method coverage: 8 / 8 (100.00%)

No uncovered lines, branches and methods in this file.

When it comes up empty, it says which of the three reasons applies, since only one of them means you mistyped a pattern:

-- Reporting uncovered lines, branches and methods for 0 of 218 files --

No files matched, so no coverage was reported.

-- Reporting uncovered lines, branches and methods for 0 of 218 files --

1 file matched, but it is not in the coverage result. It may not have been loaded by
this run, or may be excluded by a SimpleCov filter.

-- Reporting uncovered lines, branches and methods for 0 of 218 files --

No files were requested, so no coverage was reported.

Scoping to the code under test

Naming the files by hand is the awkward part of a focused run: the file you want is whatever you happen to be testing right now. :described resolves to the source files defining the classes the run described, so it follows you from run to run:

SimpleCov::RSpec.start(list_uncovered: :all, list_uncovered_files: :described)

# Report on lib/example_project/parser.rb, because that is what these specs describe
bundle exec rspec spec/example_project/parser_spec.rb

It walks nested groups too, so a describe of one class inside another contributes both. A group describing something that is not a class contributes nothing, as does one whose class is anonymous, defined in C, or no longer reachable by name — there is no source file to report on in those cases.

The scope is the classes the run described, not the ones it exercised. A class that a described class delegates to is not included, and a group written as describe 'the parser' do names no class at all. Where that matters, build the scope yourself: SimpleCov::RSpec.described_source_files is public, and you can add to what it returns.

SimpleCov::RSpec.start(
  list_uncovered: :all,
  list_uncovered_files: -> { SimpleCov::RSpec.described_source_files + ['lib/example_project/lexer.rb'] }
)

The lambda is doing real work there, and leaving it off is a mistake worth naming. SimpleCov::RSpec.start runs before any example is defined, so calling described_source_files at that point returns an empty list and you get a report scoped to nothing. Any scope derived from the run has to be passed as a callable and resolved afterwards — which is exactly what :described does for you.

Configuration block

A configuration block can be given to the start method to further configure SimpleCov:

# Initialize SimpleCov with a specific formatter
SimpleCov::RSpec.start { formatter SimpleCov::Formatter::LcovFormatter }

This block is passed on to SimpleCov.start. See Configuring SimpleCov for details.

Configuration from environment variables

Environment variables can be used to configure simplecov-rspec. These environment variables take precedence over the values passed to SimpleCov::RSpec.start.

  • COVERAGE_THRESHOLD: Sets the minimum line coverage threshold (0-100). Overrides minimum_coverage[:line].
  • COVERAGE_THRESHOLD_BRANCH: Sets the minimum branch coverage threshold (0-100), and enables branch coverage. Overrides minimum_coverage[:branch].
  • COVERAGE_THRESHOLD_METHOD: Sets the minimum method coverage threshold (0-100), and enables method coverage. Overrides minimum_coverage[:method].
  • FAIL_ON_LOW_COVERAGE: Controls whether tests fail if coverage is below the threshold. Set to 'true', 'yes', 'on', or '1' (case insensitive) to enable.
  • LIST_UNCOVERED: Controls which criteria to list uncovered items for. Set to 'all', 'true', 'yes', 'on', or '1' to report every criterion; 'false', 'no', 'off', or '0' to report none; or a comma-separated list, e.g. line,branch.
  • LIST_UNCOVERED_DETAIL: Controls whether uncovered items are listed individually, or just summarized as a count per criterion. Set to 'true', 'yes', 'on', or '1' (case insensitive) to show individual items.
  • LIST_UNCOVERED_FILES: Controls which files uncovered items are listed for. Set to a comma-separated list of Dir.glob patterns, relative to SimpleCov.root; to 'described', for the files defining the classes the run described; or to 'all' (or 'false', 'no', 'off', '0', or empty) to list them for every file. Since the separator is a comma, a brace pattern such as lib/{a,b}.rb cannot be used here — give the alternatives separately, as lib/a.rb,lib/b.rb. Like list_uncovered_files, this scopes the listing rather than asking for one: it has no effect unless LIST_UNCOVERED (or list_uncovered:) names at least one criterion.

For example, here is a bash script to run tests in an infinite loop while writing test output to fail.txt:

while true; do FAIL_ON_LOW_COVERAGE=false rspec >> fail.txt; done

In a CI system, you might want to set LIST_UNCOVERED=all in order to list uncovered lines, branches, and methods on a platform other than the one you use for local development.

Development

If you want to contribute or experiment with the gem, follow these steps to set up your development environment:

After checking out the repo, run bin/setup to install dependencies. Then, run rake to run linting, tests, etc. just like the CI build. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/main-branch/simplecov-rspec. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.

Commit message guidelines

All commit messages must follow the Conventional Commits standard. This helps us maintain a clear and structured commit history, automate versioning, and generate changelogs effectively.

To ensure compliance, this project includes:

  • A git commit-msg hook that validates your commit messages before they are accepted.

To activate the hook, you must have node installed and run npm install.

  • A GitHub Actions workflow that will enforce the Conventional Commit standard as part of the continuous integration pipeline.

Any commit message that does not conform to the Conventional Commits standard will cause the workflow to fail and not allow the PR to be merged.

Pull request guidelines

All pull requests must be merged using rebase merges. This ensures that commit messages from the feature branch are preserved in the release branch, keeping the history clean and meaningful.

License

The gem is available as open source under the terms of the MIT License.

Code of conduct

Everyone interacting in the Simplecov::Rspec project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.