Class: SpecGuard::RSpec::Configuration

Inherits:
Object
  • Object
show all
Defined in:
lib/specguard/rspec/configuration.rb

Overview

What SpecGuard::RSpecFormatter needs to know about the run it is watching: which checkout produced it, and where to send what was captured.

Why the environment is the default source

commit_sha and branch are facts about the checkout, not about the suite, and every CI provider already publishes them. Reading them from ENV means the common case — a CI job on any of the five major providers — needs no configuration at all. When no provider named either, both fall through to GitCheckout, which asks git itself — so a laptop run and a hand-rolled container report their checkout too, without being told to.

The escape hatch stays open on top of all of it, for a value neither source can know:

SpecGuard::RSpec.configure do |config|
config.branch = "release/2.0"
end

An explicitly assigned value always wins; the ENV names and the git fallback are only consulted to seed the defaults.

Why an unset value is nil rather than an error

Nothing here may ever be the reason a test run fails (see SpecGuard::RSpecFormatter's never-block-CI contract). Somebody running bundle exec rspec on a laptop has no GITHUB_SHA, and that is not a misconfiguration to shout about — it is a run whose envelope honestly records "unknown commit". A blank or whitespace-only variable is treated the same as an unset one, because CI providers routinely export "" for a variable that does not apply to the current event.

Transport, and what an unset api_key means

endpoint, api_key and timeout configure the POST to SpecGuard's ingest endpoint. api_key is the switch: set it and the run is delivered over HTTP, leave it unset and the run is appended to output_path exactly as before. Local development is therefore the default, and it needs no opt-out — which is the whole reason the switch is the credential rather than a separate enabled flag nobody would remember to turn off.

There is deliberately no default endpoint. SpecGuard is self-hostable and there is no address that is right for everybody; guessing one would mean a misconfigured project POSTing its test names to whatever happens to answer at that name.

Constant Summary collapse

DEFAULT_OUTPUT_PATH =

Relative to the working directory the suite was started from, which is the project root for every normal bundle exec rspec invocation.

"log/test_results.jsonl"
DEFAULT_TIMEOUT_SECONDS =

Applied to opening the connection and to reading the response.

Net::HTTP's own defaults are 60s each, which is up to two minutes added to every CI run against a hung endpoint — squarely against the roadmap's "a 20,000-example run must not be meaningfully slowed". Ten seconds is the number the client-gem spec settles on, and it is the whole budget: there are no retries, because a retry doubles the worst case for telemetry that is explicitly allowed to be lost.

10
COMMIT_SHA_KEYS =

Checked in order, first non-blank wins. The SPECGUARD_-prefixed name comes first everywhere so a project can always override whatever its CI provider decided to export.

The provider list is not decoration. commit_sha is the one field the platform rejects a run over, so a list that only knew GITHUB_SHA meant every GitLab, CircleCI, Buildkite and Jenkins run 400ing as a matter of course — the default outcome on four of the five providers this gem is likely to meet.

%w[
  SPECGUARD_COMMIT_SHA
  GITHUB_SHA
  CI_COMMIT_SHA
  CIRCLE_SHA1
  BUILDKITE_COMMIT
  GIT_COMMIT
].freeze
BRANCH_KEYS =

The same providers, in the same order. GITHUB_REF_NAME is the short name (main), not the full ref (refs/heads/main) that GITHUB_REF carries; Jenkins's GIT_BRANCH is passed through as it comes (origin/main on many setups), because branch is a free-form string to the platform and inventing a normalization here would be this gem guessing at somebody's ref layout.

%w[
  SPECGUARD_BRANCH
  GITHUB_REF_NAME
  CI_COMMIT_REF_NAME
  CIRCLE_BRANCH
  BUILDKITE_BRANCH
  GIT_BRANCH
].freeze
RUN_ID_KEYS =

The id the CI provider gave the build, which every shard of a sharded run shares.

This is the field that makes a 20,000-example suite land as one run. Nobody runs a suite that size in a single process: under parallel_tests, Knapsack or a CI matrix each shard loads this formatter and POSTs its own slice, and every shard carries the same commit_sha and branch — so the platform had nothing to tell four shards of one run apart from four separate runs, and recorded four TestRun rows each holding a quarter of the denominator. Annotations tend to cluster in whatever area a team is currently working on rather than spreading evenly across shards, so the headline ratio then moved from re-run to re-run without the suite changing at all.

It identifies the run, and deliberately survives a re-run

These ids are stable across re-attempts of the same build, and that is documented behaviour rather than an accident. GitHub Actions, verbatim: GITHUB_RUN_ID is "A unique number for each workflow run within a repository. This number does not change if you re-run the workflow run." (GITHUB_RUN_ATTEMPT is the value that increments per attempt.) Buildkite retries a job inside the same BUILDKITE_BUILD_ID, bumping BUILDKITE_RETRY_COUNT; GitLab retries a job inside the same CI_PIPELINE_ID.

The attempt is deliberately not part of this id, and that is the correction this field carries over its first implementation. Pressing "re-run failed jobs" — the mainline recovery gesture for a sharded suite, because re-running all 20,000 examples for one flaky shard is the thing sharding exists to avoid — re-runs a subset of shards. Had the attempt been folded in, that subset would have formed a brand-new run holding only the shards that happened to be retried, which is the split denominator this whole field exists to abolish, one gesture further down. Keeping the id stable instead lets a retried shard land back on the run it belongs to and replace its own previous slice.

That replacement is what SHARD_ID_KEYS is for, and the two fields are only correct together: without a shard identity the platform can add a re-delivered slice but cannot recognise it, so a re-run inflates the denominator instead of refreshing it.

Same providers in the same order as above, and the same "unset is nil, never an error" rule — a laptop run has none of these, and the platform treats a run with no id as its own run, which is exactly right.

The requirement on whatever a provider exports is only this: every shard of one run reads the same value, and no other run reads it. A project whose CI layout does not satisfy that for the variable below — Jenkins BUILD_TAG interpolates JOB_NAME, which some matrix layouts vary per axis — sets SPECGUARD_RUN_ID itself, which wins over all of them.

%w[
  SPECGUARD_RUN_ID
  GITHUB_RUN_ID
  CI_PIPELINE_ID
  CIRCLE_WORKFLOW_ID
  BUILDKITE_BUILD_ID
  BUILD_TAG
].freeze
SHARD_ID_KEYS =

Which shard of the run this process is.

The platform keys each slice of a run by this, so a shard that reports twice — a retried job, a re-run of the whole build — replaces its own previous numbers rather than adding to them. Without it, ingest can only ever add, and "re-run failed jobs" reports a suite larger than the suite.

It only has to be unique within one run; it is never compared across runs, so a bare parallel-process index is a perfectly good value.

nil is allowed and is not an error: the slice is still counted, it just cannot be recognised if it arrives twice. See the "anonymous contribution" note in the platform's Ingest::RunRecorder for exactly what that costs.

Two traps that are the reason this is not a plain key list

  1. parallel_tests — by far the most likely way a Ruby suite is sharded — sets TEST_ENV_NUMBER to the empty string for its first process ('', '2', '3', … so that the first process uses the plain yourproject_test database). A first-non-blank rule would therefore skip straight past it and leave process 1, and only process 1, anonymous — the single hardest version of this bug to see, because three shards of four would be idempotent. Presence of the variable is the signal here, not its value, so a set-but-empty TEST_ENV_NUMBER resolves to "1". See #resolve_shard_id.

  2. GitHub Actions exports no per-leg index for a matrix: job. GITHUB_JOB is the job's id as written in the workflow YAML and is identical across every leg of the matrix, so it is not in this list — it would make all N shards claim to be the same shard, and the run would keep only whichever finished last. A matrix-sharded suite sets SPECGUARD_SHARD_ID from the matrix value itself, e.g. SPECGUARD_SHARD_ID: ${{ matrix.shard }}.

The same rule applies to any nesting: parallel_tests inside a matrix leg makes TEST_ENV_NUMBER repeat across legs, so those projects set SPECGUARD_SHARD_ID to something that composes both.

%w[
  SPECGUARD_SHARD_ID
  TEST_ENV_NUMBER
  CI_NODE_INDEX
  CIRCLE_NODE_INDEX
  BUILDKITE_PARALLEL_JOB
].freeze
BLANK_MEANS_FIRST_SHARD_KEY =

The one key in SHARD_ID_KEYS whose empty value is meaningful rather than absent. See trap 1 above.

"TEST_ENV_NUMBER"
OUTPUT_PATH_KEYS = 
%w[SPECGUARD_OUTPUT_PATH].freeze
ENDPOINT_KEYS = 
%w[SPECGUARD_ENDPOINT].freeze
API_KEY_KEYS = 
%w[SPECGUARD_API_KEY].freeze
TIMEOUT_KEYS = 
%w[SPECGUARD_TIMEOUT].freeze

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(env: ENV, git: GitCheckout) ⇒ Configuration

Returns a new instance of Configuration.

Parameters:

  • env (#[]) (defaults to: ENV) —

    the environment to seed defaults from. Injectable so the mapping can be tested without mutating the process's own ENV.

  • git (#commit_sha, #branch) (defaults to: GitCheckout) —

    the checkout to fall back to when no variable named the commit or the branch. Injectable for the same reason, and so a test does not silently pick up the sha or the branch of whatever repository it runs in.



354
355
356
357
358
359
360
361
362
363
364
365
 
# File 'lib/specguard/rspec/configuration.rb', line 354

def initialize(env: ENV, git: GitCheckout)
  @commit_sha = first_present(env, COMMIT_SHA_KEYS) || blank_to_nil(git.commit_sha)
  # `||` short-circuits, which is the whole of "do not spawn a subprocess
  # when a provider already answered".
  @branch = first_present(env, BRANCH_KEYS) || blank_to_nil(git.branch)
  @run_id = first_present(env, RUN_ID_KEYS)
  @shard_id = resolve_shard_id(env)
  @output_path = first_present(env, OUTPUT_PATH_KEYS) || DEFAULT_OUTPUT_PATH
  @endpoint = first_present(env, ENDPOINT_KEYS)
  @api_key = first_present(env, API_KEY_KEYS)
  @timeout = seconds(first_present(env, TIMEOUT_KEYS)) || DEFAULT_TIMEOUT_SECONDS
end

Instance Attribute Details

#api_key ⇒ Object

The repository's SpecGuard API key. Present means "deliver over HTTP".

Deliberately not format-checked. The platform mints sgk_-prefixed keys today; a client that gated on that prefix would start rejecting valid keys the day the platform rotated it, from a version of the gem nobody can retroactively fix. Send whatever is configured and let a 401 be the answer.



344
345
346
 
# File 'lib/specguard/rspec/configuration.rb', line 344

def api_key
  @api_key
end

#branch ⇒ Object

The branch the suite ran on. nil when nothing said — including a detached checkout, which has no branch to name and says so rather than inventing one. See GitCheckout::BRANCH_COMMAND.



321
322
323
 
# File 'lib/specguard/rspec/configuration.rb', line 321

def branch
  @branch
end

#commit_sha ⇒ Object

The commit the suite ran against. nil when nothing said.



317
318
319
 
# File 'lib/specguard/rspec/configuration.rb', line 317

def commit_sha
  @commit_sha
end

#endpoint ⇒ Object

The SpecGuard installation to POST to, scheme and host only: https://specguard.example.com. The /api/v1/ingest path is the platform's contract, not a setting. nil when nothing said.



336
337
338
 
# File 'lib/specguard/rspec/configuration.rb', line 336

def endpoint
  @endpoint
end

#output_path ⇒ Object

Where the run's JSON payload is appended, one object per line — the local sink when there is no API key, and the fallback when delivery fails.



332
333
334
 
# File 'lib/specguard/rspec/configuration.rb', line 332

def output_path
  @output_path
end

#run_id ⇒ Object

The CI provider's id for the build this process is one shard of. nil when nothing said, which is how a laptop run spells "I am my own run". See RUN_ID_KEYS.



325
326
327
 
# File 'lib/specguard/rspec/configuration.rb', line 325

def run_id
  @run_id
end

#shard_id ⇒ Object

Which shard of that build this process is. nil when nothing said. See SHARD_ID_KEYS.



328
329
330
 
# File 'lib/specguard/rspec/configuration.rb', line 328

def shard_id
  @shard_id
end

#timeout ⇒ Object

Seconds, applied to opening the connection and to reading the response.



346
347
348
 
# File 'lib/specguard/rspec/configuration.rb', line 346

def timeout
  @timeout
end