Module: SpecGuard::RSpec::ValidatorBackend

Defined in:: lib/specguard/rspec/validator_backend.rb

Overview

The opt-in Go validator backend: validate-intent --source --json.

What this is, and what it deliberately is not

SPGD-96's Definition of Done ends "…and the Ruby gem invoking it for linting … with the Ruby hand-rolled validation logic removed". This file is the first half of that sentence. The second half is not done here, and the reason is distribution rather than confidence: bin/validate-intent-go is a build artifact in open-test-intent (.gitignore), /dist/ is ignored there with the comment "this repo does not publish them", and this gem's whole premise is that it has no cross-repo runtime dependency (SCHEMA_PATH). There is no release to depend on. A default-on shell-out would therefore turn every existing user's working linter into specguard-lint: error: … No such file.

So the backend is opt-in and off by default. With SPECGUARD_VALIDATE_INTENT unset, ValidatorBackend.resolve returns nil and CLI runs exactly the code it ran before this file existed — not "a code path that should be equivalent", the same objects.

Why a dedicated seam rather than Configuration

The originating proposal put this on Configuration. That is the wrong object twice over: Configuration is the telemetry config — its only consumers are Formatter and Transport, and CLI does not reference it at all — so threading the lint backend through it would make the linter depend on the telemetry object solely to read one env var. It also memoizes process-wide (SpecGuard::RSpec.configuration), with reset_configuration! existing only for tests, which is a poor fit for something a spec wants to vary per example. CLI takes an env: instead and asks this module, so the seam is one hash away in a test and one method here in production.

The mapping, and the one thing the JSON document does not carry

RunSourceJSON (open-test-intent, cmd/validate-intent/report.go) NORMALIZES the reference's problem and errors into a single errors list, saying so in a comment: "problem and errors are the same thing to a consumer… kind is what tells them apart". Linter::Result keeps them apart, and CLI#report_failure renders them differently — an em dash on one line for a problem, an indented -> line per reason.

So the split has to be reconstructed from kind, which is exactly what the Go comment says kind is for:

schema                     -> reasons: errors
extraction / parse / read  -> problem: errors.first
no-match                   -> problem, and see below

For the three problem kinds an errors list of any length other than 1 is a contract change, not something to paper over by joining the strings: the renderer would silently emit one line where the tool meant several. Runner raises, and the run exits 2.

`no-match`: the gem's arguments are PATHS, the binary's are GLOBS

bin/validate-intent expands its arguments with Python's glob.glob(..., recursive=True); specguard-lint's are paths, checked as given (CLI#select, Scanner.scan_file). Handing a path straight through would silently re-expand it: spec/fixtures/bracket[1]_spec.rb becomes a character class, and a file containing * becomes a wildcard that may match other files. Every path is therefore escaped with ValidatorBackend.escape_glob, the port of Python's glob.escape, so each argument matches exactly the file it names or nothing at all.

"Or nothing at all" is Go's no-match kind, which the gem has no Finding::KIND_* for. It is mapped to Finding::KIND_READ, because that is what it means on the gem's side of the seam: a named path that could not be opened. The classification, the dropped line number, the "N files could not be read" clause of the summary and the exit code all follow the Ruby path exactly. Two things about it are deliberate:

* the reported file is the ORIGINAL path, not the escaped pattern —
`[[]1]` is an artifact of this file and printing it would misreport
what the caller asked for;
* the wording is the gem's own, not Go's `no file(s) match <pattern>`,
which is a statement about a *pattern* — a concept `specguard-lint`
does not have, and whose text would carry the escaped form.

This is one of the ratified differences between the backends, all of which are asserted in spec/specguard/rspec/validator_backend_spec.rb and in open-test-intent's tests/parity/run_ruby_parity.sh. That harness counts them two ways and both are right, so expect it: its header GROUPS them as four mechanisms, (a)-(d), while the section that asserts them is headed "8b. the six enumerated backend differences" and numbers them (i)-(vi) — this one and the acceptance set each cover two cases. Follow the cross-reference and you are reading the six.

The other three mechanisms are below. Only the first two are about TEXT — the backend passing the port's wording through where the Ruby path spells it its own way. The third is not a wording difference at all, which is why this list no longer calls the group "text differences" as an earlier revision did:

* the read-failure tail — see {Scanner.scan_text}, which records why
the gem emits a fixed string where CPython and the port emit a
decoder diagnostic;
* the parse-failure tail — see {Scanner#parse} (1). A payload that
survives normalisation and still is not JSON is described by
whichever JSON parser saw it, so the Ruby path carries Ruby's
`JSON::ParserError#message` and the backend carries CPython's. This
is the most commonly hit of the three: `parse` is one of the three
things the linter exists to report.
* THE ACCEPTANCE SET — see {Scanner#parse} (2), and note that this one
is not a wording difference at all. The two JSON parsers do not
accept the same language: CPython takes non-finite literals, lone
high surrogates and unbounded nesting, and Ruby takes none of the
three. For such a payload the backend does not merely word the
failure differently, it does not have one — it parses the payload and
validates it, so the finding arrives as KIND_SCHEMA against the Ruby
path's KIND_PARSE, and where the payload is schema-VALID the backend
reports nothing at all and the two exit codes disagree.

That last case is the only known input on which the two backends return different verdicts for the same file, and it is ratified rather than closed for reasons Scanner#parse sets out. Nothing here can detect it: a document reporting no findings is exactly what a clean run looks like, so the mapping below is correct and the disagreement is upstream of it.

Everything that can go wrong here is exit 2

A missing binary, a binary that will not execute, a non-zero exit this cannot read a document out of, or unparseable output are all "the linter could not do its job". They must never reach exit 1, which the contract has already spent on "an annotation is malformed" (CLI). Hence ValidatorError, rescued beside UsageError so the message reads specguard-lint: error: … rather than the backstop's internal error:.

Naming what produced the verdicts — and why identity is NOT in that band

The paragraph above is about failures to obtain a VERDICT. The binary's identity is not one, and the distinction is the whole of #identity.

This file already refuses to let "which validator ran" be ambiguous in the two places it can be decided: a named-but-missing binary is a hard exit 2 (Runner#verify!), and a bare command name is refused rather than PATH-resolved (Runner#path_hint) because "a run that succeeded against a different validator … nothing downstream can detect". Both close the hole for binaries that do not resolve. For every binary that DOES resolve, the run was still silent about it: a report produced by the port and a report produced by Linter were the same bytes.

So Runner#verify! asks the binary who it is, once, before anything is selected or scanned — the same fail-early placement as the checks beside it — and Runner#provenance renders the answer for the one stderr line CLI prints per run. Three properties make that safe:

* it is asked ONCE per run, so it cannot become a per-batch cost on a
large audit. Not by memoizing the answer — {Runner#verify!} re-probes
every time it is called, exactly as it re-runs the three file checks
beside it — but because {ValidatorBackend.resolve} is the only thing
that calls {Runner#verify!} and {CLI} resolves once per run;
* the answer is passed through VERBATIM. A line the gem composed about
the binary would be a claim by the gem; the point is to carry the
binary's own statement, which is the only thing that can distinguish
two builds this gem has never heard of;
* a binary that cannot answer still validates. `--version` arrived in
open-test-intent slice 6; an older build reads it as a filename,
reports "no file(s) match" on stderr and exits 1. That must not cost
a verdict, so the probe treats a non-zero exit, empty output, or
output that cannot be rendered as one line as "identity unavailable"
and never as a {ValidatorError}. `SystemCallError` is caught in the
probe rather than left to {Runner#run}'s rescue for the same reason.

"Unavailable" is then reported IN WORDS by Runner#provenance. Dropping the line instead would make a run that could not name its validator look exactly like a run nobody looked at — the silent-omission shape this project keeps naming, arrived at from a third direction.

Asking the answer a question: the schema contract across the seam

The identity above was carried through and rendered, and nothing read it. One token in it is not decoration: open-test-intent's VersionLine (cmd/validate-intent/version.go) ends schema sha256:<64-hex>, the digest of the schema COMPILED INTO that binary, and schema.go's SchemaSHA256 says in as many words why it exists —

"a gem that vendors schema A can be pointed at a binary built when
canonical was B, and all three guards stay green while the two halves
enforce different contracts."

Every drift guard in this ecosystem compares a matched pair inside ONE checkout: schema_test.go digests the Go embed against the Go tree's schemas/, spec/specguard/rspec/schema_packaging_spec.rb digests this gem's vendored copy against this gem's pin, tests/parity/ compares two files in two checkouts. None of them looks at an INSTALLED artifact, and none of them can. The seam is ordinary, not contrived: install.sh fetches a released binary by version with nothing tying that release's vintage to the gem beside it, and ENV_VAR accepts any path on the host.

And on this path the gem does not even read its own schema — CLI#run loads it only when the backend is nil, deliberately, because with the backend on the binary's copy is what governs. So the two contracts never met. Runner#verify! is where they now do, beside the three file checks and before anything is selected or scanned: the same fail-early placement this file already argues for, and for the same reason — a divergence is not a verdict about anyone's annotations.

CARRIED is not ENFORCED, and only one of them is the question

The paragraphs above are how this check was first built, and they compare the wrong digest. --version reports the schema the artifact CARRIES — SchemaSHA256 is a pure fold of the compiled-in bytes — while LoadSchema (cmd/validate-intent/fileio.go) gives a schemas/open-test-intent.v1.json found beside the executable priority over that copy and falls back to it only on ENOENT. --version returns some thirty lines above that decision and never reaches it; the binary's own --help trailer says the digest "is not a claim about what a given run enforced".

So the carried digest is the one answer that cannot settle the question, and asking it fails in BOTH directions:

* a binary whose embedded schema is ours, sitting beside a `schemas/`
file that is not, reports a matching digest — the guard returns
`:matched` and stays silent — and then enforces different bytes. That
is precisely the case this check exists to refuse, passing;
* the mirror: a binary whose embedded schema differs from ours but whose
on-disk schema IS ours gets refused, for a run that would have been
correct. Planting a schema beside the binary is not exotic — it is how
open-test-intent's own `tests/parity/run_parity.sh` operates.

open-test-intent slice 19 added --schema-source, which calls the REAL loader and prints schema <origin> sha256:<hex> for the bytes a verdict run on this host would load. It is the only surface that answers the question, so Runner#verify! asks it — once per run, beside the identity probe — and the comparison uses the ENFORCED digest whenever one comes back, falling back to the carried digest, unchanged, when it does not.

The bands, and the boundaries between them are the whole design:

* ENFORCED AND EQUAL — the run proceeds, and {Runner#provenance} says so
WITHOUT the hedge below, naming the origin the bytes were loaded from.
This arm may state what the run enforces because that is the question
`--schema-source` answers.

* ENFORCED AND DIFFERENT — {ValidatorError}, exit 2, naming BOTH digests
and the origin. This is the one addition to the exit-2 band, and it
belongs there for the reason the band exists: a verdict produced under
a different contract is not a verdict this gem can stand behind, and
nothing downstream can notice — the report is a normal report, the
exit code is a normal exit code, and the findings are whatever the
other contract implies. Note this is the OPPOSITE of the identity
rule above rather than an exception to it: "I could not ask" stays
out of the band, "I asked and the answer was wrong" goes in.

* CARRIED AND EQUAL, CARRIED AND DIFFERENT — the fallback, reached only
when `--schema-source` did not answer, and byte for byte what this
file did before it was asked: the same two outcomes, the same two
messages, and the hedge kept on the matched arm because on that path
it is still true.

* NOT REPORTED — never a refusal, in any of its three shapes: a
pre-slice-17 build whose `--version` carries no digest token, a binary
that could not answer `--version` at all, and this gem being unable to
read its own vendored schema. Each says so IN WORDS in its own
wording, because "could not check" and "checked and clean" are two
different statements and a checker owes both.

The rule the second probe inherits, and the one it cannot

A binary predating slice 19 reads --schema-source as a filename, writes "no file(s) match" to stderr and exits 1; a binary whose schema exists and cannot be loaded exits 2 with the "could not load schema" diagnostic that belongs to it, which #check reaches a moment later anyway. Both are the identity probe's rule unchanged — a non-zero exit, empty output or output this cannot read is "unavailable" and never a ValidatorError — so with the flag absent the run is byte-identical to the one before this existed.

That rule is also why SCHEMA_SOURCE_PATTERN is pinned against RECORDED output of the real binary in spec/fixtures/validator/schema-source-probes.json rather than against a hand-written line. A parse bug does not announce itself here: every miss reads as "this binary is too old", the guard silently reverts to comparing the carried digest, and the run goes green — the exact defect this file is closing, re-created inside the fix.

What two probes CANNOT promise is what one probe did. #verify_schema_contract! reads #identity rather than re-asking, so the digest it compared and the name in the provenance line come from the same process; a second probe is a second process, so the enforced digest carries no such guarantee. It says what a run STARTED at that moment would load, which is the strongest thing any answer to this question can say — the schema beside a binary can change between two lines of a shell script — and the identity/carried pair keeps the promise it always had.

That third shape is the one judgment call here, so it is recorded rather than left to be re-derived. Schema.load's precedent is that an unreadable SCHEMA_PATH is exit 2 — but that is a schema the run is about to ENFORCE, and this one is not: CLI#run skips the load on this path precisely so an unrelated packaging accident cannot fail a run that never reads it, and making the digest fatal here would re-introduce exactly the dependency that comment removed. An unreadable vendored copy is also not what the exit-2 arm is for — divergence is a POSITIVE finding, two digests that differ, and a missing operand is not a difference. So it lands in the third band and is said out loud. (Hence Digest::SHA256.file rather than Schema.load: this needs bytes, not a parsed and validated document, so only an unreadable file can fail it and a schema this run does not use cannot fail it twice.)

The digest is computed from SCHEMA_PATH at runtime and never written down here. A constant would be a fourth copy of a hex string that already exists in three places, drifting independently of the file it claims to describe — which is the precise failure this whole check was added to detect, re-created inside the detector.

Defined Under Namespace

Classes: Runner

Constant Summary collapse

ENV_VAR = Set it to a validate-intent binary to route linting through the Go port. Blank or unset means the Ruby path, which is the default and the only behaviour any existing user has.

"SPECGUARD_VALIDATE_INTENT"

Class Method Summary collapse

.escape_glob(path) ⇒ String
Python's glob.escape for POSIX paths: wrap each of *, ? and [ in a character class, which makes it match itself literally. ] is not escaped and does not need to be — outside a class it is already a literal — and CPython does not escape it either.
.resolve(env: ENV) ⇒ Runner^?
Nil when the backend is not requested.

Class Method Details

.escape_glob(path) ⇒ `String`

Python's glob.escape for POSIX paths: wrap each of *, ? and [ in a character class, which makes it match itself literally. ] is not escaped and does not need to be — outside a class it is already a literal — and CPython does not escape it either.

Parameters:

path (String)

Returns:

(String) —
a glob pattern matching exactly path



338
339
340

# File 'lib/specguard/rspec/validator_backend.rb', line 338

def self.escape_glob(path)
  path.gsub(/([*?\[])/, '[\1]')
end

.resolve(env: ENV) ⇒ `Runner`^?

Returns nil when the backend is not requested.