Class: SpecGuard::RSpec::IngestCLI

Inherits:

Object

• Object
• SpecGuard::RSpec::IngestCLI

Defined in:

lib/specguard/rspec/ingest_cli.rb

Overview

Replays a saved run: reads a log/test_results.jsonl back and re-delivers each line through the Transport this gem already ships.

The file is not a new wire format, and that is the whole design

SpecGuard::RSpecFormatter#deliver hands the same Hash to Transport#deliver and to its own append, and each of them calls JSON.generate on it once. A line in the sink is therefore byte-for-byte the body the endpoint refused, so replaying it needs no parser, no migration and no versioning — only something that reads a line back and POSTs it. That is all this class is.

The exit contract, which is CLI's reasoning transferred verbatim

0  every line was accepted
1  at least one line was refused by the endpoint
2  this tool could not do its job

Ruby exits 1 for an uncaught exception and for an uncaught OptionParser::InvalidOption, and 1 is already spent here on "the endpoint said no". Left alone, specguard-ingest --dry-runn and an unreadable file would both report that the platform refused a run it was never offered — a tool failure wearing the costume of a content failure, which is the defect CLI exists to keep out of the linter. So #run rescues and returns rather than exits, and EXIT_REFUSED is produced in exactly one place — #exit_code, over a :refused LineResult — so it means that and nothing else.

Interrupt, SignalException and SystemExit are deliberately not caught. Ctrl-C halfway through a 40-line file must stay Ctrl-C.

Where the line between 1 and 2 actually falls

Not where Transport::Result draws it. That struct answers :rejected for any non-2xx, and "a non-2xx came back" is not the same claim as "the endpoint read this payload and judged it" — so this class redraws the line rather than inheriting it.

The platform emits exactly one verdict about a payload: Api::V1::IngestsController#create reaches render_bad_request(payload.errors), a 400. Every other non-2xx is the endpoint declining to look at the run — a 401 is answered by authenticate_api_key!'s before_action before the action runs at all, so Ingest::Payload is never even constructed — or the endpoint not being there (404), or the platform failing (429, 5xx). Nothing was stored in any of them, and none of them is a statement about anyone's suite. They are :undelivered, and they are a 2.

:failed — connection refused, DNS, TLS, a read timeout — is a 2 for the same reason arrived at from further away: nothing was delivered and no verdict exists. A socket error reported as a 1 would be this tool telling an operator their run is bad on the strength of a broken pipe; a 404 reported as a 1 would be it telling them the same thing on the strength of a typo in SPECGUARD_ENDPOINT, while its own advice line says to go fix that variable.

That last case is the one that fixes the rule in place: an unset SPECGUARD_ENDPOINT is a 2 from #build_transport, so an endpoint set to the wrong URL must be a 2 as well. Same operator mistake, same fix, and the case where this tool knows more must not report worse.

2 also dominates: a file where line 3 was refused and line 7 never arrived exits 2, because the second fact is the one that leaves work undone. Both are printed either way — the exit code chooses what to shout, never what to say.

It re-delivers EVERY line, and will not guess which ones were failures

The sink is not a failure log. #deliver writes to it when a delivery failed and when no API key was set at all (return append(data) if blank?(configuration.api_key)), and the two are indistinguishable on the line — there is no field to tell them apart. A developer's local file is therefore a file of ordinary laptop runs, and pointing this command at it sends all of them.

A heuristic here would be worse than the gap: it would be this tool guessing at somebody's intent from data that does not carry it, and being confidently wrong about which runs reach the platform. So the tool is explicit and user-initiated instead, --help says so in as many words, and no line is filtered.

--from-line and --lines are the two things that narrow the set, and both narrow it by numbers the user typed after reading the report — which is the opposite of a guess. They exist because LineResult's numbering is only half of criterion 3: a report that says line 7 was refused is worth little if acting on it means re-sending lines 1 through 6, and re-sending a line that carries no ci_run_id is not free — RunRecorder has no identity to fold it onto, so it becomes a second row.

Why a suffix was not enough, and why --lines is not a heuristic

--from-line N can express only a suffix, and the set a report points at is a suffix at most once. The sink is append-only and mixes both sources, so ordinary keyless laptop runs keep landing after the CI failures somebody wants to replay. Worse, CONTENT_REFUSAL_CODES is a content verdict — a 400 will be refused every time it is offered — so a 400 sitting at line 3 of a 40-line file is a line no suffix can step over, and the file can never be replayed to completion.

--lines 3,7,12-15 is the same category of thing as --from-line: a set the user typed, over the file's own numbering, after reading --list. Nothing about the line's content is consulted, which is the line the paragraph above draws and this flag stays on the explicit side of.

The two are refused together — UsageError, a 2. They answer the same question, and an intersection would silently drop a number the user typed (--from-line 5 --lines 3,7 delivers only 7, and the 3 vanishes without a word). Quietly narrowing what it was asked for is the failure this whole file is arranged against, so the combination is refused rather than resolved. Carving the file up with sed is not the alternative either: a carved temp file renumbers, and the whole value of resuming from a report is that line 12 is still line 12.

A repeated selector is the other half of that question and gets the opposite answer deliberately, not by omission: --lines 1,2 --lines 4 delivers line 4, and --from-line 2 --from-line 5 starts at 5, both by last-wins. The reason the cross-flag case is refused does not reach this one. Refusing --from-line with --lines is about two different flags answering one question, where combining them yields a set smaller than either one names and the user cannot see which of their numbers went. A repeat is one flag answering its own question twice, and the later answer replaces the earlier rather than intersecting it — the delivered set is exactly the last one typed, which is the one thing on the command line that is unambiguously current. It is also what makes a selector overridable at all: a wrapper script or shell alias that bakes in --lines is corrected by appending a new one, and refusing the repeat would take that away for no gain in clarity. Last-wins is OptionParser's convention and the shell's; this file adopts it on purpose and pins it in the spec, so it is a decision rather than a default nobody looked at.

--list, which is what makes "check the file first" an instruction

Having refused to guess for the user, this command owes them what they need to decide. #list prints one row per line off the fields the envelope already carries — branch, commit_sha, ci_run_id or its absence, how many examples, how long — and delivers nothing. Reading the file by hand is not the alternative: one line is one whole run, and at this project's design point that is megabytes of JSON on a single physical line.

It short-circuits ahead of #build_transport, deliberately, and that ordering is the load-bearing part. build_transport raises for a blank SPECGUARD_ENDPOINT or SPECGUARD_API_KEY, and the file that most needs checking is the one written because no API key was set (Formatter — return append(data) if blank?(configuration.api_key)). Requiring a key to look at the file would withdraw the instrument in exactly the situation that produces the hazard.

Listing delivers nothing, so it can never be a content verdict: it answers 0 or 2 and never routes through #exit_code, which stays the one place EXIT_REFUSED is produced.

What it will not claim

The natural question about a replay is whether a line folded onto an existing run or created a new one, and the platform does not answer it: Ingest::RunRecorder#record find-or-creates by ci_run_id and the 202 body carries no created-versus-updated flag. So this class reports what it can see — the test_run_id that came back, and whether the line carried a ci_run_id at all — and states folding only where two lines observed it: same ci_run_id in, same test_run_id out is one row, not an inference about one. See #folded_runs.

--json, and the one thing the human report cannot say

A 400 is the only permanent verdict here (CONTENT_REFUSAL_CODES), so the only way to land a refused line is to learn which specs the platform objected to and fix the payload. It names every one of them — one error per bad spec — and Transport::Result#reason renders three, truncated to 300 characters, because it exists for the one stderr line an in-run CI warning is allowed. On a systemic client bug over a 20,000-example suite, the command whose entire job is to fix and re-send a refused run was showing three of twenty thousand reasons.

--json is the second channel that cap's own grounds hand over, and it is a RENDERER: stdout carries one document instead of the human report, in both modes, over the same LineResults, the same counts, the same groupings and the same #exit_code. Nothing about what this command decides moves — the cap, #reason and the formatter's warning are untouched, and the default output is byte-identical (pinned in spec/specguard/rspec/regression_targets_spec.rb). See IngestReporter for the document, and for which runs emit one.

Defined Under Namespace

Classes: Folding, LineResult, ListedLine, Options, Source

Constant Summary

BANNER =

"Usage: specguard-ingest [options] <file>"

DESCRIPTION =

Shown by --help. The second paragraph is the one that has to be there: the sink mixes failed deliveries with ordinary keyless local runs and nothing on the line tells them apart, so a developer must not be able to discover only afterwards that they pushed their laptop's history.

<<~TEXT.freeze
  Re-delivers a saved run to SpecGuard's ingest endpoint. <file> is a
  log/test_results.jsonl written by the RSpec formatter — one whole run per
  line, byte-for-byte the body the endpoint was offered.

  EVERY line in <file> is delivered — or, when you narrow it, every line
  --from-line or --lines names. The formatter writes to this file both
  when a delivery failed and when no API key was configured at all, and
  the two are indistinguishable on the line, so a laptop's file is a file
  of ordinary local runs and all of them will be sent. Nothing is
  filtered and nothing is guessed at.

  So check the file first: --list prints one row per line — branch, commit,
  ci_run_id or its absence, how many examples, how long — and delivers
  nothing at all. It needs no SPECGUARD_ENDPOINT and no SPECGUARD_API_KEY,
  because the file most worth checking is the one written when no API key
  was set. It composes with --from-line and --lines, so you can list the
  exact set you are about to send.

  Each line is delivered once, with no retry, and reported by its line
  number in <file>. Use --from-line to resume a file that was only partly
  accepted, or --lines to send an arbitrary set of them — 3,7,12-15 — over
  that same numbering, instead of re-sending all of it. Both narrow the
  same file, so give one or the other and never both.

  Reads SPECGUARD_ENDPOINT, SPECGUARD_API_KEY and SPECGUARD_TIMEOUT.

  --json replaces the human report with one JSON document on stdout, in
  both modes. It carries every line's status, the endpoint's HTTP code and
  the FULL list of reasons a refusal named — the human line has room for
  three of them — plus the same summary counts and folding observations.
  Warnings stay on stderr, and a run that never got as far as reading
  <file> writes no document at all.

  Exit codes:
    0  every line was accepted — or, with --list, the file was listed
    1  at least one line was refused by the endpoint — it read the payload
       and said no (HTTP 400). Unreachable with --list, which delivers
       nothing and so can never carry a verdict about a run
    2  this tool could not do its job — bad flags, no endpoint or API key,
       an unreadable file, an unparseable line, a delivery that never
       reached the endpoint, or one the endpoint answered without ever
       reading it (401, 404, 429, 5xx — nothing was stored, so none of
       them is a verdict about your run). With --list the only reachable
       2s are a bad flag and a file that cannot be read: listing needs no
       credentials, and an unparseable line becomes a row in the listing
       rather than an exit code
TEXT

EXIT_OK =

Every line in the file was accepted by the endpoint — including the vacuous case of a file with no lines in it, which is loud on stderr for exactly the reason specguard-lint's empty selection is: the contract has no code for "there was nothing to do", so the warning carries it.

0

EXIT_REFUSED =

At least one line was refused. The only code that says something about the content of a run, and the only path to it is a non-2xx whose status is in CONTENT_REFUSAL_CODES.

1

EXIT_MISUSE =

This tool could not do its job: bad flags, no endpoint or API key configured, a file it could not read, a line it could not parse, a delivery that never reached the endpoint at all, or one the endpoint answered without ever reading — a 401, 404, 429 or 5xx.

2

CONTENT_REFUSAL_CODES =

The status codes that carry a verdict about the payload — which on this platform is exactly one.

Api::V1::IngestsController#create reaches render_bad_request(payload.errors) and nothing else. A 401 comes from authenticate_api_key!'s before_action before the action runs, so Ingest::Payload is never constructed and no verdict about the run exists to report; a 403, 404, 429 or 5xx never gets as far as a body either. See the class comment for what that costs an operator when it is got wrong.

A list of one rather than code == 400, so a platform that grows a second verdict is a one-line change here — and deliberately not pre-seeded with a 422 the platform does not send. An unlisted refusal is reported as :undelivered, which under-claims, and under-claiming is the safe direction: it sends an operator to look at their setup rather than at a suite that was never judged.

[400].freeze

LINE_SPEC_ENTRY =

One entry of a --lines spec: 12 or 12-15, and nothing else. No sign, no open end, no whitespace inside — #parse_line_set strips each entry before matching, so 3, 7 is fine, but 12- and 5 - 7 are not near-misses to be repaired, they are typos to be reported. This is the Integer-coercion rationale on --from-line applied to a richer grammar: a spec that half-parses would silently deliver the wrong set, and delivering the wrong set is the one outcome a selector exists to prevent.

/\A(\d+)(?:-(\d+))?\z/

STATUS_LABELS =

{
  accepted: "accepted",
  refused: "refused",
  undelivered: "not delivered",
  unparseable: "unparseable"
}.freeze

Instance Method Summary

• #initialize(stdout: $stdout, stderr: $stderr, env: ENV) ⇒ IngestCLI constructor

  A new instance of IngestCLI.

• #run(argv) ⇒ Integer

  0, 1 or 2 — never anything else, and never by letting an exception reach the shell.

Constructor Details

#initialize(stdout: $stdout, stderr: $stderr, env: ENV) ⇒ IngestCLI

Returns a new instance of IngestCLI.

# File 'lib/specguard/rspec/ingest_cli.rb', line 372

def initialize(stdout: $stdout, stderr: $stderr, env: ENV)
  @stdout = stdout
  @stderr = stderr
  @env = env
end

Instance Method Details

#run(argv) ⇒ Integer

Returns 0, 1 or 2 — never anything else, and never by letting an exception reach the shell.

Parameters:

• argv (Array<String>)

Returns:

• (Integer) —

  0, 1 or 2 — never anything else, and never by letting an exception reach the shell

# File 'lib/specguard/rspec/ingest_cli.rb', line 381

def run(argv)
  options = parse_options(argv)
  return EXIT_OK if options.nil? # --help / --version already printed

  # Ahead of `build_transport`, and that is the whole point of the branch
  # being here rather than after it. Listing sends nothing, so it needs no
  # endpoint and no key — and the file that most wants looking at is the
  # one the formatter wrote *because* no key was set. A listing that
  # demanded credentials would be unavailable in exactly the case it
  # exists for.
  return list(options) if options.list

  # Before the file is opened, deliberately. "There is nowhere to send
  # this" is the earlier question — which lines to send does not matter
  # when nothing is going to accept them — and asking it first means an
  # unconfigured run reads its one real problem instead of a complaint
  # about a path that was never the point.
  transport = build_transport

  source = read_source(options)
  results = source.lines.map { |number, text| deliver_line(number, text, transport) }

  report(source, results, json: options.json)
  exit_code(results)
rescue UsageError => e
  @stderr.puts "specguard-ingest: error: #{e.message}"
  EXIT_MISUSE
rescue ScriptError, StandardError => e
  # The backstop that makes exit 1 mean one thing. Anything reaching here
  # is a bug in this tool, not a verdict from the endpoint about anyone's
  # run, so it is a 2 and it says so in those words.
  @stderr.puts "specguard-ingest: internal error: #{e.class}: #{e.message}"
  EXIT_MISUSE
end