Class: SpecGuard::RSpec::Transport

Inherits:
Object
  • Object
show all
Defined in:
lib/specguard/rspec/transport.rb

Overview

The one HTTP call this gem makes: POST <endpoint>/api/v1/ingest, carrying a whole run.

Why this returns a result instead of raising

SpecGuard::RSpecFormatter's never-block-CI guard is a rescue around each hook, and a rescue is structurally blind to the failure that matters most here: Net::HTTP hands back Net::HTTPUnauthorized as an ordinary return value. A wrong API key therefore raises nothing, warns nothing and logs nothing — the run's telemetry disappears in complete silence, which is precisely the outcome the client-gem spec's "if the API key is wrong … it logs a warning to stderr" forbids.

So the two failure families are made the same shape rather than left to two different mechanisms: #deliver answers a Result for a non-2xx response and a Result for a raised exception, and the caller has one thing to check. Nothing escapes this class except Interrupt, SignalException and SystemExit — Ctrl-C must stay Ctrl-C.

One request per process, gzipped once it is big enough

A run goes in a single POST, and the body is compressed above a size threshold. Both halves are decisions rather than defaults, and the roadmap asked for them to be made deliberately rather than discovered in production, so they are written down here.

Batching: no, and this is settled

* `Ingest::Payload` derives `total_specs_count` from the specs of *that*
request. Splitting a run across N POSTs with no way to say they are
one run would produce N `TestRun` rows with a split denominator,
corrupting the headline annotated-ratio metric.
* That is qualified rather than absolute, and the qualification is
`ci_run_id` + `shard_id`. The platform folds every POST carrying the
same run id onto one `TestRun`, keyed by shard so a slice that arrives
twice replaces itself, which is what makes a *sharded* run — N
processes, N POSTs, one run — land as one row. So the rule this class
keeps is narrower than it was: **one process sends one request.**
* Batching a single process's own run into several POSTs stays wrong,
and not only for the denominator: every part would carry the same
`shard_id`, so the parts would overwrite one another and the row would
keep only the last. Fixing that means a new part-of-a-shard concept on
the platform, which is a schema change bought to solve a problem that
compression already solved.

Streaming: no, for the same reason, and the reason is measured

The pressure that made batching and streaming look necessary was size, and size is now a number rather than a worry. Measured by running a real 200-file / 20,000-example suite through this gem's own formatter, half of the examples annotated:

identity   7,354,782 bytes   7.01 MiB    needs 5.9 Mbit/s to write in 10s
gzip         346,206 bytes   0.33 MiB    needs 0.3 Mbit/s
ratio           21.2x        95.3% saved       60 ms to compress

Uncompressed, that body has to be written inside Configuration::DEFAULT_TIMEOUT_SECONDS (10). At 5 Mbit/s of uplink it takes 11.8s and fails as a write_timeout; the run then lands in log/test_results.jsonl and the platform never sees it. Compressed, the same run takes 0.55s on that link and 2.8s on a 1 Mbit/s one. The 60 ms spent compressing is noise against a suite that took minutes.

Two honesties about that ratio. It is below the 35x this change was proposed on, because SPGD-159 has since added id and spec_file_path to every row — re-measure rather than quote, is the lesson. And it is probably above what a real suite gets: synthetic example names repeat more than human ones do, so treat 21x as the optimistic end. Nothing here depends on the exact figure. Even a pessimistic 5x moves the 20k case from "cannot ship on a slow link" to "ships with room to spare", and 0.33 MiB is not a payload anyone needs to chunk.

So: compression yes, batching and streaming no. Recorded rather than left implicit — an undocumented decision is one that gets re-opened in six months by someone who cannot tell it was ever made.

Why a threshold rather than always

Below GZIP_THRESHOLD_BYTES the round trip buys nothing worth paying for in opacity: a small run stays identity-encoded, so it is still readable with curl and tcpdump, and the local-file and stub-server paths still show a human a JSON body. Compression is for the case that could not ship at all, not a uniform policy.

Compression also sits inside the never-block-CI contract. A Zlib failure falls back to the identity body — which the platform still accepts — rather than raising: a run must not be lost to an optimisation.

The version floor this creates, which is a deploy order and not a merge order

Sending Content-Encoding: gzip requires a platform that can inflate one, and that arrived with GzipRequestBody (SPGD-175). "Merge the platform PR first" is the half of this that is easy to say and is not sufficient: merge order only settles the source trees. What this gem actually talks to is a deployment.

So a gem at this version pointed at an installation deployed before GzipRequestBody will 400 every run over GZIP_THRESHOLD_BYTES — the inflater is not there, the body reaches the JSON parser still gzipped, and Api::V1::IngestsController refuses it. #deliver answers Result(outcome: :rejected, code: 400); there is no retry-as-identity, so the formatter falls back to log/test_results.jsonl and the run is lost to the platform. That is precisely the failure this change exists to close, reintroduced for precisely the large suites it targets — and it is silent apart from one stderr line, because CI still goes green.

Written down rather than fixed, deliberately. A 400-triggered retry with the identity body would paper over it, but it doubles the request count on every genuinely-malformed payload and makes a client-side bug look like a flake; that is a contract decision, not a detail to slip in here. The honest statement of the constraint is a version floor: this gem requires a platform deployment that includes GzipRequestBody.

Defined Under Namespace

Classes: Result

Constant Summary collapse

PATH =

config/routes.rb mounts post "ingest" under the /api/v1 scope. Part of the platform's contract, so it is not configurable — the endpoint setting is the installation's address and nothing more.

"/api/v1/ingest"
CONTENT_TYPE = 
"application/json"
USER_AGENT = 
"specguard-rspec/#{SpecGuard::RSpec::VERSION}"
CONTENT_ENCODING = 
"gzip"
GZIP_THRESHOLD_BYTES =

Bodies at least this large are gzipped; smaller ones go identity. See the class comment for why this is a threshold and not a switch.

256 KiB is roughly where the round trip starts paying — a few thousand examples. It is a judgement call, not a measured optimum, and the only thing that depends on the exact number is how large a run has to be before curl stops showing you a readable body.

256 * 1024

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(endpoint:, api_key:, timeout: Configuration::DEFAULT_TIMEOUT_SECONDS) ⇒ Transport

Returns a new instance of Transport.

Parameters:

  • endpoint (String, nil) —

    the installation's base URL. Any trailing slashes are dropped; a path prefix is preserved, so an installation behind https://tools.example.com/specguard works.

  • api_key (String, nil) —

    sent verbatim as a Bearer token.

  • timeout (Numeric, String, nil) (defaults to: Configuration::DEFAULT_TIMEOUT_SECONDS) —

    seconds. Anything that is not a positive finite number falls back to the default rather than raising: a typo in SPECGUARD_TIMEOUT must not be able to fail a suite.



187
188
189
190
191
 
# File 'lib/specguard/rspec/transport.rb', line 187

def initialize(endpoint:, api_key:, timeout: Configuration::DEFAULT_TIMEOUT_SECONDS)
  @endpoint = endpoint
  @api_key = api_key
  @timeout = sanitize_timeout(timeout)
end

Instance Attribute Details

#timeout ⇒ Object (readonly)

Returns the value of attribute timeout.



193
194
195
 
# File 'lib/specguard/rspec/transport.rb', line 193

def timeout
  @timeout
end

Instance Method Details

#deliver(payload) ⇒ Result

Returns never nil, never raised through.

Parameters:

  • payload (Hash) —

    the run, as SpecGuard::RSpecFormatter#payload assembles it. Sent as-is: its key names are already the platform's ingest contract, and reshaping it here would put the wire format two files away from the code that decides it.

Returns:

  • (Result) —

    never nil, never raised through.



211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
 
# File 'lib/specguard/rspec/transport.rb', line 211

def deliver(payload)
  response = post(JSON.generate(payload))
  code = response.code.to_i

  return Result.new(outcome: :success, code: code) if response.is_a?(Net::HTTPSuccess)

  Result.new(outcome: :rejected, code: code)
rescue ScriptError, StandardError => e
  # Connection refused, DNS failure, TLS failure, open/read timeout, a
  # malformed endpoint — one family, one shape. `ScriptError` is in the
  # list for the same reason the formatter's guard names it: an autoload
  # blowing up under `net/http` is not a `StandardError`, and a bare
  # rescue would let it escape and take the suite's exit code with it.
  Result.new(outcome: :failed, error: e)
end

#uri ⇒ URI::HTTP

Where this transport would POST.

Returns:

  • (URI::HTTP)

Raises:

  • (ArgumentError) —

    when the endpoint is missing or is not an http(s) URL. Raised rather than returned because #deliver converts it into a Result like every other failure, and a caller asking for the URI directly wants to know.



202
203
204
 
# File 'lib/specguard/rspec/transport.rb', line 202

def uri
  @uri ||= build_uri
end