Class: Rubino::Compression::LogCompressor

Inherits:
Object
  • Object
show all
Defined in:
lib/rubino/compression/log_compressor.rb

Overview

Deterministic, ML-free compression of COMMAND OUTPUT (test runs, linters, build logs, long shell dumps) for the model-facing channel. Unlike the Ruby SKELETON path (whole-file reads), this is the high-ROI channel: the agent reads command output WHOLE, and the signal — failures + the summary tally — is a tiny fraction of the bytes. We keep every error/failure and the final tally VERBATIM and drop the passing-test / progress noise.

The FIDELITY INVARIANT is the whole contract: a line that names an ERROR / FAIL / Failure, and the final N examples, M failures summary, MUST survive compression. We may drop a passing dot, an INFO line, a green describe header — NEVER a failure descriptor. The measurement (eval/) and the spec both assert this directly.

Pure regex + counting, no AST, no gem. Small outputs (< min_lines) pass through unchanged — the marker indirection isn't worth it.

Defined Under Namespace

Classes: Config, Line

Constant Summary collapse

ERROR_RE =

--- Severity lexicon (word-boundary anchored so error_count doesn't trip on a passing-context line and passed doesn't read as pass). ---

/\b(?:error|errors|fail|failed|failure|failures|fatal|exception|panic|assert(?:ion)?)\b/i
WARN_RE = 
/\b(?:warn|warning|warnings|deprecat\w+|pending|skipped|todo)\b/i
INFO_RE = 
/\b(?:info|debug|pass|passed|passing|ok|success|done|examples?)\b/i
FAILURE_SHAPE_RE =

In a STRUCTURED test runner the per-test progress section names tests ("handles an error", "fails fast") whose keywords are NOT failures — the real failures live in dedicated report SHAPES. These match those shapes so the 8k-line green progress section can't masquerade as 364 failures. rspec: Failure/Error:, N) <desc>, rspec ./path:NN rerun list pytest: FAILED path::test, E <assert>, > assert ... jest: ✕ test, ● Component › test cargo: test name ... FAILED, ---- name stdout ---- rubocop path:line:col: C: Offense

%r{
  \A\s*\d+\)\s                          # rspec/cargo numbered failure
  | \bFailure/Error:                    # rspec body anchor
  | \A\s*rspec\s+['"]?\.?/?\S+:\d+      # rspec rerun line
  | \A\s*FAILED\b                       # pytest / generic
  | \A\s*E\s{2,}\S                       # pytest assertion line
  | \A\s*[✕✗✘×]\s                       # jest/mocha fail mark
  | \A\s*[●•]\s.*›                  # jest failure header (› )
  | \.{3}\s*FAILED\s*\z                  # cargo `test x ... FAILED`
  | \A----\s.*\bstdout\b                 # cargo failure capture header
  | \A\S+:\d+:\d+:\s+[A-Z]:\s            # rubocop offense (path:l:c: C:)
  | \A\s*(?:error|panic)\[             # rust/compiler `error[E…]`
}x
STACK_RE =

Stack-trace / backtrace frame: rspec # ./spec/...:NN, a bare from path:line:in, a path:line:in Ruby frame, or a at File.fn / pytest File "x", line N. Indented continuation lines of a trace.

%r{
  \A\s*(?:\#\s+)?(?:from\s+)?[^\s:]+\.\w+:\d+(?::in\b)?   # path:line[:in]
  | \A\s*at\s+\S+                                         # JS/Java at frame
  | \A\s*File\s+"[^"]+",\s+line\s+\d+                     # pytest frame
}x
SUMMARY_RE =

The final tally / framing lines that MUST survive: rspec N examples, M failures, Finished in ..., the Failures: header, rubocop's NN files inspected, MM offenses, pytest's === N failed ===.

/
  \b\d+\s+examples?\b
  | \bFinished\sin\b
  | \A\s*Failures:\s*\z
  | \b\d+\s+files?\s+inspected\b
  | \b\d+\s+offenses?\b
  | ^={3,}.*\b(?:failed|passed|error)\b
  | \b\d+\s+(?:passed|failed|error)\b
/xi
STRUCTURED =

STRUCTURED runners report failures in dedicated shapes; the green progress section's keyword-bearing test names are NOT failures. Generic logs have no such structure, so keyword severity is all we have.

%i[rspec pytest jest cargo].freeze

Instance Method Summary collapse

Constructor Details

#initialize(config) ⇒ LogCompressor

Returns a new instance of LogCompressor.



99
100
101
 
# File 'lib/rubino/compression/log_compressor.rb', line 99

def initialize(config)
  @cfg = config.is_a?(Config) ? config : Config.from(config)
end

Instance Method Details

#compress(text) ⇒ Object

Returns a CompressionResult. applied? == false means "send the original".



104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
 
# File 'lib/rubino/compression/log_compressor.rb', line 104

def compress(text)
  original_bytes = text.bytesize
  raw = text.split("\n", -1)
  # split("\n", -1) leaves a trailing "" for a final newline; drop it so
  # the line count and the rebuilt output match the input.
  raw.pop if raw.last == "" && text.end_with?("\n")

  return CompressionResult.noop(strategy: :too_small) if raw.length < @cfg.min_lines

  @format = detect_format(raw)
  lines = classify(raw)
  select!(lines)
  kept = lines.select(&:kept)

  return CompressionResult.noop(strategy: :insufficient_saving) if kept.length >= raw.length

  out = render(lines)
  build_result(out, original_bytes)
end