Class: Rubino::Compression::JsonCompressor

Inherits:
Object
  • Object
show all
Defined in:
lib/rubino/compression/json_compressor.rb

Overview

Deterministic, ML-free compression of a whole-document JSON tool output (a curl | jq, kubectl get -o json, gh api, docker inspect, aws --output json dump, or an MCP/custom-tool JSON result). Modelled on headroom's "SmartCrusher": a LOSSLESS schema-fold first, a LOSSY row selection only as a fallback, and a sentinel marking what was dropped.

The high-ROI case (headroom's ~90%) is an ARRAY OF UNIFORM OBJECTS — the same keys repeated on every element. JSON spends most of its bytes on those repeated key names; we emit the keys ONCE as a header line and one compact val | val | … row per item. That fold is LOSSLESS per item.

Stages, in order (mirrors crusher.rs):

1. parse — JSON.parse the (stripped) text. Not a JSON array/object ⇒
 :not_json signal, the router falls through (this is NOT our content).
2. size gate — below `min_items` array elements / `min_lines` text lines
 there is nothing worth the pointer indirection ⇒ passthrough.
3. ARRAY of mostly-uniform objects:
   a. LOSSLESS schema-fold (header + compact rows). Ship if ≥ min_saving.
   b. LOSSY fallback (only if the fold didn't save enough AND the array
      is large): keep MUST-KEEP rows — error/exception-bearing items
      (fidelity: errors always survive), statistical outliers (a numeric
      field > outlier_sigma σ from the mean), and the first+last item
      (boundary). Dropped rows collapse to one `{"_elided": N}` sentinel.
      (headroom's query-anchors are skipped — there is no query here.)
4. SINGLE large object: keep every key + the whole structure; elide only
 very large STRING values (> max_string_chars) behind a short
 `"<elided N chars>"` placeholder. Never drops keys.
5. saving guard — only apply when the result is ≥ min_saving smaller;
 else byte-identical passthrough. Small JSON the model wants verbatim
 stays untouched.

Any parse/strategy error ⇒ noop. Compression must never break a tool call.

Defined Under Namespace

Classes: Config

Constant Summary collapse

ERROR_KEYS =

Keys whose presence (or whose value, when stringy) marks an item as error-bearing — such items always survive the lossy fallback.

%w[error errors exception err fault failure failures].freeze
ERROR_MARKERS =

Value substrings (case-insensitive) that also mark an item as error-bearing, scanned across the item's stringified scalar fields.

/\b(?:error|exception|fail(?:ed|ure)?|fatal|panic|traceback)\b/i

Instance Method Summary collapse

Constructor Details

#initialize(config) ⇒ JsonCompressor

Returns a new instance of JsonCompressor.



59
60
61
 
# File 'lib/rubino/compression/json_compressor.rb', line 59

def initialize(config)
  @cfg = config.is_a?(Config) ? config : Config.from(config)
end

Instance Method Details

#compress(text) ⇒ Object

Returns a CompressionResult. applied? == false ⇒ "send the original". On non-JSON content the strategy is :not_json so the router can tell a "this isn't mine, fall through" from a "JSON but not worth it" noop.



66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
 
# File 'lib/rubino/compression/json_compressor.rb', line 66

def compress(text)
  original_bytes = text.bytesize
  data = parse(text)
  return CompressionResult.noop(strategy: :not_json) if data == :not_json

  out =
    case data
    when Array then compress_array(text, data)
    when Hash  then compress_object(text, data)
    end
  return CompressionResult.noop(strategy: :too_small) if out.nil?

  build_result(out, original_bytes)
rescue StandardError
  CompressionResult.noop(strategy: :parse_error)
end