Module: Foam::Otel::Redaction

Defined in:
lib/foam/otel/redaction.rb

Overview

The central redaction engine: the always-on CREDENTIAL FLOOR plus the customer's opt-in key lists (holistic redesign 2026-07-26, amended the same day by the credential-floor fleet ruling — docs/decisions/credential-denylist-design.md).

THE FLOOR (unconditional, no off switch): every attribute/field whose NAME is on the frozen CREDENTIAL_KEY_DENYLIST (constants.rb) — including the semconv http.request,response.header. forms of the CREDENTIAL_HEADER_DENYLIST — masks in FULL to the literal [REDACTED], in every signal, at every depth this engine walks, before the customer's lists run. Matching is canon-exact (lowercase + dash→underscore fold, design §1.0/§2.1): NEVER substring — "authorization" does not mask "authorization_url"; "secret" does not mask "secretary". The floor consults the module-level frozen constant directly (never Config), so no configuration path can disable, shrink, or re-spell it.

ABOVE the floor, redaction stays fully opt-in: with neither redact_keys nor redact_pii_keys set (or both empty) every OTHER value is captured RAW — no value-pattern auto-masking. Customer redaction runs ONLY over the keys the customer enumerates, ADDITIVE on top of the floor:

* redact_keys     — tail-mask matching scalar values (shape-preserving,
                  e.g. ********f456); non-scalars redact in full.
* redact_pii_keys — erase matching values in full ([REDACTED]).

CUSTOMER matching is SUBSTRING, case-insensitive, over the lowercased key (documented Ruby behavior, unchanged). The FLOOR deliberately does NOT ride that matcher (design §2.4 — substring would over-match): it uses its own canon-exact predicate, floor_kind, below. A floor match is TERMINAL: listing a floor name in redact_keys never downgrades it to a tail mask.

Two entry points:

* redact(value)            — the public helper: mask a value the caller
                           KNOWS is sensitive (scalar → tail mask,
                           everything else → full [REDACTED]). This is
                           unconditional — the caller opted in by
                           calling it.
* mask_span_data / mask_log_record_data / mask_metric_data — the
                           exporter-boundary pass that applies the
                           floor plus the customer's key lists, run by
                           the redacting exporters
                           (redacting_exporter.rb) on door 1 AND all
                           three door-2 taps. Runs UNCONDITIONALLY —
                           the floor has no off switch; values under
                           unlisted names pass through raw.

Ruby SDK constraint (see GOTCHAS.md): a Span freezes its attributes at finish (opentelemetry-sdk span.rb) BEFORE any SpanProcessor#on_finish runs, so the key pass cannot be a processor — it runs at the exporter boundary, rebuilding the mutable SpanData/LogRecordData Structs.

Constant Summary collapse

FLOOR_HEADER_PREFIXES =

The two semconv header-attribute prefixes — the ONLY prefixes the floor peels (design §2.2). Canonical (lowercase, no dashes), so matching the canon'd name covers any case variant of the prefix too.

["http.request.header.", "http.response.header."].freeze
FLOOR_KEY_SET =

O(1) lookup set for the frozen key list (the array stays the shipped, fixture-gated constant; this is a derived view, same object identity of entries).

CREDENTIAL_KEY_DENYLIST.to_set.freeze
URL_ATTRIBUTES =

---- URL / query redaction ------------------------------------------- The instrumentation-set URL attributes (rack, http clients). Redact the query PER KEY so non-secret pairs survive: ?token=x&user=bob exports as token=[REDACTED]&user=bob. A pair is touched only when its key is on the credential floor (canon-exact — the floor's signature entry covers OTel's own PARAMS_TO_REDACT Signature case-insensitively) or on one of the customer's lists.

%w[http.target url.query url.full http.url uri.query].freeze

Class Method Summary collapse

Class Method Details

.canon(name) ⇒ Object

---- the credential-floor predicate (design §2, self-contained) ------- canon(name): trim ASCII whitespace → ASCII-lowercase → fold '-' to '_' (design §1.0). Applied to BOTH sides of every floor comparison; the stored CREDENTIAL_KEY_DENYLIST entries are already canonical.



139
140
141
 
# File 'lib/foam/otel/redaction.rb', line 139

def canon(name)
  name.to_s.strip.downcase(:ascii).tr("-", "_")
end

.classify_key(key, config) ⇒ Object

---- key classification ---------------------------------------------- The credential floor is checked FIRST and is TERMINAL (design §4): a floor-matched name is [REDACTED] in full even when the customer also lists it in redact_keys — the tail pass must never see a floor value (a tail mask over the [REDACTED] literal would emit ******** and destroy the contract literal). Below the floor: redact_keys (:mask) wins over redact_pii_keys (:erase) when both claim a key. Customer matching is substring, case-insensitive (unchanged).



258
259
260
261
262
263
264
265
266
267
 
# File 'lib/foam/otel/redaction.rb', line 258

def classify_key(key, config)
  return :floor if floor_match?(key)
  return nil if config.nil?

  lower = key.downcase
  return :mask if match_any_substring?(lower, config.redact_keys)
  return :erase if match_any_substring?(lower, config.redact_pii_keys)

  nil
end

.deep_mask(value, config, depth) ⇒ Object 



439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
 
# File 'lib/foam/otel/redaction.rb', line 439

def deep_mask(value, config, depth)
  return REDACTED if depth > MAX_REDACT_DEPTH

  case value
  when Hash
    value.each_with_object({}) do |(key, val), out|
      case classify_key(key.to_s, config)
      when :floor then out[key] = floor_mask(floor_kind(key.to_s), val)
      when :erase then out[key] = REDACTED
      when :mask  then out[key] = scalar_or_full(val)
      else out[key] = deep_mask(val, config, depth + 1)
      end
    end
  when Array
    value.map { |v| deep_mask(v, config, depth + 1) }
  else
    value
  end
rescue StandardError
  REDACTED
end

.descend(value, config = nil, depth = 0) ⇒ Object

Unmatched key: the value itself is captured RAW (no value-pattern auto-masking). Structured values still DESCEND so a floor-listed or redact_keys/redact_pii_keys key NESTED under an innocent key is honored — bounded to MAX_REDACT_DEPTH so an adversarial/cyclic structure can never drive a downstream encoder into SystemStackError (rule 9).



243
244
245
246
247
248
 
# File 'lib/foam/otel/redaction.rb', line 243

def descend(value, config = nil, depth = 0)
  case value
  when Hash, Array then deep_mask(value, config || Foam::Otel.active_config, depth)
  else value
  end
end

.floor_kind(name) ⇒ Object

Classify a name against the FLOOR ONLY (never the customer lists):

:header — a semconv header attribute (http.{request,response}
        .header.<name>) whose peeled suffix canon-matches the key
        list (which contains all seven headers — subset invariant,
        design §1.1; the peel checks the FULL 52-entry list, §2.2,
        so e.g. http.request.header.x_forwarded_for masks too);
:key    — the full name itself canon-matches a key-list entry
        (full-name equality: "user.session" does NOT match
        "session"; "connect.sid" matches its own verbatim entry);
nil     — no floor match (customer lists may still apply).


163
164
165
166
167
168
169
170
171
 
# File 'lib/foam/otel/redaction.rb', line 163

def floor_kind(name)
  c = canon(name)
  FLOOR_HEADER_PREFIXES.each do |prefix|
    next unless c.start_with?(prefix)

    return FLOOR_KEY_SET.include?(c[prefix.length..]) ? :header : nil
  end
  FLOOR_KEY_SET.include?(c) ? :key : nil
end

.floor_mask(kind, value) ⇒ Object

The floor mask (design §3): a semconv header attribute is a string array (one element per header instance) — mask PER ELEMENT, preserving arity and the string-array type (OTel's own sanitize_header_values shape). Everything else replaces the ENTIRE value with the single literal [REDACTED] — scalar, object, array, number, binary. Never a tail, never length-preserving: these are credentials, not debug aids.



183
184
185
186
187
 
# File 'lib/foam/otel/redaction.rb', line 183

def floor_mask(kind, value)
  return value.map { REDACTED } if kind == :header && value.is_a?(Array)

  REDACTED
end

.floor_match?(name) ⇒ Boolean

Returns:

  • (Boolean) 


173
174
175
 
# File 'lib/foam/otel/redaction.rb', line 173

def floor_match?(name)
  !floor_kind(name).nil?
end

.mask_attributes(attributes, config) ⇒ Object

---- the floor + customer key pass over exported attributes ----------- Returns a NEW attributes hash with the credential floor and the customer's key lists applied. Runs UNCONDITIONALLY (the floor has no off switch); values under unmatched names pass through raw (same objects). Never raises; a key that fails to mask redacts in full rather than leaking.



195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
 
# File 'lib/foam/otel/redaction.rb', line 195

def mask_attributes(attributes, config)
  return attributes unless attributes.is_a?(Hash)

  attributes.each_with_object({}) do |(key, value), out|
    out[key] = mask_one(key.to_s, value, config)
  rescue StandardError
    # Fail closed at the VALUE level, loudly (rule 15): the key name only
    # — never the value — reaches the diagnostics channel. The warning is
    # itself guarded: a poisoned key (#to_s raises) must still redact.
    begin
      Diagnostics.warn("masking failed for attribute #{key.to_s.inspect} — value redacted in full " \
                       "(fail-closed, rule 14)")
    rescue StandardError, SystemStackError
      Diagnostics.warn("masking failed for an attribute (unprintable key) — value redacted in full " \
                       "(fail-closed, rule 14)")
    end
    out[key] = REDACTED
  end
end

.mask_body(body, config) ⇒ Object

A log body is AnyValue: a free-text string is captured RAW (no value-pattern masking — the floor matches NAMES, never scans content), a structured map/array DESCENDS so a floor-listed or customer-listed key nested inside is honored (depth-guarded). Runs unconditionally.



433
434
435
436
437
 
# File 'lib/foam/otel/redaction.rb', line 433

def mask_body(body, config)
  deep_mask(body, config, 0)
rescue StandardError
  REDACTED
end

.mask_exemplars!(point, config) ⇒ Object

Exemplars carry filtered_attributes the OTLP exporter encodes RAW, one field over from the point attributes just masked — the floor must reach them too (design §2.3). Two postures, both fail-closed:

* customer keys configured → DROP exemplars entirely (the documented
pre-floor behavior for configured redaction, unchanged);
* floor only (the default path) → keep the exemplars but run their
filtered_attributes through the same engine pass, so floor-listed
names are [REDACTED] and everything else stays raw (§9: the only
default-path wire delta is floor values becoming [REDACTED]). An
exemplar whose rebuild fails is dropped, never exported raw.


408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
 
# File 'lib/foam/otel/redaction.rb', line 408

def mask_exemplars!(point, config)
  return unless point.respond_to?(:exemplars=)

  if redaction_configured?(config)
    point.exemplars = nil
    return
  end
  exemplars = point.respond_to?(:exemplars) ? point.exemplars : nil
  return if exemplars.nil?

  point.exemplars = exemplars.filter_map do |exemplar|
    copy = exemplar.dup
    copy.filtered_attributes = mask_attributes(exemplar.filtered_attributes || {}, config)
    copy
  rescue StandardError
    nil # fail closed: a poisoned exemplar drops, never ships raw
  end
rescue StandardError
  point.exemplars = nil # fail closed at the list level too
end

.mask_log_record_data(record, config) ⇒ Object 



364
365
366
367
368
369
370
371
 
# File 'lib/foam/otel/redaction.rb', line 364

def mask_log_record_data(record, config)
  masked = record.dup
  masked.attributes = mask_attributes(record.attributes, config) if record.attributes
  masked.body = mask_body(record.body, config) if record.body
  masked
rescue StandardError, SystemStackError
  nil
end

.mask_metric_data(metric_data, config) ⇒ Object

Door 1 masks metric attributes at capture; foreign (door-2) instrument data never passes those helpers, so the ingest metric path applies the floor + key lists here. Rebuilds every data point as a COPY (the collected points' attributes hash is the LIVE aggregation key upstream — masking in place would corrupt the customer's aggregation state). Runs UNCONDITIONALLY (the floor has no off switch).

FAIL CLOSED: returns NIL on a hard failure — the caller drops the metric loudly rather than exporting a half-masked point.



382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
 
# File 'lib/foam/otel/redaction.rb', line 382

def mask_metric_data(metric_data, config)
  masked = metric_data.dup
  points = metric_data.data_points
  if points
    masked.data_points = points.map do |point|
      copy = point.dup
      copy.attributes = mask_attributes(point.attributes || {}, config)
      mask_exemplars!(copy, config)
      copy
    end
  end
  masked
rescue StandardError, SystemStackError
  nil
end

.mask_one(key, value, config) ⇒ Object 



215
216
217
218
219
220
221
222
223
224
225
226
227
 
# File 'lib/foam/otel/redaction.rb', line 215

def mask_one(key, value, config)
  case classify_key(key, config)
  when :floor then floor_mask(floor_kind(key), value)
  when :erase then REDACTED
  when :mask  then value.is_a?(Array) ? value.map { REDACTED } : scalar_or_full(value)
  else
    if url_attribute?(key)
      redact_url_value(value, config)
    else
      descend(value, config)
    end
  end
end

.mask_span_data(span_data, config) ⇒ Object

---- exporter-boundary pass (SpanData / LogRecordData) ---------------- Rebuild the mutable Struct with the credential floor and the customer's key lists applied. A dup keeps foam's export copy independent of the span the tenant seam saw. Runs UNCONDITIONALLY (the floor has no off switch); values under unmatched names are carried through raw.

FAIL CLOSED: on a hard failure these return NIL — never the raw payload once the pass has begun. The redacting exporters drop a nil entry (with a loud [foam] warning) rather than export a half-masked record.



338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
 
# File 'lib/foam/otel/redaction.rb', line 338

def mask_span_data(span_data, config)
  masked = span_data.dup
  masked.attributes = mask_attributes(span_data.attributes, config) if span_data.attributes
  # Link attributes ride the key pass too: links are reachable on BOTH
  # doors and the OTLP exporter encodes link.attributes RAW. Copy-not-
  # mutate, exactly as events are handled (Link freezes its attributes at
  # construction).
  if span_data.links && !span_data.links.empty?
    masked.links = span_data.links.map do |link|
      next link unless link.respond_to?(:attributes) && link.attributes

      link.class.new(link.span_context, mask_attributes(link.attributes, config))
    end
  end
  if span_data.events && !span_data.events.empty?
    masked.events = span_data.events.map do |event|
      next event unless event.respond_to?(:attributes) && event.attributes

      event.class.new(event.name, mask_attributes(event.attributes, config), event.timestamp)
    end
  end
  masked
rescue StandardError, SystemStackError
  nil
end

.match_any_substring?(lower, keys) ⇒ Boolean

Returns:

  • (Boolean) 


269
270
271
272
273
 
# File 'lib/foam/otel/redaction.rb', line 269

def match_any_substring?(lower, keys)
  return false if keys.nil?

  keys.any? { |k| lower.include?(k) }
end

.redact(value) ⇒ Object

---- public helper (section 0 helper set) ---------------------------- Mask a value the caller knows is sensitive. Scalars get the tail mask; anything structured redacts in full. Fail closed: never return the unredacted payload — empty string on hard failure.



69
70
71
72
73
74
75
76
77
78
 
# File 'lib/foam/otel/redaction.rb', line 69

def redact(value)
  case value
  when String, Numeric, Symbol then tail_mask(value.to_s)
  else REDACTED
  end
rescue StandardError, SystemStackError
  # SystemStackError is not a StandardError: a value whose #to_s recurses
  # unboundedly must still degrade to empty here, never escape (rule 9).
  ""
end

.redact_query(query, config) ⇒ Object

Per-key query redaction, never re-encoding, never leaking on failure.



304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
 
# File 'lib/foam/otel/redaction.rb', line 304

def redact_query(query, config)
  return query if query.nil? || query.empty?

  query.split("&", -1).map do |pair|
    eq = pair.index("=")
    next pair if eq.nil?

    raw_key = pair[0...eq]
    decoded = begin
      URI.decode_www_form_component(raw_key).downcase
    rescue StandardError
      raw_key.downcase
    end
    if floor_match?(decoded) ||
       match_any_substring?(decoded, config&.redact_keys) ||
       match_any_substring?(decoded, config&.redact_pii_keys)
      "#{raw_key}=#{REDACTED}"
    else
      pair
    end
  end.join("&")
rescue StandardError
  ""
end

.redact_url_value(value, config) ⇒ Object 



288
289
290
291
292
293
294
295
296
297
298
299
300
301
 
# File 'lib/foam/otel/redaction.rb', line 288

def redact_url_value(value, config)
  return value unless value.is_a?(String)

  if value.include?("?")
    head, _, query = value.partition("?")
    return value if query.empty?

    "#{head}?#{redact_query(query, config)}"
  else
    redact_query(value, config)
  end
rescue StandardError
  REDACTED
end

.redaction_configured?(config) ⇒ Boolean

True when the customer enumerated at least one key to redact — the CUSTOMER lists only. The credential floor runs regardless: this predicate gates only customer-list behavior (e.g. the exemplar-drop posture below), never the floor.

Returns:

  • (Boolean) 


125
126
127
128
129
130
131
132
133
 
# File 'lib/foam/otel/redaction.rb', line 125

def redaction_configured?(config)
  return false if config.nil?

  keys = config.redact_keys
  pii = config.redact_pii_keys
  (!keys.nil? && !keys.empty?) || (!pii.nil? && !pii.empty?)
rescue StandardError
  false
end

.scalar_or_full(value) ⇒ Object

A redact_keys match: tail-mask a scalar leaf, redact anything else in full (partial masking only helps where it hides the value).



231
232
233
234
235
236
 
# File 'lib/foam/otel/redaction.rb', line 231

def scalar_or_full(value)
  case value
  when String, Numeric then tail_mask(value.to_s)
  else REDACTED
  end
end

.scrub_utf8(value) ⇒ Object

---- encoding hygiene (NOT redaction) --------------------------------- OTLP protobuf requires valid UTF-8, and the upstream Ruby exporter's encode failure drops the WHOLE batch — one invalid-encoding attribute from a helper call would silently kill every healthy span/log/metric batched beside it (the rule-15 silent-loss shape; observed live: the conformance /hostile route's bad-UTF-8 attribute dropping the /health span's batch). Foam scrubs String values to valid UTF-8 AT CAPTURE on its helper surface: invalid bytes become U+FFFD — content otherwise untouched (this is byte-validity, not masking; raw capture stands).



89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
 
# File 'lib/foam/otel/redaction.rb', line 89

def scrub_utf8(value)
  case value
  when String
    if value.encoding == Encoding::UTF_8
      value.valid_encoding? ? value : value.scrub("�")
    elsif value.encoding == Encoding::ASCII_8BIT
      # binary payloads carry no declared charset — treat as UTF-8 bytes
      value.dup.force_encoding(Encoding::UTF_8).scrub("�")
    else
      value.encode(Encoding::UTF_8, invalid: :replace, undef: :replace)
    end
  when Array
    value.map { |entry| scrub_utf8(entry) }
  else
    value
  end
rescue StandardError, SystemStackError
  "" # a value that cannot be made encodable degrades, never poisons a batch
end

.tail_mask(str) ⇒ Object

---- the two mask shapes --------------------------------------------- A redact_keys match on a SCALAR leaf: eight mask chars, plus the last four when the value is long enough to make a tail safe. The mask body is always exactly eight chars — a secret's length is itself information (entropy-safe masking).



114
115
116
117
118
119
 
# File 'lib/foam/otel/redaction.rb', line 114

def tail_mask(str)
  s = str.to_s
  return MASK_BODY if s.length < TAIL_MIN_LENGTH

  "#{MASK_BODY}#{s[-4..]}"
end

.url_attribute?(key) ⇒ Boolean

Returns:

  • (Boolean) 


284
285
286
 
# File 'lib/foam/otel/redaction.rb', line 284

def url_attribute?(key)
  URL_ATTRIBUTES.include?(key)
end