Module: Foam::Otel::Redaction

Defined in:
lib/foam/otel/redaction.rb

Overview

The central, fail-closed redaction engine (BASE_PACKAGE_SPEC rule 14).

Two entry points:

* redact(value)            — the public helper: mask a value the caller
                           KNOWS is sensitive (scalar → tail mask,
                           everything else → full [REDACTED]).
* mask_span_data / mask_log_record_data — the always-on secrets floor
                           applied to exported telemetry, run by the
                           redacting exporters (redacting_exporter.rb).

Ruby SDK constraint (see GOTCHAS.md): a Span freezes its attributes at finish (opentelemetry-sdk span.rb) BEFORE any SpanProcessor#on_finish runs, so the floor cannot be a processor — it runs at the exporter boundary, rebuilding the mutable SpanData/LogRecordData Structs.

Constant Summary collapse

URL_ATTRIBUTES =

---- URL / query redaction ------------------------------------------- The instrumentation-set URL attributes (rack, http clients). Redact the query PER KEY so non-secret pairs survive: ?token=x&user=bob exports as token=[REDACTED]&user=bob.

%w[http.target url.query url.full http.url uri.query].freeze

Class Method Summary collapse

Class Method Details

.auth_root?(lower) ⇒ Boolean

auth must hit authorization/authentication/x-auth-token and MISS author/author_id/authored_at.

Returns:

  • (Boolean) 


135
136
137
138
 
# File 'lib/foam/otel/redaction.rb', line 135

def auth_root?(lower)
  lower.include?("authorization") || lower.include?("authentication") ||
    /(?<![a-z0-9])auth(?![a-z0-9])/.match?(lower)
end

.card_root?(lower) ⇒ Boolean

card must hit card/card_number/credit_card and MISS dashboard/wildcard/postcard.

Returns:

  • (Boolean) 


142
143
144
145
146
 
# File 'lib/foam/otel/redaction.rb', line 142

def card_root?(lower)
  norm = normalize_key(lower)
  norm.include?("creditcard") || norm.include?("cardnumber") ||
    /(?<![a-z0-9])card(?![a-z0-9])/.match?(lower)
end

.classify_key(key, config) ⇒ Object

---- key classification ---------------------------------------------- :secret wins over :pii when both claim a key (a value under a secret-shaped key is better tail-masked than fully hidden). Substring, case-insensitive, over-redaction-curated (constants.rb).



108
109
110
111
112
113
114
 
# File 'lib/foam/otel/redaction.rb', line 108

def classify_key(key, config)
  lower = key.downcase
  return :secret if secret_key?(lower) || match_any_substring?(lower, config.redact_keys)
  return :pii if match_any_substring?(lower, config.redact_pii_keys)

  nil
end

.deep_mask(value, config, depth) ⇒ Object 



323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
 
# File 'lib/foam/otel/redaction.rb', line 323

def deep_mask(value, config, depth)
  return REDACTED if depth > MAX_REDACT_DEPTH

  case value
  when String
    scan_value_patterns(value)
  when Hash
    value.each_with_object({}) do |(key, val), out|
      case classify_key(key.to_s, config)
      when :pii    then out[key] = REDACTED
      when :secret then out[key] = scalar_or_full(val)
      else out[key] = deep_mask(val, config, depth + 1)
      end
    end
  when Array
    value.map { |v| deep_mask(v, config, depth + 1) }
  else
    value
  end
rescue StandardError
  REDACTED
end

.mask_attributes(attributes, config) ⇒ Object

---- the always-on floor over exported attributes -------------------- Returns a NEW attributes hash with the floor applied. Never raises; a key that fails to mask redacts in full rather than leaking.



57
58
59
60
61
62
63
64
65
 
# File 'lib/foam/otel/redaction.rb', line 57

def mask_attributes(attributes, config)
  return attributes unless attributes.is_a?(Hash)

  attributes.each_with_object({}) do |(key, value), out|
    out[key] = mask_one(key.to_s, value, config)
  rescue StandardError
    out[key] = REDACTED
  end
end

.mask_body(body, config) ⇒ Object

A log body is AnyValue: a free-text string (where raw tokens live — scan it), OR a structured map/array (where secret KEYS live — apply the floor recursively, depth-guarded). rule 14 / door-2 logs clause.



317
318
319
320
321
 
# File 'lib/foam/otel/redaction.rb', line 317

def mask_body(body, config)
  deep_mask(body, config, 0)
rescue StandardError
  REDACTED
end

.mask_by_value(value, config = nil, depth = 0) ⇒ Object

Unmatched key: the value-pattern pass still masks a secret-shaped token embedded in the value (a token under an innocent key name). Hash/Array values ride the DEPTH-GUARDED deep_mask — never a raw recursion: an adversarial/cyclic structure (a cyclic Hash log attribute) is bounded to MAX_REDACT_DEPTH here, so it can never drive a downstream encoder into SystemStackError (rule 9) or leak a secret nested under an innocent key (rule 14).



96
97
98
99
100
101
102
 
# File 'lib/foam/otel/redaction.rb', line 96

def mask_by_value(value, config = nil, depth = 0)
  case value
  when String then scan_value_patterns(value)
  when Hash, Array then deep_mask(value, config || Foam::Otel.active_config, depth)
  else value
  end
end

.mask_log_record_data(record, config) ⇒ Object 



270
271
272
273
274
275
276
277
 
# File 'lib/foam/otel/redaction.rb', line 270

def mask_log_record_data(record, config)
  masked = record.dup
  masked.attributes = mask_attributes(record.attributes, config) if record.attributes
  masked.body = mask_body(record.body, config) if record.body
  masked
rescue StandardError, SystemStackError
  nil
end

.mask_metric_data(metric_data, config) ⇒ Object

The door-2 engine extension (fleet contract ruling: rule 14 on FOREIGN instruments). Door 1 masks metric attributes at capture (metrics.rb stringify) — foreign instrumentation data never passes those helpers, so the ingest metric path needs the floor at the exporter boundary. Rebuilds every data point as a COPY with masked attributes: the collected points are SHALLOW dups whose attributes hash is the LIVE aggregation key upstream (metrics-sdk aggregation/sum.rb #collect) — masking in place would corrupt the customer's aggregation state, so mask_attributes builds a fresh hash and the point struct is dup'd. Same two mask shapes, same curated roots, same value-pattern pass.

FAIL CLOSED (rule 14): returns NIL on a hard failure — the caller drops the metric loudly rather than exporting unmasked (never a partial item mixing masked and raw fields — fleet audit R6).



293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
 
# File 'lib/foam/otel/redaction.rb', line 293

def mask_metric_data(metric_data, config)
  masked = metric_data.dup
  points = metric_data.data_points
  if points
    masked.data_points = points.map do |point|
      copy = point.dup
      copy.attributes = mask_attributes(point.attributes || {}, config)
      # Exemplars carry filtered_attributes — the (often high-cardinality)
      # measurement attributes dropped from the point — and the OTLP
      # metrics exporter encodes them RAW, one field over from the point
      # attributes the floor just masked. Drop them: fail-closed, and no
      # observability cost for foam's own use (rule 14).
      copy.exemplars = nil if copy.respond_to?(:exemplars=)
      copy
    end
  end
  masked
rescue StandardError, SystemStackError
  nil
end

.mask_one(key, value, config) ⇒ Object 



67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/foam/otel/redaction.rb', line 67

def mask_one(key, value, config)
  case classify_key(key, config)
  when :pii    then REDACTED
  when :secret then value.is_a?(Array) ? value.map { REDACTED } : scalar_or_full(value)
  else
    if url_attribute?(key)
      redact_url_value(value, config)
    else
      mask_by_value(value, config)
    end
  end
end

.mask_span_data(span_data, config) ⇒ Object

---- exporter-boundary masking (SpanData / LogRecordData) ------------- Rebuild the mutable Struct with masked attributes/events/body. A dup keeps foam's export copy independent of the span the tenant seam saw.

FAIL CLOSED (rule 14): on a hard failure these return NIL — never the raw payload. The redacting exporters drop a nil entry (with a loud [foam] warning) rather than export unmasked; "on hard failure return empty, never the payload".



219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
 
# File 'lib/foam/otel/redaction.rb', line 219

def mask_span_data(span_data, config)
  masked = span_data.dup
  masked.attributes = mask_attributes(span_data.attributes, config) if span_data.attributes
  # Link attributes ride the floor too (rule 14): links are reachable on
  # BOTH doors (the get_tracer passthrough carries them; door-2 foreign
  # pipelines routinely emit them — batch/messaging instrumentation) and
  # the OTLP exporter encodes link.attributes RAW. A secret in a link
  # attribute passes NEITHER capture-time masking (no foam helper builds
  # links) NOR the point floor — mask it here, copy-not-mutate, exactly
  # as events are handled (Link freezes its attributes at construction).
  if span_data.links && !span_data.links.empty?
    masked.links = span_data.links.map do |link|
      next link unless link.respond_to?(:attributes) && link.attributes

      link.class.new(link.span_context, mask_attributes(link.attributes, config))
    end
  end
  if span_data.events && !span_data.events.empty?
    masked.events = span_data.events.map do |event|
      next event unless event.respond_to?(:attributes) && event.attributes

      event.class.new(event.name, mask_attributes(event.attributes, config), event.timestamp)
    end
  end
  # The status DESCRIPTION is free text (record_exception sets it from the
  # exception message — exactly where a raw bearer token/header leaks). It
  # bypasses the attribute floor entirely, so the same secret masked in an
  # attribute would survive one field over. Run the value-pattern pass over
  # it (rule 14). A failure here bubbles to the fail-closed rescue below.
  masked.status = mask_status(span_data.status) if span_data.status
  masked
rescue StandardError, SystemStackError
  nil
end

.mask_status(status) ⇒ Object

Rebuild a span Status with the value-pattern pass applied to its description; codes without a description are returned untouched. Status' constructor is private, so build through the public code factories.



257
258
259
260
261
262
263
264
265
266
267
268
 
# File 'lib/foam/otel/redaction.rb', line 257

def mask_status(status)
  return status unless status.respond_to?(:description) && status.description
  return status if status.description.empty?

  desc = scan_value_patterns(status.description.to_s)
  klass = status.class
  case status.code
  when klass::ERROR then klass.error(desc)
  when klass::OK then klass.ok(desc)
  else klass.unset(desc)
  end
end

.match_any_substring?(lower, keys) ⇒ Boolean

Returns:

  • (Boolean) 


148
149
150
 
# File 'lib/foam/otel/redaction.rb', line 148

def match_any_substring?(lower, keys)
  keys.any? { |k| lower.include?(k) }
end

.normalize_key(key) ⇒ Object 



129
130
131
 
# File 'lib/foam/otel/redaction.rb', line 129

def normalize_key(key)
  key.downcase.gsub(/[-_.\s]/, "")
end

.redact(value) ⇒ Object

---- public helper (section 0 helper set) ---------------------------- Mask a value the caller knows is sensitive. Scalars get the tail mask; anything structured redacts in full. Fail closed: never return the unredacted payload — empty string on hard failure.



31
32
33
34
35
36
37
38
39
40
 
# File 'lib/foam/otel/redaction.rb', line 31

def redact(value)
  case value
  when String, Numeric, Symbol then tail_mask(value.to_s)
  else REDACTED
  end
rescue StandardError, SystemStackError
  # SystemStackError is not a StandardError: a value whose #to_s recurses
  # unboundedly must still degrade to empty here, never escape (rule 9).
  ""
end

.redact_query(query, config) ⇒ Object

Per-key query redaction, never re-encoding, never leaking on failure.



187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
 
# File 'lib/foam/otel/redaction.rb', line 187

def redact_query(query, config)
  return query if query.nil? || query.empty?

  query.split("&", -1).map do |pair|
    eq = pair.index("=")
    next pair if eq.nil?

    raw_key = pair[0...eq]
    decoded = begin
      URI.decode_www_form_component(raw_key).downcase
    rescue StandardError
      raw_key.downcase
    end
    if secret_key?(decoded) || match_any_substring?(decoded, config.redact_keys) ||
       match_any_substring?(decoded, config.redact_pii_keys)
      "#{raw_key}=#{REDACTED}"
    else
      pair
    end
  end.join("&")
rescue StandardError
  ""
end

.redact_url_value(value, config) ⇒ Object 



171
172
173
174
175
176
177
178
179
180
181
182
183
184
 
# File 'lib/foam/otel/redaction.rb', line 171

def redact_url_value(value, config)
  return value unless value.is_a?(String)

  if value.include?("?")
    head, _, query = value.partition("?")
    return value if query.empty?

    "#{head}?#{redact_query(query, config)}"
  else
    redact_query(value, config)
  end
rescue StandardError
  REDACTED
end

.scalar_or_full(value) ⇒ Object

A matched secret key: tail-mask a scalar leaf, redact anything else in full (partial masking only helps where it hides the value).



82
83
84
85
86
87
 
# File 'lib/foam/otel/redaction.rb', line 82

def scalar_or_full(value)
  case value
  when String, Numeric then tail_mask(value.to_s)
  else REDACTED
  end
end

.scan_value_patterns(text) ⇒ Object

One-pass gsub over the unioned value patterns, replacing only the matched token (keeps surrounding non-secret text intact). Used for attribute values and log bodies.



155
156
157
158
159
 
# File 'lib/foam/otel/redaction.rb', line 155

def scan_value_patterns(text)
  text.gsub(SECRET_VALUE_RE) { |m| tail_mask(m) }
rescue StandardError
  REDACTED
end

.secret_key?(lower) ⇒ Boolean

lower keeps separators (for the boundary-guarded short roots); norm folds them away (for the distinctive bare roots).

Returns:

  • (Boolean) 


118
119
120
121
122
123
124
125
126
127
 
# File 'lib/foam/otel/redaction.rb', line 118

def secret_key?(lower)
  norm = normalize_key(lower)
  return true if BARE_SECRET_ROOTS.any? { |root| norm.include?(root) }
  return true if auth_root?(lower) || card_root?(lower)
  # Short/ambiguous roots as whole tokens (`user_cvv` hits,
  # `classname`/`consider` do not) — one precompiled boundary regex.
  return true if GUARDED_SECRET_RE.match?(lower)

  false
end

.tail_mask(str) ⇒ Object

---- the two mask shapes (rule 14) ----------------------------------- Secret-key + redact_keys matches on SCALAR leaves: eight mask chars, plus the last four when the value is long enough to make a tail safe. The mask body is always exactly eight chars — a secret's length is itself information (entropy-safe masking).



47
48
49
50
51
52
 
# File 'lib/foam/otel/redaction.rb', line 47

def tail_mask(str)
  s = str.to_s
  return MASK_BODY if s.length < TAIL_MIN_LENGTH

  "#{MASK_BODY}#{s[-4..]}"
end

.url_attribute?(key) ⇒ Boolean

Returns:

  • (Boolean) 


167
168
169
 
# File 'lib/foam/otel/redaction.rb', line 167

def url_attribute?(key)
  URL_ATTRIBUTES.include?(key)
end