Module: Foam::Otel::Redaction

Defined in:
lib/foam/otel/redaction.rb

Overview

Opt-in, key-list redaction (holistic redesign, 2026-07-26). The package does NO redaction by default: with neither redact_keys nor redact_pii_keys set (or both empty) every value is captured RAW — there is NO always-on secret floor and NO value-pattern auto-masking. Redaction runs ONLY over the keys the customer enumerates:

* redact_keys     — tail-mask matching scalar values (shape-preserving,
                  e.g. ********f456); non-scalars redact in full.
* redact_pii_keys — erase matching values in full ([REDACTED]).

Matching is SUBSTRING, case-insensitive, over the lowercased key.

Two entry points:

* redact(value)            — the public helper: mask a value the caller
                           KNOWS is sensitive (scalar → tail mask,
                           everything else → full [REDACTED]). This is
                           unconditional — the caller opted in by
                           calling it.
* mask_span_data / mask_log_record_data / mask_metric_data — the
                           exporter-boundary pass that applies the
                           customer's key lists, run by the redacting
                           exporters (redacting_exporter.rb). A true
                           pass-through when no keys are configured.

Ruby SDK constraint (see GOTCHAS.md): a Span freezes its attributes at finish (opentelemetry-sdk span.rb) BEFORE any SpanProcessor#on_finish runs, so the key pass cannot be a processor — it runs at the exporter boundary, rebuilding the mutable SpanData/LogRecordData Structs.

Constant Summary collapse

URL_ATTRIBUTES =

---- URL / query redaction ------------------------------------------- The instrumentation-set URL attributes (rack, http clients). Redact the query PER KEY so non-secret pairs survive: with redact_keys=["token"], ?token=x&user=bob exports as token=[REDACTED]&user=bob. Only keys the customer listed are ever touched.

%w[http.target url.query url.full http.url uri.query].freeze

Class Method Summary collapse

Class Method Details

.classify_key(key, config) ⇒ Object

---- key classification ---------------------------------------------- redact_keys (:mask) wins over redact_pii_keys (:erase) when both claim a key. Substring, case-insensitive.



163
164
165
166
167
168
169
 
# File 'lib/foam/otel/redaction.rb', line 163

def classify_key(key, config)
  lower = key.downcase
  return :mask if match_any_substring?(lower, config.redact_keys)
  return :erase if match_any_substring?(lower, config.redact_pii_keys)

  nil
end

.deep_mask(value, config, depth) ⇒ Object 



319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
 
# File 'lib/foam/otel/redaction.rb', line 319

def deep_mask(value, config, depth)
  return REDACTED if depth > MAX_REDACT_DEPTH

  case value
  when Hash
    value.each_with_object({}) do |(key, val), out|
      case classify_key(key.to_s, config)
      when :erase then out[key] = REDACTED
      when :mask  then out[key] = scalar_or_full(val)
      else out[key] = deep_mask(val, config, depth + 1)
      end
    end
  when Array
    value.map { |v| deep_mask(v, config, depth + 1) }
  else
    value
  end
rescue StandardError
  REDACTED
end

.descend(value, config = nil, depth = 0) ⇒ Object

Unmatched key: the value itself is captured RAW (no value-pattern auto-masking). Structured values still DESCEND so a redact_keys/ redact_pii_keys key NESTED under an innocent key is honored — bounded to MAX_REDACT_DEPTH so an adversarial/cyclic structure can never drive a downstream encoder into SystemStackError (rule 9).



153
154
155
156
157
158
 
# File 'lib/foam/otel/redaction.rb', line 153

def descend(value, config = nil, depth = 0)
  case value
  when Hash, Array then deep_mask(value, config || Foam::Otel.active_config, depth)
  else value
  end
end

.mask_attributes(attributes, config) ⇒ Object

---- the customer key pass over exported attributes ------------------ Returns a NEW attributes hash with the customer's key lists applied, or the same hash untouched when no keys are configured. Never raises; a key that fails to mask redacts in full rather than leaking.



113
114
115
116
117
118
119
120
121
122
 
# File 'lib/foam/otel/redaction.rb', line 113

def mask_attributes(attributes, config)
  return attributes unless attributes.is_a?(Hash)
  return attributes unless redaction_configured?(config)

  attributes.each_with_object({}) do |(key, value), out|
    out[key] = mask_one(key.to_s, value, config)
  rescue StandardError
    out[key] = REDACTED
  end
end

.mask_body(body, config) ⇒ Object

A log body is AnyValue: a free-text string is captured RAW (no value-pattern masking), a structured map/array DESCENDS so a listed key nested inside is honored (depth-guarded). Pass-through when no keys are configured.



311
312
313
314
315
316
317
 
# File 'lib/foam/otel/redaction.rb', line 311

def mask_body(body, config)
  return body unless redaction_configured?(config)

  deep_mask(body, config, 0)
rescue StandardError
  REDACTED
end

.mask_log_record_data(record, config) ⇒ Object 



265
266
267
268
269
270
271
272
273
274
 
# File 'lib/foam/otel/redaction.rb', line 265

def mask_log_record_data(record, config)
  return record unless redaction_configured?(config)

  masked = record.dup
  masked.attributes = mask_attributes(record.attributes, config) if record.attributes
  masked.body = mask_body(record.body, config) if record.body
  masked
rescue StandardError, SystemStackError
  nil
end

.mask_metric_data(metric_data, config) ⇒ Object

Door 1 masks metric attributes at capture; foreign (door-2) instrument data never passes those helpers, so the ingest metric path applies the key lists here. Rebuilds every data point as a COPY (the collected points' attributes hash is the LIVE aggregation key upstream — masking in place would corrupt the customer's aggregation state). A true pass-through when no keys are configured.

FAIL CLOSED: returns NIL on a hard failure — the caller drops the metric loudly rather than exporting a half-masked point.



285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
 
# File 'lib/foam/otel/redaction.rb', line 285

def mask_metric_data(metric_data, config)
  return metric_data unless redaction_configured?(config)

  masked = metric_data.dup
  points = metric_data.data_points
  if points
    masked.data_points = points.map do |point|
      copy = point.dup
      copy.attributes = mask_attributes(point.attributes || {}, config)
      # Exemplars carry filtered_attributes the OTLP exporter encodes RAW,
      # one field over from the point attributes the key pass just masked.
      # Once redaction is configured, drop them: fail-closed, no
      # observability cost for foam's own use.
      copy.exemplars = nil if copy.respond_to?(:exemplars=)
      copy
    end
  end
  masked
rescue StandardError, SystemStackError
  nil
end

.mask_one(key, value, config) ⇒ Object 



124
125
126
127
128
129
130
131
132
133
134
135
136
137
 
# File 'lib/foam/otel/redaction.rb', line 124

def mask_one(key, value, config)
  return value unless redaction_configured?(config)

  case classify_key(key, config)
  when :erase then REDACTED
  when :mask  then value.is_a?(Array) ? value.map { REDACTED } : scalar_or_full(value)
  else
    if url_attribute?(key)
      redact_url_value(value, config)
    else
      descend(value, config)
    end
  end
end

.mask_span_data(span_data, config) ⇒ Object

---- exporter-boundary key pass (SpanData / LogRecordData) ------------ Rebuild the mutable Struct with the customer's key lists applied. A dup keeps foam's export copy independent of the span the tenant seam saw. With no keys configured this is a true PASS-THROUGH: the original struct is returned untouched (ZERO redaction — raw capture).

FAIL CLOSED: on a hard failure these return NIL — never the raw payload once a key pass has begun. The redacting exporters drop a nil entry (with a loud [foam] warning) rather than export a half-masked record.



237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
 
# File 'lib/foam/otel/redaction.rb', line 237

def mask_span_data(span_data, config)
  return span_data unless redaction_configured?(config)

  masked = span_data.dup
  masked.attributes = mask_attributes(span_data.attributes, config) if span_data.attributes
  # Link attributes ride the key pass too: links are reachable on BOTH
  # doors and the OTLP exporter encodes link.attributes RAW. Copy-not-
  # mutate, exactly as events are handled (Link freezes its attributes at
  # construction).
  if span_data.links && !span_data.links.empty?
    masked.links = span_data.links.map do |link|
      next link unless link.respond_to?(:attributes) && link.attributes

      link.class.new(link.span_context, mask_attributes(link.attributes, config))
    end
  end
  if span_data.events && !span_data.events.empty?
    masked.events = span_data.events.map do |event|
      next event unless event.respond_to?(:attributes) && event.attributes

      event.class.new(event.name, mask_attributes(event.attributes, config), event.timestamp)
    end
  end
  masked
rescue StandardError, SystemStackError
  nil
end

.match_any_substring?(lower, keys) ⇒ Boolean

Returns:

  • (Boolean) 


171
172
173
174
175
 
# File 'lib/foam/otel/redaction.rb', line 171

def match_any_substring?(lower, keys)
  return false if keys.nil?

  keys.any? { |k| lower.include?(k) }
end

.redact(value) ⇒ Object

---- public helper (section 0 helper set) ---------------------------- Mask a value the caller knows is sensitive. Scalars get the tail mask; anything structured redacts in full. Fail closed: never return the unredacted payload — empty string on hard failure.



45
46
47
48
49
50
51
52
53
54
 
# File 'lib/foam/otel/redaction.rb', line 45

def redact(value)
  case value
  when String, Numeric, Symbol then tail_mask(value.to_s)
  else REDACTED
  end
rescue StandardError, SystemStackError
  # SystemStackError is not a StandardError: a value whose #to_s recurses
  # unboundedly must still degrade to empty here, never escape (rule 9).
  ""
end

.redact_query(query, config) ⇒ Object

Per-key query redaction, never re-encoding, never leaking on failure.



204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
 
# File 'lib/foam/otel/redaction.rb', line 204

def redact_query(query, config)
  return query if query.nil? || query.empty?

  query.split("&", -1).map do |pair|
    eq = pair.index("=")
    next pair if eq.nil?

    raw_key = pair[0...eq]
    decoded = begin
      URI.decode_www_form_component(raw_key).downcase
    rescue StandardError
      raw_key.downcase
    end
    if match_any_substring?(decoded, config.redact_keys) ||
       match_any_substring?(decoded, config.redact_pii_keys)
      "#{raw_key}=#{REDACTED}"
    else
      pair
    end
  end.join("&")
rescue StandardError
  ""
end

.redact_url_value(value, config) ⇒ Object 



188
189
190
191
192
193
194
195
196
197
198
199
200
201
 
# File 'lib/foam/otel/redaction.rb', line 188

def redact_url_value(value, config)
  return value unless value.is_a?(String)

  if value.include?("?")
    head, _, query = value.partition("?")
    return value if query.empty?

    "#{head}?#{redact_query(query, config)}"
  else
    redact_query(value, config)
  end
rescue StandardError
  REDACTED
end

.redaction_configured?(config) ⇒ Boolean

True when the customer enumerated at least one key to redact. With neither list set the package does ZERO redaction (raw capture).

Returns:

  • (Boolean) 


99
100
101
102
103
104
105
106
107
 
# File 'lib/foam/otel/redaction.rb', line 99

def redaction_configured?(config)
  return false if config.nil?

  keys = config.redact_keys
  pii = config.redact_pii_keys
  (!keys.nil? && !keys.empty?) || (!pii.nil? && !pii.empty?)
rescue StandardError
  false
end

.scalar_or_full(value) ⇒ Object

A redact_keys match: tail-mask a scalar leaf, redact anything else in full (partial masking only helps where it hides the value).



141
142
143
144
145
146
 
# File 'lib/foam/otel/redaction.rb', line 141

def scalar_or_full(value)
  case value
  when String, Numeric then tail_mask(value.to_s)
  else REDACTED
  end
end

.scrub_utf8(value) ⇒ Object

---- encoding hygiene (NOT redaction) --------------------------------- OTLP protobuf requires valid UTF-8, and the upstream Ruby exporter's encode failure drops the WHOLE batch — one invalid-encoding attribute from a helper call would silently kill every healthy span/log/metric batched beside it (the rule-15 silent-loss shape; observed live: the conformance /hostile route's bad-UTF-8 attribute dropping the /health span's batch). Foam scrubs String values to valid UTF-8 AT CAPTURE on its helper surface: invalid bytes become U+FFFD — content otherwise untouched (this is byte-validity, not masking; raw capture stands).



65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
 
# File 'lib/foam/otel/redaction.rb', line 65

def scrub_utf8(value)
  case value
  when String
    if value.encoding == Encoding::UTF_8
      value.valid_encoding? ? value : value.scrub("�")
    elsif value.encoding == Encoding::ASCII_8BIT
      # binary payloads carry no declared charset — treat as UTF-8 bytes
      value.dup.force_encoding(Encoding::UTF_8).scrub("�")
    else
      value.encode(Encoding::UTF_8, invalid: :replace, undef: :replace)
    end
  when Array
    value.map { |entry| scrub_utf8(entry) }
  else
    value
  end
rescue StandardError, SystemStackError
  "" # a value that cannot be made encodable degrades, never poisons a batch
end

.tail_mask(str) ⇒ Object

---- the two mask shapes --------------------------------------------- A redact_keys match on a SCALAR leaf: eight mask chars, plus the last four when the value is long enough to make a tail safe. The mask body is always exactly eight chars — a secret's length is itself information (entropy-safe masking).



90
91
92
93
94
95
 
# File 'lib/foam/otel/redaction.rb', line 90

def tail_mask(str)
  s = str.to_s
  return MASK_BODY if s.length < TAIL_MIN_LENGTH

  "#{MASK_BODY}#{s[-4..]}"
end

.url_attribute?(key) ⇒ Boolean

Returns:

  • (Boolean) 


184
185
186
 
# File 'lib/foam/otel/redaction.rb', line 184

def url_attribute?(key)
  URL_ATTRIBUTES.include?(key)
end