foam-otel (Ruby)

Foam's OpenTelemetry core for Ruby services. A thin, safe wrapper over the official OpenTelemetry libraries: foam owns the pipeline (providers, batch processors, OTLP export to the foam endpoint), turns on automatic tier-1/2 instrumentation, and hands you a small set of never-throw helpers — and nothing else. Redaction is fully opt-in above two always-on exceptions: by default foam captures every value RAW (no PII preset) except (1) the credential floor — a fixed list of credential/secret header and field NAMES always masked to [REDACTED] (see "The default credential denylist" below) — and (2) the value-pattern secret layer — credential-SHAPED value spans (cloud keys, tokens, JWTs, private-key blocks, user:pass@ URLs…) masked to [REDACTED] regardless of field name (see "The value-pattern secret layer" below); you enumerate any further fields to mask via redact_keys/redact_pii_keys.

Built to docs/BASE_PACKAGE_SPEC.md. This README is the manual you integrate from.

Foam builds on the OpenTelemetry authors' work (https://opentelemetry.io) — see THIRD-PARTY-NOTICES. Licensed, not sold — see LICENSE.

Install

# Gemfile
gem "foam-otel"
# foam-otel bundles the UNIVERSAL FLOOR and auto-activates each piece when its
# target library is present (presence-checked, and gated on foam owning the
# slot it produces into): Rack + the Rails family, the HTTP clients (Net::HTTP,
# Faraday), the primary datastores (pg, mysql2, redis, mongo) with RAW
# db.statement capture, Sidekiq, the stdlib Logger bridge, hand-written Ruby
# runtime + GC metrics, session stitching (browser session.id via baggage),
# default-on HTTP header capture (a documented default header list on the
# official rack server span; every Faraday client header — see "Header
# capture" below), and the LLM shims
# (OpenAI, Anthropic, Gemini, ruby_llm — activity + raw content). Add only
# the niche long tail your app needs beyond the floor:
gem "opentelemetry-instrumentation-graphql"  # e.g. GraphQL
gem "opentelemetry-instrumentation-resque"   # e.g. Resque

Supported versions (tested in CI, spec/version_spec.rb): Ruby >= 3.4 (the version the CI matrix runs — the floor follows the tested matrix); OpenTelemetry API opentelemetry-api ~> 1.1 (1.x). The metrics and logs SDKs are pre-1.0 upstream (metrics alpha, logs development) and are pinned with pessimistic constraints; foam wraps them behind stable helpers so app code never touches the churning API (see GOTCHAS R2).

Quick start — scenario 1: a clean service (the common case)

Put init FIRST, before your app boots. In Rails, that means an initializer (and, for cluster mode, per worker — see "Puma / Unicorn" below).

# config/initializers/foam.rb  (the one config module you own)
require "foam/otel"

Foam::Otel.init(
  name: "checkout-api",                       # → service.name
  environment: ENV.fetch("APP_ENV", "development"),  # → deployment.environment.name (verbatim)
  enabled: !%w[test ci].include?(ENV["APP_ENV"]),    # silence tests/CI; export everywhere else
  token: ENV.fetch("FOAM_OTEL_TOKEN"),             # required WHEN enabled — wire it explicitly
  version: ENV["GIT_SHA"]                      # → service.version (recommended; optional)
)

Outcome: foam registers the tracer/meter/logger providers, exports OTLP to the foam endpoint, and auto-activates the universal floor — the official rack/rails instrumentation plus the bundled HTTP (Net::HTTP/Faraday), datastore (pg/mysql2/redis/mongo), and Sidekiq instrumentations — each presence-gated, and gated on foam owning the traces slot (a foreign-owned tracer stands the activation down). The datastore set is activated with db_statement: :include — foam explicitly overrides the upstream :obfuscate default so RAW SQL / redis args / mongo commands land on the wire. On top of the contrib set the floor adds foam's own pieces: the stdlib Logger bridge (every host Logger line also lands as a trace-correlated OTel log record, body raw), hand-written Ruby runtime + GC metrics (GC counts + timing, heap slots, allocation, thread count, RSS — Ruby has no drop-in upstream metrics gem, so foam ships its own collector), session stitching (inbound baggage: session.id from the foam browser package is stamped onto every span and log record), and the LLM shims (below). Every inbound request, DB call, HTTP call, job, log line, and model call now flows to foam. Redaction is opt-in above the credential floor: by default every value — attributes, URLs/query strings, DB statement text, log bodies, LLM content — is captured RAW, except values whose NAME is on the always-on credential denylist (masked [REDACTED]; see "The default credential denylist" below). Pass redact_keys/redact_pii_keys to mask or erase further specific fields (see below).

Sidekiq trace shape (GOTCHAS G16): the bundled instrumentation keeps its upstream default propagation_style: :link — a performed job runs as its OWN trace whose first span LINKS back to the enqueuing span (context is always propagated across the hop; proven against a real Redis in spec/floor_sidekiq_wire_spec.rb). Prefer one continuous web→job trace? Set the standard contrib lever: OTEL_RUBY_INSTRUMENTATION_SIDEKIQ_CONFIG_OPTS='propagation_style=child'.

The LLM surface — OpenAI, Anthropic, Gemini (plus ruby_llm, ruby-openai, and hand-rolled HTTP.rb Anthropic)

The Ruby contrib registry ships no LLM instrumentation, so foam does: thin, presence-checked shims over the SDKs' public call sites, emitting the standard gen_ai.* semantic conventions — activity (model, response id, finish reasons, token usage, latency) AND prompt/response content, RAW (gen_ai.input.messages / gen_ai.output.messages / gen_ai.system_instructions). Agent tool loops ride in full: tool-call turns keep their tool names/arguments and tool-result turns keep their tool_call_id linkage inside the message content. gen_ai.operation.name (and the {operation} {model} span name) follows the semconv operation for each API shape — chat for the OpenAI/Anthropic/ruby_llm chat seams, generate_content for Gemini's generateContent, identical to foam's js/python Gemini paths (wire change in 1.2.1: Gemini spans previously said chat). Covered: the official openai SDK (chat.completions.create + responses.create), the official anthropic SDK (messages.create), Gemini via the gemini-ai gem (Google ships no official Ruby SDK), ruby_llm (one seam covering all its providers), the community ruby-openai gem (client.chat / client.embeddings — presence is discriminated from the official gem by call shape, since both reopen the OpenAI namespace; embeddings emit tokens + models, never the input documents), and hand-rolled HTTP.rb Anthropic clients (HTTP.post / HTTP.persistent(...).post to api.anthropic.com/v1/messages — exactly that endpoint; HTTP.rb's non-raising 4xx/5xx become ERROR spans with the observed status, and foam never consumes a response stream: SSE, chunked, and oversized bodies pass through untouched, and buffered JSON bodies are handed back readable). Absent SDK = inert; version-guarded; fail-to-dark — a shim failure can NEVER break the model call, and the SDK's own error re-raises identically. One model call = one gen_ai client span (the SDK's HTTP send rides the same suppression context foam's exporters use; transport gems that ignore that context, like the contrib http instrumentation, nest beneath it instead). Streaming call sites (#stream/#stream_raw on the official SDKs) are a recorded follow-up.

Session stitching — browser `session.id` on backend telemetry

The foam browser package stamps session.id on every browser signal and sends it to your API as W3C Baggage beside traceparent. This gem's floor copies session.id (and session.previous_id across a rotation) from the inbound baggage onto every span and log record foam emits — browser session → backend traces/logs, stitched with no ingest-side join. No-op when the baggage is absent. CORS note (the FDE wires this on every frontend-called API): the allowlist must include BOTH headers — Access-Control-Allow-Headers: traceparent, baggage.

Header capture — request + response headers on the official spans

Inbound (default-on, no init option): foam configures the OFFICIAL opentelemetry-instrumentation-rack gem's own header options (allowed_request_headers: / allowed_response_headers:) with a documented default list at init, so the rack SERVER span carries the listed request and response headers as http.request.header.<name> / http.response.header.<name> attributes (the gem's form: lowercase with - folded to _, e.g. http.request.header.x_request_id), on error requests too.

Honest limit (deliberate divergence from foam's js/python cores): Ruby inbound captures the named default list below, NOT all headers — the official gem's options are an enumerated allowlist with no capture-all form, and that official option is the chosen seam. A custom header outside the list is not captured unless you extend the list (recipe below).

The default lists (Foam::Otel::HeaderCapture::DEFAULT_REQUEST_HEADERS / DEFAULT_RESPONSE_HEADERS — frozen, documented constants):

request — content-type, content-length, content-encoding, accept, accept-charset, accept-encoding, accept-language, user-agent, referer, origin, host, cache-control, pragma, if-none-match, if-modified-since, range, via, forwarded, x-forwarded-for, x-forwarded-proto, x-forwarded-host, x-forwarded-port, x-real-ip, x-request-id, x-correlation-id, plus the request half of the credential floor — authorization, proxy-authorization, cookie, x-api-key, x-auth-token;
response — content-type, content-length, content-encoding, content-language, content-range, cache-control, pragma, expires, age, etag, last-modified, vary, location, retry-after, x-request-id, x-correlation-id, x-runtime, server-timing, plus the response half of the floor — set-cookie, www-authenticate.

Propagation headers (traceparent, tracestate, baggage) are deliberately not listed — they are extracted as span context, not captured as attributes; hop-by-hop plumbing (connection, keep-alive, …) is skipped too.

Extending (or narrowing) the list — the gem's own standard env var, always respected, never overridden by foam:

# The value REPLACES the list for that option — name every header you want:
export OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS='allowed_request_headers=content-type,accept,x-request-id,x-tenant-id'

Precedence: (1) an explicit rack instrumentation instance you pass via additional_instrumentations: installs first and its header config wins entirely; (2) any header option you set in OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS governs that option (foam's default fills only the option you did not touch — a narrower operator list is never widened, even the narrow-to-empty allowed_request_headers= spelling); (3) otherwise foam's defaults apply. Foam never installs the rack instrumentation (or its header config) when a foreign SDK owns the traces slot.

Floor masking, zero config: the always-on credential floor masks the seven credential headers (authorization, proxy-authorization, cookie, set-cookie, x-api-key, x-auth-token, www-authenticate) to [REDACTED] in the emitted attribute forms (dash/underscore-normalized match) — they ride the default lists deliberately so their PRESENCE is visible, value never (see "The default credential denylist"; x-forwarded-for / x-real-ip are on the 52-entry key floor and arrive [REDACTED] too). Masking further captured headers is redaction's job, not a capture switch: list the emitted name form in redact_keys / redact: { secrets: [...] } (tail mask) or redact_pii_keys / redact: { pii: [...] } (full [REDACTED]) — the substring key match reaches the attribute names (inbound names are underscored by the rack gem, so list e.g. x_runtime; Faraday's outbound names keep dashes).

Outbound (Faraday): the official faraday instrumentation has no header option at all, so outbound stays foam's own Faraday middleware (its first-class public extension API), enriching the official CLIENT span with every request header actually sent (the injected traceparent included) and every response header received — string-array attributes, dash-form names. It never creates a span and stands down when foam is disabled/killed, when a foreign SDK owns traces, or when the current span is not the official faraday instrumentation's own (a mis-ordered stack captures nothing rather than writing onto the wrong span). One line per connection (Faraday has no public add-to-every-connection hook — the middleware name is registered by init, placement is yours; ORDER MATTERS, the official middleware first):

conn = Faraday.new(url: "https://partner-api.example") do |f|
  f.use :open_telemetry     # the official client span (explicit, so it sits OUTSIDE)
  f.use :foam_otel_headers  # foam's header capture, directly inside it
end

Deliberately out of scope (no standard extension point exists — skipped, not monkey-patched): outbound Net::HTTP / Excon / HTTP (httprb) / HTTPX headers (their official instrumentations expose no request/response hook or header option) and gRPC metadata. Outbound header coverage today is Faraday. Bodies/payloads are not captured by header capture — bodies are the separate, opt-in capture_payloads: capability below.

Payload (body) capture — opt-in, `capture_payloads:`

Off by default. One init option turns on inbound HTTP body capture on the official rack SERVER span (headers stay the header-capture seam above — this never duplicates them):

Foam::Otel.init(
  name: "checkout-api", environment: ENV.fetch("APP_ENV"),
  enabled: ENV.fetch("APP_ENV") == "production", token: ENV.fetch("FOAM_OTEL_TOKEN"),
  capture_payloads: :errors # :off (default) | :errors | :always — equivalent strings accepted
)

:off (default) — zero body teeing, zero per-request allocation, and zero middleware shipped: with the default, foam's Rails wiring installs nothing at all.
:errors — bodies are teed per request but attributes attach to the span only when the request errored: Foam::Otel.record_exception called while the rack server span is current, an exception raised through the middleware, or a response status >= 500. A clean 2xx/4xx attaches nothing.
:always — attach on every request.

An invalid option value raises ArgumentError at boot. The operator env clamp FOAM_CAPTURE_PAYLOADS=off|errors|always overrides the option in both directions (force payloads off on a misbehaving deploy, or force them on without a code change — one loud [foam] line when it changes the mode); an invalid env value warns and falls back to the option, never a crashed boot.

What lands on the span (in :always, or in :errors on an errored request): http.request.body / http.response.body for textual payloads (text/*, JSON/+json, urlencoded forms, XML/+xml, GraphQL; skipped when Content-Encoding is not identity), capped at 8192 chars with an …[truncated] marker, plus http.request.body.size / http.response.body.size — true byte sizes when known (declared Content-Length, else the bytes actually observed). Non-textual payloads contribute sizes only.

Stream safety (GOTCHAS F15): bodies are TEED, never consumed — foam records exactly what your app reads from rack.input (an app that never reads its body captures nothing; a re-read form body — rewound, or repositioned via seek/pos= — is captured and counted once), and streaming/SSE responses pass through unbuffered chunk-by-chunk with close preserved. Rack 3 #call-only streaming bodies are never wrapped. The request and response are never altered, and an app exception re-raises identically.

Redaction stays central, on export: a JSON-shaped body is deep-redacted by field name at the exporter boundary — the credential floor (password, authorization, …) and your redact_keys / redact: {secrets:/pii:} lists reach inside the body JSON; urlencoded bodies get their k=v pair names masked by the same tokenizer that covers query strings; and the value-pattern secret layer scans all captured body text. (A body truncated at the cap no longer parses as JSON, so field-name deep-redaction cannot apply to it — the value-shape scans still run.)

Wiring: in Rails, automatic when the mode is not :off (foam's railtie inserts Foam::Otel::PayloadCapture directly inside the official rack middleware after your initializers run; when foam-otel loads before rails, init wires it instead, as long as init runs during boot — the standard config/initializers/foam.rb recipe). Plain Rack / Sinatra apps add one line under the official middleware:

# config.ru
use(*OpenTelemetry::Instrumentation::Rack::Instrumentation.instance.middleware_args)
use Foam::Otel::PayloadCapture   # opt-in body capture (inert while capture_payloads is :off)
run MyApp

The middleware enriches only: it never creates a span, and it stands down per request when foam is disabled/killed, when the span is not recording, or when a foreign SDK owns the traces slot. Outbound (client) bodies are not captured — inbound only.

The `init` options

Option	Type	Required	Default	What it does
`name:`	String	yes	—	`service.name`. Blank → raises at boot.
`environment:`	String	yes	—	`deployment.environment.name`, exported verbatim. A value outside `{production, staging, development, test}` warns but is never rewritten. Blank → raises.
`enabled:`	Boolean	yes	—	The config switch. `false` = fully inert (no SDK, no providers, no network, helpers no-op). `true` = export, in EVERY environment. No default — you write the logic.
`token:`	String	when enabled	—	`Authorization: Bearer` for export. Required (and validated) only when `enabled: true`; wired explicitly from your secret source — never an env fallback read by the gem. The convention across all foam SDKs: read it from the `FOAM_OTEL_TOKEN` env var. Blank while enabled → raises at boot.
`version:`	String	no	nil	`service.version`, verbatim (git SHA recommended). Missing → warns and continues. Never detected at runtime.
`redact:`	Hash	no	nil	The grouped redaction object (1.5.0, the preferred surface): `{ secrets: [...], pii: [...], detect: [...] }` — symbol or string keys. `secrets:` has `redact_keys` semantics (tail mask) and `pii:` has `redact_pii_keys` semantics (full `[REDACTED]`); when both a legacy option and its `redact` field are set, the lists UNION. `detect:` is the opt-in PII detection tier's entity list — value-shape detection with typed placeholders (see "PII detection (opt-in)" below). An unknown field, a non-Array value, or an unknown `detect` entity name raises at boot. Absent (the default) → behavior byte-identical to before.
`redact_keys:`	Array	no	`[]`	Alias — the preferred spelling is `redact: { secrets: [...] }` (same semantics; the lists union when both are set). The ONLY field names tail-masked (shape-preserving, e.g. `********f456`). Opt-in and ADDITIVE on top of the always-on credential floor — empty (the default) means no masking beyond the floor. Matches by case-insensitive substring, including keys nested in structured values and query-string keys. A key that is also on the floor stays fully `[REDACTED]` (the floor wins; it is never downgraded to a tail).
`redact_pii_keys:`	Array	no	`[]`	Alias — the preferred spelling is `redact: { pii: [...] }` (same semantics; the lists union when both are set). Field names fully erased to `[REDACTED]` (no tail). Opt-in and additive above the floor — empty (the default) means no erasure beyond the floor. Foam ships no PII preset; this is your enumeration.
`additional_span_processors:`	Array	no	`[]`	Tenant seam: constructed SpanProcessor instances added to foam's pipeline (additive; never replace foam's export). See coexistence.
`additional_log_record_processors:`	Array	no	`[]`	Tenant seam, logs.
`additional_metric_readers:`	Array	no	`[]`	Tenant seam, metrics.
`additional_instrumentations:`	Array	no	`[]`	Constructed tier-2 instrumentation instances to register (fault-isolated: one that throws is skipped with a `[foam]` warning).
`ignored_outbound_hosts:`	Array	no	`[]`	Hosts whose outbound calls produce no spans — EXTENDS the built-in export-loop guard. For a co-resident agent's intake or a tenant exporter's endpoint. Applies to the clients whose official instrumentation supports host suppression: Net::HTTP and Excon. Faraday/HTTP (httprb)/HTTPX have no upstream `untraced_hosts` option — foam warns loudly at init when one of those is bundled (GOTCHAS G13); foam's own export loop is guarded for every client regardless.
`before_send:`	Proc/Array	no	`[]`	Export hook (1.8.0): one callable — or an Array run as a pipeline — called with every span and log record at the export boundary, after batching and BEFORE foam's redaction pass and serialization, so nothing a hook drops ever leaves the process. Return the record (or a same-type replacement) to export it — in-place mutation supported; return `nil` to drop it; a raise or foreign return drops that record loudly, fail-closed, never a raise into your threads. Runs BEFORE redaction — a hook can never widen what ships. Spans and logs only; door 1 only. A non-callable (or a one-arg-incompatible lambda) raises at boot. Process-global. Hooks run on the export thread — keep them non-blocking. Full contract: "The `before_send` hook" below.
`capture_payloads:`	Symbol/String	no	`:off`	Inbound HTTP body capture on the rack server span (see "Payload (body) capture"). `:off` = zero teeing, zero middleware (the default); `:errors` = tee per request, attach only on exception/5xx; `:always` = attach on every request. Equivalent strings accepted; anything else raises at boot. Overridden in both directions by the `FOAM_CAPTURE_PAYLOADS` env clamp (env table below). Textual payloads only, 8192-char cap + `…[truncated]`, true `.size` attributes; sizes-only for binary/compressed.
`diagnostics:`	Boolean	no	false	Verbose `[foam]` self-reporting of init/health. Warnings and errors are always loud regardless.
`secret_heuristics:`	Boolean	no	true	The value-pattern secret layer's HEURISTIC tier (generic keyword+entropy detection, see "The value-pattern secret layer"). `false` disables ONLY the heuristics — the named provider patterns, the credential floor and the redaction-coverage contract have no off switch. Disabling logs one loud `[foam]` line at init (an explicit, audited decision for telemetry whose legitimate values collide with the heuristics).

Examples for the remaining options (each is runnable as-is):

# Redaction is OPT-IN (default: nothing is masked). List the fields to redact:
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"),
                redact_keys: %w[password api_key internal_ref voucher_code], # tail-masked
                redact_pii_keys: %w[customer_email full_name])               # fully [REDACTED]

# The grouped redact object (1.5.0, preferred — the flat options above are
# aliases and UNION with it when both are set), plus the opt-in PII
# detection tier (typed placeholders; see "PII detection (opt-in)"):
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"),
                redact: {
                  secrets: %w[internal_ref voucher_code],   # == redact_keys (tail mask)
                  pii: %w[customer_email full_name],        # == redact_pii_keys (full [REDACTED])
                  detect: %w[email phone ssn credit_card ip], # value-shape detection → [EMAIL], [PHONE], …
                })
# An unknown field, a non-Array value, or an unknown detect entity raises at boot:
# Foam::Otel.init(..., redact: { detect: ["name"] })
# => raises ArgumentError: Foam::Otel.init redact: unknown detect entity "name" — the valid
#    entity names are exactly {email, phone, ssn, credit_card, ip} (contract/pii-detect.json)

# Only if your legitimate telemetry collides with the generic secret
# heuristics (e.g. base64 content-addressed ids masked as high-entropy
# tokens): disable the HEURISTIC tier only. The named provider patterns and
# the credential floor stay on — they have no off switch. Logs one loud
# [foam] line.
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"),
                secret_heuristics: false)

# Constructed tier-2 instrumentation instances (fault-isolated; one that
# throws is skipped with a [foam] warning, never a crashed boot):
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"),
                additional_instrumentations: [
                  OpenTelemetry::Instrumentation::Sidekiq::Instrumentation.instance,
                  [OpenTelemetry::Instrumentation::PG::Instrumentation.instance, { db_statement: :omit }],
                ])

# Tenant seam, logs + metrics (spans are shown in scenario 3 below):
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"),
                additional_log_record_processors: [EvalTool::LogRecordProcessor.new],
                additional_metric_readers: [EvalTool::MetricReader.new],
                ignored_outbound_hosts: ["ingest.eval-tool.example"]) # the tenant loop guard — required

# The before_send export hook (1.8.0): transform or drop spans/logs right
# before they are exported to foam — after batching, BEFORE redaction and
# serialization (a dropped record never leaves the process). Return the
# record (mutate in place freely) or nil to drop; a raising hook drops ONLY
# its record, loudly, and never raises into your threads (fail-closed).
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"),
                before_send: lambda { |record|
                  next nil if record.respond_to?(:name) && record.name == "GET /healthz" # drop noise

                  record.attributes.delete("internal.debug.blob") # trim before it ships
                  record
                })
# Or an Array pipeline — hooks run in order, each receiving the previous
# hook's return; the first nil drops the record and short-circuits:
# Foam::Otel.init(..., before_send: [DropHealthchecks, TrimDebugAttrs, StampTeam])

# Verbose self-reporting while wiring foam up (warnings are loud regardless):
Foam::Otel.init(name: "checkout-api", environment: "development", enabled: true,
                token: ENV.fetch("FOAM_OTEL_TOKEN"), diagnostics: true)

And the unhappy path — a blank required argument fails AT BOOT, on your machine, never silently:

Foam::Otel.init(name: "", environment: "production", enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"))
# => raises ArgumentError: Foam::Otel.init requires a non-empty name

Deliberately absent (and why): endpoint, cadence/sampling, is_production, force_export, disabled_environments, and any processor/exporter injection beyond the tenant seam. Customers identify their service and turn foam on or off; they can never drop a share of the data, inject their own pipeline pieces, or redirect export from code. The only endpoint override is the operator-level OTEL_EXPORTER_OTLP_ENDPOINT env var (below).

The default credential denylist (the always-on floor)

Always on in every foam SDK, identical everywhere (frozen fixture: contract/credential-denylist.json). This is the ONE exception to foam's raw-by-default capture, and it has no off switch — no init option, no env var, and no door-2 parameter can disable, shrink, or re-spell it.

What is masked. The VALUE of every header/attribute/field whose NAME is on the frozen list, in every signal (spans — events and links included — logs, metrics), on door 1, on all three door-2 ingest taps, and in the view handed to tenant additional_* processors:

(a) the seven credential headers, matched as header names wherever headers are captured — including the semconv http.request.header.<name> / http.response.header.<name> span-attribute forms: authorization, proxy-authorization, cookie, set-cookie, x-api-key, x-auth-token, www-authenticate;
(b) the frozen 52-entry key list of contract/credential-denylist.json (the deduplicated union of those seven, sentry-python's default denylists, and foam's documented reference roots — password, token, secret, api_key, ssn, jwt, private_key, connect.sid, phpsessid, …), matched as attribute/field names at every depth the engine walks, and per-key in URL query strings (?token=x&user=bob → token=[REDACTED]&user=bob).

The matching rule. Case-insensitive, dash/underscore-normalized EXACT name-equality — never substring: authorization masks the header/field authorization (any casing, -≡_) and nothing else — authorization_url, x_api_key_id, secretary, and session_count all ride RAW. Dotted entries match verbatim and whole (connect.sid masks; user.session does not). This is deliberately STRICTER matching than your own redact_keys lists, which keep their documented substring behavior.

The mask. The full literal [REDACTED] — no tail, no length preservation. These are credentials, not debug aids. Semconv header attributes (string arrays) mask per element, preserving arity.

Precedence. The floor runs unconditionally BEFORE your redact_keys/redact_pii_keys, which stay purely additive on top. Listing a floor name in redact_keys does NOT downgrade it to a tail mask — the floor is terminal. enabled: false stays fully inert exactly as before (nothing exports at all — the floor masks data that leaves; inert mode has none).

Everything else stays RAW — above the floor and the value layer. The floor matches NAMES only, never values. Credential-SHAPED value spans are the value-pattern secret layer's job (next section); UNSHAPED values — bodies, query values, SQL text, LLM content, and PII under unlisted names — export byte-identical raw, exactly as documented everywhere else in this README.

Why. Coverage-over-masking remains the mission, but raw Authorization headers, cookies, and API keys on the wire are a breach in waiting for every customer at once — foam carves out exactly this list (and only this list) as non-negotiable. The list is byte-identical in every foam SDK and gate-checked in CI against the shipped fixture (spec/credential_floor_spec.rb).

Param names across the foam SDKs (the customer options above the floor, identical concept in every SDK — the floor itself has NO init surface anywhere):

Concept	js/otel	js/browser	python	ruby	java
secret keys (tail-mask)	`redactKeys`	`redactKeys`	`redact_keys`	`redact_keys:`	`.redactKeys(String…)`
PII keys (full `[REDACTED]`)	`redactPiiKeys`	`redactPiiKeys`	`redact_pii_keys`	`redact_pii_keys:`	`.redactPiiKeys(String…)`

Migration note — 1.3.0 (minor): always-on credential masking. As of this version foam masks, by default and in every signal, the VALUES of a fixed list of credential/secret header and field NAMES (authorization, cookie, set-cookie, proxy-authorization, x-api-key, x-auth-token, www-authenticate, and the 52-name key list in contract/credential-denylist.json — Sentry-parity plus foam's documented roots) to the literal [REDACTED]. Matching is exact name-equality, case-insensitive, dash/underscore-insensitive — never substring: authorization_url is untouched. Everything else still exports RAW exactly as before; redact_keys/redact_pii_keys are unchanged and additive. There is no off switch — if a dashboard keyed off a raw credential value (it should not have), it will now see [REDACTED]. Customers who previously received these header/field values raw stop receiving them.

The value-pattern secret layer (always-on)

The credential floor masks by NAME and is deliberately value-shape-blind — a live AWS key under the field name note would sail past it. Foam telemetry is read downstream by LLMs, so a leaked credential is exfiltratable by prompt injection; "no leakage at all" is the bar. The value-pattern secret layer is the second always-on control:

What it masks. Credential-SHAPED value spans, regardless of the field name they hide under: AWS access-key ids and keyword-anchored AWS secrets, GCP API keys, Azure client/storage secrets, GitHub tokens and fine-grained PATs, GitLab tokens, Slack tokens + webhooks, Stripe secret keys (publishable pk_ keys are deliberately NOT masked), JWTs, PEM private-key blocks (-----BEGIN … PRIVATE KEY — a truncated block masks to the end of the value, fail-closed; PUBLIC KEY/CERTIFICATE blocks never match) and PuTTY PPK keys, scheme://user:pass@ URI credentials (the whole userinfo, username included), Bearer/Basic auth values, and Anthropic/OpenAI API keys. The matched span becomes the literal [REDACTED] — no tail, no prefix, zero reconstructable bytes; surrounding text keeps its diagnostic value (conn to postgres://[REDACTED]@db:5432 refused).

The heuristic tier additionally catches the long tail: a keyword-anchored generic pass (password = hunter2SecretXyz99 inside a log body masks its value — gitleaks-derived, entropy-gated, with placeholder/ stopword/key-context false-positive suppression) and a whole-value high-entropy base64 token pass (detect-secrets-derived; pure hex and UUIDs are deliberately exempt — trace ids and digests are telemetry's ambient vocabulary). Only this tier is disableable: secret_heuristics: false (one loud [foam] line; the named patterns above are not configurable).

Where it runs. Every value-bearing string on every export path, both doors: span/event/link attribute values, status.message, exception messages, log bodies (string and structured leaves), SQL (db.statement), LLM content (BEFORE serialization and again at the boundary), URL query AND fragment pair values, baggage-derived attributes, exemplar filtered_attributes, and resource identity strings (once, at init). Metric datapoint attributes (metric labels) are the one exempt surface — low-cardinality by construction; a value mask there would break your aggregation dimensions.

Bounded by design. Compile-once rules, a literal-anchor pre-filter (no anchor in the value → zero regexes run), a 256 KiB value cap (an oversized value with an anchor hit anywhere masks WHOLE, fail-closed; without one it passes untouched), a match-flood cap, a 256-pair query/fragment tokenizer cap, and a per-Regexp timeout on Ruby ≥ 3.2. ANY scanner fault masks the value rather than shipping it raw.

Provenance. The ruleset derives from MIT/Apache-licensed OSS secret scanners (gitleaks, detect-secrets, secretlint) — see THIRD-PARTY-NOTICES. Rules are frozen module constants, byte-consistent across all foam SDKs — not configurable per app.

Migration note — 1.4.0 (minor): value-pattern secret masking + redaction coverage fix + TLS pin. foam now masks credential-SHAPED values (AWS/GCP/Azure keys, GitHub/GitLab tokens, Slack, Stripe, Anthropic/OpenAI keys, JWTs, private-key blocks, user:pass@ URI credentials, bearer/basic tokens) to [REDACTED] in every exported string value — regardless of field name — plus a generic keyword+entropy heuristic you can disable with secret_heuristics: false (the named patterns and the credential floor cannot be disabled). Redaction now also reaches URL FRAGMENTS (OAuth implicit-flow tokens, hash-router params), bare k=v pair lists, nested attribute containers (fail-closed), and structured LLM content BEFORE serialization. Query/fragment tokenizing is capped at 256 pairs (the overflow remainder masks whole). Metric labels are not value-scanned. TLS peer verification is now PINNED on every exporter: OTEL_RUBY_EXPORTER_OTLP_SSL_VERIFY_NONE is INERT and loudly warned about. If a dashboard keyed off a raw token value (it should not have), it will now see [REDACTED].

PII detection (opt-in)

The floor and the secret layer protect credentials. PII in VALUES — an email address inside a log line, a card number inside an exception message — still exports RAW by default, because coverage-over-masking is the mission and only you know your privacy posture. As of 1.5.0 you can opt into value-shape PII detection per entity, via redact: { detect: [...] } (frozen fixture: contract/pii-detect.json — the entity names and placeholders are byte-identical in every foam SDK and gate-checked in CI by spec/pii_detect_spec.rb). Nothing detects unless you list the entity — an empty/absent detect list is exactly today's behavior.

The exact entity set (these five, nothing else — an unknown name raises at boot):

Entity	Placeholder	Catches
`email`	`[EMAIL]`	`john.smith+test@example.co.uk` — requires a real TLD (`user@localhost` rides raw)
`phone`	`[PHONE]`	`+1 (415) 555-0142`, `415-555-0199` — separated groups; bare digit runs, dotted versions (`2024.10.05`) and clock times ride raw
`ssn`	`[SSN]`	`536-90-4399`, `536 90 4399` — delimited 3-2-4 only; structurally invalid SSNs (`000-…`, group `00`, serial `0000`) and undelimited runs ride raw
`credit_card`	`[CREDIT_CARD]`	13–19 digit PANs (spaced/dashed/contiguous) that pass Luhn — a card-shaped tracking id failing the checksum is never masked
`ip`	`[IP]`	IPv4 (octet-validated — `999.1.1.1` and `10.4.1.2000` ride raw) and IPv6, `::`-compressed included

How it masks. Only the matched character span is replaced with the entity's typed placeholder — surrounding text keeps its diagnostic value (login from [IP] flagged). It runs everywhere the value-pattern secret layer runs (span/event/link attributes, status.message, log bodies, URL query and fragment leaves, nested structures, both doors), immediately AFTER the secret layer (a shaped credential still becomes [REDACTED] — detect never weakens it) and BEFORE your key lists; metric datapoint attributes stay exempt exactly like the secret layer. Detection is idempotent (placeholders never re-match), rides the same execution caps and fail-closed discipline (any detector fault masks the whole value to [REDACTED], loudly), and the patterns follow the same bounded-quantifier authoring rules.

What it can NOT do — read this before relying on it. This is value-SHAPE detection, nothing more: person names and free-text prose are NOT detectable — recognizing "the patient, John Smith, reported…" takes server-side NER, which is platform scope, not an SDK regex. Field-NAME-keyed PII (patient_name) is what redact: { pii: [...] } is for; enumerate those keys yourself. Foam still ships no PII preset — the detect list is your explicit, audited enumeration.

# Door-2 taps take the same redact object, scoped to that tap:
processor = Foam::Otel.create_ingest_span_processor(
  token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV"),
  redact: { pii: %w[customer_email], detect: %w[email credit_card] })

The `before_send` hook

before_send: is the last word your code gets on every span and log record before it is exported to foam. It runs at the export boundary — after the batch processor hands foam the finished records, immediately before foam's redaction pass and the OTLP serialization — so a record your hook drops is never serialized and never leaves the process. (This is the same seam redaction lives at, and for the same Ruby SDK reason: a span's attributes freeze at finish before any on_finish processor runs, so a SpanProcessor could neither mutate nor drop. The export boundary is the first point where the record is a mutable struct.)

The pipeline order is pinned:

BatchProcessor → before_send hooks → redaction (floor + your lists) → OTLP wire

Your hooks run first, on the raw record (it is your own data, still in-process), and redaction runs on whatever they return — so before_send can never widen what ships: an attribute a hook adds is masked by the credential floor and your redact lists exactly like one an instrumentation set.

The per-record contract (Sentry beforeSend semantics):

return the record — or a replacement of the same type → it continues to the next hook / to export. Mutating it in place is supported — record.attributes arrives as your own unfrozen copy (values included), so record.attributes["k"] = v, record.attributes.delete("k"), record.attributes["k"].gsub!(...) and (for logs) editing record.body all just work without bleeding into the span your app still holds or the view a tenant processor saw. Events, links and status are the exception: replace them wholesale (record.events = [...]) — never mutate them in place, they are shared structures. Foam re-normalizes the record's total_recorded_* bookkeeping after your hooks run, so adding/deleting attributes never corrupts the wire's dropped-count accounting. record.resource and record.instrumentation_scope are identity, not payload: read them freely, but edits and replacements are discarded — the original identity objects always ship (they are process-global and the encoder groups by them; a writable resource would also bypass redaction, which never masks resource attributes).
return nil → the record is dropped, silently (that is the filter mechanism, not an error).
raise, or return a foreign object → that record is dropped loudly — one aggregated [foam] warning per batch, error classes only (messages can carry your data) — and the batch reports failure: fail-closed, a faulted hook never ships a record you may have meant to scrub, and never raises into your application or export threads. Healthy records in the same batch still export. Because a fault marks the whole batch failed, a hook that faults deterministically also makes the at-exit flush report failure — fix faulting hooks promptly (GOTCHAS F16).

Hooks run on foam's export thread, inside the batch processor's export lock. Keep them fast and non-blocking: no network calls, no DB lookups, no sleeping — a blocked hook stalls the export pipeline and can wedge the at-exit flush. Anything a hook needs from the request/job thread (tenant context, user identity) must be stamped onto the record as an attribute at capture time (see the additional_span_processors stamper pattern); thread-locals are gone by the time the hook runs.

Boot validation is strict: a non-callable — or a lambda/Method whose signature cannot accept one positional argument (e.g. a Sentry-port ->(event, hint) { ... } with two required params, or a required keyword) — raises ArgumentError at init, because it would otherwise fault on every record and ship zero telemetry from a green boot. Make the extra Sentry hint param optional (->(event, hint = nil)) and it is accepted.

Scope: spans and log records (the two record-shaped signals — the hook receives SpanData or LogRecordData; distinguish them with is_a? or respond_to?). Metrics are aggregated state, not discrete records, and are deliberately not routed through before_send. Door-2 ingest taps are unchanged. Like enabled: and the endpoint, before_send is process-global: the exporter chain is built at the first init, and a second init never changes it.

The helpers

All helpers never raise, and no-op silently before init and when disabled.

Lifecycle

Foam::Otel.flush       # force-flush buffered pipelines (call at handler/worker exit)
Foam::Otel.shutdown    # flush + shut down providers (normal exit is auto via at_exit)
Foam::Otel.after_fork! # restart metric-reader threads after a fork (see cluster mode)

Traces

result = Foam::Otel.span("nightly-sync") do |span|
  # ... work ...  the span always ends; a raised error is recorded + re-raised
end

Foam::Otel.set_attribute("user.tier", "pro")           # on the active span; false if none recording
Foam::Otel.set_attributes("a" => 1, "b" => 2)
Foam::Otel.add_event("cache.miss")
Foam::Otel.record_exception(e)                          # records on active span; never notifies a tracker
ctx = Foam::Otel.get_trace_context                      # {trace_id:, span_id:} hex, or nil — never fabricated

Metrics (one helper per synchronous instrument; names pass through verbatim)

Foam::Otel.increment_counter("checkout.completed", by: 1, attributes: { "plan" => "pro" })
Foam::Otel.record_histogram("checkout.duration_ms", 428)
Foam::Otel.add_up_down_counter("queue.depth", by: -1)
Foam::Otel.set_metric("cache.size_bytes", 10_485_760)   # gauge (last value wins)

⚠️ Metric attributes are a loaded gun: an UNBOUNDED value (user id, raw path, full URL) creates one metric stream per value — a memory leak that the SDK's cardinality cap then turns into silently flat dashboards. Use bounded, low-cardinality attribute values only.

Logs

Foam::Otel.log(:info, "checkout finished", attributes: { "order.id" => id })
# severity is a level symbol (:trace :debug :info :warn :error :fatal) or an
# OTel severity number (1..24). Trace-correlated when a span is active.

Redaction

Foam::Otel.redact(user_supplied_token)  # tail-masks a scalar you know is sensitive; [REDACTED] for structures

Passthroughs (the real OTel objects on foam's pipeline — links, span kinds, observable instruments, baggage all reachable here)

Foam::Otel.get_tracer("my.lib")
Foam::Otel.get_meter("my.lib")
Foam::Otel.get_logger("my.lib")

Coexistence — foam arriving in a monitored service

init classifies each signal's global slot independently and does the right, non-destructive thing. The classifier's output is information for you, never a decision it makes for you.

Scenario 2 — beside a proprietary APM agent

Disjoint pipelines: foam runs its own beside the agent, touching nothing. Add the agent's intake host so foam's outbound instrumentation doesn't shadow its egress.

Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: FOAM_OTEL_TOKEN,
                ignored_outbound_hosts: ["agent-intake.vendor.example"])

Scenario 3 — a scoped tenant rides foam's pipeline (LLM-eval, AI-observability)

Attach the tenant's processor via the seam, and add its exporter host to the ignore list (its exports would otherwise be shadow-spanned into a loop).

Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: FOAM_OTEL_TOKEN,
                additional_span_processors: [EvalTool::SpanProcessor.new(project: "prod")],
                ignored_outbound_hosts: ["ingest.eval-tool.example"])

Ruby note: when you configure redact_keys/redact_pii_keys, foam's HELPERS apply the key pass at capture, so a tenant processor receives foam-helper data (span/log helper attributes, log bodies, metric labels) ALREADY masked. The Ruby SDK freezes span attributes at finish, so historically a tenant processor saw third-party instrumentation attributes UNMASKED — as of 1.3.0 that gap is closed: tenant span/log processors are handed a READ-ONLY MASKED VIEW (the credential floor plus your configured keys, the same pass foam's own export applies — GOTCHAS F1). A tenant never sees a raw floor value. With no keys configured (the default) only the credential floor is masked on either path.

Loop-guard note: ignored_outbound_hosts suppresses spans for tenant/agent egress made through Net::HTTP or Excon. A tenant exporter using Faraday/HTTP (httprb)/HTTPX cannot be host-suppressed today (no upstream untraced_hosts option — foam warns at init; GOTCHAS G13).

Scenario 4 — a foreign OTel SDK already owns a signal (door 2)

If another OTel SDK registered (say) the tracer before foam, foam does NOT displace it. init warns once, naming the owner, and foam is dark for that signal — your tier-3 telemetry for it rides the foreign SDK to ITS backend. To ALSO send that pipeline to foam, add ONE LINE per claimed signal in THEIR setup — the ingest entries (see "Door 2 — the ingest entries" below). YOU place the tap: foam never attaches itself to a pipeline it does not own (their data moves only when a human says so), and the placement line doubles as the off-switch — gate it on the same condition you would give enabled:.

# config/initializers/opentelemetry.rb — THEIR setup, plus the foam lines
require "opentelemetry/sdk"
require "foam/otel"

OpenTelemetry::SDK.configure do |c|
  c.service_name = "their-service"
  c.use_all
  # REQUIRED loop step (see "Door 2 — the required loop step" below):
  c.use "OpenTelemetry::Instrumentation::Net::HTTP", { untraced_hosts: ["otel.api.foam.ai"] }
  c.add_span_processor(their_vendor_processor)           # their export, untouched
  if ENV["APP_ENV"] == "production"                       # your consent AND your off-switch
    c.add_span_processor(Foam::Otel.create_ingest_span_processor(
      token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))
  end
end

Outcome: their pipeline keeps working exactly as before — their data, their resource, their export, byte-identical — and foam ALSO receives the signal, carrying THEIR resource identity plus foam's export-time stamp, the always-on credential floor applied to foam's copy, and any per-tap redact_keys/redact_pii_keys you set (opt-in — none by default; details in the Door 2 section).

Unhappy paths:

Foam::Otel.create_ingest_span_processor(token: "", environment: "production")
# => raises ArgumentError: [foam] Foam::Otel.create_ingest_span_processor requires a non-empty token

OTEL_SDK_DISABLED=true at construction → the factory returns an inert no-op instance (their setup line keeps working, nothing is sent) and warns: [foam] OTEL_SDK_DISABLED=true — this foam ingest entry is disabled by the operator kill switch and will send nothing.
Tap wired onto FOAM'S OWN pipeline (self-ingest wiring bug) → one loud warning and the tap STANDS DOWN — door 1 already exports that data, so nothing is lost and nothing is doubled.
init() already exports a signal AND a tap sits on a separate foreign pipeline for the same signal → one loud warning; BOTH keep flowing (different pipelines; foam.ingest.tier distinguishes the copies).

When the TRACES slot is foreign-owned, foam also skips its automatic tier-1/2 instrumentation activation (it never adds span producers or Rack middleware to a pipeline it does not own) and warns once; additional_instrumentations you pass explicitly are still installed. The same holds for the process-wide propagator: a propagator the incumbent installed first is KEPT (foam installs its W3C set only over the API default), with one warning.

Scenario 5 — per-signal composition

Signals are independent. A foreign SDK owning only traces still gets foam for metrics and logs on the full contract:

Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: FOAM_OTEL_TOKEN)
# → warns: traces foreign-owned (foam dark for traces); metrics + logs → foam, full contract.

# The doors COMPOSE: add the traces tap in THEIR setup and foam receives
# traces too — the tap rides THEIR tracer, and trace ids correlate with
# foam-exported logs/metrics through the shared context:
their_provider.add_span_processor(Foam::Otel.create_ingest_span_processor(
  token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))
# (no double-claim warning: init() serves metrics+logs, the tap serves traces)

Scenario 6 — tests / CI (fully inert, no token needed)

Foam::Otel.init(name: "checkout-api", environment: "test", enabled: false)

Scenario 7 — serverless / short-lived (flush at the end)

Foam::Otel.init(name: "orders-fn", environment: "production", enabled: true, token: FOAM_OTEL_TOKEN)

def handler(event:, context:)
  run(event)
ensure
  Foam::Otel.flush   # the platform freezes the process before the batch timer fires
end

Puma / Unicorn (cluster mode) — CRITICAL

Exporter threads do not survive fork. Foam auto-recovers metric readers via a Process._fork hook, but the cleanest pattern is to init PER WORKER so each worker gets its own pipeline and a unique service.instance.id:

# config/puma.rb
on_worker_boot do
  Foam::Otel.init(name: "checkout-api", environment: ENV["APP_ENV"], enabled: true, token: ENV["FOAM_OTEL_TOKEN"])
  # or, if you init in an initializer: Foam::Otel.after_fork!
end

Door 2 — the ingest entries

When a foreign OTel SDK owns a signal (scenario 4), the ingest entries are how that pipeline's data ALSO reaches foam: constructed instances you add — one line per claimed signal — to the CUSTOMER'S OWN OTel setup. Additive readers only: they never mutate their data, their resource, or their export.

All three entries are SHIPPED. The traces entry rides the stable 1.x SDK; the log and metric entries ride upstream's pre-1.0 SDKs (logs "development", metrics alpha), pinned to this gem's capped ranges (GOTCHAS R2/R5) so a 0.x drift is a deliberate re-prove, never a silent change. The pre-1.0 SDK maturity is a version-pin caveat, not a deferral — the entries are available and supported today.

Version-pin posture: because the log and metric taps ride those pre-1.0 upstream SDKs, they are version-pinned and re-proven on every 0.x bump rather than assumed stable — shipped and supported either way.

The three entries

All three share one signature (token: and environment: required; the redaction kwargs — the grouped redact: object and its legacy aliases redact_keys:/redact_pii_keys:, union semantics exactly as on init — are the only CUSTOMER redaction that tap applies — opt-in, none by default, scoped to that tap, additive above the always-on credential floor, which every tap applies with zero configuration; diagnostics: is tap-scoped narration). Construction validates loudly at boot and NEVER throws afterwards — a failure inside a tap can never break their pipeline. Naming across the foam SDKs (pinned by spec/export_surface_spec.rb): create_ingest_span_processor = createFoamIngestSpanProcessor, create_ingest_log_record_processor = createFoamIngestLogRecordProcessor, create_ingest_metric_reader = createFoamIngestMetricReader.

Traces — a SpanProcessor for THEIR TracerProvider (scenario 4 snippet):

processor = Foam::Otel.create_ingest_span_processor(
  token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV"),
  redact_keys: %w[voucher_code],          # optional: extra secret keys, this tap only
  redact_pii_keys: %w[customer_email],    # optional: your PII enumeration, this tap only
  diagnostics: false)                     # optional: per-tap [foam] narration
their_tracer_provider.add_span_processor(processor)
# returns a new instance per call; no-op inert instance under OTEL_SDK_DISABLED=true;
# raises ArgumentError at construction for a blank token/environment

Logs — a LogRecordProcessor for THEIR LoggerProvider. The credential floor and the tap's redact_keys/redact_pii_keys (if any) match over record attributes and keys nested in structured bodies; free-text bodies ride RAW (no value-pattern pass):

provider = OpenTelemetry::SDK::Logs::LoggerProvider.new(resource: their_resource)
provider.add_log_record_processor(their_log_processor)
provider.add_log_record_processor(Foam::Otel.create_ingest_log_record_processor(
  token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))
OpenTelemetry.logger_provider = provider

Metrics — a PeriodicMetricReader for THEIR MeterProvider. Attach order is forgiving: add_metric_reader back-fills instruments created before the tap attached. Foam's reader collects on its own schedule beside theirs:

OpenTelemetry.meter_provider.add_metric_reader(Foam::Otel.create_ingest_metric_reader(
  token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))

Door 2 — the required loop step

The tap's exports to foam are outbound HTTP calls THEIR instrumentation will span, and their pipeline hands those spans back to the tap — a feedback loop in THEIR pipeline. Three layers keep it closed:

The SDK suppression context (automatic): foam's exporters send inside untraced, which the official http instrumentations respect.
REQUIRED — suppress foam's endpoint host in THEIR http instrumentation (belt for non-contrib patches and older contrib versions): add untraced_hosts: ["otel.api.foam.ai"] to their Net::HTTP/Excon config (scenario 4 snippet). When the OTEL_EXPORTER_OTLP_ENDPOINT override is active, list THAT host instead. Faraday/HTTP (httprb)/HTTPX cannot take the option, but they also never see the tap's exports — foam's exporters use Net::HTTP only.
The echo filter (structural backstop): a client-kind span naming the ACTIVE foam export host is telemetry about foam's own export call — the tap refuses to re-export its own echo, with one loud warning naming the missing step. Their pipeline keeps that span untouched.

Identity on the wire

	their pipeline	foam's copy
`service.name` / `service.version` / `service.instance.id`	theirs	theirs, untouched (the data carries THEIR identity)
all attributes / events / bodies	raw, byte-identical	the credential floor + your per-tap `redact_keys`/`redact_pii_keys` applied (default: floor only — everything off the frozen list identical to theirs)
`deployment.environment.name`	theirs (if any)	the tap's `environment:` argument (stamped at export time)
`foam.ingest.tier`	absent	`"external"` (the wire-contract marker)
`telemetry.distro.name` / `telemetry.distro.version`	absent	`"foam"` / the gem version (support reads the tap's presence and version from telemetry alone)

The stamp is merged onto FOAM'S COPY at export serialization — never onto the shared objects; it changes exactly those four keys and nothing else.

External-tier caveats (recorded, never fought)

THEIR sampler gates what the traces tap sees (unsampled spans never reach processors).
THEIR Views shape what the metrics tap receives; per-reader temporality is not a Ruby knob — aggregations default cumulative (or the operator's OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE), and cumulative totals start at attach time.
Lifecycle rides THEIR pipeline: their provider's shutdown/force_flush drives the tap's. Serverless mirror: flush THEIR provider at handler end (the platform freezes the process before batch timers fire).
Kill switch and endpoint override apply to taps exactly as to init: OTEL_SDK_DISABLED=true at construction → inert instance + one warning; OTEL_EXPORTER_OTLP_ENDPOINT honored with the loud warning; per-signal OTLP endpoint variants stay INERT + warned.

Puma / Unicorn note (door 2)

Construct taps where THEIR setup runs — with SDK.configure in an initializer under cluster preload, taps are inherited by workers. The span and log taps self-heal in the child (the stock batch processors reset on first use after a fork); the metrics tap is restarted by foam's own Process._fork hook, which the factory installs (a separate registry from init's own readers, so a per-worker re-init never orphans a tap — GOTCHAS R7). Foam::Otel.after_fork! covers both if you wire it manually.

Tests / CI

Gate the add-line on the same condition you would give enabled: — the placement line IS the switch (no foam-invented env var):

if ENV["APP_ENV"] == "production"
  c.add_span_processor(Foam::Otel.create_ingest_span_processor(
    token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))
end

Failure modes — what happens and where the warning appears

Situation	Behavior
blank `name`/`environment`, or `enabled` omitted	raises `ArgumentError` at boot (on your machine)
`enabled: true` but blank `token`	raises `ArgumentError` at boot
`enabled: false`	fully inert; helpers no-op silently
missing `version`	`[foam]` warning, continues
a foreign OTel SDK owns a signal	`[foam]` warning once, naming the owner AND the matching door-2 ingest entry; foam dark for that signal
`init` called twice	`[foam]` warning; returns the first instance (providers are process-global)
ingest entry with blank `token:`/`environment:`	raises `ArgumentError` at construction (on your machine)
ingest entry under `OTEL_SDK_DISABLED=true`	inert no-op instance + one `[foam]` warning; their setup line never crashes
ingest tap wired onto foam's OWN pipeline	`[foam]` wiring-bug warning once; the tap STANDS DOWN (zero duplicates)
`init()` exports a signal AND a tap on a separate foreign pipeline	`[foam]` wiring-bug warning once; both keep flowing (`foam.ingest.tier` distinguishes them)
their instrumentation traces foam's export calls (missing loop step)	echo filter refuses foam's own echo + one `[foam]` warning naming the required `untraced_hosts` step

Environment variables

Var	Posture
`OTEL_SDK_DISABLED=true`	HONORED — full kill switch, supersedes `enabled: true` (matched trimmed + case-insensitive, so `TRUE` / `true` also kill — this one emergency switch behaves the same in every foam SDK). Door-2 ingest entries honor it too: construction returns an inert instance (read once at construction; changing it implies a restart).
`OTEL_EXPORTER_OTLP_ENDPOINT`	HONORED — the one operator-level override of the pinned foam endpoint (for foam's own conformance rig / enterprise egress). Active → loud `[foam]` warning naming the destination. Applies to door-2 taps identically (and moves the host the required loop step must name).
`OTEL_PROPAGATORS=none`	HONORED — turns trace propagation OFF (links lost) while telemetry keeps flowing; warns. Any other value warns and is ignored (foam's propagator set is fixed: W3C tracecontext + baggage).
`OTEL_BSP_SCHEDULE_DELAY` / `OTEL_BLRP_SCHEDULE_DELAY` / `OTEL_METRIC_EXPORT_INTERVAL`	HONORED — batch cadence, read natively by the upstream SDK.
`FOAM_CAPTURE_PAYLOADS`	HONORED — the operator clamp over the `capture_payloads:` init option (`off`/`errors`/`always`, case-insensitive), overriding it in BOTH directions with one loud `[foam]` line when it changes the mode. An invalid value warns and falls back to the init option (never crashes a boot). This is the ONE foam-named env var the gem reads — an override valve over an init-declared option, never an on/off switch, token, or config fallback (those still arrive only through `init`'s explicit arguments). Read once at init; changing it implies a restart.
`OTEL_RUBY_INSTRUMENTATION_RACK_CONFIG_OPTS`	HONORED (by the contrib rack gem itself) and RESPECTED by foam — a header option you set here (`allowed_request_headers=…` / `allowed_response_headers=…`) governs that option; foam's default header list fills only the option you did not touch (never widened, never overridden — see "Header capture"). The other `OTEL_RUBY_INSTRUMENTATION_<NAME>_CONFIG_OPTS` vars are likewise the contrib gems' own standard levers (e.g. the Sidekiq `propagation_style` note above).
`OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`	INERT — foam wires ONE resolved endpoint into all three exporters explicitly, so per-signal endpoint vars never redirect (or split) foam's export. Set → loud `[foam]` warning that it is inert.
`OTEL_EXPORTER_OTLP_METRICS_ENDPOINT`	INERT — as above (warns when set).
`OTEL_EXPORTER_OTLP_LOGS_ENDPOINT`	INERT — as above (warns when set).
`OTEL_RUBY_EXPORTER_OTLP_SSL_VERIFY_NONE`	INERT — foam pins TLS peer verification (`ssl_verify_mode: VERIFY_PEER`) on every exporter constructor, both doors, so this vendored-exporter knob can never strip certificate verification off the credential-bearing export connection (1.4.0 security fix, CWE-295). Set → loud `[foam]` warning that it is inert; fix the trust chain (CA bundle) instead.
every other `OTEL_*`	INERT — `OTEL_SERVICE_NAME` / `OTEL_RESOURCE_ATTRIBUTES` never override the init-declared identity; sampler/exporter/span-limit vars are ignored.

Troubleshooting

Telemetry halved or doubled after a dependency bump → check for a second copy of the OTel API first: bundle list | grep opentelemetry-api must show ONE. Two copies mean two global registries (silent no-op or doubled data).
No data at all → set diagnostics: true and read the [foam] lines; check init runs before your app code, that enabled is true, and that a token is wired.
No door-2 data → check, in order: OTEL_SDK_DISABLED unset (a killed tap is inert and warned once at boot); the tap is on the RIGHT provider (the one their setup actually registers — a wiring-bug warning at first traffic means self-ingest stand-down); the [foam] wiring-bug/echo warnings in the boot log; construct with diagnostics: true for per-export narration; for metrics under Puma, see the Door 2 Puma note.
A downstream rejects traceparent → set OTEL_PROPAGATORS=none to stop injection while keeping telemetry flowing.

See GOTCHAS.md (the traps and how foam handles them) and RESEARCH.md (the primitive audit, instrumentation census, and redaction provenance).