foam-otel (Ruby)

Foam's OpenTelemetry core for Ruby services. A thin, safe wrapper over the official OpenTelemetry libraries: foam owns the pipeline (providers, batch processors, OTLP export to the foam fleet endpoint), turns on automatic tier-1/2 instrumentation, and hands you a small set of never-throw helpers — and nothing else. Redaction is fully opt-in: by default foam captures every value RAW (no secret floor, no value-pattern masking); you enumerate the fields to mask via redact_keys/redact_pii_keys.

Built to docs/BASE_PACKAGE_SPEC.md. This README is the manual you integrate from.

Foam builds on the OpenTelemetry authors' work (https://opentelemetry.io) — see THIRD-PARTY-NOTICES. Licensed, not sold — see LICENSE.


Install

# Gemfile
gem "foam-otel"
# foam-otel bundles the UNIVERSAL FLOOR and auto-activates each piece when its
# target library is present (presence-checked, and gated on foam owning the
# slot it produces into): Rack + the Rails family, the HTTP clients (Net::HTTP,
# Faraday), the primary datastores (pg, mysql2, redis, mongo) with RAW
# db.statement capture, Sidekiq, the stdlib Logger bridge, hand-written Ruby
# runtime + GC metrics, session stitching (browser session.id via baggage),
# and the LLM shims (OpenAI, Anthropic, Gemini, ruby_llm — activity + raw
# content). Add only the niche long tail your app needs beyond the floor:
gem "opentelemetry-instrumentation-graphql"  # e.g. GraphQL
gem "opentelemetry-instrumentation-resque"   # e.g. Resque

Supported versions (tested in CI, spec/version_spec.rb): Ruby >= 3.1; OpenTelemetry API opentelemetry-api ~> 1.1 (1.x). The metrics and logs SDKs are pre-1.0 upstream (metrics alpha, logs development) and are pinned with pessimistic constraints; foam wraps them behind stable helpers so app code never touches the churning API (see GOTCHAS R2).


Quick start — scenario 1: a clean service (the common case)

Put init FIRST, before your app boots. In Rails, that means an initializer (and, for cluster mode, per worker — see "Puma / Unicorn" below).

# config/initializers/foam.rb  (the one config module you own)
require "foam/otel"

Foam::Otel.init(
  name: "checkout-api",                       # → service.name
  environment: ENV.fetch("APP_ENV", "development"),  # → deployment.environment.name (verbatim)
  enabled: !%w[test ci].include?(ENV["APP_ENV"]),    # silence tests/CI; export everywhere else
  token: ENV.fetch("FOAM_OTEL_TOKEN"),             # required WHEN enabled — wire it explicitly
  version: ENV["GIT_SHA"]                      # → service.version (recommended; optional)
)

Outcome: foam registers the tracer/meter/logger providers, exports OTLP to the foam fleet endpoint, and auto-activates the universal floor — the official rack/rails instrumentation plus the bundled HTTP (Net::HTTP/Faraday), datastore (pg/mysql2/redis/mongo), and Sidekiq instrumentations — each presence-gated, and gated on foam owning the traces slot (a foreign-owned tracer stands the activation down). The datastore set is activated with db_statement: :include — foam explicitly overrides the upstream :obfuscate default so RAW SQL / redis args / mongo commands land on the wire. On top of the contrib set the floor adds foam's own pieces: the stdlib Logger bridge (every host Logger line also lands as a trace-correlated OTel log record, body raw), hand-written Ruby runtime + GC metrics (GC counts + timing, heap slots, allocation, thread count, RSS — Ruby has no drop-in upstream metrics gem, so foam ships its own collector), session stitching (inbound baggage: session.id from the foam browser package is stamped onto every span and log record), and the LLM shims (below). Every inbound request, DB call, HTTP call, job, log line, and model call now flows to foam. Redaction is opt-in: by default every value — attributes, URLs/query strings, DB statement text, log bodies, LLM content — is captured RAW. Pass redact_keys/redact_pii_keys to mask or erase specific fields (see below).

The LLM surface — OpenAI, Anthropic, Gemini (and ruby_llm)

The Ruby contrib registry ships no LLM instrumentation, so foam does: thin, presence-checked shims over the SDKs' public call sites, emitting the standard gen_ai.* semantic conventions — activity (model, response id, finish reasons, token usage, latency) AND prompt/response content, RAW (gen_ai.input.messages / gen_ai.output.messages / gen_ai.system_instructions). Covered: the official openai SDK (chat.completions.create + responses.create), the official anthropic SDK (messages.create), Gemini via the gemini-ai gem (Google ships no official Ruby SDK), and ruby_llm (one seam covering all its providers). Absent SDK = inert; version-guarded; fail-to-dark — a shim failure can NEVER break the model call, and the SDK's own error re-raises identically. One model call = one gen_ai client span (the SDK's HTTP send rides the same suppression context foam's exporters use). Streaming call sites (#stream/#stream_raw on the official SDKs) are a recorded follow-up.

Session stitching — browser session.id on backend telemetry

The foam browser package stamps session.id on every browser signal and sends it to your API as W3C Baggage beside traceparent. This gem's floor copies session.id (and session.previous_id across a rotation) from the inbound baggage onto every span and log record foam emits — browser session → backend traces/logs, stitched with no ingest-side join. No-op when the baggage is absent. CORS note (the FDE wires this on every frontend-called API): the allowlist must include BOTH headers — Access-Control-Allow-Headers: traceparent, baggage.


The init options

Option Type Required Default What it does
name: String yes service.name. Blank → raises at boot.
environment: String yes deployment.environment.name, exported verbatim. A value outside {production, staging, development, test} warns but is never rewritten. Blank → raises.
enabled: Boolean yes The config switch. false = fully inert (no SDK, no providers, no network, helpers no-op). true = export, in EVERY environment. No default — you write the logic.
token: String when enabled Authorization: Bearer for export. Required (and validated) only when enabled: true; wired explicitly from your secret source — never an env fallback read by the gem. Fleet convention: read it from the FOAM_OTEL_TOKEN env var. Blank while enabled → raises at boot.
version: String no nil service.version, verbatim (git SHA recommended). Missing → warns and continues. Never detected at runtime.
redact_keys: Array no [] The ONLY field names tail-masked (shape-preserving, e.g. ********f456). Opt-in — empty (the default) means NO masking. Matches by case-insensitive substring, including keys nested in structured values and query-string keys.
redact_pii_keys: Array no [] Field names fully erased to [REDACTED] (no tail). Opt-in — empty (the default) means NO erasure. Foam ships no preset; this is your enumeration.
additional_span_processors: Array no [] Tenant seam: constructed SpanProcessor instances added to foam's pipeline (additive; never replace foam's export). See coexistence.
additional_log_record_processors: Array no [] Tenant seam, logs.
additional_metric_readers: Array no [] Tenant seam, metrics.
additional_instrumentations: Array no [] Constructed tier-2 instrumentation instances to register (fault-isolated: one that throws is skipped with a [foam] warning).
ignored_outbound_hosts: Array no [] Hosts whose outbound calls produce no spans — EXTENDS the built-in export-loop guard. For a co-resident agent's intake or a tenant exporter's endpoint. Applies to the clients whose official instrumentation supports host suppression: Net::HTTP and Excon. Faraday/HTTP (httprb)/HTTPX have no upstream untraced_hosts option — foam warns loudly at init when one of those is bundled (GOTCHAS G13); foam's own export loop is guarded for every client regardless.
diagnostics: Boolean no false Verbose [foam] self-reporting of init/health. Warnings and errors are always loud regardless.

Examples for the remaining options (each is runnable as-is):

# Redaction is OPT-IN (default: nothing is masked). List the fields to redact:
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"),
                redact_keys: %w[password api_key internal_ref voucher_code], # tail-masked
                redact_pii_keys: %w[customer_email full_name])               # fully [REDACTED]
# Constructed tier-2 instrumentation instances (fault-isolated; one that
# throws is skipped with a [foam] warning, never a crashed boot):
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"),
                additional_instrumentations: [
                  OpenTelemetry::Instrumentation::Sidekiq::Instrumentation.instance,
                  [OpenTelemetry::Instrumentation::PG::Instrumentation.instance, { db_statement: :omit }],
                ])
# Tenant seam, logs + metrics (spans are shown in scenario 3 below):
Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"),
                additional_log_record_processors: [EvalTool::LogRecordProcessor.new],
                additional_metric_readers: [EvalTool::MetricReader.new],
                ignored_outbound_hosts: ["ingest.eval-tool.example"]) # the tenant loop guard — required
# Verbose self-reporting while wiring foam up (warnings are loud regardless):
Foam::Otel.init(name: "checkout-api", environment: "development", enabled: true,
                token: ENV.fetch("FOAM_OTEL_TOKEN"), diagnostics: true)

And the unhappy path — a blank required argument fails AT BOOT, on your machine, never silently:

Foam::Otel.init(name: "", environment: "production", enabled: true, token: ENV.fetch("FOAM_OTEL_TOKEN"))
# => raises ArgumentError: Foam::Otel.init requires a non-empty name

Deliberately absent (and why): endpoint, cadence/sampling, is_production, force_export, disabled_environments, and any processor/exporter injection beyond the tenant seam. Customers identify their service and turn foam on or off; they can never drop a share of the data, inject their own pipeline pieces, or redirect export from code. The only endpoint override is the operator-level OTEL_EXPORTER_OTLP_ENDPOINT env var (below).


The helpers

All helpers never raise, and no-op silently before init and when disabled.

Lifecycle

Foam::Otel.flush       # force-flush buffered pipelines (call at handler/worker exit)
Foam::Otel.shutdown    # flush + shut down providers (normal exit is auto via at_exit)
Foam::Otel.after_fork! # restart metric-reader threads after a fork (see cluster mode)

Traces

result = Foam::Otel.span("nightly-sync") do |span|
  # ... work ...  the span always ends; a raised error is recorded + re-raised
end

Foam::Otel.set_attribute("user.tier", "pro")           # on the active span; false if none recording
Foam::Otel.set_attributes("a" => 1, "b" => 2)
Foam::Otel.add_event("cache.miss")
Foam::Otel.record_exception(e)                          # records on active span; never notifies a tracker
ctx = Foam::Otel.get_trace_context                      # {trace_id:, span_id:} hex, or nil — never fabricated

Metrics (one helper per synchronous instrument; names pass through verbatim)

Foam::Otel.increment_counter("checkout.completed", by: 1, attributes: { "plan" => "pro" })
Foam::Otel.record_histogram("checkout.duration_ms", 428)
Foam::Otel.add_up_down_counter("queue.depth", by: -1)
Foam::Otel.set_metric("cache.size_bytes", 10_485_760)   # gauge (last value wins)

⚠️ Metric attributes are a loaded gun: an UNBOUNDED value (user id, raw path, full URL) creates one metric stream per value — a memory leak that the SDK's cardinality cap then turns into silently flat dashboards. Use bounded, low-cardinality attribute values only.

Logs

Foam::Otel.log(:info, "checkout finished", attributes: { "order.id" => id })
# severity is a level symbol (:trace :debug :info :warn :error :fatal) or an
# OTel severity number (1..24). Trace-correlated when a span is active.

Redaction

Foam::Otel.redact(user_supplied_token)  # tail-masks a scalar you know is sensitive; [REDACTED] for structures

Passthroughs (the real OTel objects on foam's pipeline — links, span kinds, observable instruments, baggage all reachable here)

Foam::Otel.get_tracer("my.lib")
Foam::Otel.get_meter("my.lib")
Foam::Otel.get_logger("my.lib")

Coexistence — foam arriving in a monitored service

init classifies each signal's global slot independently and does the right, non-destructive thing. The classifier's output is information for you, never a decision it makes for you.

Scenario 2 — beside a proprietary APM agent

Disjoint pipelines: foam runs its own beside the agent, touching nothing. Add the agent's intake host so foam's outbound instrumentation doesn't shadow its egress.

Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: FOAM_OTEL_TOKEN,
                ignored_outbound_hosts: ["agent-intake.vendor.example"])

Scenario 3 — a scoped tenant rides foam's pipeline (LLM-eval, AI-observability)

Attach the tenant's processor via the seam, and add its exporter host to the ignore list (its exports would otherwise be shadow-spanned into a loop).

Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: FOAM_OTEL_TOKEN,
                additional_span_processors: [EvalTool::SpanProcessor.new(project: "prod")],
                ignored_outbound_hosts: ["ingest.eval-tool.example"])

Ruby note: when you configure redact_keys/redact_pii_keys, foam's HELPERS apply the key pass at capture, so a tenant processor receives foam-helper data (span/log helper attributes, log bodies, metric labels) ALREADY masked. Because the Ruby SDK freezes span attributes at finish, a tenant processor sees third-party instrumentation attributes UNMASKED (for those, foam's key pass runs at the exporter boundary — GOTCHAS F1). With no keys configured (the default) nothing is masked on either path.

Loop-guard note: ignored_outbound_hosts suppresses spans for tenant/agent egress made through Net::HTTP or Excon. A tenant exporter using Faraday/HTTP (httprb)/HTTPX cannot be host-suppressed today (no upstream untraced_hosts option — foam warns at init; GOTCHAS G13).

Scenario 4 — a foreign OTel SDK already owns a signal (door 2)

If another OTel SDK registered (say) the tracer before foam, foam does NOT displace it. init warns once, naming the owner, and foam is dark for that signal — your tier-3 telemetry for it rides the foreign SDK to ITS backend. To ALSO send that pipeline to foam, add ONE LINE per claimed signal in THEIR setup — the ingest entries (see "Door 2 — the ingest entries" below). YOU place the tap: foam never attaches itself to a pipeline it does not own (their data moves only when a human says so), and the placement line doubles as the off-switch — gate it on the same condition you would give enabled:.

# config/initializers/opentelemetry.rb — THEIR setup, plus the foam lines
require "opentelemetry/sdk"
require "foam/otel"

OpenTelemetry::SDK.configure do |c|
  c.service_name = "their-service"
  c.use_all
  # REQUIRED loop step (see "Door 2 — the required loop step" below):
  c.use "OpenTelemetry::Instrumentation::Net::HTTP", { untraced_hosts: ["otel.api.foam.ai"] }
  c.add_span_processor(their_vendor_processor)           # their export, untouched
  if ENV["APP_ENV"] == "production"                       # your consent AND your off-switch
    c.add_span_processor(Foam::Otel.create_ingest_span_processor(
      token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))
  end
end

Outcome: their pipeline keeps working exactly as before — their data, their resource, their export, byte-identical — and foam ALSO receives the signal, carrying THEIR resource identity plus foam's export-time stamp and any per-tap redact_keys/redact_pii_keys you set (opt-in — none by default; details in the Door 2 section).

Unhappy paths:

Foam::Otel.create_ingest_span_processor(token: "", environment: "production")
# => raises ArgumentError: [foam] Foam::Otel.create_ingest_span_processor requires a non-empty token
  • OTEL_SDK_DISABLED=true at construction → the factory returns an inert no-op instance (their setup line keeps working, nothing is sent) and warns: [foam] OTEL_SDK_DISABLED=true — this foam ingest entry is disabled by the operator kill switch and will send nothing.
  • Tap wired onto FOAM'S OWN pipeline (self-ingest wiring bug) → one loud warning and the tap STANDS DOWN — door 1 already exports that data, so nothing is lost and nothing is doubled.
  • init() already exports a signal AND a tap sits on a separate foreign pipeline for the same signal → one loud warning; BOTH keep flowing (different pipelines; foam.ingest.tier distinguishes the copies).

When the TRACES slot is foreign-owned, foam also skips its automatic tier-1/2 instrumentation activation (it never adds span producers or Rack middleware to a pipeline it does not own) and warns once; additional_instrumentations you pass explicitly are still installed. The same holds for the process-wide propagator: a propagator the incumbent installed first is KEPT (foam installs its W3C set only over the API default), with one warning.

Scenario 5 — per-signal composition

Signals are independent. A foreign SDK owning only traces still gets foam for metrics and logs on the full contract:

Foam::Otel.init(name: "checkout-api", environment: env, enabled: true, token: FOAM_OTEL_TOKEN)
# → warns: traces foreign-owned (foam dark for traces); metrics + logs → foam, full contract.

# The doors COMPOSE: add the traces tap in THEIR setup and foam receives
# traces too — the tap rides THEIR tracer, and trace ids correlate with
# foam-exported logs/metrics through the shared context:
their_provider.add_span_processor(Foam::Otel.create_ingest_span_processor(
  token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))
# (no double-claim warning: init() serves metrics+logs, the tap serves traces)

Scenario 6 — tests / CI (fully inert, no token needed)

Foam::Otel.init(name: "checkout-api", environment: "test", enabled: false)

Scenario 7 — serverless / short-lived (flush at the end)

Foam::Otel.init(name: "orders-fn", environment: "production", enabled: true, token: FOAM_OTEL_TOKEN)

def handler(event:, context:)
  run(event)
ensure
  Foam::Otel.flush   # the platform freezes the process before the batch timer fires
end

Puma / Unicorn (cluster mode) — CRITICAL

Exporter threads do not survive fork. Foam auto-recovers metric readers via a Process._fork hook, but the cleanest pattern is to init PER WORKER so each worker gets its own pipeline and a unique service.instance.id:

# config/puma.rb
on_worker_boot do
  Foam::Otel.init(name: "checkout-api", environment: ENV["APP_ENV"], enabled: true, token: ENV["FOAM_OTEL_TOKEN"])
  # or, if you init in an initializer: Foam::Otel.after_fork!
end

Door 2 — the ingest entries

When a foreign OTel SDK owns a signal (scenario 4), the ingest entries are how that pipeline's data ALSO reaches foam: constructed instances you add — one line per claimed signal — to the CUSTOMER'S OWN OTel setup. Additive readers only: they never mutate their data, their resource, or their export.

All three entries are SHIPPED. The traces entry rides the stable 1.x SDK; the log and metric entries ride upstream's pre-1.0 SDKs (logs "development", metrics alpha), pinned to this gem's capped ranges (GOTCHAS R2/R5) so a 0.x drift is a deliberate re-prove, never a silent change. The pre-1.0 SDK maturity is a version-pin caveat, not a deferral — the entries are available and supported today.

Version-pin posture: because the log and metric taps ride those pre-1.0 upstream SDKs, they are version-pinned and re-proven on every 0.x bump rather than assumed stable — shipped and supported either way.

The three entries

All three share one signature (token: and environment: required; the redaction kwargs are the ONLY keys that tap redacts — opt-in, none by default, scoped to that tap; diagnostics: is tap-scoped narration). Construction validates loudly at boot and NEVER throws afterwards — a failure inside a tap can never break their pipeline. Fleet canon mapping (pinned by spec/export_surface_spec.rb): create_ingest_span_processor = createFoamIngestSpanProcessor, create_ingest_log_record_processor = createFoamIngestLogRecordProcessor, create_ingest_metric_reader = createFoamIngestMetricReader.

Traces — a SpanProcessor for THEIR TracerProvider (scenario 4 snippet):

processor = Foam::Otel.create_ingest_span_processor(
  token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV"),
  redact_keys: %w[voucher_code],          # optional: extra secret keys, this tap only
  redact_pii_keys: %w[customer_email],    # optional: your PII enumeration, this tap only
  diagnostics: false)                     # optional: per-tap [foam] narration
their_tracer_provider.add_span_processor(processor)
# returns a new instance per call; no-op inert instance under OTEL_SDK_DISABLED=true;
# raises ArgumentError at construction for a blank token/environment

Logs — a LogRecordProcessor for THEIR LoggerProvider. The tap's redact_keys/redact_pii_keys (if any) match over record attributes and keys nested in structured bodies; free-text bodies ride RAW (no value-pattern pass):

provider = OpenTelemetry::SDK::Logs::LoggerProvider.new(resource: their_resource)
provider.add_log_record_processor(their_log_processor)
provider.add_log_record_processor(Foam::Otel.create_ingest_log_record_processor(
  token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))
OpenTelemetry.logger_provider = provider

Metrics — a PeriodicMetricReader for THEIR MeterProvider. Attach order is forgiving: add_metric_reader back-fills instruments created before the tap attached. Foam's reader collects on its own schedule beside theirs:

OpenTelemetry.meter_provider.add_metric_reader(Foam::Otel.create_ingest_metric_reader(
  token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))

Door 2 — the required loop step

The tap's exports to foam are outbound HTTP calls THEIR instrumentation will span, and their pipeline hands those spans back to the tap — a feedback loop in THEIR pipeline. Three layers keep it closed:

  1. The SDK suppression context (automatic): foam's exporters send inside untraced, which the official http instrumentations respect.
  2. REQUIRED — suppress foam's endpoint host in THEIR http instrumentation (belt for non-contrib patches and older contrib versions): add untraced_hosts: ["otel.api.foam.ai"] to their Net::HTTP/Excon config (scenario 4 snippet). When the OTEL_EXPORTER_OTLP_ENDPOINT override is active, list THAT host instead. Faraday/HTTP (httprb)/HTTPX cannot take the option, but they also never see the tap's exports — foam's exporters use Net::HTTP only.
  3. The echo filter (structural backstop): a client-kind span naming the ACTIVE foam export host is telemetry about foam's own export call — the tap refuses to re-export its own echo, with one loud warning naming the missing step. Their pipeline keeps that span untouched.

Identity on the wire

their pipeline foam's copy
service.name / service.version / service.instance.id theirs theirs, untouched (the data carries THEIR identity)
all attributes / events / bodies raw, byte-identical your per-tap redact_keys/redact_pii_keys applied (default: none → identical to theirs)
deployment.environment.name theirs (if any) the tap's environment: argument (stamped at export time)
foam.ingest.tier absent "external" (the wire-contract marker)
telemetry.distro.name / telemetry.distro.version absent "foam" / the gem version (support reads the tap's presence and version from telemetry alone)

The stamp is merged onto FOAM'S COPY at export serialization — never onto the shared objects; it changes exactly those four keys and nothing else.

External-tier caveats (recorded, never fought)

  • THEIR sampler gates what the traces tap sees (unsampled spans never reach processors).
  • THEIR Views shape what the metrics tap receives; per-reader temporality is not a Ruby knob — aggregations default cumulative (or the operator's OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE), and cumulative totals start at attach time.
  • Lifecycle rides THEIR pipeline: their provider's shutdown/force_flush drives the tap's. Serverless mirror: flush THEIR provider at handler end (the platform freezes the process before batch timers fire).
  • Kill switch and endpoint override apply to taps exactly as to init: OTEL_SDK_DISABLED=true at construction → inert instance + one warning; OTEL_EXPORTER_OTLP_ENDPOINT honored with the loud warning; per-signal OTLP endpoint variants stay INERT + warned.

Puma / Unicorn note (door 2)

Construct taps where THEIR setup runs — with SDK.configure in an initializer under cluster preload, taps are inherited by workers. The span and log taps self-heal in the child (the stock batch processors reset on first use after a fork); the metrics tap is restarted by foam's own Process._fork hook, which the factory installs (a separate registry from init's own readers, so a per-worker re-init never orphans a tap — GOTCHAS R7). Foam::Otel.after_fork! covers both if you wire it manually.

Tests / CI

Gate the add-line on the same condition you would give enabled: — the placement line IS the switch (no foam-invented env var):

if ENV["APP_ENV"] == "production"
  c.add_span_processor(Foam::Otel.create_ingest_span_processor(
    token: ENV.fetch("FOAM_OTEL_TOKEN"), environment: ENV.fetch("APP_ENV")))
end

Failure modes — what happens and where the warning appears

Situation Behavior
blank name/environment, or enabled omitted raises ArgumentError at boot (on your machine)
enabled: true but blank token raises ArgumentError at boot
enabled: false fully inert; helpers no-op silently
missing version [foam] warning, continues
a foreign OTel SDK owns a signal [foam] warning once, naming the owner AND the matching door-2 ingest entry; foam dark for that signal
init called twice [foam] warning; returns the first instance (providers are process-global)
ingest entry with blank token:/environment: raises ArgumentError at construction (on your machine)
ingest entry under OTEL_SDK_DISABLED=true inert no-op instance + one [foam] warning; their setup line never crashes
ingest tap wired onto foam's OWN pipeline [foam] wiring-bug warning once; the tap STANDS DOWN (zero duplicates)
init() exports a signal AND a tap on a separate foreign pipeline [foam] wiring-bug warning once; both keep flowing (foam.ingest.tier distinguishes them)
their instrumentation traces foam's export calls (missing loop step) echo filter refuses foam's own echo + one [foam] warning naming the required untraced_hosts step

Environment variables

Var Posture
OTEL_SDK_DISABLED=true HONORED — full kill switch, supersedes enabled: true (matched trimmed + case-insensitive, so TRUE / true also kill — the fleet's one emergency switch behaves the same across languages). Door-2 ingest entries honor it too: construction returns an inert instance (read once at construction; changing it implies a restart).
OTEL_EXPORTER_OTLP_ENDPOINT HONORED — the one operator-level override of the pinned fleet endpoint (for foam's own conformance rig / enterprise egress). Active → loud [foam] warning naming the destination. Applies to door-2 taps identically (and moves the host the required loop step must name).
OTEL_PROPAGATORS=none HONORED — turns trace propagation OFF (links lost) while telemetry keeps flowing; warns. Any other value warns and is ignored (foam's propagator set is fixed: W3C tracecontext + baggage).
OTEL_BSP_SCHEDULE_DELAY / OTEL_BLRP_SCHEDULE_DELAY / OTEL_METRIC_EXPORT_INTERVAL HONORED — batch cadence, read natively by the upstream SDK.
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT INERT — foam wires ONE resolved endpoint into all three exporters explicitly, so per-signal endpoint vars never redirect (or split) foam's export. Set → loud [foam] warning that it is inert.
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT INERT — as above (warns when set).
OTEL_EXPORTER_OTLP_LOGS_ENDPOINT INERT — as above (warns when set).
every other OTEL_* INERT — OTEL_SERVICE_NAME / OTEL_RESOURCE_ATTRIBUTES never override the init-declared identity; sampler/exporter/span-limit vars are ignored.

Troubleshooting

  • Telemetry halved or doubled after a dependency bump → check for a second copy of the OTel API first: bundle list | grep opentelemetry-api must show ONE. Two copies mean two global registries (silent no-op or doubled data).
  • No data at all → set diagnostics: true and read the [foam] lines; check init runs before your app code, that enabled is true, and that a token is wired.
  • No door-2 data → check, in order: OTEL_SDK_DISABLED unset (a killed tap is inert and warned once at boot); the tap is on the RIGHT provider (the one their setup actually registers — a wiring-bug warning at first traffic means self-ingest stand-down); the [foam] wiring-bug/echo warnings in the boot log; construct with diagnostics: true for per-export narration; for metrics under Puma, see the Door 2 Puma note.
  • A downstream rejects traceparent → set OTEL_PROPAGATORS=none to stop injection while keeping telemetry flowing.

See GOTCHAS.md (the traps and how foam handles them) and RESEARCH.md (the primitive audit, instrumentation census, and redaction provenance).