Module: Apartment::TransactionTaint

Defined in:
lib/apartment/transaction_taint.rb

Overview

Heals a tenant connection left in an aborted transaction (PostgreSQL's PQTRANS_INERROR) at the moment it is checked back into its pool.

WHY CHECKIN. ActiveRecord's active? probes with an empty query, which does NOT error in an aborted transaction, so verify! pronounces a poisoned connection healthy and checkin does not reset it. The pool then serves it to the next caller. Under pool-per-tenant that connection is the ONLY connection for its tenant, so the tenant is dead on that worker until the process restarts, while every other tenant looks fine and nothing in the logs explains it. That amplification is what makes this ours to fix: the defect is generic Rails/PG, the consequence is specific to pool-per-tenant.

Checkin is also the one seam that serves both populations correctly. A FIXTURE-PINNED connection must KEEP its transaction (teardown_fixtures owns the rollback) and never reaches ConnectionPool#checkin's body; a production connection must be reset before reuse. We still test pinned explicitly, because this override runs BEFORE the early return in AR's own checkin.

NEVER issue a raw ROLLBACK here. It destroys the ENCLOSING transaction while ActiveRecord still believes its stack is intact, raises nothing, and lets subsequent writes autocommit — turning a loud failure into silent, permanent database pollution. reset! is ActiveRecord's own primitive: it rolls back, DISCARDs session state, and resets AR's transaction bookkeeping together. It also re-applies the pool's schema_search_path, so a healed tenant connection still points at its own schema (verified — the alternative would be a cross-tenant leak, which is strictly worse than the bug being fixed).

Design: docs/designs/transaction-taint-detection.md

Defined Under Namespace

Modules: PoolHeal

Class Method Summary collapse

Class Method Details

.heal(conn, pool) ⇒ Object

Best-effort, and it must never raise: an exception here would abort ConnectionPool#checkin and leak the connection out of the pool permanently, which is worse than the taint we came to fix.



60
61
62
63
64
65
66
67
68
69
70
 
# File 'lib/apartment/transaction_taint.rb', line 60

def heal(conn, pool)
  return if conn.pinned
  return unless Apartment.adapter&.aborted_transaction?(conn)

  open_transactions = conn.open_transactions
  conn.reset!
  warn_once(pool)
  instrument(pool, open_transactions: open_transactions, healed: true)
rescue StandardError => e
  discard(conn, pool, e)
end

.install(pool, tenant:, pool_key:) ⇒ Object

Extend pool with the heal. Called once, at tenant-pool creation.



48
49
50
51
52
53
54
55
 
# File 'lib/apartment/transaction_taint.rb', line 48

def install(pool, tenant:, pool_key:)
  return pool unless Apartment.config&.heal_tainted_connections

  pool.extend(PoolHeal)
  pool.apartment_tenant = tenant.to_s
  pool.apartment_pool_key = pool_key
  pool
end