Module: Hecks::Adapters::PostgresEra::Lineage::ResumableBackfill

Included in:
Hecks::Adapters::PostgresEra::Lineage
Defined in:
lib/hecks/adapters/driven/postgres_era/lineage/resumable_backfill.rb

Overview

THE ONE CHUNKED, LOCK-FREE, RESUMABLE BACKFILL LOOP — shared by backfill_head_snapshot! (era 1's existing one-shot blocking backfill, retrofit) and every field-cache table's own initial backfill (new). Governing principle 1 (docs/implemented/postgres-era-adapter- split-plan.md): no operation this plan touches may hold a lock across a scan whose duration scales with table size — a single INSERT ... SELECT over the whole journal (what backfill_head_snapshot! used to be) is exactly that, and so is a naive "populate every cache row in one statement" field-cache backfill.

THE SHAPE: read one bounded chunk (real rows, real ordinals) with a plain SELECT — no lock held across it, so an ordinary reader or writer is never blocked by a backfill in progress — then upsert that chunk under the SAME transactionally-scoped advisory lock + ordinal-guard idiom append's own snapshot upsert already uses (WHERE ordinal < EXCLUDED.ordinal), then persist a cursor before moving to the next chunk. Repeat until a chunk reads back short of a full page — that page was the last one.

RESUMABLE, not merely restartable. A crash (or a second concurrent boot) mid-backfill leaves the cursor exactly where the last COMMITTED chunk left it — hecks_backfill_progress is updated in the SAME transaction as the chunk's own upsert, so cursor and data can never observably disagree (see run_chunk!). The next attempt reads that cursor and continues; it does not rescan what a prior attempt already committed. RESTARTABLE would also be CORRECT here (every upsert is idempotent and ordinal-guarded — rerunning an already-done chunk from id 1 changes nothing) but wastes real work on a large table; resumability is what keeps a crash near the END of a large backfill cheap to recover from instead of starting over.

THE LOCK KEY PREFIX is hecks_field_cache: — deliberately disjoint from the three families already in use elsewhere in this adapter (hecks_ordinal:, hecks_eras:, hecks_head_snapshot: — see lineage.rb/head_compiler.rb/ mint_transaction.rb/tail_merge.rb) so a backfill chunk NEVER contends with a plain write, a mint, or a snapshot-table's own first-creation lock. It is held for exactly ONE CHUNK's own transaction, never across the whole backfill — two concurrent backfillers of the SAME target simply take turns one chunk at a time rather than racing to duplicate work; neither blocks an unrelated reader or writer for even an instant.

Constant Summary collapse

CHUNK_SIZE = 
5_000

Instance Method Summary collapse

Instance Method Details

#chunked_backfill!(target, source_sql:, upsert:) ⇒ Object

Drives target (an already-created, currently-empty-or- partially-filled table) through chunks until a source read comes back short of CHUNK_SIZE rows. Two distinct callables, not one — a head-snapshot row and a field-cache row carry different columns (state jsonb vs. a single extracted value), so there is no one generic "upsert this row" shape to share; only the LOOP, the lock, and the cursor are generic.

source_sql.call(cursor) — given the last-processed id (nil
before the first chunk), returns a SQL SELECT whose result
has an `id` column (text, ordered ascending) plus whatever
other columns `upsert` below needs. Must read `id >
cursor` (or unconditional when cursor is nil), `ORDER BY
id`, `LIMIT CHUNK_SIZE` — the caller owns the actual
column list/source tables; this method only owns the loop,
the lock, and the cursor.

upsert.call(rows) — given the PG::Result of one chunk's
read, performs the actual guarded upsert into `target` and
returns nothing meaningful; runs INSIDE the same
transaction/advisory-lock scope as the cursor update below,
so a crash between "wrote the chunk" and "advanced the
cursor" is impossible — they commit together or not at
all.


90
91
92
93
94
95
96
 
# File 'lib/hecks/adapters/driven/postgres_era/lineage/resumable_backfill.rb', line 90

def chunked_backfill!(target, source_sql:, upsert:)
  ensure_backfill_progress_table!
  loop do
    done = run_chunk!(target, source_sql: source_sql, upsert: upsert)
    break if done
  end
end

#ensure_backfill_progress_table! ⇒ Object

Idempotent, unguarded — same idiom as every other DDL helper in this file tree (ensure_head_snapshot! et al.): cheap, runs on every boot, only ever does real work once.



55
56
57
58
59
60
61
62
63
64
 
# File 'lib/hecks/adapters/driven/postgres_era/lineage/resumable_backfill.rb', line 55

def ensure_backfill_progress_table!
  @db.exec(<<~SQL)
    CREATE TABLE IF NOT EXISTS hecks_backfill_progress (
      target     text PRIMARY KEY,
      cursor     text,
      completed  boolean NOT NULL DEFAULT false,
      updated_at timestamptz NOT NULL DEFAULT now()
    )
  SQL
end