Class: Hecks::Adapters::PostgresEra::Lineage

Inherits:
Object
  • Object
show all
Includes:
EraStore, FieldCache, HeadCompiler, MintTransaction, Provisioning, ResumableBackfill, TailMerge, TransformInstaller
Defined in:
lib/hecks/ports/persistence/plugins/era/postgres_era/lineage.rb,
lib/hecks/ports/persistence/plugins/era/postgres_era/lineage/era_store.rb,
lib/hecks/ports/persistence/plugins/era/postgres_era/lineage/tail_merge.rb,
lib/hecks/ports/persistence/plugins/era/postgres_era/lineage/field_cache.rb,
lib/hecks/ports/persistence/plugins/era/postgres_era/lineage/provisioning.rb,
lib/hecks/ports/persistence/plugins/era/postgres_era/lineage/head_compiler.rb,
lib/hecks/ports/persistence/plugins/era/postgres_era/lineage/mint_transaction.rb,
lib/hecks/ports/persistence/plugins/era/postgres_era/lineage/resumable_backfill.rb,
lib/hecks/ports/persistence/plugins/era/postgres_era/lineage/transform_installer.rb

Overview

The lineage topology inside one Postgres database: one journal per domain, LIST-partitioned by era, with a single ordinal sequence spanning partitions (total order across eras is structural); a hecks_eras table holding each era's frozen source text, its once-minted hash/label, and the watermark cut into its ancestor; and, per aggregate, a HEAD derived from the journal — never a table anything rewrites.

For era 1 the head is a plain view (latest save per id). From era 2 on, the ancestor tail is a MATERIALIZED view whose definition is the compiled, chained edge sequence — one CTE per original edge, in mint order, never a flattened merged rule set — with the watermark baked into the definition, so post-cut old-era writes cannot leak into the new head even on REFRESH. Live current-era writes overlay it through the head view.

Writing to a superseded schema drops the instant the new one materializes: ONE shared row policy admits INSERTs to whichever era was just established (era 1 at first hold, era N at mint), and advancing it is part of the SAME transaction that builds the new era's matview. There is no persisted per-role fork — any granted role, app or the table's own OWNER (FORCE ROW LEVEL SECURITY applies this to the owner too, not only ordinary roles), writes the current era or nothing, from the moment that transaction commits. A stale-era write during the narrow window before that commit — RLS is checked once, when the statement EXECUTES, never re-checked at commit, so a transaction that inserted while the old era was still current can still land after a concurrent mint has already moved the fence on — is exactly what diverged_count/merge_tail exist to reconcile; it is the residual of an unavoidable race (ordinals are sequence-assigned, not transactional — see below), not a supported way to keep operating two schemas side by side. Only an actual Postgres superuser (or a role granted BYPASSRLS) sits above FORCE and keeps writing at will, forever.

Lineage order is ordinal-assignment order: the ordinal comes from a sequence, and a sequence's nextval() is never rolled back with its transaction — accepted and documented rather than papered over.

ONE PART OF THAT IS CLOSED. Two concurrent PLAIN writes could call nextval() in one order and COMMIT in the other — nothing about a single autocommit INSERT statement stops a slower one from finishing after a faster one that started later — so "ordinal order" and "commit order" were formally two different total orders even with no mint anywhere near either write. PostgresEra#append now holds pg_advisory_xact_lock(hashtext('hecks_ordinal:' || domain)) for the length of its own transaction, a DIFFERENT key from mint_era! and merge_tail!'s hecks_eras:domain — so plain writes serialize against EACH OTHER only, never against a mint, and ordinal order equals commit order for them now.

THE OTHER PART IS NOT, ON PURPOSE. A stale-era write during the narrow window before a mint's fence-move commits is the SAME race by a different name — and closing it would mean a plain write serializing against a mint, which is exactly the guarantee postgres_lineage_spec.rb's "an old checkout keeps writing its own era THROUGH a mint" pins the ABSENCE of. That race stays the residual diverged_count/merge_tail! exist to reconcile, not something a plain write should ever block for.

One concern per file under lineage/: provisioning (DDL and the RLS posture), era_store (the hecks_eras rows and their integrity), mint_transaction (the one transaction that makes an era real), tail_merge (the one deliberate merge command), resumable_backfill (the one chunked/lock-free/resumable scan loop, shared by head_compiler's own backfill and field_cache's), head_compiler (the chained-edge SQL a head derives through), field_cache (the per-where-field read cache that lets a query skip the reduction entirely), transform_installer (the hecks_tr_* jsonb helpers).

Defined Under Namespace

Modules: EraStore, FieldCache, HeadCompiler, MintTransaction, Provisioning, ResumableBackfill, TailMerge, TransformInstaller

Constant Summary collapse

JOURNAL_COLUMNS = 
"ordinal, era, aggregate, aggregate_id, operation, state, mirrors".freeze

Constants included from ResumableBackfill

ResumableBackfill::CHUNK_SIZE

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from TransformInstaller

#install_transform_functions!, #install_transforms!

Methods included from FieldCache

#backfill_field_cache!, #delete_field_cache_row!, #ensure_field_cache!, #field_cache, #upsert_field_cache_row!

Methods included from HeadCompiler

#ancestor_latest, #ancestor_tail_sql, #backfill_head_snapshot!, #chain_sql, #compile_head!, #ensure_first_head!, #ensure_head_snapshot!, #latest_of, #latest_per_id, #layered_chain_sql, #names_by_era, #nested_transaction, #table_exists?, #translated_latest, #view_exists?

Methods included from ResumableBackfill

#chunked_backfill!, #ensure_backfill_progress_table!

Methods included from TailMerge

#conflict_ids, #diverged_count, #merge_tail!

Methods included from MintTransaction

#advance_era!, #grant_role!, #mint_era!

Methods included from EraStore

#approval_for, #archive_text!, #backfill_frozen_facts!, #current_era, #eras, #hold_first!, #last_ordinal, #mint_name!, #raw_era, #reattest!, #record_approval!, #verify_integrity!

Methods included from Provisioning

#ensure_base!, #ensure_partition!, #partition_attached?, #provisioner?, #rename_domain!

Constructor Details

#initialize(db, domain, formerly_known_as: nil) ⇒ Lineage

Returns a new instance of Lineage.



101
102
103
104
105
 
# File 'lib/hecks/ports/persistence/plugins/era/postgres_era/lineage.rb', line 101

def initialize(db, domain, formerly_known_as: nil)
  @db = db
  @domain = domain.to_s
  @formerly_known_as = formerly_known_as&.to_s
end

Instance Attribute Details

#db ⇒ Object (readonly)

Returns the value of attribute db.



99
100
101
 
# File 'lib/hecks/ports/persistence/plugins/era/postgres_era/lineage.rb', line 99

def db
  @db
end

#domain ⇒ Object (readonly)

Returns the value of attribute domain.



99
100
101
 
# File 'lib/hecks/ports/persistence/plugins/era/postgres_era/lineage.rb', line 99

def domain
  @domain
end

#formerly_known_as ⇒ Object (readonly)

Returns the value of attribute formerly_known_as.



99
100
101
 
# File 'lib/hecks/ports/persistence/plugins/era/postgres_era/lineage.rb', line 99

def formerly_known_as
  @formerly_known_as
end

Instance Method Details

#head_snapshot(storage_name, era) ⇒ Object

The transactionally-upserted read cache behind head_view — one row per LIVE id, keyed by id, carrying the ordinal it was last written at. Scoped by ERA, not just storage_name — an aggregate that ISN'T renamed across a mint keeps the SAME storage_name in both eras, so storage_name alone would have era N+1 sharing one physical table with era N: a freshly-minted era would inherit every pre-mint (and, worse, pre-rekey/pre-translation) row instead of starting empty. era-qualified naming is what partition/matview already do for exactly this reason.



121
 
# File 'lib/hecks/ports/persistence/plugins/era/postgres_era/lineage.rb', line 121

def head_snapshot(storage_name, era) = "#{storage_name}_head_snapshot_#{era}"

#head_view(storage_name) ⇒ Object 



111
112
113
114
115
116
117
118
119
120
 
# File 'lib/hecks/ports/persistence/plugins/era/postgres_era/lineage.rb', line 111

def head_view(storage_name) = "#{storage_name}_head"
# The transactionally-upserted read cache behind head_view — one row
# per LIVE id, keyed by id, carrying the ordinal it was last written
# at. Scoped by ERA, not just storage_name — an aggregate that
# ISN'T renamed across a mint keeps the SAME storage_name in both
# eras, so storage_name alone would have era N+1 sharing one
# physical table with era N: a freshly-minted era would inherit
# every pre-mint (and, worse, pre-rekey/pre-translation) row
# instead of starting empty. era-qualified naming is what
# `partition`/`matview` already do for exactly this reason.

#journal ⇒ Object 



107
 
# File 'lib/hecks/ports/persistence/plugins/era/postgres_era/lineage.rb', line 107

def journal = "hecks_journal_#{Naming.snake(@domain)}"

#matview(storage_name, era, label) ⇒ Object 



122
 
# File 'lib/hecks/ports/persistence/plugins/era/postgres_era/lineage.rb', line 122

def matview(storage_name, era, label) = "#{storage_name}_lineage_#{era}_#{label}"

#partition(era) ⇒ Object 



110
 
# File 'lib/hecks/ports/persistence/plugins/era/postgres_era/lineage.rb', line 110

def partition(era) = "#{journal}_era_#{era}"

#quoted_journal ⇒ Object 



108
 
# File 'lib/hecks/ports/persistence/plugins/era/postgres_era/lineage.rb', line 108

def quoted_journal = quote(journal)

#sequence ⇒ Object 



109
 
# File 'lib/hecks/ports/persistence/plugins/era/postgres_era/lineage.rb', line 109

def sequence = "#{journal}_ordinal"