Class: Mammoth::Sources::Postgres

Inherits:
Object
  • Object
show all
Defined in:
lib/mammoth/sources/postgres.rb

Overview

Concrete PostgreSQL CDC source for Mammoth.

Postgres realizes the CDC Ecosystem libraries for Mammoth's product boundary. It composes pgoutput-client, pgoutput-parser, pgoutput-decoder, and pgoutput-source-adapter into a single source that yields CDC::Core work to the delivery runtime.

This class may mention pgoutput implementation details because it is the concrete PostgreSQL source adapter used by Mammoth. The rest of Mammoth should remain source-agnostic and consume only the work yielded here. rubocop:disable Metrics/ClassLength

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(config, runner: nil, parser: nil, decoder: nil, adapter: nil, checkpoint_store: nil, publication_inspector: nil) ⇒ Postgres

Build a PostgreSQL CDC source.

Parameters:

  • config (Mammoth::Configuration) —

    loaded configuration

  • runner (#start, nil) (defaults to: nil) —

    injected pgoutput-client runner

  • parser (Object, nil) (defaults to: nil) —

    injected pgoutput parser or relation tracker

  • decoder (Object, nil) (defaults to: nil) —

    injected pgoutput decoder

  • adapter (Object, nil) (defaults to: nil) —

    injected source adapter

  • checkpoint_store (Mammoth::CheckpointStore, nil) (defaults to: nil) —

    persisted checkpoints for restart resume

  • publication_inspector (#inspect, nil) (defaults to: nil) —

    injected publication metadata inspector



42
43
44
45
46
47
48
49
50
51
 
# File 'lib/mammoth/sources/postgres.rb', line 42

def initialize(config, runner: nil, parser: nil, decoder: nil, adapter: nil, checkpoint_store: nil,
               publication_inspector: nil)
  @config = config
  @runner = runner
  @parser = parser
  @decoder = decoder
  @adapter = adapter
  @checkpoint_store = checkpoint_store
  @publication_inspector = publication_inspector
end

Instance Attribute Details

#adapter ⇒ Object? (readonly)

Returns injected CDC source adapter.

Returns:

  • (Object, nil) —

    injected CDC source adapter



27
28
29
 
# File 'lib/mammoth/sources/postgres.rb', line 27

def adapter
  @adapter
end

#checkpoint_store ⇒ Mammoth::CheckpointStore? (readonly)

Returns checkpoint store used for restart resume.

Returns:



29
30
31
 
# File 'lib/mammoth/sources/postgres.rb', line 29

def checkpoint_store
  @checkpoint_store
end

#config ⇒ Mammoth::Configuration (readonly)

Returns loaded Mammoth configuration.

Returns:



19
20
21
 
# File 'lib/mammoth/sources/postgres.rb', line 19

def config
  @config
end

#decoder ⇒ Object? (readonly)

Returns injected pgoutput decoder.

Returns:

  • (Object, nil) —

    injected pgoutput decoder



25
26
27
 
# File 'lib/mammoth/sources/postgres.rb', line 25

def decoder
  @decoder
end

#parser ⇒ Object? (readonly)

Returns injected pgoutput protocol parser.

Returns:

  • (Object, nil) —

    injected pgoutput protocol parser



23
24
25
 
# File 'lib/mammoth/sources/postgres.rb', line 23

def parser
  @parser
end

#publication_inspector ⇒ #inspect? (readonly)

Returns injected PostgreSQL publication inspector.

Returns:

  • (#inspect, nil) —

    injected PostgreSQL publication inspector



31
32
33
 
# File 'lib/mammoth/sources/postgres.rb', line 31

def publication_inspector
  @publication_inspector
end

#runner ⇒ #start? (readonly)

Returns injected pgoutput-client runner.

Returns:

  • (#start, nil) —

    injected pgoutput-client runner



21
22
23
 
# File 'lib/mammoth/sources/postgres.rb', line 21

def runner
  @runner
end

Instance Method Details

#acknowledge(lsn) ⇒ Integer

Acknowledge a durably handled PostgreSQL WAL position.

Parameters:

  • lsn (String, Integer) —

    pgoutput-client compatible WAL position

Returns:

  • (Integer) —

    normalized acknowledged WAL position



88
89
90
91
92
 
# File 'lib/mammoth/sources/postgres.rb', line 88

def acknowledge(lsn)
  effective_runner.ack(lsn)
rescue StandardError => e
  raise ReplicationError, "PostgreSQL WAL acknowledgement failed: #{e.message}"
end

#each {|work| ... } ⇒ Enumerator?

Stream CDC::Core work from PostgreSQL logical replication.

Calling this method starts the injected or configured pgoutput-client runner. The runner owns the PostgreSQL replication connection and slot lifecycle, while pgoutput-source-adapter owns transaction buffering and normalization into CDC::Core work items. This class only composes those layers and forwards transport source positions to the adapter.

Yield Parameters:

  • work (CDC::Core::ChangeEvent, CDC::Core::TransactionEnvelope)

Returns:

  • (Enumerator, nil)

Raises:



64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
 
# File 'lib/mammoth/sources/postgres.rb', line 64

def each(&block)
  return enum_for(:each) unless block_given?

  preflight_slot!
  preflight_replica_identity!
  normalizer = effective_adapter
  unless normalizer.respond_to?(:each_normalized)
    raise ReplicationError, "pgoutput source adapter must respond to #each_normalized"
  end

  normalizer.each_normalized(decoded_stream) do |work|
    yield_with_progress_position(validate_core_work!(work), &block)
  end
  nil
rescue StandardError => e
  raise e if e.is_a?(ReplicationError)

  raise ReplicationError, "PostgreSQL CDC source failed: #{e.message}"
end

#progress_position_for(work) ⇒ String, ...

Resolve the acknowledgement-compatible transport position for work currently being yielded by this source.

Parameters:

  • work (CDC::Core::ChangeEvent, CDC::Core::TransactionEnvelope) —

    yielded work

Returns:

  • (String, Integer, nil) —

    pgoutput-client compatible WAL position



99
100
101
102
103
 
# File 'lib/mammoth/sources/postgres.rb', line 99

def progress_position_for(work)
  return nil unless yielded_work_includes?(work)

  @yielded_progress_position
end

#slot_health ⇒ PostgresSlotHealth

Inspect the configured slot for readiness and operator metrics.

pgoutput-client owns catalog access. This method converts its snapshot into Mammoth's PostgreSQL-specific health policy without leaking transport-library types into the observability layer.

Returns:



112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
 
# File 'lib/mammoth/sources/postgres.rb', line 112

def slot_health
  status = inspected_slot_status
  return PostgresSlotHealth.missing(required_config("replication", "slot")) unless status

  PostgresSlotHealth.new(
    slot_name: value_from(status, :slot_name),
    present: true,
    active: value_from(status, :active) == true,
    retained_wal_bytes: value_from(status, :retained_wal_bytes),
    wal_status: value_from(status, :wal_status),
    safe_wal_size: value_from(status, :safe_wal_size),
    inactive_since: value_from(status, :inactive_since),
    invalidation_reason: value_from(status, :invalidation_reason),
    restart_lsn: value_from(status, :restart_lsn),
    restart_lsn_bytes: metric_lsn(value_from(status, :restart_lsn)),
    confirmed_flush_lsn: value_from(status, :confirmed_flush_lsn),
    confirmed_flush_lsn_bytes: metric_lsn(value_from(status, :confirmed_flush_lsn)),
    conflicting: value_from(status, :conflicting) == true
  )
rescue StandardError => e
  raise e if e.is_a?(ReplicationError)

  raise ReplicationError, "PostgreSQL slot health inspection failed: #{e.message}"
end