Class: Igniter::Store::FileBackend

Inherits:
Object
  • Object
show all
Includes:
NativeFileBackendSnapshotSupport, WireProtocol
Defined in:
lib/igniter/store/file_backend.rb,
lib/igniter/store/file_backend.rb

Overview

Pure-Ruby FileBackend — skipped when the Rust native extension is loaded.

WAL format (v2): length-prefixed frames with CRC32 integrity check.

Each frame:

[4 bytes BE uint32: body_len][body_len bytes: JSON][4 bytes BE uint32: CRC32(body)]

Snapshot format (path + “.snap”):

[header frame: JSON { type: "snapshot_header", fact_count: N, written_at: T }]
[fact frame 1] ... [fact frame N]

On open, if a snapshot file exists: snapshot facts are loaded first, WAL facts whose IDs are already in the snapshot are skipped. Combined result is sorted by timestamp. Startup cost is O(snapshot_size + delta_wal_size) instead of O(total_wal_size).

Constant Summary collapse

SNAPSHOT_SUFFIX = 
NativeFileBackendSnapshotSupport::SNAPSHOT_SUFFIX

Constants included from WireProtocol

WireProtocol::FRAME_CRC_SIZE, WireProtocol::FRAME_HEADER_SIZE

Class Method Summary collapse

Instance Method Summary collapse

Methods included from WireProtocol

#encode_frame, #read_frame

Constructor Details

#initialize(path) ⇒ FileBackend

Returns a new instance of FileBackend.



32
33
34
35
36
 
# File 'lib/igniter/store/file_backend.rb', line 32

def initialize(path)
  @path = path.to_s
  @file = File.open(@path, "ab")
  @file.sync = true
end

Class Method Details

._native_new ⇒ Object 



258
 
# File 'lib/igniter/store/file_backend.rb', line 258

alias_method :_native_new, :new

.new(path) ⇒ Object 



260
261
262
263
264
 
# File 'lib/igniter/store/file_backend.rb', line 260

def new(path)
  obj = _native_new(path)
  obj.instance_variable_set(:@_ruby_path, path.to_s)
  obj
end

Instance Method Details

#_native_replay_wal ⇒ Object

Combines snapshot (if present) + WAL delta into a chronologically ordered list of facts. Facts in the snapshot are deduplicated against the WAL by ID so a checkpoint never causes double-replay. The native extension defines ‘replay` (WAL-only) as a class-level method, which shadows the module’s snapshot-aware ‘replay`. Save the WAL-only version under an alias, then override `replay` in the class body so the snapshot-aware path is used on store open.



248
249
250
251
252
 
# File 'lib/igniter/store/file_backend.rb', line 248

def replay
  snapshot_facts, seen_ids = load_snapshot
  wal_facts = read_wal_frames.reject { |f| seen_ids.include?(f.id) }
  (snapshot_facts + wal_facts).sort_by(&:transaction_time)
end

#close ⇒ Object 



98
99
100
 
# File 'lib/igniter/store/file_backend.rb', line 98

def close
  @file.close
end

#replace_with_snapshot!(facts) ⇒ Object

Pruning-safe barrier: atomically replace the snapshot with facts AND truncate the WAL so that dropped facts cannot resurface on reopen.

Normal checkpoint (#write_snapshot) is non-destructive — it leaves the WAL intact, which means any fact not present in the snapshot will still be loaded from the WAL on next open. For physical purge that is wrong: the dropped fact ids would not be in the new snapshot, so the WAL would replay them back into existence.

This method:

1. Writes the new snapshot atomically (tmp → rename).
2. Closes the current WAL file handle.
3. Truncates the WAL to 0 bytes (new open in write mode).
4. Reopens for future appends.

After a successful call, close/reopen will load only the snapshot facts.



86
87
88
89
90
91
92
 
# File 'lib/igniter/store/file_backend.rb', line 86

def replace_with_snapshot!(facts)
  write_snapshot(facts)
  @file.close
  File.open(@path, "wb") {}   # truncate WAL
  @file = File.open(@path, "ab")
  @file.sync = true
end

#replay ⇒ Object

Combines snapshot (if present) + WAL delta into a chronologically ordered list of facts. Facts in the snapshot are deduplicated against the WAL by ID so a checkpoint never causes double-replay.



46
47
48
49
50
 
# File 'lib/igniter/store/file_backend.rb', line 46

def replay
  snapshot_facts, seen_ids = load_snapshot
  wal_facts = read_wal_frames.reject { |f| seen_ids.include?(f.id) }
  (snapshot_facts + wal_facts).sort_by(&:transaction_time)
end

#snapshot_path ⇒ Object 



94
95
96
 
# File 'lib/igniter/store/file_backend.rb', line 94

def snapshot_path
  @path + SNAPSHOT_SUFFIX
end

#write_fact(fact) ⇒ Object 



38
39
40
41
 
# File 'lib/igniter/store/file_backend.rb', line 38

def write_fact(fact)
  body  = JSON.generate(fact.to_h)
  @file.write(encode_frame(body))
end

#write_snapshot(facts) ⇒ Object

Atomically writes all facts to a snapshot file (<wal_path>.snap). Uses a tmp file + rename so a partial write never corrupts an existing snapshot. The WAL file is untouched; the snapshot is a parallel read artefact only.



56
57
58
59
60
61
62
63
64
65
66
67
68
 
# File 'lib/igniter/store/file_backend.rb', line 56

def write_snapshot(facts)
  tmp = "#{snapshot_path}.tmp"
  File.open(tmp, "wb") do |f|
    header = JSON.generate({
      type:       "snapshot_header",
      fact_count: facts.size,
      written_at: Process.clock_gettime(Process::CLOCK_REALTIME)
    })
    f.write(encode_frame(header))
    facts.each { |fact| f.write(encode_frame(JSON.generate(fact.to_h))) }
  end
  FileUtils.mv(tmp, snapshot_path)
end