Class: Metanorma::Collection::ArtifactStore

Inherits:
Object
  • Object
show all
Defined in:
lib/metanorma/collection/artifact_store.rb

Overview

Durable, content-addressed store for staged collection-build artefacts.

Enables incremental (batched) builds and resumption across runs: an artefact is named <docid-slug>.<content-hash>.<stage>.<format>, so a document already built with identical inputs is detected by filename alone and reused, and an interrupted run resumes by recomputing which artefacts are already present.

Reuse is keyed on the caller-supplied content hash of the document's input closure (source + resolved includes + templates + referenced schemas + tool versions + options) -- never on file presence alone. See metanorma/iso-10303#455 for why presence-keyed reuse of compiled XML is unsound; the store deliberately requires a content hash it does not infer.

Disabled by default: the collection renderer uses NullArtifactStore unless an incremental build is explicitly requested (opt-in, never default).

Constant Summary collapse

DEFAULT_DIRNAME = 
".metanorma-collection-cache"
STAGE_FORMATS =

The artefact-stage vocabulary, and the one non-redundant property each stage carries: the format it is serialised as (also its file extension). THIS IS ITS HOME: every producer and consumer names its stage from this table by symbol, never a bare string literal elsewhere; an unknown stage raises (KeyError via fetch), so the vocabulary cannot drift outside this file. The stage's name is the filename token, so it is not stored here -- that would just restate the key. The stages trace the pipeline (isolated compile -> index -> reinflate); each role is documented inline:

{
  # Compiled Semantic XML, unresolved cross-document repo:() references
  # PRESERVED as deterministic stubs (neither stripped nor resolved). The
  # unit of an isolated per-document compile; a pure function of the
  # document's input closure, hence content-addressable and reusable.
  semantic: "xml",
  # This document's contribution to the global anchor index: its anchors
  # and ids (type => {label/UUID => anchor-id}). Aggregated across all
  # documents to build the anchor -> owning-document index used at
  # reinflation.
  anchors: "json",
  # Reinflated Presentation XML: the preserved stubs resolved against the
  # global index into real cross-document links. Shared source for the PDF
  # and Word collection outputs, and per-document for HTML.
  presentation: "xml",
}.freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(dir) ⇒ ArtifactStore

Returns a new instance of ArtifactStore.



54
55
56
57
 
# File 'lib/metanorma/collection/artifact_store.rb', line 54

def initialize(dir)
  @dir = dir
  FileUtils.mkdir_p(@dir)
end

Instance Attribute Details

#dir ⇒ Object (readonly)

Returns the value of attribute dir.



52
53
54
 
# File 'lib/metanorma/collection/artifact_store.rb', line 52

def dir
  @dir
end

Class Method Details

.content_hash(*inputs) ⇒ Object

SHA256 over the input closure, truncated to 16 hex (64 bits -- ample for a per-collection store, and short enough to stay well inside filename length limits). Each input is hashed to a fixed-width digest and the digests concatenated before the final hash, so no two distinct closures collide on the pre-truncation digest. The caller assembles the closure; the store does not infer it.



117
118
119
120
 
# File 'lib/metanorma/collection/artifact_store.rb', line 117

def self.content_hash(*inputs)
  digests = inputs.map { |i| Digest::SHA256.hexdigest(i.to_s) }
  Digest::SHA256.hexdigest(digests.join)[0, 16]
end

Instance Method Details

#clear ⇒ Object

Wipe the entire store. The escape hatch when inputs the content hash cannot see -- imported EXPRESS schemas, templates, included files -- have changed, so the cache can no longer be trusted; the next build recompiles everything. Tracking such shared-input changes is the collection maintainer's responsibility, not the build's.



106
107
108
109
 
# File 'lib/metanorma/collection/artifact_store.rb', line 106

def clear
  FileUtils.rm_rf(@dir)
  FileUtils.mkdir_p(@dir)
end

#key?(docid, content_hash, stage) ⇒ Boolean

Is this exact (document, input-closure, stage) artefact already present? This is the resumption / skip predicate: a hit is byte-identical to a fresh build because the hash covers every input that determines output.

Returns:

  • (Boolean) 


69
70
71
 
# File 'lib/metanorma/collection/artifact_store.rb', line 69

def key?(docid, content_hash, stage)
  File.exist?(path(docid, content_hash, stage))
end

#path(docid, content_hash, stage) ⇒ Object

... stage must be a key of STAGE_FORMATS; an unknown stage raises KeyError.



61
62
63
64
 
# File 'lib/metanorma/collection/artifact_store.rb', line 61

def path(docid, content_hash, stage)
  format = STAGE_FORMATS.fetch(stage)
  File.join(@dir, "#{slug(docid)}.#{content_hash}.#{stage}.#{format}")
end

#prune_superseded(docid, keep_hash) ⇒ Object

Remove every stored version of this document whose content hash is not keep_hash -- the superseded artefacts left behind when the document's content changed. Keeps exactly one (current) version per document per stage, bounding disk to current state without losing the cache the next build resumes from.



92
93
94
95
96
97
98
99
 
# File 'lib/metanorma/collection/artifact_store.rb', line 92

def prune_superseded(docid, keep_hash)
  STAGE_FORMATS.each do |stage, ext|
    keep = path(docid, keep_hash, stage)
    Dir[File.join(@dir, "#{slug(docid)}.*.#{stage}.#{ext}")].each do |f|
      f == keep or File.delete(f)
    end
  end
end

#read(docid, content_hash, stage) ⇒ Object 



73
74
75
76
 
# File 'lib/metanorma/collection/artifact_store.rb', line 73

def read(docid, content_hash, stage)
  f = path(docid, content_hash, stage)
  File.exist?(f) ? File.read(f, encoding: "UTF-8") : nil
end

#write(docid, content_hash, stage, content) ⇒ Object

Idempotent write: the same (docid, content_hash, stage) always maps to the same path and the content is a pure function of the hash, so a re-run overwrites with identical bytes -- no side effects on repeat.



81
82
83
84
85
 
# File 'lib/metanorma/collection/artifact_store.rb', line 81

def write(docid, content_hash, stage, content)
  f = path(docid, content_hash, stage)
  File.write(f, content, encoding: "UTF-8")
  f
end