Class: Opencdd::Parcel::ShardedDirReader

Inherits:
Object
  • Object
show all
Defined in:
lib/opencdd/parcel/sharded_dir_reader.rb

Overview

Reads the per-class sharded Parcel layout emitted by cdd.iec.ch downloads: a top-level directory containing one subdir per class code, each subdir holding 1-6 export_*_<code>.xls files (CLASS, PROPERTY, RELATION, UNIT, VALUELIST, VALUETERMS).

Two on-disk layouts are supported:

[flat (legacy)] <CODE>/export_*.xls directly under the code dir. [per-version] <CODE>/<UNID>/export_*.xls nested under the current version's UNID, with <CODE>/_entity.json identifying which <UNID> folder is the current version. Emitted by harvest/download_versions.py.

The reader is a thin orchestrator over three deep modules:

- +Opencdd::Parcel::LayoutDetector+ — single source of truth
for "is this a class-code dir? where do the XLS live?"
- +Opencdd::Parcel::EntityManifest+ — parses the
+_entity.json+ sidecar and produces VersionHistory /
stub entities.
- +Opencdd::Parcel::FlatDirReader+ — actually reads the
+.xls+ files into a Workbook.

Example:

reader = Opencdd::Parcel::ShardedDirReader.new("downloads/iec63213")
db = Opencdd::Database.new
reader.load_into(db)

Constant Summary collapse

CLASS_CODE_PATTERN =

Detection patterns owned by Opencdd::Parcel::LayoutDetector (SSOT). Aliased here for back-compat with external callers that reference these constants by class.

Opencdd::Parcel::LayoutDetector::CLASS_CODE_PATTERN
UNID_PATTERN = 
Opencdd::Parcel::LayoutDetector::UNID_PATTERN

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(path) ⇒ ShardedDirReader

Returns a new instance of ShardedDirReader.



42
43
44
 
# File 'lib/opencdd/parcel/sharded_dir_reader.rb', line 42

def initialize(path)
  @path = path
end

Instance Attribute Details

#path ⇒ Object (readonly)

Returns the value of attribute path.



40
41
42
 
# File 'lib/opencdd/parcel/sharded_dir_reader.rb', line 40

def path
  @path
end

Instance Method Details

#active_xls_dir(code_dir) ⇒ Object

Delegates to LayoutDetector — single source of truth for "where do the active XLS files live for this class-code dir?"



130
131
132
 
# File 'lib/opencdd/parcel/sharded_dir_reader.rb', line 130

def active_xls_dir(code_dir)
  Opencdd::Parcel::LayoutDetector.active_xls_dir(code_dir)
end

#attach_unloaded_entities(database) ⇒ Object

Scans _entities/ for manifests whose entity wasn't loaded by the XLS round (e.g. a property whose .xls export only had CLASS rows). Creates minimal stub entities so cross-references resolve.



82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
 
# File 'lib/opencdd/parcel/sharded_dir_reader.rb', line 82

def attach_unloaded_entities(database)
  entities_root = File.join(@path, "_entities")
  return unless File.directory?(entities_root)

  Dir.children(entities_root).sort.each do |code|
    entity_dir = File.join(entities_root, code)
    next unless File.directory?(entity_dir)

    manifest = Opencdd::Parcel::EntityManifest.read(entity_dir)
    next unless manifest

    irdi = manifest.irdi
    next unless irdi
    next if database.find(irdi)

    stub = manifest.to_stub_entity
    database.add_entity(stub) if stub
  end
end

#attach_version_histories(database) ⇒ Object

Walks each class subdir's _entity.json and attaches its versions array to the corresponding entity.



61
62
63
64
65
66
67
68
69
 
# File 'lib/opencdd/parcel/sharded_dir_reader.rb', line 61

def attach_version_histories(database)
  class_subdirs.each do |subdir|
    manifest = Opencdd::Parcel::EntityManifest.read(subdir)
    next unless manifest
    next if manifest.versions_empty?
    target = database.find_by_code(File.basename(subdir))
    target&.attach_version_history(manifest.version_history)
  end
end

#class_subdirs ⇒ Object 



117
118
119
120
121
122
123
124
125
126
 
# File 'lib/opencdd/parcel/sharded_dir_reader.rb', line 117

def class_subdirs
  return [] unless File.directory?(@path)
  dirs = list_class_subdirs(@path)

  # Also scan _entities/ for manifest-driven entity dirs.
  entities_root = File.join(@path, "_entities")
  dirs.concat(list_class_subdirs(entities_root)) if File.directory?(entities_root)

  dirs
end

#load_into(database) ⇒ Object 



46
47
48
49
50
51
52
53
54
55
56
57
 
# File 'lib/opencdd/parcel/sharded_dir_reader.rb', line 46

def load_into(database)
  class_subdirs.each do |subdir|
    active = active_xls_dir(subdir)
    next unless active
    workbook = Opencdd::Parcel::FlatDirReader.new(active).read_workbook
    database.add_workbook(workbook)
  end
  attach_unloaded_entities(database)
  attach_version_histories(database)
  database.finalize!
  database
end

#parse_version_history(code_dir) ⇒ Object

Back-compat: external callers may reach the per-class version history directly. New code should use EntityManifest.



73
74
75
76
 
# File 'lib/opencdd/parcel/sharded_dir_reader.rb', line 73

def parse_version_history(code_dir)
  manifest = Opencdd::Parcel::EntityManifest.read(code_dir)
  manifest&.version_history || Opencdd::Entity::VersionHistory.new
end

#read_workbook ⇒ Object 



102
103
104
105
106
107
108
109
110
111
112
113
114
115
 
# File 'lib/opencdd/parcel/sharded_dir_reader.rb', line 102

def read_workbook
  sub_workbooks = class_subdirs.filter_map do |subdir|
    active = active_xls_dir(subdir)
    next nil unless active
    Opencdd::Parcel::FlatDirReader.new(active).read_workbook
  end

  Opencdd::Parcel::Workbook.new(
    sheets: sub_workbooks.flat_map(&:sheets),
    sheetmap: sub_workbooks.flat_map(&:sheetmap),
    project: sub_workbooks.first&.project,
    source_path: @path.to_s,
  )
end