Class: Stupidedi::Schema::Generation::FlatFileReader

Inherits:
Object
  • Object
show all
Defined in:
lib/stupidedi/schema/generation/flat_file_reader.rb

Overview

Reads ASC X12 Table Data flat files (the official .TXT distribution, CSV despite the extension) and builds the in-memory Models tree the generators consume. This is Layer A - the inverse of the Tediware x12:import importer, with no database.

release = FlatFileReader.read("vendor/x12/table_data/005010", "005010")

Files consumed: ELEHEAD/ELEDETL (simple elements), COMHEAD/COMDETL (composites), SEGHEAD/SEGDETL (segments + element uses), SETHEAD/SETDETL (transaction sets + structure), FREEFORM (code lists + syntax notes).

The distribution's encoding varies by release - see SOURCE_ENCODINGS. It is not ISO-8859-1 for any release we support, despite that being the obvious guess: releases through 007010 are Windows-1252 and 008010 is UTF-8.

Constant Summary collapse

SOURCE_ENCODINGS =

Declared source encoding per ASC X12 release. CP1252 and ISO-8859-1 agree everywhere except 0x80-0x9F, which carries smart punctuation in the former and undefined C1 controls in the latter - so reading a CP1252 distribution as Latin-1 turns a curly apostrophe into U+0092 while leaving accented letters intact, which is why the damage hides. 003060's and 004010's .TXT files are pure ASCII, so their entries are a formality - declared anyway, because an undeclared release falls through to the UTF-8 default and that is a decision, not an oversight.

{
  "003060" => "Windows-1252",
  "004010" => "Windows-1252",
  "004060" => "Windows-1252",
  "005010" => "Windows-1252",
  "006010" => "Windows-1252",
  "007010" => "Windows-1252",
  "008010" => "UTF-8"
}.freeze
DEFAULT_SOURCE_ENCODING =

An undeclared release decodes as UTF-8, deliberately. Single-byte encodings decode every possible byte, so guessing one can only fail silently; UTF-8 raises on the first byte that is not valid UTF-8, so a new distribution nobody declared stops generation instead of writing mojibake into the grammar.

"UTF-8"
PUNCTUATION =

CP1252 smart punctuation, normalized to ASCII after decoding. This is not cosmetic: consumers derive identifiers and translated values from these strings with ASCII-only munging (delete("'") and friends), so a U+2019 survives the munging and stays damaged downstream. 008010's own source already uses straight apostrophes, so this converges the older releases onto what the newest one says. Punctuation only - accented letters are left alone ("Fiancee", "Denominacion" and "Marzen" keep their real characters).

{
  "‘" => "'",  # left single quotation mark
  "’" => "'",  # right single quotation mark
  "“" => '"',  # left double quotation mark
  "”" => '"',  # right double quotation mark
  "–" => "-",  # en dash
  "—" => "-"   # em dash
}.freeze
PUNCTUATION_PATTERN = 
Regexp.union(PUNCTUATION.keys).freeze
C1_CONTROLS =

C1 controls are never legitimate in this data. Finding one after a successful decode means the declared encoding is wrong (Latin-1 read of CP1252 smart punctuation lands squarely here), so reject rather than carry it into the generated grammar.

(0x80..0x9F).map { |cp| cp.chr(Encoding::UTF_8) }.join.freeze
BOM =

U+FEFF, spelled numerically because it is invisible in source.

0xFEFF.chr(Encoding::UTF_8).freeze

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(dir, release_code) ⇒ FlatFileReader

Returns a new instance of FlatFileReader.



81
82
83
84
85
86
87
 
# File 'lib/stupidedi/schema/generation/flat_file_reader.rb', line 81

def initialize(dir, release_code)
  @dir = dir
  @release_code = release_code
  @elements_by_code = {}
  @segments_by_code = {}
  @transaction_sets_by_code = {}
end

Class Method Details

.read(dir, release_code) ⇒ Object 



77
78
79
 
# File 'lib/stupidedi/schema/generation/flat_file_reader.rb', line 77

def self.read(dir, release_code)
  new(dir, release_code).read
end

Instance Method Details

#read ⇒ Object 



89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
 
# File 'lib/stupidedi/schema/generation/flat_file_reader.rb', line 89

def read
  read_elements
  read_composites
  read_segments
  read_element_uses
  read_component_uses
  read_transaction_sets
  read_freeform

  release = Models::Release.new(
    code: @release_code,
    elements: @elements_by_code.values,
    segments: @segments_by_code.values,
    transaction_sets: @transaction_sets_by_code.values
  )
  release.transaction_sets.each { |ts| ts.release = release }
  release
end