Class: Pubid::Nist::Parser
- Inherits:
-
Parslet::Parser
- Object
- Parslet::Parser
- Pubid::Nist::Parser
- Defined in:
- lib/pubid/nist/parser.rb
Overview
Parser class for NIST identifiers Single Responsibility: Parsing NIST identifier syntax
Class Method Summary collapse
-
.class_parse_with_preprocessing(input) ⇒ Object
Class-level parse method with preprocessing Handles data quality normalization before parsing Named explicitly to avoid conflict with Parslet’s built-in parse method.
-
.detect_format(input) ⇒ Object
Detect format from input string :mr if contains dots (machine-readable: NIST.SP.800-53) :short otherwise (default: NIST SP 800-53).
-
.roman_to_arabic(roman) ⇒ Object
Convert Roman numerals to Arabic numbers I→1, II→2, III→3, IV→4, V→5, VI→6, VII→7, VIII→8, IX→9, X→10.
Class Method Details
.class_parse_with_preprocessing(input) ⇒ Object
Class-level parse method with preprocessing Handles data quality normalization before parsing Named explicitly to avoid conflict with Parslet’s built-in parse method
13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 |
# File 'lib/pubid/nist/parser.rb', line 13 def self.class_parse_with_preprocessing(input) # Apply legacy update_codes normalization first, before any other preprocessing cleaned = Core::UpdateCodes.apply(input.to_s.strip, :nist) # Fix lowercase publisher at start cleaned = cleaned.sub(/^nbs\b/i, "NBS") cleaned = cleaned.sub(/^nist\b/i, "NIST") # Fix publisher+series concatenation: "NISTIR" → "NIST IR", "NBSIR" → "NBS IR" # Must come after lowercase publisher fix to catch "nistir" → "NISTIR" → "NIST IR" cleaned = cleaned.gsub( /^(NBS|NIST)(IR|FIPS|GCR|HB|MONO|MP|NCSTAR|NSRDS)/i, '\1 \2' ) # Fix lowercase series (ir, sp, tn, etc.) cleaned = cleaned.sub(/\b(ir|sp|tn|hb|fips|ams|vts)\b/i, &:upcase) # Normalize LC to LCIRC (single definition of truth) # Pattern: "LC" followed by space/dot/end should become "LCIRC" # But don't change if already "LCIRC" cleaned = cleaned.gsub(/\bLC\b(?!IRC)/, "LCIRC") # Combine "NBS LCIRC" with space into "NBS.LCIRC" ONLY when followed by supplement marker # This allows the circ_supplement_identifier rule to match the pattern # Only apply to supplement cases, not regular LCIRC identifiers cleaned = cleaned.gsub(/\bNBS LCIRC\b(?=.*\b(?:supp?|sup\+|r\d+\/)\d)/, "NBS.LCIRC") # Convert MR format LCIRC supplements to space-separated format # "NBS.LCIRC.145r11/1925" → "NBS LCIRC 145r11/1925" (convert series dot to space) cleaned = cleaned.gsub(/\bNBS\.LCIRC\.(\d+r\d+\/\d{4})/, "NBS LCIRC \\1") # Also handle without year: "NBS.LCIRC.145r11" → "NBS LCIRC 145r11" cleaned = cleaned.gsub(/\bNBS\.LCIRC\.(\d+r\d+)\b/, "NBS LCIRC \\1") # Fix Roman numerals: "1011-I-2" → keep as is, but fix spaces: "1011-I-2 0" → "1011-I-2.0" cleaned = cleaned.gsub(/([-\d]+[IVX]+[-\d]+)\s+(\d+)/, '\1.\2') # Fix rev without space: "126rev2013" → "126 rev2013" (separate number from rev+year) # BUT preserve edition+revision patterns: "e2rev1908" stays as-is cleaned = cleaned.gsub(/(?<!e)(\d)(rev\d{4})/, '\1 \2') # Fix LCIRC revision with slash and year: "145r6/1925" → "145 r6/1925" # BUT NOT for LCIRC series (keep "NBS LCIRC 145r11/1925" as-is for parser) # The circ_supplement_identifier rule expects "145r11" (no space) unless cleaned.include?("LCIRC") || cleaned.include?("CIRC") cleaned = cleaned.gsub(/(\d)(r\d+\/\d{4})/, '\1 \2') end # Fix LCIRC revision with just year (no slash): "1128r1995" → "1128 r1995" # BUT preserve edition+revision patterns: "13e2rev1908" stays as-is # AND preserve month abbreviations in patterns like "107-Mar1985" (ar1985 contains 'r') # Use word boundary to ensure 'r' is standalone, not part of a month name # AND preserve "rv" (revision year) patterns: "1013rv1953" stays as-is cleaned = cleaned.gsub(/\b(r(?!v)\d{4})\b/, ' \1') # Fix month in revision: "4743rJun1992" → "4743 rJun1992" (NEW) cleaned = cleaned.gsub(/(\d)(r[A-Z][a-z]{2,8}\d{4})/, '\1 \2') # REMOVED: Revision with 1-2 digits + lowercase letter preprocessing # This is now handled by the more comprehensive fix at lines 131-142 # which keeps "22r1a" together (no space) for second_number pattern matching # CRITICAL: Normalize lowercase letter suffix to uppercase # Fix dash-letter pattern: "6529-a" → "6529-A" (FIXED - was incorrect) # BUT preserve lowercase for NCSTAR series when letter is followed by volume (e.g., "1-1av1") cleaned = cleaned.gsub(/(\d)-([a-z])$/) { "#{$1}-#{$2.upcase}" } # Fix direct letter suffix (no dash): "378g" → "378G", "1000a" → "1000A" # MUST come after dash pattern to avoid conflicts # Fix letter suffix at end: "1011-A" → "1011A", "97-3b" → "97-3B" # CRITICAL: Exclude r+digit pattern (e.g., "73-197r", "6945r") from this conversion # These should remain as lowercase for edition pattern matching # Only match single letter at end, not part of words like "index", "sec", etc. cleaned = cleaned.gsub(/(\d)([a-z&&[^r]])$/) { "#{$1}#{$2.upcase}" } # Also fix r+letter patterns (e.g., "22r1a" → "22r1A") separately cleaned = cleaned.gsub(/(\d)(r)(\d+)([a-z])$/) do "#{$1}#{$2}#{$3}#{$4.upcase}" end # NEW: Fix letter suffix before r (e.g., "53ar1" → "53Ar1") # For patterns like NIST SP 800-53ar1 where letter is between number and revision cleaned = cleaned.gsub(/(\d)([a-z])(r\d)/) { "#{$1}#{$2.upcase}#{$3}" } # NOTE: Removed uppercase letter before r rule - it was breaking 800-56Ar2 parsing # The parser should handle 56Ar2 as a single unit (letter suffix + revision) # Fix letter suffix before volume: "1-2bv1" → "1-2Bv1" (MR format) # BUT preserve "rv" (revision year) patterns: "1013rv1953" stays as-is # Skip for NCSTAR to preserve lowercase letters (patterns like "1-1av1" should stay lowercase) is_ncstar = cleaned.include?("NCSTAR") unless is_ncstar cleaned = cleaned.gsub(/(\d)([a-z&&[^r]])(v\d+)/) do "#{$1}#{$2.upcase}#{$3}" end end # Fix space before volume number: "80-2073 2" → "80-2073 v2" (Session 219) # This handles NBS IR 80-2073 2 and NBS IR 80-2073 3 as volume identifiers cleaned = cleaned.gsub(/(\d{2}-\d{4})\s+(\d)$/, '\1 v\2') # Fix draft with number: "8270-draft2" → "8270 -draft 2" (Session 253) # Space BEFORE dash AND after draft to separate it from report_number cleaned = cleaned.gsub(/(\d)-draft(\d)/, '\1 -draft \2') # NEW FIX 2: Draft without dash: "8270draft2" → "8270 -draft 2" # More lenient pattern to catch missing dash before draft cleaned = cleaned.gsub(/(\d)draft(\d)/, '\1 -draft \2') # Fix supplement typo: "154suprev" → "154supprev" (Session 219) cleaned = cleaned.gsub(/(\d)suprev/, '\1supprev') # Fix letter suffix + revision before draft: "140Cr1-draft2" → "140C r1-draft2" (Session 221) # Must be BEFORE general draft preprocessing at line 47 cleaned = cleaned.gsub(/(\d{2,})([A-Z])(r\d+)([-\s]draft\d*)/, '\1\2 \3\4') # Convert Roman numeral volumes to Arabic per NIST spec (page 7) # "1011-I-2.0" → "1011 v1 ver2.0" # "1011-II-1.0" → "1011 v2 ver1.0" cleaned = cleaned.gsub(/(\d+)-([IVX]+)-(\d+(?:\.\d+)*)/) do number = $1 roman = $2 version_part = $3 # Convert Roman to Arabic arabic = roman_to_arabic(roman) # Convert to volume+version format "#{number} v#{arabic} ver#{version_part}" end # Fix LCIRC supplement with slash and year: "118supp3/1926" → "118 supp3/1926" cleaned = cleaned.gsub(/(\d)(supp\d+\/\d{4})/, '\1 \2') # Fix Pt pattern: "800-57Pt3r1" → "800-57 pt3 r1" cleaned = cleaned.gsub(/(\d)Pt(\d+)(r\d+)/, '\1 pt\2 \3') # Fix version patterns: "ver1e2006" → "ver1 e2006", "ver2v1" → "ver2 v1" cleaned = cleaned.gsub(/(\d)ver(\d)/, '\1 ver \2') cleaned = cleaned.gsub(/ver(\d+)e(\d{4})/, 'ver\1 e\2') cleaned = cleaned.gsub(/ver(\d+)v(\d+)/, 'ver\1 v\2') # Fix dotted version: separate from number "268v1.1" → "268 v1.1" cleaned = cleaned.gsub(/(\d)(v\d+\.\d+)/, '\1 \2') # CRITICAL: Now separate dotted versions from preceding digits: "268v1.1" → "268 v1.1" (NEW) cleaned = cleaned.gsub(/(\d)(v\d+\.\d+)/, '\1 \2') # NEW: Separate version from number AND convert spaces to dots in one step cleaned = cleaned.gsub(/(\d)(v\d+)\s+(\d+)$/, '\1 \2.\3') # Two-part: "268v1 1" → "268 v1.1" cleaned = cleaned.gsub(/(\d)(v\d+)\s+(\d+)\s+(\d+)$/, '\1 \2.\3.\4') # Three-part: "63v1 0 1" → "63 v1.0.1" # Fix volume ranges: "535v2a-l" → "535 v2a-l", "535v2m-z" → "535 v2m-z" cleaned = cleaned.gsub(/(\d)(v\d+[a-z]-[a-z])/, '\1 \2') # NEW: Fix volume with uppercase letter: "48v3B" → "48 v3B" (Session 220) cleaned = cleaned.gsub(/(\d)(v\d+[A-Z])/, '\1 \2') # NEW: Fix volume ranges with uppercase: "v2A-L" → "v2a-l" (normalize to lowercase) (Session 220) cleaned = cleaned.gsub(/(v\d+)([A-Z])-([A-Z])/, '\1\2-\3'.downcase) # NEW: Fix edition with "ed." suffix: "2006ed." → "e2006" (V1 compatibility) # Pattern appears at end of identifier: "NIST SP 260-162 2006ed." cleaned = cleaned.gsub(/(\d{4})ed\./, 'e\1') # CRITICAL: Fix revision attached to number BEFORE update patterns! # "8115r1-upd" → "8115 r1-upd" so that later "r1-upd" → "r1 -upd" works # But preserve r6/1925 format (don't add space before slash/year) # And preserve 300-8r1/upd format (don't separate r1/upd) # ENHANCED: Also handle r1a (revision with letter suffix) - "800-22r1a" → "800-22r1A" # FIXED: When there's a letter suffix, keep together for second_number pattern # CRITICAL: Use \d{1,2} instead of \d+ to limit revision to 1-2 digits, allowing [a-z] to match # First rule: Match r+digit+letter (keep together) cleaned = cleaned.gsub(/(\d+)(r\d{1,2})([a-z])(?=-|[A-Z]|$)/) do num = $1 rev = $2 letter = $3 # Keep together when there's a letter suffix "#{num}#{rev}#{letter.upcase}" end # Second rule: Match r+digit WITHOUT letter suffix # CRITICAL: Use negative lookahead (?![a-zA-Z]) to avoid matching when there's a letter # PRESERVE compact format (no space) when at end of string (NIST SP 800-53r4) # ADD space only when followed by: dash+uppercase, uppercase letter, or /upd, /errata, /insert cleaned = cleaned.gsub(/(\d+)(r\d{1,2})(?![a-zA-Z])(?=[A-Z]|-(?=[A-Z])|\/(?:upd|errata|insert))/) do num = $1 rev = $2 # Add space when followed by dash+uppercase, uppercase, or update keyword "#{num} #{rev}" end # Fix spaces in version/volume numbers: "v1 1" → "v1.1", "1011-I-2 0" → "1011-I-2.0" # ENHANCED to handle multiple spaces: "v1 0 1" → "v1.0.1", "v1 0 2" → "v1.0.2" # FIXED: Pattern must start with "v" or digit to avoid matching "rev 2013" as "v" + " 2013" # CRITICAL: Added word boundary \b to prevent matching "v" within "rev" # CRITICAL FIX: Use \b to ensure match starts at word boundary cleaned = cleaned.gsub(/(\b(?:v|\d)[v\d]*[-A-Z]*)\s+(\d+)\s+(\d+)/, '\1.\2.\3') # Three parts # CRITICAL FIX: Use \b to ensure match starts at word boundary cleaned = cleaned.gsub(/(\b(?:v|\d)[v\d]*)\s+(\d+)/, '\1.\2') # Two parts # Fix update patterns: ensure space before -upd or /upd (not just at end) # Enhanced to handle optional digits after upd: -upd, -upd1, /upd, /upd1 cleaned = cleaned.gsub(/(\d+)-upd(\d*)/, '\1 -upd\2') # -upd or -upd1 cleaned = cleaned.gsub(/(\d+)\/upd(\d*)/, '\1 /upd\2') # /upd or /upd1 cleaned = cleaned.gsub(/([a-z]\d+)-upd/, '\1 -upd') # r1-upd → r1 -upd cleaned = cleaned.gsub(/([a-z]\d+)\/upd/, '\1 /upd') # After revision: r1/upd → r1 /upd # NEW FIX 3: MR format with letter suffix before update: "8286C-upd1" → "8286C -upd1" # Must handle uppercase letters before -upd in MR format cleaned = cleaned.gsub(/(\d+[A-Z])-upd(\d*)/, '\1 -upd\2') # Letter suffix + update cleaned = cleaned.gsub(/(\d+[A-Z])\/upd(\d*)/, '\1 /upd\2') # Letter suffix + /upd variant # Fix supplement patterns: ensure space before supplement (1st variant) # "118supp3" already handled at line 32-33, but add "sup" variant cleaned = cleaned.gsub(/(\d)(sup\d)/, '\1 \2') # 100-2sup1 → 100-2 sup1 # Fix supplement patterns: ensure space before supplement (2nd variant) cleaned = cleaned.gsub(/(\d)(sup+)(\d)/, '\1 \2\3') # 100-2sup+1 → 100-2 sup+1 # Fix supplement patterns: ensure space before supplement (3rd variant) cleaned = cleaned.gsub(/(\d)(sup\+)(\d)/, '\1 \2\3') # 100-2sup+1 → 100-2 sup+1 # Fix supplement patterns: ensure space before supplement (4th variant) cleaned = cleaned.gsub(/(\d)(sup\d+)/, '\1 \2') # 100-2sup1 → 100-2 sup1 # Fix supplement patterns: ensure space before supplement (5th variant) cleaned = cleaned.gsub(/(\d)(sup\d+\b)/, '\1 \2') # 100-2sup1 → 100-2 sup1 # Fix letter suffix + supplement: "378Gsup" → "378Gsupp" (NEW for LCIRC patterns) # Normalize "sup" to "supp" for letter suffix patterns to match circ_supplement_identifier rule cleaned = cleaned.gsub(/(\d+[A-Z])sup(\b)/, '\1supp\2') # 378Gsup → 378Gsupp # Fix LCIRC supplement without letter suffix: "118sup12/1926" → "118supp12/1926" # Normalize "sup" to "supp" for LCIRC patterns to match circ_supplement_identifier rule cleaned = cleaned.gsub(/(\d+)sup(\d+\/\d{4})/, '\1supp\2') # 118sup12/1926 → 118supp12/1926 # REMOVED: Revision letter patterns that add space before revision with letter # These conflicted with the fix at lines 131-142 which keeps "22r1a" together # for second_number pattern matching. The comprehensive fix now handles: # - "800-22r1a" → "800-22r1A" (kept together, uppercase letter) # - "800-22r1" → "800-22 r1" (space added when no letter suffix) # Fix number with letter suffix followed by standalone 'r': "56ar" → "56a r" (NEW) cleaned = cleaned.gsub(/(\d[a-z])r\b/, '\1 r') # Fix revision followed by language code: "r1es" → "r1 es", "r1pt" → "r1 pt" (NEW) cleaned = cleaned.gsub(/(r\d+)(es|pt|chi|viet|port|esp)\b/, '\1 \2') # Fix MR format translation codes: ".spa" → " spa", ".por" → " por", ".ind" → " ind" (NEW) # Prevents 3-letter translation codes from being parsed as letter suffixes # "NIST.SP.1262.spa" → "NIST.SP.1262 spa" (convert dot to space) cleaned = cleaned.gsub(/^([A-Z]+)\.SP\.(\d+)\.([a-z]{2,4})$/, '\1.SP.\2 \3') cleaned = cleaned.gsub(/^([A-Z]+)\.([A-Z]+)\.(\d+)\.([a-z]{2,4})$/, '\1.\2.\3 \4') # ENHANCEMENT 1: Edition year normalization (-YYYY → eYYYY) # Per NIST spec, trailing -YYYY should normalize to eYYYY format # Pattern: number (optionally with non-e letter suffix) followed by dash and 4-digit year # Examples: "330-2019" → "330e2019", "304a-2017" → "304Ae2017" # Must NOT match existing edition patterns like "11e2-1915" (e2 is edition, -1915 is separate) # Must be at end or before space to avoid breaking number-number patterns like "800-53" # Negative lookbehind (?<![eE-]) prevents matching after e/E or dash (avoids e2-1915 and 105-1-1990) # EXCLUSION: Do NOT convert -YYYY for HB series (handbooks) - preserve original format # Example: "NBS HB 130-1979" should stay as "NBS HB 130-1979" (not convert to e1979) # EXCLUSION: Do NOT convert -YYYY when preceded by "e\d+" (edition+year pattern like "44e2-1955") # EXCLUSION: Only convert years in NBS (1901-1988) or NIST (1988-2099) range # Numbers outside this range are part numbers, not edition years (e.g., SP 250-1039) # Use a more specific pattern: only convert when NOT preceded by "e" + digits (edition) # AND only convert when year is in valid range (1901-2099) cleaned = cleaned.gsub(/(?<!e\d)(?<![eE-])(\d(?:[A-DF-Z]?))-(\d{4})(?=\s|$)/) do |match| prefix = $1 # Number with optional letter year = $2.to_i # Only convert to edition format if year is in valid range if year.between?(1901, 2099) "#{prefix}e#{year}" else match # Keep dash format for part numbers (e.g., 250-1039) end end # Revert the conversion for HB series to preserve -YYYY format # Matches both "HB 130e1979" and "HB 105-1e1990" patterns # Use [^:\s.]*? (exclude dots) to avoid consuming MR format dot separators # This prevents "NIST.HB.135e2022" from being incorrectly reverted cleaned = cleaned.gsub(/\b(HB|HB\s+)[^:\s.]*?(\d+)e(\d{4})(?=\s|$)/, '\1\2-\3') # Revert the conversion for OWMWP series to preserve date format MM-DD-YYYY # OWMWP uses date as the number: "06-13-2018" (not an edition) # Pattern: "OWMWP 06-13e2018" → "OWMWP 06-13-2018" cleaned = cleaned.gsub( /\b(OWMWP|OWMWP\s*)[^:\s]*?(\d{2})-(\d{2})e(\d{4})(?=\s|$)/, '\1\2-\3-\4' ) # Revert the conversion for RPT series to preserve year range format YYYY-YYYY # Report series uses year ranges as the number: "1946-1947" (not an edition) # Pattern: "RPT 1946e1947" → "RPT 1946-1947" # Note: This must check that first year < second year (forward range) cleaned = cleaned.gsub(/\b(RPT|RPT\s*)([^:\s]*?)(\d{4})e(\d{4})(?=\s|$)/) do |match| prefix = $1 # "RPT" or "RPT " separator = $2 # "." or "" or other non-colon, non-space chars first_year = $3.to_i second_year = $4.to_i # Only revert if first < second (year range like 1946-1947) if first_year < second_year "#{prefix}#{separator}#{first_year}-#{second_year}" else match # Keep e format for editions like e2018e2019 end end # ENHANCEMENT 2: Version normalization (v1.1 → ver1.1, Ver. 2.0 → ver2.0) # Normalize short v format to verbose ver format per NIST spec # Already handled in version rule, but normalize in preprocessing for consistency # CRITICAL: MR format version normalization must come BEFORE general v normalization # Pattern: "NIST.SP.500-281-v1.0" → "NIST.SP.500-281.ver1.0" # This allows report_number to match "500-281" and version rule to match ".ver1.0" cleaned = cleaned.gsub(/-v(\d+\.\d+)/, '.ver\1') # Handle Ver. with period: "Ver. 2.0" → "ver2.0" (remove period and space) cleaned = cleaned.gsub(/\bVer\.\s+(\d+(?:\.\d+)*)/, 'ver\1') # Handle verbose "v" to "ver": "v1.1" → "ver1.1" (only with dots - versions have dots) cleaned = cleaned.gsub(/\bv(\d+\.\d+(?:\.\d+)*)/, 'ver\1') # Fix uppercase P for part: "428P1" → "428 p1", "647P2" → "647 p2" (NEW) cleaned = cleaned.gsub(/(\d)P(\d)/, '\1 p\2') # Normalize part notation: "p1" → "pt1", "n1" → "pt1" for consistency # This handles patterns like "61p1" → "61pt1" and "467n1" → "467pt1" # MUST come AFTER uppercase P normalization # EXCLUDE pattern: {number}p{digit}{4-digit-year} like "28p11969" (part + year, not part notation) # Use negative lookahead to avoid matching when p/n + digit is followed by exactly 4 digits (year) cleaned = cleaned.gsub(/\b([pn])(\d+)(?!\d{4}\b)/, 'pt\2') # Fix complex part patterns in MR format: ensure space before part cleaned = cleaned.gsub(/(\d)([pP]\d+)/, '\1 \2') # .467p1adde1 → .467 p1adde1, 800-57p1 → 800-57 p1 # Fix CRPL-F series: ensure space after series (e.g., "CRPL-F-B150" → "CRPL-F-B 150") cleaned = cleaned.gsub(/(NBS CRPL-F-[AB])(\d)/, '\1 \2') cleaned = cleaned.gsub(/(CRPL-F-[AB])(\d)/, '\1 \2') # Extract volume from number: "17-917v3" → "17-917 v3", "1-1v1" → "1-1 v1" # Pattern: digits-digits followed by v and digits (GCR, NCSTAR patterns) # MUST be specific to avoid breaking existing "v1.1" patterns cleaned = cleaned.gsub(/(\d+-\d+)(v\d+)(?![.\d])/, '\1 \2') # Negative lookahead for dots # pd_suffix rule handles " 2pd" directly (space >> digits >> str("pd")) # No preprocessing needed - adding space before "pd" breaks the parser # Fix "Suppl" with space: "955 Suppl" → "955Suppl" cleaned = cleaned.gsub(/(\d+)\s+Suppl\b/, '\1Suppl') # Fix verbose "Version" format: " Version 2" → " ver 2" cleaned = cleaned.gsub(/\s+Version\s+(\d+)/, ' ver \1') # Fix verbose "Revision" format: " Revision (r)" → " r" cleaned = cleaned.gsub(/\s+Revision\s+\(r\)/, " r") # Fix verbose "rev YYYY" format: "126 rev 2013" → "126r2013" # Removes space between number and "rev", and converts to "r" prefix # Handles patterns like "NIST SP 260-126 rev 2013" → "NIST SP 260-126r2013" cleaned = cleaned.gsub(/(\d+)\s+rev\s+(\d{4})/, '\1r\2') # Fix historical "report ;" format: "NBS report ; 8079" → "NBS RPT 8079" # The semicolon and "report" (spelled out) are historical formats cleaned = cleaned.gsub(/\breport\s*;\s*/, "RPT ") cleaned = cleaned.gsub(/\breport\b/, "RPT") # REMOVED: Incorrect dot preprocessing that treated dots as number separators # This was semantically wrong - dots are PART separators in NIST! # DELETE: cleaned = cleaned.gsub(/(\d{3,})\.(\d{1,4})(?=\s|$)/, '\1_\2') # REMOVED: Incorrect space-to-underscore that treated as single number # DELETE: cleaned = cleaned.gsub(/(\d{3,})\s+(\d{1,2})$/, '\1_\2') # Detect format before parsing format = detect_format(input.to_s) # Use parslet parser instance result = new.parse(cleaned) # Add format to result if result.is_a?(Hash) result.merge(parsed_format: format) elsif result.is_a?(Array) # For array results, merge all hashes into one # This handles cases where identifier rule returns multiple components (e.g., compound_series + edition) merged = result.inject({}) do |acc, hash| next acc unless hash.is_a?(Hash) acc.merge(hash) end merged.merge(parsed_format: format) else result end end |
.detect_format(input) ⇒ Object
Detect format from input string :mr if contains dots (machine-readable: NIST.SP.800-53) :short otherwise (default: NIST SP 800-53)
407 408 409 410 411 412 413 414 415 416 417 418 419 |
# File 'lib/pubid/nist/parser.rb', line 407 def self.detect_format(input) # Check if it has dot separators (MR format pattern) # Patterns include: # - "NIST.SP.800-53" (publisher.series.number) # - "FIPS.46e1977" (series.numberWithEdition) # - "NBS.HB.28pt1e1969" (publisher.series.part.edition) # Key indicator: dots between components instead of spaces if input.include?(".") && !input.match?(/\s/) :mr else :short end end |
.roman_to_arabic(roman) ⇒ Object
Convert Roman numerals to Arabic numbers I→1, II→2, III→3, IV→4, V→5, VI→6, VII→7, VIII→8, IX→9, X→10
423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 |
# File 'lib/pubid/nist/parser.rb', line 423 def self.roman_to_arabic(roman) case roman when "I" then "1" when "II" then "2" when "III" then "3" when "IV" then "4" when "V" then "5" when "VI" then "6" when "VII" then "7" when "VIII" then "8" when "IX" then "9" when "X" then "10" else roman # Fallback for unexpected patterns end end |