Module: CAFrame::CSVParser

Defined in:
lib/carray/frame/csv_parser.rb

Overview

CSV tokenizer behind CAFrame.from_csv. It produces raw String cells and does no type inference — casting is a separate step.

Records with no quote character take a String#split fast path; only quote-bearing records fall back to the field scanner, which handles embedded separators, embedded newlines and doubled-quote escapes. Spacing follows RFC 4180 (significant and preserved) unless strip: is given. An empty unquoted field is nil (missing); an empty quoted field is the empty String.

Defined Under Namespace

Classes: MalformedCSV, Tokenizer

Class Method Summary collapse

Class Method Details

.parse(io, sep: ",", quote: '"', strip: false) ⇒ Object

Parse an IO (or anything answering +gets+). The first record supplies the headers; the rest are data rows. Fully blank lines are skipped.



51
52
53
54
55
56
57
58
59
60
61
62
63
 
# File 'lib/carray/frame/csv_parser.rb', line 51

def parse(io, sep: ",", quote: '"', strip: false)
  tok = Tokenizer.new(sep, quote, strip)
  headers = nil
  rows    = []
  while (fields = tok.read(io))
    if headers.nil?
      headers = fields.map(&:to_s)
    else
      rows << fields
    end
  end
  [headers, rows]
end

.parse_file(path, encoding: "bom|utf-8", **opts) ⇒ Object

Parse a file into [headers, rows]. +encoding+ is an IO open-mode encoding string; the default strips a leading BOM and reads UTF-8.



45
46
47
 
# File 'lib/carray/frame/csv_parser.rb', line 45

def parse_file(path, encoding: "bom|utf-8", **opts)
  File.open(path, "r:#{encoding}") { |io| parse(io, **opts) }
end

.read_record(io, quote) ⇒ Object

Read one logical record, joining continuation lines while a quoted field is still open (an odd number of quote characters means unbalanced).



67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/carray/frame/csv_parser.rb', line 67

def read_record(io, quote)
  line = io.gets
  return nil if line.nil?
  rec = line.dup
  while rec.count(quote).odd?
    more = io.gets
    raise MalformedCSV, "unterminated quoted field at end of input" if more.nil?
    rec << more
  end
  rec.chomp!
  rec
end