Class: Ask::RAG::Loader::CSV

Inherits:
Base
  • Object
show all
Defined in:
lib/ask/rag/loader/csv.rb

Overview

Loads a CSV file, producing one document per row.

Each document's content is the row's cell values joined by newlines. Metadata includes the source path and any identifying columns.

Instance Method Summary collapse

Methods inherited from Base

#lazy_load

Constructor Details

#initialize(content_columns: nil, metadata_columns: nil, separator: ",") ⇒ CSV

Returns a new instance of CSV.

Parameters:

  • path (String) —

    path to the CSV file

  • content_columns (Array<String>, nil) (defaults to: nil) —

    columns to include in content (nil = all columns)

  • metadata_columns (Array<String>, nil) (defaults to: nil) —

    columns to include in metadata (nil = all columns except content_columns)

  • separator (String) (defaults to: ",") —

    column separator (default: ",")



19
20
21
22
23
 
# File 'lib/ask/rag/loader/csv.rb', line 19

def initialize(content_columns: nil, metadata_columns: nil, separator: ",")
  @content_columns = content_columns
  @metadata_columns = metadata_columns
  @separator = separator
end

Instance Method Details

#load(path) ⇒ Object 



25
26
27
28
29
30
31
32
33
34
35
36
 
# File 'lib/ask/rag/loader/csv.rb', line 25

def load(path)
  docs = []
  options = { headers: true, col_sep: @separator }

  ::CSV.foreach(path, **options) do |row|
    content = build_content(row)
    metadata = build_metadata(row, path)
    docs << Ask::Document.new(content: content, metadata: metadata)
  end

  docs
end