Module: Rogalik::Plugins::Core::LoaderMethods

Defined in:
lib/rogalik/plugins/core/loader_methods.rb

Overview

Instance methods mixed into Rogalik::Pipeline::Loader.

Provides file-based document loading that yields Document objects to the pipeline. Each loader is initialized with a source path and a logger, then drives the pipeline by calling #load.

Examples:

class MyLoader
  include Rogalik::Plugins::Core::LoaderMethods
end

MyLoader.new("path/to/file.txt", logger).load { |doc| process(doc) }

Instance Attribute Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#logger ⇒ Object (readonly)

Returns the value of attribute logger.



19
20
21
 
# File 'lib/rogalik/plugins/core/loader_methods.rb', line 19

def logger
  @logger
end

#source ⇒ Object (readonly)

Returns the value of attribute source.



19
20
21
 
# File 'lib/rogalik/plugins/core/loader_methods.rb', line 19

def source
  @source
end

Instance Method Details

#initialize(source, logger) ⇒ Object

Parameters:

  • source (String) —

    path to the file to load

  • logger (Logger) —

    logger instance for progress output



23
24
25
26
 
# File 'lib/rogalik/plugins/core/loader_methods.rb', line 23

def initialize(source, logger)
  @source = source
  @logger = logger
end

#load {|document| ... } ⇒ void

This method returns an undefined value.

Reads the source file and yields a single Document to the caller.

Yield Parameters:

  • document (Document) —

    the loaded document



32
33
34
35
36
37
38
39
40
41
42
 
# File 'lib/rogalik/plugins/core/loader_methods.rb', line 32

def load
  logger.info("Loading data...")
  File.open(source, "r") do |file|
    yield Pipeline::Document.new(
      id: nil,
      source: source,
      page_content: file.read,
      metadata: { source: source }
    )
  end
end