Class: Spotlight::Etl::Pipeline

Inherits:
Object
  • Object
show all
Includes:
ActiveSupport::Benchmarkable
Defined in:
app/services/spotlight/etl/pipeline.rb

Overview

ETL pipeline definition

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize {|_self| ... } ⇒ Pipeline

Returns a new instance of Pipeline.

Yields:

  • (_self)

Yield Parameters:



50
51
52
 
# File 'app/services/spotlight/etl/pipeline.rb', line 50

def initialize
  yield(self) if block_given?
end

Instance Attribute Details

#context ⇒ Object (readonly)

Returns the value of attribute context.



9
10
11
 
# File 'app/services/spotlight/etl/pipeline.rb', line 9

def context
  @context
end

#source ⇒ Object (readonly)

Returns the value of attribute source.



9
10
11
 
# File 'app/services/spotlight/etl/pipeline.rb', line 9

def source
  @source
end

Instance Method Details

#call(context, data: {}, cache: nil) {|optioanlly..| ... } ⇒ Object

Execute the ETL pipeline

Parameters:

  • context (Spotlight::Etl::Context)
  • data (Hash) (defaults to: {}) —

    the initial data structure to pass through to the transform steps

Yields:

  • (optioanlly..) —

    each transformed document after it is transformed but before it is sent to the loaders



61
62
63
 
# File 'app/services/spotlight/etl/pipeline.rb', line 61

def call(context, data: {}, cache: nil, &block)
  executor(context, cache:).call(data:, &block)
end

#estimated_size(context) ⇒ Number

Estimate the number of documents that will be produced by the pipeline

Parameters:

Returns:

  • (Number) 


70
71
72
 
# File 'app/services/spotlight/etl/pipeline.rb', line 70

def estimated_size(context)
  executor(context).estimated_size
end