Class: Spree::Import

Inherits:
Object
  • Object
show all
Includes:
NumberIdentifier
Defined in:
app/models/spree/import.rb

Constant Summary collapse

SAMPLE_DATA_BASE_URL_TEMPLATE =

Where the downloadable example CSVs are served from — see sample_csv_url.

Pinned to the installed version's tag rather than main: whether a type has an example is answered by the local db/sample_data, so pointing elsewhere would let a released install link to a newer, incompatible schema (or a file that has since been removed).

'https://raw.githubusercontent.com/spree/spree/refs/tags/v%<version>s/spree/core/db/sample_data'.freeze

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.available_models ⇒ Array<Class>

Returns the available models for the import

Returns:

  • (Array<Class>) 


401
402
403
 
# File 'app/models/spree/import.rb', line 401

def available_models
  available_types.map(&:model_class)
end

.available_types ⇒ Array<Class>

Returns the available types for the import

Returns:

  • (Array<Class>) 


395
396
397
 
# File 'app/models/spree/import.rb', line 395

def available_types
  Spree.import_types
end

.model_class ⇒ Object

eg. Spree::Imports::Orders => Spree::Order

Raises:

  • (NameError) 


456
457
458
459
460
461
462
463
464
 
# File 'app/models/spree/import.rb', line 456

def model_class
  return Spree.user_class if to_s == 'Spree::Imports::Customers'

  klass = "Spree::#{to_s.demodulize.singularize}".safe_constantize

  raise NameError, "Missing model class for #{self}" unless klass

  klass
end

.required_scope ⇒ Symbol?

Admin API scope family gating this import type — an import is a bulk write, so an API key needs write_<required_scope> to create and manage it. Derived from the class name (Spree::Imports::Customers => :customers); override in subclasses gated by a different scope. Returns nil on the base class, so unmapped types are only accessible to write_all keys.

Returns:

  • (Symbol, nil) 


419
420
421
422
423
 
# File 'app/models/spree/import.rb', line 419

def required_scope
  return nil if self == Spree::Import

  to_s.demodulize.underscore.to_sym
end

.sample_csv_filename ⇒ String

eg. Spree::Imports::ProductTranslations => "product_translations.csv"

Returns:

  • (String) 


451
452
453
 
# File 'app/models/spree/import.rb', line 451

def sample_csv_filename
  "#{api_type}.csv"
end

.sample_csv_url ⇒ String?

Publicly downloadable example CSV for this import type — the same file rake spree:load_sample_data feeds through this very pipeline, so it is a working example rather than a hand-maintained sample that can drift from the schema.

Whether a type has an example is answered by db/sample_data itself, so adding or removing a CSV there needs no change here; a type with no sample file returns nil and the UI omits the link. The URL is pinned to the installed version's tag so the file served always matches the schema that check was made against.

Returns:

  • (String, nil) 


437
438
439
440
441
442
 
# File 'app/models/spree/import.rb', line 437

def sample_csv_url
  return nil if self == Spree::Import
  return nil unless Spree::Core::Engine.root.join('db', 'sample_data', sample_csv_filename).exist?

  [sample_data_base_url, sample_csv_filename].join('/')
end

.sample_data_base_url ⇒ String

Returns:

  • (String) 


445
446
447
 
# File 'app/models/spree/import.rb', line 445

def sample_data_base_url
  format(SAMPLE_DATA_BASE_URL_TEMPLATE, version: Spree.version)
end

.type_for_model(model) ⇒ Class

Returns the type for the model

Returns:

  • (Class) 


407
408
409
 
# File 'app/models/spree/import.rb', line 407

def type_for_model(model)
  available_types.find { |type| type.model_class.to_s == model.to_s }
end

Instance Method Details

#attachment_file_content ⇒ String

Returns the content of the attachment file

Returns:

  • (String) 


331
332
333
 
# File 'app/models/spree/import.rb', line 331

def attachment_file_content
  @attachment_file_content ||= attachment.attached? ? attachment.blob.download&.force_encoding('UTF-8') : nil
end

#can_be_deleted? ⇒ Boolean

Imports are deletable only while no processing jobs can be in flight — CreateRowsJob/ProcessGroupJob re-load the record mid-run.

Returns:

  • (Boolean) 


188
189
190
 
# File 'app/models/spree/import.rb', line 188

def can_be_deleted?
  %w[pending mapping completed failed].include?(status)
end

#complete? ⇒ Boolean

Returns true if the import is complete

Returns:

  • (Boolean) 


181
182
183
 
# File 'app/models/spree/import.rb', line 181

def complete?
  status == 'completed'
end

#create_mappings ⇒ Object

Creates mappings from the schema fields TODO: get mappings from the previous import if it exists, so user won't have to map the same columns again



337
338
339
340
341
342
343
 
# File 'app/models/spree/import.rb', line 337

def create_mappings
  schema_fields.each do |schema_field|
    mapping = mappings.find_or_create_by!(schema_field: schema_field[:name])
    mapping.try_to_auto_assign_file_column(csv_headers)
    mapping.save!
  end
end

#create_rows_async ⇒ void

This method returns an undefined value.

Creates rows asynchronously



347
348
349
350
351
352
353
 
# File 'app/models/spree/import.rb', line 347

def create_rows_async
  # The 2s delay lets the attachment settle before the job reads it; running
  # inline there is nothing to wait for.
  return Spree::Imports::CreateRowsJob.perform_now(id) if preferred_inline

  Spree::Imports::CreateRowsJob.set(wait: 2.seconds).perform_later(id)
end

#csv_headers ⇒ Array<String>

Returns the headers of the csv file

Returns:

  • (Array<String>) 


320
321
322
323
324
325
326
327
 
# File 'app/models/spree/import.rb', line 320

def csv_headers
  return [] if attachment_file_content.blank?

  @csv_headers ||= ::CSV.parse_line(
    attachment_file_content,
    col_sep: preferred_delimiter
  )
end

#current_ability ⇒ Spree::Ability

Returns the current ability for the import

Returns:



375
376
377
 
# File 'app/models/spree/import.rb', line 375

def current_ability
  @current_ability ||= Spree.ability_class.new(user, { store: store })
end

#display_name ⇒ String

Returns the display name for the import

Returns:

  • (String) 


306
307
308
 
# File 'app/models/spree/import.rb', line 306

def display_name
  "#{Spree.t(type.demodulize.pluralize.downcase)} #{number}"
end

#event_serializer_class ⇒ Object 



388
389
390
 
# File 'app/models/spree/import.rb', line 388

def event_serializer_class
  'Spree::Api::V3::ImportSerializer'.safe_constantize
end

#group_column ⇒ String?

Returns the schema field name used to group rows for parallel processing. Rows sharing the same value in this field are processed together in one job. Returns nil for imports where rows are independent (default — batched in chunks).

Returns:

  • (String, nil) 


163
164
165
 
# File 'app/models/spree/import.rb', line 163

def group_column
  nil
end

#import_schema ⇒ Spree::ImportSchema

Returns the import schema for the import

Returns:



250
251
252
 
# File 'app/models/spree/import.rb', line 250

def import_schema
  "Spree::ImportSchemas::#{type.demodulize}".safe_constantize.new
end

#large_import? ⇒ Boolean

Returns true if the import has more rows than the large import threshold. Large imports skip per-row UI broadcasts and use bulk processing.

Returns:

  • (Boolean) 


155
156
157
 
# File 'app/models/spree/import.rb', line 155

def large_import?
  rows_count >= Spree::Config[:large_import_threshold]
end

#mapped_fields ⇒ Array<String>

Returns the mapped fields for the import schema

Returns:

  • (Array<String>) 


294
295
296
 
# File 'app/models/spree/import.rb', line 294

def mapped_fields
  @mapped_fields ||= mappings.mapped.where(schema_field: required_fields)
end

#mapping? ⇒ Boolean

Returns true if the import is in mapping state

Returns:

  • (Boolean) 


169
170
171
 
# File 'app/models/spree/import.rb', line 169

def mapping?
  status == 'mapping'
end

#mapping_done? ⇒ Boolean

Returns true if the mapping is done

Returns:

  • (Boolean) 


300
301
302
 
# File 'app/models/spree/import.rb', line 300

def mapping_done?
  mapped_fields.count == required_fields.count
end

#model_class ⇒ Class

Returns the model class for the import

Returns:

  • (Class) 


240
241
242
243
244
245
246
 
# File 'app/models/spree/import.rb', line 240

def model_class
  if type == 'Spree::Imports::Customers'
    Spree.user_class
  else
    "Spree::#{type.demodulize.singularize}".safe_constantize
  end
end

#process_rows_async ⇒ void

This method returns an undefined value.

Processes rows asynchronously



357
358
359
360
361
 
# File 'app/models/spree/import.rb', line 357

def process_rows_async
  return Spree::Imports::ProcessRowsJob.perform_now(id) if preferred_inline

  Spree::Imports::ProcessRowsJob.perform_later(id)
end

#processing? ⇒ Boolean

Returns true if the import is processing or completed mapping

Returns:

  • (Boolean) 


175
176
177
 
# File 'app/models/spree/import.rb', line 175

def processing?
  ['processing', 'completed_mapping'].include?(status)
end

#publish_import_completed_event ⇒ Object 



314
315
316
 
# File 'app/models/spree/import.rb', line 314

def publish_import_completed_event
  publish_event('import.completed')
end

#required_fields ⇒ Array<String>

Returns the required fields for the import schema

Returns:

  • (Array<String>) 


288
289
290
 
# File 'app/models/spree/import.rb', line 288

def required_fields
  import_schema.required_fields
end

#results_page_url ⇒ String?

URL of the wizard/results view for this import — the caller-provided results_url with the wizard's ?import= param appended (a no-op for the legacy admin's show URL, whose path already identifies the import). Nil when the preference is absent; the mailer then renders no link.

Returns:

  • (String, nil) 


214
215
216
217
218
219
220
221
222
223
224
 
# File 'app/models/spree/import.rb', line 214

def results_page_url
  return if preferred_results_url.blank?

  uri = URI.parse(preferred_results_url)
  params = URI.decode_www_form(uri.query.to_s)
  params << ['import', prefixed_id]
  uri.query = URI.encode_www_form(params)
  uri.to_s
rescue URI::InvalidURIError
  nil
end

#results_url ⇒ Object

Public API name for the results_url preference (read/write symmetry). String preferences round-trip nil as "" — normalize blank to nil.



201
202
203
 
# File 'app/models/spree/import.rb', line 201

def results_url
  preferred_results_url.presence
end

#results_url=(value) ⇒ Object 



205
206
207
 
# File 'app/models/spree/import.rb', line 205

def results_url=(value)
  self.preferred_results_url = value
end

#row_lookup_cache ⇒ Hash

Per-instance cache shared by row processors within a single processing job. Group jobs funnel every row through the same Import instance, so lookups of shared records (tax/shipping categories, option types, metafield definitions) resolve once per job instead of once per row.

Returns:

  • (Hash) 


384
385
386
 
# File 'app/models/spree/import.rb', line 384

def row_lookup_cache
  @row_lookup_cache ||= {}
end

#row_processor_class ⇒ Class

Returns the row processor class for the import

Returns:

  • (Class) 


256
257
258
 
# File 'app/models/spree/import.rb', line 256

def row_processor_class
  "Spree::ImportRowProcessors::#{type.demodulize.singularize}".safe_constantize
end

#rows_status_counts ⇒ Hash{String => Integer}

Per-status row counts for API status payloads, memoized so one request issues a single grouped COUNT instead of one per status.

Returns:

  • (Hash{String => Integer}) 


195
196
197
 
# File 'app/models/spree/import.rb', line 195

def rows_status_counts
  @rows_status_counts ||= rows.group(:status).count
end

#sample_csv_url ⇒ String?

Publicly downloadable example CSV for this import's type, or nil when the type has no sample file. Resolved from the type column rather than the instance's class, so it works on a record built as the base Spree::Import with type assigned (which is how the admin's new-import form builds it).

Returns:

  • (String, nil) 


132
133
134
 
# File 'app/models/spree/import.rb', line 132

def sample_csv_url
  self.class.available_types.find { |klass| klass.to_s == type.to_s }&.sample_csv_url
end

#sample_row ⇒ Hash

First data row as { header => value }, reading a single line of the attached CSV — used by the mapping UI to show example values.

Returns:

  • (Hash) 


229
230
231
232
233
234
235
236
 
# File 'app/models/spree/import.rb', line 229

def sample_row
  return {} if attachment_file_content.blank?

  row = ::CSV.foreach(StringIO.new(attachment_file_content), headers: true, col_sep: preferred_delimiter).first
  row&.to_h || {}
rescue ::CSV::MalformedCSVError, EncodingError
  {}
end

#schema_fields ⇒ Array<Hash>

Returns the fields for the import schema If model supports metafields, it will include the metafield definitions for this model

Returns:

  • (Array<Hash>) 


263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
 
# File 'app/models/spree/import.rb', line 263

def schema_fields
  base_fields = import_schema.fields

  # Dynamically add metafield definitions if the model supports metafields
  if model_class_supports_metafields?
    metafield_fields = metafield_definitions_for_model.map do |definition|
      {
        name: definition.csv_header_name,
        label: definition.name
      }
    end
    base_fields + metafield_fields
  else
    base_fields
  end
end

#store ⇒ Spree::Store

Returns the store for the import

Returns:



365
366
367
368
369
370
371
 
# File 'app/models/spree/import.rb', line 365

def store
  if owner.is_a?(Spree::Store)
    owner
  else
    owner.respond_to?(:store) ? owner.store : Spree::Store.default
  end
end

#template_csv ⇒ String

Header-only CSV for this import's schema — the blank counterpart to sample_csv_url. One line, so callers can inline it (e.g. as a data: URI) instead of round-tripping to the server for it.

Returns:

  • (String) 


141
142
143
 
# File 'app/models/spree/import.rb', line 141

def template_csv
  ::CSV.generate_line(schema_fields.map { |field| field[:name] })
end

#template_csv_filename ⇒ String

Suggested filename for template_csv, e.g. "products_import_template.csv".

Returns:

  • (String) 


148
149
150
 
# File 'app/models/spree/import.rb', line 148

def template_csv_filename
  "#{self.class.api_type_for(type)}_import_template.csv"
end

#touch_store ⇒ Object 



310
311
312
 
# File 'app/models/spree/import.rb', line 310

def touch_store
  store.touch
end

#unmapped_file_columns ⇒ Array<String>

Returns the file columns that are not mapped

Returns:

  • (Array<String>) 


282
283
284
 
# File 'app/models/spree/import.rb', line 282

def unmapped_file_columns
  csv_headers.reject { |header| mappings.mapped.exists?(file_column: header) }
end