Class: ColdStorage::Archiver

Inherits:
Object
  • Object
show all
Includes:
Logging, RowReader
Defined in:
lib/cold_storage/archiver.rb

Overview

Moves rows of one model from the primary database to the archive database.

The two databases cannot share a transaction, so every batch is:

1. read from the primary database,
2. upserted into the archive database (idempotent, keyed on the primary
 key, so a retry can never duplicate a row),
3. deleted from the primary database.

A crash between 2 and 3 leaves the row in both databases; the next run upserts it again and deletes it. Archiving is therefore at-least-once, and never loses a row.

Defined Under Namespace

Classes: Result

Constant Summary

Constants included from Logging

Logging::PREFIX

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(model, relation: nil, force: false, limit: nil, dry_run: ColdStorage.config.dry_run, batch_size: nil, track: true, cascade: true, delete_after_archive: nil) ⇒ Archiver

Returns a new instance of Archiver.

Parameters:

  • model (Class) —

    the ActiveRecord model to archive

  • relation (ActiveRecord::Relation, nil) (defaults to: nil) —

    rows to archive, defaults to what the model's policy selects

  • force (Boolean) (defaults to: false) —

    ignore the every: window

  • limit (Integer, nil) (defaults to: nil) —

    stop after this many rows

  • dry_run (Boolean) (defaults to: ColdStorage.config.dry_run) —

    report what would move without moving it

  • track (Boolean) (defaults to: true) —

    write a cold_storage_runs row

  • cascade (Boolean, Symbol, Array, Hash) (defaults to: true) —

    true to follow the policy's cascade, false for none, or an explicit association tree

  • delete_after_archive (Boolean, nil) (defaults to: nil) —

    override the policy's setting

Raises:



44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
 
# File 'lib/cold_storage/archiver.rb', line 44

def initialize(model, relation: nil, force: false, limit: nil,
               dry_run: ColdStorage.config.dry_run, batch_size: nil, track: true,
               cascade: true, delete_after_archive: nil)
  @model      = model
  @policy     = model.respond_to?(:archiving_policy) ? model.archiving_policy : nil
  @relation   = relation
  @force      = force
  @limit      = limit
  @dry_run    = dry_run
  @batch_size = batch_size
  @track      = track && relation.nil?
  @cascade    = cascade
  @delete_after_archive = delete_after_archive

  return if @policy || @relation

  raise NotArchivableError,
        "#{model} is not archivable. Add `archivable ...` to the model, or pass an explicit relation."
end

Instance Attribute Details

#model ⇒ Object (readonly)

Returns the value of attribute model.



32
33
34
 
# File 'lib/cold_storage/archiver.rb', line 32

def model
  @model
end

#policy ⇒ Object (readonly)

Returns the value of attribute policy.



32
33
34
 
# File 'lib/cold_storage/archiver.rb', line 32

def policy
  @policy
end

#relation ⇒ Object (readonly)

Returns the value of attribute relation.



32
33
34
 
# File 'lib/cold_storage/archiver.rb', line 32

def relation
  @relation
end

#result ⇒ Object (readonly)

Returns the value of attribute result.



32
33
34
 
# File 'lib/cold_storage/archiver.rb', line 32

def result
  @result
end

Instance Method Details

#call ⇒ Result

Returns:



65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
 
# File 'lib/cold_storage/archiver.rb', line 65

def call
  return skipped(:disabled) unless ColdStorage.config.enabled
  return skipped(:destroy_only) if sweep_less?
  return skipped(:not_due) unless due?

  policy&.validate_against_schema!(model)
  ensure_archive_table!

  run = start_run
  archived = 0
  deleted = 0

  each_batch do |rows, ids|
    archived += copy(rows)
    deleted  += cascade_and_delete(ids)
    break if limit_reached?(archived)

    throttle
  end

  run&.succeed!(archived_count: archived, deleted_count: deleted)
  finish(archived, deleted)
rescue StandardError => e
  run&.fail!(e, archived_count: archived.to_i, deleted_count: deleted.to_i)
  raise
end