Class: OvertureMaps::Syncer

Inherits:
Object
  • Object
show all
Defined in:
lib/overture_maps/syncer.rb

Overview

Brings imported areas up to a newer Overture release without full reloads. For each tracked area (see Models::ImportedArea):

  1. When every intermediate release's changelog is known, delete the ids marked removed in each step (bbox-scoped), then re-extract and upsert the area at the target release — upserts cover added and data_changed, deletions cover removed.
  2. When the chain is incomplete (the area's release is no longer in the catalog), fall back to a full refresh: delete the area's rows and re-import.

Overlapping areas sharing a table converge once all areas are synced to the same target.

Defined Under Namespace

Classes: Result

Constant Summary collapse

DELETE_BATCH_SIZE = 
1000

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(target_release: nil) ⇒ Syncer

Returns a new instance of Syncer.



22
23
24
 
# File 'lib/overture_maps/syncer.rb', line 22

def initialize(target_release: nil)
  @target = Releases.validate!(target_release || Releases.current)
end

Instance Attribute Details

#target ⇒ Object (readonly)

Returns the value of attribute target.



26
27
28
 
# File 'lib/overture_maps/syncer.rb', line 26

def target
  @target
end

Instance Method Details

#releases_between(from, to) ⇒ Object

The releases after from up to and including to, oldest first, or nil when the chain can't be established from the known release list.



59
60
61
62
63
64
65
66
 
# File 'lib/overture_maps/syncer.rb', line 59

def releases_between(from, to)
  known = Releases.all.sort
  from_idx = known.index(from)
  to_idx = known.index(to)
  return nil if from_idx.nil? || to_idx.nil? || to_idx <= from_idx

  known[(from_idx + 1)..to_idx]
end

#sync_all ⇒ Object 



28
29
30
 
# File 'lib/overture_maps/syncer.rb', line 28

def sync_all
  Models::ImportedArea.find_each.map { |area| sync_area(area) }
end

#sync_area(area) ⇒ Object 



32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
 
# File 'lib/overture_maps/syncer.rb', line 32

def sync_area(area)
  return Result.new(area: area, status: :up_to_date, removed: 0, imported: 0, errors: 0) if area.release == target

  model = area.model_class
  bbox = area.to_bounding_box
  steps = releases_between(area.release, target)

  if steps
    removed = delete_removed(area, model, bbox, steps)
    runner = reimport(area, model, bbox)
    finalize(area, runner)
    Result.new(area: area, status: :synced, removed: removed,
               imported: runner.imported_count, errors: runner.error_count)
  else
    purged = purge_area(model, bbox)
    runner = reimport(area, model, bbox)
    finalize(area, runner)
    Result.new(area: area, status: :refreshed, removed: purged,
               imported: runner.imported_count, errors: runner.error_count,
               message: "changelog chain from #{area.release} unavailable; did a full refresh")
  end
rescue StandardError => e
  Result.new(area: area, status: :failed, removed: 0, imported: 0, errors: 1, message: e.message)
end