Module: Twin::Sync

Defined in:
lib/twin/sync.rb

Constant Summary collapse

ITEMIZE_CHANGE =

A line from rsync --itemize-changes describing a real change: an itemize code whose first column is the update type (< > c h *) and second the file type (f d L D S), e.g. ">f+++++++++", "cd+++++++++", "*deleting". No-op runs emit no such line; headers ("sending …", "created directory …") and the summary don't match. Deterministic — no scraping of prose.

/\A[<>ch*][fdLDS]/
RSYNC_NOISE =

rsync's own noise: the header, the blank line, and the transfer summary. Everything twin adds itself (cmd output, "skipped:", error text) survives, as does any itemize line describing an actual change. Spaces are escaped on purpose: /x ignores literal whitespace, and these patterns are all multi-word.

/
  \A(sending|receiving)\ incremental\ file\ list\z |
  \Acreated\ directory\  |
  \Asent\ [\d,]+\ bytes |
  \Atotal\ size\ is\ [\d,]+
/x

Class Method Summary collapse

Class Method Details

.backup_args(job) ⇒ Object

Safety net for --delete: deleted and overwritten files land in a per-run backup dir on the target side (/.twin-backup/). rsync only creates the dir when it actually backs something up. The exclude keeps a backup dir inside the transfer root (Path: ".") from being deleted by the very sync it protects against.



185
186
187
188
189
 
# File 'lib/twin/sync.rb', line 185

def backup_args(job)
  root = job.remote? ? Twin::Remote.split(job.target).last : job.target
  dir  = File.join(root, ".twin-backup", run_stamp)
  ["--backup", "--backup-dir=#{dir}", "--exclude=.twin-backup/"]
end

.mounted?(path) ⇒ Boolean

True if the path lives on a mounted volume other than the root filesystem. Walks up parents until it finds a mount point (different device than parent) or hits "/" (path is on the root volume, not externally mounted).

Returns:

  • (Boolean) 


47
48
49
50
51
52
53
54
55
56
57
58
 
# File 'lib/twin/sync.rb', line 47

def mounted?(path)
  return false unless File.exist?(path)
  current = File.expand_path(path)
  until current == "/"
    parent = File.expand_path("..", current)
    return true if File.stat(current).dev != File.stat(parent).dev
    current = parent
  end
  false
rescue Errno::ENOENT
  false
end

.render_job(cfg, job, dry_run: false) ⇒ Object

Render a template Job: read source, substitute {vars}, write if changed. Returns [success, output, changed].



62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
 
# File 'lib/twin/sync.rb', line 62

def render_job(cfg, job, dry_run: false)
  return [false, "render: remote targets are not supported", false] if job.remote?

  src = job.source_path
  tgt = job.target_path

  return [false, "source not found: #{src}", false] unless File.exist?(src)
  return [false, "render: source must be a file, not a directory: #{src}", false] if File.directory?(src)

  begin
    rendered = Twin::Template.render_file(src, cfg.var_map, context: job.path)
  rescue => e
    return [false, e.message, false]
  end

  if dry_run
    return [true, "(dry-run) would render #{File.basename(src)} → #{tgt}", false]
  end

  FileUtils.mkdir_p(File.dirname(tgt))

  current = File.exist?(tgt) ? File.binread(tgt) : nil
  changed = (current != rendered)

  if changed
    File.binwrite(tgt, rendered)
    output = "rendered #{File.basename(src)} → #{tgt}"
  else
    output = "#{File.basename(src)}: content unchanged"
  end

  if !job.cmd.empty?
    if changed
      cmd_out, cmd_status = run(["sh", "-c", job.cmd])
      output += "\ncmd: #{job.cmd}\n#{cmd_out}"
      unless cmd_status.success?
        output += "cmd failed (exit #{cmd_status.exitstatus})"
        return [false, output, changed]
      end
    else
      output += "\ncmd skipped (content unchanged)"
    end
  end

  [true, output, changed]
end

.rsync_args(cfg, job, dry_run: false, force: false) ⇒ Object

Full rsync argument vector for a job.

force: drop --update, so a file that is newer on the target is overwritten anyway. Only ever set after the user agreed to it (see Twin::Conflict), or via twin sync --force.



158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
 
# File 'lib/twin/sync.rb', line 158

def rsync_args(cfg, job, dry_run: false, force: false)
  src = job.source_path
  tgt = job.target_path

  args = ["rsync", "-av", "--itemize-changes"]
  args << "--update" unless force
  if job.delete
    args << "--delete"
    args.concat(backup_args(job))
  end
  args << "--dry-run" if dry_run
  cfg.global_excludes.each { |ex| args << "--exclude=#{ex}" }
  job.all_excludes.each   { |ex| args << "--exclude=#{ex}" }

  if File.directory?(src)
    args << "#{src}/" << "#{tgt}/"
  else
    args << src << tgt
  end
  args
end

.run(args) ⇒ Object 



201
202
203
204
205
206
207
 
# File 'lib/twin/sync.rb', line 201

def run(args)
  require "open3"
  stdout, stderr, status = Open3.capture3(*args)
  [stdout + stderr, status]
rescue Errno::ENOENT
  raise "command not found: #{args.first}"
end

.run_job(cfg, job, dry_run: false, force: false) ⇒ Object

Sync one Job. Returns [success, combined_output, transferred]. transferred is true when rsync actually moved bytes (false on no-op or dry_run).



111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
 
# File 'lib/twin/sync.rb', line 111

def run_job(cfg, job, dry_run: false, force: false)
  return render_job(cfg, job, dry_run: dry_run) if job.render

  src = job.source_path
  tgt = job.target_path

  return [false, "source not found: #{src}", false] unless File.exist?(src)

  if job.remote?
    host, rpath = Twin::Remote.split(tgt)
    unless dry_run || Twin::Remote.mkdir_p(host, File.dirname(rpath))
      return [false, "ssh: could not create #{File.dirname(rpath)} on #{host}", false]
    end
  else
    FileUtils.mkdir_p(File.dirname(tgt))
  end

  output, status = run(rsync_args(cfg, job, dry_run: dry_run, force: force))
  return [false, output, false] unless status.success?

  xfr = !dry_run && transferred?(output)

  if job.conflict && !xfr && !dry_run && !force
    output += "\nskipped: target is newer, source not synced"
  end

  if !job.cmd.empty? && !dry_run
    if xfr
      cmd_out, cmd_status = run(["sh", "-c", job.cmd])
      output += "\ncmd: #{job.cmd}\n#{cmd_out}"
      unless cmd_status.success?
        output += "cmd failed (exit #{cmd_status.exitstatus})"
        return [false, output, xfr]
      end
    else
      output += "\ncmd skipped (nothing transferred)"
    end
  end

  [true, output, xfr]
end

.run_program(cfg, program, dry_run: false, force: false) ⇒ Object

Sync all jobs in a Program. Returns array of [job, success, output].



197
198
199
 
# File 'lib/twin/sync.rb', line 197

def run_program(cfg, program, dry_run: false, force: false)
  program.active_jobs.map { |job| [job, *run_job(cfg, job, dry_run: dry_run, force: force)] }
end

.run_stamp ⇒ Object

One timestamp per twin process, so a multi-job run shares a backup dir.



192
193
194
 
# File 'lib/twin/sync.rb', line 192

def run_stamp
  @run_stamp ||= Time.now.strftime("%Y-%m-%d_%H%M%S")
end

.summarize(output) ⇒ Object

Drop everything from an rsync run that does not describe a change. An itemize line whose first column is "." moved no data — permissions or a timestamp on an existing file — and is exactly the material that made the interesting lines hard to find.



37
38
39
40
41
42
 
# File 'lib/twin/sync.rb', line 37

def summarize(output)
  output.lines.reject do |line|
    l = line.rstrip
    l.empty? || RSYNC_NOISE.match?(l) || (l.start_with?(".") && l =~ /\A\.[fdLDS]/)
  end.join
end

.transferred?(output) ⇒ Boolean

True when rsync reported at least one changed item.

Returns:

  • (Boolean) 


17
18
19
 
# File 'lib/twin/sync.rb', line 17

def transferred?(output)
  output.lines.any? { |l| ITEMIZE_CHANGE.match?(l) }
end