Yobi

Yobi is a Ruby interface for the Restic backup program.

Installation

Install the gem

bundle add yobi

or, without Bundler:

gem install yobi

Install Restic

Yobi shells out to a real restic binary; it doesn't install or bundle one. You need restic on PATH (or point Yobi at it explicitly, see below) before any of this works.

See Restic's installation guide for how to install it on your platform.

Confirm it's reachable:

restic version

Minimum supported version

Yobi requires Restic 0.17.1 or newer. This isn't arbitrary: Yobi::Restic's exit-code-based error dispatch (mapping Restic's own exit codes to Yobi::RepositoryNotFound/RepositoryLocked/AuthenticationFailed) depends on a convention that's only guaranteed since that version. An older Restic could reuse those same exit codes for different failures, so rather than risk silently misclassifying an error, Yobi checks the installed version before running any real command and raises Yobi::UnsupportedResticVersion if it's too old:

begin
  repo.snapshots
rescue Yobi::UnsupportedResticVersion => e
  puts "Restic #{e.installed_version} is too old, need >= #{e.minimum_version}"
end

See "Minimum-version enforcement, in detail" below for the mechanics. If you'd rather fail fast at application startup instead of on the first backup attempt, call it explicitly:

RESTIC = Yobi::Restic.new
RESTIC.ensure_minimum_version! # raises immediately if too old, otherwise a no-op

A non-default Restic binary

If restic isn't on PATH, or you want to pin a specific binary:

# via $RESTIC_PATH
ENV["RESTIC_PATH"] = "/opt/restic/bin/restic"
Yobi::Restic.new

# or explicitly
Yobi::Restic.new("/opt/restic/bin/restic")
Yobi::Repository.new(url: "...", password: "...", restic: "/opt/restic/bin/restic")

Core concepts

Two classes make up the whole API:

• Yobi::Restic: a specific restic binary plus its process-level settings that apply no matter which repository you're talking to (cache directory, bandwidth limits, compression, and so on).
• Yobi::Repository: one repository, covering its URL, password, and backend credentials, plus every Restic subcommand as a method. Holds a Restic internally, either one you hand it explicitly via restic: (useful when several repositories should share the same operational settings) or a default one it creates for you.

require "yobi"

repo = Yobi::Repository.new(
  url: "sftp:backup@nas.local:/repos/home",
  password: [:command, "security find-generic-password -a restic -s home -w"]
)

Quick start

A repository has to exist before anything else works: #init creates one at url. From there, #backup creates a snapshot, #snapshots lists what exists, and #restore pulls one back out:

repo.init

outcome = repo.backup(
  source: "/Users/zia/Documents",
  excludes: ["*.tmp", "node_modules"],
  tags: ["documents", "daily"]
)
outcome.errors.none? # => true
outcome.summary["snapshot_id"]
outcome.summary["data_added"]

repo.snapshots.each do |snapshot|
  puts "#{snapshot.short_id}  #{snapshot.time}  #{snapshot.paths.join(", ")}"
end

repo.restore(snapshot_id: "latest", target: "/tmp/restore")

Most verb methods return something typed rather than a raw Hash. #backup returns a BackupOutcome (#summary, #errors), #snapshots returns an Enumerable of Snapshot objects, and so on throughout the API - the low-level cat_* introspection commands are the exception, returning Restic's own raw JSON directly (see "Low-level (cat) and object listing" below).

Constructing a Repository

repo = Yobi::Repository.new(
  url: "sftp:backup@nas.local:/repos/home",
  password: [:command, "security find-generic-password -a restic -s home -w"]
)

url:

The repository location, in whatever form Restic itself accepts:

• a plain path for a local repository
• or a URL prefixed with backend type such as sftp:, s3:, azure:, b2:, gs:, rest:, rclone:, etc.

See Restic's documentation for the full list of supported backends and their URL-ish schemes.

password:

Identifies the repository's encryption password. Required - pass one of five shapes:

# A literal password
password = "correct horse battery staple"

# Restic resolves this itself, Yobi never sees the value
password = [:command, "security find-generic-password -a restic -w"]

# Same idea, from a file
password = [:file, "/etc/restic/password"]

# Resolved on the Ruby side, called fresh immediately before every Restic
# invocation, use this for a secret store (see Recipes below)
password = -> { Vault.logical.read("secret/restic").data[:password] }

# An explicit "there is no password" declaration (Restic's own
# --insecure-no-password flag)
password = :insecure_no_password

Yobi::Repository.new(url: "...", password: password)

backend_credentials:

Separate from password:: this is whatever the storage backend needs to authenticate, unrelated to whether the repository's contents can be decrypted. A Hash of env vars, or a callable returning one, resolved fresh on every call the same way a callable password: is:

# S3
Yobi::Repository.new(
  url: "s3:s3.amazonaws.com/my-bucket",
  password: "...",
  backend_credentials: {"AWS_ACCESS_KEY_ID" => "...", "AWS_SECRET_ACCESS_KEY" => "..."}
)

# Azure
Yobi::Repository.new(
  url: "azure:my-container:/",
  password: "...",
  backend_credentials: {"AZURE_ACCOUNT_NAME" => "...", "AZURE_ACCOUNT_KEY" => "..."}
)

# Google Cloud Storage
Yobi::Repository.new(
  url: "gs:my-bucket:/",
  password: "...",
  backend_credentials: {"GOOGLE_PROJECT_ID" => "...", "GOOGLE_APPLICATION_CREDENTIALS" => "/path/to/service-account.json"}
)

# REST server, credentials embedded directly in the URL instead of backend_credentials:
Yobi::Repository.new(url: "rest:https://user:pass@host/", password: "...")

For that last case, Yobi extracts the embedded username/password into RESTIC_REST_USERNAME/RESTIC_REST_PASSWORD and strips them from the stored url so it won't leak via repo#inspect, logs, or a stray puts. Explicit backend_credentials: values win over extracted ones if both are given.

See Restic's documentation for the full list of supported backends and their env vars.

restic:

A Yobi::Restic instance to share, a bare Restic binary path, or just leave it out to get a default one:

Yobi::Repository.new(url: "...", password: "...") # default Restic

Yobi::Repository.new(url: "...", password: "...", restic: "/opt/restic/bin/restic") # bare path

Yobi::Repository.new(url: "...", password: "...", restic: restic) # shared Yobi::Restic instance

See "Sharing a Restic across repositories" below for why you'd want to share one explicitly (global settings like bandwidth limits or cache directory).

Error handling

Every failure raises a specific, typed error rather than a generic one, so you can handle the cases that matter to you and let the rest propagate:

begin
  repo.snapshots
rescue Yobi::RepositoryLocked
  # another Restic process holds the lock, see Recipes below for a retry pattern
rescue Yobi::AuthenticationFailed
  # wrong password
rescue Yobi::RepositoryNotFound
  # url doesn't point to an initialized repository yet, maybe call #init first
end

The full hierarchy, all under Yobi::Error < StandardError:

• Yobi::ResticNotFound: the restic binary itself couldn't be found or executed.
• Yobi::UnsupportedResticVersion: the installed Restic is older than the minimum supported version.
• Yobi::RepositoryNotFound, Yobi::RepositoryLocked, Yobi::AuthenticationFailed: Restic's own typed exit codes (10/11/12).
• Yobi::ResticCommandFailed: any other non-zero exit, for a failure that doesn't fit one of the above.
• Yobi::MountTimeout: #mount didn't report itself ready within its timeout.

Every method below lives on Yobi::Repository unless noted otherwise.

Repository lifecycle

#init

Creates the repository at url.

repo.init
# => #<Yobi::Initialized id="..." repository="...">

copy_chunker_params: copies chunker parameters from another repository (from_repo:/from_password:/etc.), so a later #copy between the two can deduplicate - see #init_mirror under "Across repositories" for a shortcut that sets this up in one call. See the YARD docs for the full list of options. Returns an Initialized (#id/#repository).

#cat_config

The repository's own config document (cat config), as a Hash. A cheap way to confirm a repository exists and the password is correct without doing anything else. Aliased #config.

repo.cat_config
# => {"version"=>2, "id"=>"...", "chunker_polynomial"=>"..."}

Raises Yobi::RepositoryNotFound if url doesn't point to an initialized repository, Yobi::AuthenticationFailed if the password is wrong.

Backup and restore

#backup

outcome = repo.backup(
  source: "/Users/zia/Documents",
  excludes: ["*.tmp", "node_modules"],
  tags: ["documents", "daily"]
)
outcome.errors.none?      # => true
outcome.summary["snapshot_id"]
outcome.summary["files_new"]
outcome.summary["data_added"]

source: is either a path String, or a [:stdin_from_command, command]/[:stdin_from_command, command, filename] tuple: Restic spawns and executes command itself, capturing its stdout as the backup content (see "Database dumps via stdin_from_command" below for a pg_dump example). command can be a String (tokenized with Shellwords.split, so quoted arguments survive) or an Array of already-discrete arguments (used as-is; needed when an argument itself contains a literal space, which Shellwords would otherwise split incorrectly).

host: records a hostname on the new snapshot - a single value, since this is metadata being written, not a filter, unlike every hosts: elsewhere in this API. See the YARD docs for the full list of options (filtering, retention, timing, and more).

Returns a BackupOutcome: #summary (aliased #report; a BackupSummary, Restic's own summary fields plus #backup_start/#backup_end parsed into Time), #errors (lazy Enumerable of BackupError - empty unless some files were skipped, e.g. permission errors; a full failure raises before an outcome exists at all, see "Error handling" below), #command_output (the source: [:stdin_from_command, ...] subprocess's own stderr, de-prefixed, if any).

Also accepts a block, to receive typed progress objects as the backup runs, on top of (not instead of) the final BackupOutcome once it returns. The block also receives a Yobi::BackupSummary, the same object outcome.summary returns, once the run finishes:

outcome = repo.backup(source: "/Users/zia/Documents") do |message|
  case message
  when Yobi::BackupStatus
    puts "#{((message.percent_done || 0) * 100).round}% (#{message.files_done}/#{message.total_files})"
  when Yobi::BackupError
    warn "#{message.item}: #{message.message}"
  when Yobi::BackupVerboseStatus
    puts "#{message.action} #{message.item}"
  when Yobi::BackupSummary
    puts "done: #{message["data_added"]} bytes added"
  end
end
outcome.summary["snapshot_id"]

Yobi::BackupVerboseStatus (one message per file) only streams when verbose: true is passed; it's not emitted by default, since a large backup can otherwise produce hundreds of MB of these. Behavior without a block is identical, just without the live callbacks; the returned BackupOutcome is the same either way.

#restore

repo.restore(snapshot_id: "latest", target: "/tmp/restore")

snapshot_id: accepts "latest" or a real ID; target: is the destination directory. delete: removes files in target: not present in the snapshot - worth knowing about since it's the one option here that can destroy data outside the snapshot itself. See the YARD docs for the full list of options (filtering, overwrite: behavior, and more).

Returns a RestoreOutcome: #summary (aliased #report; Restic's own summary fields as a plain Hash) - restore has no exit code 3, so any item-level failure is a hard error that raises before an outcome exists.

Accepts a block the same way #backup does, yielding Yobi::RestoreStatus/Yobi::RestoreVerboseStatus (the latter only when verbose: true) instead of Yobi::BackupStatus/Yobi::BackupVerboseStatus.

#dump

Extracts a single file (raw bytes) or a whole folder (as a tar/zip archive) from a snapshot, without a full restore:

# Straight to a file
repo.dump(snapshot_id: "latest", file: "/etc/hosts", target: "/tmp/hosts")

# Or stream it yourself
repo.dump(snapshot_id: "latest", file: "/etc/hosts") do |io|
  IO.copy_stream(io, "/tmp/hosts")
end

# A whole folder as an archive (default to tar):
repo.dump(snapshot_id: "latest", file: "/var/www", target: "/tmp/www.tar")
repo.dump(snapshot_id: "latest", file: "/var/www", target: "/tmp/www.zip", archive: "zip")

Give at most one of target: or a block. Without either, returns a Yobi::IOHandle instead. hosts:/paths:/tags: filters are also available when snapshot_id: is "latest" - see the YARD docs for the full list of options.

handle = repo.dump(snapshot_id: "latest", file: "/var/www")
begin
  # ... hand handle.io to something else that reads from it later ...
ensure
  # put this in ensure block so the restic subprocess doesn't become a zombie:
  handle.close # may raise error based on restic's exit code.
end

#diff

outcome = repo.diff(from: older_snapshot.id, to: newer_snapshot.id)
outcome.statistics.changed_files
outcome.changes.each { |change| puts "#{change.modifier} #{change.path}" }

metadata: true also reports metadata-only changes (permissions, timestamps) alongside content changes. change.modifier is Restic's own concatenation of single-character codes: + added, - removed, U metadata updated, M content modified, T type changed, ? bitrot detected. See the YARD docs for the full list of options.

Returns a DiffOutcome: #statistics (aliased #report; a DiffStatistics with #changed_files plus #added/#removed DiffStat breakdowns) and lazy #changes (Enumerable of DiffChange, with predicates like #modified?/#added? alongside the raw #modifier).

Snapshot management

#snapshots

repo.snapshots(tags: ["daily"], hosts: "web-1").each do |snapshot|
  puts "#{snapshot.short_id}  #{snapshot.time}  #{snapshot.paths.join(", ")}"
end

Returns an Enumerable of Snapshot (#id, #short_id, #time, #host, #tags, #paths, #parent_id, #summary - a SnapshotSummary with the same stats fields #backup's own summary has). group_by:/latest: group and limit results, e.g. latest: 1 per group for "the most recent backup of each host." See the YARD docs for the full list of options.

#tag

repo.tag(snapshot_ids: snapshot.id, add: "verified")

add:/remove:/set: (mutually exclusive with add:/remove: in Restic itself) modify tags on snapshots matched by snapshot_ids: (or by hosts:/paths:/tags: filters when no explicit IDs are given). Since tags are part of a snapshot's content-addressed identity, tagging produces a new snapshot ID for every snapshot touched. See the YARD docs for the full list of options.

Returns a TagOutcome: #summary (aliased #report; a TagSummary - just #changed_snapshots) and lazy #changes (Enumerable of TagChange, the old-ID/new-ID pairs).

#forget

repo.forget(keep_daily: 7, keep_weekly: 4, keep_monthly: 12, prune: true)

Applies a retention policy, removing snapshots that don't match any keep_* rule (counts like keep_daily:, or durations like keep_within:). prune: true also reclaims the disk space the forgotten snapshots held (equivalent to a separate #prune call afterward). See the YARD docs for the full list of options.

Returns an Enumerable of ForgetGroup (Restic evaluates the policy per group, by default grouped by host+paths). Each exposes #keep/#remove (Arrays of Snapshot) and #reasons (an Array of KeepReason explaining which rule kept each surviving snapshot, e.g. "daily snapshot").

#find

repo.find(patterns: "*.pem").each do |matches|
  puts "in #{matches.snapshot_id}: #{matches.hits} hits"
  matches.matches.each { |m| puts "  #{m.path}" }
end

Searches for files/directories by name pattern across snapshots. blob:/pack:/tree: switch to matching object IDs instead, for low-level troubleshooting. See the YARD docs for the full list of options.

Returns an Enumerable of MatchesPerSnapshot (Restic's own docs describe find's output as "organized by snapshot," not a flat list). Each has #snapshot/#snapshot_id (just the matched snapshot's ID as a String - unlike elsewhere in this API, Restic's own find output doesn't include the full record), #hits, and #matches (an Array of FindMatch, with the usual file metadata: #path, #size, #mtime, etc.).

#ls

outcome = repo.ls(snapshot_id: "latest", dirs: "/var/www")
outcome.snapshot.short_id
outcome.nodes.each { |node| puts node.path }

Lists a snapshot's files/directories. recursive: descends into subdirectories. See the YARD docs for the full list of options.

Returns an LsOutcome: #snapshot (the resolved Snapshot, useful to see what "latest" actually resolved to) and lazy #nodes (aliased #entries; Enumerable of Node - Restic's own term for a file/directory entry).

Maintenance and health

#check

outcome = repo.check(read_data_subset: "5%")
outcome.summary.num_errors
outcome.errors.each { |error| puts error.message }

Verifies repository integrity. read_data: also reads and verifies every pack file's actual contents, not just structure (slow, thorough); read_data_subset: does a partial version of the same (e.g. "5%", or "1/20" for a fifth each day in rotation). See the YARD docs for the full list of options.

Returns a CheckOutcome: #summary (aliased #report; a CheckSummary covering the error count and what to run next - repair or prune) and #errors (lazy Enumerable of CheckError) for problems found.

#prune

repo.prune(max_unused: "5%")

Removes data no longer referenced by any snapshot. dry_run: reports what would happen without doing it; max_unused: targets a maximum acceptable unused-space ratio after pruning. See the YARD docs for the full list of options. Returns true.

#repair_index, #repair_packs, #repair_snapshots

repo.repair_index
repo.repair_packs(ids: damaged_pack_ids)
repo.repair_snapshots(snapshot_ids: affected_ids, forget: true)

#repair_index rebuilds the index from the pack files present (the modern successor to Restic's now-deprecated rebuild-index). #repair_packs extracts intact blobs from damaged pack files and drops the rest. Restic also writes a backup copy of each given pack (named pack-<id>) into the calling process's own current working directory first, with no flag to disable this; be aware of where your process runs from before calling it. #repair_snapshots regenerates snapshots with damaged content removed. This is real data loss for whatever gets removed, so prefer a fresh #backup where the source data is still available, and run #repair_index first since this depends on a correct index. See the YARD docs for the full list of options.

#recover

repo.recover

Builds a new snapshot from any data present in the repository but not referenced by an existing snapshot (e.g. after an accidental #forget). Call #snapshots afterward to see whether anything was actually recovered. Restic reports "nothing to do" as plain text with no structured way to tell the two cases apart.

#rewrite

repo.rewrite(snapshot_ids: old.id, excludes: "*.log", new_time: Time.now.iso8601)

Creates new snapshots from existing ones with filters applied or metadata changed. With no snapshot_ids:/hosts:/tags:/paths: given, rewrites every snapshot in the repository - Restic's own default, not something Yobi adds. dry_run: reports what would happen without doing it. forget: false (the default) tags the new snapshots "rewrite" and keeps the originals; forget: true removes the originals instead (their data isn't reclaimed until a later #prune). See the YARD docs for the full list of options.

#migrate

repo.migrate(names: "upgrade_repo_v2")

Applies the given named migrations; with names: left empty, only lists what's available (as Restic's own plain-text output; there's no programmatic way to enumerate migrations, since --json is ignored for this command).

#unlock

repo.unlock

Removes stale locks left by other Restic processes. remove_all: true removes every lock, not just stale ones.

#stats

repo.stats(mode: "raw-data")
# => #<Yobi::RepositoryStats total_size=... total_file_count=... snapshots_count=...>

Repository size/file-count statistics. mode: picks the counting strategy, as a String or Symbol: "restore-size"/:restore_size (default), "files-by-contents"/:files_by_contents, "blobs-per-file"/:blobs_per_file, "raw-data"/:raw_data (actual on-disk size, accounting for deduplication). Anything else raises ArgumentError. See the YARD docs for more. Returns a RepositoryStats covering size, file/blob counts, and (for a repository using compression) how much space it's saving.

Key management

repo.key_add(user: "ci-runner", new_password: [:file, "/etc/restic/ci-runner-password"])
repo.key_list.each { |key| puts "#{key.id} #{key.user_name}@#{key.host_name} current=#{key.current?}" }
repo.key_passwd(new_password: [:file, "/etc/restic/new-password"])
repo.key_remove(id: old_key.id)

#key_add/#key_list/#key_passwd/#key_remove (aliased #add_key/#keys/#change_password/#remove_key) manage passwords. Every key here is a different way to unlock the same underlying master key, not a separate encryption key of its own (see "Low-level (cat) and object listing" below's note on #cat_masterkey_and_game_over_if_this_leaks).

#key_add/#key_passwd share the same kwargs: new_password: is required, accepting the same shapes as #initialize's password: minus [:command, ...]: a literal String; a [:file, "..."] tuple, resolved natively by Restic; :insecure_no_password; or anything responding to #call. Restic itself only accepts a new password by file, unlike the repository's own password:, so a literal String or callable is written to a briefly-lived, 0600-permissioned tempfile by Yobi first. See the YARD docs for the full list of options.

#key_passwd rotates the password, not the underlying master key itself (see the note on #cat_masterkey_and_game_over_if_this_leaks below if that distinction matters for your threat model). On success, it also updates this same Repository instance's own password: to match, so it keeps working against the repository right afterward:

repo.key_passwd(new_password: "new-password-value")
repo.snapshots # already using the new password - no need to rebuild `repo`

Restic refuses to #key_remove the key currently in use.

Low-level (cat) and object listing

repo.cat_snapshot(snapshot)      # => raw stored record, as a Hash
repo.cat_tree(snapshot, subfolder: "var/log")
repo.cat_pack(pack_id) { |io| ... }   # raw, still-encrypted bytes
repo.cat_blob(blob_id) { |io| ... }   # raw, decrypted bytes
repo.list(:packs)                     # => Array of pack IDs

#cat_snapshot/#cat_index/#cat_key/#cat_tree return a repository object's raw stored record as a Hash, lower-level than #snapshots/#key_list, which wrap entries in Snapshot/Key instead. #cat_snapshot/#cat_key/#cat_tree each accept either a bare ID String or the corresponding wrapper object (Snapshot/Key/Snapshot) directly; #cat_index only takes a bare ID String, since index entries have no wrapper class of their own. #cat_pack/#cat_blob are raw-bytes commands with the same block/IOHandle shape as #dump. #list(type) enumerates every object ID of a given type (:blobs/:packs/:index/:snapshots/:keys/:locks) as plain strings. Restic ignores --json for this command, so these come back exactly as Restic prints them.

#cat_masterkey_and_game_over_if_this_leaks returns the repository's actual encryption/MAC key material: not a redacted reference, the real thing. There is no operation that rotates this key; every password change made afterward protects nothing already backed up if this value ever leaks. Treat the return value with more care than you'd give password: itself: unlike a password, this can't be rotated away after a leak.

Across repositories

#init_mirror

mirror_repo = primary_repo.init_mirror(url: "b2:my-bucket:/", password: "...")

A shortcut for setting up a #copy destination: constructs a new Repository and initializes it with chunker parameters copied from primary_repo, so #copy between the two can deduplicate (skipped this way, Restic picks each repository's chunker polynomial independently at #init time, so nothing between them would ever dedupe). A one-time setup call, not an ongoing sync - pair it with repeated #copy calls below to actually move snapshots across.

#copy

mirror_repo.copy(from_repo: primary_repo, tags: "nightly")

Replicates snapshots from another repository into this one. from_repo: accepts a plain URL String or a Repository instance. Given the latter, its own #url/#password are used automatically unless from_password: is given explicitly. Already-copied snapshots are skipped automatically on a repeat run. See the YARD docs for the full list of options. Returns true.

Mounting

#mount

repo.mount(mountpoint: "/mnt/restic") do |mount|
  Dir.children("#{mount.mountpoint}/snapshots/latest")
end

Serves the repository as a read-only FUSE filesystem at mountpoint (which must already exist). With a block, yields a Yobi::MountHandle and stops it automatically afterward. Without one, returns the MountHandle for you to #stop yourself:

mount = repo.mount(mountpoint: "/mnt/restic")
begin
  # ... the mount stays live until something calls #stop ...
ensure
  mount.stop
end

#stop is safe to call more than once, and MountHandle#pid/MountHandle#stop are safe even if something else already triggered the unmount externally (kill -INT <pid>, or the OS's own umount/fusermount directly). hosts:/paths:/tags: restrict which snapshots appear under the mount's own snapshots/ directory. ready_timeout: (default 10 seconds) bounds how long #mount waits for Restic's own readiness message before raising Yobi::MountTimeout. See the YARD docs for the full list of options.

Restic-level (not scoped to a repository)

restic = Yobi::Restic.new
restic.version.version # => "0.19.1"
restic.cache(cleanup: true)

#version and #cache are the two Restic subcommands that don't touch a repository at all, so they live on Yobi::Restic rather than Repository. #version returns a Yobi::ResticVersion (#version, #go_version, #go_os, #go_arch); if the installed binary is old enough to ignore --json for this command entirely, Yobi falls back to parsing its plain-text output for the version number instead of raising a JSON parse error. #cache lists (or, with cleanup: true, cleans) local cache directories, returning true. See the YARD docs for its remaining options.

Global Restic settings

Settings that apply regardless of which repository is being operated on live on Yobi::Restic, not Repository:

restic = Yobi::Restic.new(
  cache_dir: "/var/cache/restic",
  limit_upload: 5000,
  no_lock: true
)

Two genuinely different kinds, both exposed as plain attr_accessors:

• Env-var-backed (e.g. cache_dir:, compression:, pack_size:): Restic#env computes the corresponding RESTIC_* vars fresh from the current accessor values on every call. Mutating one after construction is picked up by the very next command.
• CLI-flag-only, no env var equivalent (e.g. limit_upload:, no_lock:, options: - Restic's own --option/-o, an escape hatch for backend-specific tuning like s3.connections=10).

See the YARD docs for the full list of settings.

"Picked up by the very next command" assumes calls happen one at a time. If you mutate one of these accessors on a Restic shared across threads while another thread has a command already in flight on it, which value that in-flight command actually uses is a race with no ordering guarantee, though not a crash: each accessor read/write is still atomic, so you won't get a corrupted argv, just an unpredictable choice between the old and new value.

Sharing a Restic across repositories

restic = Yobi::Restic.new(limit_upload: 5000)

customer_a = Yobi::Repository.new(url: "s3:.../customer-a", password: "...", restic: restic)
customer_b = Yobi::Repository.new(url: "s3:.../customer-b", password: "...", restic: restic)

Both repositories now use the same limit_upload: value, and changing restic.limit_upload later affects every repository built from it. This isn't a shared, pooled cap: Restic applies --limit-upload per invocation, so two backups running concurrently through this same restic are each independently capped at 5000, not splitting one combined 5000 between them. Useful when one process backs up many repositories under one operational policy, without repeating the same settings on every Repository.new call.

Minimum-version enforcement, in detail

Yobi::Restic#run/#run_dump/#run_mount all call #ensure_minimum_version! before executing a real command. It's memoized per Restic instance. The check runs once, not on every call, since the binary on disk isn't expected to change mid-process. See "Minimum supported version" above for what triggers it and why 0.17.1 specifically.

Redaction

Both Repository#inspect and Restic#inspect redact sensitive values, so a stray pp/puts/log call never prints a credential in plaintext:

repo.inspect
# => #<Yobi::Repository url="s3:..." password="[FILTERED]" backend_credentials={"AWS_ACCESS_KEY_ID"=>"[FILTERED]", "AWS_SECRET_ACCESS_KEY"=>"[FILTERED]"}>

A literal or command/file-sourced password: shows as "[FILTERED]"; anything callable shows as "[RESOLVER]" without ever being called. Printing a Repository never triggers a real secret-store lookup as a side effect. :insecure_no_password shows plainly, since it's an explicit "there is no secret" declaration, not a value that could leak anything. backend_credentials: keys not in Restic's own documented list of non-identity operational env vars (Yobi::Restic::ALLOWED_ENV_VARS) are filtered the same way; the allowed ones (RESTIC_CACHE_DIR, AWS_DEFAULT_REGION, and similar non-secret operational vars) stay visible.

#env itself is never redacted. It's the method that actually builds what gets passed to Process.spawn, so it has to return the real values. Only #inspect (the thing a stray debug print might accidentally call) filters.

Recipes

Examples for common day-to-day scenarios.

Scheduled backup with retention

A nightly job (cron, whenever, a Sidekiq-cron/sidekiq-scheduler job, whatever runs on your schedule):

repo = Yobi::Repository.new(
  url: ENV.fetch("RESTIC_REPOSITORY"),
  password: [:file, "/etc/restic/password"]
)

outcome = repo.backup(
  source: "/var/myapp/uploads",
  tags: ["myapp", "nightly"],
  exclude_caches: true
)

outcome.errors.each { |e| log_error("backup: #{e.item}: #{e.message}") }

repo.forget(
  keep_daily: 7,
  keep_weekly: 4,
  keep_monthly: 12,
  tags: ["myapp"],
  prune: true
)

A non-empty outcome.errors (exit code 3) means some files were skipped but the snapshot was still created (often fine to continue past, e.g. permission-denied on a handful of files); a full failure raises before outcome even exists via the usual Yobi::ResticCommandFailed/Yobi::RepositoryLocked/etc. hierarchy, so wrap the whole thing in the usual rescue if you want to alert on that separately (see "Graceful lock-contention handling" below).

Restore workflows

Restoring the latest snapshot, the common case:

repo.restore(snapshot_id: "latest", target: "/var/myapp/uploads")

Restoring a specific snapshot, found by filtering:

snapshot = repo.snapshots(tags: "nightly").max_by(&:time)
repo.restore(snapshot_id: snapshot.id, target: "/tmp/restore-#{snapshot.short_id}")

Browsing before committing to a restore: mount the repository and look around, rather than restoring blind:

repo.mount(mountpoint: "/mnt/restic") do |mount|
  Dir.children("#{mount.mountpoint}/snapshots/latest/var/myapp/uploads")
end

or list without mounting anything:

repo.ls(snapshot_id: "latest", dirs: "/var/myapp/uploads").nodes.each do |node|
  puts "#{node.type}  #{node.size}  #{node.path}"
end

Partial restore: only what you actually need back:

repo.restore(
  snapshot_id: "latest",
  target: "/tmp/restore",
  includes: "/var/myapp/uploads/2026-*"
)

Credentials from a secret store

Any callable works for password:/backend_credentials:. It's invoked fresh immediately before every Restic command, never cached by Yobi itself:

repo = Yobi::Repository.new(
  url: "s3:s3.amazonaws.com/my-bucket",
  password: -> { Rails.application.credentials.dig(:restic, :password) },
  backend_credentials: -> {
    {
      "AWS_ACCESS_KEY_ID" => Rails.application.credentials.dig(:aws, :access_key_id),
      "AWS_SECRET_ACCESS_KEY" => Rails.application.credentials.dig(:aws, :secret_access_key)
    }
  }
)

Or from Vault:

password: -> {
  Vault.logical.read("secret/data/restic")&.data&.dig(:data, :password)
}

If the resolver itself is slow or rate-limited (a network round-trip per Restic invocation adds up over many commands), add your own caching inside the lambda; how long the result stays valid is entirely your call.

Database dumps via stdin_from_command

Backing up a live database consistently means dumping it first, not copying its on-disk files while they might be mid-write. source: [:stdin_from_command, ...] has Restic spawn the dump command itself and back up its stdout directly, with no intermediate file ever touching disk:

repo.backup(
  source: [:stdin_from_command, "pg_dump --no-owner mydb", "mydb.sql"],
  tags: ["mydb", "nightly"]
)

The third element ("mydb.sql") names the resulting virtual file inside the snapshot. Omit it to fall back to Restic's own "stdin" default. If the command needs an argument containing a literal space (a password with a space in it, an unusual path), pass it as an Array of already-discrete arguments instead of a single String, since Shellwords-splitting a string would otherwise break it apart incorrectly. There's no shell involved either way (Restic execs the tokenized argv directly), but if any part of the String form is ever built from untrusted input rather than a hardcoded command like above, Shellwords.split will still tokenize whatever that input contains, letting it inject extra arguments into what gets executed. Use the Array form with untrusted values kept to their own discrete elements instead.

repo.backup(source: [:stdin_from_command, ["mysqldump", "-u", "backup", "my app db"]])

If the dump command itself fails partway (wrong credentials, database down), Restic detects the non-zero exit and does not persist a snapshot. It raises Yobi::ResticCommandFailed the same as any other command failure, so a broken dump never silently becomes a "successful" backup of garbage data.

Live progress reporting

Streaming #backup's block into something other than stdout: a logger, a progress bar, a broadcast to a browser:

# A logger
repo.backup(source: source_path) do |message|
  case message
  when Yobi::BackupStatus
    logger.info("backup: #{((message.percent_done || 0) * 100).round}%")
  when Yobi::BackupError
    logger.error("backup: #{message.item}: #{message.message}")
  end
end

# ActionCable, for a live-updating admin dashboard
repo.backup(source: source_path) do |message|
  next unless message.is_a?(Yobi::BackupStatus)

  BackupChannel.broadcast_to(job, {
    percent: message.percent_done,
    files_done: message.files_done,
    total_files: message.total_files
  })
end

Multi-repository operations

One Restic (holding shared operational settings) backing many Repository instances, e.g. one repository per customer or per managed server:

restic = Yobi::Restic.new(limit_upload: 10_000, cache_dir: "/var/cache/restic")

def repository_for(customer)
  Yobi::Repository.new(
    url: "s3:s3.amazonaws.com/backups/#{customer.id}",
    password: -> { customer.restic_password },
    restic: restic
  )
end

Customer.find_each do |customer|
  repository_for(customer).backup(source: customer.data_path, tags: [customer.id.to_s])
end

Running commands concurrently through a shared Repository/Restic is safe (see "Global Restic settings" above for the one caveat, mutating restic's tuning accessors from another thread while backups are in flight), so backing up many repositories doesn't have to run strictly one at a time. A Thread.new per customer would spawn an unbounded number of concurrent restic subprocesses, and Ruby silently drops an unhandled exception from a thread that's never joined, so a single customer's failure could vanish instead of getting logged. A small fixed-size worker pool avoids both:

queue = Queue.new
Customer.find_each { |customer| queue << customer }
queue.close

workers = 4.times.map do
  Thread.new do
    while (customer = queue.pop)
      begin
        repository_for(customer).backup(source: customer.data_path, tags: [customer.id.to_s])
      rescue Yobi::Error => e
        log_error("backup failed for customer #{customer.id}: #{e.message}")
      end
    end
  end
end
workers.each(&:join)

Copying between repositories (e.g. replicating to a second, offsite backend for a 3-2-1 strategy), see #init_mirror/#copy under "Across repositories":

mirror_repo = primary_repo.init_mirror(url: "b2:my-bucket:/", password: "...") # once, before the first #copy
mirror_repo.copy(from_repo: primary_repo, tags: "nightly") # run nightly, e.g. from cron

Repository maintenance and health checks

A separate, less-frequent job (weekly, say) than the nightly backup itself:

check = repo.check(read_data_subset: "10%")
if check.summary.num_errors.positive?
  check.errors.each { |e| AlertService.notify("Restic check: #{e.message}") }
end

repo.prune(max_unused: "5%")

read_data_subset: rotates through the repository over time (e.g. "1/20" today, "2/20" tomorrow) rather than reading everything in one run, so a full verification eventually happens without one job spending hours reading the entire dataset.

Graceful lock-contention handling

Two processes touching the same repository at once (a backup job and a prune job overlapping, say): retry instead of failing the whole run:

def with_retry(max_attempts: 3, delay: 30)
  attempts = 0
  begin
    yield
  rescue Yobi::RepositoryLocked
    attempts += 1
    raise if attempts >= max_attempts

    sleep(delay + rand(delay)) # jittered, so lockstep retries against the same lock spread out
    retry
  end
end

with_retry { repo.backup(source: source_path) }

For a one-off manual fix instead of retrying, repo.unlock removes stale locks left by a process that crashed without cleaning up after itself, but don't call it reflexively inside a retry loop that might be racing a legitimately still-running second process; that would double-run the backup.

Non-goals

• Windows. Relies on Process.spawn/POSIX unlink semantics; only verified on Linux and macOS so far.
• Bundled secret-vault integration. Credentials are your own responsibility. Pass a callable for password:/backend_credentials: if you need one resolved from Vault, 1Password, or similar (see "Credentials from a secret store" above).
• A built-in scheduler/orchestrator. Yobi runs a command when you call it; deciding when that happens (cron, whenever, a Sidekiq-cron job) is your own responsibility, see "Scheduled backup with retention" above.

Contributing

Bug reports and pull requests are welcome at https://codeberg.org/ukazap/yobi.

After checking out the repo, run bin/setup to install dependencies.

• bin/test-unit: the unit suite, no I/O, no Restic/minio needed.
• bin/test-integration: the integration suite, shelling out to a real Restic binary; skipped automatically if Restic isn't installed.
• bin/test: both, in parallel.
• bin/standardrb: lint.
• bin/console: an interactive prompt with the gem loaded.
• bin/docs: serves YARD docs at http://localhost:8808, reparsing on every request.

See MAINTAINERS.md for what to keep in sync as the API changes, and the release checklist.

License

The gem is available as open source under the terms of the MIT License.

☂️☔️☂️☔️☂️