Class: RSpecTracer::RemoteCache::RedisBackend Private

Inherits:
Object
  • Object
show all
Defined in:
lib/rspec_tracer/remote_cache/redis_backend.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

Redis implementation of ‘RemoteCache::Backend`. Each cache ref is one Redis hash keyed under the two-tier layout:

<prefix>:main:<sha>[:<test_suite_id>]           -> HASH
<prefix>:pr:<branch>:<sha>[:<test_suite_id>]    -> HASH
<prefix>:pr:<branch>:branch_refs                -> STRING (JSON)

Hash fields per ref:

_timestamp       -> epoch float (string; microsecond resolution
                     to keep within-second orderings stable for
                     count-based prune)
last_run.json    -> JSON content verbatim
<run_id>/<f>.json -> JSON content per file in the 15-file layout

Why hashmap and not a binary archive (like S3 / LocalFs): hash- per-ref is the idiomatic Redis data model, matches the brief, and gives operational visibility via ‘redis-cli HGETALL` / `HKEYS` / `HLEN` without extracting an archive first. The storage cost (no gzip) is negligible for realistic cache sizes.

Retention: identical dispatch to S3 / LocalFs. The orchestrator calls ‘prune!(count:, duration_seconds:, pr_branch_ttl_seconds:)` after each upload; this backend enumerates via SCAN + HGET on the per-ref `_timestamp` field, then DELs stale keys. TTL-on-SET (i.e. letting Redis EXPIRE handle it natively) is a reasonable ergonomic followup but is not required for correctness - the explicit prune pass already achieves the same eviction outcome.

Graceful-degradation contract:

- `redis` gem missing -> constructor raises RedisBackendError;
  UserTasks rescues at the top, logs a clear "add gem to your
  Gemfile" message, falls back to cold run. Never propagates.
- Wire failure (connection refused, timeout) -> redis-rb raises
  Redis::BaseError subclasses. `download` catches and returns
  false; `upload` lets them propagate for the Rake task to log.
  Same rescue model as S3Backend.

The ‘redis` gem is an OPTIONAL runtime dependency - users add `gem ’redis’‘ to their own Gemfile to use RedisBackend. The constructor calls `require ’redis’‘ lazily and raises a clear RedisBackendError if the gem is absent, which UserTasks converts into a warning + cold run.

Defined Under Namespace

Classes: RedisBackendError

Constant Summary collapse

MAIN_TIER =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future.

Internal constant.

'main'
PR_TIER =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future.

Internal constant.

'pr'
BRANCH_REFS_SUFFIX =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future.

Internal constant.

'branch_refs'
PR_BRANCHES_SUFFIX =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future.

Internal constant.

'pr_branches'
LAST_RUN_FIELD =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future.

Internal constant.

'last_run.json'
TIMESTAMP_FIELD =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future.

Internal constant.

'_timestamp'
ENCODING =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future.

Internal constant.

'UTF-8'
DEFAULT_SCAN_COUNT =

This constant is part of a private API. You should avoid using this constant if possible, as it may be removed or be changed in the future.

Internal constant.

200

Instance Method Summary collapse

Constructor Details

#initialize(prefix:, branch:, default_branch:, cache_path:, url: nil, redis_client: nil, test_suite_id: nil, logger: nil, ttl: nil) ⇒ RedisBackend

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

rubocop:disable Metrics/ParameterLists



87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
 
# File 'lib/rspec_tracer/remote_cache/redis_backend.rb', line 87

def initialize(prefix:, branch:, default_branch:, cache_path:,
               url: nil, redis_client: nil, test_suite_id: nil, logger: nil,
               ttl: nil)
  validate_required!(prefix: prefix, branch: branch,
                     default_branch: default_branch, cache_path: cache_path)
  validate_connection_source!(url: url, redis_client: redis_client)
  validate_ttl!(ttl)

  @prefix = trim_trailing_colons(prefix.to_s)
  @branch = branch.to_s.chomp
  @default_branch = default_branch.to_s.chomp
  @test_suite_id = normalize_test_suite_id(test_suite_id)
  @cache_path = cache_path.to_s
  @logger = logger
  @ttl = ttl
  @redis = redis_client || build_client(url)
end

Instance Method Details

#branch_refs(branch_name) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Read branch_refs for the given branch. Returns ‘=> ts_epoch` or `{}` when missing / malformed.



144
145
146
147
148
149
150
151
152
153
154
155
 
# File 'lib/rspec_tracer/remote_cache/redis_backend.rb', line 144

def branch_refs(branch_name)
  return {} if blank?(branch_name)

  raw = @redis.get(branch_refs_key(branch_name))
  return {} if raw.nil? || raw.empty?

  parsed = JSON.parse(raw)
  parsed.is_a?(Hash) ? parsed.transform_values(&:to_i) : {}
rescue StandardError => e
  log_debug("branch_refs read failed (#{e.class}: #{e.message}); treating as empty")
  {}
end

#download(ref, tree_sha: nil) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Download cache for ‘ref`. Tries own tier, falls back to main tier. Returns true on validated success, false otherwise. Cleans up partially-written files on failure.

‘tree_sha:` is accepted for protocol uniformity with S3Backend but is currently a no-op: the tree-SHA secondary index is an S3-only feature. Future enhancement may extend it here; the orchestrator already forwards the kwarg.



114
115
116
117
118
119
120
121
122
 
# File 'lib/rspec_tracer/remote_cache/redis_backend.rb', line 114

def download(ref, tree_sha: nil)
  _ = tree_sha
  return false if blank?(ref)

  tiers_to_try = [own_tier_segment]
  tiers_to_try << MAIN_TIER if pr_tier?

  tiers_to_try.any? { |tier| try_download_from(tier, ref) }
end

#prune!(count: nil, duration_seconds: nil, pr_branch_ttl_seconds: nil) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Apply retention to own tier. Returns count removed. Two or more knobs may be set; each applies independently. Never raises - a wire-level Redis error on any sub-prune logs + is absorbed. rubocop:disable Metrics/PerceivedComplexity



172
173
174
175
176
177
178
179
180
181
 
# File 'lib/rspec_tracer/remote_cache/redis_backend.rb', line 172

def prune!(count: nil, duration_seconds: nil, pr_branch_ttl_seconds: nil)
  removed = 0
  removed += prune_by_count!(count) if count&.positive?
  removed += prune_by_duration!(duration_seconds) if duration_seconds&.positive?
  removed += prune_dead_pr_branch!(pr_branch_ttl_seconds) if pr_tier? && pr_branch_ttl_seconds&.positive?
  removed
rescue StandardError => e
  log_warn("prune! failed (#{e.class}: #{e.message}); returning #{removed}")
  removed
end

#prune_all!(pr_branch_ttl_seconds: nil) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Cross-tier PR-branch cleanup. Enumerates every PR branch under the configured prefix (by scanning for ‘<prefix>:pr:<branch>:branch_refs` keys and deriving branch names), applies the TTL to each, deletes dead branches whole. Returns total refs removed. No-op on nil / non-positive TTL.



189
190
191
192
193
194
195
196
197
198
 
# File 'lib/rspec_tracer/remote_cache/redis_backend.rb', line 189

def prune_all!(pr_branch_ttl_seconds: nil)
  return 0 unless pr_branch_ttl_seconds&.positive?

  cutoff = Time.now.to_i - pr_branch_ttl_seconds.to_i
  branches = discover_pr_branches
  branches.sum { |branch| maybe_prune_branch(branch, cutoff) }
rescue StandardError => e
  log_warn("prune_all! failed (#{e.class}: #{e.message})")
  0
end

#unbounded_warning(warn_threshold: 500) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Warn when main tier has grown beyond threshold and no retention is configured.



202
203
204
205
206
207
208
 
# File 'lib/rspec_tracer/remote_cache/redis_backend.rb', line 202

def unbounded_warning(warn_threshold: 500)
  count = count_tier_refs(MAIN_TIER)
  return nil unless count > warn_threshold

  "rspec-tracer remote cache has #{count} refs in #{@prefix}:#{MAIN_TIER}; " \
    'configure cache_retention_count or cache_retention_duration to cap growth'
end

#upload(ref, tree_sha: nil) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Upload local cache as a hash under own-tier key. Raises on I/O or Redis wire failure.

‘tree_sha:` is accepted for protocol uniformity with S3Backend (no-op here; see `download`).

Raises:



129
130
131
132
133
134
135
136
137
138
139
140
 
# File 'lib/rspec_tracer/remote_cache/redis_backend.rb', line 129

def upload(ref, tree_sha: nil)
  _ = tree_sha
  raise RedisBackendError, 'ref is required' if blank?(ref)

  run_id = read_local_run_id
  raise RedisBackendError, "no local cache to upload (missing #{LAST_RUN_FIELD})" if run_id.nil?

  fields = build_upload_fields(run_id)
  key = ref_key(own_tier_segment, ref)
  write_upload_hash(key, fields)
  log_debug("uploaded cache for #{ref} to #{own_tier_segment} (#{fields.size} fields)")
end

#write_branch_refs(branch_name, refs) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Persist branch_refs for the given branch. No-op for main-branch writes. Raises on Redis wire failure for PR tier.



159
160
161
162
163
164
165
166
 
# File 'lib/rspec_tracer/remote_cache/redis_backend.rb', line 159

def write_branch_refs(branch_name, refs)
  return if blank?(branch_name)
  return if branch_name.to_s.chomp == @default_branch
  return if refs.nil? || refs.empty?

  @redis.set(branch_refs_key(branch_name), JSON.pretty_generate(refs))
  log_debug("wrote branch_refs for #{branch_name}")
end