Class: RoundhouseUi::Backends::Sidekiq

Inherits:
Object
  • Object
show all
Defined in:
lib/roundhouse_ui/backends/sidekiq.rb

Overview

The Sidekiq implementation of the Roundhouse backend port. It is a thin seam over Sidekiq's own API objects — controllers ask the backend for a "set" or "stats" instead of naming Sidekiq::* directly, so a second backend (Solid Queue) can supply duck-typed equivalents without touching controllers or views.

Mostly thin — most methods return the Sidekiq object the controllers used before the port was extracted. The exceptions are the batched reads (queue_summaries) and the capability list, which the Solid Queue backend firmed up.

Constant Summary collapse

CAPABILITIES =

Capabilities let the UI hide what a backend can't do. Sidekiq supports all the sets/views. On OSS Sidekiq pause is NOT native (it needs our fetcher), so :native_pause is withheld and the "not enforced" warning applies — but Sidekiq Pro ships its own enforced pause, so there it is advertised and the warning drops away.

%i[retries dead scheduled busy workers redis capsules cancel history
retry_job enqueue enqueue_now snapshots].freeze
CONCURRENCY_TTL =

Total worker threads across every reporting process, for the capacity figure (#36). Deliberately not Stats#workers_size, which counts threads busy this instant — that goes to zero on an idle fleet, and dividing a required rate by it would claim the fleet has infinite headroom.

Two things this call has to get right, because it runs on the poll endpoint that every open tab hits every few seconds:

* `ProcessSet.new(false)` skips Sidekiq's dead-process cleanup, which
issues a SET and can issue an SREM. Roundhouse is an observer; a
read endpoint must not write to Redis, least of all on a read-only
install. Sidekiq's own workers prune the set.
* The result is cached briefly. Thread counts change on deploy or
scale, not between polls, so re-reading the process set on every
poll spends round-trips on an answer that has not moved.
15.0

Class Attribute Summary collapse

Instance Method Summary collapse

Class Attribute Details

.concurrency_cache ⇒ Object

Returns the value of attribute concurrency_cache.



26
27
28
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 26

def concurrency_cache
  @concurrency_cache
end

Instance Method Details

#busy ⇒ Object

Currently-executing jobs, normalized to the Busy view's shape. (This normalization used to live in BusyController; it belongs to the backend.)



132
133
134
135
136
137
138
139
140
141
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 132

def busy
  ::Sidekiq::WorkSet.new.map do |process_id, tid, work|
    if work.respond_to?(:queue) # Sidekiq 7+: Sidekiq::Work struct
      { process: process_id, tid: tid, queue: work.queue, run_at: work.run_at, job: work.job }
    else                         # Sidekiq 6.x: a plain Hash
      { process: process_id, tid: tid, queue: work["queue"],
        run_at: Time.at(work["run_at"]), job: ::Sidekiq::JobRecord.new(work["payload"]) }
    end
  end
end

#concurrency ⇒ Object

seconds



73
74
75
76
77
78
79
80
81
82
83
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 73

def concurrency
  now = ::Process.clock_gettime(::Process::CLOCK_MONOTONIC)
  cached = self.class.concurrency_cache
  return cached[:value] if cached && now - cached[:at] < CONCURRENCY_TTL

  value = ::Sidekiq::ProcessSet.new(false).sum { |p| p["concurrency"].to_i }
  self.class.concurrency_cache = { value: value, at: now }
  value
rescue StandardError
  nil
end

#dead_set ⇒ Object 



113
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 113

def dead_set     = ::Sidekiq::DeadSet.new

#fetch_installed? ⇒ Boolean

Returns:

  • (Boolean) 


186
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 186

def fetch_installed?  = RoundhouseUi::Pause.fetch_installed?

#history(days) ⇒ Object

Sidekiq keeps a per-day counter whether anyone reads it or not, so this is free history — no storage of ours, no migration, works everywhere. Oldest first: a chart should not have to reverse it.



47
48
49
50
51
52
53
54
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 47

def history(days)
  h = ::Sidekiq::Stats::History.new(days)
  processed = h.processed
  failed = h.failed
  processed.keys.sort.map do |date|
    RoundhouseUi::Day.new(date: date, processed: processed[date].to_i, failed: failed[date].to_i)
  end
end

#name ⇒ Object 



13
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 13

def name = "Sidekiq"

#pause(name) ⇒ Object 



184
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 184

def pause(name)       = RoundhouseUi::Pause.pause!(name)

#paused_queues ⇒ Object

--- pause (via the opt-in Roundhouse fetcher; not native) ---



183
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 183

def paused_queues     = RoundhouseUi::Pause.paused_set

#process_set ⇒ Object 



116
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 116

def process_set  = ::Sidekiq::ProcessSet.new

#push(payload) ⇒ Object 



128
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 128

def push(payload) = ::Sidekiq::Client.push(payload)

#queue(name) ⇒ Object 



37
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 37

def queue(name)  = ::Sidekiq::Queue.new(name)

#queue_jobs(name) ⇒ Object

The jobs waiting on one queue. Sidekiq::Queue is Enumerable and pages through Redis in chunks, so callers that stop early (browse fills one page and breaks) never load a large queue into memory.



42
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 42

def queue_jobs(name) = ::Sidekiq::Queue.new(name)

#queue_summaries ⇒ Object

Depth and latency for every queue in two Redis round-trips, regardless of how many queues there are.

Sidekiq::Queue#size and #latency each issue their own command, so the obvious queues.map { ... } costs one SSCAN plus a LLEN and an LRANGE per queue — on an app with sixty queues, a couple of hundred round-trips to render one page. Sidekiq's own Queues#lengths pipelines the LLENs for the same reason; this pipelines the LRANGEs alongside them.

Queues with no ready work are kept, at size 0: an operator has to be able to pause or snapshot a queue that happens to be empty right now.



96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 96

def queue_summaries
  # Sidekiq 8 ships an equivalent (Stats#queue_summaries) but 6.5 and 7 do
  # not, and its Data objects carry a `paused` field ours cannot. Branching
  # on it would hand callers a different shape depending on which Sidekiq
  # the host runs, so this stays one implementation for every supported
  # version — the reason the gem is portable in the first place.
  names = ::Sidekiq.redis { |c| c.call("SMEMBERS", "queues") }.sort
  return [] if names.empty?

  sizes, oldest = batch_read(names).each_slice(names.size).to_a
  names.each_with_index.map do |name, i|
    RoundhouseUi::QueueSummary.new(name: name, size: sizes[i].to_i,
                                  latency: latency_from(oldest[i]))
  end
end

#queues ⇒ Object 



36
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 36

def queues       = ::Sidekiq::Queue.all

#resume(name) ⇒ Object 



185
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 185

def resume(name)      = RoundhouseUi::Pause.unpause!(name)

#retry_set ⇒ Object 



112
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 112

def retry_set    = ::Sidekiq::RetrySet.new

#scheduled_set ⇒ Object 



114
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 114

def scheduled_set = ::Sidekiq::ScheduledSet.new

#set(kind) ⇒ Object

The job set a given UI section maps to (used by the job-detail page). Lazy — only builds the requested set.



120
121
122
123
124
125
126
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 120

def set(kind)
  case kind.to_s
  when "dead"      then dead_set
  when "retry"     then retry_set
  when "scheduled" then scheduled_set
  end
end

#stats ⇒ Object 



35
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 35

def stats        = ::Sidekiq::Stats.new

#supports?(capability) ⇒ Boolean

Returns:

  • (Boolean) 


29
30
31
32
33
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 29

def supports?(capability)
  return RoundhouseUi::Pause.native? if capability == :native_pause

  CAPABILITIES.include?(capability)
end

#work_set ⇒ Object 



115
 
# File 'lib/roundhouse_ui/backends/sidekiq.rb', line 115

def work_set     = ::Sidekiq::WorkSet.new