Class: GitLab::Exporter::SidekiqProber

Inherits:
Object
  • Object
show all
Defined in:
lib/gitlab_exporter/sidekiq.rb

Overview

A prober for Sidekiq queues

It takes the Redis URL Sidekiq is connected to rubocop:disable Metrics/ClassLength

Constant Summary collapse

PROBE_JOBS_LIMIT =

The maximum depth (from the head) of each queue to probe. Probing the entirety of a very large queue will take longer and run the risk of timing out. But when we have a very large queue, we are most in need of reliable metrics. This trades off completeness for predictability by only taking a limited amount of items from the head of the queue.

1_000
POOL_SIZE = 
3
POOL_TIMEOUT =

This timeout is configured to higher interval than scrapping of Prometheus to ensure that connection is kept instead of needed to be re-initialized

90
SIDEKIQ_REDIS_LOCK =

Lock for Sidekiq.redis which we need to modify, but is not concurrency safe.

Mutex.new

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(metrics: PrometheusMetrics.new, logger: nil, **opts) ⇒ SidekiqProber

Returns a new instance of SidekiqProber.



40
41
42
43
44
45
46
47
48
49
50
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 40

def initialize(metrics: PrometheusMetrics.new, logger: nil, **opts)
  @opts    = opts
  @metrics = metrics
  @logger  = logger

  @probe_namespaced = !!opts[:probe_namespaced] # rubocop:disable Style/DoubleNegation
  @probe_non_namespaced = !!opts[:probe_non_namespaced] # rubocop:disable Style/DoubleNegation

  # to maintain backward compatibility if the config is missing values
  @probe_namespaced = true if opts[:probe_namespaced].nil? && opts[:probe_non_namespaced].nil?
end

Class Method Details

.connection_pool ⇒ Object 



32
33
34
35
36
37
38
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 32

def self.connection_pool
  @@connection_pool ||= Hash.new do |h, connection_hash| # rubocop:disable Style/ClassVars
    config = connection_hash.merge(pool_timeout: POOL_TIMEOUT, size: POOL_SIZE)

    h[connection_hash] = Sidekiq::RedisConnection.create(config)
  end
end

Instance Method Details

#probe_dead ⇒ Object 



105
106
107
108
109
110
111
112
113
114
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 105

def probe_dead
  puts "[DEPRECATED] probe_dead is now considered obsolete and will be removed in future major versions,"\
       " please use probe_stats instead"

  with_sidekiq do
    @metrics.add("sidekiq_dead_jobs", Sidekiq::Stats.new.dead_size)
  end

  self
end

#probe_future_sets ⇒ Object 



73
74
75
76
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 73

def probe_future_sets
  with_sidekiq { _probe_future_sets } if @probe_namespaced
  with_sidekiq(false) { _probe_future_sets(false) } if @probe_non_namespaced
end

#probe_jobs ⇒ Object 



66
67
68
69
70
71
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 66

def probe_jobs
  puts "[REMOVED] probe_jobs is now considered obsolete and does not emit any metrics,"\
       " please use probe_jobs_limit instead"

  self
end

#probe_jobs_limit ⇒ Object

Count worker classes present in Sidekiq queues. This only looks at the first PROBE_JOBS_LIMIT jobs in each queue. This means that we run a single LRANGE command for each queue, which does not block other commands. For queues over PROBE_JOBS_LIMIT in size, this means that we will not have completely accurate statistics, but the probe performance will also not degrade as the queue gets larger.



84
85
86
87
88
89
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 84

def probe_jobs_limit
  with_sidekiq { _probe_jobs_limit } if @probe_namespaced
  with_sidekiq(false) { _probe_jobs_limit(false) } if @probe_non_namespaced

  self
end

#probe_queues ⇒ Object 



59
60
61
62
63
64
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 59

def probe_queues
  with_sidekiq { _probe_queues } if @probe_namespaced
  with_sidekiq(false) { _probe_queues(false) } if @probe_non_namespaced

  self
end

#probe_retries ⇒ Object 



98
99
100
101
102
103
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 98

def probe_retries
  with_sidekiq { _probe_retries } if @probe_namespaced
  with_sidekiq(false) { _probe_retries(false) } if @probe_non_namespaced

  self
end

#probe_stats ⇒ Object 



52
53
54
55
56
57
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 52

def probe_stats
  with_sidekiq { _probe_stats } if @probe_namespaced
  with_sidekiq(false) { _probe_stats(false) } if @probe_non_namespaced

  self
end

#probe_workers ⇒ Object 



91
92
93
94
95
96
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 91

def probe_workers
  with_sidekiq { _probe_workers } if @probe_namespaced
  with_sidekiq(false) { _probe_workers(false) } if @probe_non_namespaced

  self
end

#write_to(target) ⇒ Object 



116
117
118
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 116

def write_to(target)
  target.write(@metrics.to_s)
end