Class: GitLab::Exporter::SidekiqProber

Inherits:
Object
  • Object
show all
Defined in:
lib/gitlab_exporter/sidekiq.rb

Overview

A prober for Sidekiq queues

It takes the Redis URL Sidekiq is connected to rubocop:disable Metrics/ClassLength

Constant Summary collapse

PROBE_JOBS_LIMIT =

The maximum depth (from the head) of each queue to probe. Probing the entirety of a very large queue will take longer and run the risk of timing out. But when we have a very large queue, we are most in need of reliable metrics. This trades off completeness for predictability by only taking a limited amount of items from the head of the queue.

1_000
POOL_SIZE = 
3
POOL_TIMEOUT =

This timeout is configured to higher interval than scrapping of Prometheus to ensure that connection is kept instead of needed to be re-initialized

90

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(metrics: PrometheusMetrics.new, logger: nil, **opts) ⇒ SidekiqProber

Returns a new instance of SidekiqProber.



37
38
39
40
41
42
43
44
45
46
47
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 37

def initialize(metrics: PrometheusMetrics.new, logger: nil, **opts)
  @opts    = opts
  @metrics = metrics
  @logger  = logger

  @probe_namespaced = !!opts[:probe_namespaced] # rubocop:disable Style/DoubleNegation
  @probe_non_namespaced = !!opts[:probe_non_namespaced] # rubocop:disable Style/DoubleNegation

  # to maintain backward compatibility if the config is missing values
  @probe_namespaced = true if opts[:probe_namespaced].nil? && opts[:probe_non_namespaced].nil?
end

Class Method Details

.connection_pool ⇒ Object 



29
30
31
32
33
34
35
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 29

def self.connection_pool
  @@connection_pool ||= Hash.new do |h, connection_hash| # rubocop:disable Style/ClassVars
    config = connection_hash.merge(pool_timeout: POOL_TIMEOUT, size: POOL_SIZE)

    h[connection_hash] = Sidekiq::RedisConnection.create(config)
  end
end

Instance Method Details

#probe_dead ⇒ Object 



102
103
104
105
106
107
108
109
110
111
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 102

def probe_dead
  puts "[DEPRECATED] probe_dead is now considered obsolete and will be removed in future major versions,"\
       " please use probe_stats instead"

  with_sidekiq do
    @metrics.add("sidekiq_dead_jobs", Sidekiq::Stats.new.dead_size)
  end

  self
end

#probe_future_sets ⇒ Object 



70
71
72
73
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 70

def probe_future_sets
  with_sidekiq { _probe_future_sets } if @probe_namespaced
  with_sidekiq(false) { _probe_future_sets(false) } if @probe_non_namespaced
end

#probe_jobs ⇒ Object 



63
64
65
66
67
68
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 63

def probe_jobs
  puts "[REMOVED] probe_jobs is now considered obsolete and does not emit any metrics,"\
       " please use probe_jobs_limit instead"

  self
end

#probe_jobs_limit ⇒ Object

Count worker classes present in Sidekiq queues. This only looks at the first PROBE_JOBS_LIMIT jobs in each queue. This means that we run a single LRANGE command for each queue, which does not block other commands. For queues over PROBE_JOBS_LIMIT in size, this means that we will not have completely accurate statistics, but the probe performance will also not degrade as the queue gets larger.



81
82
83
84
85
86
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 81

def probe_jobs_limit
  with_sidekiq { _probe_jobs_limit } if @probe_namespaced
  with_sidekiq(false) { _probe_jobs_limit(false) } if @probe_non_namespaced

  self
end

#probe_queues ⇒ Object 



56
57
58
59
60
61
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 56

def probe_queues
  with_sidekiq { _probe_queues } if @probe_namespaced
  with_sidekiq(false) { _probe_queues(false) } if @probe_non_namespaced

  self
end

#probe_retries ⇒ Object 



95
96
97
98
99
100
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 95

def probe_retries
  with_sidekiq { _probe_retries } if @probe_namespaced
  with_sidekiq(false) { _probe_retries(false) } if @probe_non_namespaced

  self
end

#probe_stats ⇒ Object 



49
50
51
52
53
54
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 49

def probe_stats
  with_sidekiq { _probe_stats } if @probe_namespaced
  with_sidekiq(false) { _probe_stats(false) } if @probe_non_namespaced

  self
end

#probe_workers ⇒ Object 



88
89
90
91
92
93
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 88

def probe_workers
  with_sidekiq { _probe_workers } if @probe_namespaced
  with_sidekiq(false) { _probe_workers(false) } if @probe_non_namespaced

  self
end

#write_to(target) ⇒ Object 



113
114
115
 
# File 'lib/gitlab_exporter/sidekiq.rb', line 113

def write_to(target)
  target.write(@metrics.to_s)
end