Module: Hyperion::PrometheusExporter

Defined in:
lib/hyperion/prometheus_exporter.rb

Overview

Renders Hyperion.stats as Prometheus text exposition format (v0.0.4). Mounted by AdminMiddleware on GET /-/metrics; the returned content-type is ‘text/plain; version=0.0.4; charset=utf-8`.

Mapping rules:

  • keys listed in KNOWN_METRICS get their canonical name + curated HELP/TYPE

  • keys matching ‘responses_<3-digit>` are grouped under a single `hyperion_responses_status_total` family with a `status` label

  • any other key is auto-exported as ‘hyperion_<key>` with a generic HELP line, so newly-added counters surface in Prometheus without code changes here (the curated-name path is just nicer presentation, not gating)

Output ordering is deterministic for stable scrape diffs:

  • known metrics in KNOWN_METRICS declaration order

  • status codes ascending

  • other keys alphabetically

Constant Summary collapse

KNOWN_METRICS = 
{
  requests: { name: 'hyperion_requests_total',
              help: 'Total HTTP requests handled',
              type: 'counter' },
  bytes_read: { name: 'hyperion_bytes_read_total',
                help: 'Total bytes read from request sockets',
                type: 'counter' },
  bytes_written: { name: 'hyperion_bytes_written_total',
                   help: 'Total bytes written to response sockets',
                   type: 'counter' },
  rejected_connections: { name: 'hyperion_rejected_connections_total',
                          help: 'Connections rejected due to backpressure (max_pending)',
                          type: 'counter' },
  sendfile_responses: { name: 'hyperion_sendfile_responses_total',
                        help: 'Responses sent via plain-TCP sendfile(2) zero-copy path',
                        type: 'counter' },
  tls_zerobuf_responses: { name: 'hyperion_tls_zerobuf_responses_total',
                           help: 'Responses sent via TLS IO.copy_stream (avoids userspace String build, but TLS encryption forces copy)',
                           type: 'counter' }
}.freeze
STATUS_KEY_PATTERN = 
/\Aresponses_(\d{3})\z/
STATUS_FAMILY_NAME = 
'hyperion_responses_status_total'
STATUS_FAMILY_HELP = 
'Responses by HTTP status code'
METRIC_DOCS =

2.4-C: curated HELP/TYPE preamble for the new histogram + gauge + labeled-counter families. Looking up by name keeps the rendered scrape body human-friendly even when the caller registered the family from a deep code path with no docstring.

{
  hyperion_request_duration_seconds: {
    help: 'HTTP request duration in seconds, by route template + method + status class',
    type: 'histogram'
  },
  hyperion_websocket_deflate_ratio: {
    help: 'WebSocket permessage-deflate compression ratio (original_bytes / compressed_bytes)',
    type: 'histogram'
  },
  hyperion_per_conn_rejections_total: {
    help: 'Per-connection in-flight cap rejections (503 + Retry-After), by worker',
    type: 'counter'
  },
  hyperion_tls_ktls_active_connections: {
    help: 'Active TLS connections currently driven by kernel TLS_TX, by worker',
    type: 'gauge'
  },
  hyperion_io_uring_workers_active: {
    help: 'Whether io_uring accept policy is active for this worker (1 = active, 0 = epoll)',
    type: 'gauge'
  },
  hyperion_threadpool_queue_depth: {
    help: 'In-flight count in the worker ThreadPool inbox (snapshot at scrape time)',
    type: 'gauge'
  },
  # 2.12-E — per-worker request counter for the SO_REUSEPORT
  # load-balancing audit. One series per worker (label_value =
  # `Process.pid.to_s`); ticks on every dispatched request from
  # every dispatch shape. Operators scrape /-/metrics N times in
  # cluster mode to gather distribution across workers.
  hyperion_requests_dispatch_total: {
    help: 'Requests dispatched per worker (PID-labeled), across all dispatch modes',
    type: 'counter'
  }
}.freeze
REQUESTS_DISPATCH_TOTAL =

2.12-E — name of the per-worker request counter family. Hoisted to a constant so the C-loop fold-in below stays declarative.

:hyperion_requests_dispatch_total

Class Method Summary collapse

Class Method Details

.c_loop_requests_total ⇒ Object 



200
201
202
203
204
205
206
207
208
209
 
# File 'lib/hyperion/prometheus_exporter.rb', line 200

def c_loop_requests_total
  return 0 unless defined?(::Hyperion::Http::PageCache)
  return 0 unless ::Hyperion::Http::PageCache.respond_to?(:c_loop_requests_total)

  ::Hyperion::Http::PageCache.c_loop_requests_total.to_i
rescue StandardError
  # The scrape path must never raise — observability code degrades
  # to "no fold-in" rather than failing the metrics endpoint.
  0
end

.default_label_keys(n) ⇒ Object 



281
282
283
 
# File 'lib/hyperion/prometheus_exporter.rb', line 281

def default_label_keys(n)
  Array.new(n) { |i| "label_#{i}" }
end

.escape_label(value) ⇒ Object 



292
293
294
295
296
297
298
299
300
301
302
303
304
 
# File 'lib/hyperion/prometheus_exporter.rb', line 292

def escape_label(value)
  out = +''
  value.each_char do |c|
    out << case c
           when '\\' then '\\\\'
           when '"'  then '\\"'
           when "\n" then '\\n'
           else
             c
           end
  end
  out
end

.format_float(v) ⇒ Object

Render histogram bucket edges with a stable representation. Integer- valued floats stay as ‘2.5` (not `2.5000000000000004`); fractional ones round to 6 places, plenty for scrape stability.



309
310
311
312
 
# File 'lib/hyperion/prometheus_exporter.rb', line 309

def format_float(v)
  f = v.to_f
  f == f.to_i ? f.to_i.to_s : format('%.6g', f)
end

.labels_block(pairs) ⇒ Object 



285
286
287
288
289
290
 
# File 'lib/hyperion/prometheus_exporter.rb', line 285

def labels_block(pairs)
  return '' if pairs.empty?

  inside = pairs.map { |k, v| %(#{k}="#{escape_label(v.to_s)}") }.join(',')
  "{#{inside}}"
end

.merge_c_loop_into_dispatch_snapshot(snap) ⇒ Object

2.12-E — merge ‘Hyperion::Http::PageCache.c_loop_requests_total` (process-global atomic ticked by the C accept4 + io_uring loops) into the `hyperion_requests_dispatch_totalworker_id=PID` series for the current worker. Without this fold-in, a `-w 4` cluster serving from the C accept loop would scrape zeros from every worker even though the loop’s atomic counter is ticking — the loop bypasses ‘Connection#serve`, so no Ruby-side `tick_worker_request` call ever lands.

We only fold in when the labeled-counter family is ALREADY in the snapshot — i.e., something on the Ruby side has called ‘Metrics#tick_worker_request` at least once on this sink. The `Server#run_c_accept_loop` boot path performs a single registration tick on the runtime’s metrics sink so this condition holds for any production cluster engaging the C loop.

Why the gate matters: spec fixtures that ‘Hyperion::Metrics.new` to assert “empty body when no metrics have been recorded” would otherwise pull in a process-global atomic bumped by an earlier spec’s C-loop run. Those fixtures never register the family, so the fold-in skips them cleanly.

Idempotent on snapshots that already contain a series for the current PID. Pure on the input — we build a shallow copy of the snapshot so the live Hash behind ‘Metrics#labeled_counter_snapshot` isn’t mutated.

Defensive: when the C ext isn’t loaded (JRuby / TruffleRuby) we silently skip — the snapshot stays Ruby-only.



179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
 
# File 'lib/hyperion/prometheus_exporter.rb', line 179

def merge_c_loop_into_dispatch_snapshot(snap)
  family = snap[REQUESTS_DISPATCH_TOTAL]
  return snap if family.nil?

  c_loop_count = c_loop_requests_total
  return snap if c_loop_count <= 0

  pid_label = Process.pid.to_s
  merged_series = family[:series].dup
  key = [pid_label].freeze
  existing_key = merged_series.keys.find { |k| k.first == pid_label } || key
  merged_series[existing_key] = (merged_series[existing_key] || 0) + c_loop_count

  merged = snap.dup
  merged[REQUESTS_DISPATCH_TOTAL] = {
    meta: family[:meta] || { label_keys: %w[worker_id].freeze },
    series: merged_series
  }
  merged
end

.render(stats) ⇒ Object 



93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
 
# File 'lib/hyperion/prometheus_exporter.rb', line 93

def render(stats)
  buf = +''
  grouped_status = {}
  other = {}
  known = {}

  stats.each do |key, value|
    if (match = key.to_s.match(STATUS_KEY_PATTERN))
      grouped_status[match[1]] = value
    elsif KNOWN_METRICS.key?(key)
      known[key] = value
    else
      other[key] = value
    end
  end

  # Known metrics first, in declaration order — gives the scrape a stable,
  # human-friendly preamble regardless of hash insertion order.
  KNOWN_METRICS.each do |key, meta|
    next unless known.key?(key)

    append_metric(buf, meta[:name], meta[:help], meta[:type], known[key])
  end

  unless grouped_status.empty?
    buf << "# HELP #{STATUS_FAMILY_NAME} #{STATUS_FAMILY_HELP}\n"
    buf << "# TYPE #{STATUS_FAMILY_NAME} counter\n"
    grouped_status.sort.each do |status, value|
      buf << %(#{STATUS_FAMILY_NAME}{status="#{status}"} #{value}\n)
    end
  end

  other.sort_by { |k, _| k.to_s }.each do |key, value|
    name = "hyperion_#{key}"
    append_metric(buf, name, 'Hyperion internal counter (auto-exported)', 'counter', value)
  end

  buf
end

.render_full(metrics_sink) ⇒ Object

2.4-C — render histograms, gauges, and labeled counters from a live Metrics instance. Called by AdminMiddleware in addition to ‘render` so /-/metrics surfaces the full 2.4-C observability surface in one scrape body. Order: legacy counters first (existing render), then histograms, gauges, labeled counters — each curated families first, auto-exported last, alphabetical within each section so scrape diffs stay stable.



140
141
142
143
144
145
146
147
148
 
# File 'lib/hyperion/prometheus_exporter.rb', line 140

def render_full(metrics_sink)
  buf = +''
  buf << render(metrics_sink.snapshot)
  buf << render_histograms(metrics_sink.histogram_snapshot)
  buf << render_gauges(metrics_sink.gauge_snapshot)
  labeled = merge_c_loop_into_dispatch_snapshot(metrics_sink.labeled_counter_snapshot)
  buf << render_labeled_counters(labeled)
  buf
end

.render_gauges(snap) ⇒ Object 



228
229
230
231
232
233
234
235
236
237
238
239
240
241
 
# File 'lib/hyperion/prometheus_exporter.rb', line 228

def render_gauges(snap)
  buf = +''
  snap.each do |name, payload|
    series = payload[:series]
    meta   = payload[:meta] || { label_keys: [].freeze }
    doc = METRIC_DOCS[name] || { help: 'Hyperion gauge', type: 'gauge' }
    buf << "# HELP #{name} #{doc[:help]}\n"
    buf << "# TYPE #{name} gauge\n"
    series.each do |label_values, value|
      buf << render_labeled_value(name, meta[:label_keys], label_values, value)
    end
  end
  buf
end

.render_histogram_series(buf, name, label_keys, label_values, snap_data) ⇒ Object 



258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
 
# File 'lib/hyperion/prometheus_exporter.rb', line 258

def render_histogram_series(buf, name, label_keys, label_values, snap_data)
  buckets = snap_data[:buckets]
  counts  = snap_data[:counts]
  label_keys ||= []
  label_keys = label_keys.size != label_values.size ? %w[method path status][0, label_values.size] : label_keys
  base_pairs = label_keys.zip(label_values)
  buckets.each_with_index do |edge, idx|
    pairs = base_pairs + [['le', format_float(edge)]]
    buf << "#{name}_bucket#{labels_block(pairs)} #{counts[idx]}\n"
  end
  pairs_inf = base_pairs + [['le', '+Inf']]
  buf << "#{name}_bucket#{labels_block(pairs_inf)} #{snap_data[:count]}\n"
  buf << "#{name}_sum#{labels_block(base_pairs)} #{snap_data[:sum]}\n"
  buf << "#{name}_count#{labels_block(base_pairs)} #{snap_data[:count]}\n"
end

.render_histograms(snap) ⇒ Object 



211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
 
# File 'lib/hyperion/prometheus_exporter.rb', line 211

def render_histograms(snap)
  buf = +''
  snap.each do |name, payload|
    meta   = payload[:meta]
    series = payload[:series]
    next if meta.nil?

    doc = METRIC_DOCS[name] || { help: 'Hyperion histogram', type: 'histogram' }
    buf << "# HELP #{name} #{doc[:help]}\n"
    buf << "# TYPE #{name} histogram\n"
    series.each do |label_values, snap_data|
      render_histogram_series(buf, name, meta[:label_keys], label_values, snap_data)
    end
  end
  buf
end

.render_labeled_counters(snap) ⇒ Object 



243
244
245
246
247
248
249
250
251
252
253
254
255
256
 
# File 'lib/hyperion/prometheus_exporter.rb', line 243

def render_labeled_counters(snap)
  buf = +''
  snap.each do |name, payload|
    series = payload[:series]
    meta   = payload[:meta] || { label_keys: [].freeze }
    doc = METRIC_DOCS[name] || { help: 'Hyperion labeled counter', type: 'counter' }
    buf << "# HELP #{name} #{doc[:help]}\n"
    buf << "# TYPE #{name} counter\n"
    series.each do |label_values, value|
      buf << render_labeled_value(name, meta[:label_keys], label_values, value)
    end
  end
  buf
end

.render_labeled_value(name, label_keys, label_values, value) ⇒ Object 



274
275
276
277
278
279
 
# File 'lib/hyperion/prometheus_exporter.rb', line 274

def render_labeled_value(name, label_keys, label_values, value)
  label_keys ||= []
  label_keys = label_keys.size == label_values.size ? label_keys : default_label_keys(label_values.size)
  pairs = label_keys.zip(label_values)
  "#{name}#{labels_block(pairs)} #{value}\n"
end