Class: Usps::Support::HealthController

Inherits:
ActionController::Base
  • Object
show all
Defined in:
app/controllers/usps/support/health_controller.rb

Overview

Base controller for external synthetic health probes (see the infrastructure repo's doc/application_health_probes.md). Unlike Rails' /up (boot-only) and nginx's static /instance-health-check, this exercises an app's critical dependencies and returns the sentinel "HEALTHCHECK_OK " only when every check passes, so a Route 53 HTTPS_STR_MATCH probe asserting that prefix flips the Statuspage component even with no organic traffic.

Host apps subclass this and declare which dependencies to probe with a CHECKS constant — a list of symbols, each naming a #check_ method:

class HealthController < Usps::Support::HealthController
CHECKS = %i[database redis hq_database imis].freeze
end

and route to it (get 'health' => 'health#show'). For a bespoke dependency, define a #check_ in the subclass (e.g. wrapping #check_http(url)) and add to CHECKS.

Checks run on a background thread, not the request thread: a per-process refresher updates a cached snapshot every health_refresh_interval seconds and #show serves that snapshot, so a slow or wedged dependency can't tie up the Passenger worker pool. Each check also runs under health_check_timeout, and a raise or timeout reads as a failed check (its error captured) so the action always responds. See doc/health_check_refresh_design.md.

Per-component results (ok, latency, error) are handed to Usps::Support::HealthDiagnostics once per refresh which — when enabled — publishes them to CloudWatch so a past failure can be attributed to a specific component after it has recovered.

Inherits ActionController::Base directly — NOT the host's ApplicationController — to bypass Devise auth, Pundit authorization, PaperTrail, and especially allow_browser, whose modern-browser gate would 406 Route 53's non-browser prober and fail the probe for the wrong reason.

Defined Under Namespace

Classes: CheckTimeout

Constant Summary collapse

SENTINEL_PREFIX =

rubocop:disable Rails/ApplicationController

'HEALTHCHECK_OK'
CHECKS =

Symbols naming the dependency checks to run; each invokes the matching #check_. Override per app by defining a CHECKS constant. Default probes the primary database only, which presumably every app needs.

%i[database].freeze

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.dependency_status ⇒ Object

Read-only, non-blocking view of the last completed round, for callers on a user request thread — specifically Usps::Support::DependencyGate, which uses it to fail a request fast rather than discovering the outage by blocking on the dead dependency itself:

{ hq_database: { ok: false, consecutive_failures: 4 }, database: { ok: true, ... } }

Runs no check and takes no lock; it reads whatever the background refresher last stored. Returns {} — "no opinion", so callers fail open — when no round has completed yet (cold worker) or the snapshot has aged past health_status_max_age, which means the refresher itself is wedged and its last verdict is no longer evidence about the dependency.



85
86
87
88
89
90
 
# File 'app/controllers/usps/support/health_controller.rb', line 85

def dependency_status
  snapshot = health_monitor.cached
  return {} unless snapshot && health_monitor.age_of(snapshot) <= health_status_max_age

  snapshot.data&.[](:status) || {}
end

.health_monitor ⇒ Object

One monitor per concrete subclass, per process. Built lazily so the background thread starts after Passenger forks its workers (threads do not survive fork).



65
66
67
68
69
70
71
72
73
 
# File 'app/controllers/usps/support/health_controller.rb', line 65

def health_monitor
  klass = self
  @health_monitor ||= HealthMonitor.new(
    interval: health_refresh_interval,
    max_age: health_snapshot_max_age,
    background: health_background_refresh,
    runner: -> { klass.new.send(:run_and_record) }
  )
end

.record_streaks(results) ⇒ Object

Consecutive failed rounds per check, updated once per refresh so a gate can require a sustained failure and ignore a single slow round. Called only from the monitor's runner, and the monitor serialises refreshes under its own mutex, so this needs no lock of its own.



95
96
97
98
99
 
# File 'app/controllers/usps/support/health_controller.rb', line 95

def record_streaks(results)
  @streaks ||= Hash.new(0)
  results.each { |name, ok| ok ? @streaks[name] = 0 : @streaks[name] += 1 }
  results.to_h { |name, ok| [name, { ok:, consecutive_failures: @streaks[name] }] }
end

.reset_health_monitor! ⇒ Object

Drops the memoized monitor (stopping its thread) and the failure streaks. For tests.



102
103
104
105
106
 
# File 'app/controllers/usps/support/health_controller.rb', line 102

def reset_health_monitor!
  @health_monitor&.stop
  @health_monitor = nil
  @streaks = nil
end

Instance Method Details

#show ⇒ Object 



109
110
111
112
113
114
 
# File 'app/controllers/usps/support/health_controller.rb', line 109

def show
  data = self.class.health_monitor.current
  return render_unavailable unless data

  render_health(data)
end