Class: HotCell::Supervisor

Inherits:
Object
  • Object
show all
Defined in:
lib/hot_cell/supervisor.rb

Overview

Accepts, queues, dispatches, times, kills, reaps, and cleans up. It never evaluates image data.

That last rule is mechanical rather than defensive: libvips starts its thread pool on the first evaluation and that pool does not survive fork, so a supervisor that has touched an image forks workers that deadlock forever. Reading a request line would be harmless — it is a control message from the trusted side on a bounded buffer — but the supervisor does not need to, and staying out of the request is what lets it dispatch a connection whose descriptors are still queued on it.

Dispatching rather than letting workers accept is what makes the rest work. The supervisor needs to own the accept anyway, for the queue, for queued_ms, and to answer capacity. It also means the supervisor knows when every worker started its current request, which is what the deadline needs.

Defined Under Namespace

Classes: Child, Pending

Constant Summary collapse

SUN_PATH_MAX =

A path longer than this fails to bind with an error that does not say so. Darwin allows four fewer bytes than Linux, and control.sock is the longer of the two names, so it overflows first.

RUBY_PLATFORM.include?("darwin") ? 104 : 108
SIGNAL_CAUSES =

The signals this cell can attribute to the request the worker was holding. XFSZ is that worker passing RLIMIT_FSIZE, and SEGV, ABRT and TRAP are how libvips and GLib die on their own allocation failures — libvips dereferences null after printing the correct diagnostic, and g_malloc aborts.

These three are the worker hitting its own per-worker RLIMIT_DATA, which is a property of the input this worker held, so the same bytes do it again and the verdict is permanent. Codes says a signal tells how a process died and never why, so a signal is transient by default — and that is not in conflict with a permanent verdict here, because aggregate pressure the worker did not cause arrives as SIGKILL, not as these. The two are different signals, and SIGKILL is excluded below.

SIGKILL is deliberately absent, and its absence is the point. The supervisor's own deadline kill is already named by killed_for, so a SIGKILL reaching this table came from somewhere this process cannot see: a cgroup OOM chosen on aggregate pressure, or a sibling worker, which shares a uid and is not prevented from signalling. Reading it as this request's memory condemned an input for someone else's pressure.

Anything not here is crashed, which is also where a worker that exited without a signal lands. They were two names, signal and crashed, for one amount of knowledge: the worker died and nothing says why. One name is honest about that.

{
  "XFSZ" => Codes::FSIZE,
  "SEGV" => Codes::MEMORY,
  "ABRT" => Codes::MEMORY,
  "TRAP" => Codes::MEMORY,
}.freeze
SOCKETS = 
[ "work.sock", "control.sock" ].freeze
PTRACE_SCOPE =

Request memory is protected by kernel.yama.ptrace_scope >= 1, and nothing else protects it. That is a host sysctl no container flag can supply.

"/proc/sys/kernel/yama/ptrace_scope"
CONTROL_BACKLOG =

Only to bound the list. The channel's whole value is answering when nothing else does, so this is set far above any real scrape rate rather than as a throttle.

64

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(directory:, workspace: nil, configuration: HotCell.configuration, log: Log.new, ptrace_scope_path: PTRACE_SCOPE) ⇒ Supervisor

Returns a new instance of Supervisor.



135
136
137
138
139
140
141
142
143
144
145
146
147
 
# File 'lib/hot_cell/supervisor.rb', line 135

def initialize(directory:, workspace: nil, configuration: HotCell.configuration, log: Log.new,
               ptrace_scope_path: PTRACE_SCOPE)
  @directory = directory
  @workspace = workspace || File.join(Dir.tmpdir, "hotcell-workspace")
  @configuration = configuration
  @log = log
  @ptrace_scope_path = ptrace_scope_path
  @children = {}
  @queue = []
  @control_pending = []
  @counters = Counters.new
  @stopping = false
end

Instance Attribute Details

#configuration ⇒ Object (readonly)

Returns the value of attribute configuration.



133
134
135
 
# File 'lib/hot_cell/supervisor.rb', line 133

def configuration
  @configuration
end

#counters ⇒ Object (readonly)

Returns the value of attribute counters.



133
134
135
 
# File 'lib/hot_cell/supervisor.rb', line 133

def counters
  @counters
end

#directory ⇒ Object (readonly)

Returns the value of attribute directory.



133
134
135
 
# File 'lib/hot_cell/supervisor.rb', line 133

def directory
  @directory
end

#log ⇒ Object (readonly)

Returns the value of attribute log.



133
134
135
 
# File 'lib/hot_cell/supervisor.rb', line 133

def log
  @log
end

#workspace ⇒ Object (readonly)

Returns the value of attribute workspace.



133
134
135
 
# File 'lib/hot_cell/supervisor.rb', line 133

def workspace
  @workspace
end

Instance Method Details

#boot ⇒ Object 



149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
 
# File 'lib/hot_cell/supervisor.rb', line 149

def boot
  verify_socket_paths!
  verify_limits!
  verify_ptrace_scope!
  prepare_directories
  preload
  @work = listen "work.sock"
  @control = listen "control.sock"
  @control_handler = Control.new(configuration: configuration, counters: counters)
  trap_signals

  log.write "cell.boot", pid: Process.pid, directory: directory, operations: Registry.names,
                         configuration: configuration.to_h
  self
end

#run ⇒ Object 



165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
 
# File 'lib/hot_cell/supervisor.rb', line 165

def run
  until stopped?
    readable, = IO.select(sources, nil, nil, wait_for)
    Array(readable).each { |source| handle source }

    enforce_deadlines
    enforce_retirements
    expire_queue
    expire_control
    retire_idle if @stopping
    pump
  end
ensure
  shutdown
end