Class: Hyperion::Server

Inherits:
Object
  • Object
show all
Defined in:
lib/hyperion/server.rb,
lib/hyperion/server/route_table.rb,
lib/hyperion/server/connection_loop.rb

Overview

Phase 2a server: bind a TCPServer, accept connections, schedule each on its own fiber via Async. Multiple in-flight requests run concurrently on a single OS thread. Keep-alive is still off — connection closes after one request (Phase 2b will add keep-alive).

Phase 7 (scoped): when ‘tls:` is supplied, wrap the listener in an OpenSSL::SSL::SSLServer with ALPN advertising `h2` + `http/1.1`. After the handshake, dispatch on the negotiated protocol — http/1.1 goes through Connection (real path); h2 goes to Http2Handler (505 stub until Phase 8).

Defined Under Namespace

Modules: ConnectionLoop Classes: RouteTable

Constant Summary collapse

DEFAULT_READ_TIMEOUT_SECONDS = 
30
DEFAULT_THREAD_COUNT = 
5
REJECT_503 =

Pre-built minimal 503 response for the backpressure path. We bypass ResponseWriter / Rack entirely — no env build, no app dispatch, no access-log line. The bytes are frozen and reused across every rejection so the overload path stays allocation-free. Body is JSON so JSON-only API consumers don’t have to special-case the format.

lambda {
  body = +%({"error":"server_busy","retry_after_seconds":1}\n)
  body.force_encoding(Encoding::ASCII_8BIT)
  head = +"HTTP/1.1 503 Service Unavailable\r\n" \
          "content-type: application/json\r\n" \
          "content-length: #{body.bytesize}\r\n" \
          "retry-after: 1\r\n" \
          "connection: close\r\n" \
          "\r\n"
  head.force_encoding(Encoding::ASCII_8BIT)
  (head + body).freeze
}.call

Class Attribute Summary collapse

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(app:, host: '127.0.0.1', port: 9292, read_timeout: DEFAULT_READ_TIMEOUT_SECONDS, tls: nil, thread_count: DEFAULT_THREAD_COUNT, max_pending: nil, max_request_read_seconds: 60, h2_settings: nil, async_io: nil, runtime: nil, accept_fibers_per_worker: 1, h2_max_total_streams: nil, admin_listener_port: nil, admin_listener_host: '127.0.0.1', admin_token: nil, tls_session_cache_size: TLS::DEFAULT_SESSION_CACHE_SIZE, tls_ktls: :auto, io_uring: :off, max_in_flight_per_conn: nil, tls_handshake_rate_limit: :unlimited, route_table: nil, preload_static_dirs: nil) ⇒ Server

1.7.0 added kwargs (all default to current behaviour):

* `runtime:`             — `Hyperion::Runtime` instance (default
                            `Runtime.default`). Threaded through to
                            every per-connection / per-stream code
                            path so per-server metrics/logger
                            isolation works.
* `accept_fibers_per_worker:` — Integer, default 1. When > 1 and the
                            accept loop is async-wrapped, spawn N
                            accept fibers that race on the same
                            listening fd. Linear scaling on
                            `:reuseport` (Linux); Darwin honours the
                            knob silently with no scaling benefit
                            (RFC §5 Q5).
* `h2_max_total_streams:` — Integer or nil (default nil). Process-
                            wide cap on simultaneously-open h2
                            streams across all connections. nil
                            disables (current behaviour); set to
                            opt into RFC A7 admission control.
* `admin_listener_port:`  — Integer or nil (default nil). When set,
                            spawn a sibling HTTP listener on
                            `127.0.0.1:<port>` that serves only
                            `/-/quit` and `/-/metrics`. nil keeps
                            admin mounted in-app (current shape).



189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
 
# File 'lib/hyperion/server.rb', line 189

def initialize(app:, host: '127.0.0.1', port: 9292, read_timeout: DEFAULT_READ_TIMEOUT_SECONDS,
               tls: nil, thread_count: DEFAULT_THREAD_COUNT, max_pending: nil,
               max_request_read_seconds: 60, h2_settings: nil, async_io: nil,
               runtime: nil, accept_fibers_per_worker: 1,
               h2_max_total_streams: nil, admin_listener_port: nil,
               admin_listener_host: '127.0.0.1', admin_token: nil,
               tls_session_cache_size: TLS::DEFAULT_SESSION_CACHE_SIZE,
               tls_ktls: :auto,
               io_uring: :off,
               max_in_flight_per_conn: nil,
               tls_handshake_rate_limit: :unlimited,
               route_table: nil,
               preload_static_dirs: nil)
  validate_async_io!(async_io)
  @host                     = host
  @port                     = port
  @app                      = app
  @read_timeout             = read_timeout
  @tls                      = tls
  @thread_count             = thread_count
  @max_pending              = max_pending
  @max_request_read_seconds = max_request_read_seconds
  @h2_settings              = h2_settings
  @async_io                 = async_io
  # `@explicit_runtime` toggles between 1.7.0 isolation (an
  # explicitly-passed Runtime) and 1.6.x compat (legacy module-level
  # accessors honoured for stub seams). All record_dispatch /
  # reject_connection / log lines route through `runtime_metrics` /
  # `runtime_logger` helpers below.
  @runtime                  = runtime || Hyperion::Runtime.default
  @explicit_runtime         = !runtime.nil?
  @accept_fibers_per_worker = [accept_fibers_per_worker.to_i, 1].max
  # 2.0: `h2_max_total_streams` is normally a positive integer (the
  # default-flipped cap from `Config#finalize!`) or nil (operator
  # opted out via `h2.max_total_streams :unbounded`). Defensive
  # branch: treat the `:auto` / `:unbounded` sentinels as "no cap"
  # if a caller bypasses Config and constructs Server directly.
  @h2_admission             = if h2_max_total_streams.is_a?(Integer) && h2_max_total_streams.positive?
                                Hyperion::H2Admission.new(max_total_streams: h2_max_total_streams)
                              end
  @admin_listener_port      = admin_listener_port
  @admin_listener_host      = admin_listener_host
  @admin_token              = admin_token
  @admin_listener           = nil
  @thread_pool              = nil
  @stopped                  = false
  @tls_session_cache_size   = tls_session_cache_size
  @tls_ktls                 = tls_ktls
  @ktls_logged              = false
  # 2.3-A: resolve the io_uring accept policy. `:off` (the 2.3.0
  # default) skips the resolve step entirely so hosts without the
  # cdylib don't trigger any Fiddle.dlopen probe at boot.
  # Workers don't share rings across fork — each child opens its
  # own ring lazily on first use inside `run_accept_fiber`.
  @io_uring_policy          = io_uring
  @io_uring_active          = io_uring != :off && Hyperion::IOUring.resolve_policy!(io_uring)
  log_io_uring_state_once
  # 2.3-B: per-conn fairness cap (validated/finalized upstream by
  # `Config#finalize!`; constructor accepts the resolved value, not
  # a sentinel). nil = no cap (default). The cap propagates to
  # every Connection the ThreadPool's `:connection` worker builds.
  @max_in_flight_per_conn   = max_in_flight_per_conn
  # 2.3-B: TLS handshake CPU throttle. One limiter per worker
  # (per-Server). `:unlimited` short-circuits every `acquire_token!`
  # to true so the hot path stays branchless. Built eagerly so
  # bench harnesses can introspect via `server.tls_handshake_limiter`.
  @tls_handshake_limiter    = Hyperion::TLS::HandshakeRateLimiter.new(tls_handshake_rate_limit)
  # 2.10-D: per-instance route table (defaults to the class-level
  # singleton).  Tests can inject a fresh table to isolate
  # registrations from other examples.
  @route_table              = route_table || Hyperion::Server.route_table
  # 2.10-E: list of `{path:, immutable:}` entries the worker warms
  # into `Hyperion::Http::PageCache` at boot. Resolved by
  # `Config#resolved_preload_static_dirs` and threaded through
  # Master → Worker → Server. nil/[] = no preload (1.x cold-cache
  # behaviour).
  @preload_static_dirs      = preload_static_dirs
  @preloaded                = false
end

Class Attribute Details

.route_table ⇒ Object

2.10-D — process-wide direct-dispatch route table. Operators register routes via ‘Hyperion::Server.handle(:GET, ’/hello’, handler)‘ BEFORE forking workers; each forked worker inherits the populated table via copy-on-write. Per-Server instances can override by passing `route_table:` to the constructor (a test seam — production code uses the class singleton).

Lazily initialized so ‘require ’hyperion’‘ itself doesn’t pay the allocation when the operator never registers a direct route (the common 1.x deployment).



56
57
58
 
# File 'lib/hyperion/server.rb', line 56

def self.route_table
  @route_table ||= RouteTable.new
end

Instance Attribute Details

#host ⇒ Object (readonly)

Returns the value of attribute host.



44
45
46
 
# File 'lib/hyperion/server.rb', line 44

def host
  @host
end

#port ⇒ Object (readonly)

Returns the value of attribute port.



44
45
46
 
# File 'lib/hyperion/server.rb', line 44

def port
  @port
end

#route_table ⇒ Object (readonly)

2.10-D — read-only handle to the per-instance route table. Connection#serve consults this after parse to decide whether to engage the direct-dispatch fast path. Defaults to the process-wide ‘Hyperion::Server.route_table` singleton.



276
277
278
 
# File 'lib/hyperion/server.rb', line 276

def route_table
  @route_table
end

#runtime ⇒ Object (readonly)

Returns the value of attribute runtime.



44
45
46
 
# File 'lib/hyperion/server.rb', line 44

def runtime
  @runtime
end

#ssl_ctx ⇒ Object (readonly)

Read-only handle to the per-worker SSL context (nil when the listener is plain TCP). Exposed so the worker can call ‘Hyperion::TLS.rotate!(server.ssl_context)` from its SIGUSR2 handler without reaching into Server internals.



282
283
284
 
# File 'lib/hyperion/server.rb', line 282

def ssl_ctx
  @ssl_ctx
end

#tls_handshake_limiter ⇒ Object (readonly)

Read-only handle for tests + bench harness introspection.



270
271
272
 
# File 'lib/hyperion/server.rb', line 270

def tls_handshake_limiter
  @tls_handshake_limiter
end

Class Method Details

.handle(method_sym, path, handler = nil, &block) ⇒ Object

2.10-D — register a direct-dispatch handler. Bypasses the Rack adapter on hit: when a request whose method + path matches this entry arrives, ‘Connection#serve` skips the env-hash build, the middleware chain, and the body-iteration loop —the handler is called directly with a `Hyperion::Request` value object.

‘method_sym` is one of `:GET`, `:POST`, `:PUT`, `:DELETE`, `:HEAD`, `:PATCH`, `:OPTIONS` (case-insensitive — `:get` works too). `path` is an exact-match String (regex / glob routing is intentionally out of scope; future work). `handler` is any object responding to `#call(request)` that returns a `[status, headers, body]` Rack tuple.

Lifecycle hooks (‘Runtime#on_request_start` / `on_request_end`) still fire on direct routes so NewRelic / AppSignal / OpenTelemetry instrumentation works regardless of dispatch shape.

On a non-match (any path / method not registered here) the request falls through to the regular Rack adapter dispatch — existing behaviour for un-handled routes is unchanged.

Raises:

  • (ArgumentError) 


91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
 
# File 'lib/hyperion/server.rb', line 91

def self.handle(method_sym, path, handler = nil, &block)
  raise ArgumentError, 'pass a handler OR a block, not both' if handler && block
  raise ArgumentError, 'must pass a handler or block' if handler.nil? && block.nil?

  if block
    # 2.14-A — block form: `Server.handle(:GET, '/x') { |env| ... }`.
    # Wraps the block in a `DynamicBlockEntry` so the C accept loop
    # (when engaged) can recognise the entry and dispatch via the
    # registered C-loop helper. The block receives a Rack env hash
    # — same shape Rack apps see — and must return a `[status,
    # headers, body]` triple per the Rack spec.
    method_key = method_sym.to_s.upcase.to_sym
    entry = RouteTable::DynamicBlockEntry.new(method_key, path.dup.freeze, block).freeze
    route_table.register(method_sym, path, entry)
    entry
  else
    # Legacy 2.10-D handler form: `handler#call(request)` returning
    # a `[status, headers, body]` triple. The C accept loop does
    # NOT engage on these — they fall through to the Connection
    # path so the Hyperion::Request shape contract holds.
    route_table.register(method_sym, path, handler)
  end
end

.handle_static(method_sym, path, body_bytes, content_type: 'text/plain') ⇒ Object

2.10-D — register a direct-dispatch route whose response is FULLY known at registration time. The full HTTP/1.1 response buffer (status line + Content-Type + Content-Length + body) is built ONCE here and stashed in a ‘RouteTable::StaticEntry`; on hit, `Connection#serve` issues a single `socket.write` of the pre-built bytes — no header build, no body iteration, zero per-request allocation past the Connection ivars.

Mirrors agoo’s optimal hello-world path. ‘body_bytes` is the response body (frozen automatically); `content_type` defaults to `text/plain`. Returns the registered `StaticEntry` for inspection.

Raises:

  • (ArgumentError) 


127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
 
# File 'lib/hyperion/server.rb', line 127

def self.handle_static(method_sym, path, body_bytes, content_type: 'text/plain')
  raise ArgumentError, 'body_bytes must be a String' unless body_bytes.is_a?(String)
  raise ArgumentError, 'content_type must be a String' unless content_type.is_a?(String)

  body = body_bytes.dup.b.freeze
  head = +"HTTP/1.1 200 OK\r\n" \
          "content-type: #{content_type}\r\n" \
          "content-length: #{body.bytesize}\r\n" \
          "\r\n"
  head.force_encoding(Encoding::ASCII_8BIT)
  buffer = (head + body).freeze

  method_key = method_sym.to_s.upcase.to_sym
  # 2.10-F — record the headers prefix length on the StaticEntry
  # struct so HEAD-method writes can serve a headers-only prefix.
  entry = RouteTable::StaticEntry.new(method_key, path.dup.freeze, buffer, head.bytesize).freeze
  # 2.10-F — register the entry DIRECTLY (StaticEntry responds to
  # `#call`) instead of wrapping it in a closure, so the dispatch
  # path can branch on `is_a?(StaticEntry)` BEFORE invoking the
  # handler — that's what unlocks the C-ext fast path.
  route_table.register(method_sym, path, entry)
  # 2.10-F — also register HEAD for any GET registration.  HTTP
  # mandates HEAD-on-a-GET-resource, and the C fast path strips
  # the body bytes for HEAD requests inside `serve_request`.
  # Idiomatic for static-asset routes (every CDN-shaped GET URL
  # MUST also answer HEAD with the same headers).  No-op on a
  # POST/PUT/etc. registration — those don't get a HEAD twin.
  route_table.register(:HEAD, path, entry) if method_key == :GET
  # 2.10-F — fold the prebuilt response into the C-side PageCache so
  # `PageCache.serve_request` can write it without ever crossing
  # back into Ruby.  Best-effort: if the C ext isn't available
  # (JRuby / TruffleRuby), the dispatcher silently falls back to
  # the Ruby `socket.write` path that's been there since 2.10-D.
  if defined?(::Hyperion::Http::PageCache) && ::Hyperion::Http::PageCache.respond_to?(:register_prebuilt)
    ::Hyperion::Http::PageCache.register_prebuilt(path, buffer, body.bytesize)
  end
  entry
end

Instance Method Details

#adopt_listener(sock) ⇒ Object

Phase 3: workers pass in a pre-bound, SO_REUSEPORT-set socket built by Hyperion::Worker. Bypasses #listen but keeps the rest of the accept loop intact since Socket and TCPServer both quack #accept_nonblock.

Phase 8: when ‘tls:` was supplied to the constructor, also build the SSL context here so the accept loop can wrap incoming connections. Each worker builds its own context — they don’t share state.



322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
 
# File 'lib/hyperion/server.rb', line 322

def adopt_listener(sock)
  @server = sock
  @tcp_server = sock
  @port = case sock
          when ::TCPServer
            sock.addr[1]
          else
            sock.local_address.ip_port
          end
  if @tls
    @ssl_ctx = TLS.context(cert: @tls[:cert], key: @tls[:key], chain: @tls[:chain],
                           session_cache_size: @tls_session_cache_size,
                           ktls: @tls_ktls)
  end
  self
end

#listen ⇒ Object 



296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
 
# File 'lib/hyperion/server.rb', line 296

def listen
  tcp = ::TCPServer.new(@host, @port)
  @port = tcp.addr[1]

  if @tls
    @ssl_ctx = TLS.context(cert: @tls[:cert], key: @tls[:key], chain: @tls[:chain],
                           session_cache_size: @tls_session_cache_size,
                           ktls: @tls_ktls)
    ssl_server = ::OpenSSL::SSL::SSLServer.new(tcp, @ssl_ctx)
    ssl_server.start_immediately = false
    @server = ssl_server
    @tcp_server = tcp
  else
    @server = tcp
    @tcp_server = tcp
  end
  self
end

#preload_static!(logger: runtime_logger) ⇒ Object

2.10-E — Walk every configured preload directory, populate ‘Hyperion::Http::PageCache`, and mark every entry immutable when asked. Called from `start` once per worker. Idempotent — second call is a no-op so test harnesses + Worker respawn paths don’t re-walk the tree.

‘logger` is exposed as a kwarg purely for the spec suite; production callers omit it and the runtime logger is used.



545
546
547
548
549
550
551
552
553
 
# File 'lib/hyperion/server.rb', line 545

def preload_static!(logger: runtime_logger)
  return 0 if @preloaded

  @preloaded = true
  entries = @preload_static_dirs
  return 0 if entries.nil? || entries.empty?

  Hyperion::StaticPreload.run(entries, logger: logger)
end

#run_one ⇒ Object 



339
340
341
342
343
344
345
346
347
 
# File 'lib/hyperion/server.rb', line 339

def run_one
  Async do
    socket = blocking_accept
    next unless socket

    apply_timeout(socket)
    dispatch(socket)
  end.wait
end

#start ⇒ Object 



349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
 
# File 'lib/hyperion/server.rb', line 349

def start
  listen unless @server
  # 2.10-E: warm the page cache before any request can land. Idempotent
  # via `@preloaded`, so repeated `start` calls (test harnesses,
  # Worker#run respawn) don't re-walk the tree. Runs after `listen`
  # (so `@server` exists for the operator's introspection hooks if any
  # future runtime fires off boot-side instrumentation) but before the
  # accept loop fires up — first request hits warm cache.
  preload_static!
  if @thread_count.positive?
    @thread_pool = ThreadPool.new(size: @thread_count, max_pending: @max_pending,
                                  max_in_flight_per_conn: @max_in_flight_per_conn,
                                  route_table: @route_table)
  end
  maybe_start_admin_listener

  if @tls || @async_io
    # TLS path: ALPN may pick `h2`, and h2 spawns one fiber per stream
    # inside Http2Handler. Keep the Async wrapper so the scheduler is
    # available for those fibers and for handshake yields. Plain
    # HTTP/1.1-over-TLS dispatch is also handled inline on the calling
    # fiber by default in 1.4.0+ (see #dispatch) — fiber-cooperative
    # libraries (async-pg, async-redis) work without --async-io.
    #
    # async_io: true: operator opt-in for plain HTTP/1.1. The Async wrap
    # is required when callers want fiber cooperative I/O — e.g.
    # `hyperion-async-pg` yielding while a Postgres query is in flight.
    # Pays ~5% throughput vs the raw-loop fast path; in exchange one
    # OS thread can serve N concurrent in-flight DB queries instead of 1.
    start_async_loop
  else
    # Plain HTTP/1.1, async_io: nil (default with no TLS) or
    # async_io: false (explicit opt-out): the worker thread owns each
    # connection for its lifetime, so the Async wrapper adds zero value
    # (no fibers ever run on this loop's task). Skip it — pure
    # IO.select + accept_nonblock shaves measurable overhead off the
    # accept hot path.
    start_raw_loop
  end
ensure
  @thread_pool&.shutdown
  @admin_listener&.stop
end

#stop ⇒ Object

2.14-B — graceful stop sequence.

Pre-2.14-B this was three lines: flip the Ruby ‘@stopped` flag, `close()` the listener, drop the references. That was enough for the Ruby/Async accept loops on every kernel — those poll `@stopped` every 100 ms via the `IO.select` timeout in `accept_or_nil` and exit at the next tick. It was NOT enough for the C accept loop introduced by 2.12-C: that loop calls a blocking `accept(2)` with the GVL released and only checks `hyp_cl_stop` between accepts. On Linux ≥ 6.x, calling `close()` on a listening socket from one thread does NOT interrupt another thread that is currently parked in `accept(2)` on that same fd — so the C loop stayed parked until a real connection arrived. SIGTERM-driven graceful shutdown then hung until the master’s ‘graceful_timeout` (default 30 s) expired and SIGKILL fired. See CHANGELOG ### 2.13-C for the full discovery story.

Fix surface: only the C accept loop needs the wake-connect dance. The wake gate (‘wake_required?`) keeps the change surgical: TLS, async-IO, and thread-pool servers see the same close-then-drop sequence they had pre-2.14-B; only the C-loop server pays the burst cost. Wiring the wake into the Async path would be unnecessary (it polls @stopped) and would introduce a close-vs-`IO.select`-EBADF race on macOS kqueue.

Order rationale (C-loop case).

  1. The wake-connect dial happens BEFORE ‘close_listeners` so THIS process’s listener fd is still in the SO_REUSEPORT pool when the kernel hashes the SYN. Closing first would drop us from the pool — every dial would hash to a sibling worker (in ‘:reuseport` cluster mode) and never reach our own parked accept thread.

  2. The burst (‘WAKE_CONNECT_BURST` dials) drives the miss probability down for the SO_REUSEPORT-distributes-unevenly case. Single-server / `:share` cluster mode (Darwin/BSD) just sees K extra zero-byte connects — cheap.

  3. ‘close_listeners` runs last as a belt-and-braces close on macOS / *BSD where the close-on-accept-wake guarantee still holds, and to release the bound port to the OS promptly.

Idempotent: a second ‘stop` call is a no-op — `wake_target` returns `[nil, nil]` once the listener references are nilled, and `close_listeners` swallows the EBADF.



437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
 
# File 'lib/hyperion/server.rb', line 437

def stop
  @stopped = true
  if wake_required?
    # C-loop path: flip the C-side flag, dial the wake-connect
    # burst, THEN close. The wake makes any thread parked in
    # `accept(2)` return; the loop checks the flag, exits cleanly.
    stop_c_accept_loop
    host, port = wake_target
    ConnectionLoop.wake_listener(host, port, count: ConnectionLoop::WAKE_CONNECT_BURST) \
      if host && port
  end
  # Pre-2.14-B `close` path. For TLS / async-IO / thread-pool
  # servers this is the entire stop sequence and matches the
  # behaviour the spec suite (and operators) have been observing
  # since 1.0 — the wake-connect dance is a no-op for them and
  # has been deliberately gated out via `wake_required?`.
  close_listeners
  nil
end