Module: Hyperion::IOUring

Defined in:
lib/hyperion/io_uring.rb

Overview

2.3-A — io_uring accept on Linux 5.6+ (opt-in).

The biggest unmovable bottleneck below the GVL on the plaintext h1 path is the kernel accept loop: every accept costs accept_nonblock + IO.select on the EAGAIN edge (two syscalls per accepted connection under burst). io_uring lets us submit accept SQEs and reap CQEs in one syscall, with the kernel batching multiple accepts in a single CQE drain when connections arrive faster than the fiber can consume them.

## Surface

Hyperion::IOUring.supported?   # bool — Linux ≥ 5.6 + cdylib loaded
                               #        + runtime probe succeeds
Hyperion::IOUring::Ring.new(queue_depth: 256)
                               # per-fiber ring; #accept(fd) → fd
                               # or :wouldblock; #close releases
                               # the ring's SQ/CQ memory.

## Per-fiber, NEVER per-process or per-thread

io_uring under fork+threads has known sharp edges:

* Submission queue is process-shared by default — under fork, the
  parent's outstanding SQEs leak into the child's CQ.
* IORING_SETUP_SQPOLL kernel thread does not survive fork.
* Threads sharing a ring need IORING_SETUP_SINGLE_ISSUER + careful
  submission discipline.

Hyperion’s safe pattern, matching the fiber-per-conn architecture:

* One ring per fiber that needs it (the accept fiber, optionally
  per-connection read fibers in a future phase).
* Ring is opened lazily on first use:
    Fiber[:hyperion_io_uring] ||=
      Hyperion::IOUring::Ring.new(queue_depth: 256)
* Ring is closed when the fiber exits.
* Workers don't share rings across fork — each child opens its own.

## Default off in 2.3.0

Mirrors the 2.2.0 fix-B HYPERION_H2_NATIVE_HPACK pattern: ship the plumbing in 2.3.0 with the default OFF, give operators an env-var to A/B (HYPERION_IO_URING=on,auto), flip the default to :auto in 2.4 only after 6 months of soak. io_uring code in production has too many sharp edges to default-on without field validation.

Defined Under Namespace

Classes: HotpathRing, Ring, Unsupported

Constant Summary collapse

EXPECTED_ABI = 
2
MIN_LINUX_KERNEL =

Linux 5.6 stabilized IORING_OP_ACCEPT (commit 17f2fe35d080, mainlined Mar 2020). 5.5 had a buggy precursor that the io-uring crate refuses to use. We gate on 5.6 to match the crate’s stance.

[5, 6].freeze

Class Method Summary collapse

Class Method Details

.candidate_paths ⇒ Object 



453
454
455
456
457
458
459
460
461
462
463
464
465
466
 
# File 'lib/hyperion/io_uring.rb', line 453

def candidate_paths
  gem_lib = File.expand_path('../hyperion_io_uring', __dir__)
  ext_target = File.expand_path('../../ext/hyperion_io_uring/target/release', __dir__)
  # Prefer the platform-native extension first so Fiddle.dlopen doesn't
  # accidentally pick up a cross-platform binary (e.g. a macOS .dylib
  # rsync'd into a Linux lib dir) which causes an ArgumentError on read.
  native, other = linux? ? %w[.so .dylib] : %w[.dylib .so]
  [
    File.join(gem_lib,    "libhyperion_io_uring#{native}"),
    File.join(ext_target, "libhyperion_io_uring#{native}"),
    File.join(gem_lib,    "libhyperion_io_uring#{other}"),
    File.join(ext_target, "libhyperion_io_uring#{other}"),
  ]
end

.compute_hotpath_supported ⇒ Object 



290
291
292
293
294
295
296
 
# File 'lib/hyperion/io_uring.rb', line 290

def compute_hotpath_supported
  return false unless supported?
  return false unless @hotpath_supported_fn
  @hotpath_supported_fn.call.zero?
rescue StandardError
  false
end

.compute_supported ⇒ Object

—- Internal: feature gate —-



300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
 
# File 'lib/hyperion/io_uring.rb', line 300

def compute_supported
  # Gate 1: Linux only. macOS/BSD don't have io_uring.
  return false unless linux?

  # Gate 2: Kernel ≥ 5.6.
  return false unless kernel_supports_io_uring?

  # Gate 3: cdylib loaded.
  load!
  return false unless @lib

  # Gate 4: runtime probe — try to set up a tiny ring. Catches
  # sandboxed containers (seccomp blocking io_uring_setup,
  # locked-down environments returning -EPERM, kernels with
  # io_uring disabled via /proc/sys/kernel/io_uring_disabled).
  rc = @probe_fn.call
  rc.zero?
rescue StandardError
  false
end

.hotpath_copy_buffer(ptr, buf_id, len, out_ptr, out_cap) ⇒ Object

Plan #2 Task 2.3.4 — copy ‘len` bytes from kernel buffer-ring slot `buf_id` into the caller-supplied output buffer. Returns the number of bytes written, or a negative errno on failure.



525
526
527
 
# File 'lib/hyperion/io_uring.rb', line 525

def hotpath_copy_buffer(ptr, buf_id, len, out_ptr, out_cap)
  @hotpath_copy_buffer_fn.call(ptr, buf_id, len, out_ptr, out_cap)
end

.hotpath_force_unhealthy(ptr) ⇒ Object 



514
515
516
 
# File 'lib/hyperion/io_uring.rb', line 514

def hotpath_force_unhealthy(ptr)
  @hotpath_force_unhealthy_fn.call(ptr)
end

.hotpath_is_healthy(ptr) ⇒ Object 



518
519
520
 
# File 'lib/hyperion/io_uring.rb', line 518

def hotpath_is_healthy(ptr)
  @hotpath_is_healthy_fn.call(ptr)
end

.hotpath_release_buf(ptr, buf_id) ⇒ Object 



510
511
512
 
# File 'lib/hyperion/io_uring.rb', line 510

def hotpath_release_buf(ptr, buf_id)
  @hotpath_release_buf_fn.call(ptr, buf_id)
end

.hotpath_ring_free(ptr) ⇒ Object 



492
 
# File 'lib/hyperion/io_uring.rb', line 492

def hotpath_ring_free(ptr); @hotpath_ring_free_fn.call(ptr); end

.hotpath_ring_new(qd, n_bufs, buf_size) ⇒ Object 



487
488
489
490
 
# File 'lib/hyperion/io_uring.rb', line 487

def hotpath_ring_new(qd, n_bufs, buf_size)
  ptr = @hotpath_ring_new_fn.call(qd, n_bufs, buf_size)
  ptr.null? ? nil : ptr
end

.hotpath_submit_accept(ptr, fd) ⇒ Object 



494
495
496
 
# File 'lib/hyperion/io_uring.rb', line 494

def hotpath_submit_accept(ptr, fd)
  @hotpath_submit_accept_fn.call(ptr, fd)
end

.hotpath_submit_recv(ptr, fd) ⇒ Object 



498
499
500
 
# File 'lib/hyperion/io_uring.rb', line 498

def hotpath_submit_recv(ptr, fd)
  @hotpath_submit_recv_fn.call(ptr, fd)
end

.hotpath_submit_send(ptr, fd, iov, n) ⇒ Object 



502
503
504
 
# File 'lib/hyperion/io_uring.rb', line 502

def hotpath_submit_send(ptr, fd, iov, n)
  @hotpath_submit_send_fn.call(ptr, fd, iov, n)
end

.hotpath_supported? ⇒ Boolean

Plan #2 — true when (a) accept-only ‘supported?` true, AND (b) the kernel actually accepts pbuf-ring registration (5.19+). NOTE: this does NOT probe RecvMulti (6.0+); the Ruby caller must treat the first recv CQE failure as a feature-unavailable signal — see hotpath.rs’s hyperion_io_uring_hotpath_supported doc comment for the full kernel-version matrix.

Returns:

  • (Boolean) 


285
286
287
288
 
# File 'lib/hyperion/io_uring.rb', line 285

def hotpath_supported?
  return @hotpath_supported unless @hotpath_supported.nil?
  @hotpath_supported = compute_hotpath_supported
end

.hotpath_wait(ptr, mc, t, out, cap) ⇒ Object 



506
507
508
 
# File 'lib/hyperion/io_uring.rb', line 506

def hotpath_wait(ptr, mc, t, out, cap)
  @hotpath_wait_fn.call(ptr, mc, t, out, cap)
end

.kernel_supports_io_uring? ⇒ Boolean

Returns:

  • (Boolean) 


327
328
329
330
331
332
333
334
335
336
 
# File 'lib/hyperion/io_uring.rb', line 327

def kernel_supports_io_uring?
  return false unless linux?

  release = parse_kernel_release
  return false unless release

  major, minor = release
  min_major, min_minor = MIN_LINUX_KERNEL
  major > min_major || (major == min_major && minor >= min_minor)
end

.linux? ⇒ Boolean

Returns:

  • (Boolean) 


321
322
323
324
325
 
# File 'lib/hyperion/io_uring.rb', line 321

def linux?
  Etc.uname[:sysname] == 'Linux'
rescue StandardError
  false
end

.load! ⇒ Object

—- Internal: Fiddle loader —-



356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
 
# File 'lib/hyperion/io_uring.rb', line 356

def load!
  return @lib if defined?(@lib) && !@lib.nil?

  path = candidate_paths.find { |p| File.exist?(p) }
  unless path
    @lib = nil
    return nil
  end

  @lib = Fiddle.dlopen(path)
  @abi_fn = Fiddle::Function.new(@lib['hyperion_io_uring_abi_version'],
                                 [], Fiddle::TYPE_INT)
  abi = @abi_fn.call
  if abi != EXPECTED_ABI
    warn "[hyperion] IOUring ABI mismatch (got #{abi}, expected #{EXPECTED_ABI}); falling back"
    @lib = nil
    return nil
  end

  @probe_fn = Fiddle::Function.new(@lib['hyperion_io_uring_probe'],
                                   [], Fiddle::TYPE_INT)
  @ring_new_fn = Fiddle::Function.new(@lib['hyperion_io_uring_ring_new'],
                                      [Fiddle::TYPE_INT], Fiddle::TYPE_VOIDP)
  @ring_free_fn = Fiddle::Function.new(@lib['hyperion_io_uring_ring_free'],
                                       [Fiddle::TYPE_VOIDP], Fiddle::TYPE_VOID)
  @accept_fn = Fiddle::Function.new(@lib['hyperion_io_uring_accept'],
                                    [Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT, Fiddle::TYPE_VOIDP],
                                    Fiddle::TYPE_INT)
  @read_fn = Fiddle::Function.new(@lib['hyperion_io_uring_read'],
                                  [Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT,
                                   Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT,
                                   Fiddle::TYPE_VOIDP],
                                  Fiddle::TYPE_INT)

  # Plan #2 — hotpath surface (5.19+ for PBUF_RING, 6.0+ for RecvMulti).
  @hotpath_supported_fn = Fiddle::Function.new(
    @lib['hyperion_io_uring_hotpath_supported'], [], Fiddle::TYPE_INT
  )
  @hotpath_ring_new_fn = Fiddle::Function.new(
    @lib['hyperion_io_uring_hotpath_ring_new'],
    [Fiddle::TYPE_INT, Fiddle::TYPE_SHORT, Fiddle::TYPE_INT],
    Fiddle::TYPE_VOIDP
  )
  @hotpath_ring_free_fn = Fiddle::Function.new(
    @lib['hyperion_io_uring_hotpath_ring_free'],
    [Fiddle::TYPE_VOIDP], Fiddle::TYPE_VOID
  )
  @hotpath_submit_accept_fn = Fiddle::Function.new(
    @lib['hyperion_io_uring_hotpath_submit_accept_multishot'],
    [Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT], Fiddle::TYPE_INT
  )
  @hotpath_submit_recv_fn = Fiddle::Function.new(
    @lib['hyperion_io_uring_hotpath_submit_recv_multishot'],
    [Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT], Fiddle::TYPE_INT
  )
  @hotpath_submit_send_fn = Fiddle::Function.new(
    @lib['hyperion_io_uring_hotpath_submit_send'],
    [Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT, Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT],
    Fiddle::TYPE_INT
  )
  @hotpath_wait_fn = Fiddle::Function.new(
    @lib['hyperion_io_uring_hotpath_wait_completions'],
    [Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT, Fiddle::TYPE_INT,
     Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT],
    Fiddle::TYPE_INT
  )
  @hotpath_release_buf_fn = Fiddle::Function.new(
    @lib['hyperion_io_uring_hotpath_release_buffer'],
    [Fiddle::TYPE_VOIDP, Fiddle::TYPE_SHORT], Fiddle::TYPE_VOID
  )
  @hotpath_force_unhealthy_fn = Fiddle::Function.new(
    @lib['hyperion_io_uring_hotpath_force_unhealthy'],
    [Fiddle::TYPE_VOIDP], Fiddle::TYPE_VOID
  )
  @hotpath_is_healthy_fn = Fiddle::Function.new(
    @lib['hyperion_io_uring_hotpath_is_healthy'],
    [Fiddle::TYPE_VOIDP], Fiddle::TYPE_INT
  )
  # Plan #2 Task 2.3.4 — one-copy buffer extraction: copies `len`
  # bytes from kernel buffer-ring slot `buf_id` into a
  # caller-supplied output buffer. Returns byte count or negative errno.
  @hotpath_copy_buffer_fn = Fiddle::Function.new(
    @lib['hyperion_io_uring_hotpath_copy_buffer'],
    [Fiddle::TYPE_VOIDP,  # ptr
     Fiddle::TYPE_SHORT,  # buf_id (u16)
     Fiddle::TYPE_INT,    # len (u32)
     Fiddle::TYPE_VOIDP,  # out_ptr
     Fiddle::TYPE_INT],   # out_cap (u32)
    Fiddle::TYPE_INT
  )
  @lib
rescue Fiddle::DLError, StandardError => e
  warn "[hyperion] IOUring failed to load (#{e.class}: #{e.message}); falling back to epoll"
  @lib = nil
  nil
end

.parse_kernel_release ⇒ Object

‘Etc.uname` is the canonical source. Falls back to `/proc/sys/kernel/osrelease` when uname isn’t available (e.g. specs that stub Etc.uname but leave release alone).



341
342
343
344
345
346
347
348
349
350
351
352
 
# File 'lib/hyperion/io_uring.rb', line 341

def parse_kernel_release
  release = Etc.uname[:release].to_s
  if release.empty? && File.exist?('/proc/sys/kernel/osrelease')
    release = File.read('/proc/sys/kernel/osrelease').strip
  end
  m = release.match(/\A(\d+)\.(\d+)/)
  return nil unless m

  [m[1].to_i, m[2].to_i]
rescue StandardError
  nil
end

.reset! ⇒ Object

Test seam: clear cached probe so ‘supported?` re-runs. Used by specs that stub Etc.uname or RbConfig.



273
274
275
276
277
 
# File 'lib/hyperion/io_uring.rb', line 273

def reset!
  @supported = nil
  @hotpath_supported = nil
  @lib = nil
end

.resolve_hotpath_policy!(policy) ⇒ Object

Plan #2 — resolve ‘:off | :auto | :on` for the hotpath gate. Mirrors `resolve_policy!` semantics: `:on` raises on unsupported, `:auto` quietly falls back, `:off` returns false.



563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
 
# File 'lib/hyperion/io_uring.rb', line 563

def self.resolve_hotpath_policy!(policy)
  case policy
  when :off, nil, false
    false
  when :auto
    hotpath_supported?
  when :on, true
    unless hotpath_supported?
      raise Unsupported,
            'io_uring hotpath required (io_uring_hotpath: :on) but unsupported on this host ' \
            "(linux=#{linux?}, kernel_ok=#{kernel_supports_io_uring?}, " \
            "hotpath_supported=#{hotpath_supported?})"
    end
    true
  else
    raise ArgumentError, "io_uring_hotpath must be :off, :auto, or :on (got #{policy.inspect})"
  end
end

.resolve_policy!(policy) ⇒ Object

Resolve the operator’s ‘io_uring` policy + the runtime gate into a boolean “use io_uring on this server”. Called by Server at boot.

Policy values:

:off  → never. Returns false. Used for the 2.3.0 default.
:auto → use it when supported; quietly fall back otherwise.
:on   → demand it. Raise UnsupportedError if not available
        so the operator's misconfig surfaces at boot, not as
        a slow-fallback mystery hours later.



542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
 
# File 'lib/hyperion/io_uring.rb', line 542

def self.resolve_policy!(policy)
  case policy
  when :off, nil, false
    false
  when :auto
    supported?
  when :on, true
    unless supported?
      raise Unsupported,
            'io_uring required (io_uring: :on) but not supported on this host ' \
            "(linux=#{linux?}, kernel_ok=#{kernel_supports_io_uring?}, lib_loaded=#{!@lib.nil?})"
    end
    true
  else
    raise ArgumentError, "io_uring must be :off, :auto, or :on (got #{policy.inspect})"
  end
end

.ring_accept(ptr, fd, errno_buf) ⇒ Object 



479
480
481
 
# File 'lib/hyperion/io_uring.rb', line 479

def ring_accept(ptr, fd, errno_buf)
  @accept_fn.call(ptr, fd, errno_buf)
end

.ring_free(ptr) ⇒ Object 



475
476
477
 
# File 'lib/hyperion/io_uring.rb', line 475

def ring_free(ptr)
  @ring_free_fn.call(ptr)
end

.ring_new(depth) ⇒ Object

—- FFI wrappers —-



470
471
472
473
 
# File 'lib/hyperion/io_uring.rb', line 470

def ring_new(depth)
  ptr = @ring_new_fn.call(depth)
  ptr.null? ? nil : ptr
end

.ring_read(ptr, fd, buf, max, errno_buf) ⇒ Object 



483
484
485
 
# File 'lib/hyperion/io_uring.rb', line 483

def ring_read(ptr, fd, buf, max, errno_buf)
  @read_fn.call(ptr, fd, buf, max, errno_buf)
end

.supported? ⇒ Boolean

Cached three-state result: nil = not-yet-probed, true/false = result.

The probe is intentionally process-local (not Fiber-local) — the answer is the same for every fiber in this process, and probing once at boot avoids per-request syscall overhead.

Returns:

  • (Boolean) 


265
266
267
268
269
 
# File 'lib/hyperion/io_uring.rb', line 265

def supported?
  return @supported unless @supported.nil?

  @supported = compute_supported
end