Module: Hyperion::H2Codec

Defined in:
lib/hyperion/h2_codec.rb,
ext/hyperion_http/h2_codec_glue.c

Overview

Phase 6 (RFC §3 2.0.0) — native HPACK encoder/decoder + frame primitives implemented in Rust. The Ruby side here is a thin Fiddle-based loader; all real work happens in ‘ext/hyperion_h2_codec`.

The integration is OPT-IN at runtime: ‘Http2Handler` checks `Hyperion::H2Codec.available?` and uses the native path only when the cdylib loaded successfully. Operators on systems without Rust (older Debians, locked-down CI runners, JRuby) get the existing `protocol-http2` Ruby HPACK path automatically — no boot-time error.

ABI version: bumped on any breaking C ABI change. Ruby refuses to load a binary that disagrees so a stale on-disk codec from an older gem install can’t crash the process.

Defined Under Namespace

Modules: CGlue Classes: Decoder, Encoder, OutputOverflow

Constant Summary collapse

EXPECTED_ABI = 
1

Class Method Summary collapse

Class Method Details

.available? ⇒ Boolean

Try to load the native cdylib. Sets ‘@available = true/false`. Idempotent — second call is a no-op.

Returns:

  • (Boolean) 


32
33
34
35
 
# File 'lib/hyperion/h2_codec.rb', line 32

def self.available?
  load!
  @available == true
end

.candidate_paths ⇒ Object 



444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
 
# File 'lib/hyperion/h2_codec.rb', line 444

def self.candidate_paths
  gem_lib = File.expand_path('../hyperion_h2_codec', __dir__)
  ext_target = File.expand_path('../../ext/hyperion_h2_codec/target/release', __dir__)
  # 2.11-B fix: order suffixes by host OS. Pre-2.11-B this was a
  # static `[dylib, so]` order, which broke on Linux hosts that
  # had a stale macOS `.dylib` on the path (e.g. a developer rsync
  # leaking the `target/release` artifact across platforms). Fiddle
  # would try the `.dylib` first, choke on the Mach-O binary with
  # `ArgumentError: invalid byte sequence in UTF-8` from libffi,
  # and the rescue in `load!` would silently fall back to the Ruby
  # HPACK path with no warning visible to bench harnesses.
  #
  # Ordering by `host_os` makes Linux pick `.so` first and ignore
  # any orphan `.dylib`; macOS keeps the `.dylib`-first behavior
  # for back-compat with existing dev environments.
  suffixes = if /darwin|mac/i.match?(RbConfig::CONFIG['host_os'])
               %w[libhyperion_h2_codec.dylib libhyperion_h2_codec.so]
             else
               %w[libhyperion_h2_codec.so libhyperion_h2_codec.dylib]
             end
  suffixes.flat_map { |name| [File.join(gem_lib, name), File.join(ext_target, name)] }
end

.cglue_active? ⇒ Boolean

2.11-B — operator-controllable gate that overlays CGlue availability. The Encoder/Decoder hot paths probe this (NOT ‘cglue_available?`) so a `HYPERION_H2_NATIVE_HPACK=v2` boot can force the Fiddle path even on a host where the C glue loaded successfully. This is the bench-isolation knob 2.11-B’s ‘bench/h2_rails_shape.sh` needs to compare native-v2 against native-v3 honestly — without it, “native” and “cglue” variants would always pick the same physical path.

‘Http2Handler#initialize` writes the gate based on the env var; tests can flip `@cglue_disabled` directly. Default false (i.e., gate is OPEN — same physical behavior as 2.4-A through 2.10).

Returns:

  • (Boolean) 


60
61
62
 
# File 'lib/hyperion/h2_codec.rb', line 60

def self.cglue_active?
  cglue_available? && !@cglue_disabled
end

.cglue_available? ⇒ Boolean

2.4-A — has the C glue (‘Hyperion::H2Codec::CGlue`) loaded AND successfully resolved the Rust HPACK symbols via dlopen/dlsym? Distinct from `available?` because CGlue can fail to load (older systems without dlfcn, hardened sandboxes blocking dlopen) while the Fiddle path still works. When this is true the per-call encode/decode hot path bypasses Fiddle entirely.

Returns:

  • (Boolean) 


43
44
45
46
 
# File 'lib/hyperion/h2_codec.rb', line 43

def self.cglue_available?
  load!
  @cglue_available == true
end

.cglue_disabled ⇒ Object 



68
69
70
 
# File 'lib/hyperion/h2_codec.rb', line 68

def self.cglue_disabled
  @cglue_disabled == true
end

.cglue_disabled=(value) ⇒ Object 



64
65
66
 
# File 'lib/hyperion/h2_codec.rb', line 64

def self.cglue_disabled=(value)
  @cglue_disabled = value ? true : false
end

.decoder_decode(ptr, input, in_len, out, cap) ⇒ Object 



495
496
497
 
# File 'lib/hyperion/h2_codec.rb', line 495

def self.decoder_decode(ptr, input, in_len, out, cap)
  @decoder_dec_fn.call(ptr, input, in_len, out, cap)
end

.decoder_free(ptr) ⇒ Object 



491
492
493
 
# File 'lib/hyperion/h2_codec.rb', line 491

def self.decoder_free(ptr)
  @decoder_free_fn.call(ptr)
end

.decoder_new ⇒ Object 



487
488
489
 
# File 'lib/hyperion/h2_codec.rb', line 487

def self.decoder_new
  @decoder_new_fn.call
end

.encoder_encode(ptr, names, name_lens, vals, val_lens, count, out, cap) ⇒ Object 



478
479
480
 
# File 'lib/hyperion/h2_codec.rb', line 478

def self.encoder_encode(ptr, names, name_lens, vals, val_lens, count, out, cap)
  @encoder_enc_fn.call(ptr, names, name_lens, vals, val_lens, count, out, cap)
end

.encoder_encode_v2(ptr, blob, blob_len, argv, argv_count, out, out_cap) ⇒ Object

fix-B (2.2.x) — v2 flat-blob encode. See lib.rs:hyperion_h2_codec_encoder_encode_v2.



483
484
485
 
# File 'lib/hyperion/h2_codec.rb', line 483

def self.encoder_encode_v2(ptr, blob, blob_len, argv, argv_count, out, out_cap)
  @encoder_enc_v2_fn.call(ptr, blob, blob_len, argv, argv_count, out, out_cap)
end

.encoder_free(ptr) ⇒ Object 



474
475
476
 
# File 'lib/hyperion/h2_codec.rb', line 474

def self.encoder_free(ptr)
  @encoder_free_fn.call(ptr)
end

.encoder_new ⇒ Object

FFI wrappers — kept thin so callers don’t see Fiddle::Pointer types. Each method is a one-liner that the Encoder/Decoder classes above invoke.



470
471
472
 
# File 'lib/hyperion/h2_codec.rb', line 470

def self.encoder_new
  @encoder_new_fn.call
end

.install_cglue(path) ⇒ Object

2.4-A — wire the C extension’s dlopen-based path. We require the bundled C extension (already loaded by ‘c_parser.rb` at gem boot in normal use, but we guard the constant lookup in case someone required `hyperion/h2_codec` directly without the C ext). Returns true iff CGlue is now installed and `encoder_encode_v3` is safe to call.



434
435
436
437
438
439
440
441
442
 
# File 'lib/hyperion/h2_codec.rb', line 434

def self.install_cglue(path)
  @cglue_available = false
  return unless defined?(Hyperion::H2Codec::CGlue)
  return unless Hyperion::H2Codec::CGlue.respond_to?(:install)

  @cglue_available = Hyperion::H2Codec::CGlue.install(path) ? true : false
rescue StandardError
  @cglue_available = false
end

.load! ⇒ Object

rubocop:disable Metrics/MethodLength



361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
 
# File 'lib/hyperion/h2_codec.rb', line 361

def self.load!
  return unless @available.nil?

  @available = false

  path = candidate_paths.find { |p| File.exist?(p) }
  unless path
    @lib = nil
    return
  end

  @lib = Fiddle.dlopen(path)

  @abi_fn = Fiddle::Function.new(@lib['hyperion_h2_codec_abi_version'],
                                 [], Fiddle::TYPE_INT)
  abi = @abi_fn.call
  if abi != EXPECTED_ABI
    warn "[hyperion] H2Codec ABI mismatch (got #{abi}, expected #{EXPECTED_ABI}); using Ruby fallback"
    @lib = nil
    return
  end

  @encoder_new_fn  = Fiddle::Function.new(@lib['hyperion_h2_codec_encoder_new'],
                                          [], Fiddle::TYPE_VOIDP)
  @encoder_free_fn = Fiddle::Function.new(@lib['hyperion_h2_codec_encoder_free'],
                                          [Fiddle::TYPE_VOIDP], Fiddle::TYPE_VOID)
  @encoder_enc_fn  = Fiddle::Function.new(@lib['hyperion_h2_codec_encoder_encode'],
                                          [Fiddle::TYPE_VOIDP, Fiddle::TYPE_VOIDP, Fiddle::TYPE_VOIDP,
                                           Fiddle::TYPE_VOIDP, Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT,
                                           Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT],
                                          Fiddle::TYPE_INT)
  # fix-B (2.2.x) — v2 flat-blob encode entry.
  # Signature: (handle, blob_ptr, blob_len, argv_ptr, argv_count, out_ptr, out_cap) -> i64
  # Sizes are passed as size_t so Fiddle::TYPE_SIZE_T matches the
  # Rust `usize` exactly on both 64-bit Linux and macOS arm64.
  @encoder_enc_v2_fn = Fiddle::Function.new(@lib['hyperion_h2_codec_encoder_encode_v2'],
                                            [Fiddle::TYPE_VOIDP, Fiddle::TYPE_VOIDP, Fiddle::TYPE_SIZE_T,
                                             Fiddle::TYPE_VOIDP, Fiddle::TYPE_SIZE_T,
                                             Fiddle::TYPE_VOIDP, Fiddle::TYPE_SIZE_T],
                                            Fiddle::TYPE_LONG_LONG)
  @decoder_new_fn  = Fiddle::Function.new(@lib['hyperion_h2_codec_decoder_new'],
                                          [], Fiddle::TYPE_VOIDP)
  @decoder_free_fn = Fiddle::Function.new(@lib['hyperion_h2_codec_decoder_free'],
                                          [Fiddle::TYPE_VOIDP], Fiddle::TYPE_VOID)
  @decoder_dec_fn  = Fiddle::Function.new(@lib['hyperion_h2_codec_decoder_decode'],
                                          [Fiddle::TYPE_VOIDP, Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT,
                                           Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT],
                                          Fiddle::TYPE_INT)

  @available = true

  # 2.4-A — try to install the C glue with the same path the
  # Fiddle loader just used. CGlue is defined by the bundled C
  # extension (`hyperion_http/hyperion_http.bundle`); if that
  # extension didn't compile or the dlopen fails, `@cglue_available`
  # stays nil and Encoder/Decoder transparently fall back to the
  # v2 (Fiddle) path. No warning on this branch — it's purely a
  # perf optimization, not a correctness gate.
  install_cglue(path)
rescue Fiddle::DLError, StandardError => e
  warn "[hyperion] H2Codec failed to load (#{e.class}: #{e.message}); using Ruby fallback"
  @lib = nil
  @available = false
  @cglue_available = false
end

.reset! ⇒ Object

Force a reload (test seam). Unsets the memoized state so the next ‘available?` call probes the filesystem again.



74
75
76
77
78
79
 
# File 'lib/hyperion/h2_codec.rb', line 74

def self.reset!
  @available = nil
  @cglue_available = nil
  @cglue_disabled = false
  @lib = nil
end