Class: Bundler::Spinel::Probe

Inherits:
Object
  • Object
show all
Defined in:
lib/bundler/spinel/probe.rb

Overview

Decides a compatibility verdict for an unpacked gem against the current Spinel engine. Two complementary signals, because Spinel’s failure modes are not exit codes:

1. COMPILE SIGNAL. Spinel never exits non-zero on unsupported Ruby; it
   emits `warning: ... cannot resolve call to 'X' ... (emitting 0)` and
   degrades the call to a no-op. So we compile the gem's lib entrypoints
   with `spinel -c` and parse stderr. An unresolved-call warning names
   the exact missing feature (`unresolved:eval`), which is what makes the
   verdict forward-compatible — re-probing under a newer engine clears it
   the moment Spinel learns that call.

2. STATIC RISK SIGNAL. Some constructs (define_method, instance_eval, a
   bare `send`) are degraded with *no* warning at all, and dead-code
   elimination can hide an unsupported call that's defined-but-uncalled
   in a library. So we also scan the source for known-risky tokens. These
   don't reject on their own (the gem may never hit that path) but they
   downgrade `clean` → `risky`.

NEITHER signal catches Spinel’s *silent miscompiles* (local-var-name collapse, Int-0-as-nil). Only the ‘verified` rung — running the gem’s own tests through a Spinel-compiled harness — does. The lock-time gate trusts ‘clean`; only a curated whitelist / platform variant trusts `verified`.

Constant Summary collapse

UNRESOLVED_CALL =

An unsupported call — the strongest, most precise signal, and the one that’s forward-compatible (names the exact feature Spinel lacks today).

/cannot resolve call to '([^']+)'/.freeze
UNRESOLVED_REQUIRE =

A ‘require “x”` Spinel couldn’t follow. Spinel has no load path (plain require resolves only against <spinel>/lib), so a gem’s own split files and stdlib deps surface here. Informational, NOT a standalone reject: it’s as often a probe limitation as a real incompatibility.

/require "([^"]+)" could not be resolved/.freeze
REQUIRE_CALL_FAIL =

Since the #1383 fix, an unresolvable plain ‘require` no longer warns + continues (exit 0) — it’s emitted as an unsupported CallNode and the compile EXITS NON-ZERO. On the whole 189k corpus that turned ~thousands of previously-‘clean` gems (the universal `require “gem/version”` idiom) into spurious `analyze-failed`. We detect this exact form so a require-ONLY compile failure is classified as the no-load-path limitation, not a codegen failure. (b60fbd7; see harness/findings.)

/unsupported call:.*CallNode `require`/.freeze
HARD_COMPILE_ERROR =

A genuine codegen/compile failure — the C compiler choked, the analyzer died, or an unsupported construct OTHER than a plain require. Presence of any of these means the failure is real, not just the load-path limit.

%r{
  out\.c:\d+.*\berror:        |   # gcc error on emitted C
  \bfatal\b                   |
  \bSegmentation\ fault\b     |
  \banalyze\ failed\b         |
  unsupported\ puts\ argument |   # an unsupported non-require construct
  unsupported\ call:\ (?!.*CallNode\ `require`)  # any non-require unsupported call
}ix.freeze
ANALYZE_FAILED = 
/\b(analyze failed|fatal)\b/i.freeze
COMPILE_TIMEOUT =

Spinel’s analyze pass can spin for minutes on pathological inputs (no internal bound). In a wholesale survey that’s indistinguishable from a hang, so we cap each compile and treat an overrun as its own reject reason — ‘analyze-timeout` is itself a useful roadmap signal (which gems blow up the analyzer). Override with SPINEL_COMPILE_TIMEOUT (seconds).

Integer(ENV.fetch("SPINEL_COMPILE_TIMEOUT", "60"))
HARD_REJECT_TOKENS =

Tokens whose mere presence in ‘lib/*/.rb` makes the gem a definite Spinel reject — there’s no path under which the compile would succeed, so we skip the (expensive) spinel call entirely and record a ‘rejected` verdict from the static scan alone. Conservative set: only constructs Spinel will never support (threads, Mutex, TracePoint). Metaprogramming tokens like `define_method` stay in RISK_TOKENS below — they degrade silently, so the compile signal is still the right call there. Constructs that put a gem outside the AOT closed-world model entirely —rejected from a static scan, before a compile is even attempted. Thread/Mutex lived here until matz/spinel#1360 made them run (single- threaded, carrying the block’s value): they’re now compiled + flagged ‘risky` (below), not hard-rejected. TracePoint/set_trace_func stay —there is no degenerate-but-correct lowering for runtime tracing.

{
  /\bTracePoint\b/     => "TracePoint",
  /\bset_trace_func\b/ => "set_trace_func"
}.freeze
RISK_TOKENS =

token => reason. Tokens Spinel cannot honour and may silently no-op.

{
  # Thread/Mutex run single-threaded since matz/spinel#1360 — correct for
  # defensive use (a mutex guarding state, Thread.new for a value), but
  # degenerate for genuine concurrency: compiles, flagged, fails
  # `check --strict`. (Demoted from HARD_REJECT after #1360.)
  /\bThread\.(new|start|fork)\b/ => "thread",
  /\bMutex\.new\b/               => "mutex",
  /\bMutex_m\b/                  => "mutex_m",
  /\beval\s*\(/                  => "eval",
  /\binstance_eval\b/            => "instance_eval",
  /\b(class|module)_eval\b/      => "class_eval",
  /\bdefine_method\b/            => "define_method",
  /\bmethod_missing\b/           => "method_missing",
  /\brespond_to_missing\?/       => "respond_to_missing",
  /\bconst_missing\b/            => "const_missing",
  /\.send\s*\(/                  => "send",
  /\bpublic_send\b/              => "public_send",
  /\bObjectSpace\b/              => "objectspace",
  /\b(TracePoint|set_trace_func)\b/ => "tracepoint",
  /\bbinding\b/                  => "binding"
}.freeze

Instance Method Summary collapse

Constructor Details

#initialize(engine, ledger) ⇒ Probe

Returns a new instance of Probe.



109
110
111
112
 
# File 'lib/bundler/spinel/probe.rb', line 109

def initialize(engine, ledger)
  @engine = engine
  @ledger = ledger
end

Instance Method Details

#probe(gem_name, version, dir) ⇒ Object

gem_name, version, dir(unpacked) -> recorded Ledger::Verdict



115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
 
# File 'lib/bundler/spinel/probe.rb', line 115

def probe(gem_name, version, dir)
  @engine.ensure!

  # Fail fast on cheap static signals before spending the spinel-compile
  # budget — no entrypoint, a C extension, or a top-level token Spinel
  # will never support → record `rejected` and skip the compile.
  if (fast = static_hard_reject(dir, gem_name))
    return @ledger.record(@ledger.build(
      gem: gem_name, version: version, rev: @engine.rev,
      verdict: "rejected", reasons: fast, risks: [], probe: "static"
    ))
  end

  sig = compile_signal(dir, gem_name)
  risks = static_signal(dir)
  # Unfollowed requires are notes, not rejections — record them so a human
  # can see when a verdict is entangled with the no-load-path limitation.
  risks += sig[:requires].map { |r| "needs:#{r}" }

  verdict, reasons =
    if !sig[:calls].empty?
      # Genuine unsupported call(s): unambiguous, forward-compatible reject.
      ["rejected", sig[:calls].map { |s| "unresolved:#{s}" }]
    elsif sig[:timed_out]
      # Analyzer ran past the cap — pathological for Spinel, not the gem.
      ["rejected", ["analyze-timeout"]]
    elsif sig[:require_only_fail]
      # Compile failed solely on an unresolvable plain `require` (b60fbd7
      # hard-fails these where cb23cc6 warned + continued). That's the
      # no-load-path limitation, not a codegen failure — a real Spinel
      # project vendors deps so the require resolves. Classify as the
      # load-path limit (risky), keeping the needs: notes below.
      ["risky", ["load-path:require"]]
    elsif sig[:analyze_failed] || !sig[:exit_ok]
      ["rejected", ["analyze-failed"]]
    elsif !static_only_risks(risks).empty?
      ["risky", []]
    else
      ["clean", []]
    end

  @ledger.record(@ledger.build(
    gem: gem_name, version: version, rev: @engine.rev,
    verdict: verdict, reasons: reasons.uniq, risks: risks.uniq, probe: "compile+scan"
  ))
end