Module: Mutineer::ExternalBackend

Defined in:
lib/mutineer/external_backend.rb

Overview

#27 (U3): the external execution backend. Runs the user's --test-command as a subprocess in the app's OWN runtime (whatever Ruby its bundle resolves to), so mutineer (Ruby >= 3.4) can mutation-test apps pinned to an older Ruby.

This is deliberately NOT a TestRunners framework adapter: those return an Integer 0/1 from inside a fork and are dispatched by framework name. This is a whole backend — it spawns a process, enforces a wall-clock timeout, and maps the exit status to a Result. The mapping is the SAME direction as in-process (suite passes => survived, suite fails => killed) but coarser: it cannot tell an infrastructure error from a genuine kill, so the smoke check (below) guards the persistent case and the score is disclosed as an upper bound (KTD-3/KTD-6).

Constant Summary collapse

SMOKE_TIMEOUT =

Generous ceiling for the one-off smoke/calibration run (a cold app boot plus the full suite). The per-mutant timeout is derived from how long this took.

900
POLL =

Poll interval for the deadline wait loop. Independent of Isolation's loop — this backend waits on an external process TREE, not an in-process fork.

0.02

Class Method Summary collapse

Class Method Details

.build_argv(command, files) ⇒ Array<String>

Turn a command template into an argv array (no shell → no eval, no injection). The %{files} token expands IN PLACE to N separate argv elements — one per path, unescaped — so a path containing a space stays a single argument. It is not a space-joined string.

Parameters:

  • command (String) —

    the --test-command template (contains %files).

  • files (Array<String>) —

    test file paths to substitute.

Returns:

  • (Array<String>) —

    argv.



40
41
42
 
# File 'lib/mutineer/external_backend.rb', line 40

def self.build_argv(command, files)
  Shellwords.split(command).flat_map { |tok| tok == "%{files}" ? files : [tok] }
end

.run(command, files, timeout:, verbose: false) ⇒ Mutineer::Result

Runs the command for ONE mutant against whatever is currently on disk (the caller has already swapped the mutant in via FileSwap). Maps the outcome to a Result. Env is inherited by the subprocess, so RAILS_ENV=test mutineer … reaches the child with no parsing here.

Parameters:

  • command (String) —

    the --test-command template.

  • files (Array<String>) —

    test file paths.

  • timeout (Numeric) —

    per-mutant wall-clock timeout in seconds.

  • verbose (Boolean) (defaults to: false) —

    print the child's captured output on a non-pass.

Returns:



54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
 
# File 'lib/mutineer/external_backend.rb', line 54

def self.run(command, files, timeout:, verbose: false)
  kind, code, output, = spawn_capture(command, files, timeout)
  case kind
  when :timeout
    # A timeout is the one non-pass we flag by default — a normal kill is also
    # a non-zero exit, so notifying on every non-zero would spam every kill.
    warn "[mutineer] test-command exceeded #{timeout}s and was killed (scored timeout)."
    warn output if verbose && !output.empty?
    Result.timeout
  else # :exited
    return Result.survived if code&.zero?

    warn output if verbose && !output.empty?
    Result.killed
  end
end

.smoke_check!(command, files, timeout: SMOKE_TIMEOUT) ⇒ Float

Pre-flight: run the command once against the UNMUTATED tree. Green (exit 0) returns the elapsed seconds (used to calibrate the per-mutant timeout); anything else raises SmokeCheckError so the run aborts before scoring.

Parameters:

  • command (String) —

    the --test-command template.

  • files (Array<String>) —

    test file paths.

  • timeout (Numeric) (defaults to: SMOKE_TIMEOUT) —

    ceiling for the calibration run.

Returns:

  • (Float) —

    elapsed seconds of the clean run.

Raises:



80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
 
# File 'lib/mutineer/external_backend.rb', line 80

def self.smoke_check!(command, files, timeout: SMOKE_TIMEOUT)
  kind, code, output, elapsed = spawn_capture(command, files, timeout)
  return elapsed if kind == :exited && code&.zero?

  reason =
    if kind == :timeout then "did not finish within #{timeout}s"
    elsif code.nil?      then "was terminated by a signal"
    else                      "exited #{code}"
    end
  detail = output.empty? ? "" : "\n--- last output ---\n#{tail(output)}"
  raise SmokeCheckError,
        "the test command #{reason} against the UNMUTATED source — the " \
        "environment looks broken (check DB, RAILS_ENV, migrations), not the " \
        "tests weak.#{detail}"
end

.spawn_capture(command, files, timeout) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Spawns the command to a captured combined-output tempfile, enforces a wall-clock timeout (SIGKILL past the deadline), and returns [kind, exit_code, output, elapsed]. Mirrors Isolation's single-waiter loop: we are the only caller of waitpid on this pid, so the kill can never hit a reaped/recycled pid.



103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
 
# File 'lib/mutineer/external_backend.rb', line 103

def self.spawn_capture(command, files, timeout)
  argv = build_argv(command, files)
  # Unreachable in practice (validate_test_command! guarantees a non-empty
  # command with %{files}); a neutral ArgumentError, not the smoke-specific
  # SmokeCheckError, since this is not a smoke-check failure.
  raise ArgumentError, "--test-command produced an empty command" if argv.empty?

  out = Tempfile.create("mutineer_ext")
  start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  # `pgroup: true` puts the child in its OWN process group so a timeout can
  # kill the whole tree (`bundle exec rails test` forks parallel workers;
  # spring/bundler add more) — killing only the leader would orphan workers
  # that keep holding the shared DB, corrupting later serial mutants. The
  # explicit [program, argv0] form guarantees the no-shell exec path even for a
  # degenerate single-element argv (Process.spawn(*argv) would route a lone
  # metachar-bearing string through /bin/sh, breaking the argv-only invariant).
  pid = Process.spawn([argv.first, argv.first], *argv[1..], out: out, err: %i[child out], pgroup: true)
  kind, code = wait_with_timeout(pid, timeout)
  elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - start
  out.rewind
  [kind, code, out.read, elapsed]
ensure
  if out
    out.close
    File.unlink(out.path) rescue nil # rubocop:disable Style/RescueModifier
  end
end

.tail(output, lines = 40) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Last ~40 lines of captured output, for a smoke-failure message.



164
165
166
 
# File 'lib/mutineer/external_backend.rb', line 164

def self.tail(output, lines = 40)
  output.lines.last(lines).join
end

.wait_with_timeout(pid, timeout) ⇒ Array(Symbol, Integer, nil)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Waits for the spawned pid, SIGKILLing its process group past the deadline. Single-waiter deadline loop (mirrors Isolation), so the kill can never hit a reaped/recycled pid.

Parameters:

  • pid (Integer) —

    the spawned child pid.

  • timeout (Numeric) —

    wall-clock deadline in seconds.

Returns:

  • (Array(Symbol, Integer, nil)) —

    [:exited, code] or [:timeout, nil].



139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
 
# File 'lib/mutineer/external_backend.rb', line 139

def self.wait_with_timeout(pid, timeout)
  deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + timeout
  loop do
    reaped, status = Process.waitpid2(pid, Process::WNOHANG)
    return [:exited, status.exitstatus] if reaped

    if Process.clock_gettime(Process::CLOCK_MONOTONIC) >= deadline
      # Kill the whole process GROUP (negative pid) so forked test workers die
      # with the leader; fall back to the leader alone if the group is already
      # gone. The child led its group (pgroup: true at spawn).
      begin
        Process.kill(:KILL, -pid)
      rescue Errno::ESRCH, Errno::EPERM
        Process.kill(:KILL, pid) rescue nil # rubocop:disable Style/RescueModifier
      end
      Process.waitpid(pid) rescue nil # rubocop:disable Style/RescueModifier
      return [:timeout, nil]
    end
    sleep POLL
  end
end