Class: Mutineer::Runner

Inherits:
Object
  • Object
show all
Defined in:
lib/mutineer/runner.rb

Overview

Orchestrates one mutation end-to-end: apply it textually, validate the result, select its covering test files from the coverage map, then run only those against the mutated source in an isolated child process (strategy whole-file reload via load).

The source file path is passed explicitly because Mutation carries only byte offsets, not its file. Coverage-map selection replaces a hardcoded test file: a mutation whose line no test exercises is :no_coverage (no fork); otherwise exactly the covering test files run in the child.

Class Method Summary collapse

Class Method Details

.collect_jobs(config, operator_classes) ⇒ Array(Array, Array<Result>, Hash<String,String>)

Collect every (subject, mutation, id) up front so a backend can run them. A mutant the user marked known-equivalent (inline disable-line comment or .mutineer.yml ignore id) is classified :ignored here and NEVER run. It is removed from the killed+survived denominator so a strong file reaches 100%. The stable id is computed per subject (occurrence needs the full list) and carried on every job so the parent can reattach it after the run. Shared by the in-process, external, and daemon backends so job selection can never drift.

Returns:

  • (Array(Array, Array<Result>, Hash<String,String>)) —

    jobs, ignored, source_map.



148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
 
# File 'lib/mutineer/runner.rb', line 148

def self.collect_jobs(config, operator_classes)
  source_map = {}
  disabled_map = {}
  ignore_set = config.ignore.to_set
  jobs = []
  ignored_results = []
  Project.discover(config.sources, only: config.only).each do |subject|
    source = (source_map[subject.file] ||= File.read(subject.file))
    disabled = (disabled_map[subject.file] ||= suppress_map(source))
    mutations = operator_classes.flat_map { |klass| klass.new.mutations_for(subject, source) }
    ids = MutantId.for_subject(subject, source, mutations)
    mutations.each_with_index do |mutation, i|
      id = ids[i]
      line = source.byteslice(0, mutation.start_offset).count("\n") + 1
      if suppressed?(mutation.operator, line, id, disabled, ignore_set)
        ignored_results << Result.ignored.with(subject: subject, mutation: mutation, id: id)
      else
        jobs << [subject, mutation, id]
      end
    end
  end
  [jobs, ignored_results, source_map]
end

.coverage_selection(source_file, mutation, subject, source, coverage_map) ⇒ Array(Symbol, Object)

Coverage-based test selection, shared by the in-process (run) and daemon paths so both narrow identically (score parity). Returns [:run, abs_test_paths] when some test covers the mutant's line, or [:verdict, Result] (no_coverage / uncapturable) when none do.

An empty selection is :uncapturable (not :no_coverage) when the mutant's enclosing method body got coverage from no successful capture but a sibling test failed to capture: the coverage was lost, not absent. Both are excluded from the score denominator, so this distinction is reporting-only and never changes the daemon-vs-in-process score.

Parameters:

  • source_file (String) —

    the mutated source file path.

  • mutation (Mutineer::Mutation) —

    the mutation (for its line offset).

  • subject (Mutineer::Subject, nil) —

    the subject (for its method body range).

  • source (String) —

    the original source text.

  • coverage_map (Mutineer::CoverageMap) —

    the built/loaded coverage map.

Returns:

  • (Array(Symbol, Object)) —

    [:run, Array<String>] or [:verdict, Result].



256
257
258
259
260
261
262
263
264
265
266
267
268
 
# File 'lib/mutineer/runner.rb', line 256

def self.coverage_selection(source_file, mutation, subject, source, coverage_map)
  line   = source.byteslice(0, mutation.start_offset).count("\n") + 1
  chosen = coverage_map.tests_for(source_file, line)
  if chosen.empty?
    # Method BODY range, not the whole def: the def/end lines are "covered" at
    # class-load even when the body never runs (body_loc is the statements' span).
    loc   = subject&.body_loc
    range = loc ? (loc.start_line..loc.end_line) : (line..line)
    return [:verdict, coverage_map.method_uncapturable?(source_file, range) ? Result.uncapturable : Result.no_coverage]
  end

  [:run, chosen.map { |t| File.expand_path(t, coverage_map.project_root) }]
end

.ensure_rails_env(config) ⇒ Object

When --rails is on and RAILS_ENV is unset, default it to "test" (and say so) before the app boots. Otherwise it boots development and nothing is measured. An explicitly-set RAILS_ENV is always respected.



339
340
341
342
343
344
345
 
# File 'lib/mutineer/runner.rb', line 339

def self.ensure_rails_env(config)
  return unless config.rails
  return unless ENV["RAILS_ENV"].nil? || ENV["RAILS_ENV"].empty?

  ENV["RAILS_ENV"] = "test"
  warn "[mutineer] RAILS_ENV was unset; defaulting to 'test' for --rails."
end

.execute(config) ⇒ Array(Mutineer::AggregateResult, Hash<String, String>)

Full orchestration: resolve operators, discover subjects, build the coverage map, run every mutation, and aggregate. Returns [AggregateResult, source_map]. The CLI then reports + applies the exit code; the integration test asserts directly on the AggregateResult.

The parent process requires each source file so its classes exist; forked children inherit them, so a covering test file's own require_relative of the source is a no-op and does not clobber the mutated load (spec ยง7).

Parameters:

Returns:



41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
 
# File 'lib/mutineer/runner.rb', line 41

def self.execute(config)
  operator_classes = MutatorRegistry.resolve(config.operators || MutatorRegistry::DEFAULT_NAMES)

  # External backend: run the suite as a subprocess in the app's own runtime.
  # It does no in-process boot/require or coverage build, so branch before any
  # of that. The in-process path below is untouched.
  return execute_external(config, operator_classes) if config.test_command

  # Daemon backend: boot the app ONCE in a persistent subprocess under the
  # app's bundle and fork per mutant. Tool-side we only discover jobs + build
  # payloads (Prism), so branch before any in-process boot.
  return DaemonBackend.execute(config, operator_classes) if config.daemon

  # Boot mode: require the boot file ONCE so the app env (e.g. Rails) is booted
  # in the parent and inherited by every fork. Do NOT manually require the
  # sources. Under Zeitwerk a manual require of an autoloadable file raises;
  # the booted env autoloads them, and subject discovery is a static Prism
  # parse that needs nothing loaded. Standalone mode requires the sources as
  # before so their classes exist for the children to inherit.
  if config.boot
    # Under --rails an unset RAILS_ENV boots development, where the test
    # suite is not loaded. Coverage comes back empty and EVERY mutant is
    # falsely reported no_coverage (score N/A, exit 0). Default it to test.
    ensure_rails_env(config)

    # Coverage instruments only files loaded AFTER it starts. Start it BEFORE
    # the boot require so the entire app loaded during boot is instrumented;
    # forked children then measure each test's coverage delta against it.
    require "coverage"
    Coverage.start(lines: true) unless Coverage.running?
    require File.expand_path(config.boot, config.project_root)
  else
    config.sources.each { |f| require File.expand_path(f, config.project_root) }
  end
  config.require_paths.each { |f| require File.expand_path(f, config.project_root) }

  if config.boot
    # Rails/Minitest test files do `require "test_helper"`, which needs the
    # test root on $LOAD_PATH (`bin/rails test` adds it). Prepend each test
    # file's helper root here in the parent so loading them in the fork
    # children (both coverage capture and per-mutant) resolves.
    boot_tests = config.tests.map { |t| File.expand_path(t, config.project_root) }
    test_load_roots(boot_tests).each { |d| $LOAD_PATH.unshift(d) unless $LOAD_PATH.include?(d) }

    # Boot mode now uses coverage selection too: capture each test's coverage
    # by forking the booted parent, then select covering tests per mutant.
    coverage_map = CoverageMap.new(
      source_paths: config.sources, test_paths: config.tests,
      cache_dir: config.cache_dir, project_root: config.project_root,
      load_paths: config.load_paths, framework: config.framework,
      boot_path: File.expand_path(config.boot, config.project_root),
      verbose: config.verbose
    ).build_via_fork(after_fork: (config.rails ? -> { reconnect_active_record } : nil))
  else
    coverage_map = CoverageMap.new(
      source_paths: config.sources, test_paths: config.tests,
      cache_dir: config.cache_dir, project_root: config.project_root,
      load_paths: config.load_paths, framework: config.framework
    ).build_or_load
  end

  # Collect every (subject, mutation) up front so the pool can fan them out.
  jobs, ignored_results, source_map = collect_jobs(config, operator_classes)

  jobs = filter_since(jobs, source_map, config) if config.since

  # Whole-file reload writes mutineer_mutant*.rb into each source dir (so
  # require_relative resolves). A SIGKILL'd child skips the tempfile's
  # ensure-unlink, orphaning it. `ensure` is unreliable vs SIGKILL, so the
  # PARENT sweeps each source dir before and after the run. Orphans are
  # impossible after a normal run.
  dirs = source_dirs(config)
  sweep_orphans(dirs)

  strategy = config.strategy
  # Fail-fast must be serial: a parallel stop_when fires on the first survivor
  # by wall-clock, not input order, so the survivor set would diverge from
  # --jobs 1 (daemon already forces serial for the same reason).
  jobs_n = config.fail_fast ? 1 : config.jobs
  results =
    begin
      framework = config.framework
      stop_when = config.fail_fast ? ->(r) { r.survived? } : nil
      bare = WorkerPool.new(jobs_n).run(jobs, stop_when: stop_when) do |subject, mutation|
        run(mutation, source_file: subject.file, coverage_map: coverage_map,
            subject: subject, strategy: strategy, rails: config.rails, framework: framework)
      end
      # The bare Results carry only status (Subjects hold live AST nodes that
      # do not marshal); reattach subject+mutation+id in the parent, in order.
      # filter_map drops nils for jobs --fail-fast left unscheduled.
      bare.each_with_index.filter_map { |r, i| r&.with(subject: jobs[i][0], mutation: jobs[i][1], id: jobs[i][2]) }
    ensure
      sweep_orphans(dirs)
    end

  [AggregateResult.new(results + ignored_results), source_map]
end

.execute_external(config, operator_classes) ⇒ Array(Mutineer::AggregateResult, Hash<String,String>)

External backend orchestration. Runs each mutant's whole-file mutation on disk (crash-safe swap) and executes the user's --test-command as a subprocess in the app's own runtime. Serial by construction (one shared DB, no per-worker isolation yet). No coverage narrowing: every mutant runs the full --test set; the score is therefore an upper bound and not comparable to an in-process run (the CLI discloses this).

Parameters:

  • config (Mutineer::Config) —

    run configuration (test_command set).

  • operator_classes (Array<Class>) —

    resolved operators.

Returns:



182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
 
# File 'lib/mutineer/runner.rb', line 182

def self.execute_external(config, operator_classes)
  abs_tests = config.tests.map { |t| File.expand_path(t, config.project_root) }
  dirs      = source_dirs(config)

  # Heal any file a prior hard-killed run left mutated BEFORE reading source.
  # collect_jobs computes mutation offsets/ids from the on-disk bytes, so a
  # still-mutated file would yield garbage offsets against the later-healed
  # source. Heal first, then discover jobs from the clean tree.
  FileSwap.restore_orphans(dirs)

  jobs, ignored_results, source_map = collect_jobs(config, operator_classes)
  jobs = filter_since(jobs, source_map, config) if config.since

  # Nothing to mutate: return before the smoke check, which runs the whole
  # --test set to calibrate a timeout no mutant would use (#76).
  return [AggregateResult.new(ignored_results), source_map] if jobs.empty?

  # Calibrate the per-mutant timeout from the clean run (a real suite far
  # outlasts the 10s in-process fork budget), and abort if it is not green.
  # 3x the clean run, floor 30s, ceiling 300s: a heuristic. The floor covers
  # a fast suite; the ceiling bounds a hung mutant (infinite loop) so a
  # handful cannot stall a serial run for ~45min on a slow suite.
  smoke_elapsed = ExternalBackend.smoke_check!(config.test_command, abs_tests)
  timeout = [[smoke_elapsed * 3, 30].max, 300].min.ceil

  results = []
  begin
    jobs.each do |subject, mutation, id|
      r = run_external(subject, mutation, config.test_command, abs_tests,
                       timeout: timeout, verbose: config.verbose)
      results << r.with(subject: subject, mutation: mutation, id: id)
      break if config.fail_fast && r.survived? # stop at the first survivor
    end
  ensure
    FileSwap.restore_orphans(dirs)
  end

  [AggregateResult.new(results + ignored_results), source_map]
end

.filter_since(jobs, source_map, config) ⇒ Object

--since: keep only jobs whose mutation lands on a line changed since the git ref. Composes with coverage selection (it only narrows the job list; each surviving mutant still goes through Runner.run's coverage check). A file with no changed lines (absent from the diff) contributes no jobs. Line is computed exactly as Runner.run does, from the already-read source in source_map.



304
305
306
307
308
309
310
311
312
313
 
# File 'lib/mutineer/runner.rb', line 304

def self.filter_since(jobs, source_map, config)
  changed = ChangedLines.for(ref: config.since, files: config.sources,
                             project_root: config.project_root)
  jobs.select do |subject, mutation|
    source = source_map[subject.file]
    line = source.byteslice(0, mutation.start_offset).count("\n") + 1
    abs = File.expand_path(subject.file, config.project_root)
    changed.fetch(abs, []).include?(line)
  end
end

.run(mutation, source_file:, coverage_map: nil, subject: nil, strategy: "reload", timeout: Isolation::DEFAULT_TIMEOUT, rails: false, framework: "minitest") ⇒ Mutineer::Result

Runs a single mutation through isolation.

Parameters:

  • mutation (Mutineer::Mutation) —

    mutation to run.

  • source_file (String) —

    source file path.

  • coverage_map (Mutineer::CoverageMap, nil) (defaults to: nil) —

    coverage map.

  • subject (Mutineer::Subject, nil) (defaults to: nil) —

    subject for surgical strategy.

  • strategy (String) (defaults to: "reload") —

    mutation strategy.

  • timeout (Integer) (defaults to: Isolation::DEFAULT_TIMEOUT) —

    child timeout in seconds.

  • rails (Boolean) (defaults to: false) —

    whether Rails reconnect handling is enabled.

  • framework (String) (defaults to: "minitest") —

    test framework name.

Returns:



384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
 
# File 'lib/mutineer/runner.rb', line 384

def self.run(mutation, source_file:, coverage_map: nil, subject: nil, strategy: "reload",
             timeout: Isolation::DEFAULT_TIMEOUT, rails: false, framework: "minitest")
  source  = File.read(source_file)
  mutated = mutation.apply(source)

  # Validity rule: a mutant that does not re-parse is skipped before forking.
  return Result.skipped if Parser.parse_string(mutated).errors.any?

  # Coverage selection (both standalone and boot mode): a mutation on a line
  # no test exercises is :no_coverage (no fork); otherwise exactly the
  # covering test files run in the child. Shared with the daemon path so both
  # narrow identically (score parity).
  kind, payload = coverage_selection(source_file, mutation, subject, source, coverage_map)
  return payload if kind == :verdict

  abs_tests = payload

  Isolation.run(timeout: timeout) do
    # Forking inherits the parent's live DB connection; sharing one socket
    # across processes corrupts it. Drop it so AR reconnects per child.
    reconnect_active_record if rails
    if strategy == "redefine"
      Isolation.apply_surgical(mutation, subject, source)
    else
      Isolation.apply_whole_file(mutated, source_file)
    end
    TestRunners.for(framework).run(abs_tests)
  end
end

.run_external(subject, mutation, command, abs_tests, timeout:, verbose:) ⇒ Mutineer::Result

Runs one mutant through the external backend: apply the whole-file mutation on disk, run the command, restore. An invalid (non-reparsing) mutant would fail to load and score a false killed, so skip it tool-side (Prism, already cheap) and never write the file, preserving the skipped verdict the in-process path gives at the pre-fork check.

Returns:



229
230
231
232
233
234
235
236
237
 
# File 'lib/mutineer/runner.rb', line 229

def self.run_external(subject, mutation, command, abs_tests, timeout:, verbose:)
  source  = File.read(subject.file)
  mutated = mutation.apply(source)
  return Result.skipped if Parser.parse_string(mutated).errors.any?

  FileSwap.with(subject.file, mutated) do
    ExternalBackend.run(command, abs_tests, timeout: timeout, verbose: verbose)
  end
end

.source_dirs(config) ⇒ Array<String>

The unique absolute directories holding the sources. Sweep target for both orphan mechanisms (in-process mutant tempfiles and external backup files), and shipped to the daemon via DaemonBackend.boot_config so it can sweep too. Shared so the path-expansion rule cannot drift between the paths.

Parameters:

Returns:

  • (Array<String>) —

    unique absolute source directories.



354
355
356
 
# File 'lib/mutineer/runner.rb', line 354

def self.source_dirs(config)
  config.sources.map { |f| File.dirname(File.expand_path(f, config.project_root)) }.uniq
end

.suppress_map(source) ⇒ Object

Scan a source once into { line_number => :all | Set } from inline # mutineer:disable-line [ops] markers (RuboCop semantics: the marker sits on the same physical line as the code it silences). A bare marker disables every operator on that line; disable-line a, b only the listed operators. Block-form disable/enable ranges are intentionally not supported.



275
276
277
278
279
280
281
282
283
284
 
# File 'lib/mutineer/runner.rb', line 275

def self.suppress_map(source)
  map = {}
  source.each_line.with_index(1) do |text, line|
    next unless (m = text.match(/#\s*mutineer:disable-line(?:\s+([\w,\s]+))?/))

    ops = m[1]
    map[line] = ops ? ops.split(",").map { |o| o.strip.to_sym }.reject(&:empty?).to_set : :all
  end
  map
end

.suppressed?(operator, line, id, disabled, ignore_set) ⇒ Boolean

True when this mutant is suppressed: its line bears a disable-line marker (bare, or scoped to its operator), OR its stable id is in the config ignore list. Checked at job-build time so a suppressed mutant is never forked.

Returns:

  • (Boolean) 


289
290
291
292
293
294
295
296
297
 
# File 'lib/mutineer/runner.rb', line 289

def self.suppressed?(operator, line, id, disabled, ignore_set)
  return true if ignore_set.include?(id)

  case (entry = disabled[line])
  when :all then true
  when Set  then entry.include?(operator)
  else false
  end
end

.sweep_orphans(dirs, glob = "mutineer_mutant*.rb") ⇒ void

This method returns an undefined value.

Removes stale mutant tempfiles from the given directories. The daemon writes a differently-named temp, so DaemonBackend passes its glob when it has to sweep tool-side (nothing boots on an empty run, so the daemon's own sweep never runs).

Parameters:

  • dirs (Array<String>) —

    directories to sweep.

  • glob (String) (defaults to: "mutineer_mutant*.rb") —

    filename pattern to remove.



365
366
367
368
369
370
371
 
# File 'lib/mutineer/runner.rb', line 365

def self.sweep_orphans(dirs, glob = "mutineer_mutant*.rb")
  dirs.each do |dir|
    Dir.glob(File.join(dir, glob)).each do |f|
      File.unlink(f) rescue nil # rubocop:disable Style/RescueModifier
    end
  end
end

.test_load_roots(test_files) ⇒ Object

For each test file, the directory to add to $LOAD_PATH so its require "test_helper" (or spec_helper) resolves: the nearest ancestor holding that helper, plus the file's own dir as a fallback.



318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
 
# File 'lib/mutineer/runner.rb', line 318

def self.test_load_roots(test_files)
  test_files.flat_map do |f|
    dir = File.dirname(f)
    root = nil
    loop do
      if File.exist?(File.join(dir, "test_helper.rb")) || File.exist?(File.join(dir, "spec_helper.rb"))
        root = dir
        break
      end
      parent = File.dirname(dir)
      break if parent == dir

      dir = parent
    end
    [root, File.dirname(f)].compact
  end.uniq
end