Class: Mutineer::CLI

Inherits:
Object
  • Object
show all
Defined in:
lib/mutineer/cli.rb

Overview

Command-line entry point. start is the single public method called by bin/mutineer; it parses argv, acts, and exits with a pinned code.

Exit codes:

0  success / requested output (--version, --help, score >= threshold)
1  survivors below threshold, or a runtime error
2  usage / flag error (unknown subcommand, invalid flag, unknown operator,
 out-of-range threshold)

Constant Summary collapse 

<<~USAGE
  Usage: mutineer [options] <command> [args]

  Commands:
    run [options] <source...> --test <test...>   Mutate, run, and report
    run --dry-run [options] <source...>          Print candidate mutations only

  Run options:
    --test FILE          Test file covering the sources (repeatable)
    --operators LIST     Comma-separated operator names (default: Tier 1 set)
    --threshold FLOAT    Fail (exit 1) when score < FLOAT (default: 0 = off)
    --baseline FILE      Fail (exit 1) on NEW survivors / score drop vs a prior
                         --format json run (CI delta gate)
    --baseline-epsilon FLOAT  Score-drop tolerance for --baseline (default: 0)
    --only NAME          Restrict to one fully-qualified subject
    --since REF          Only mutate lines changed since git REF (e.g. origin/main)
    --jobs N             Parallel worker count (default: processor count)
    --strategy NAME      reload (whole-file) or redefine (surgical); default: reload
    --framework NAME     minitest or rspec (default: auto-detect from --test names)
    --boot FILE          Require FILE once in the parent to boot the app env, then
                         fork per mutant (Rails apps; requires --test)
    --rails              Sugar for --boot config/environment --strategy redefine
    --test-command CMD   Run the target suite in the app's own runtime as a
                         subprocess (for apps on Ruby < 3.4). CMD must contain
                         %{files}. Scrubs Mutineer Ruby PATH pins; set RAILS_ENV
                         on the mutineer command (not as KEY=val inside CMD)
    --daemon             Boot the app ONCE in a persistent daemon and fork per
                         mutant, with per-worker DB isolation so --jobs N is safe
                         under Rails (needs --rails/--boot; not with --test-command)
    --format human|json|html  Report format (default: human)
    --output FILE        Write the report to FILE instead of stdout
    --dry-run            List mutations without executing
    --fail-fast          Stop at the first surviving mutant
    --verbose            Surface the real error when a fork capture fails (alias: --debug)

  Options:
    --list-operators  List available operators (default vs optional) and exit
    --version         Print version and exit
    --help            Print this help and exit
USAGE
PRECEDENCE_FLAGS =

Field symbols whose config-file value is suppressed when the flag is typed.

%i[operators jobs threshold only].freeze
STRATEGY_ALIASES =

Deprecated internal strategy names, mapped to their canonical equivalents.

{ "7a" => "reload", "7b" => "redefine" }.freeze

Class Method Summary collapse

Class Method Details

.autopair!(config, explicit) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Auto-pair sources to tests by path convention when no --test was given (explicit --test wins). Each source with an inferred test on disk joins the run; a source with none is dropped with a one-line stderr warning and the run continues with the rest. If every source is dropped: in boot mode the dedicated --boot/--rails-requires-test check reports it; otherwise exit 2 with a usage message. The framework is re-detected from the inferred set unless it was set explicitly (a spec-only project loads/reports as rspec).

Parameters:

  • config (Mutineer::Config) —

    run configuration.

  • explicit (Set<Symbol>) —

    explicit CLI fields.



403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
 
# File 'lib/mutineer/cli.rb', line 403

def self.autopair!(config, explicit)
  return unless config.tests.empty?

  paired = config.sources.filter_map do |s|
    t = Pairing.infer_test(s, project_root: config.project_root, prefer: config.framework)
    [s, t] if t
  end
  (config.sources - paired.map(&:first)).each do |s|
    warn "[mutineer] no test found by convention for #{s}; skipping"
  end
  config.sources = paired.map(&:first)
  config.tests   = paired.map(&:last).uniq
  config.framework = Config.detect_framework(config.tests) unless explicit.include?(:framework)

  return unless config.sources.empty?
  return if config.boot # let the --boot/--rails-requires-test check report it

  warn "mutineer: no test files found by convention; pass --test or add tests"
  exit 2
end

.dry_run(config) ⇒ void

This method returns an undefined value.

Runs dry-run mode. Reuses Runner.collect_jobs (+ filter_since) so the candidate list cannot drift from a real run's job selection.

Parameters:



516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
 
# File 'lib/mutineer/cli.rb', line 516

def self.dry_run(config)
  operator_classes = MutatorRegistry.resolve(config.operators || MutatorRegistry::DEFAULT_NAMES)
  jobs, ignored_results, source_map = Runner.collect_jobs(config, operator_classes)
  # Narrow jobs and ignored the same way so the summary matches the printed list.
  if config.since
    jobs = Runner.filter_since(jobs, source_map, config)
    ignored_jobs = ignored_results.map { |r| [r.subject, r.mutation, r.id] }
    ignored = Runner.filter_since(ignored_jobs, source_map, config).size
  else
    ignored = ignored_results.size
  end

  per_operator = Hash.new(0)
  skipped = 0
  jobs.each do |subject, mutation, _id|
    source = source_map[subject.file]
    unless mutation.valid?(source)
      skipped += 1
      next
    end

    line = source.byteslice(0, mutation.start_offset).count("\n") + 1
    per_operator[mutation.operator] += 1
    original = source.byteslice(mutation.start_offset...mutation.end_offset)
    puts "[#{mutation.operator}] #{subject.qualified_name}  " \
         "#{subject.file}:#{line}  `#{original}` -> `#{mutation.replacement}`"
  end

  total = per_operator.values.sum
  breakdown = per_operator.map { |op, n| "#{op}: #{n}" }.join(", ")
  summary = breakdown.empty? ? "" : "#{breakdown} — "
  puts "#{summary}#{total} mutations (dry run, not executed); " \
       "#{skipped} skipped (invalid); #{ignored} ignored (suppressed)"
  exit 0
end

.execute(config) ⇒ void

This method returns an undefined value.

Executes the run command.

Parameters:



457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
 
# File 'lib/mutineer/cli.rb', line 457

def self.execute(config)
  if config.tests.empty?
    warn "mutineer: run requires at least one --test file (or use --dry-run)"
    exit 2
  end

  aggregate, source_map = Runner.execute(config)
  reporter = Reporter.new(aggregate, source_map)

  # Diff the current run against the baseline (preflighted above) by the
  # stable survivor id. The delta is rendered inline (human section / additive
  # json block) and gates exit independently of --threshold.
  delta = (Baseline.load(config.baseline).diff(aggregate, epsilon: config.baseline_epsilon) if config.baseline)

  reporter.report(out: $stdout, err: $stderr, threshold: config.threshold,
                  format: config.format, output: config.output, baseline: delta)

  # Warn (stderr, so it never pollutes json/html) that an external run's score
  # is not comparable to an in-process run: no coverage narrowing (uncovered
  # mutants count as survivors), and an infra failure is scored as a kill
  # (upper bound). Daemon coverage fallback warnings are emitted from the runner
  # only when the map is unavailable, not on every --daemon run.
  if config.test_command
    warn "[mutineer] --test-command score is an upper bound, not comparable to an " \
         "in-process run: no coverage narrowing (uncovered mutants count as survivors) " \
         "and an infra failure is scored as a kill."
  end

  # Nudge toward the opt-in tier-2 operators (human report only: never
  # pollute JSON output).
  if !%w[json html].include?(config.format) && (hint = tier2_hint(config.operators))
    puts hint
  end

  # --baseline and --threshold are independent gates OR'd together.
  # `max` of two 0/1 codes is the OR; usage (2) is handled earlier and wins.
  baseline_exit = delta&.regressed ? 1 : 0
  exit [reporter.exit_code(threshold: config.threshold), baseline_exit].max
end

.list_operators ⇒ void

This method returns an undefined value.

Lists available operators.



174
175
176
177
178
179
180
 
# File 'lib/mutineer/cli.rb', line 174

def self.list_operators
  MutatorRegistry::ALL.each_key do |name|
    state = MutatorRegistry.default?(name) ? "default" : "disabled"
    puts format("%-20s tier %d  %-9s %s",
                name, MutatorRegistry.tier(name), state, MutatorRegistry::DESCRIPTIONS[name])
  end
end

.preflight_baseline!(path) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

A missing/unreadable/unparseable baseline is a usage error (exit 2), mirroring --output/--since preflight, so CI sees "bad invocation," not a backtrace mid-run. Validating up front = attempting the load (it raises ConfigError/SystemCallError; the actual diff reloads in execute).

Parameters:

  • path (String) —

    baseline file path.



432
433
434
435
436
437
 
# File 'lib/mutineer/cli.rb', line 432

def self.preflight_baseline!(path)
  Baseline.load(path)
rescue Mutineer::ConfigError, SystemCallError => e
  warn "mutineer: #{e.message}"
  exit 2
end

.preflight_output!(path) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Preflights an output path.

Parameters:

  • path (String) —

    output file path.



444
445
446
447
448
449
450
451
 
# File 'lib/mutineer/cli.rb', line 444

def self.preflight_output!(path)
  dir = File.dirname(File.expand_path(path))
  return if File.directory?(dir) && File.writable?(dir)

  reason = File.directory?(dir) ? "directory is not writable" : "no such directory"
  warn "mutineer: cannot write to #{path}: #{reason}"
  exit 2
end

.run(config, explicit = Set.new) ⇒ void

This method returns an undefined value.

Runs the requested command after validation.

Parameters:

  • config (Mutineer::Config) —

    run configuration.

  • explicit (Set<Symbol>) (defaults to: Set.new) —

    explicit CLI fields.



187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
 
# File 'lib/mutineer/cli.rb', line 187

def self.run(config, explicit = Set.new)
  if config.sources.empty?
    warn "mutineer: run requires at least one source file"
    exit 2
  end
  validate!(config, explicit)

  config.dry_run ? dry_run(config) : execute(config)
rescue ArgumentError => e
  # Unknown --operators value surfaces here; no backtrace reaches the user.
  warn "mutineer: #{e.message}"
  exit 2
rescue SystemCallError => e
  # A missing/unreadable path reaches here as Errno::ENOENT etc. A plain
  # message and usage exit, never a raw backtrace.
  warn "mutineer: #{e.message}"
  exit 2
rescue SyntaxError => e
  # A syntactically invalid source file surfaces when `require`d; report it
  # cleanly rather than dumping a backtrace.
  warn "mutineer: cannot load source: #{e.message}"
  exit 1
rescue Mutineer::ParseError => e
  warn "mutineer: error reading: #{e.message}"
  exit 1
rescue Mutineer::SmokeCheckError => e
  # The unmutated suite is not green under --test-command: a broken
  # environment, not weak tests. Runtime error (exit 1), not usage (exit 2).
  warn "mutineer: #{e.message}"
  exit 1
rescue Mutineer::DaemonBootError => e
  # The daemon is gone for good, so the run ended rather than scoring the rest
  # against it. A deliberate stop deserves a message, not a raw backtrace.
  warn "mutineer: #{e.message}"
  exit 1
end

.start(argv) ⇒ void

This method returns an undefined value.

Parses arguments, executes the command, and exits.

Parameters:

  • argv (Array<String>) —

    raw command-line arguments.



79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
 
# File 'lib/mutineer/cli.rb', line 79

def self.start(argv)
  opts = {}            # symbol => value, the CLI-provided Config fields
  explicit = Set.new   # precedence keys the user typed
  show_operators = false

  parser = OptionParser.new do |o|
    o.banner = BANNER
    o.on("--version") do
      puts Mutineer::VERSION
      exit 0
    end
    o.on("--help") do
      puts BANNER
      exit 0
    end
    o.on("--list-operators") { show_operators = true }
    o.on("--dry-run") { opts[:dry_run] = true }
    o.on("--fail-fast") { opts[:fail_fast] = true; explicit << :fail_fast }
    o.on("--only NAME") { |v| opts[:only] = v; explicit << :only }
    o.on("--since REF") { |v| opts[:since] = v; explicit << :since }
    o.on("--test FILE") { |v| (opts[:tests] ||= []) << v }
    o.on("--operators LIST") { |v| opts[:operators] = v.split(",").map(&:strip); explicit << :operators }
    o.on("--threshold FLOAT") do |v|
      f = Float(v, exception: false)
      if f.nil?
        warn "mutineer: --threshold requires a number between 0 and 100 (got: #{v.inspect})"
        exit 2
      end
      opts[:threshold] = f
      explicit << :threshold
    end
    o.on("--jobs N") { |v| opts[:jobs] = v; explicit << :jobs }
    o.on("--strategy STRAT") { |v| opts[:strategy] = v; explicit << :strategy }
    o.on("--framework NAME") { |v| opts[:framework] = v; explicit << :framework }
    o.on("--boot FILE") { |v| opts[:boot] = v; explicit << :boot }
    o.on("--rails") { opts[:rails] = true }
    o.on("--verbose") { opts[:verbose] = true }
    o.on("--debug") { opts[:verbose] = true } # alias of --verbose
    o.on("--format FORMAT") { |v| opts[:format] = v }
    o.on("--output FILE") { |v| opts[:output] = v }
    # --baseline is also a .mutineer.yml key, so mark it explicit when typed
    # (CLI wins over the file). --baseline-epsilon is CLI-only.
    o.on("--baseline FILE") { |v| opts[:baseline] = v; explicit << :baseline }
    o.on("--baseline-epsilon FLOAT") { |v| opts[:baseline_epsilon] = v.to_f }
    # Run the target suite as a subprocess in the app's OWN runtime so
    # mutineer (Ruby >= 3.4) can mutation-test apps pinned to an older Ruby.
    o.on("--test-command CMD") { |v| opts[:test_command] = v; explicit << :test_command }
    # Boot the app ONCE in a persistent daemon and fork per mutant, with
    # per-worker DB isolation so --jobs N is safe under Rails.
    o.on("--daemon") { opts[:daemon] = true; explicit << :daemon }
  end

  begin
    parser.parse!(argv)
  rescue OptionParser::InvalidOption, OptionParser::MissingArgument => e
    warn "mutineer: #{e.message}"
    exit 2
  end

  if show_operators
    list_operators
    exit 0
  end

  if argv.empty?
    puts BANNER
    exit 0
  end

  begin
    file_path = Config.find_file
    file_hash = file_path ? Config.from_file(file_path) : {}
    config = Config.resolve(opts, file_hash, explicit)
  rescue Mutineer::ConfigError => e
    # The lib layer raises instead of killing the host; the CLI maps a
    # config (usage) error to exit 2.
    warn "mutineer: #{e.message}"
    exit 2
  end

  case argv.first
  when "run"
    # A directory source expands to its **/*.rb files; literal files pass
    # through. Test inference (when --test is omitted) happens in validate!.
    config.sources = Pairing.expand_sources(argv[1..], project_root: config.project_root)
    run(config, explicit)
  else
    warn "mutineer: unknown command '#{argv.first}'"
    exit 2
  end
end

.tier2_hint(active) ⇒ String?

The tier-2 operators not in the active set, as a one-line hint (or nil when they are all already enabled). active nil means the default (Tier-1) set.

Parameters:

  • active (Array<String>, nil) —

    active operator names.

Returns:

  • (String, nil) —

    hint text or nil.



502
503
504
505
506
507
508
509
 
# File 'lib/mutineer/cli.rb', line 502

def self.tier2_hint(active)
  active ||= MutatorRegistry::DEFAULT_NAMES
  unused = MutatorRegistry::TIER2_NAMES - active
  return if unused.empty?

  "#{unused.size} tier-2 operators available (#{unused.join(', ')}) — " \
    "enable with --operators <list>."
end

.validate!(config, explicit = Set.new) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Flag validation: every flag/usage failure exits 2, consistent with the taxonomy above. CI can tell "mistyped flag" from "tests too weak."

Parameters:

  • config (Mutineer::Config) —

    run configuration.

  • explicit (Set<Symbol>) (defaults to: Set.new) —

    explicit CLI fields.



231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
 
# File 'lib/mutineer/cli.rb', line 231

def self.validate!(config, explicit = Set.new)
  unless (0.0..100.0).cover?(config.threshold)
    warn "mutineer: --threshold must be between 0 and 100"
    exit 2
  end

  jobs = Integer(config.jobs.to_s, exception: false)
  if jobs.nil? || jobs < 1
    warn "mutineer: --jobs requires a positive integer (got: #{config.jobs})"
    exit 2
  end
  config.jobs = jobs

  unless %w[human json html].include?(config.format)
    warn %(mutineer: unknown format "#{config.format}". Expected: human, json, html)
    exit 2
  end

  # Canonical strategies are reload|redefine; 7a/7b are accepted as deprecated
  # aliases. Normalize to canonical so the rest of the pipeline sees one name.
  config.strategy = STRATEGY_ALIASES.fetch(config.strategy, config.strategy)
  unless %w[reload redefine].include?(config.strategy)
    warn %(mutineer: unknown strategy "#{config.strategy}". Expected: reload, redefine)
    exit 2
  end

  unless %w[minitest rspec].include?(config.framework)
    warn %(mutineer: unknown framework "#{config.framework}". Expected: minitest, rspec)
    exit 2
  end

  validate_test_command!(config) if config.test_command

  validate_since!(config) if config.since
  preflight_output!(config.output) if config.output
  preflight_baseline!(config.baseline) if config.baseline

  # When --test is omitted, infer each source's test by convention. Autopair
  # also re-detects framework from inferred tests when --framework was not set.
  autopair!(config, explicit) unless config.dry_run

  # Daemon validation runs AFTER autopair so auto-inferred *_spec.rb tests
  # cannot bypass the RSpec rejection (framework would still be minitest if we
  # validated before discovery).
  validate_daemon!(config) if config.daemon

  # Boot mode needs at least one --test file (nothing to select from otherwise).
  if config.boot && config.tests.empty?
    warn "mutineer: --boot/--rails requires at least one --test file"
    exit 2
  end

  validate_paths!(config)
end

.validate_daemon!(config) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

--daemon selects the persistent-daemon backend (boot once, fork per mutant, per-worker DB isolation on SQLite). Usage errors exit 2: cannot combine with --test-command; requires --rails or --boot; minitest only; reload strategy only (redefine needs a shared VM surgical path the daemon does not ship).

Parameters:



329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
 
# File 'lib/mutineer/cli.rb', line 329

def self.validate_daemon!(config)
  if config.test_command
    warn "mutineer: choose one backend — --daemon and --test-command cannot be combined"
    exit 2
  end
  unless config.rails || config.boot
    warn "mutineer: --daemon needs an app to boot; add --rails (or --boot FILE)"
    exit 2
  end
  if config.framework == "rspec"
    warn "mutineer: --daemon supports only --framework minitest " \
         "(rspec is not implemented on the daemon path yet)"
    exit 2
  end
  return if config.strategy == "reload"

  # --rails defaults strategy to redefine; daemon always whole-file loads.
  warn "[mutineer] --daemon uses --strategy reload " \
       "(redefine is not supported on the daemon path); forcing reload."
  config.strategy = "reload"
end

.validate_paths!(config) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Validate path existence up front so a typo is a clean usage error (exit 2), not an Errno::ENOENT backtrace from deep in the run. Flag checks run first so a bad flag still reports the flag, not the missing file.

Parameters:



382
383
384
385
386
387
388
389
 
# File 'lib/mutineer/cli.rb', line 382

def self.validate_paths!(config)
  missing = (config.sources + config.tests)
            .reject { |p| File.exist?(File.expand_path(p, config.project_root)) }
  return if missing.empty?

  warn "mutineer: no such file: #{missing.join(', ')}"
  exit 2
end

.validate_since!(config) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

--since needs a real git repo and a resolvable ref; either failure is a usage error (exit 2) so CI sees "bad invocation," not "tests too weak."

Parameters:



357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
 
# File 'lib/mutineer/cli.rb', line 357

def self.validate_since!(config)
  _out, _err, status = Open3.capture3(
    "git", "-C", config.project_root, "rev-parse", "--verify", "--quiet",
    "#{config.since}^{commit}"
  )
  return if status.success?

  inside, = Open3.capture3(
    "git", "-C", config.project_root, "rev-parse", "--is-inside-work-tree"
  )
  msg = inside.strip == "true" ? "unknown git ref: #{config.since}" : "--since requires a git repository"
  warn "mutineer: #{msg}"
  exit 2
rescue Errno::ENOENT
  warn "mutineer: --since requires git on PATH"
  exit 2
end

.validate_test_command!(config) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

--test-command runs the target suite in the app's own runtime. Validate its shape up front (usage errors → exit 2) and force serial execution: each subprocess boots the app and opens its own fixture transaction against the same DB, so --jobs > 1 would corrupt results (fixture-contention hazard). Unlike --rails, this path has NO per-worker DB isolation to opt into, so an explicit --jobs N is forced to 1 rather than honored.

Parameters:



296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
 
# File 'lib/mutineer/cli.rb', line 296

def self.validate_test_command!(config)
  if config.test_command.strip.empty?
    warn "mutineer: --test-command must not be empty"
    exit 2
  end
  unless config.test_command.include?("%{files}")
    warn "mutineer: --test-command must contain %{files} (where the --test paths are substituted)"
    exit 2
  end
  if config.boot
    warn "mutineer: --test-command cannot be combined with --boot/--rails " \
         "(the external subprocess boots the app itself)"
    exit 2
  end
  if config.strategy == "redefine"
    warn "mutineer: --test-command supports only --strategy reload " \
         "(surgical redefine needs a shared VM; the subprocess has its own)"
    exit 2
  end
  return unless config.jobs > 1

  warn "[mutineer] --test-command runs serially (no per-worker DB isolation yet); forcing --jobs 1."
  config.jobs = 1
end