Class: SkillBench::Cli::CompareCommand

Inherits:
Object
  • Object
show all
Defined in:
lib/skill_bench/cli/compare_command.rb

Overview

Handles the ‘skill-bench compare` command. Runs the same eval with two skill variants and reports the comparison.

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(argv) ⇒ CompareCommand

Returns a new instance of CompareCommand.

Parameters:

  • argv (Array<String>) —

    Raw CLI arguments



23
24
25
 
# File 'lib/skill_bench/cli/compare_command.rb', line 23

def initialize(argv)
  @argv = argv
end

Class Method Details

.call(argv) ⇒ Integer

Parses argv and executes the comparison.

Parameters:

  • argv (Array<String>) —

    Raw CLI arguments

Returns:

  • (Integer) —

    Exit code



18
19
20
 
# File 'lib/skill_bench/cli/compare_command.rb', line 18

def self.call(argv)
  new(argv).call
end

Instance Method Details

#call ⇒ Integer

Parses options, runs both variants, and prints a comparison report.

Returns:

  • (Integer) —

    Exit code (0 if both pass, 1 otherwise)



30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
 
# File 'lib/skill_bench/cli/compare_command.rb', line 30

def call
  options = Services::CompareOptionParser.call(@argv)

  skill_name = @argv.shift
  return error_missing_skill unless skill_name
  return error_missing_variant_a unless options[:variant_a]
  return error_missing_variant_b unless options[:variant_b]
  return error_missing_eval unless options[:eval]

  variant_a = Services::VariantParser.call(options[:variant_a])
  variant_b = Services::VariantParser.call(options[:variant_b])

  puts "--- Running Variant A: #{options[:variant_a]} ---"
  puts "--- Running Variant B: #{options[:variant_b]} ---"

  results = Services::ComparisonRunner.call(
    variant_a,
    variant_b,
    skill_name,
    options[:eval]
  )

  Services::ComparisonReporter.call(
    results[:result_a],
    results[:result_b],
    options[:variant_a],
    options[:variant_b]
  )

  Services::ExitCodeCalculator.call(results[:result_a], results[:result_b])
rescue SkillBench::HelpRequested
  0
rescue StandardError => e
  warn "Error: #{e.message}"
  1
end