Class: PerformanceComparator

Inherits:
Object
  • Object
show all
Defined in:
lib/tasks/performance_comparator.rb

Overview

Compares performance between the current branch and a base branch (default: main) by running the same benchmark suite in two separate Ruby processes —one per branch. Each process loads its own Canon implementation from disk, fully isolated from the other, and emits a JSON result. The comparator then diffs the two JSON payloads and reports regressions.

Running each branch in its own process is required because Canon uses Ruby autoload extensively; loading both branches’ code into a single process causes constant resolution and LOAD_PATH conflicts.

Constant Summary collapse

REPO_ROOT = 
File.expand_path(File.join(__dir__, "..", ".."))
DEFAULT_RUN_TIME = 
Integer(ENV.fetch("CANON_PERF_RUN_TIME", "10"))
DEFAULT_THRESHOLD =

10%

0.10
DEFAULT_BASE = 
"main"
TMP_PERF_DIR = 
File.join(REPO_ROOT, "tmp", "performance")
REPORT_SCRIPT = 
File.expand_path("performance_report.rb", __dir__)
RED = 
"\e[31m"
GREEN = 
"\e[32m"
YELLOW = 
"\e[33m"
CYAN = 
"\e[36m"
BOLD = 
"\e[1m"
DIM = 
"\e[2m"
CLEAR = 
"\e[0m"

Instance Method Summary collapse

Instance Method Details

#run ⇒ Object 



33
34
35
36
37
38
39
40
41
 
# File 'lib/tasks/performance_comparator.rb', line 33

def run
  clone_base
  current = run_report(REPO_ROOT, "current")
  base = run_report(base_clone_dir, "base (#{DEFAULT_BASE})")
  print_report(current, base)
  exit(1) if regressions?(current, base)
ensure
  cleanup
end