Module: Perfgate::Comparison::MetricDecision

Defined in:
lib/perfgate/comparison/metric_decision.rb

Overview

Decides pass/warn/fail/inconclusive for a single metric on a single workload (spec section 16). A regression is only ever reported as "fail" when it clears both bars: a practically-significant change (spec 16.4's thresholds) and statistical support that it isn't just sampling noise (spec 16.3's Mann-Whitney test) -- see spikes/FINDINGS.md for why Milestone 0 settled on requiring both. sql_count is the one exception: it's compared deterministically (spec 16.3), since query counts are rarely continuous/noisy the way timing and allocation metrics are.

Constant Summary collapse

DETERMINISTIC_METRICS = 
%w[sql_count].freeze

Class Method Summary collapse

Class Method Details

.call(metric:, baseline_samples:, candidate_samples:, config: Perfgate.configuration) ⇒ Object 



23
24
25
26
27
28
29
30
31
 
# File 'lib/perfgate/comparison/metric_decision.rb', line 23

def call(metric:, baseline_samples:, candidate_samples:, config: Perfgate.configuration)
  return MetricChange.inconclusive(metric) unless enough_samples?(baseline_samples, candidate_samples, config)

  if DETERMINISTIC_METRICS.include?(metric.to_s)
    DeterministicMetricDecision.call(metric, baseline_samples, candidate_samples, config)
  else
    StatisticalMetricDecision.call(metric, baseline_samples, candidate_samples, config)
  end
end

.enough_samples?(baseline_samples, candidate_samples, config) ⇒ Boolean

Returns:

  • (Boolean) 


33
34
35
36
 
# File 'lib/perfgate/comparison/metric_decision.rb', line 33

def enough_samples?(baseline_samples, candidate_samples, config)
  minimum = config.comparison_minimum_samples
  baseline_samples.size >= minimum && candidate_samples.size >= minimum
end