Class: Phronomy::Eval::Metrics

Inherits:
Object
  • Object
show all
Defined in:
lib/phronomy/eval/metrics.rb

Overview

Aggregates a collection of EvalResult objects into summary statistics.

Examples:

metrics = Metrics.new(results)
puts metrics.pass_rate        # => 0.8
puts metrics.average_score    # => 0.9
puts metrics.to_h

Instance Method Summary collapse

Constructor Details

#initialize(results) ⇒ Metrics

Returns a new instance of Metrics.

Parameters:



15
16
17
 
# File 'lib/phronomy/eval/metrics.rb', line 15

def initialize(results)
  @results = results
end

Instance Method Details

#average_latency_ms ⇒ Float

Arithmetic mean of latency_ms across all results.

Returns:

  • (Float) 


46
47
48
49
 
# File 'lib/phronomy/eval/metrics.rb', line 46

def average_latency_ms
  return 0.0 if @results.empty?
  @results.sum(&:latency_ms).to_f / @results.size
end

#average_score ⇒ Float

Arithmetic mean of all scores.

Returns:

  • (Float) 


30
31
32
33
 
# File 'lib/phronomy/eval/metrics.rb', line 30

def average_score
  return 0.0 if @results.empty?
  @results.sum(&:score) / @results.size
end

#pass_rate ⇒ Float

Fraction of results that passed (score == 1.0).

Returns:

  • (Float) —

    in [0.0, 1.0]



22
23
24
25
 
# File 'lib/phronomy/eval/metrics.rb', line 22

def pass_rate
  return 0.0 if @results.empty?
  @results.count(&:pass?).to_f / @results.size
end

#to_h ⇒ Hash

Returns a plain Hash summary suitable for logging or serialisation.

Returns:

  • (Hash) 


54
55
56
57
58
59
60
61
62
63
 
# File 'lib/phronomy/eval/metrics.rb', line 54

def to_h
  {
    total: @results.size,
    pass_count: @results.count(&:pass?),
    pass_rate: pass_rate,
    average_score: average_score,
    total_usage: total_usage.to_h,
    average_latency_ms: average_latency_ms
  }
end

#total_usage ⇒ Phronomy::TokenUsage

Sum of all TokenUsage objects present in the results. Results without usage are skipped.

Returns:



39
40
41
 
# File 'lib/phronomy/eval/metrics.rb', line 39

def total_usage
  @results.map(&:usage).compact.reduce(TokenUsage.zero, :+)
end