Class: Phronomy::Eval::Metrics

Inherits:
Object
  • Object
show all
Defined in:
lib/phronomy/eval/metrics.rb

Overview

Aggregates a collection of EvalResult objects into summary statistics.

Examples:

metrics = Metrics.new(results)
puts metrics.pass_rate        # => 0.8
puts metrics.average_score    # => 0.9
puts metrics.to_h

Instance Method Summary collapse

Constructor Details

#initialize(results) ⇒ Metrics

Returns a new instance of Metrics.

Parameters:



14
15
16
 
# File 'lib/phronomy/eval/metrics.rb', line 14

def initialize(results)
  @results = results
end

Instance Method Details

#average_latency_ms ⇒ Float

Arithmetic mean of latency_ms across all results.

Returns:

  • (Float) 


41
42
43
44
 
# File 'lib/phronomy/eval/metrics.rb', line 41

def average_latency_ms
  return 0.0 if @results.empty?
  @results.sum(&:latency_ms).to_f / @results.size
end

#average_score ⇒ Float

Arithmetic mean of all scores.

Returns:

  • (Float) 


27
28
29
30
 
# File 'lib/phronomy/eval/metrics.rb', line 27

def average_score
  return 0.0 if @results.empty?
  @results.sum(&:score) / @results.size
end

#pass_rate ⇒ Float

Fraction of results that passed (score == 1.0).

Returns:

  • (Float) —

    in [0.0, 1.0]



20
21
22
23
 
# File 'lib/phronomy/eval/metrics.rb', line 20

def pass_rate
  return 0.0 if @results.empty?
  @results.count(&:pass?).to_f / @results.size
end

#to_h ⇒ Hash

Returns a plain Hash summary suitable for logging or serialisation.

Returns:

  • (Hash) 


48
49
50
51
52
53
54
55
56
57
 
# File 'lib/phronomy/eval/metrics.rb', line 48

def to_h
  {
    total: @results.size,
    pass_count: @results.count(&:pass?),
    pass_rate: pass_rate,
    average_score: average_score,
    total_usage: total_usage.to_h,
    average_latency_ms: average_latency_ms
  }
end

#total_usage ⇒ Phronomy::TokenUsage

Sum of all TokenUsage objects present in the results. Results without usage are skipped.

Returns:



35
36
37
 
# File 'lib/phronomy/eval/metrics.rb', line 35

def total_usage
  @results.map(&:usage).compact.reduce(TokenUsage.zero, :+)
end