Class: Riffer::Evals::Metric

Inherits:
Object
  • Object
show all
Defined in:
lib/riffer/evals/metric.rb

Overview

Represents a metric configuration with thresholds.

Metrics define which evaluator to use and what thresholds determine pass/fail.

metric = Riffer::Evals::Metric.new(
  evaluator_class: Riffer::Evals::Evaluators::AnswerRelevancy,
  min: 0.85,
  weight: 1.0
)

metric.passes?(result)  # => true/false based on thresholds

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(evaluator_class:, min: nil, max: nil, weight: 1.0) ⇒ Metric

Initializes a new metric.

Raises Riffer::ArgumentError if evaluator_class is not a subclass of Riffer::Evals::Evaluator.

: (evaluator_class: singleton(Riffer::Evals::Evaluator), ?min: Float?, ?max: Float?, ?weight: Float) -> void



34
35
36
37
38
39
40
41
42
43
 
# File 'lib/riffer/evals/metric.rb', line 34

def initialize(evaluator_class:, min: nil, max: nil, weight: 1.0)
  unless evaluator_class.is_a?(Class) && evaluator_class < Riffer::Evals::Evaluator
    raise Riffer::ArgumentError, "evaluator_class must be a subclass of Riffer::Evals::Evaluator, got #{evaluator_class.inspect}"
  end

  @evaluator_class = evaluator_class
  @min = min&.to_f
  @max = max&.to_f
  @weight = weight.to_f
end

Instance Attribute Details

#evaluator_class ⇒ Object (readonly)

The evaluator class to use.



18
19
20
 
# File 'lib/riffer/evals/metric.rb', line 18

def evaluator_class
  @evaluator_class
end

#max ⇒ Object (readonly)

Maximum acceptable score (for lower_is_better evaluators).



24
25
26
 
# File 'lib/riffer/evals/metric.rb', line 24

def max
  @max
end

#min ⇒ Object (readonly)

Minimum acceptable score (for higher_is_better evaluators).



21
22
23
 
# File 'lib/riffer/evals/metric.rb', line 21

def min
  @min
end

#weight ⇒ Object (readonly)

Weight for aggregate scoring (default: 1.0).



27
28
29
 
# File 'lib/riffer/evals/metric.rb', line 27

def weight
  @weight
end

Instance Method Details

#passes?(result) ⇒ Boolean

Checks if a result passes this metric’s thresholds.

: (Riffer::Evals::Result) -> bool

Returns:

  • (Boolean) 


48
49
50
51
52
 
# File 'lib/riffer/evals/metric.rb', line 48

def passes?(result)
  return false if min && result.score < min
  return false if max && result.score > max
  true
end

#to_h ⇒ Object

Returns a hash representation of the metric.

: () -> Hash[Symbol, untyped]



57
58
59
60
61
62
63
64
 
# File 'lib/riffer/evals/metric.rb', line 57

def to_h
  {
    evaluator_class: evaluator_class,
    min: min,
    max: max,
    weight: weight
  }
end