Class: Riffer::Evals::Metric

Inherits:
Object
  • Object
show all
Defined in:
lib/riffer/evals/metric.rb

Overview

Represents a metric configuration with thresholds.

Metrics define which evaluator to use and what thresholds determine pass/fail.

metric = Riffer::Evals::Metric.new(
  evaluator_identifier: "answer_relevancy",
  min: 0.85,
  weight: 1.0
)

metric.passes?(result)  # => true/false based on thresholds

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(evaluator_identifier:, min: nil, max: nil, weight: 1.0) ⇒ Metric

Initializes a new metric.

: (evaluator_identifier: String, ?min: Float?, ?max: Float?, ?weight: Float) -> void



32
33
34
35
36
37
 
# File 'lib/riffer/evals/metric.rb', line 32

def initialize(evaluator_identifier:, min: nil, max: nil, weight: 1.0)
  @evaluator_identifier = evaluator_identifier.to_s
  @min = min&.to_f
  @max = max&.to_f
  @weight = weight.to_f
end

Instance Attribute Details

#evaluator_identifier ⇒ Object (readonly)

The identifier of the evaluator to use.



18
19
20
 
# File 'lib/riffer/evals/metric.rb', line 18

def evaluator_identifier
  @evaluator_identifier
end

#max ⇒ Object (readonly)

Maximum acceptable score (for lower_is_better evaluators).



24
25
26
 
# File 'lib/riffer/evals/metric.rb', line 24

def max
  @max
end

#min ⇒ Object (readonly)

Minimum acceptable score (for higher_is_better evaluators).



21
22
23
 
# File 'lib/riffer/evals/metric.rb', line 21

def min
  @min
end

#weight ⇒ Object (readonly)

Weight for aggregate scoring (default: 1.0).



27
28
29
 
# File 'lib/riffer/evals/metric.rb', line 27

def weight
  @weight
end

Instance Method Details

#evaluator_class ⇒ Object

Returns the evaluator class for this metric.

: () -> singleton(Riffer::Evals::Evaluator)?



42
43
44
 
# File 'lib/riffer/evals/metric.rb', line 42

def evaluator_class
  Riffer::Evals::Evaluators::Repository.find(evaluator_identifier)
end

#passes?(result) ⇒ Boolean

Checks if a result passes this metric’s thresholds.

: (Riffer::Evals::Result) -> bool

Returns:

  • (Boolean) 


49
50
51
52
53
 
# File 'lib/riffer/evals/metric.rb', line 49

def passes?(result)
  return false if min && result.score < min
  return false if max && result.score > max
  true
end

#to_h ⇒ Object

Returns a hash representation of the metric.

: () -> Hash[Symbol, untyped]



58
59
60
61
62
63
64
65
 
# File 'lib/riffer/evals/metric.rb', line 58

def to_h
  {
    evaluator_identifier: evaluator_identifier,
    min: min,
    max: max,
    weight: weight
  }
end