Class: Riffer::Evals::Result

Inherits:
Object
  • Object
show all
Defined in:
lib/riffer/evals/result.rb,
sig/generated/riffer/evals/result.rbs

Overview

Represents the result of a single evaluation.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(evaluator:, score:, reason: nil, metadata: {}, higher_is_better: true, token_usage: nil) ⇒ Result

Raises Riffer::ArgumentError if score is not between 0.0 and 1.0.

: (evaluator: singleton(Riffer::Evals::Evaluator), score: Float, ?reason: String?, ?metadata: Hash[Symbol, untyped], ?higher_is_better: bool, ?token_usage: Riffer::Providers::TokenUsage?) -> void

Parameters:

  • evaluator: (singleton(Riffer::Evals::Evaluator))
  • score: (Float)
  • reason: (String, nil) (defaults to: nil)
  • metadata: (Hash[Symbol, untyped]) (defaults to: {})
  • higher_is_better: (Boolean) (defaults to: true)
  • token_usage: (Riffer::Providers::TokenUsage, nil) (defaults to: nil) 


28
29
30
31
32
33
34
35
36
 
# File 'lib/riffer/evals/result.rb', line 28

def initialize(evaluator:, score:, reason: nil, metadata: {}, higher_is_better: true, token_usage: nil)
  @evaluator = evaluator
  @score = score.to_f
  validate_score!
  @reason = reason
  @metadata = metadata
  @higher_is_better = higher_is_better
  @token_usage = token_usage
end

Instance Attribute Details

#evaluator ⇒ singleton(Riffer::Evals::Evaluator) (readonly)

The evaluator class that produced this result.

Returns:



7
8
9
 
# File 'lib/riffer/evals/result.rb', line 7

def evaluator
  @evaluator
end

#higher_is_better ⇒ Boolean (readonly)

Whether higher scores are better for this evaluator.

Returns:

  • (Boolean) 


19
20
21
 
# File 'lib/riffer/evals/result.rb', line 19

def higher_is_better
  @higher_is_better
end

#metadata ⇒ Hash[Symbol, untyped] (readonly)

Additional metadata from the evaluation.

Returns:

  • (Hash[Symbol, untyped]) 


16
17
18
 
# File 'lib/riffer/evals/result.rb', line 16

def metadata
  @metadata
end

#reason ⇒ String? (readonly)

Human-readable explanation of the score.

Returns:

  • (String, nil) 


13
14
15
 
# File 'lib/riffer/evals/result.rb', line 13

def reason
  @reason
end

#score ⇒ Float (readonly)

The evaluation score (0.0 to 1.0).

Returns:

  • (Float) 


10
11
12
 
# File 'lib/riffer/evals/result.rb', line 10

def score
  @score
end

#token_usage ⇒ Riffer::Providers::TokenUsage? (readonly)

Token usage for the judge call that produced this result, when the evaluator used an LLM. Nil for rule-based evaluators.

Returns:



23
24
25
 
# File 'lib/riffer/evals/result.rb', line 23

def token_usage
  @token_usage
end

Instance Method Details

#to_h ⇒ Hash[Symbol, untyped]

Returns a hash representation of the result.

-- : () -> Hash[Symbol, untyped]

Returns:

  • (Hash[Symbol, untyped]) 


42
43
44
45
46
47
48
49
50
51
 
# File 'lib/riffer/evals/result.rb', line 42

def to_h
  {
    evaluator: evaluator.name,
    score: score,
    reason: reason,
    metadata: metadata,
    higher_is_better: higher_is_better,
    token_usage: token_usage&.to_h
  }
end

#validate_score! ⇒ void

This method returns an undefined value.

-- : () -> void



57
58
59
60
61
 
# File 'lib/riffer/evals/result.rb', line 57

def validate_score!
  return if score.is_a?(Numeric) && score >= 0.0 && score <= 1.0

  raise Riffer::ArgumentError, "score must be between 0.0 and 1.0, got #{score}"
end