Class: RubricLLM::Metrics::Base

Inherits:
Object
  • Object
show all
Defined in:
lib/rubric_llm/metrics/base.rb

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(judge:) ⇒ Base

Returns a new instance of Base.



22
23
24
 
# File 'lib/rubric_llm/metrics/base.rb', line 22

def initialize(judge:)
  @judge = judge
end

Instance Attribute Details

#judge ⇒ Object (readonly)

Returns the value of attribute judge.



20
21
22
 
# File 'lib/rubric_llm/metrics/base.rb', line 20

def judge
  @judge
end

Class Method Details

.normalize_context(context) ⇒ Object

Single source of truth for what counts as usable context. Blank and whitespace-only chunks are dropped.



8
9
10
 
# File 'lib/rubric_llm/metrics/base.rb', line 8

def self.normalize_context(context)
  Array(context).map { |chunk| chunk.to_s.strip }.reject(&:empty?)
end

.require_context!(context) ⇒ Object

An empty context cannot produce a faithfulness score. Reject it as a caller error instead of letting a nil score read as a quality verdict.

Raises:

  • (ArgumentError) 


14
15
16
17
18
 
# File 'lib/rubric_llm/metrics/base.rb', line 14

def self.require_context!(context)
  return unless normalize_context(context).empty?

  raise ArgumentError, "context must contain at least one non-empty entry"
end

Instance Method Details

#call(question:, answer:, context: [], ground_truth: nil) ⇒ Object

Evaluate a single sample. Subclasses must implement this.

Returns { score: Float (0.0-1.0), details: Hash }

Raises:

  • (NotImplementedError) 


29
30
31
 
# File 'lib/rubric_llm/metrics/base.rb', line 29

def call(question:, answer:, context: [], ground_truth: nil, **)
  raise NotImplementedError, "#{self.class}#call must be implemented"
end