Class: RubyLLM::Contract::Eval::Evaluator::Regex

Inherits:
Object
  • Object
show all
Defined in:
lib/ruby_llm/contract/eval/evaluator/regex.rb

Overview

Matches a regex against the flattened textual representation of output.

Instance Method Summary collapse

Constructor Details

#initialize(pattern) ⇒ Regex

Returns a new instance of Regex.



9
10
11
 
# File 'lib/ruby_llm/contract/eval/evaluator/regex.rb', line 9

def initialize(pattern)
  @pattern = pattern.is_a?(::Regexp) ? pattern : ::Regexp.new(pattern)
end

Instance Method Details

#call(output:, expected: nil, input: nil) ⇒ Object

rubocop:disable Lint/UnusedMethodArgument



13
14
15
16
17
18
19
 
# File 'lib/ruby_llm/contract/eval/evaluator/regex.rb', line 13

def call(output:, expected: nil, input: nil) # rubocop:disable Lint/UnusedMethodArgument
  pattern = @pattern.inspect
  details = text_for(output).match?(@pattern) ? "matches #{pattern}" : "does not match #{pattern}"
  passed = details.start_with?("matches")

  EvaluationResult.new(score: passed ? 1.0 : 0.0, passed: passed, details: details)
end