Module: Riffer::Evals::Profile

Defined in:
lib/riffer/evals/profile.rb

Overview

Module factory providing the ai_evals DSL for defining eval profiles.

Include this module in a module to create an eval profile that can be included in agents.

module EvalProfiles::QualityEvals
  include Riffer::Evals::Profile

  ai_evals do
    metric :answer_relevancy, min: 0.85
    metric :hallucination, max: 0.10
  end
end

class MyAgent < Riffer::Agent
  include EvalProfiles::QualityEvals
  model "openai/gpt-4o"
end

result = MyAgent.run_eval(input: "What is Ruby?")
result.passed?  # => true/false

Defined Under Namespace

Modules: AgentClassMethods, ClassMethods Classes: Builder

Class Method Summary collapse

Class Method Details

.included(base) ⇒ Object

: (Module) -> void



28
29
30
31
32
33
34
35
36
37
38
 
# File 'lib/riffer/evals/profile.rb', line 28

def self.included(base)
  base.extend(ClassMethods)

  # When the profile module is included in an Agent, add the eval method
  base.define_singleton_method(:included) do |target|
    if target < Riffer::Agent
      target.extend(AgentClassMethods)
      target.instance_variable_set(:@eval_profile, base)
    end
  end
end