Class: Langfuse::Evaluator

Inherits:

BaseObservation

• Object
• BaseObservation
• Langfuse::Evaluator

Defined in:

lib/langfuse/observations.rb

Overview

Observation for tracking quality assessment and evaluation operations.

Examples:

Block-based API

Langfuse.observe("quality-check", as_type: :evaluator) do |evaluator|
  evaluator.input = { response: "Ruby is a language", expected: "Ruby is a programming language" }
  score = calculate_similarity(evaluator.input[:response], evaluator.input[:expected])
  evaluator.update(
    output: { score: score, passed: score > 0.8 },
    metadata: { metric: "similarity" }
  )
end

Stateful API

evaluator = Langfuse.start_observation("response-evaluation", {
  input: { response: llm_output, criteria: "accuracy" }
}, as_type: :evaluator)
evaluation_result = evaluate_response(evaluator.input[:response], evaluator.input[:criteria])
evaluator.update(output: evaluation_result)
evaluator.end

Instance Attribute Summary

Attributes inherited from BaseObservation

#otel_span, #otel_tracer, #type

Instance Method Summary

• #initialize(otel_span, otel_tracer, attributes: nil) ⇒ Evaluator constructor

  A new instance of Evaluator.

• #update(attrs) ⇒ self

Methods inherited from BaseObservation

#current_span, #end, #event, #id, #input=, #level=, #metadata=, #output=, #start_observation, #trace_id, #trace_url, #update_trace

Constructor Details

#initialize(otel_span, otel_tracer, attributes: nil) ⇒ Evaluator

Returns a new instance of Evaluator.

# File 'lib/langfuse/observations.rb', line 518

def initialize(otel_span, otel_tracer, attributes: nil)
  super(otel_span, otel_tracer, attributes: attributes, type: OBSERVATION_TYPES[:evaluator])
end

Instance Method Details

#update(attrs) ⇒ self

Parameters:

• attrs (Hash, Types::EvaluatorAttributes) —

  Evaluator attributes to set

Returns:

• (self)

# File 'lib/langfuse/observations.rb', line 524

def update(attrs)
  update_observation_attributes(attrs)
  self
end