Class: Braintrust::Eval::Context::Factory

Inherits:
Object
  • Object
show all
Defined in:
lib/braintrust/eval/context.rb

Overview

Encapsulates normalization of raw user inputs into typed wrappers.

Instance Method Summary collapse

Constructor Details

#initialize(state: nil, tracer_provider: nil, project_id: nil, project_name: nil) ⇒ Factory

Returns a new instance of Factory.

Parameters:

  • state (Braintrust::State, nil) (defaults to: nil) —

    Authenticated API state; passed through to scorer resolution

  • tracer_provider (#tracer, nil) (defaults to: nil) —

    OpenTelemetry tracer provider; passed through to remote scorers

  • project_id (String, nil) (defaults to: nil) —

    Project ID; passed through to the built Context

  • project_name (String, nil) (defaults to: nil) —

    Project name; required when resolving scorer slugs



85
86
87
88
89
90
 
# File 'lib/braintrust/eval/context.rb', line 85

def initialize(state: nil, tracer_provider: nil, project_id: nil, project_name: nil)
  @state = state
  @tracer_provider = tracer_provider
  @project_id = project_id
  @project_name = project_name
end

Instance Method Details

#build(task:, scorers:, cases:, classifiers: [], experiment_id: nil, experiment_name: nil, on_progress: nil, parent: nil, parameters: nil) ⇒ Context

Normalize raw inputs and construct a Braintrust::Eval::Context.

Parameters:

  • task (Task, Proc, #call) —

    Raw task

  • scorers (Array) —

    Raw scorers

  • classifiers (Array) (defaults to: []) —

    Raw classifiers

  • cases (Cases, Array, Enumerable) —

    Raw eval cases

  • experiment_id (String, nil) (defaults to: nil)
  • experiment_name (String, nil) (defaults to: nil)
  • on_progress (Proc, nil) (defaults to: nil)
  • parent (Hash, nil) (defaults to: nil) —

    Parent span info with keys :object_type, :object_id, and optionally :generation

Returns:



102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
 
# File 'lib/braintrust/eval/context.rb', line 102

def build(task:, scorers:, cases:, classifiers: [], experiment_id: nil,
  experiment_name: nil, on_progress: nil, parent: nil, parameters: nil)
  Context.new(
    task: normalize_task(task),
    scorers: normalize_scorers(scorers),
    classifiers: normalize_classifiers(classifiers),
    cases: normalize_cases(cases),
    experiment_id: experiment_id,
    experiment_name: experiment_name,
    project_id: @project_id,
    project_name: @project_name,
    state: @state,
    tracer_provider: @tracer_provider || OpenTelemetry.tracer_provider,
    on_progress: on_progress,
    parent_span_attr: resolve_parent_span_attr(parent),
    generation: parent&.dig(:generation),
    parameters: parameters
  )
end