Module: Braintrust

Defined in:
lib/braintrust.rb,
lib/braintrust/api.rb,
lib/braintrust/eval.rb,
lib/braintrust/task.rb,
lib/braintrust/setup.rb,
lib/braintrust/state.rb,
lib/braintrust/trace.rb,
lib/braintrust/config.rb,
lib/braintrust/logger.rb,
lib/braintrust/prompt.rb,
lib/braintrust/scorer.rb,
lib/braintrust/server.rb,
lib/braintrust/contrib.rb,
lib/braintrust/dataset.rb,
lib/braintrust/version.rb,
lib/braintrust/eval/case.rb,
lib/braintrust/functions.rb,
lib/braintrust/eval/cases.rb,
lib/braintrust/eval/trace.rb,
lib/braintrust/server/sse.rb,
lib/braintrust/eval/result.rb,
lib/braintrust/eval/runner.rb,
lib/braintrust/eval/scorer.rb,
lib/braintrust/server/rack.rb,
lib/braintrust/api/datasets.rb,
lib/braintrust/eval/context.rb,
lib/braintrust/eval/summary.rb,
lib/braintrust/internal/env.rb,
lib/braintrust/api/functions.rb,
lib/braintrust/contrib/setup.rb,
lib/braintrust/internal/http.rb,
lib/braintrust/internal/time.rb,
lib/braintrust/server/router.rb,
lib/braintrust/eval/evaluator.rb,
lib/braintrust/eval/formatter.rb,
lib/braintrust/eval/functions.rb,
lib/braintrust/internal/retry.rb,
lib/braintrust/contrib/context.rb,
lib/braintrust/contrib/patcher.rb,
lib/braintrust/internal/origin.rb,
lib/braintrust/server/rack/app.rb,
lib/braintrust/contrib/registry.rb,
lib/braintrust/trace/attachment.rb,
lib/braintrust/api/internal/auth.rb,
lib/braintrust/api/internal/btql.rb,
lib/braintrust/internal/callable.rb,
lib/braintrust/internal/encoding.rb,
lib/braintrust/internal/template.rb,
lib/braintrust/trace/span_filter.rb,
lib/braintrust/contrib/integration.rb,
lib/braintrust/server/auth/no_auth.rb,
lib/braintrust/trace/span_exporter.rb,
lib/braintrust/contrib/support/otel.rb,
lib/braintrust/internal/thread_pool.rb,
lib/braintrust/server/handlers/eval.rb,
lib/braintrust/server/handlers/list.rb,
lib/braintrust/trace/span_processor.rb,
lib/braintrust/api/internal/projects.rb,
lib/braintrust/contrib/rails/railtie.rb,
lib/braintrust/vendor/mustache/utils.rb,
lib/braintrust/contrib/openai/patcher.rb,
lib/braintrust/contrib/support/openai.rb,
lib/braintrust/server/handlers/health.rb,
lib/braintrust/server/middleware/auth.rb,
lib/braintrust/server/middleware/cors.rb,
lib/braintrust/vendor/mustache/parser.rb,
lib/braintrust/server/auth/clerk_token.rb,
lib/braintrust/vendor/mustache/context.rb,
lib/braintrust/api/internal/experiments.rb,
lib/braintrust/contrib/ruby_llm/patcher.rb,
lib/braintrust/vendor/mustache/mustache.rb,
lib/braintrust/vendor/mustache/settings.rb,
lib/braintrust/vendor/mustache/template.rb,
lib/braintrust/contrib/anthropic/patcher.rb,
lib/braintrust/contrib/openai/deprecated.rb,
lib/braintrust/vendor/mustache/generator.rb,
lib/braintrust/contrib/openai/integration.rb,
lib/braintrust/vendor/mustache/enumerable.rb,
lib/braintrust/contrib/rails/server/engine.rb,
lib/braintrust/contrib/ruby_llm/deprecated.rb,
lib/braintrust/contrib/ruby_openai/patcher.rb,
lib/braintrust/contrib/anthropic/deprecated.rb,
lib/braintrust/contrib/ruby_llm/integration.rb,
lib/braintrust/server/services/eval_service.rb,
lib/braintrust/server/services/list_service.rb,
lib/braintrust/vendor/mustache/context_miss.rb,
lib/braintrust/contrib/anthropic/integration.rb,
lib/braintrust/contrib/rails/server/generator.rb,
lib/braintrust/contrib/ruby_openai/deprecated.rb,
lib/braintrust/contrib/ruby_openai/integration.rb,
lib/braintrust/contrib/openai/instrumentation/chat.rb,
lib/braintrust/contrib/rails/server/eval_controller.rb,
lib/braintrust/contrib/rails/server/list_controller.rb,
lib/braintrust/contrib/openai/instrumentation/common.rb,
lib/braintrust/contrib/ruby_llm/instrumentation/chat.rb,
lib/braintrust/contrib/rails/server/health_controller.rb,
lib/braintrust/contrib/ruby_llm/instrumentation/common.rb,
lib/braintrust/contrib/anthropic/instrumentation/common.rb,
lib/braintrust/contrib/openai/instrumentation/responses.rb,
lib/braintrust/contrib/ruby_openai/instrumentation/chat.rb,
lib/braintrust/contrib/anthropic/instrumentation/messages.rb,
lib/braintrust/contrib/openai/instrumentation/moderations.rb,
lib/braintrust/contrib/ruby_openai/instrumentation/common.rb,
lib/braintrust/contrib/rails/server/application_controller.rb,
lib/braintrust/contrib/ruby_openai/instrumentation/responses.rb,
lib/braintrust/contrib/anthropic/instrumentation/beta_messages.rb,
lib/braintrust/contrib/ruby_openai/instrumentation/moderations.rb

Overview

Backward compatibility shim for the old ruby-openai integration API. This file now just delegates to the new API.

Defined Under Namespace

Modules: Contrib, Eval, Functions, Internal, Log, Scorer, Server, Setup, Task, Trace, Vendor Classes: API, Config, Dataset, Error, Prompt, State

Constant Summary collapse

ScorerId =
Deprecated.

Use Braintrust::Scorer::ID instead.

Scorer::ID
DatasetId =
Deprecated.

Use Braintrust::Dataset::ID instead.

Dataset::ID
VERSION = 
"0.3.2"

Class Method Summary collapse

Class Method Details

.auto_instrument!(config = nil) ⇒ Array<Symbol>?

Auto-instrument available integrations.

Discovers which integrations have their target libraries loaded and instruments them automatically. This is the primary public API for enabling auto-instrumentation.

Environment variables:

- BRAINTRUST_AUTO_INSTRUMENT: set to "false" to disable (default: enabled)
- BRAINTRUST_INSTRUMENT_ONLY: comma-separated list of integrations to include
- BRAINTRUST_INSTRUMENT_EXCEPT: comma-separated list of integrations to exclude

Examples:

Enable with defaults

Braintrust.auto_instrument!

Explicitly enable

Braintrust.auto_instrument!(true)

Disable

Braintrust.auto_instrument!(false)

Only specific integrations

Braintrust.auto_instrument!(only: [:openai, :anthropic])

Exclude specific integrations

Braintrust.auto_instrument!(except: [:ruby_llm])

Parameters:

  • config (nil, Boolean, Hash) (defaults to: nil) —

    Auto-instrumentation configuration:

    • nil: use BRAINTRUST_AUTO_INSTRUMENT env var (defaults to enabled)

    • true: explicitly enable auto-instrumentation

    • false: explicitly disable auto-instrumentation

    • Hash with :only or :except keys for filtering integrations

Returns:

  • (Array<Symbol>, nil) —

    names of instrumented integrations, or nil if disabled



42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
 
# File 'lib/braintrust/contrib.rb', line 42

def self.auto_instrument!(config = nil)
  should_instrument = case config
  when nil
    Internal::Env.auto_instrument
  when false
    false
  when true, Hash
    true
  end

  return unless should_instrument

  only = Internal::Env.instrument_only
  except = Internal::Env.instrument_except

  if config.is_a?(Hash)
    only = config[:only] || only
    except = config[:except] || except
  end

  Contrib.auto_instrument!(only: only, except: except)
end

.current_state ⇒ State?

Get the current global state

Returns:

  • (State, nil) —

    the global state, or nil if not set



76
77
78
 
# File 'lib/braintrust.rb', line 76

def self.current_state
  State.global
end

.init(api_key: nil, org_name: nil, default_project: nil, app_url: nil, api_url: nil, set_global: true, blocking_login: false, enable_tracing: true, tracer_provider: nil, filter_ai_spans: nil, span_filter_funcs: nil, exporter: nil, auto_instrument: nil) ⇒ State

Initialize Braintrust SDK

Parameters:

  • api_key (String, nil) (defaults to: nil) —

    Braintrust API key (overrides BRAINTRUST_API_KEY env var)

  • org_name (String, nil) (defaults to: nil) —

    Organization name (overrides BRAINTRUST_ORG_NAME env var)

  • default_project (String, nil) (defaults to: nil) —

    Default project for spans (overrides BRAINTRUST_DEFAULT_PROJECT env var, format: “project_name:my-project” or “project_id:uuid”)

  • app_url (String, nil) (defaults to: nil) —

    App URL (overrides BRAINTRUST_APP_URL env var, default: www.braintrust.dev)

  • api_url (String, nil) (defaults to: nil) —

    API URL (overrides BRAINTRUST_API_URL env var, default: api.braintrust.dev)

  • set_global (Boolean) (defaults to: true) —

    Whether to set as global state (default: true)

  • blocking_login (Boolean) (defaults to: false) —

    Whether to block and login synchronously (default: false - async background login)

  • enable_tracing (Boolean) (defaults to: true) —

    Whether to enable OpenTelemetry tracing (default: true)

  • tracer_provider (TracerProvider, nil) (defaults to: nil) —

    Optional tracer provider to use instead of creating one

  • filter_ai_spans (Boolean, nil) (defaults to: nil) —

    Enable AI span filtering (overrides BRAINTRUST_OTEL_FILTER_AI_SPANS env var)

  • span_filter_funcs (Array<Proc>, nil) (defaults to: nil) —

    Custom span filter functions

  • exporter (Exporter, nil) (defaults to: nil) —

    Optional exporter override (for testing)

  • auto_instrument (Boolean, Hash, nil) (defaults to: nil) —

    Auto-instrumentation config:

    • nil (default): use BRAINTRUST_AUTO_INSTRUMENT env var, default true if not set

    • true: explicitly enable

    • false: explicitly disable

    • Hash with :only or :except keys for filtering

Returns:

  • (State) —

    the created state



52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
 
# File 'lib/braintrust.rb', line 52

def self.init(api_key: nil, org_name: nil, default_project: nil, app_url: nil, api_url: nil, set_global: true, blocking_login: false, enable_tracing: true, tracer_provider: nil, filter_ai_spans: nil, span_filter_funcs: nil, exporter: nil, auto_instrument: nil)
  state = State.from_env(
    api_key: api_key,
    org_name: org_name,
    default_project: default_project,
    app_url: app_url,
    api_url: api_url,
    blocking_login: ,
    enable_tracing: enable_tracing,
    tracer_provider: tracer_provider,
    filter_ai_spans: filter_ai_spans,
    span_filter_funcs: span_filter_funcs,
    exporter: exporter
  )

  State.global = state if set_global

  auto_instrument!(auto_instrument)

  state
end

.instrument!(name, **options) ⇒ void

This method returns an undefined value.

Instrument a registered integration by name. This is the main entry point for activating integrations.

Examples:

Instrument all OpenAI clients

Braintrust.instrument!(:openai)

Instrument specific OpenAI client instance

client = OpenAI::Client.new
Braintrust.instrument!(:openai, target: client, tracer_provider: my_provider)

Parameters:

  • name (Symbol) —

    The integration name (e.g., :openai, :anthropic)

  • options (Hash) —

    Optional configuration

Options Hash (**options):

  • :target (Object) —

    Optional target instance to instrument specifically

  • :tracer_provider (OpenTelemetry::SDK::Trace::TracerProvider) —

    Optional tracer provider



80
81
82
 
# File 'lib/braintrust/contrib.rb', line 80

def self.instrument!(name, **options)
  Contrib.instrument!(name, **options)
end