Class: OpenAI::Resources::Evals

Inherits:
Object
  • Object
show all
Defined in:
lib/openai/resources/evals.rb,
lib/openai/resources/evals/runs.rb,
lib/openai/resources/evals/runs/output_items.rb

Overview

Manage and run evals in the OpenAI platform.

Defined Under Namespace

Classes: Runs

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(client:) ⇒ Evals

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of Evals.

Parameters:



152
153
154
155
 
# File 'lib/openai/resources/evals.rb', line 152

def initialize(client:)
  @client = client
  @runs = OpenAI::Resources::Evals::Runs.new(client: client)
end

Instance Attribute Details

#runs ⇒ OpenAI::Resources::Evals::Runs (readonly)

Manage and run evals in the OpenAI platform.

Returns:



9
10
11
 
# File 'lib/openai/resources/evals.rb', line 9

def runs
  @runs
end

Instance Method Details

#create(data_source_config:, testing_criteria:, metadata: nil, name: nil, request_options: {}) ⇒ OpenAI::Models::EvalCreateResponse

Some parameter documentations has been truncated, see Models::EvalCreateParams for more details.

Create the structure of an evaluation that can be used to test a model’s performance. An evaluation is a set of testing criteria and the config for a data source, which dictates the schema of the data used in the evaluation. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the [Evals guide](platform.openai.com/docs/guides/evals).

Parameters:

Returns:

See Also:



36
37
38
39
40
41
42
43
44
45
 
# File 'lib/openai/resources/evals.rb', line 36

def create(params)
  parsed, options = OpenAI::EvalCreateParams.dump_request(params)
  @client.request(
    method: :post,
    path: "evals",
    body: parsed,
    model: OpenAI::Models::EvalCreateResponse,
    options: options
  )
end

#delete(eval_id, request_options: {}) ⇒ OpenAI::Models::EvalDeleteResponse

Delete an evaluation.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to delete.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



140
141
142
143
144
145
146
147
 
# File 'lib/openai/resources/evals.rb', line 140

def delete(eval_id, params = {})
  @client.request(
    method: :delete,
    path: ["evals/%1$s", eval_id],
    model: OpenAI::Models::EvalDeleteResponse,
    options: params[:request_options]
  )
end

#list(after: nil, limit: nil, order: nil, order_by: nil, request_options: {}) ⇒ OpenAI::Internal::CursorPage<OpenAI::Models::EvalListResponse>

Some parameter documentations has been truncated, see Models::EvalListParams for more details.

List evaluations for a project.

Parameters:

Returns:

See Also:



116
117
118
119
120
121
122
123
124
125
126
127
 
# File 'lib/openai/resources/evals.rb', line 116

def list(params = {})
  parsed, options = OpenAI::EvalListParams.dump_request(params)
  query = OpenAI::Internal::Util.encode_query_params(parsed)
  @client.request(
    method: :get,
    path: "evals",
    query: query,
    page: OpenAI::Internal::CursorPage,
    model: OpenAI::Models::EvalListResponse,
    options: options
  )
end

#retrieve(eval_id, request_options: {}) ⇒ OpenAI::Models::EvalRetrieveResponse

Get an evaluation by ID.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to retrieve.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



58
59
60
61
62
63
64
65
 
# File 'lib/openai/resources/evals.rb', line 58

def retrieve(eval_id, params = {})
  @client.request(
    method: :get,
    path: ["evals/%1$s", eval_id],
    model: OpenAI::Models::EvalRetrieveResponse,
    options: params[:request_options]
  )
end

#update(eval_id, metadata: nil, name: nil, request_options: {}) ⇒ OpenAI::Models::EvalUpdateResponse

Some parameter documentations has been truncated, see Models::EvalUpdateParams for more details.

Update certain properties of an evaluation.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to update.

  • metadata (Hash{Symbol=>String}, nil) —

    Set of 16 key-value pairs that can be attached to an object. This can be

  • name (String) —

    Rename the evaluation.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



85
86
87
88
89
90
91
92
93
94
 
# File 'lib/openai/resources/evals.rb', line 85

def update(eval_id, params = {})
  parsed, options = OpenAI::EvalUpdateParams.dump_request(params)
  @client.request(
    method: :post,
    path: ["evals/%1$s", eval_id],
    body: parsed,
    model: OpenAI::Models::EvalUpdateResponse,
    options: options
  )
end