Class: OpenAI::Resources::Evals

Inherits:
Object
  • Object
show all
Defined in:
lib/openai/resources/evals.rb,
lib/openai/resources/evals/runs.rb,
lib/openai/resources/evals/runs/output_items.rb

Overview

Manage and run evals in the OpenAI platform.

Defined Under Namespace

Classes: Runs

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(client:) ⇒ Evals

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of Evals.

Parameters:



157
158
159
160
 
# File 'lib/openai/resources/evals.rb', line 157

def initialize(client:)
  @client = client
  @runs = OpenAI::Resources::Evals::Runs.new(client: client)
end

Instance Attribute Details

#runs ⇒ OpenAI::Resources::Evals::Runs (readonly)

Manage and run evals in the OpenAI platform.

Returns:



9
10
11
 
# File 'lib/openai/resources/evals.rb', line 9

def runs
  @runs
end

Instance Method Details

#create(data_source_config:, testing_criteria:, metadata: nil, name: nil, request_options: {}) ⇒ OpenAI::Models::EvalCreateResponse

Some parameter documentations has been truncated, see Models::EvalCreateParams for more details.

Create the structure of an evaluation that can be used to test a model’s performance. An evaluation is a set of testing criteria and the config for a data source, which dictates the schema of the data used in the evaluation. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the [Evals guide](platform.openai.com/docs/guides/evals).

Parameters:

Returns:

See Also:



36
37
38
39
40
41
42
43
44
45
46
 
# File 'lib/openai/resources/evals.rb', line 36

def create(params)
  parsed, options = OpenAI::EvalCreateParams.dump_request(params)
  @client.request(
    method: :post,
    path: "evals",
    body: parsed,
    model: OpenAI::Models::EvalCreateResponse,
    security: {bearer_auth: true},
    options: options
  )
end

#delete(eval_id, request_options: {}) ⇒ OpenAI::Models::EvalDeleteResponse

Delete an evaluation.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to delete.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



144
145
146
147
148
149
150
151
152
 
# File 'lib/openai/resources/evals.rb', line 144

def delete(eval_id, params = {})
  @client.request(
    method: :delete,
    path: ["evals/%1$s", eval_id],
    model: OpenAI::Models::EvalDeleteResponse,
    security: {bearer_auth: true},
    options: params[:request_options]
  )
end

#list(after: nil, limit: nil, order: nil, order_by: nil, request_options: {}) ⇒ OpenAI::Internal::CursorPage<OpenAI::Models::EvalListResponse>

Some parameter documentations has been truncated, see Models::EvalListParams for more details.

List evaluations for a project.

Parameters:

Returns:

See Also:



119
120
121
122
123
124
125
126
127
128
129
130
131
 
# File 'lib/openai/resources/evals.rb', line 119

def list(params = {})
  parsed, options = OpenAI::EvalListParams.dump_request(params)
  query = OpenAI::Internal::Util.encode_query_params(parsed)
  @client.request(
    method: :get,
    path: "evals",
    query: query,
    page: OpenAI::Internal::CursorPage,
    model: OpenAI::Models::EvalListResponse,
    security: {bearer_auth: true},
    options: options
  )
end

#retrieve(eval_id, request_options: {}) ⇒ OpenAI::Models::EvalRetrieveResponse

Get an evaluation by ID.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to retrieve.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



59
60
61
62
63
64
65
66
67
 
# File 'lib/openai/resources/evals.rb', line 59

def retrieve(eval_id, params = {})
  @client.request(
    method: :get,
    path: ["evals/%1$s", eval_id],
    model: OpenAI::Models::EvalRetrieveResponse,
    security: {bearer_auth: true},
    options: params[:request_options]
  )
end

#update(eval_id, metadata: nil, name: nil, request_options: {}) ⇒ OpenAI::Models::EvalUpdateResponse

Some parameter documentations has been truncated, see Models::EvalUpdateParams for more details.

Update certain properties of an evaluation.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to update.

  • metadata (Hash{Symbol=>String}, nil) —

    Set of 16 key-value pairs that can be attached to an object. This can be

  • name (String) —

    Rename the evaluation.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



87
88
89
90
91
92
93
94
95
96
97
 
# File 'lib/openai/resources/evals.rb', line 87

def update(eval_id, params = {})
  parsed, options = OpenAI::EvalUpdateParams.dump_request(params)
  @client.request(
    method: :post,
    path: ["evals/%1$s", eval_id],
    body: parsed,
    model: OpenAI::Models::EvalUpdateResponse,
    security: {bearer_auth: true},
    options: options
  )
end