Class: OpenAI::Resources::Evals

Inherits:
Object
  • Object
show all
Defined in:
lib/openai/resources/evals.rb,
lib/openai/resources/evals/runs.rb,
lib/openai/resources/evals/runs/output_items.rb

Defined Under Namespace

Classes: Runs

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(client:) ⇒ Evals

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of Evals.

Parameters:



149
150
151
152
 
# File 'lib/openai/resources/evals.rb', line 149

def initialize(client:)
  @client = client
  @runs = OpenAI::Resources::Evals::Runs.new(client: client)
end

Instance Attribute Details

#runs ⇒ OpenAI::Resources::Evals::Runs (readonly)

Returns:



7
8
9
 
# File 'lib/openai/resources/evals.rb', line 7

def runs
  @runs
end

Instance Method Details

#create(data_source_config: , testing_criteria: , metadata: nil, name: nil, request_options: {}) ⇒ OpenAI::Models::EvalCreateResponse

Some parameter documentations has been truncated, see Models::EvalCreateParams for more details.

Create the structure of an evaluation that can be used to test a model’s performance. An evaluation is a set of testing criteria and the config for a data source, which dictates the schema of the data used in the evaluation. After creating an evaluation, you can run it on different models and model parameters. We support several types of graders and datasources. For more information, see the [Evals guide](platform.openai.com/docs/guides/evals).

Parameters:

Returns:

See Also:



34
35
36
37
38
39
40
41
42
43
 
# File 'lib/openai/resources/evals.rb', line 34

def create(params)
  parsed, options = OpenAI::EvalCreateParams.dump_request(params)
  @client.request(
    method: :post,
    path: "evals",
    body: parsed,
    model: OpenAI::Models::EvalCreateResponse,
    options: options
  )
end

#delete(eval_id, request_options: {}) ⇒ OpenAI::Models::EvalDeleteResponse

Delete an evaluation.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to delete.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



137
138
139
140
141
142
143
144
 
# File 'lib/openai/resources/evals.rb', line 137

def delete(eval_id, params = {})
  @client.request(
    method: :delete,
    path: ["evals/%1$s", eval_id],
    model: OpenAI::Models::EvalDeleteResponse,
    options: params[:request_options]
  )
end

#list(after: nil, limit: nil, order: nil, order_by: nil, request_options: {}) ⇒ OpenAI::Internal::CursorPage<OpenAI::Models::EvalListResponse>

Some parameter documentations has been truncated, see Models::EvalListParams for more details.

List evaluations for a project.

Parameters:

Returns:

See Also:



114
115
116
117
118
119
120
121
122
123
124
 
# File 'lib/openai/resources/evals.rb', line 114

def list(params = {})
  parsed, options = OpenAI::EvalListParams.dump_request(params)
  @client.request(
    method: :get,
    path: "evals",
    query: parsed,
    page: OpenAI::Internal::CursorPage,
    model: OpenAI::Models::EvalListResponse,
    options: options
  )
end

#retrieve(eval_id, request_options: {}) ⇒ OpenAI::Models::EvalRetrieveResponse

Get an evaluation by ID.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to retrieve.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



56
57
58
59
60
61
62
63
 
# File 'lib/openai/resources/evals.rb', line 56

def retrieve(eval_id, params = {})
  @client.request(
    method: :get,
    path: ["evals/%1$s", eval_id],
    model: OpenAI::Models::EvalRetrieveResponse,
    options: params[:request_options]
  )
end

#update(eval_id, metadata: nil, name: nil, request_options: {}) ⇒ OpenAI::Models::EvalUpdateResponse

Some parameter documentations has been truncated, see Models::EvalUpdateParams for more details.

Update certain properties of an evaluation.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to update.

  • metadata (Hash{Symbol=>String}, nil) —

    Set of 16 key-value pairs that can be attached to an object. This can be

  • name (String) —

    Rename the evaluation.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



83
84
85
86
87
88
89
90
91
92
 
# File 'lib/openai/resources/evals.rb', line 83

def update(eval_id, params = {})
  parsed, options = OpenAI::EvalUpdateParams.dump_request(params)
  @client.request(
    method: :post,
    path: ["evals/%1$s", eval_id],
    body: parsed,
    model: OpenAI::Models::EvalUpdateResponse,
    options: options
  )
end