Class: OpenAI::Resources::Evals::Runs

Inherits:
Object
  • Object
show all
Defined in:
lib/openai/resources/evals/runs.rb,
lib/openai/resources/evals/runs/output_items.rb

Overview

Manage and run evals in the OpenAI platform.

Defined Under Namespace

Classes: OutputItems

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(client:) ⇒ Runs

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of Runs.

Parameters:



164
165
166
167
 
# File 'lib/openai/resources/evals/runs.rb', line 164

def initialize(client:)
  @client = client
  @output_items = OpenAI::Resources::Evals::Runs::OutputItems.new(client: client)
end

Instance Attribute Details

#output_items ⇒ OpenAI::Resources::Evals::Runs::OutputItems (readonly)

Manage and run evals in the OpenAI platform.

Returns:



10
11
12
 
# File 'lib/openai/resources/evals/runs.rb', line 10

def output_items
  @output_items
end

Instance Method Details

#cancel(run_id, eval_id:, request_options: {}) ⇒ OpenAI::Models::Evals::RunCancelResponse

Cancel an ongoing evaluation run.

Parameters:

  • run_id (String) —

    The ID of the run to cancel.

  • eval_id (String) —

    The ID of the evaluation whose run you want to cancel.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



147
148
149
150
151
152
153
154
155
156
157
158
159
 
# File 'lib/openai/resources/evals/runs.rb', line 147

def cancel(run_id, params)
  parsed, options = OpenAI::Evals::RunCancelParams.dump_request(params)
  eval_id =
    parsed.delete(:eval_id) do
      raise ArgumentError.new("missing required path argument #{_1}")
    end
  @client.request(
    method: :post,
    path: ["evals/%1$s/runs/%2$s", eval_id, run_id],
    model: OpenAI::Models::Evals::RunCancelResponse,
    options: options
  )
end

#create(eval_id, data_source:, metadata: nil, name: nil, request_options: {}) ⇒ OpenAI::Models::Evals::RunCreateResponse

Some parameter documentations has been truncated, see Models::Evals::RunCreateParams for more details.

Kicks off a new run for a given evaluation, specifying the data source, and what model configuration to use to test. The datasource will be validated against the schema specified in the config of the evaluation.

Parameters:

Returns:

See Also:



34
35
36
37
38
39
40
41
42
43
 
# File 'lib/openai/resources/evals/runs.rb', line 34

def create(eval_id, params)
  parsed, options = OpenAI::Evals::RunCreateParams.dump_request(params)
  @client.request(
    method: :post,
    path: ["evals/%1$s/runs", eval_id],
    body: parsed,
    model: OpenAI::Models::Evals::RunCreateResponse,
    options: options
  )
end

#delete(run_id, eval_id:, request_options: {}) ⇒ OpenAI::Models::Evals::RunDeleteResponse

Delete an eval run.

Parameters:

  • run_id (String) —

    The ID of the run to delete.

  • eval_id (String) —

    The ID of the evaluation to delete the run from.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



120
121
122
123
124
125
126
127
128
129
130
131
132
 
# File 'lib/openai/resources/evals/runs.rb', line 120

def delete(run_id, params)
  parsed, options = OpenAI::Evals::RunDeleteParams.dump_request(params)
  eval_id =
    parsed.delete(:eval_id) do
      raise ArgumentError.new("missing required path argument #{_1}")
    end
  @client.request(
    method: :delete,
    path: ["evals/%1$s/runs/%2$s", eval_id, run_id],
    model: OpenAI::Models::Evals::RunDeleteResponse,
    options: options
  )
end

#list(eval_id, after: nil, limit: nil, order: nil, status: nil, request_options: {}) ⇒ OpenAI::Internal::CursorPage<OpenAI::Models::Evals::RunListResponse>

Some parameter documentations has been truncated, see Models::Evals::RunListParams for more details.

Get a list of runs for an evaluation.

Parameters:

  • eval_id (String) —

    The ID of the evaluation to retrieve runs for.

  • after (String) —

    Identifier for the last run from the previous pagination request.

  • limit (Integer) —

    Number of runs to retrieve.

  • order (Symbol, OpenAI::Models::Evals::RunListParams::Order) —

    Sort order for runs by timestamp. Use ‘asc` for ascending order or `desc` for de

  • status (Symbol, OpenAI::Models::Evals::RunListParams::Status) —

    Filter runs by status. One of ‘queued` | `in_progress` | `failed` | `completed`

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



94
95
96
97
98
99
100
101
102
103
104
105
 
# File 'lib/openai/resources/evals/runs.rb', line 94

def list(eval_id, params = {})
  parsed, options = OpenAI::Evals::RunListParams.dump_request(params)
  query = OpenAI::Internal::Util.encode_query_params(parsed)
  @client.request(
    method: :get,
    path: ["evals/%1$s/runs", eval_id],
    query: query,
    page: OpenAI::Internal::CursorPage,
    model: OpenAI::Models::Evals::RunListResponse,
    options: options
  )
end

#retrieve(run_id, eval_id:, request_options: {}) ⇒ OpenAI::Models::Evals::RunRetrieveResponse

Get an evaluation run by ID.

Parameters:

  • run_id (String) —

    The ID of the run to retrieve.

  • eval_id (String) —

    The ID of the evaluation to retrieve runs for.

  • request_options (OpenAI::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



58
59
60
61
62
63
64
65
66
67
68
69
70
 
# File 'lib/openai/resources/evals/runs.rb', line 58

def retrieve(run_id, params)
  parsed, options = OpenAI::Evals::RunRetrieveParams.dump_request(params)
  eval_id =
    parsed.delete(:eval_id) do
      raise ArgumentError.new("missing required path argument #{_1}")
    end
  @client.request(
    method: :get,
    path: ["evals/%1$s/runs/%2$s", eval_id, run_id],
    model: OpenAI::Models::Evals::RunRetrieveResponse,
    options: options
  )
end