Class: Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1EvaluateInstancesRequest

Inherits:
Object
  • Object
show all
Includes:
Core::Hashable, Core::JsonObjectSupport
Defined in:
lib/google/apis/aiplatform_v1beta1/classes.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb

Overview

Request message for EvaluationService.EvaluateInstances.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(**args) ⇒ GoogleCloudAiplatformV1beta1EvaluateInstancesRequest

Returns a new instance of GoogleCloudAiplatformV1beta1EvaluateInstancesRequest.



15271
15272
15273
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15271

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

#autorater_config ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1AutoraterConfig

The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. Corresponds to the JSON property autoraterConfig

Returns:



15082
15083
15084
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15082

def autorater_config
  @autorater_config
end

#bleu_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1BleuInput

Input for bleu metric. Corresponds to the JSON property bleuInput

Returns:



15087
15088
15089
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15087

def bleu_input
  @bleu_input
end

#coherence_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1CoherenceInput

Input for coherence metric. Corresponds to the JSON property coherenceInput

Returns:



15092
15093
15094
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15092

def coherence_input
  @coherence_input
end

#comet_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1CometInput

Input for Comet metric. Corresponds to the JSON property cometInput

Returns:



15097
15098
15099
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15097

def comet_input
  @comet_input
end

#exact_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ExactMatchInput

Input for exact match metric. Corresponds to the JSON property exactMatchInput

Returns:



15102
15103
15104
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15102

def exact_match_input
  @exact_match_input
end

#fluency_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1FluencyInput

Input for fluency metric. Corresponds to the JSON property fluencyInput

Returns:



15107
15108
15109
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15107

def fluency_input
  @fluency_input
end

#fulfillment_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1FulfillmentInput

Input for fulfillment metric. Corresponds to the JSON property fulfillmentInput

Returns:



15112
15113
15114
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15112

def fulfillment_input
  @fulfillment_input
end

#groundedness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1GroundednessInput

Input for groundedness metric. Corresponds to the JSON property groundednessInput

Returns:



15117
15118
15119
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15117

def groundedness_input
  @groundedness_input
end

#instance ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1EvaluationInstance

A single instance to be evaluated. Instances are used to specify the input data for evaluation, from simple string comparisons to complex, multi-turn model evaluations Corresponds to the JSON property instance

Returns:



15124
15125
15126
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15124

def instance
  @instance
end

#location ⇒ String

Required. The resource name of the Location to evaluate the instances. Format: projects/project/locations/location` Corresponds to the JSON propertylocation`

Returns:

  • (String) 


15130
15131
15132
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15130

def location
  @location
end

#metric_sources ⇒ Array<Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1MetricSource>

Optional. The metrics (either inline or registered) used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated. Corresponds to the JSON property metricSources

Returns:



15137
15138
15139
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15137

def metric_sources
  @metric_sources
end

#metrics ⇒ Array<Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1Metric>

The metrics used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated. Corresponds to the JSON property metrics

Returns:



15144
15145
15146
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15144

def metrics
  @metrics
end

#metricx_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1MetricxInput

Input for MetricX metric. Corresponds to the JSON property metricxInput

Returns:



15149
15150
15151
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15149

def metricx_input
  @metricx_input
end

#pairwise_metric_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PairwiseMetricInput

Input for pairwise metric. Corresponds to the JSON property pairwiseMetricInput

Returns:



15154
15155
15156
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15154

def pairwise_metric_input
  @pairwise_metric_input
end

#pairwise_question_answering_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PairwiseQuestionAnsweringQualityInput

Input for pairwise question answering quality metric. Corresponds to the JSON property pairwiseQuestionAnsweringQualityInput

Returns:



15159
15160
15161
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15159

def pairwise_question_answering_quality_input
  @pairwise_question_answering_quality_input
end

#pairwise_summarization_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PairwiseSummarizationQualityInput

Input for pairwise summarization quality metric. Corresponds to the JSON property pairwiseSummarizationQualityInput

Returns:



15164
15165
15166
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15164

def pairwise_summarization_quality_input
  @pairwise_summarization_quality_input
end

#pointwise_metric_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PointwiseMetricInput

Input for pointwise metric. Corresponds to the JSON property pointwiseMetricInput

Returns:



15169
15170
15171
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15169

def pointwise_metric_input
  @pointwise_metric_input
end

#question_answering_correctness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringCorrectnessInput

Input for question answering correctness metric. Corresponds to the JSON property questionAnsweringCorrectnessInput

Returns:



15174
15175
15176
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15174

def question_answering_correctness_input
  @question_answering_correctness_input
end

#question_answering_helpfulness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringHelpfulnessInput

Input for question answering helpfulness metric. Corresponds to the JSON property questionAnsweringHelpfulnessInput

Returns:



15179
15180
15181
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15179

def question_answering_helpfulness_input
  @question_answering_helpfulness_input
end

#question_answering_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringQualityInput

Input for question answering quality metric. Corresponds to the JSON property questionAnsweringQualityInput

Returns:



15184
15185
15186
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15184

def question_answering_quality_input
  @question_answering_quality_input
end

#question_answering_relevance_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringRelevanceInput

Input for question answering relevance metric. Corresponds to the JSON property questionAnsweringRelevanceInput

Returns:



15189
15190
15191
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15189

def question_answering_relevance_input
  @question_answering_relevance_input
end

#rouge_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1RougeInput

Input for rouge metric. Corresponds to the JSON property rougeInput

Returns:



15194
15195
15196
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15194

def rouge_input
  @rouge_input
end

#rubric_based_instruction_following_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1RubricBasedInstructionFollowingInput

Instance and metric spec for RubricBasedInstructionFollowing metric. Corresponds to the JSON property rubricBasedInstructionFollowingInput

Returns:



15199
15200
15201
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15199

def rubric_based_instruction_following_input
  @rubric_based_instruction_following_input
end

#safety_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SafetyInput

Input for safety metric. Corresponds to the JSON property safetyInput

Returns:



15204
15205
15206
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15204

def safety_input
  @safety_input
end

#summarization_helpfulness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SummarizationHelpfulnessInput

Input for summarization helpfulness metric. Corresponds to the JSON property summarizationHelpfulnessInput

Returns:



15209
15210
15211
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15209

def summarization_helpfulness_input
  @summarization_helpfulness_input
end

#summarization_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SummarizationQualityInput

Input for summarization quality metric. Corresponds to the JSON property summarizationQualityInput

Returns:



15214
15215
15216
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15214

def summarization_quality_input
  @summarization_quality_input
end

#summarization_verbosity_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SummarizationVerbosityInput

Input for summarization verbosity metric. Corresponds to the JSON property summarizationVerbosityInput

Returns:



15219
15220
15221
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15219

def summarization_verbosity_input
  @summarization_verbosity_input
end

#tool_call_valid_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolCallValidInput

Input for tool call valid metric. Corresponds to the JSON property toolCallValidInput

Returns:



15224
15225
15226
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15224

def tool_call_valid_input
  @tool_call_valid_input
end

#tool_name_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolNameMatchInput

Input for tool name match metric. Corresponds to the JSON property toolNameMatchInput

Returns:



15229
15230
15231
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15229

def tool_name_match_input
  @tool_name_match_input
end

#tool_parameter_key_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolParameterKeyMatchInput

Input for tool parameter key match metric. Corresponds to the JSON property toolParameterKeyMatchInput

Returns:



15234
15235
15236
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15234

def tool_parameter_key_match_input
  @tool_parameter_key_match_input
end

#tool_parameter_kv_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolParameterKvMatchInput

Input for tool parameter key value match metric. Corresponds to the JSON property toolParameterKvMatchInput

Returns:



15239
15240
15241
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15239

def tool_parameter_kv_match_input
  @tool_parameter_kv_match_input
end

#trajectory_any_order_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryAnyOrderMatchInput

Instances and metric spec for TrajectoryAnyOrderMatch metric. Corresponds to the JSON property trajectoryAnyOrderMatchInput

Returns:



15244
15245
15246
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15244

def trajectory_any_order_match_input
  @trajectory_any_order_match_input
end

#trajectory_exact_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryExactMatchInput

Instances and metric spec for TrajectoryExactMatch metric. Corresponds to the JSON property trajectoryExactMatchInput

Returns:



15249
15250
15251
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15249

def trajectory_exact_match_input
  @trajectory_exact_match_input
end

#trajectory_in_order_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryInOrderMatchInput

Instances and metric spec for TrajectoryInOrderMatch metric. Corresponds to the JSON property trajectoryInOrderMatchInput

Returns:



15254
15255
15256
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15254

def trajectory_in_order_match_input
  @trajectory_in_order_match_input
end

#trajectory_precision_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryPrecisionInput

Instances and metric spec for TrajectoryPrecision metric. Corresponds to the JSON property trajectoryPrecisionInput

Returns:



15259
15260
15261
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15259

def trajectory_precision_input
  @trajectory_precision_input
end

#trajectory_recall_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryRecallInput

Instances and metric spec for TrajectoryRecall metric. Corresponds to the JSON property trajectoryRecallInput

Returns:



15264
15265
15266
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15264

def trajectory_recall_input
  @trajectory_recall_input
end

#trajectory_single_tool_use_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectorySingleToolUseInput

Instances and metric spec for TrajectorySingleToolUse metric. Corresponds to the JSON property trajectorySingleToolUseInput

Returns:



15269
15270
15271
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15269

def trajectory_single_tool_use_input
  @trajectory_single_tool_use_input
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object



15276
15277
15278
15279
15280
15281
15282
15283
15284
15285
15286
15287
15288
15289
15290
15291
15292
15293
15294
15295
15296
15297
15298
15299
15300
15301
15302
15303
15304
15305
15306
15307
15308
15309
15310
15311
15312
15313
15314
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15276

def update!(**args)
  @autorater_config = args[:autorater_config] if args.key?(:autorater_config)
  @bleu_input = args[:bleu_input] if args.key?(:bleu_input)
  @coherence_input = args[:coherence_input] if args.key?(:coherence_input)
  @comet_input = args[:comet_input] if args.key?(:comet_input)
  @exact_match_input = args[:exact_match_input] if args.key?(:exact_match_input)
  @fluency_input = args[:fluency_input] if args.key?(:fluency_input)
  @fulfillment_input = args[:fulfillment_input] if args.key?(:fulfillment_input)
  @groundedness_input = args[:groundedness_input] if args.key?(:groundedness_input)
  @instance = args[:instance] if args.key?(:instance)
  @location = args[:location] if args.key?(:location)
  @metric_sources = args[:metric_sources] if args.key?(:metric_sources)
  @metrics = args[:metrics] if args.key?(:metrics)
  @metricx_input = args[:metricx_input] if args.key?(:metricx_input)
  @pairwise_metric_input = args[:pairwise_metric_input] if args.key?(:pairwise_metric_input)
  @pairwise_question_answering_quality_input = args[:pairwise_question_answering_quality_input] if args.key?(:pairwise_question_answering_quality_input)
  @pairwise_summarization_quality_input = args[:pairwise_summarization_quality_input] if args.key?(:pairwise_summarization_quality_input)
  @pointwise_metric_input = args[:pointwise_metric_input] if args.key?(:pointwise_metric_input)
  @question_answering_correctness_input = args[:question_answering_correctness_input] if args.key?(:question_answering_correctness_input)
  @question_answering_helpfulness_input = args[:question_answering_helpfulness_input] if args.key?(:question_answering_helpfulness_input)
  @question_answering_quality_input = args[:question_answering_quality_input] if args.key?(:question_answering_quality_input)
  @question_answering_relevance_input = args[:question_answering_relevance_input] if args.key?(:question_answering_relevance_input)
  @rouge_input = args[:rouge_input] if args.key?(:rouge_input)
  @rubric_based_instruction_following_input = args[:rubric_based_instruction_following_input] if args.key?(:rubric_based_instruction_following_input)
  @safety_input = args[:safety_input] if args.key?(:safety_input)
  @summarization_helpfulness_input = args[:summarization_helpfulness_input] if args.key?(:summarization_helpfulness_input)
  @summarization_quality_input = args[:summarization_quality_input] if args.key?(:summarization_quality_input)
  @summarization_verbosity_input = args[:summarization_verbosity_input] if args.key?(:summarization_verbosity_input)
  @tool_call_valid_input = args[:tool_call_valid_input] if args.key?(:tool_call_valid_input)
  @tool_name_match_input = args[:tool_name_match_input] if args.key?(:tool_name_match_input)
  @tool_parameter_key_match_input = args[:tool_parameter_key_match_input] if args.key?(:tool_parameter_key_match_input)
  @tool_parameter_kv_match_input = args[:tool_parameter_kv_match_input] if args.key?(:tool_parameter_kv_match_input)
  @trajectory_any_order_match_input = args[:trajectory_any_order_match_input] if args.key?(:trajectory_any_order_match_input)
  @trajectory_exact_match_input = args[:trajectory_exact_match_input] if args.key?(:trajectory_exact_match_input)
  @trajectory_in_order_match_input = args[:trajectory_in_order_match_input] if args.key?(:trajectory_in_order_match_input)
  @trajectory_precision_input = args[:trajectory_precision_input] if args.key?(:trajectory_precision_input)
  @trajectory_recall_input = args[:trajectory_recall_input] if args.key?(:trajectory_recall_input)
  @trajectory_single_tool_use_input = args[:trajectory_single_tool_use_input] if args.key?(:trajectory_single_tool_use_input)
end