Class: Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1EvaluateInstancesRequest

Inherits:
Object
  • Object
show all
Includes:
Core::Hashable, Core::JsonObjectSupport
Defined in:
lib/google/apis/aiplatform_v1beta1/classes.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb

Overview

Request message for EvaluationService.EvaluateInstances.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(**args) ⇒ GoogleCloudAiplatformV1beta1EvaluateInstancesRequest

Returns a new instance of GoogleCloudAiplatformV1beta1EvaluateInstancesRequest.



16550
16551
16552
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16550

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

#autorater_config ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1AutoraterConfig

The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. Corresponds to the JSON property autoraterConfig

Returns:



16361
16362
16363
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16361

def autorater_config
  @autorater_config
end

#bleu_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1BleuInput

Input for bleu metric. Corresponds to the JSON property bleuInput

Returns:



16366
16367
16368
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16366

def bleu_input
  @bleu_input
end

#coherence_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1CoherenceInput

Input for coherence metric. Corresponds to the JSON property coherenceInput

Returns:



16371
16372
16373
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16371

def coherence_input
  @coherence_input
end

#comet_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1CometInput

Input for Comet metric. Corresponds to the JSON property cometInput

Returns:



16376
16377
16378
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16376

def comet_input
  @comet_input
end

#exact_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ExactMatchInput

Input for exact match metric. Corresponds to the JSON property exactMatchInput

Returns:



16381
16382
16383
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16381

def exact_match_input
  @exact_match_input
end

#fluency_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1FluencyInput

Input for fluency metric. Corresponds to the JSON property fluencyInput

Returns:



16386
16387
16388
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16386

def fluency_input
  @fluency_input
end

#fulfillment_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1FulfillmentInput

Input for fulfillment metric. Corresponds to the JSON property fulfillmentInput

Returns:



16391
16392
16393
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16391

def fulfillment_input
  @fulfillment_input
end

#groundedness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1GroundednessInput

Input for groundedness metric. Corresponds to the JSON property groundednessInput

Returns:



16396
16397
16398
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16396

def groundedness_input
  @groundedness_input
end

#instance ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1EvaluationInstance

A single instance to be evaluated. Instances are used to specify the input data for evaluation, from simple string comparisons to complex, multi-turn model evaluations Corresponds to the JSON property instance

Returns:



16403
16404
16405
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16403

def instance
  @instance
end

#location ⇒ String

Required. The resource name of the Location to evaluate the instances. Format: projects/project/locations/location` Corresponds to the JSON propertylocation`

Returns:

  • (String) 


16409
16410
16411
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16409

def location
  @location
end

#metric_sources ⇒ Array<Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1MetricSource>

Optional. The metrics (either inline or registered) used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated. Corresponds to the JSON property metricSources

Returns:



16416
16417
16418
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16416

def metric_sources
  @metric_sources
end

#metrics ⇒ Array<Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1Metric>

The metrics used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated. Corresponds to the JSON property metrics

Returns:



16423
16424
16425
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16423

def metrics
  @metrics
end

#metricx_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1MetricxInput

Input for MetricX metric. Corresponds to the JSON property metricxInput

Returns:



16428
16429
16430
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16428

def metricx_input
  @metricx_input
end

#pairwise_metric_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PairwiseMetricInput

Input for pairwise metric. Corresponds to the JSON property pairwiseMetricInput

Returns:



16433
16434
16435
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16433

def pairwise_metric_input
  @pairwise_metric_input
end

#pairwise_question_answering_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PairwiseQuestionAnsweringQualityInput

Input for pairwise question answering quality metric. Corresponds to the JSON property pairwiseQuestionAnsweringQualityInput

Returns:



16438
16439
16440
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16438

def pairwise_question_answering_quality_input
  @pairwise_question_answering_quality_input
end

#pairwise_summarization_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PairwiseSummarizationQualityInput

Input for pairwise summarization quality metric. Corresponds to the JSON property pairwiseSummarizationQualityInput

Returns:



16443
16444
16445
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16443

def pairwise_summarization_quality_input
  @pairwise_summarization_quality_input
end

#pointwise_metric_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PointwiseMetricInput

Input for pointwise metric. Corresponds to the JSON property pointwiseMetricInput

Returns:



16448
16449
16450
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16448

def pointwise_metric_input
  @pointwise_metric_input
end

#question_answering_correctness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringCorrectnessInput

Input for question answering correctness metric. Corresponds to the JSON property questionAnsweringCorrectnessInput

Returns:



16453
16454
16455
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16453

def question_answering_correctness_input
  @question_answering_correctness_input
end

#question_answering_helpfulness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringHelpfulnessInput

Input for question answering helpfulness metric. Corresponds to the JSON property questionAnsweringHelpfulnessInput

Returns:



16458
16459
16460
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16458

def question_answering_helpfulness_input
  @question_answering_helpfulness_input
end

#question_answering_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringQualityInput

Input for question answering quality metric. Corresponds to the JSON property questionAnsweringQualityInput

Returns:



16463
16464
16465
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16463

def question_answering_quality_input
  @question_answering_quality_input
end

#question_answering_relevance_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringRelevanceInput

Input for question answering relevance metric. Corresponds to the JSON property questionAnsweringRelevanceInput

Returns:



16468
16469
16470
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16468

def question_answering_relevance_input
  @question_answering_relevance_input
end

#rouge_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1RougeInput

Input for rouge metric. Corresponds to the JSON property rougeInput

Returns:



16473
16474
16475
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16473

def rouge_input
  @rouge_input
end

#rubric_based_instruction_following_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1RubricBasedInstructionFollowingInput

Instance and metric spec for RubricBasedInstructionFollowing metric. Corresponds to the JSON property rubricBasedInstructionFollowingInput

Returns:



16478
16479
16480
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16478

def rubric_based_instruction_following_input
  @rubric_based_instruction_following_input
end

#safety_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SafetyInput

Input for safety metric. Corresponds to the JSON property safetyInput

Returns:



16483
16484
16485
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16483

def safety_input
  @safety_input
end

#summarization_helpfulness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SummarizationHelpfulnessInput

Input for summarization helpfulness metric. Corresponds to the JSON property summarizationHelpfulnessInput

Returns:



16488
16489
16490
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16488

def summarization_helpfulness_input
  @summarization_helpfulness_input
end

#summarization_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SummarizationQualityInput

Input for summarization quality metric. Corresponds to the JSON property summarizationQualityInput

Returns:



16493
16494
16495
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16493

def summarization_quality_input
  @summarization_quality_input
end

#summarization_verbosity_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SummarizationVerbosityInput

Input for summarization verbosity metric. Corresponds to the JSON property summarizationVerbosityInput

Returns:



16498
16499
16500
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16498

def summarization_verbosity_input
  @summarization_verbosity_input
end

#tool_call_valid_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolCallValidInput

Input for tool call valid metric. Corresponds to the JSON property toolCallValidInput

Returns:



16503
16504
16505
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16503

def tool_call_valid_input
  @tool_call_valid_input
end

#tool_name_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolNameMatchInput

Input for tool name match metric. Corresponds to the JSON property toolNameMatchInput

Returns:



16508
16509
16510
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16508

def tool_name_match_input
  @tool_name_match_input
end

#tool_parameter_key_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolParameterKeyMatchInput

Input for tool parameter key match metric. Corresponds to the JSON property toolParameterKeyMatchInput

Returns:



16513
16514
16515
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16513

def tool_parameter_key_match_input
  @tool_parameter_key_match_input
end

#tool_parameter_kv_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolParameterKvMatchInput

Input for tool parameter key value match metric. Corresponds to the JSON property toolParameterKvMatchInput

Returns:



16518
16519
16520
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16518

def tool_parameter_kv_match_input
  @tool_parameter_kv_match_input
end

#trajectory_any_order_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryAnyOrderMatchInput

Instances and metric spec for TrajectoryAnyOrderMatch metric. Corresponds to the JSON property trajectoryAnyOrderMatchInput

Returns:



16523
16524
16525
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16523

def trajectory_any_order_match_input
  @trajectory_any_order_match_input
end

#trajectory_exact_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryExactMatchInput

Instances and metric spec for TrajectoryExactMatch metric. Corresponds to the JSON property trajectoryExactMatchInput

Returns:



16528
16529
16530
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16528

def trajectory_exact_match_input
  @trajectory_exact_match_input
end

#trajectory_in_order_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryInOrderMatchInput

Instances and metric spec for TrajectoryInOrderMatch metric. Corresponds to the JSON property trajectoryInOrderMatchInput

Returns:



16533
16534
16535
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16533

def trajectory_in_order_match_input
  @trajectory_in_order_match_input
end

#trajectory_precision_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryPrecisionInput

Instances and metric spec for TrajectoryPrecision metric. Corresponds to the JSON property trajectoryPrecisionInput

Returns:



16538
16539
16540
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16538

def trajectory_precision_input
  @trajectory_precision_input
end

#trajectory_recall_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryRecallInput

Instances and metric spec for TrajectoryRecall metric. Corresponds to the JSON property trajectoryRecallInput

Returns:



16543
16544
16545
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16543

def trajectory_recall_input
  @trajectory_recall_input
end

#trajectory_single_tool_use_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectorySingleToolUseInput

Instances and metric spec for TrajectorySingleToolUse metric. Corresponds to the JSON property trajectorySingleToolUseInput

Returns:



16548
16549
16550
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16548

def trajectory_single_tool_use_input
  @trajectory_single_tool_use_input
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object



16555
16556
16557
16558
16559
16560
16561
16562
16563
16564
16565
16566
16567
16568
16569
16570
16571
16572
16573
16574
16575
16576
16577
16578
16579
16580
16581
16582
16583
16584
16585
16586
16587
16588
16589
16590
16591
16592
16593
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 16555

def update!(**args)
  @autorater_config = args[:autorater_config] if args.key?(:autorater_config)
  @bleu_input = args[:bleu_input] if args.key?(:bleu_input)
  @coherence_input = args[:coherence_input] if args.key?(:coherence_input)
  @comet_input = args[:comet_input] if args.key?(:comet_input)
  @exact_match_input = args[:exact_match_input] if args.key?(:exact_match_input)
  @fluency_input = args[:fluency_input] if args.key?(:fluency_input)
  @fulfillment_input = args[:fulfillment_input] if args.key?(:fulfillment_input)
  @groundedness_input = args[:groundedness_input] if args.key?(:groundedness_input)
  @instance = args[:instance] if args.key?(:instance)
  @location = args[:location] if args.key?(:location)
  @metric_sources = args[:metric_sources] if args.key?(:metric_sources)
  @metrics = args[:metrics] if args.key?(:metrics)
  @metricx_input = args[:metricx_input] if args.key?(:metricx_input)
  @pairwise_metric_input = args[:pairwise_metric_input] if args.key?(:pairwise_metric_input)
  @pairwise_question_answering_quality_input = args[:pairwise_question_answering_quality_input] if args.key?(:pairwise_question_answering_quality_input)
  @pairwise_summarization_quality_input = args[:pairwise_summarization_quality_input] if args.key?(:pairwise_summarization_quality_input)
  @pointwise_metric_input = args[:pointwise_metric_input] if args.key?(:pointwise_metric_input)
  @question_answering_correctness_input = args[:question_answering_correctness_input] if args.key?(:question_answering_correctness_input)
  @question_answering_helpfulness_input = args[:question_answering_helpfulness_input] if args.key?(:question_answering_helpfulness_input)
  @question_answering_quality_input = args[:question_answering_quality_input] if args.key?(:question_answering_quality_input)
  @question_answering_relevance_input = args[:question_answering_relevance_input] if args.key?(:question_answering_relevance_input)
  @rouge_input = args[:rouge_input] if args.key?(:rouge_input)
  @rubric_based_instruction_following_input = args[:rubric_based_instruction_following_input] if args.key?(:rubric_based_instruction_following_input)
  @safety_input = args[:safety_input] if args.key?(:safety_input)
  @summarization_helpfulness_input = args[:summarization_helpfulness_input] if args.key?(:summarization_helpfulness_input)
  @summarization_quality_input = args[:summarization_quality_input] if args.key?(:summarization_quality_input)
  @summarization_verbosity_input = args[:summarization_verbosity_input] if args.key?(:summarization_verbosity_input)
  @tool_call_valid_input = args[:tool_call_valid_input] if args.key?(:tool_call_valid_input)
  @tool_name_match_input = args[:tool_name_match_input] if args.key?(:tool_name_match_input)
  @tool_parameter_key_match_input = args[:tool_parameter_key_match_input] if args.key?(:tool_parameter_key_match_input)
  @tool_parameter_kv_match_input = args[:tool_parameter_kv_match_input] if args.key?(:tool_parameter_kv_match_input)
  @trajectory_any_order_match_input = args[:trajectory_any_order_match_input] if args.key?(:trajectory_any_order_match_input)
  @trajectory_exact_match_input = args[:trajectory_exact_match_input] if args.key?(:trajectory_exact_match_input)
  @trajectory_in_order_match_input = args[:trajectory_in_order_match_input] if args.key?(:trajectory_in_order_match_input)
  @trajectory_precision_input = args[:trajectory_precision_input] if args.key?(:trajectory_precision_input)
  @trajectory_recall_input = args[:trajectory_recall_input] if args.key?(:trajectory_recall_input)
  @trajectory_single_tool_use_input = args[:trajectory_single_tool_use_input] if args.key?(:trajectory_single_tool_use_input)
end