Class: Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1EvaluateInstancesRequest

Inherits:
Object
  • Object
show all
Includes:
Core::Hashable, Core::JsonObjectSupport
Defined in:
lib/google/apis/aiplatform_v1beta1/classes.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb

Overview

Request message for EvaluationService.EvaluateInstances.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(**args) ⇒ GoogleCloudAiplatformV1beta1EvaluateInstancesRequest

Returns a new instance of GoogleCloudAiplatformV1beta1EvaluateInstancesRequest.



15762
15763
15764
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15762

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

#autorater_config ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1AutoraterConfig

The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. Corresponds to the JSON property autoraterConfig

Returns:



15573
15574
15575
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15573

def autorater_config
  @autorater_config
end

#bleu_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1BleuInput

Input for bleu metric. Corresponds to the JSON property bleuInput

Returns:



15578
15579
15580
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15578

def bleu_input
  @bleu_input
end

#coherence_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1CoherenceInput

Input for coherence metric. Corresponds to the JSON property coherenceInput

Returns:



15583
15584
15585
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15583

def coherence_input
  @coherence_input
end

#comet_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1CometInput

Input for Comet metric. Corresponds to the JSON property cometInput

Returns:



15588
15589
15590
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15588

def comet_input
  @comet_input
end

#exact_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ExactMatchInput

Input for exact match metric. Corresponds to the JSON property exactMatchInput

Returns:



15593
15594
15595
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15593

def exact_match_input
  @exact_match_input
end

#fluency_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1FluencyInput

Input for fluency metric. Corresponds to the JSON property fluencyInput

Returns:



15598
15599
15600
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15598

def fluency_input
  @fluency_input
end

#fulfillment_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1FulfillmentInput

Input for fulfillment metric. Corresponds to the JSON property fulfillmentInput

Returns:



15603
15604
15605
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15603

def fulfillment_input
  @fulfillment_input
end

#groundedness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1GroundednessInput

Input for groundedness metric. Corresponds to the JSON property groundednessInput

Returns:



15608
15609
15610
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15608

def groundedness_input
  @groundedness_input
end

#instance ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1EvaluationInstance

A single instance to be evaluated. Instances are used to specify the input data for evaluation, from simple string comparisons to complex, multi-turn model evaluations Corresponds to the JSON property instance

Returns:



15615
15616
15617
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15615

def instance
  @instance
end

#location ⇒ String

Required. The resource name of the Location to evaluate the instances. Format: projects/project/locations/location` Corresponds to the JSON propertylocation`

Returns:

  • (String) 


15621
15622
15623
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15621

def location
  @location
end

#metric_sources ⇒ Array<Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1MetricSource>

Optional. The metrics (either inline or registered) used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated. Corresponds to the JSON property metricSources

Returns:



15628
15629
15630
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15628

def metric_sources
  @metric_sources
end

#metrics ⇒ Array<Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1Metric>

The metrics used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated. Corresponds to the JSON property metrics

Returns:



15635
15636
15637
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15635

def metrics
  @metrics
end

#metricx_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1MetricxInput

Input for MetricX metric. Corresponds to the JSON property metricxInput

Returns:



15640
15641
15642
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15640

def metricx_input
  @metricx_input
end

#pairwise_metric_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PairwiseMetricInput

Input for pairwise metric. Corresponds to the JSON property pairwiseMetricInput

Returns:



15645
15646
15647
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15645

def pairwise_metric_input
  @pairwise_metric_input
end

#pairwise_question_answering_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PairwiseQuestionAnsweringQualityInput

Input for pairwise question answering quality metric. Corresponds to the JSON property pairwiseQuestionAnsweringQualityInput

Returns:



15650
15651
15652
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15650

def pairwise_question_answering_quality_input
  @pairwise_question_answering_quality_input
end

#pairwise_summarization_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PairwiseSummarizationQualityInput

Input for pairwise summarization quality metric. Corresponds to the JSON property pairwiseSummarizationQualityInput

Returns:



15655
15656
15657
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15655

def pairwise_summarization_quality_input
  @pairwise_summarization_quality_input
end

#pointwise_metric_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PointwiseMetricInput

Input for pointwise metric. Corresponds to the JSON property pointwiseMetricInput

Returns:



15660
15661
15662
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15660

def pointwise_metric_input
  @pointwise_metric_input
end

#question_answering_correctness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringCorrectnessInput

Input for question answering correctness metric. Corresponds to the JSON property questionAnsweringCorrectnessInput

Returns:



15665
15666
15667
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15665

def question_answering_correctness_input
  @question_answering_correctness_input
end

#question_answering_helpfulness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringHelpfulnessInput

Input for question answering helpfulness metric. Corresponds to the JSON property questionAnsweringHelpfulnessInput

Returns:



15670
15671
15672
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15670

def question_answering_helpfulness_input
  @question_answering_helpfulness_input
end

#question_answering_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringQualityInput

Input for question answering quality metric. Corresponds to the JSON property questionAnsweringQualityInput

Returns:



15675
15676
15677
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15675

def question_answering_quality_input
  @question_answering_quality_input
end

#question_answering_relevance_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringRelevanceInput

Input for question answering relevance metric. Corresponds to the JSON property questionAnsweringRelevanceInput

Returns:



15680
15681
15682
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15680

def question_answering_relevance_input
  @question_answering_relevance_input
end

#rouge_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1RougeInput

Input for rouge metric. Corresponds to the JSON property rougeInput

Returns:



15685
15686
15687
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15685

def rouge_input
  @rouge_input
end

#rubric_based_instruction_following_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1RubricBasedInstructionFollowingInput

Instance and metric spec for RubricBasedInstructionFollowing metric. Corresponds to the JSON property rubricBasedInstructionFollowingInput

Returns:



15690
15691
15692
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15690

def rubric_based_instruction_following_input
  @rubric_based_instruction_following_input
end

#safety_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SafetyInput

Input for safety metric. Corresponds to the JSON property safetyInput

Returns:



15695
15696
15697
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15695

def safety_input
  @safety_input
end

#summarization_helpfulness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SummarizationHelpfulnessInput

Input for summarization helpfulness metric. Corresponds to the JSON property summarizationHelpfulnessInput

Returns:



15700
15701
15702
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15700

def summarization_helpfulness_input
  @summarization_helpfulness_input
end

#summarization_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SummarizationQualityInput

Input for summarization quality metric. Corresponds to the JSON property summarizationQualityInput

Returns:



15705
15706
15707
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15705

def summarization_quality_input
  @summarization_quality_input
end

#summarization_verbosity_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SummarizationVerbosityInput

Input for summarization verbosity metric. Corresponds to the JSON property summarizationVerbosityInput

Returns:



15710
15711
15712
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15710

def summarization_verbosity_input
  @summarization_verbosity_input
end

#tool_call_valid_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolCallValidInput

Input for tool call valid metric. Corresponds to the JSON property toolCallValidInput

Returns:



15715
15716
15717
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15715

def tool_call_valid_input
  @tool_call_valid_input
end

#tool_name_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolNameMatchInput

Input for tool name match metric. Corresponds to the JSON property toolNameMatchInput

Returns:



15720
15721
15722
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15720

def tool_name_match_input
  @tool_name_match_input
end

#tool_parameter_key_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolParameterKeyMatchInput

Input for tool parameter key match metric. Corresponds to the JSON property toolParameterKeyMatchInput

Returns:



15725
15726
15727
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15725

def tool_parameter_key_match_input
  @tool_parameter_key_match_input
end

#tool_parameter_kv_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolParameterKvMatchInput

Input for tool parameter key value match metric. Corresponds to the JSON property toolParameterKvMatchInput

Returns:



15730
15731
15732
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15730

def tool_parameter_kv_match_input
  @tool_parameter_kv_match_input
end

#trajectory_any_order_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryAnyOrderMatchInput

Instances and metric spec for TrajectoryAnyOrderMatch metric. Corresponds to the JSON property trajectoryAnyOrderMatchInput

Returns:



15735
15736
15737
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15735

def trajectory_any_order_match_input
  @trajectory_any_order_match_input
end

#trajectory_exact_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryExactMatchInput

Instances and metric spec for TrajectoryExactMatch metric. Corresponds to the JSON property trajectoryExactMatchInput

Returns:



15740
15741
15742
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15740

def trajectory_exact_match_input
  @trajectory_exact_match_input
end

#trajectory_in_order_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryInOrderMatchInput

Instances and metric spec for TrajectoryInOrderMatch metric. Corresponds to the JSON property trajectoryInOrderMatchInput

Returns:



15745
15746
15747
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15745

def trajectory_in_order_match_input
  @trajectory_in_order_match_input
end

#trajectory_precision_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryPrecisionInput

Instances and metric spec for TrajectoryPrecision metric. Corresponds to the JSON property trajectoryPrecisionInput

Returns:



15750
15751
15752
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15750

def trajectory_precision_input
  @trajectory_precision_input
end

#trajectory_recall_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryRecallInput

Instances and metric spec for TrajectoryRecall metric. Corresponds to the JSON property trajectoryRecallInput

Returns:



15755
15756
15757
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15755

def trajectory_recall_input
  @trajectory_recall_input
end

#trajectory_single_tool_use_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectorySingleToolUseInput

Instances and metric spec for TrajectorySingleToolUse metric. Corresponds to the JSON property trajectorySingleToolUseInput

Returns:



15760
15761
15762
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15760

def trajectory_single_tool_use_input
  @trajectory_single_tool_use_input
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object



15767
15768
15769
15770
15771
15772
15773
15774
15775
15776
15777
15778
15779
15780
15781
15782
15783
15784
15785
15786
15787
15788
15789
15790
15791
15792
15793
15794
15795
15796
15797
15798
15799
15800
15801
15802
15803
15804
15805
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 15767

def update!(**args)
  @autorater_config = args[:autorater_config] if args.key?(:autorater_config)
  @bleu_input = args[:bleu_input] if args.key?(:bleu_input)
  @coherence_input = args[:coherence_input] if args.key?(:coherence_input)
  @comet_input = args[:comet_input] if args.key?(:comet_input)
  @exact_match_input = args[:exact_match_input] if args.key?(:exact_match_input)
  @fluency_input = args[:fluency_input] if args.key?(:fluency_input)
  @fulfillment_input = args[:fulfillment_input] if args.key?(:fulfillment_input)
  @groundedness_input = args[:groundedness_input] if args.key?(:groundedness_input)
  @instance = args[:instance] if args.key?(:instance)
  @location = args[:location] if args.key?(:location)
  @metric_sources = args[:metric_sources] if args.key?(:metric_sources)
  @metrics = args[:metrics] if args.key?(:metrics)
  @metricx_input = args[:metricx_input] if args.key?(:metricx_input)
  @pairwise_metric_input = args[:pairwise_metric_input] if args.key?(:pairwise_metric_input)
  @pairwise_question_answering_quality_input = args[:pairwise_question_answering_quality_input] if args.key?(:pairwise_question_answering_quality_input)
  @pairwise_summarization_quality_input = args[:pairwise_summarization_quality_input] if args.key?(:pairwise_summarization_quality_input)
  @pointwise_metric_input = args[:pointwise_metric_input] if args.key?(:pointwise_metric_input)
  @question_answering_correctness_input = args[:question_answering_correctness_input] if args.key?(:question_answering_correctness_input)
  @question_answering_helpfulness_input = args[:question_answering_helpfulness_input] if args.key?(:question_answering_helpfulness_input)
  @question_answering_quality_input = args[:question_answering_quality_input] if args.key?(:question_answering_quality_input)
  @question_answering_relevance_input = args[:question_answering_relevance_input] if args.key?(:question_answering_relevance_input)
  @rouge_input = args[:rouge_input] if args.key?(:rouge_input)
  @rubric_based_instruction_following_input = args[:rubric_based_instruction_following_input] if args.key?(:rubric_based_instruction_following_input)
  @safety_input = args[:safety_input] if args.key?(:safety_input)
  @summarization_helpfulness_input = args[:summarization_helpfulness_input] if args.key?(:summarization_helpfulness_input)
  @summarization_quality_input = args[:summarization_quality_input] if args.key?(:summarization_quality_input)
  @summarization_verbosity_input = args[:summarization_verbosity_input] if args.key?(:summarization_verbosity_input)
  @tool_call_valid_input = args[:tool_call_valid_input] if args.key?(:tool_call_valid_input)
  @tool_name_match_input = args[:tool_name_match_input] if args.key?(:tool_name_match_input)
  @tool_parameter_key_match_input = args[:tool_parameter_key_match_input] if args.key?(:tool_parameter_key_match_input)
  @tool_parameter_kv_match_input = args[:tool_parameter_kv_match_input] if args.key?(:tool_parameter_kv_match_input)
  @trajectory_any_order_match_input = args[:trajectory_any_order_match_input] if args.key?(:trajectory_any_order_match_input)
  @trajectory_exact_match_input = args[:trajectory_exact_match_input] if args.key?(:trajectory_exact_match_input)
  @trajectory_in_order_match_input = args[:trajectory_in_order_match_input] if args.key?(:trajectory_in_order_match_input)
  @trajectory_precision_input = args[:trajectory_precision_input] if args.key?(:trajectory_precision_input)
  @trajectory_recall_input = args[:trajectory_recall_input] if args.key?(:trajectory_recall_input)
  @trajectory_single_tool_use_input = args[:trajectory_single_tool_use_input] if args.key?(:trajectory_single_tool_use_input)
end