Class: Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1EvaluateInstancesRequest

Inherits:
Object
  • Object
show all
Includes:
Core::Hashable, Core::JsonObjectSupport
Defined in:
lib/google/apis/aiplatform_v1beta1/classes.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb

Overview

Request message for EvaluationService.EvaluateInstances.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(**args) ⇒ GoogleCloudAiplatformV1beta1EvaluateInstancesRequest

Returns a new instance of GoogleCloudAiplatformV1beta1EvaluateInstancesRequest.



17938
17939
17940
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17938

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

#autorater_config ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1AutoraterConfig

The configs for autorater. This is applicable to both EvaluateInstances and EvaluateDataset. Corresponds to the JSON property autoraterConfig

Returns:



17749
17750
17751
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17749

def autorater_config
  @autorater_config
end

#bleu_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1BleuInput

Input for bleu metric. Corresponds to the JSON property bleuInput

Returns:



17754
17755
17756
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17754

def bleu_input
  @bleu_input
end

#coherence_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1CoherenceInput

Input for coherence metric. Corresponds to the JSON property coherenceInput

Returns:



17759
17760
17761
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17759

def coherence_input
  @coherence_input
end

#comet_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1CometInput

Input for Comet metric. Corresponds to the JSON property cometInput

Returns:



17764
17765
17766
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17764

def comet_input
  @comet_input
end

#exact_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ExactMatchInput

Input for exact match metric. Corresponds to the JSON property exactMatchInput

Returns:



17769
17770
17771
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17769

def exact_match_input
  @exact_match_input
end

#fluency_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1FluencyInput

Input for fluency metric. Corresponds to the JSON property fluencyInput

Returns:



17774
17775
17776
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17774

def fluency_input
  @fluency_input
end

#fulfillment_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1FulfillmentInput

Input for fulfillment metric. Corresponds to the JSON property fulfillmentInput

Returns:



17779
17780
17781
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17779

def fulfillment_input
  @fulfillment_input
end

#groundedness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1GroundednessInput

Input for groundedness metric. Corresponds to the JSON property groundednessInput

Returns:



17784
17785
17786
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17784

def groundedness_input
  @groundedness_input
end

#instance ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1EvaluationInstance

A single instance to be evaluated. Instances are used to specify the input data for evaluation, from simple string comparisons to complex, multi-turn model evaluations Corresponds to the JSON property instance

Returns:



17791
17792
17793
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17791

def instance
  @instance
end

#location ⇒ String

Required. The resource name of the Location to evaluate the instances. Format: projects/project/locations/location` Corresponds to the JSON propertylocation`

Returns:

  • (String) 


17797
17798
17799
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17797

def location
  @location
end

#metric_sources ⇒ Array<Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1MetricSource>

Optional. The metrics (either inline or registered) used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated. Corresponds to the JSON property metricSources

Returns:



17804
17805
17806
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17804

def metric_sources
  @metric_sources
end

#metrics ⇒ Array<Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1Metric>

The metrics used for evaluation. Currently, we only support evaluating a single metric. If multiple metrics are provided, only the first one will be evaluated. Corresponds to the JSON property metrics

Returns:



17811
17812
17813
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17811

def metrics
  @metrics
end

#metricx_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1MetricxInput

Input for MetricX metric. Corresponds to the JSON property metricxInput

Returns:



17816
17817
17818
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17816

def metricx_input
  @metricx_input
end

#pairwise_metric_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PairwiseMetricInput

Input for pairwise metric. Corresponds to the JSON property pairwiseMetricInput

Returns:



17821
17822
17823
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17821

def pairwise_metric_input
  @pairwise_metric_input
end

#pairwise_question_answering_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PairwiseQuestionAnsweringQualityInput

Input for pairwise question answering quality metric. Corresponds to the JSON property pairwiseQuestionAnsweringQualityInput

Returns:



17826
17827
17828
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17826

def pairwise_question_answering_quality_input
  @pairwise_question_answering_quality_input
end

#pairwise_summarization_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PairwiseSummarizationQualityInput

Input for pairwise summarization quality metric. Corresponds to the JSON property pairwiseSummarizationQualityInput

Returns:



17831
17832
17833
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17831

def pairwise_summarization_quality_input
  @pairwise_summarization_quality_input
end

#pointwise_metric_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1PointwiseMetricInput

Input for pointwise metric. Corresponds to the JSON property pointwiseMetricInput

Returns:



17836
17837
17838
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17836

def pointwise_metric_input
  @pointwise_metric_input
end

#question_answering_correctness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringCorrectnessInput

Input for question answering correctness metric. Corresponds to the JSON property questionAnsweringCorrectnessInput

Returns:



17841
17842
17843
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17841

def question_answering_correctness_input
  @question_answering_correctness_input
end

#question_answering_helpfulness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringHelpfulnessInput

Input for question answering helpfulness metric. Corresponds to the JSON property questionAnsweringHelpfulnessInput

Returns:



17846
17847
17848
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17846

def question_answering_helpfulness_input
  @question_answering_helpfulness_input
end

#question_answering_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringQualityInput

Input for question answering quality metric. Corresponds to the JSON property questionAnsweringQualityInput

Returns:



17851
17852
17853
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17851

def question_answering_quality_input
  @question_answering_quality_input
end

#question_answering_relevance_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1QuestionAnsweringRelevanceInput

Input for question answering relevance metric. Corresponds to the JSON property questionAnsweringRelevanceInput

Returns:



17856
17857
17858
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17856

def question_answering_relevance_input
  @question_answering_relevance_input
end

#rouge_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1RougeInput

Input for rouge metric. Corresponds to the JSON property rougeInput

Returns:



17861
17862
17863
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17861

def rouge_input
  @rouge_input
end

#rubric_based_instruction_following_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1RubricBasedInstructionFollowingInput

Instance and metric spec for RubricBasedInstructionFollowing metric. Corresponds to the JSON property rubricBasedInstructionFollowingInput

Returns:



17866
17867
17868
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17866

def rubric_based_instruction_following_input
  @rubric_based_instruction_following_input
end

#safety_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SafetyInput

Input for safety metric. Corresponds to the JSON property safetyInput

Returns:



17871
17872
17873
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17871

def safety_input
  @safety_input
end

#summarization_helpfulness_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SummarizationHelpfulnessInput

Input for summarization helpfulness metric. Corresponds to the JSON property summarizationHelpfulnessInput

Returns:



17876
17877
17878
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17876

def summarization_helpfulness_input
  @summarization_helpfulness_input
end

#summarization_quality_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SummarizationQualityInput

Input for summarization quality metric. Corresponds to the JSON property summarizationQualityInput

Returns:



17881
17882
17883
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17881

def summarization_quality_input
  @summarization_quality_input
end

#summarization_verbosity_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SummarizationVerbosityInput

Input for summarization verbosity metric. Corresponds to the JSON property summarizationVerbosityInput

Returns:



17886
17887
17888
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17886

def summarization_verbosity_input
  @summarization_verbosity_input
end

#tool_call_valid_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolCallValidInput

Input for tool call valid metric. Corresponds to the JSON property toolCallValidInput

Returns:



17891
17892
17893
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17891

def tool_call_valid_input
  @tool_call_valid_input
end

#tool_name_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolNameMatchInput

Input for tool name match metric. Corresponds to the JSON property toolNameMatchInput

Returns:



17896
17897
17898
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17896

def tool_name_match_input
  @tool_name_match_input
end

#tool_parameter_key_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolParameterKeyMatchInput

Input for tool parameter key match metric. Corresponds to the JSON property toolParameterKeyMatchInput

Returns:



17901
17902
17903
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17901

def tool_parameter_key_match_input
  @tool_parameter_key_match_input
end

#tool_parameter_kv_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1ToolParameterKvMatchInput

Input for tool parameter key value match metric. Corresponds to the JSON property toolParameterKvMatchInput

Returns:



17906
17907
17908
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17906

def tool_parameter_kv_match_input
  @tool_parameter_kv_match_input
end

#trajectory_any_order_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryAnyOrderMatchInput

Instances and metric spec for TrajectoryAnyOrderMatch metric. Corresponds to the JSON property trajectoryAnyOrderMatchInput

Returns:



17911
17912
17913
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17911

def trajectory_any_order_match_input
  @trajectory_any_order_match_input
end

#trajectory_exact_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryExactMatchInput

Instances and metric spec for TrajectoryExactMatch metric. Corresponds to the JSON property trajectoryExactMatchInput

Returns:



17916
17917
17918
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17916

def trajectory_exact_match_input
  @trajectory_exact_match_input
end

#trajectory_in_order_match_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryInOrderMatchInput

Instances and metric spec for TrajectoryInOrderMatch metric. Corresponds to the JSON property trajectoryInOrderMatchInput

Returns:



17921
17922
17923
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17921

def trajectory_in_order_match_input
  @trajectory_in_order_match_input
end

#trajectory_precision_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryPrecisionInput

Instances and metric spec for TrajectoryPrecision metric. Corresponds to the JSON property trajectoryPrecisionInput

Returns:



17926
17927
17928
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17926

def trajectory_precision_input
  @trajectory_precision_input
end

#trajectory_recall_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectoryRecallInput

Instances and metric spec for TrajectoryRecall metric. Corresponds to the JSON property trajectoryRecallInput

Returns:



17931
17932
17933
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17931

def trajectory_recall_input
  @trajectory_recall_input
end

#trajectory_single_tool_use_input ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1TrajectorySingleToolUseInput

Instances and metric spec for TrajectorySingleToolUse metric. Corresponds to the JSON property trajectorySingleToolUseInput

Returns:



17936
17937
17938
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17936

def trajectory_single_tool_use_input
  @trajectory_single_tool_use_input
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object



17943
17944
17945
17946
17947
17948
17949
17950
17951
17952
17953
17954
17955
17956
17957
17958
17959
17960
17961
17962
17963
17964
17965
17966
17967
17968
17969
17970
17971
17972
17973
17974
17975
17976
17977
17978
17979
17980
17981
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 17943

def update!(**args)
  @autorater_config = args[:autorater_config] if args.key?(:autorater_config)
  @bleu_input = args[:bleu_input] if args.key?(:bleu_input)
  @coherence_input = args[:coherence_input] if args.key?(:coherence_input)
  @comet_input = args[:comet_input] if args.key?(:comet_input)
  @exact_match_input = args[:exact_match_input] if args.key?(:exact_match_input)
  @fluency_input = args[:fluency_input] if args.key?(:fluency_input)
  @fulfillment_input = args[:fulfillment_input] if args.key?(:fulfillment_input)
  @groundedness_input = args[:groundedness_input] if args.key?(:groundedness_input)
  @instance = args[:instance] if args.key?(:instance)
  @location = args[:location] if args.key?(:location)
  @metric_sources = args[:metric_sources] if args.key?(:metric_sources)
  @metrics = args[:metrics] if args.key?(:metrics)
  @metricx_input = args[:metricx_input] if args.key?(:metricx_input)
  @pairwise_metric_input = args[:pairwise_metric_input] if args.key?(:pairwise_metric_input)
  @pairwise_question_answering_quality_input = args[:pairwise_question_answering_quality_input] if args.key?(:pairwise_question_answering_quality_input)
  @pairwise_summarization_quality_input = args[:pairwise_summarization_quality_input] if args.key?(:pairwise_summarization_quality_input)
  @pointwise_metric_input = args[:pointwise_metric_input] if args.key?(:pointwise_metric_input)
  @question_answering_correctness_input = args[:question_answering_correctness_input] if args.key?(:question_answering_correctness_input)
  @question_answering_helpfulness_input = args[:question_answering_helpfulness_input] if args.key?(:question_answering_helpfulness_input)
  @question_answering_quality_input = args[:question_answering_quality_input] if args.key?(:question_answering_quality_input)
  @question_answering_relevance_input = args[:question_answering_relevance_input] if args.key?(:question_answering_relevance_input)
  @rouge_input = args[:rouge_input] if args.key?(:rouge_input)
  @rubric_based_instruction_following_input = args[:rubric_based_instruction_following_input] if args.key?(:rubric_based_instruction_following_input)
  @safety_input = args[:safety_input] if args.key?(:safety_input)
  @summarization_helpfulness_input = args[:summarization_helpfulness_input] if args.key?(:summarization_helpfulness_input)
  @summarization_quality_input = args[:summarization_quality_input] if args.key?(:summarization_quality_input)
  @summarization_verbosity_input = args[:summarization_verbosity_input] if args.key?(:summarization_verbosity_input)
  @tool_call_valid_input = args[:tool_call_valid_input] if args.key?(:tool_call_valid_input)
  @tool_name_match_input = args[:tool_name_match_input] if args.key?(:tool_name_match_input)
  @tool_parameter_key_match_input = args[:tool_parameter_key_match_input] if args.key?(:tool_parameter_key_match_input)
  @tool_parameter_kv_match_input = args[:tool_parameter_kv_match_input] if args.key?(:tool_parameter_kv_match_input)
  @trajectory_any_order_match_input = args[:trajectory_any_order_match_input] if args.key?(:trajectory_any_order_match_input)
  @trajectory_exact_match_input = args[:trajectory_exact_match_input] if args.key?(:trajectory_exact_match_input)
  @trajectory_in_order_match_input = args[:trajectory_in_order_match_input] if args.key?(:trajectory_in_order_match_input)
  @trajectory_precision_input = args[:trajectory_precision_input] if args.key?(:trajectory_precision_input)
  @trajectory_recall_input = args[:trajectory_recall_input] if args.key?(:trajectory_recall_input)
  @trajectory_single_tool_use_input = args[:trajectory_single_tool_use_input] if args.key?(:trajectory_single_tool_use_input)
end