Class: Google::Cloud::Ces::V1beta::Evaluation::GoldenExpectation

Inherits:

Object

• Object
• Google::Cloud::Ces::V1beta::Evaluation::GoldenExpectation

Extended by:

Protobuf::MessageExts::ClassMethods

Includes:

Protobuf::MessageExts

Defined in:

proto_docs/google/cloud/ces/v1beta/evaluation.rb

Overview

Represents a single, checkable requirement.

Instance Attribute Summary

• #agent_response ⇒ ::Google::Cloud::Ces::V1beta::Message

  Optional.

• #agent_response_hallucination_metrics_config_override ⇒ ::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::HallucinationMetricsConfig

  Optional.

• #agent_response_semantic_similarity_metrics_config_override ⇒ ::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::SemanticSimilarityMetricsConfig

  Optional.

• #agent_transfer ⇒ ::Google::Cloud::Ces::V1beta::AgentTransfer

  Optional.

• #comparison_type ⇒ ::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::ComparisonType

  Optional.

• #expectation_level_metrics_thresholds_override ⇒ ::Google::Cloud::Ces::V1beta::EvaluationMetricsThresholds::GoldenEvaluationMetricsThresholds::ExpectationLevelMetricsThresholds

  Optional.

• #mock_tool_response ⇒ ::Google::Cloud::Ces::V1beta::ToolResponse

  Optional.

• #no_tool_calls ⇒ ::Boolean

  Optional.

• #note ⇒ ::String

  Optional.

• #skip_evaluation ⇒ ::Boolean

  Optional.

• #tool_call ⇒ ::Google::Cloud::Ces::V1beta::ToolCall

  Optional.

• #tool_response ⇒ ::Google::Cloud::Ces::V1beta::ToolResponse

  Optional.

• #updated_variables ⇒ ::Google::Protobuf::Struct

  Optional.

Instance Attribute Details

#agent_response ⇒ ::Google::Cloud::Ces::V1beta::Message

Returns Optional. Check that the agent responded with the correct response. The role "agent" is implied.

Note: The following fields are mutually exclusive: agent_response, tool_call, tool_response, agent_transfer, updated_variables, mock_tool_response, no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

• (::Google::Cloud::Ces::V1beta::Message) —

  Optional. Check that the agent responded with the correct response. The role "agent" is implied.

  Note: The following fields are mutually exclusive: agent_response, tool_call, tool_response, agent_transfer, updated_variables, mock_tool_response, no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.

# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 284

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#agent_response_hallucination_metrics_config_override ⇒ ::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::HallucinationMetricsConfig

Returns Optional. Overrides for agent_response hallucination metrics.

Returns:

• (::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::HallucinationMetricsConfig) —

  Optional. Overrides for agent_response hallucination metrics.

# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 284

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#agent_response_semantic_similarity_metrics_config_override ⇒ ::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::SemanticSimilarityMetricsConfig

Returns Optional. Overrides for agent_response semantic similarity metrics.

Returns:

• (::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::SemanticSimilarityMetricsConfig) —

  Optional. Overrides for agent_response semantic similarity metrics.

# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 284

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#agent_transfer ⇒ ::Google::Cloud::Ces::V1beta::AgentTransfer

Returns Optional. Check that the agent transferred the conversation to a different agent.

Note: The following fields are mutually exclusive: agent_transfer, tool_call, tool_response, agent_response, updated_variables, mock_tool_response, no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

• (::Google::Cloud::Ces::V1beta::AgentTransfer) —

  Optional. Check that the agent transferred the conversation to a different agent.

  Note: The following fields are mutually exclusive: agent_transfer, tool_call, tool_response, agent_response, updated_variables, mock_tool_response, no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.

# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 284

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#comparison_type ⇒ ::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::ComparisonType

Returns Optional. The comparison type to use for the expectation check.

Returns:

• (::Google::Cloud::Ces::V1beta::EvaluationMetricsConfig::ComparisonType) —

  Optional. The comparison type to use for the expectation check.

# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 284

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#expectation_level_metrics_thresholds_override ⇒ ::Google::Cloud::Ces::V1beta::EvaluationMetricsThresholds::GoldenEvaluationMetricsThresholds::ExpectationLevelMetricsThresholds

Returns Optional. Overrides metrics at the step level.

Returns:

• (::Google::Cloud::Ces::V1beta::EvaluationMetricsThresholds::GoldenEvaluationMetricsThresholds::ExpectationLevelMetricsThresholds) —

  Optional. Overrides metrics at the step level.

# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 284

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#mock_tool_response ⇒ ::Google::Cloud::Ces::V1beta::ToolResponse

Returns Optional. The tool response to mock, with the parameters of interest specified. Any parameters not specified will be hallucinated by the LLM.

Note: The following fields are mutually exclusive: mock_tool_response, tool_call, tool_response, agent_response, agent_transfer, updated_variables, no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

• (::Google::Cloud::Ces::V1beta::ToolResponse) —

  Optional. The tool response to mock, with the parameters of interest specified. Any parameters not specified will be hallucinated by the LLM.

  Note: The following fields are mutually exclusive: mock_tool_response, tool_call, tool_response, agent_response, agent_transfer, updated_variables, no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.

# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 284

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#no_tool_calls ⇒ ::Boolean

Returns Optional. Check that no tools were called during this turn.

Note: The following fields are mutually exclusive: no_tool_calls, tool_call, tool_response, agent_response, agent_transfer, updated_variables, mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

• (::Boolean) —

  Optional. Check that no tools were called during this turn.

  Note: The following fields are mutually exclusive: no_tool_calls, tool_call, tool_response, agent_response, agent_transfer, updated_variables, mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.

# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 284

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#note ⇒ ::String

Returns Optional. A note for this requirement, useful in reporting when specific checks fail. E.g., "Check_Payment_Tool_Called".

Returns:

• (::String) —

  Optional. A note for this requirement, useful in reporting when specific checks fail. E.g., "Check_Payment_Tool_Called".

# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 284

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#skip_evaluation ⇒ ::Boolean

Returns Optional. If set to true, this specific expectation will not be evaluated.

Returns:

• (::Boolean) —

  Optional. If set to true, this specific expectation will not be evaluated.

# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 284

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#tool_call ⇒ ::Google::Cloud::Ces::V1beta::ToolCall

Returns Optional. Check that a specific tool was called with the parameters.

Note: The following fields are mutually exclusive: tool_call, tool_response, agent_response, agent_transfer, updated_variables, mock_tool_response, no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

• (::Google::Cloud::Ces::V1beta::ToolCall) —

  Optional. Check that a specific tool was called with the parameters.

  Note: The following fields are mutually exclusive: tool_call, tool_response, agent_response, agent_transfer, updated_variables, mock_tool_response, no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.

# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 284

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#tool_response ⇒ ::Google::Cloud::Ces::V1beta::ToolResponse

Returns Optional. Check that a specific tool had the expected response.

Note: The following fields are mutually exclusive: tool_response, tool_call, agent_response, agent_transfer, updated_variables, mock_tool_response, no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

• (::Google::Cloud::Ces::V1beta::ToolResponse) —

  Optional. Check that a specific tool had the expected response.

  Note: The following fields are mutually exclusive: tool_response, tool_call, agent_response, agent_transfer, updated_variables, mock_tool_response, no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.

# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 284

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#updated_variables ⇒ ::Google::Protobuf::Struct

Returns Optional. Check that the agent updated the session variables to the expected values. Used to also capture agent variable updates for golden evals.

Note: The following fields are mutually exclusive: updated_variables, tool_call, tool_response, agent_response, agent_transfer, mock_tool_response, no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

• (::Google::Protobuf::Struct) —

  Optional. Check that the agent updated the session variables to the expected values. Used to also capture agent variable updates for golden evals.

  Note: The following fields are mutually exclusive: updated_variables, tool_call, tool_response, agent_response, agent_transfer, mock_tool_response, no_tool_calls. If a field in that set is populated, all other fields in the set will automatically be cleared.

# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 284

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end