Class: Google::Cloud::Ces::V1beta::Evaluation::GoldenExpectation

Inherits:
Object
  • Object
show all
Extended by:
Protobuf::MessageExts::ClassMethods
Includes:
Protobuf::MessageExts
Defined in:
proto_docs/google/cloud/ces/v1beta/evaluation.rb

Overview

Represents a single, checkable requirement.

Instance Attribute Summary collapse

Instance Attribute Details

#agent_response ⇒ ::Google::Cloud::Ces::V1beta::Message

Returns Optional. Check that the agent responded with the correct response. The role "agent" is implied.

Note: The following fields are mutually exclusive: agent_response, tool_call, tool_response, agent_transfer, updated_variables, mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Ces::V1beta::Message) —

    Optional. Check that the agent responded with the correct response. The role "agent" is implied.

    Note: The following fields are mutually exclusive: agent_response, tool_call, tool_response, agent_transfer, updated_variables, mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.



257
258
259
260
 
# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 257

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#agent_transfer ⇒ ::Google::Cloud::Ces::V1beta::AgentTransfer

Returns Optional. Check that the agent transferred the conversation to a different agent.

Note: The following fields are mutually exclusive: agent_transfer, tool_call, tool_response, agent_response, updated_variables, mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Ces::V1beta::AgentTransfer) —

    Optional. Check that the agent transferred the conversation to a different agent.

    Note: The following fields are mutually exclusive: agent_transfer, tool_call, tool_response, agent_response, updated_variables, mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.



257
258
259
260
 
# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 257

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#mock_tool_response ⇒ ::Google::Cloud::Ces::V1beta::ToolResponse

Returns Optional. The tool response to mock, with the parameters of interest specified. Any parameters not specified will be hallucinated by the LLM.

Note: The following fields are mutually exclusive: mock_tool_response, tool_call, tool_response, agent_response, agent_transfer, updated_variables. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Ces::V1beta::ToolResponse) —

    Optional. The tool response to mock, with the parameters of interest specified. Any parameters not specified will be hallucinated by the LLM.

    Note: The following fields are mutually exclusive: mock_tool_response, tool_call, tool_response, agent_response, agent_transfer, updated_variables. If a field in that set is populated, all other fields in the set will automatically be cleared.



257
258
259
260
 
# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 257

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#note ⇒ ::String

Returns Optional. A note for this requirement, useful in reporting when specific checks fail. E.g., "Check_Payment_Tool_Called".

Returns:

  • (::String) —

    Optional. A note for this requirement, useful in reporting when specific checks fail. E.g., "Check_Payment_Tool_Called".



257
258
259
260
 
# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 257

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#tool_call ⇒ ::Google::Cloud::Ces::V1beta::ToolCall

Returns Optional. Check that a specific tool was called with the parameters.

Note: The following fields are mutually exclusive: tool_call, tool_response, agent_response, agent_transfer, updated_variables, mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Ces::V1beta::ToolCall) —

    Optional. Check that a specific tool was called with the parameters.

    Note: The following fields are mutually exclusive: tool_call, tool_response, agent_response, agent_transfer, updated_variables, mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.



257
258
259
260
 
# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 257

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#tool_response ⇒ ::Google::Cloud::Ces::V1beta::ToolResponse

Returns Optional. Check that a specific tool had the expected response.

Note: The following fields are mutually exclusive: tool_response, tool_call, agent_response, agent_transfer, updated_variables, mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Cloud::Ces::V1beta::ToolResponse) —

    Optional. Check that a specific tool had the expected response.

    Note: The following fields are mutually exclusive: tool_response, tool_call, agent_response, agent_transfer, updated_variables, mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.



257
258
259
260
 
# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 257

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#updated_variables ⇒ ::Google::Protobuf::Struct

Returns Optional. Check that the agent updated the session variables to the expected values. Used to also capture agent variable updates for golden evals.

Note: The following fields are mutually exclusive: updated_variables, tool_call, tool_response, agent_response, agent_transfer, mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::Google::Protobuf::Struct) —

    Optional. Check that the agent updated the session variables to the expected values. Used to also capture agent variable updates for golden evals.

    Note: The following fields are mutually exclusive: updated_variables, tool_call, tool_response, agent_response, agent_transfer, mock_tool_response. If a field in that set is populated, all other fields in the set will automatically be cleared.



257
258
259
260
 
# File 'proto_docs/google/cloud/ces/v1beta/evaluation.rb', line 257

class GoldenExpectation
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end