Class: Aws::BedrockAgent::Types::InferenceConfiguration

Inherits:

Struct

• Object
• Struct
• Aws::BedrockAgent::Types::InferenceConfiguration

Includes:

Structure

Defined in:

lib/aws-sdk-bedrockagent/types.rb

Overview

Contains inference parameters to use when the agent invokes a foundation model in the part of the agent sequence defined by the ‘promptType`. For more information, see [Inference parameters for foundation models].

[1]: docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html

See Also:

• AWS API Documentation

Constant Summary

SENSITIVE =

[]

Instance Attribute Summary

• #maximum_length ⇒ Integer

  The maximum number of tokens to allow in the generated response.

• #stop_sequences ⇒ Array<String>

  A list of stop sequences.

• #temperature ⇒ Float

  The likelihood of the model selecting higher-probability options while generating a response.

• #top_k ⇒ Integer

  While generating a response, the model determines the probability of the following token at each point of generation.

• #top_p ⇒ Float

  While generating a response, the model determines the probability of the following token at each point of generation.

Instance Attribute Details

#maximum_length ⇒ Integer

The maximum number of tokens to allow in the generated response.

Returns:

• (Integer)

# File 'lib/aws-sdk-bedrockagent/types.rb', line 2126

class InferenceConfiguration < Struct.new(
  :maximum_length,
  :stop_sequences,
  :temperature,
  :top_k,
  :top_p)
  SENSITIVE = []
  include Aws::Structure
end

#stop_sequences ⇒ Array<String>

A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop generating the response.

Returns:

• (Array<String>)

# File 'lib/aws-sdk-bedrockagent/types.rb', line 2126

class InferenceConfiguration < Struct.new(
  :maximum_length,
  :stop_sequences,
  :temperature,
  :top_k,
  :top_p)
  SENSITIVE = []
  include Aws::Structure
end

#temperature ⇒ Float

The likelihood of the model selecting higher-probability options while generating a response. A lower value makes the model more likely to choose higher-probability options, while a higher value makes the model more likely to choose lower-probability options.

Returns:

• (Float)

# File 'lib/aws-sdk-bedrockagent/types.rb', line 2126

class InferenceConfiguration < Struct.new(
  :maximum_length,
  :stop_sequences,
  :temperature,
  :top_k,
  :top_p)
  SENSITIVE = []
  include Aws::Structure
end

#top_k ⇒ Integer

While generating a response, the model determines the probability of the following token at each point of generation. The value that you set for ‘topK` is the number of most-likely candidates from which the model chooses the next token in the sequence. For example, if you set `topK` to 50, the model selects the next token from among the top 50 most likely choices.

Returns:

• (Integer)

# File 'lib/aws-sdk-bedrockagent/types.rb', line 2126

class InferenceConfiguration < Struct.new(
  :maximum_length,
  :stop_sequences,
  :temperature,
  :top_k,
  :top_p)
  SENSITIVE = []
  include Aws::Structure
end

#top_p ⇒ Float

While generating a response, the model determines the probability of the following token at each point of generation. The value that you set for ‘Top P` determines the number of most-likely candidates from which the model chooses the next token in the sequence. For example, if you set `topP` to 80, the model only selects the next token from the top 80% of the probability distribution of next tokens.

Returns:

• (Float)

# File 'lib/aws-sdk-bedrockagent/types.rb', line 2126

class InferenceConfiguration < Struct.new(
  :maximum_length,
  :stop_sequences,
  :temperature,
  :top_k,
  :top_p)
  SENSITIVE = []
  include Aws::Structure
end