Class: Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SpeculativeDecodingSpecNgramSpeculation

Inherits:

Object

• Object
• Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SpeculativeDecodingSpecNgramSpeculation

Includes:

Core::Hashable, Core::JsonObjectSupport

Defined in:

lib/google/apis/aiplatform_v1beta1/classes.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb

Overview

N-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens.

Instance Attribute Summary

• #ngram_size ⇒ Fixnum

  The number of last N input tokens used as ngram to search/match against the previous prompt sequence.

Instance Method Summary

• #initialize(**args) ⇒ GoogleCloudAiplatformV1beta1SpeculativeDecodingSpecNgramSpeculation constructor

  A new instance of GoogleCloudAiplatformV1beta1SpeculativeDecodingSpecNgramSpeculation.

• #update!(**args) ⇒ Object

  Update properties of this object.

Constructor Details

#initialize(**args) ⇒ GoogleCloudAiplatformV1beta1SpeculativeDecodingSpecNgramSpeculation

Returns a new instance of GoogleCloudAiplatformV1beta1SpeculativeDecodingSpecNgramSpeculation.

# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 50855

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

#ngram_size ⇒ Fixnum

The number of last N input tokens used as ngram to search/match against the previous prompt sequence. This is equal to the N in N-Gram. The default value is 3 if not specified. Corresponds to the JSON property ngramSize

Returns:

• (Fixnum)

# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 50853

def ngram_size
  @ngram_size
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object

# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 50860

def update!(**args)
  @ngram_size = args[:ngram_size] if args.key?(:ngram_size)
end