Class: Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SpeculativeDecodingSpec

Inherits:
Object
  • Object
show all
Includes:
Core::Hashable, Core::JsonObjectSupport
Defined in:
lib/google/apis/aiplatform_v1beta1/classes.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb,
lib/google/apis/aiplatform_v1beta1/representations.rb

Overview

Configuration for Speculative Decoding.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(**args) ⇒ GoogleCloudAiplatformV1beta1SpeculativeDecodingSpec

Returns a new instance of GoogleCloudAiplatformV1beta1SpeculativeDecodingSpec.



59079
59080
59081
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 59079

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

#draft_model_speculation ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SpeculativeDecodingSpecDraftModelSpeculation

Draft model speculation works by using the smaller model to generate candidate tokens for speculative decoding. Corresponds to the JSON property draftModelSpeculation

Returns:



59066
59067
59068
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 59066

def draft_model_speculation
  @draft_model_speculation
end

#ngram_speculation ⇒ Google::Apis::AiplatformV1beta1::GoogleCloudAiplatformV1beta1SpeculativeDecodingSpecNgramSpeculation

N-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens. Corresponds to the JSON property ngramSpeculation

Returns:



59072
59073
59074
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 59072

def ngram_speculation
  @ngram_speculation
end

#speculative_token_count ⇒ Fixnum

The number of speculative tokens to generate at each step. Corresponds to the JSON property speculativeTokenCount

Returns:

  • (Fixnum) 


59077
59078
59079
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 59077

def speculative_token_count
  @speculative_token_count
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object



59084
59085
59086
59087
59088
 
# File 'lib/google/apis/aiplatform_v1beta1/classes.rb', line 59084

def update!(**args)
  @draft_model_speculation = args[:draft_model_speculation] if args.key?(:draft_model_speculation)
  @ngram_speculation = args[:ngram_speculation] if args.key?(:ngram_speculation)
  @speculative_token_count = args[:speculative_token_count] if args.key?(:speculative_token_count)
end