Class: GPT2KVStepResultCuda

Inherits:
Object
  • Object
show all
Defined in:
lib/toy/llm/engine/gpt2_kv_engine_cuda.rb

Overview

Single-step graph artefacts. Returned by build_decode_step so the caller can upload the token_id input and download the logits.

Init-param names deliberately differ from the ivar names to dodge Spinel’s whole-program local-name collapse (e.g. ‘t_logits` as a param could unify with `t_logits` locals elsewhere and box the slot as sp_RbVal — breaks the (void *) cast in download_row_major).

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(tok_ptr, logits_ptr) ⇒ GPT2KVStepResultCuda

Returns a new instance of GPT2KVStepResultCuda.



270
271
272
273
 
# File 'lib/toy/llm/engine/gpt2_kv_engine_cuda.rb', line 270

def initialize(tok_ptr, logits_ptr)
  @t_token_id     = tok_ptr
  @kv_step_logits = logits_ptr
end

Instance Attribute Details

#kv_step_logits ⇒ Object

Returns the value of attribute kv_step_logits.



269
270
271
 
# File 'lib/toy/llm/engine/gpt2_kv_engine_cuda.rb', line 269

def kv_step_logits
  @kv_step_logits
end

#t_token_id ⇒ Object

Returns the value of attribute t_token_id.



269
270
271
 
# File 'lib/toy/llm/engine/gpt2_kv_engine_cuda.rb', line 269

def t_token_id
  @t_token_id
end