Class: BlockFFICacheCuda

Inherits:
Object
  • Object
show all
Defined in:
lib/toy/ffi/tinynn_cuda.rb

Overview

FullForwardFFICacheCuda mirror (M1 family) ===

Per-block tensor handles for FullForwardFFICache. One instance per transformer block. All ivars are :ptr handles (or arrays thereof); the actual ggml tensors live in the FullForwardFFICache’s session.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ BlockFFICacheCuda

Returns a new instance of BlockFFICacheCuda.



869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
 
# File 'lib/toy/ffi/tinynn_cuda.rb', line 869

def initialize
  @t_norm1_gamma = TinyNNCuda.tnn_null_ptr
  @t_norm2_gamma = TinyNNCuda.tnn_null_ptr
  # Note: Spinel currently types these arrays as `IntArray` rather
  # than `PtrArray` because the FFI's `:ptr` value model is integer-
  # backed (a long-sized address). The runtime is correct because
  # pointer values fit in `mrb_int` on 64-bit platforms — every
  # FFI call that takes the element gets a `void *` produced by a
  # silent long-to-pointer conversion (cc emits a warning but the
  # bit pattern round-trips). On a 32-bit platform this would break.
  # Seeded with `tnn_null_ptr` rather than `[nil]` so the intent is
  # clear in source even though Spinel infers the same way either way.
  @t_w_q   = [TinyNNCuda.tnn_null_ptr]
  @t_w_k   = [TinyNNCuda.tnn_null_ptr]
  @t_w_v   = [TinyNNCuda.tnn_null_ptr]
  @t_w_o   = TinyNNCuda.tnn_null_ptr
  @t_w_ff1 = TinyNNCuda.tnn_null_ptr
  @t_w_ff2 = TinyNNCuda.tnn_null_ptr
end

Instance Attribute Details

#t_norm1_gamma ⇒ Object

Returns the value of attribute t_norm1_gamma.



865
866
867
 
# File 'lib/toy/ffi/tinynn_cuda.rb', line 865

def t_norm1_gamma
  @t_norm1_gamma
end

#t_norm2_gamma ⇒ Object

Returns the value of attribute t_norm2_gamma.



865
866
867
 
# File 'lib/toy/ffi/tinynn_cuda.rb', line 865

def t_norm2_gamma
  @t_norm2_gamma
end

#t_w_ff1 ⇒ Object

Returns the value of attribute t_w_ff1.



865
866
867
 
# File 'lib/toy/ffi/tinynn_cuda.rb', line 865

def t_w_ff1
  @t_w_ff1
end

#t_w_ff2 ⇒ Object

Returns the value of attribute t_w_ff2.



865
866
867
 
# File 'lib/toy/ffi/tinynn_cuda.rb', line 865

def t_w_ff2
  @t_w_ff2
end

#t_w_k ⇒ Object

Returns the value of attribute t_w_k.



865
866
867
 
# File 'lib/toy/ffi/tinynn_cuda.rb', line 865

def t_w_k
  @t_w_k
end

#t_w_o ⇒ Object

Returns the value of attribute t_w_o.



865
866
867
 
# File 'lib/toy/ffi/tinynn_cuda.rb', line 865

def t_w_o
  @t_w_o
end

#t_w_q ⇒ Object

Returns the value of attribute t_w_q.



865
866
867
 
# File 'lib/toy/ffi/tinynn_cuda.rb', line 865

def t_w_q
  @t_w_q
end

#t_w_v ⇒ Object

Returns the value of attribute t_w_v.



865
866
867
 
# File 'lib/toy/ffi/tinynn_cuda.rb', line 865

def t_w_v
  @t_w_v
end