Class: BlockFFICache

Inherits:
Object
  • Object
show all
Defined in:
lib/toy/ffi/tinynn.rb

Overview

Per-block tensor handles for FullForwardFFICache. One instance per transformer block. All ivars are :ptr handles (or arrays thereof); the actual ggml tensors live in the FullForwardFFICache’s session.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ BlockFFICache

Returns a new instance of BlockFFICache.



91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
 
# File 'lib/toy/ffi/tinynn.rb', line 91

def initialize
  # Post-spinel `85a4670` (Array<:ptr> -> PtrArray): every `:ptr`
  # ivar seeds with TinyNN.tnn_null_ptr so the inferred type stays
  # `void *` end-to-end. A bare `nil` seed boxes the ivar as
  # `sp_RbVal` (nilable) and FFI dispatch then fails the `(void *)`
  # cast.
  @t_norm1_gamma = TinyNN.tnn_null_ptr
  @t_norm2_gamma = TinyNN.tnn_null_ptr
  @t_w_q   = [TinyNN.tnn_null_ptr]
  @t_w_k   = [TinyNN.tnn_null_ptr]
  @t_w_v   = [TinyNN.tnn_null_ptr]
  @t_w_o   = TinyNN.tnn_null_ptr
  @t_w_ff1 = TinyNN.tnn_null_ptr
  @t_w_ff2 = TinyNN.tnn_null_ptr
end

Instance Attribute Details

#t_norm1_gamma ⇒ Object

Returns the value of attribute t_norm1_gamma.



87
88
89
 
# File 'lib/toy/ffi/tinynn.rb', line 87

def t_norm1_gamma
  @t_norm1_gamma
end

#t_norm2_gamma ⇒ Object

Returns the value of attribute t_norm2_gamma.



87
88
89
 
# File 'lib/toy/ffi/tinynn.rb', line 87

def t_norm2_gamma
  @t_norm2_gamma
end

#t_w_ff1 ⇒ Object

Returns the value of attribute t_w_ff1.



87
88
89
 
# File 'lib/toy/ffi/tinynn.rb', line 87

def t_w_ff1
  @t_w_ff1
end

#t_w_ff2 ⇒ Object

Returns the value of attribute t_w_ff2.



87
88
89
 
# File 'lib/toy/ffi/tinynn.rb', line 87

def t_w_ff2
  @t_w_ff2
end

#t_w_k ⇒ Object

Returns the value of attribute t_w_k.



87
88
89
 
# File 'lib/toy/ffi/tinynn.rb', line 87

def t_w_k
  @t_w_k
end

#t_w_o ⇒ Object

Returns the value of attribute t_w_o.



87
88
89
 
# File 'lib/toy/ffi/tinynn.rb', line 87

def t_w_o
  @t_w_o
end

#t_w_q ⇒ Object

Returns the value of attribute t_w_q.



87
88
89
 
# File 'lib/toy/ffi/tinynn.rb', line 87

def t_w_q
  @t_w_q
end

#t_w_v ⇒ Object

Returns the value of attribute t_w_v.



87
88
89
 
# File 'lib/toy/ffi/tinynn.rb', line 87

def t_w_v
  @t_w_v
end