Class: Toy::LLM::Blocks::TransformerBlockCtx

Inherits:
Object
  • Object
show all
Defined in:
lib/toy/llm/blocks/transformer_block.rb,
lib/toy/llm/blocks/transformer_block_cuda.rb,
lib/toy/llm/blocks/transformer_block_metal.rb

Overview

Per-forward read-only context: the 14 config/handle values the body previously read off the cache as @ivars, packed fresh each forward.

Plain class with an explicit POSITIONAL initialize (NO default args, NO kwargs) — the same Spinel-safe pattern as RoPE::Cfg. We do NOT use Struct.new here, and the member names keep the original cache’s ‘@seq_*` / `@t_seq_*` prefixes VERBATIM. Both choices are forced by Spinel’s whole-program type inference, which unifies an accessor’s return type across every class exposing the same accessor name:

- a Struct's auto-generated `#n_heads` collided with
  SmolLM2Config#n_heads / CausalSelfAttention#n_heads, breaking an
  unrelated call (Toy::SmolLM2#algorithm's `@cfg.n_heads.to_s`);
- short names `#t` / `#b` collided with Toy::Linear#b (a weight
  tensor / FloatArray), so `ctx.b`/`ctx.t` were inferred as
  FloatArray* and poisoned RoPE.apply_2d / GQA.attention's integer
  batch args.

The ‘@seq_*`-prefixed names were chosen on the cache precisely for this type-isolation (see lib/llama_seq_forward_ffi.rb header), so reusing them here keeps every accessor’s return type local and concrete. Carries values, no behavior.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(seq_scale, seq_eps, seq_n_kv, seq_n_heads, seq_group_size, seq_has_qkv_bias, seq_weight_dtype, seq_lora_q_enabled, t_seq_positions, t_seq_rope_freq_factors, seq_rope_cfg, seq_t, seq_b, t_seq_attn_mask) ⇒ TransformerBlockCtx

Returns a new instance of TransformerBlockCtx.



67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 67

def initialize(seq_scale, seq_eps, seq_n_kv, seq_n_heads, seq_group_size,
               seq_has_qkv_bias, seq_weight_dtype, seq_lora_q_enabled,
               t_seq_positions, t_seq_rope_freq_factors, seq_rope_cfg,
               seq_t, seq_b, t_seq_attn_mask)
  @seq_scale               = seq_scale
  @seq_eps                 = seq_eps
  @seq_n_kv                = seq_n_kv
  @seq_n_heads             = seq_n_heads
  @seq_group_size          = seq_group_size
  @seq_has_qkv_bias        = seq_has_qkv_bias
  @seq_weight_dtype        = seq_weight_dtype
  @seq_lora_q_enabled      = seq_lora_q_enabled
  @t_seq_positions         = t_seq_positions
  @t_seq_rope_freq_factors = t_seq_rope_freq_factors
  @seq_rope_cfg            = seq_rope_cfg
  @seq_t                   = seq_t
  @seq_b                   = seq_b
  @t_seq_attn_mask         = t_seq_attn_mask
end

Instance Attribute Details

#seq_b ⇒ Object

Returns the value of attribute seq_b.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def seq_b
  @seq_b
end

#seq_eps ⇒ Object

Returns the value of attribute seq_eps.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def seq_eps
  @seq_eps
end

#seq_group_size ⇒ Object

Returns the value of attribute seq_group_size.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def seq_group_size
  @seq_group_size
end

#seq_has_qkv_bias ⇒ Object

Returns the value of attribute seq_has_qkv_bias.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def seq_has_qkv_bias
  @seq_has_qkv_bias
end

#seq_lora_q_enabled ⇒ Object

Returns the value of attribute seq_lora_q_enabled.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def seq_lora_q_enabled
  @seq_lora_q_enabled
end

#seq_n_heads ⇒ Object

Returns the value of attribute seq_n_heads.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def seq_n_heads
  @seq_n_heads
end

#seq_n_kv ⇒ Object

Returns the value of attribute seq_n_kv.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def seq_n_kv
  @seq_n_kv
end

#seq_rope_cfg ⇒ Object

Returns the value of attribute seq_rope_cfg.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def seq_rope_cfg
  @seq_rope_cfg
end

#seq_scale ⇒ Object

Returns the value of attribute seq_scale.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def seq_scale
  @seq_scale
end

#seq_t ⇒ Object

Returns the value of attribute seq_t.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def seq_t
  @seq_t
end

#seq_weight_dtype ⇒ Object

Returns the value of attribute seq_weight_dtype.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def seq_weight_dtype
  @seq_weight_dtype
end

#t_seq_attn_mask ⇒ Object

Returns the value of attribute t_seq_attn_mask.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def t_seq_attn_mask
  @t_seq_attn_mask
end

#t_seq_positions ⇒ Object

Returns the value of attribute t_seq_positions.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def t_seq_positions
  @t_seq_positions
end

#t_seq_rope_freq_factors ⇒ Object

Returns the value of attribute t_seq_rope_freq_factors.



62
63
64
 
# File 'lib/toy/llm/blocks/transformer_block.rb', line 62

def t_seq_rope_freq_factors
  @t_seq_rope_freq_factors
end