Class: ViTTinyConfig

Inherits:
Object
  • Object
show all
Defined in:
lib/toy/models/toy_vit.rb

Overview

ViTTinyConfig — hyperparams for the ViT-Tiny variant. Reference: vit_tiny_patch16_224 (timm) — image 224×224, patch 16, d_model 192, n_heads 3 (d_head 64), d_ff 768, n_layers 12. For the smoke we run at smaller dims (image 16×16, patch 4, d_model 64, n_heads 4, d_ff 128, n_layers 2).

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(image_size, patch_size, num_channels, d_model, n_heads, d_ff, n_layers, num_classes, ln_eps) ⇒ ViTTinyConfig

Returns a new instance of ViTTinyConfig.



25
26
27
28
29
30
31
32
33
34
35
36
37
 
# File 'lib/toy/models/toy_vit.rb', line 25

def initialize(image_size, patch_size, num_channels, d_model, n_heads,
               d_ff, n_layers, num_classes, ln_eps)
  @image_size   = image_size
  @patch_size   = patch_size
  @num_channels = num_channels
  @d_model      = d_model
  @n_heads      = n_heads
  @d_head       = n_heads > 0 ? d_model / n_heads : 0
  @d_ff         = d_ff
  @n_layers     = n_layers
  @num_classes  = num_classes
  @ln_eps       = ln_eps
end

Instance Attribute Details

#d_ff ⇒ Object

Returns the value of attribute d_ff.



21
22
23
 
# File 'lib/toy/models/toy_vit.rb', line 21

def d_ff
  @d_ff
end

#d_head ⇒ Object

Returns the value of attribute d_head.



21
22
23
 
# File 'lib/toy/models/toy_vit.rb', line 21

def d_head
  @d_head
end

#d_model ⇒ Object

Returns the value of attribute d_model.



21
22
23
 
# File 'lib/toy/models/toy_vit.rb', line 21

def d_model
  @d_model
end

#image_size ⇒ Object

Returns the value of attribute image_size.



21
22
23
 
# File 'lib/toy/models/toy_vit.rb', line 21

def image_size
  @image_size
end

#ln_eps ⇒ Object

Returns the value of attribute ln_eps.



21
22
23
 
# File 'lib/toy/models/toy_vit.rb', line 21

def ln_eps
  @ln_eps
end

#n_heads ⇒ Object

Returns the value of attribute n_heads.



21
22
23
 
# File 'lib/toy/models/toy_vit.rb', line 21

def n_heads
  @n_heads
end

#n_layers ⇒ Object

Returns the value of attribute n_layers.



21
22
23
 
# File 'lib/toy/models/toy_vit.rb', line 21

def n_layers
  @n_layers
end

#num_channels ⇒ Object

Returns the value of attribute num_channels.



21
22
23
 
# File 'lib/toy/models/toy_vit.rb', line 21

def num_channels
  @num_channels
end

#num_classes ⇒ Object

Returns the value of attribute num_classes.



21
22
23
 
# File 'lib/toy/models/toy_vit.rb', line 21

def num_classes
  @num_classes
end

#patch_size ⇒ Object

Returns the value of attribute patch_size.



21
22
23
 
# File 'lib/toy/models/toy_vit.rb', line 21

def patch_size
  @patch_size
end

Class Method Details

.tiny ⇒ Object

The canonical ViT-Tiny shape (toy#73 A.3) — timm vit_tiny_patch16_224 with the 10-class smoke head: image 224, patch 16 (→ 196 patches), 3 channels, d=192, 3 heads (d_head 64 derives in initialize), d_ff 768, 12 layers, ln_eps 1e-5. This is the exact shape example 07 / lib/toy/run/train_vit.rb train and the one data/vit_smoke is preprocessed for. Additive sugar over .new — byte-identical to the equivalent 9-positional call. (The vit smoke’s 16/4/64-dim block is a DIFFERENT, env-driven shape —it stays on .new.)



48
49
50
 
# File 'lib/toy/models/toy_vit.rb', line 48

def self.tiny
  ViTTinyConfig.new(224, 16, 3, 192, 3, 768, 12, 10, 1.0e-5)
end