Class: State

Inherits:
Object
  • Object
show all
Defined in:
lib/toy/serve/openai/server.rb

Overview

—- Inference state. Class instance held as a CONSTANT so spinel

emits a typed slot for it (same pattern as inference_api.rb /
openai_api.rb). ----

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ State

Returns a new instance of State.



57
58
59
60
61
62
63
64
 
# File 'lib/toy/serve/openai/server.rb', line 57

def initialize
  @cfg  = nil
  @kv   = nil
  @gguf = TinyNN.tnn_null_ptr
  @model_name = MODEL_NAME
  @ready      = false
  @req_seq    = 0
end

Instance Attribute Details

#cfg ⇒ Object

:req_seq is a monomorphic Integer slot (always Integer) for the per-request monotonic counter the serving events emitter uses as a deterministic-shaped request_id. Held on the already-CONSTANT-held STATE so Spinel emits a typed slot; NO new polymorphic accessor (landmine #16).



56
57
58
 
# File 'lib/toy/serve/openai/server.rb', line 56

def cfg
  @cfg
end

#gguf ⇒ Object

:req_seq is a monomorphic Integer slot (always Integer) for the per-request monotonic counter the serving events emitter uses as a deterministic-shaped request_id. Held on the already-CONSTANT-held STATE so Spinel emits a typed slot; NO new polymorphic accessor (landmine #16).



56
57
58
 
# File 'lib/toy/serve/openai/server.rb', line 56

def gguf
  @gguf
end

#kv ⇒ Object

:req_seq is a monomorphic Integer slot (always Integer) for the per-request monotonic counter the serving events emitter uses as a deterministic-shaped request_id. Held on the already-CONSTANT-held STATE so Spinel emits a typed slot; NO new polymorphic accessor (landmine #16).



56
57
58
 
# File 'lib/toy/serve/openai/server.rb', line 56

def kv
  @kv
end

#model_name ⇒ Object

:req_seq is a monomorphic Integer slot (always Integer) for the per-request monotonic counter the serving events emitter uses as a deterministic-shaped request_id. Held on the already-CONSTANT-held STATE so Spinel emits a typed slot; NO new polymorphic accessor (landmine #16).



56
57
58
 
# File 'lib/toy/serve/openai/server.rb', line 56

def model_name
  @model_name
end

#ready ⇒ Object

:req_seq is a monomorphic Integer slot (always Integer) for the per-request monotonic counter the serving events emitter uses as a deterministic-shaped request_id. Held on the already-CONSTANT-held STATE so Spinel emits a typed slot; NO new polymorphic accessor (landmine #16).



56
57
58
 
# File 'lib/toy/serve/openai/server.rb', line 56

def ready
  @ready
end

#req_seq ⇒ Object

:req_seq is a monomorphic Integer slot (always Integer) for the per-request monotonic counter the serving events emitter uses as a deterministic-shaped request_id. Held on the already-CONSTANT-held STATE so Spinel emits a typed slot; NO new polymorphic accessor (landmine #16).



56
57
58
 
# File 'lib/toy/serve/openai/server.rb', line 56

def req_seq
  @req_seq
end