Class: Ignis::AI::Server

Inherits:
Object
  • Object
show all
Defined in:
lib/nnw/ai/server.rb

Overview

OpenAI-compatible inference server.

Routes:

POST /v1/completions       — text completion (SSE streaming)
POST /v1/chat/completions  — chat completion (SSE streaming)
GET  /v1/models            — list loaded models
POST /v1/embeddings        — compute embeddings

Integrates with WNAIS HTTP/2 server if available, otherwise uses a simple TCP server for standalone operation.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(model, tokenizer, model_name: "nnw-model", host: "0.0.0.0", port: 8080) ⇒ Server

Returns a new instance of Server.

Parameters:

  • model (Transformer::Model)
  • tokenizer (Tokenizer)
  • model_name (String) (defaults to: "nnw-model")
  • host (String) (defaults to: "0.0.0.0")
  • port (Integer) (defaults to: 8080) 


30
31
32
33
34
35
36
 
# File 'lib/nnw/ai/server.rb', line 30

def initialize(model, tokenizer, model_name: "nnw-model", host: "0.0.0.0", port: 8080)
  @generator = TextGenerator.new(model, tokenizer)
  @model_name = model_name
  @host = host
  @port = port
  @batch_processor = BatchProcessor.new(model, tokenizer)
end

Instance Attribute Details

#generator ⇒ TextGenerator (readonly)

Returns:



20
21
22
 
# File 'lib/nnw/ai/server.rb', line 20

def generator
  @generator
end

#model_name ⇒ String (readonly)

Returns model name for API responses.

Returns:

  • (String) —

    model name for API responses



23
24
25
 
# File 'lib/nnw/ai/server.rb', line 23

def model_name
  @model_name
end

Instance Method Details

#start! ⇒ void

This method returns an undefined value.

Start the server.



40
41
42
43
44
45
46
47
48
49
50
51
 
# File 'lib/nnw/ai/server.rb', line 40

def start!
  Ignis.logger.info("Ignis AI Server starting on #{@host}:#{@port}")
  Ignis.logger.info("Model: #{@model_name} (#{@generator.model.num_parameters} params)")

  @batch_processor.start!

  if defined?(WNAIS::Server)
    start_wnais_server!
  else
    start_tcp_server!
  end
end

#stop! ⇒ void

This method returns an undefined value.

Stop the server.



55
56
57
58
59
 
# File 'lib/nnw/ai/server.rb', line 55

def stop!
  @batch_processor.stop!
  @server_thread&.kill
  Ignis.logger.info("Server stopped")
end