Class: Tep::Llm::OpenAI::Completion

Inherits:
Object
  • Object
show all
Defined in:
lib/tep/openai_server.rb

Overview

A backend’s generation result: the decoded text + token usage.

token_ids carries the GENERATED token IDs for an IDs-only backend (no detokenizer): when non-empty, CompletionsHandler emits them as choices.ids alongside text (which such a backend leaves “”), matching the “tokenize/detokenize client-side” serving contract. Text backends leave token_ids empty and the ids field is omitted. finish_reason defaults to “stop”; a fixed-length greedy backend sets “length”.

id is the completion id echoed as the response ‘id` (and the inference event’s request_id). It defaults to “cmpl-tep”; a backend that mints its own per-request ids (e.g. so a downstream byte-exact ingest keeps unique ids) sets it. Leaving it default keeps existing consumers byte-identical.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Completion

Returns a new instance of Completion.



286
287
288
289
290
291
292
293
294
295
296
 
# File 'lib/tep/openai_server.rb', line 286

def initialize
  @text              = ""
  @prompt_tokens     = 0
  @completion_tokens = 0
  # Typed-empty Array[Integer] seed (the [0]; delete_at(0) landmine
  # pattern) so Spinel emits an IntArray slot, not poly.
  @token_ids         = [0]
  @token_ids.delete_at(0)
  @finish_reason     = "stop"
  @id                = "cmpl-tep"
end

Instance Attribute Details

#completion_tokens ⇒ Object

Returns the value of attribute completion_tokens.



282
283
284
 
# File 'lib/tep/openai_server.rb', line 282

def completion_tokens
  @completion_tokens
end

#finish_reason ⇒ Object

Returns the value of attribute finish_reason.



283
284
285
 
# File 'lib/tep/openai_server.rb', line 283

def finish_reason
  @finish_reason
end

#id ⇒ Object

Returns the value of attribute id.



284
285
286
 
# File 'lib/tep/openai_server.rb', line 284

def id
  @id
end

#prompt_tokens ⇒ Object

Returns the value of attribute prompt_tokens.



282
283
284
 
# File 'lib/tep/openai_server.rb', line 282

def prompt_tokens
  @prompt_tokens
end

#text ⇒ Object

Returns the value of attribute text.



282
283
284
 
# File 'lib/tep/openai_server.rb', line 282

def text
  @text
end

#token_ids ⇒ Object

Returns the value of attribute token_ids.



283
284
285
 
# File 'lib/tep/openai_server.rb', line 283

def token_ids
  @token_ids
end