Module: Legion::Extensions::Llamacpp::Helpers::Usage

Defined in:
lib/legion/extensions/llamacpp/helpers/usage.rb

Constant Summary collapse

EMPTY_USAGE = 
{
  input_tokens:  0,
  output_tokens: 0,
  total_tokens:  0
}.freeze

Class Method Summary collapse

Class Method Details

.from_response(body) ⇒ Object 



16
17
18
19
20
21
22
23
24
25
26
27
 
# File 'lib/legion/extensions/llamacpp/helpers/usage.rb', line 16

def from_response(body)
  return EMPTY_USAGE.dup unless body.is_a?(Hash)

  usage = body['usage']
  return EMPTY_USAGE.dup unless usage.is_a?(Hash)

  {
    input_tokens:  usage['prompt_tokens'] || 0,
    output_tokens: usage['completion_tokens'] || 0,
    total_tokens:  usage['total_tokens'] || 0
  }
end