Module: RubyLLM::Contract::TokenEstimator

Defined in:
lib/ruby_llm/contract/token_estimator.rb

Constant Summary collapse

CHARS_PER_TOKEN =

Heuristic: ~4 characters per token for English text. This is a rough estimate — actual tokenization varies by model and content. Intentionally conservative (overestimates slightly) to avoid surprise costs.

4

Class Method Summary collapse

Class Method Details

.estimate(messages) ⇒ Object 



11
12
13
14
15
16
 
# File 'lib/ruby_llm/contract/token_estimator.rb', line 11

def self.estimate(messages)
  return 0 unless messages.is_a?(Array)

  total_chars = messages.sum { |m| m[:content].to_s.length }
  (total_chars.to_f / CHARS_PER_TOKEN).ceil
end