Module: Brute::TokenCounter::Rendering

Defined in:
lib/brute/token_counter.rb

Overview

The text a counter measures.

One rendering, so two counters cannot disagree about what the conversation even is -- and it is the same text a summariser reads, which is why it says who spoke and what was called rather than being the shortest thing that could be measured.

Class Method Summary collapse

Class Method Details

.conversation(messages) ⇒ Object 



61
 
# File 'lib/brute/token_counter.rb', line 61

def self.conversation(messages) = Array(messages).map { |message| message(message) }.join("\n")

.message(message) ⇒ Object 



63
64
65
66
67
68
69
70
71
72
73
74
75
 
# File 'lib/brute/token_counter.rb', line 63

def self.message(message)
  parts = ["#{message.role}:", message.content.to_s]

  message.tool_calls&.each do |call|
    parts << "#{call.name}(#{JSON.generate(call.arguments)}) -> #{call.id}"
  end

  unless message.tool_call_id.nil?
    parts << "(answering #{message.tool_call_id})"
  end

  parts.reject(&:empty?).join(" ")
end

.tools(tools) ⇒ Object

The schemas as the provider is given them, which is what they cost.



78
79
80
81
82
83
84
85
86
 
# File 'lib/brute/token_counter.rb', line 78

def self.tools(tools)
  wrapped = Brute::Tools::Adapter.wrap_all(tools || [])

  if wrapped.empty?
    ""
  else
    JSON.generate(wrapped.values.map(&:to_h))
  end
end