Module: Brute::TokenCounter::Rendering

Defined in:
lib/brute/token_counter.rb

Overview

The text a counter measures.

One rendering, so two counters cannot disagree about what the conversation even is -- and it is the same text a summariser reads, which is why it says who spoke and what was called rather than being the shortest thing that could be measured.

Class Method Summary collapse

Class Method Details

.conversation(messages) ⇒ Object 



63
 
# File 'lib/brute/token_counter.rb', line 63

def self.conversation(messages) = Array(messages).map { |message| message(message) }.join("\n")

.message(message) ⇒ Object 



65
66
67
68
69
70
71
72
73
74
75
76
77
 
# File 'lib/brute/token_counter.rb', line 65

def self.message(message)
  parts = ["#{message.role}:", message.content.to_s]

  message.tool_calls&.each do |call|
    parts << "#{call.name}(#{JSON.generate(call.arguments)}) -> #{call.id}"
  end

  unless message.tool_call_id.nil?
    parts << "(answering #{message.tool_call_id})"
  end

  parts.reject(&:empty?).join(" ")
end

.tools(tools) ⇒ Object

The schemas as the provider is given them, which is what they cost.



80
81
82
83
84
85
86
87
88
 
# File 'lib/brute/token_counter.rb', line 80

def self.tools(tools)
  wrapped = Brute::Tools::Adapter.wrap_all(tools || [])

  if wrapped.empty?
    ""
  else
    JSON.generate(wrapped.values.map(&:to_h))
  end
end