Module: Ask::TokenUsage

Defined in:
lib/ask/token_usage.rb,
lib/ask/token_usage/wallet.rb,
lib/ask/token_usage/pricing.rb,
lib/ask/token_usage/version.rb,
lib/ask/token_usage/activity.rb,
lib/ask/token_usage/counting.rb,
lib/ask/token_usage/ledger_entry.rb,
lib/ask/token_usage/stores/store.rb,
lib/ask/token_usage/configuration.rb,
lib/ask/token_usage/stores/memory.rb,
lib/ask/token_usage/instrumentation.rb,
lib/ask/token_usage/callback_context.rb,
lib/ask/token_usage/activity_registry.rb,
lib/ask/token_usage/insufficient_tokens.rb

Overview

Token accounting for anything you sell by measured tokens.

The gem is Railless by design: it counts real tokens (tiktoken), prices them against a per-million rate (money), and runs a wallet/ledger engine on top of a pluggable store. Persistence is the host's job — ship the in-memory store for scripts/tests, or hand in an ActiveRecord-backed store (ask-token-usage-rails) inside Rails.

Ask::TokenUsage.configure do |c|
c.price_per_1m = Money.from_amount(100, "USD")
end

Ask::TokenUsage.activity(:chat_message) do |params|
Ask::TokenUsage.count_tokens(params[:input]) +
  Ask::TokenUsage.count_tokens(params[:output])
end

wallet = Ask::TokenUsage.wallet_for("user:42")
wallet.grant!(1000, reason: :signup)
wallet.spend!(:chat_message, input: "hi", output: "yo") do
LLM.chat(...)   # only charged when the block succeeds
end
wallet.balance   # => 996

Defined Under Namespace

Modules: Counting, Instrumentation, Pricing, Stores Classes: Activity, ActivityRegistry, CallbackContext, Configuration, Error, InsufficientTokens, LedgerEntry, Wallet

Constant Summary collapse

VERSION = 
"0.1.0"

Class Method Summary collapse

Class Method Details

.activities ⇒ Object 



98
99
100
 
# File 'lib/ask/token_usage.rb', line 98

def activities
  @activities ||= ActivityRegistry.new
end

.activity(name, cost: nil, &block) ⇒ Object

Register an activity. cost: is a fixed integer; without it the block receives params and must return an integer token count.



88
89
90
 
# File 'lib/ask/token_usage.rb', line 88

def activity(name, cost: nil, &block)
  activities.register(name, cost: cost, &block)
end

.cents_for(count, currency: nil) ⇒ Object

The price of count tokens expressed in cents.



82
83
84
 
# File 'lib/ask/token_usage.rb', line 82

def cents_for(count, currency: nil)
  Pricing.cents_for(count, currency: currency)
end

.config ⇒ Object

Global configuration.



48
49
50
 
# File 'lib/ask/token_usage.rb', line 48

def config
  @config ||= Configuration.new
end

.configure {|config| ... } ⇒ Object

Yields:



52
53
54
 
# File 'lib/ask/token_usage.rb', line 52

def configure
  yield config
end

.count_tokens(text, model: nil) ⇒ Object

Count the tokens in text using tiktoken. Pass model: for a model-aware encoding; unknown models fall back to cl100k_base.



58
59
60
 
# File 'lib/ask/token_usage.rb', line 58

def count_tokens(text, model: nil)
  Counting.count(text, model: model)
end

.estimate(activity_name, params = {}) ⇒ Object

The estimated token cost of running activity with params, without spending anything.



94
95
96
 
# File 'lib/ask/token_usage.rb', line 94

def estimate(activity_name, params = {})
  activities.fetch(activity_name).cost(params)
end

.price_per(count) ⇒ Object

Price of count tokens at the configured per-token rate.

Ask::TokenUsage.price_per(1_000)      # => Money($0.10)
Ask::TokenUsage.price_per(1_000_000)   # => Money($100.00)


67
68
69
 
# File 'lib/ask/token_usage.rb', line 67

def price_per(count)
  Pricing.price_of(count)
end

.price_per_token ⇒ Object

The configured Money value of one billing token.



72
73
74
 
# File 'lib/ask/token_usage.rb', line 72

def price_per_token
  config.price_per_token
end

.reset! ⇒ Object

Reset configuration and activity registry (mainly for tests).



109
110
111
112
 
# File 'lib/ask/token_usage.rb', line 109

def reset!
  @config = Configuration.new
  @activities = ActivityRegistry.new
end

.tokens_for(money) ⇒ Object

How many billing tokens a Money amount buys at the configured rate.



77
78
79
 
# File 'lib/ask/token_usage.rb', line 77

def tokens_for(money)
  Pricing.tokens_for(money)
end

.wallet_for(owner) ⇒ Object

Build a wallet for owner. owner is an opaque identifier the store keys on.



104
105
106
 
# File 'lib/ask/token_usage.rb', line 104

def wallet_for(owner)
  Wallet.new(owner: owner, store: config.store, time: config.time)
end