Module: Batchwatch

Defined in:
lib/batchwatch/spool.rb,
lib/batchwatch/client.rb

Defined Under Namespace

Classes: Client, HTTPError, Spool, Tracking

Constant Summary collapse

MAX_BATCH =

The server's cap on one POST /v1/calls/complete.

500
MAX_BYTES =

Cap on the spool file. If batchwatch is down for a week, a busy pipeline must not fill the user's disk. When the cap is reached, the measurement is dropped - and that is the right choice: his machine is not our storage.

5 * 1024 * 1024
VERSION = 
"0.2.1"
SPOOL_INTERVAL_S =

How often, at most, we try to drain the spool on our own.

60.0
ID_WAIT_S =

How long completion waits for the start call's id to land. Only the background thread waits - the caller is long gone.

3.0
ALLOWED_FIELDS =

Field names allowed to leave the machine. Everything else is not in the body. The test test_no_content.rb pins this list. Same fields as the Python, Go, TypeScript and .NET clients.

The last four (acted_verdict, deadline_s, quoted_p50_s, quoted_p90_s) are the outcome measurement (#101): the advice we gave ourselves, attached to the later completion so the SERVER can compare its own measured duration with what we promised. All of it is numbers and a decision we made ourselves - no new PII.

%w[
  provider model mode endpoint requests
  input_tokens output_tokens started_at ended_at
  status ttfb_ms source
  acted_verdict deadline_s quoted_p50_s quoted_p90_s
].freeze

Class Method Summary collapse

Class Method Details

.default_spool ⇒ Object 



76
77
78
79
80
81
82
83
 
# File 'lib/batchwatch/client.rb', line 76

def self.default_spool
  v = ENV["BATCHWATCH_SPOOL"]
  unless v.nil?
    # An empty string turns the spool off.
    return v.empty? ? nil : v
  end
  File.join(Dir.tmpdir, "batchwatch-spool.jsonl")
end

.default_timeout ⇒ Object 



72
73
74
 
# File 'lib/batchwatch/client.rb', line 72

def self.default_timeout
  (ENV["BATCHWATCH_TIMEOUT"] || "2.0").to_f
end

.default_url ⇒ Object 



68
69
70
 
# File 'lib/batchwatch/client.rb', line 68

def self.default_url
  ENV["BATCHWATCH_URL"] || "https://batchwatch.dev"
end

.idem_complete(group) ⇒ Object

Per-request key for POST /v1/calls/complete. Derived from the group being sent, exactly like backfill.py: a single record is just a group of one. A replay of the SAME records chunks them identically (the spool preserves the order), so the same request carries the same key.



155
156
157
158
159
160
161
162
 
# File 'lib/batchwatch/client.rb', line 155

def self.idem_complete(group)
  return nil if group.nil? || group.empty?

  first = group.first
  last = group.last
  "bw-complete-#{idem_field(first, 'provider')}-#{idem_field(first, 'started_at')}-" \
    "#{group.length}-#{idem_field(last, 'ended_at')}"
end

.idem_start(body) ⇒ Object

Per-record key for POST /v1/calls (the start call).

Keeps the readable provider/model prefix for logs, then a content hash of the full (sanitized) start body. started_at is second-resolution, so two DISTINCT measurements of the same provider+model within one second would otherwise share a key - and the server refuses a reused key that carries a different body (409 body_differs), silently dropping the second measurement. Hashing the whole body keeps distinct measurements apart; an identical body (a retry or a die-and-reflush replay) still hashes the same and dedupes, so #30 holds. This key is only ever computed live at the start POST, never reconstructed from a spool line, so idem_complete's cross-client spool-replay key format is untouched.



146
147
148
149
 
# File 'lib/batchwatch/client.rb', line 146

def self.idem_start(body)
  "bw-start-#{idem_field(body, 'provider')}-#{idem_field(body, 'model')}-" \
    "#{content_hash(body)}"
end

.iso(time) ⇒ Object 



96
97
98
 
# File 'lib/batchwatch/client.rb', line 96

def self.iso(time)
  time.utc.strftime("%Y-%m-%dT%H:%M:%SZ")
end

.outcome_fields(advice) ⇒ Object

The fields from a stored advice that may be attached to a completion (#101).

Only non-nil values come along: if a percentile or a deadline is missing, the field is OMITTED entirely - we do not invent a zero (rule #30). With no advice at all the result is an empty hash, so nothing is attached.



196
197
198
199
200
201
202
203
204
 
# File 'lib/batchwatch/client.rb', line 196

def self.outcome_fields(advice)
  return {} if advice.nil?

  out = { "acted_verdict" => advice["acted_verdict"] }
  %w[deadline_s quoted_p50_s quoted_p90_s].each do |name|
    out[name] = advice[name] unless advice[name].nil?
  end
  out
end

.sanitize(body) ⇒ Object

Drop everything not on the allowlist.

The last stop before the network. Even though no public method accepts free text, THIS function is the place you can point to when someone asks "how do you know a prompt cannot get out".



90
91
92
93
94
 
# File 'lib/batchwatch/client.rb', line 90

def self.sanitize(body)
  body.each_with_object({}) do |(k, v), out|
    out[k.to_s] = v if ALLOWED_FIELDS.include?(k.to_s)
  end
end

.seconds(max_wait) ⇒ Object

Translate a max_wait into seconds, or nil if we cannot make sense of it.

The caller typically writes "15m", "1h", "30s" or "2d" - the same language /v1/should-i-batch itself accepts. A bare number is read as seconds. If we cannot parse it, we send nil rather than a guess (rule #30).



169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
 
# File 'lib/batchwatch/client.rb', line 169

def self.seconds(max_wait)
  return nil if max_wait.nil?
  return max_wait.to_f if max_wait.is_a?(Numeric)

  s = max_wait.to_s.strip.downcase
  return nil if s.empty?

  factor = { "s" => 1.0, "m" => 60.0, "h" => 3600.0, "d" => 86_400.0 }
  unit = s[-1]
  if factor.key?(unit)
    num = s[0..-2]
    mult = factor[unit]
  else
    num = s
    mult = 1.0
  end
  num = num.strip
  return nil unless num.match?(/\A-?\d+(\.\d+)?\z/)

  num.to_f * mult
end

.track(model, **kw, &blk) ⇒ Object

Shortcut that uses a shared default client, to get started quickly.



666
667
668
669
 
# File 'lib/batchwatch/client.rb', line 666

def self.track(model, **kw, &blk)
  @default_lock.synchronize { @default ||= Client.new }
  @default.track(model, **kw, &blk)
end