Class: Insika::BaselineStore

Inherits:
Object
  • Object
show all
Defined in:
lib/insika/baseline_store.rb

Overview

The ACCEPTED state of an agent's golden set (RFC-0008 §3.4, promoted to a store by RFC-0013 §3.7.3). One record per agent in the ConfigStore (scope "baselines"):

{ "at" => iso8601, "cases" => { "<case id>" => { "pass" => bool, "score" => n } } }

Exactly the shape Evals::Baseline.snapshot already produces, so the file and the record are the same document in two places — no converter to keep honest, the same discipline GoldenStore applies to the corpus.

Why it had to leave the file. evals/baseline.json works fine for the CLI, which runs from a checkout. The refinement gate does not: it runs inside a Railway deployment, scores a throwaway clone against the accepted state, and there is no checkout there to read. A gate that fell back to "no baseline found, nothing regressed" would be the most dangerous default in the system — so the baseline became a per-agent record, and the gate refuses when it is missing.

The file stays the export format and the seed for a fresh deploy (insika evals:baseline import|export), for the same reason the goldens' YAML does. It is also still what evals/run.rb --gate reads for the pre-merge check, which is a checkout-side job and should stay one.

Per AGENT and not one blob, unlike the file: a deployment serves many agents, a refinement run is about exactly one, and re-baselining the store after fixing agent A must not silently accept agent B's current state.

Constant Summary collapse

SCOPE = 
"baselines"

Instance Method Summary collapse

Constructor Details

#initialize(config_store:) ⇒ BaselineStore

Returns a new instance of BaselineStore.



33
34
35
 
# File 'lib/insika/baseline_store.rb', line 33

def initialize(config_store:)
  @cs = config_store
end

Instance Method Details

#agents ⇒ Object

-> [String] agents with a recorded baseline.



60
 
# File 'lib/insika/baseline_store.rb', line 60

def agents = @cs.keys(SCOPE)

#delete(agent_id) ⇒ Object

-> bool (did it exist?). Removing a baseline disables the gate for that agent, which is the honest consequence and not a side effect worth hiding.



57
 
# File 'lib/insika/baseline_store.rb', line 57

def delete(agent_id) = !!@cs.delete(SCOPE, agent_id.to_s)

#get(agent_id) ⇒ Object

-> { "at" =>, "cases" => … } | nil. nil means NOT RECORDED, which every caller must treat as "cannot gate", never as "nothing regressed".



39
40
41
 
# File 'lib/insika/baseline_store.rb', line 39

def get(agent_id)
  @cs.get(SCOPE, agent_id.to_s)
end

#put(agent_id, snapshot, at: nil) ⇒ Object

-> the stored record. snapshot is Evals::Baseline.snapshot output (or the equivalent hash); at is stamped here when the snapshot carries none.

Raises:



45
46
47
48
49
50
51
52
53
 
# File 'lib/insika/baseline_store.rb', line 45

def put(agent_id, snapshot, at: nil)
  raw = Coercion.deep_stringify(snapshot || {})
  cases = raw["cases"]
  raise Insika::ValidationError, "baseline needs a 'cases' mapping" unless cases.is_a?(Hash)

  record = { "at" => Coercion.presence(raw["at"]) || at || timestamp, "cases" => cases }
  @cs.put(SCOPE, agent_id.to_s, record)
  record
end

#size(agent_id) ⇒ Object

How many cases the accepted state covers. 0 and nil are different answers: nil = never recorded, 0 = recorded and empty (every case was skipped), and only the first is a configuration mistake.



65
66
67
68
 
# File 'lib/insika/baseline_store.rb', line 65

def size(agent_id)
  record = get(agent_id)
  record && (record["cases"] || {}).size
end