Class: Insika::Commands::GateRefinement

Inherits:
Object
  • Object
show all
Defined in:
lib/insika/commands/gate_refinement.rb

Overview

Control command: takes a CANDIDATE for a completed refinement run, validates it against the agent's write allowlist, and scores it by actually running it — clone the agent, apply the edits to the clone, replay the golden set, compare to the accepted baseline.

Synchronous like run_refinement, and for the same reason: it is operator-paced work with a human waiting on the answer. It is NOT cheap — the replay is a real conversation per golden case — so it is fired deliberately, never on a timer (the engine has no scheduler and this RFC does not add one).

Payload:

run_id     (required) a run in :completed — the evidence the candidate answers
candidate  { proposer?, rationale?, edits: [ {file, op, anchor?, before, after,
         addresses?} ] } — or omit it and pass `propose: true` to have the
         configured model(s) write one from the run's findings.
propose    truthy — write the candidate with the agent's proposer PANEL.
tolerance  Float — overrides the configured judge-score tolerance for this gate

-> the Run, now :awaiting_approval (gate passed), :applied (mode: auto_apply and the candidate cleared the extra bar) or :rejected (the gate said no).

propose: true is required rather than inferred from a missing candidate. Both a proposal and a gate cost real provider money, and a caller who simply forgot the candidate should get an error, not a bill.

Two refusals happen BEFORE anything is cloned, because both mean the operator has not actually enabled this: an agent still in mode: report (writing is opt-in and an absent config is report-only), and a candidate whose every edit was dropped (stale, off-allowlist, over budget). Neither is worth a provider bill.

makes the proposal a PANEL and the run's spend a BUDGET. The command's shape does not change: N models write N candidates, the gate scores each, and the best survivor is the one proposal a human is shown. A deployment with one proposer and no budget behaves exactly as did.

Constant Summary collapse

WRITE_MODES = 
%w[propose auto_apply].freeze
DEFAULT_AUTO_APPLY_MAX_EDITS =

An unattended write is held to a SMALLER bar than an approved one: one edit, by default. auto_apply exists so a well-understood, well-golden'd agent can fix a typo'd instruction overnight — not so it can rewrite three files while nobody is looking. The operator raises it deliberately.

1

Instance Method Summary collapse

Constructor Details

#initialize(profiles:, refinement_store:, agent_file_store:, gate:, event_stream:, proposer_factory: nil, resolver: nil) ⇒ GateRefinement

proposer_factory: ->(refinement_config) { [Refinement::Proposer] | Proposer | nil }. Optional — a deployment with none can still gate a candidate that arrives from the API, which is exactly what shipped before the proposer existed. resolver: the :resolve_refinement handler, used ONLY for mode: auto_apply. Reusing it rather than writing files here is what keeps auto-apply honest: it goes through the same staleness re-check, the same versioned write and the same :refinement_applied event a human approval does.



57
58
59
60
61
62
63
64
65
66
 
# File 'lib/insika/commands/gate_refinement.rb', line 57

def initialize(profiles:, refinement_store:, agent_file_store:, gate:, event_stream:,
               proposer_factory: nil, resolver: nil)
  @profiles = ProfileSource.coerce(profiles)
  @runs = refinement_store
  @agent_files = agent_file_store
  @gate = gate
  @event_stream = event_stream
  @proposer_factory = proposer_factory
  @resolver = resolver
end

Instance Method Details

#call(command) ⇒ Object

Raises:



68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/insika/commands/gate_refinement.rb', line 68

def call(command)
  p = AgentPayload.symbolize(command.payload)
  run_id = AgentPayload.presence(p[:run_id])
  raise Insika::ValidationError, "run_id is required" if run_id.nil?

  run = @runs.find(run_id) || (raise Insika::NotFoundError, "refinement run not found: #{run_id}")
  profile = @profiles[run.agent_id] ||
            (raise Insika::NotFoundError, "agent '#{run.agent_id}' not configured")
  config = refinement_config(profile)
  require_write_mode!(config, run.agent_id)
  require_completed!(run)

  result = run_panel(run, config, p)
  report = (result.winner || Refinement::Panel.best_refusal(result.entries)).report

  gated = @runs.gated(run.id, report: report, panel: result.entries,
                              cost: result.budget)
  emit(:refinement_gated, gated, passed: report.passed, reason: report.reason,
                                 cases: report.cases, passed_cases: report.passed_cases,
                                 regressions: report.regressions.size,
                                 candidates: result.entries.size,
                                 tokens: result.budget.to_h["spent"])
  auto_apply(gated, config, report) || gated
end