Class: Insika::Refinement::Panel

Inherits:
Object
  • Object
show all
Defined in:
lib/insika/refinement/panel.rb

Overview

The proposer PANEL (RFC-0013 §3.9 / §3.5): N models write N independent candidates, the gate scores each one by replaying the golden set, and the best SURVIVOR becomes the proposal a human is asked about.

Independent, not consensus-seeking. Two models agreeing on wording is weak evidence and a golden case passing is strong evidence (D7), so convergence only ever breaks a tie between candidates the gate already ranked equal.

A panel of one is phase C unchanged, which is why there is no second code path: refinement.proposer (a single ref) resolves to a one-element panel.

Defined Under Namespace

Classes: Entry, Result

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(gate:, proposers: [], budget: Budget.new, fan_out: nil) ⇒ Panel

gate: a Refinement::Gate (anything answering #score). proposers: [Refinement::Proposer], already resolved by ProposerFactory.panel. budget: a Budget. The default is unlimited — a deployment that configured none gets phase C's behaviour, which had no ceiling either.



84
85
86
87
88
89
 
# File 'lib/insika/refinement/panel.rb', line 84

def initialize(gate:, proposers: [], budget: Budget.new, fan_out: nil)
  @gate = gate
  @proposers = Array(proposers)
  @budget = budget
  @fan_out = fan_out || Insika::SubagentGraph.fan_out_cap
end

Class Method Details

.best_refusal(entries) ⇒ Object

The refusal to RECORD when nothing survived: the entry that got furthest, because "1 regression on quotes" tells an operator something and "budget exhausted" tells them only that the run stopped.



127
128
129
130
 
# File 'lib/insika/refinement/panel.rb', line 127

def self.best_refusal(entries)
  entries.find { |e| e.report&.cases.to_i.positive? } ||
    entries.find { |e| e.report } || entries.first
end

.rank(entries) ⇒ Object

-> the best SURVIVOR, or nil when none passed.

Highest graded score first; ties broken by the fewest edits (a smaller diff is a smaller bet), then by how many proposers converged on it (§3.5). min_by over a negated tuple keeps the comparison in one place and stays stable, so two genuinely indistinguishable candidates resolve to the first proposer the operator listed rather than to whichever fiber finished first.



119
120
121
122
 
# File 'lib/insika/refinement/panel.rb', line 119

def self.rank(entries)
  entries.select(&:passed?)
         .min_by { |e| [-e.report.passed_cases, e.candidate.edits.size, -e.converged] }
end

Instance Method Details

#run(agent_id:, run_id:, findings:, files:, allowlist:, contents:, limits: {}, raw: nil, tolerance: nil) {|entries| ... } ⇒ Object

Proposes (unless raw is given), builds, gates and ranks. Yields the built entries BEFORE any of them is scored, so the caller can record the panel and move the run to :gating — the gate is the slow part and a run that says nothing until it finishes looks hung.

-> Result. Raises ValidationError when there is nothing gateable at all, because that is an operator-facing refusal ("every edit was dropped, here is why"), not a verdict about the agent.

Yields:

  • (entries) 


99
100
101
102
103
104
105
106
107
108
109
110
 
# File 'lib/insika/refinement/panel.rb', line 99

def run(agent_id:, run_id:, findings:, files:, allowlist:, contents:, limits: {},
        raw: nil, tolerance: nil)
  proposals, failed = raw ? [[raw], []] : propose(agent_id, findings, files, limits)
  # A candidate that ARRIVED (Studio form, API client) cost this run nothing —
  # counting it as an unmetered leg would make the cost record read as if a model
  # had been asked and stayed quiet.
  entries = build(proposals, allowlist, contents, limits, metered: raw.nil?)
  yield entries if block_given?

  scored = entries.map { |entry| score(entry, agent_id, run_id, tolerance) }
  Result.new(entries: scored, winner: self.class.rank(scored), budget: @budget, failed: failed)
end