Class: Vangrail::Assessor

Inherits:
Object
  • Object
show all
Defined in:
lib/vangrail/assessor.rb

Overview

Combines every measured rail into a posterior, then ranks a set by it.

check_input and friends answer a different question. They run the rails in order, stop at the first block, and report a decision; that is the right shape for a request path and it is what every published defence does. It also throws away most of what was measured. A rail that fired tells you nothing about how much that hit is worth, three rails that nearly fired tell you nothing at all, and a block carries no number an operator can set a policy against.

This runs every rail that has a measured operating point, treats each verdict as evidence, and combines it with the deployment's base rate. What comes back is a probability, the bits each rail contributed, and an action under a stated policy.

Instance Method Summary collapse

Constructor Details

#initialize(engine) ⇒ Assessor

Returns a new instance of Assessor.



25
26
27
 
# File 'lib/vangrail/assessor.rb', line 25

def initialize(engine)
  @engine = engine
end

Instance Method Details

#assess(text, side: :input, prior: nil, policy: Policy::DEFAULT, evidence: nil, escalate: false, confidence: Posterior::DEFAULT_CONFIDENCE, origin: nil, **context) ⇒ Object

The prior is not optional and has no sensible default. Detector papers report their numbers on balanced corpora, where an attack is half the traffic; a documentation desk over an editable wiki might see one poisoned page in ten thousand. Those two worlds disagree about what a hit means by four orders of magnitude, and only the deployment knows which one it is in. Guessing on its behalf would be the whole error this method exists to expose.

judgement = engine.assess(page, side: :context, prior: 1e-4)
judgement.posterior    # => 0.0073
judgement.action       # => :review
judgement.fired        # => [{rail: "paraphrase", bits: 6.2, ...}]

Costs more than a check, because nothing short-circuits: every rail with an entry in the table runs, including the ones a block would have skipped.

Raises:

  • (ArgumentError) 


44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
 
# File 'lib/vangrail/assessor.rb', line 44

def assess(text, side: :input, prior: nil, policy: Policy::DEFAULT, evidence: nil,
           escalate: false, confidence: Posterior::DEFAULT_CONFIDENCE, origin: nil, **context)
  raise ArgumentError, prior_message if prior.nil?

  origin = Origin.coerce(origin || Origin.default_for(side))
  # A rail is worth different evidence on different sides, by a lot: the
  # same paraphrase rail catches a third of in-the-wild jailbreak prompts
  # and none of the published injections in retrieved documents. One table
  # for both would be an average of two unrelated things.
  evidence ||= EvidenceData.for_side(side)

  observed = observe(text, side, context, evidence, escalate ? { prior: prior, policy: policy } : nil)
  observations, direct, certain, skipped = observed
  posterior, contributions = Posterior.combine(prior: prior, observations: observations,
                                               evidence: evidence, direct: direct,
                                               confidence: confidence)
  action = policy.action_for(posterior)
  # A confidence bound is what the corpus can defend. If the point
  # estimate and the bound disagree about the action, the action is not
  # identified: reporting it as a certain decision would spend evidence
  # nobody measured.
  if confidence
    # Explicitly nil, because the bound is what `combine` defaults to now:
    # asking for the point estimate has to say so, or this compares the
    # bound against itself and reports every action as identified.
    point, = Posterior.combine(prior: prior, observations: observations,
                               evidence: evidence, direct: direct, confidence: nil)
    certain &&= policy.action_for(point) == action
  end
  Judgement.new(posterior: posterior, prior: prior, bits: contributions.sum { |c| c[:bits] },
                contributions: contributions, certain: certain, side: side.to_sym,
                skipped: skipped, action: action, origin: origin)
end

#triage(documents, prior:, policy: Policy::DEFAULT, escalate: false, **context) ⇒ Object

Screening, with the documents ranked by how suspicious they are rather than partitioned by whether one rail objected.

screen drops a document the moment a rail blocks it, which is the right shape when a rail is a switch. Given a posterior there is a better answer available: rank the set, drop what the policy says to drop, hand what it says to review to whoever reviews, and keep the rest. A page that trips one pattern at a base rate of one in ten thousand is not a page worth taking away from a reader, and it is worth putting at the bottom of the passage list.

triage = engine.triage(documents, prior: 1e-4)
triage.keep       # documents, least suspicious first
triage.review     # [{document:, judgement:}]
triage.dropped    # [{document:, judgement:}]


93
94
95
96
97
98
99
100
101
 
# File 'lib/vangrail/assessor.rb', line 93

def triage(documents, prior:, policy: Policy::DEFAULT, escalate: false, **context)
  judged = Array(documents).each_with_index.map do |document, index|
    judgement = assess(Cell.text_of(document), side: :context, prior: prior, policy: policy,
                                               escalate: escalate, **context, document: document,
                                               index: index)
    { document: document, judgement: judgement }
  end
  Triage.new(judged: judged.sort_by { |row| -row[:judgement].posterior })
end