Class: Vangrail::Rails::Linear

Inherits:
Vangrail::Rail show all
Defined in:
lib/vangrail/rails/linear.rb

Overview

The fitted classifier, reading a model somebody trained on their own traffic.

On the corpus this repository can measure, it is the best detector here by a wide margin and within two points of a published transformer:

hand-written rails                      39.5%
naive Bayes over n-grams                67.0%
this rail                               73.7%
deberta-v3-base-prompt-injection-v2     75.3%

all at the same false-alarm rate, cross-validated where fitted. Below one false alarm in a hundred it beats the transformer, 27.0% against 21.1%, which is the operating point a desk actually wants and the one a rule stack cannot be asked about at all.

It ships without a model, deliberately. Weights fitted on somebody else's traffic are the thing this repository spent a long time measuring the cost of, and the numbers above are for a model fitted on the corpus it was scored against. Fit your own:

ruby script/train_linear.rb --emit model.json

With no model configured the rail reports itself unchecked rather than passing, because a detector that is not there must not look like a detector that found nothing.

Constant Summary

Constants inherited from Vangrail::Rail

Vangrail::Rail::DEFAULT_SIDES, Vangrail::Rail::SIDES

Instance Attribute Summary collapse

Attributes inherited from Vangrail::Rail

#name, #sides

Instance Method Summary collapse

Methods inherited from Vangrail::Rail

#applies_to?, #call, #language_agnostic?, #offline?, #placeholder?, #posture?, #quantifies?, #to_s, usable

Constructor Details

#initialize(model: nil, path: nil, threshold: nil, name: 'linear', sides: %i[input context])) ⇒ Linear

Returns a new instance of Linear.



35
36
37
38
39
40
 
# File 'lib/vangrail/rails/linear.rb', line 35

def initialize(model: nil, path: nil, threshold: nil, name: 'linear', sides: %i[input context])
  super(name: name, sides: sides)
  @path = path
  @model = model || load_model(path)
  @threshold = threshold || @model&.threshold || 0.0
end

Instance Attribute Details

#model ⇒ Object (readonly)

Returns the value of attribute model.



42
43
44
 
# File 'lib/vangrail/rails/linear.rb', line 42

def model
  @model
end

#threshold ⇒ Object (readonly)

Returns the value of attribute threshold.



42
43
44
 
# File 'lib/vangrail/rails/linear.rb', line 42

def threshold
  @threshold
end

Instance Method Details

#cache_key(text, _context) ⇒ Object 



44
45
46
47
48
 
# File 'lib/vangrail/rails/linear.rb', line 44

def cache_key(text, _context)
  return nil unless model

  "#{threshold}\n#{text}"
end

#decide(text, _context) ⇒ Object 



50
51
52
53
54
55
56
57
58
59
 
# File 'lib/vangrail/rails/linear.rb', line 50

def decide(text, _context)
  return unchecked(missing_reason) unless model

  value = model.score(text)
  return pass if value <= threshold

  block(categories: ['linear'],
        reason: format('scores %<value>+.2f against a threshold of %<threshold>+.2f', value: value,
                                                                                     threshold: threshold))
end