Class: Insika::Safety::Moderator

Inherits:
Object
  • Object
show all
Defined in:
lib/insika/safety/moderator.rb

Overview

LLM content moderator (RFC-0009 Fase C / D2). The subtle tier the regex can't reach: social engineering (a fabricated prior promise), veiled hostility, tone.

Pure over an injected ask callable (prompt -> raw model text), exactly like the eval Judge — so it is unit-testable with no LLM. The real ask (RubyLLM on the utility_model, temp 0) is built by the Safety::Factory at wiring time.

FAIL-OPEN by construction: the deterministic layer already ran and caught the gross cases, so an unparseable/failed moderator reply must NOT block a legitimate customer — it degrades to allow. Blocking is the high-stakes direction (RFC §6: a false positive turns away a real buyer).

Defined Under Namespace

Classes: Verdict

Constant Summary collapse

CATEGORIES = 
%w[injection abuse sexual self_harm off_topic safe].freeze
ACTIONS = 
%w[allow refuse escalate].freeze

Instance Method Summary collapse

Constructor Details

#initialize(ask:) ⇒ Moderator

ask: ->(prompt) { "" }.



27
28
29
 
# File 'lib/insika/safety/moderator.rb', line 27

def initialize(ask:)
  @ask = ask
end

Instance Method Details

#classify(message, context: nil) ⇒ Object

Classifies a user message. Returns a Verdict; on ANY failure -> allow/safe (fail-open). context is optional free text (e.g. the agent's domain) woven into the prompt.



34
35
36
37
38
39
 
# File 'lib/insika/safety/moderator.rb', line 34

def classify(message, context: nil)
  raw = @ask.call(build_prompt(message.to_s, context)).to_s
  parse(raw)
rescue StandardError
  Verdict.new(category: "safe", action: "allow", reason: "moderator error (fail-open)")
end