Module: Scryer::FixVerifier

Defined in:
lib/scryer/fix_verifier.rb

Overview

Best-effort verification that an AI-rewritten suggested_fix actually clears the finding it was generated for — "AI-verified remediation" in the sense of "Scryer independently re-checked this," not in the sense of a human review being unnecessary (suggested_fix is still never auto-applied to a real file; see AiFixSuggester's own header comment).

How: AiFixSuggester's prompt (see static_prompt_for) asks the model to end its reply with a fenced "AFTER:" code block containing a drop-in replacement for the single offending source line. This class extracts that block, substitutes it for that one line in an in-memory copy of the file (nothing is ever written to disk), re-parses the result, and re-runs only the one rule that flagged this finding against it. If that rule no longer fires anywhere in the modified file, the fix is marked verified.

Deliberately narrow, same spirit as scryer verify (the CLI command this shares its core logic with): this confirms the ONE finding it targeted is gone, not that the fix is otherwise correct, idiomatic, or free of introducing a different problem — a full rescan (or scryer verify again with a different --rule) answers that.

Returns true (rule no longer fires), false (attempted but the rule still fires, or the rewritten line doesn't even parse), or nil (verification wasn't attempted at all — no AFTER: block in the AI's reply, the original file isn't readable, or this isn't a rule-backed Finding to begin with). nil is deliberately distinct from false: it means "we don't know," not "we checked and it's still broken."

Constant Summary collapse

AFTER_BLOCK = 
/AFTER:\s*```\w*\n(.*?)\n?```/m.freeze

Class Method Summary collapse

Class Method Details

.apply_line_replacement(lines, line_number, replacement) ⇒ Object 



75
76
77
78
79
80
 
# File 'lib/scryer/fix_verifier.rb', line 75

def apply_line_replacement(lines, line_number, replacement)
  replacement_text = replacement.end_with?("\n") ? replacement : "#{replacement}\n"
  modified = lines.dup
  modified[line_number - 1] = replacement_text
  modified.join
end

.extract_after_snippet(suggested_fix) ⇒ Object 



67
68
69
70
71
72
73
 
# File 'lib/scryer/fix_verifier.rb', line 67

def extract_after_snippet(suggested_fix)
  match = AFTER_BLOCK.match(suggested_fix.to_s)
  return nil unless match

  content = match[1].to_s
  content.strip.empty? ? nil : content
end

.verify(finding:, root:) ⇒ Object 



36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
 
# File 'lib/scryer/fix_verifier.rb', line 36

def verify(finding:, root:)
  return nil unless finding.is_a?(Scryer::Finding)
  return nil unless finding.line && finding.rule_id

  after_snippet = extract_after_snippet(finding.suggested_fix)
  return nil unless after_snippet

  abs_path = File.join(root, finding.file.to_s)
  return nil unless File.file?(abs_path)

  lines = File.read(abs_path).lines
  return nil unless finding.line.between?(1, lines.size)

  modified_source = apply_line_replacement(lines, finding.line, after_snippet)

  sexp = begin
    Ripper.sexp(modified_source)
  rescue StandardError
    nil
  end
  return false if sexp.nil? # the rewritten line doesn't even parse — not a usable fix

  rule_class = Scryer::RuleSet.all.find { |r| r.rule_id == finding.rule_id }
  return nil unless rule_class

  remaining = rule_class.new(file: finding.file, source: modified_source, sexp: sexp).scan
  remaining.none? { |f| f.rule_id == finding.rule_id }
rescue StandardError
  nil
end