Class: Vicary::PlaceholderMinter

Inherits:
Object
  • Object
show all
Defined in:
lib/vicary/minter.rb

Overview

Hands out {KIND_n} placeholders, stable per distinct original.

The Ruby port of python/src/vicary/redaction.py's minter.

Why numbering, stated as a measurement rather than a preference: a bare {NAME} standing for every person in a document is not reversible. On 25 injected essays the unnumbered masker produced 37 not-restorable violations and only 36% of essays round-tripped — one token meant Marisol in one paragraph and Terrence Okonkwo in the next, so no map keyed on the token can put either back.

Two properties, and the second is the one that needs care:

  • Injective — distinct originals never share a placeholder, which is what makes restore well-defined.
  • Stable within a document — the same original always gets the same index, so a name written five times masks to one placeholder rather than five. That matters beyond restorability: a scoring model reading {NAME_1} argued … {NAME_1} concluded can still see one person doing two things, where {NAME_1} … {NAME_5} reads as two strangers.

Keyed on the exact original text, because restore must return the exact bytes. Terrence and Terrence's are therefore different keys — correct but unsatisfying, and the reason surname-folding does NOT belong here: folding them together would make the mapping non-injective again.

Indices follow mint order, which is discovery order, not position in the text. One minter serves the whole document precisely so that holds; per-pass minters would restart each counter and emit {NAME_1} twice for two different people, which is the bug numbering exists to remove.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(number: true) ⇒ PlaceholderMinter

Returns a new instance of PlaceholderMinter.



39
40
41
42
43
44
45
46
47
48
 
# File 'lib/vicary/minter.rb', line 39

def initialize(number: true)
  @number = number
  # `[kind, original]` -> index. A tuple key rather than the joined string
  # the other two ports use: Ruby hashes take array keys directly, so there
  # is no separator to pick and no way for a name containing one to collide.
  # Insertion-ordered, which is what makes {#assigned} come back in discovery
  # order.
  @assigned = {}
  @high = Hash.new(0)
end

Instance Attribute Details

#number ⇒ Object (readonly)

Off reproduces the unnumbered output byte for byte, so the two arms stay separately measurable.



37
38
39
 
# File 'lib/vicary/minter.rb', line 37

def number
  @number
end

Instance Method Details

#assigned ⇒ Object

{placeholder => original} — the restore map, for free.

Insertion-ordered, so the map reads in the order the document discovered each span rather than in the order the placeholders sort.



78
79
80
 
# File 'lib/vicary/minter.rb', line 78

def assigned
  @assigned.keys.to_h { |kind, original| [mint(kind, original), original] }
end

#mint(kind, original) ⇒ Object

The placeholder original should be replaced by.



51
52
53
54
55
56
57
58
59
60
61
62
 
# File 'lib/vicary/minter.rb', line 51

def mint(kind, original)
  return "{#{kind}}" unless @number

  key = [kind, original]
  index = @assigned[key]
  if index.nil?
    index = @high[kind] + 1
    @high[kind] = index
    @assigned[key] = index
  end
  "{#{kind}_#{index}}"
end

#substitute(kind, pattern, text) ⇒ Object

Replace every match with a minted placeholder; returns the text and count.



65
66
67
68
69
70
71
72
 
# File 'lib/vicary/minter.rb', line 65

def substitute(kind, pattern, text)
  count = 0
  replaced = text.gsub(pattern) do |match|
    count += 1
    mint(kind, match)
  end
  [replaced, count]
end