Module: Mistri::Edit

Defined in:
lib/mistri/edit.rb

Overview

Pure fuzzy text replacement: no files, no I/O. This is the string core that a workspace-backed edit tool calls, so it works the same against a database row as against a file on disk.

Each edit's old text must match one region and only one, so an edit can never silently change the wrong place. Matching relaxes in two steps: an exact substring first, then a whitespace-tolerant line match that forgives the indentation and trailing-space drift models introduce when they reproduce code they read. Unmatched regions keep their exact bytes, including the file's original line endings.

Defined Under Namespace

Classes: Match, Result

Class Method Summary collapse

Class Method Details

.adapt_newlines(content, text) ⇒ Object

Match the document's dominant newline style so a replacement authored with bare LF does not mix endings into a CRLF document.



83
84
85
86
87
88
89
 
# File 'lib/mistri/edit.rb', line 83

def adapt_newlines(content, text)
  crlf = content.scan("\r\n").length
  bare = content.scan(/(?<!\r)\n/).length
  return text.gsub(/\r?\n/, "\r\n") if crlf > bare

  crlf.positive? || bare.positive? ? text.gsub("\r\n", "\n") : text
end

.ambiguous_message(edit, line_numbers) ⇒ Object 



115
116
117
118
119
120
121
 
# File 'lib/mistri/edit.rb', line 115

def ambiguous_message(edit, line_numbers)
  shown = line_numbers.first(4).join(", ")
  shown += ", ..." if line_numbers.length > 4
  "edits[#{edit[:index]}] old text matched #{line_numbers.length} places " \
    "(lines #{shown}). Add surrounding lines until it is unique, or set " \
    "replace_all: true to change all #{line_numbers.length}."
end

.apply(content, edits) ⇒ Object

Apply edits (each new:, string or symbol keys) to content and return the new content. Raises EditError when an edit matches nothing, matches more than once, overlaps another, or changes nothing.

Raises:



42
43
44
45
46
47
48
49
50
51
52
53
 
# File 'lib/mistri/edit.rb', line 42

def apply(content, edits)
  normalized = edits.each_with_index.map { |edit, i| normalize(edit, i) }
  matches = normalized.map { |edit| locate(content, edit) }.sort_by(&:start)
  reject_overlaps(matches)

  result = matches.reverse.reduce(content) do |text, match|
    text[0...match.start] + match.replacement + text[match.finish..]
  end
  raise EditError, "the edits changed nothing" if result == content

  result
end

.best_window(lines, wanted) ⇒ Object

Score windows by per-line bigram similarity, so a one-character typo in a one-line old_string still finds its region. Only a window at least half-similar overall is worth reporting.



183
184
185
186
187
188
189
190
191
192
193
194
 
# File 'lib/mistri/edit.rb', line 183

def best_window(lines, wanted)
  best = nil
  best_score = wanted.length / 2.0
  (0..(lines.length - wanted.length)).each do |i|
    score = (0...wanted.length).sum { |j| similarity(lines[i + j][:stripped], wanted[j]) }
    if score > best_score
      best_score = score
      best = i
    end
  end
  best
end

.bigrams(text) ⇒ Object 



207
 
# File 'lib/mistri/edit.rb', line 207

def bigrams(text) = (0...(text.length - 1)).map { |i| text[i, 2] }.uniq

.exact_match(content, edit) ⇒ Object 



91
92
93
94
95
96
97
98
99
100
101
 
# File 'lib/mistri/edit.rb', line 91

def exact_match(content, edit)
  offsets = occurrence_offsets(content, edit[:old])
  return nil if offsets.empty?

  if offsets.length > 1
    lines = offsets.map { |offset| line_number_at(content, offset) }
    raise EditError, ambiguous_message(edit, lines)
  end
  first = offsets.first
  Match.new(first, first + edit[:old].length, edit[:new], edit[:index])
end

.fuzzy_match(content, edit) ⇒ Object

Match the old text's lines against a window of content lines, comparing each line stripped of leading and trailing whitespace. The matched region is the exact original bytes those content lines span.

Raises:



126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
 
# File 'lib/mistri/edit.rb', line 126

def fuzzy_match(content, edit)
  lines = line_spans(content)
  wanted = edit[:old].lines.map(&:strip)
  wanted.pop if wanted.last == "" # a trailing newline in old text is not a line to match
  return nil if wanted.empty?

  windows = matching_windows(lines, wanted)
  return nil if windows.empty?

  raise EditError, ambiguous_message(edit, windows.map { |w| w + 1 }) if windows.length > 1

  first = windows.first
  Match.new(lines[first][:start], lines[first + wanted.length - 1][:finish],
            edit[:new], edit[:index])
end

.line_number_at(content, offset) ⇒ Object 



113
 
# File 'lib/mistri/edit.rb', line 113

def line_number_at(content, offset) = content[0...offset].count("\n") + 1

.line_spans(content) ⇒ Object

Each line with its character span in the original and its stripped form. A leading BOM is invisible to matching, and the first span starts after it, so a replacement at the top of the document never swallows it.



218
219
220
221
222
223
224
225
226
227
 
# File 'lib/mistri/edit.rb', line 218

def line_spans(content)
  offset = 0
  content.lines.map do |line|
    bom = offset.zero? && line.start_with?("\uFEFF") ? 1 : 0
    span = { start: offset + bom, finish: offset + line.length,
             stripped: line.delete_prefix("\uFEFF").strip }
    offset += line.length
    span
  end
end

.locate(content, edit) ⇒ Object

Exact match first; on a miss, a whitespace-tolerant line match. Either level must resolve to exactly one region. A total miss reports the closest region and its precise difference, so the model's retry can be one-shot.



67
68
69
70
 
# File 'lib/mistri/edit.rb', line 67

def locate(content, edit)
  exact_match(content, edit) || fuzzy_match(content, edit) ||
    raise(EditError, not_found_message(content, edit))
end

.matching_windows(lines, wanted) ⇒ Object 



209
210
211
212
213
 
# File 'lib/mistri/edit.rb', line 209

def matching_windows(lines, wanted)
  (0..(lines.length - wanted.length)).select do |i|
    wanted.each_with_index.all? { |line, j| lines[i + j][:stripped] == line }
  end
end

.nearest_region(content, old_text) ⇒ Object

The window with the most stripped-equal lines, plus its first differing line pair.



160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
 
# File 'lib/mistri/edit.rb', line 160

def nearest_region(content, old_text)
  lines = line_spans(content)
  wanted_raw = old_text.lines.map(&:chomp)
  wanted = wanted_raw.map(&:strip)
  wanted.pop && wanted_raw.pop if wanted.last == ""
  return nil if wanted.empty? || lines.length < wanted.length

  best = best_window(lines, wanted)
  return nil unless best

  diff_at = (0...wanted.length).find { |j| lines[best + j][:stripped] != wanted[j] }
  return nil unless diff_at

  yours = wanted_raw[diff_at]
  theirs = content.lines[best + diff_at].to_s.chomp
  hint = yours.strip == theirs.strip ? " (differs only in whitespace)" : ""
  { from: best + 1, to: best + wanted.length, line: best + diff_at + 1,
    yours: yours, theirs: theirs, hint: hint }
end

.normalize(edit, index) ⇒ Object

Raises:



55
56
57
58
59
60
61
 
# File 'lib/mistri/edit.rb', line 55

def normalize(edit, index)
  edit = edit.transform_keys(&:to_sym)
  old = edit[:old].to_s
  raise EditError, "edits[#{index}] has empty old text" if old.empty?

  { old: old, new: edit[:new].to_s, index: index }
end

.not_found_message(content, edit) ⇒ Object

When nothing matched, show the model the closest region and exactly how it differs, so the retry is one shot instead of a guessing loop.



144
145
146
147
148
149
150
151
152
153
154
155
156
 
# File 'lib/mistri/edit.rb', line 144

def not_found_message(content, edit)
  base = "edits[#{edit[:index]}] old text was not found"
  near = nearest_region(content, edit[:old])
  unless near
    return "#{base}. Copy old_string verbatim from read_file output, " \
           "without line-number prefixes."
  end

  "#{base}. Closest region is lines #{near[:from]}-#{near[:to]}; it differs at " \
    "line #{near[:line]}: your text #{near[:yours].inspect} vs the document's " \
    "#{near[:theirs].inspect}#{near[:hint]}. Copy old_string verbatim from " \
    "read_file output, then resend."
end

.occurrence_offsets(content, needle) ⇒ Object 



103
104
105
106
107
108
109
110
111
 
# File 'lib/mistri/edit.rb', line 103

def occurrence_offsets(content, needle)
  offsets = []
  offset = content.index(needle)
  while offset
    offsets << offset
    offset = content.index(needle, offset + 1)
  end
  offsets
end

.reject_overlaps(matches) ⇒ Object 



229
230
231
232
233
234
235
236
 
# File 'lib/mistri/edit.rb', line 229

def reject_overlaps(matches)
  matches.each_cons(2) do |a, b|
    next if a.finish <= b.start

    raise EditError, "edits[#{a.edit_index}] and edits[#{b.edit_index}] overlap; " \
                     "merge them or target separate regions"
  end
end

.replace(content, old_string, new_string, replace_all: false) ⇒ Object

The model-facing single edit: replace old_string once (unique match required) or everywhere with replace_all. Returns a Result carrying the new content and how many places changed. The replacement adapts to the document's newline style, so an LF-authored new_string dropped into a CRLF document does not mix endings.

Raises:



25
26
27
28
29
30
31
32
33
34
35
36
37
 
# File 'lib/mistri/edit.rb', line 25

def replace(content, old_string, new_string, replace_all: false)
  old = old_string.to_s
  raise EditError, "old_string is empty" if old.empty?

  new = adapt_newlines(content, new_string.to_s)
  return replace_every(content, old, new) if replace_all

  match = locate(content, { old: old, new: new, index: 0 })
  changed = content[0...match.start] + match.replacement + content[match.finish..]
  raise EditError, "the edit changed nothing" if changed == content

  Result.new(content: changed, count: 1)
end

.replace_every(content, old, new) ⇒ Object

Raises:



72
73
74
75
76
77
78
79
 
# File 'lib/mistri/edit.rb', line 72

def replace_every(content, old, new)
  count = content.enum_for(:scan, old).count
  raise EditError, not_found_message(content, { old: old, index: 0 }) if count.zero?

  # Block form keeps both sides literal; a bare string replacement would
  # interpret backslash sequences.
  Result.new(content: content.gsub(old) { new }, count: count)
end

.similarity(left, right) ⇒ Object 



196
197
198
199
200
201
202
203
204
205
 
# File 'lib/mistri/edit.rb', line 196

def similarity(left, right)
  return 1.0 if left == right
  return 0.0 if left.empty? || right.empty?

  pairs_left = bigrams(left)
  pairs_right = bigrams(right)
  return 0.0 if pairs_left.empty? || pairs_right.empty?

  (2.0 * (pairs_left & pairs_right).length) / (pairs_left.length + pairs_right.length)
end