Class: Markbridge::Parsers::BBCode::Scanner

Inherits:
Object
  • Object
show all
Defined in:
lib/markbridge/parsers/bbcode/scanner.rb

Overview

High-performance character-by-character BBCode scanner Tokenizes BBCode in O(n) time with minimal allocations and bounded backtracking

Instance Method Summary collapse

Constructor Details

#initialize(input) ⇒ Scanner

Returns a new instance of Scanner.



9
10
11
12
13
 
# File 'lib/markbridge/parsers/bbcode/scanner.rb', line 9

def initialize(input)
  @input = input
  @length = input.length
  @current_pos = 0
end

Instance Method Details

#next_token ⇒ Object 



15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
 
# File 'lib/markbridge/parsers/bbcode/scanner.rb', line 15

def next_token
  return nil if end_of_input?
  start_pos = @current_pos
  bracket_index = @input.index("[", @current_pos)

  if bracket_index.nil?
    text = @input[@current_pos..]
    @current_pos = @length
    return TextToken.new(text:, pos: start_pos)
  end

  if bracket_index > @current_pos
    text = @input[@current_pos...bracket_index]
    @current_pos = bracket_index
    return TextToken.new(text:, pos: start_pos)
  end

  if (tag_token = parse_tag_at_cursor)
    tag_token
  else
    @current_pos += 1
    TextToken.new(text: "[", pos: start_pos)
  end
end