Class: Natsuzora::Lexer::TokenProcessor

Inherits:
Object
  • Object
show all
Defined in:
lib/natsuzora/lexer/token_processor.rb

Overview

Processes tokens to handle whitespace control and comments.

Responsibilities:

  • Consume DASH tokens and apply trim rules

  • Consume comment tags entirely

  • Detect unclosed comments

Instance Method Summary collapse

Constructor Details

#initialize(tokens) ⇒ TokenProcessor

Returns a new instance of TokenProcessor.



12
13
14
15
16
 
# File 'lib/natsuzora/lexer/token_processor.rb', line 12

def initialize(tokens)
  @tokens = tokens
  @result = []
  @strip_next_text = false
end

Instance Method Details

#process ⇒ Object 



18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
 
# File 'lib/natsuzora/lexer/token_processor.rb', line 18

def process
  idx = 0

  while idx < @tokens.length
    token = @tokens[idx]

    if token.type == :TEXT
      append_text(token)
      idx += 1
    else
      idx = process_tag(idx)
    end
  end

  @result
end