Class: Mxrb::Runtime::Native::Expression::Lexer

Inherits:

Object

• Object
• Mxrb::Runtime::Native::Expression::Lexer

Defined in:

lib/mxrb/runtime/native.rb

Overview

Tokenization is cached by Expression so tight microflow loops do not repeatedly scan the same static Mendix source string.

Instance Method Summary

• #initialize(text) ⇒ Lexer constructor

  A new instance of Lexer.

• #tokens ⇒ Object

Constructor Details

#initialize(text) ⇒ Lexer

Returns a new instance of Lexer.

# File 'lib/mxrb/runtime/native.rb', line 439

def initialize(text)
  @scanner = StringScanner.new(text)
end

Instance Method Details

#tokens ⇒ Object

# File 'lib/mxrb/runtime/native.rb', line 443

def tokens
  result = []
  loop do
    @scanner.skip(/\s+/)
    token = next_token
    result << token
    return result if token.first == :eof
  end
end