Module: LexerKit::Builder::DSL

Included in:
LexerKit::Builder
Defined in:
lib/lexer_kit/builder.rb

Overview

DSL methods for defining lexers. These methods are available within the ‘LexerKit.build` block.

Instance Method Summary collapse

Instance Method Details

#define_keywords(*names) ⇒ Object

Define multiple keywords at once

Examples:

define_keywords :if, :else, :while
# equivalent to:
# keyword :IF, "if"
# keyword :ELSE, "else"
# keyword :WHILE, "while"

Parameters:

  • names (Array<Symbol>) —

    keyword values (upcased for token names)



52
53
54
55
56
57
58
 
# File 'lib/lexer_kit/builder.rb', line 52

def define_keywords(*names)
  names.each do |name|
    token_name = name.to_s.upcase.to_sym
    value = name.to_s.downcase
    keyword(token_name, value)
  end
end

#delimited(name, delimiter: nil, escape: nil, pop: false, skip: false, &block) ⇒ Object

Define a delimited section (for templates) Scans text until the delimiter is found, then switches to inner mode. The closing delimiter should be handled by a token in the inner mode with ‘pop: true`.

Examples:

delimited :TEXT, delimiter: "{["
mode :tag do
  token :CLOSE, "]}", pop: true  # closing delimiter handled here
end

with escape

delimited :TEXT, delimiter: "{[", escape: "{[{]}" do
  # {[{]} is treated as escaped delimiter and does not close TEXT
end

Parameters:

  • name (Symbol) —

    text token name

  • delimiter (String) (defaults to: nil) —

    delimiter that marks the end of text

  • escape (String, nil) (defaults to: nil) —

    escape sequence that prevents delimiter match



92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
 
# File 'lib/lexer_kit/builder.rb', line 92

def delimited(name, delimiter: nil, escape: nil, pop: false, skip: false, &block)
  unless delimiter
    location = caller_locations(1, 1).first
    message = "delimited requires `delimiter:` parameter"
    raise LexerKit::BuildError.from_location(location, message)
  end
  location = caller_locations(1, 1).first

  token_def = add_delimited_token(
    name: name,
    delimiter: delimiter,
    escape: escape,
    pop: pop,
    skip: skip,
    location: location
  )

  # If block given, define inner mode
  return unless block

  inner_mode = :"#{name}_inner"
  mode(inner_mode, &block)
  token_def.inner_mode = inner_mode
end

#keyword(name, value) ⇒ Object

Define a keyword (matched after identifier, checked by lookup)

Parameters:

  • name (Symbol) —

    keyword token name

  • value (String) —

    keyword string



40
41
42
 
# File 'lib/lexer_kit/builder.rb', line 40

def keyword(name, value)
  @keywords[value.freeze] = name
end

#mode(name) { ... } ⇒ Object

Define a mode

Parameters:

  • name (Symbol) —

    mode name

Yields:

  • mode definition block



63
64
65
66
67
68
69
70
71
72
 
# File 'lib/lexer_kit/builder.rb', line 63

def mode(name, &block)
  location = caller_locations(1, 1).first
  mode_def = ModeDef.new(name, location: location)
  @mode_defs[name] = mode_def

  old_mode = @current_mode
  @current_mode = name
  instance_eval(&block) if block
  @current_mode = old_mode
end

#scan_until(name, open: nil, close: nil, escape: nil, skip: false) ⇒ Object

Define a scan-until section with open/close delimiters.

Parameters:

  • name (Symbol) —

    token name for the scanned content

  • open (String) (defaults to: nil) —

    opening delimiter

  • close (String) (defaults to: nil) —

    closing delimiter

  • escape (String, nil) (defaults to: nil) —

    escape sequence that prevents close match

  • skip (Boolean) (defaults to: false) —

    skip emitting the content token



124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
 
# File 'lib/lexer_kit/builder.rb', line 124

def scan_until(name, open: nil, close: nil, escape: nil, skip: false)
  location = caller_locations(1, 1).first
  unless open
    message = "scan_until requires `open:` parameter"
    raise LexerKit::BuildError.from_location(location, message)
  end
  unless close
    message = "scan_until requires `close:` parameter"
    raise LexerKit::BuildError.from_location(location, message)
  end

  mode_name = next_internal_mode_name(name)
  token :"#{name}_OPEN_#{@internal_mode_counter}", open, skip: true, push: mode_name

  mode(mode_name) do
    delimited(name, delimiter: close, escape: escape, pop: true, skip: skip)
  end
end

#token(name, pattern, skip: false, push: nil, pop: false, meta: nil) ⇒ Object

Define a token

Parameters:

  • name (Symbol) —

    token name

  • pattern (String, Regexp) —

    pattern to match

  • skip (Boolean) (defaults to: false) —

    skip this token (don’t emit)

  • push (Symbol) (defaults to: nil) —

    push mode after match

  • pop (Boolean) (defaults to: false) —

    pop mode after match

  • meta (Hash) (defaults to: nil) —

    optional metadata for this token



22
23
24
25
26
27
28
29
30
31
32
33
34
35
 
# File 'lib/lexer_kit/builder.rb', line 22

def token(name, pattern, skip: false, push: nil, pop: false, meta: nil)
  location = caller_locations(1, 1).first
  token_def = TokenDef.new(
    name: name,
    pattern: pattern,
    skip: skip,
    push: push,
    pop: pop,
    meta: meta,
    location: location
  )
  current_mode_def.add_token(token_def)
  @token_defs << token_def
end

#version(v = nil) ⇒ Integer

Get or set the version

Parameters:

  • v (Integer, nil) (defaults to: nil) —

    version to set, or nil to get current

Returns:

  • (Integer) —

    current version



146
147
148
149
150
151
152
 
# File 'lib/lexer_kit/builder.rb', line 146

def version(v = nil)
  if v.nil?
    @version
  else
    @version = v
  end
end