Class: PackratParser

Inherits:
Object
  • Object
show all
Defined in:
lib/packrat_parser.rb,
lib/packrat_parser/base.rb,
lib/packrat_parser/parser.rb,
lib/packrat_parser/result.rb,
lib/packrat_parser/version.rb

Overview

A small packrat / PEG parser-combinator library whose grammar rules can be written with the for ... then comprehension from the Ruby fork.

Defined Under Namespace

Classes: Failure, ParseError, Parser, Rule, Success

Constant Summary collapse

VERSION = 
"0.2.0"

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.method_added(name) ⇒ Object

Rewrite every method defined on a subclass into a rule that returns a lazy Rule. Guards against rewriting the base class's own infrastructure and against re-entering while we install the replacement (define_method itself fires method_added).

Private methods are left alone, so they can be used as ordinary helpers rather than grammar rules. Note this only sees the visibility in effect when the method is defined, so it recognises the private section form but not private def foo ... end (which is still public at this point and only made private afterwards).



62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/packrat_parser/base.rb', line 62

def self.method_added(name)
  return if self == PackratParser
  return if name == :initialize
  return if @__defining_rule
  return if private_method_defined?(name)

  @__defining_rule = true
  @start_symbol ||= name
  begin
    body = instance_method(name)
    define_method(name) do
      Rule.new(self, name, body)
    end
  ensure
    @__defining_rule = false
  end
end

.parse(input) ⇒ Object

Convenience: parse input with a fresh instance.



48
49
50
 
# File 'lib/packrat_parser/base.rb', line 48

def self.parse(input)
  new.parse(input)
end

.skip_whitespace(pattern = /\s+/) ⇒ Object

Enable implicit whitespace skipping (Scala's RegexParsers mode). When set, every term skips leading whitespace matching pattern before attempting its match, and parse also consumes trailing whitespace before requiring full input consumption. Off by default (terminals match exactly).

class CalcParser < PackratParser
skip_whitespace            # default /\s+/
# skip_whitespace(/[ \t]+/)  # or a custom pattern
end


36
37
38
 
# File 'lib/packrat_parser/base.rb', line 36

def self.skip_whitespace(pattern = /\s+/)
  @__whitespace = pattern
end

.start_symbol(name = nil) ⇒ Object

Set (or read) the rule the parser starts from. If omitted, the first defined method is used as the start symbol.



19
20
21
22
23
24
25
 
# File 'lib/packrat_parser/base.rb', line 19

def self.start_symbol(name = nil)
  if name
    @start_symbol = name
  else
    @start_symbol
  end
end

.whitespace ⇒ Object

The configured whitespace pattern, or nil when skipping is disabled. Inherited by subclasses so a base parser can turn the mode on once.



42
43
44
45
 
# File 'lib/packrat_parser/base.rb', line 42

def self.whitespace
  return @__whitespace if defined?(@__whitespace)
  superclass.respond_to?(:whitespace) ? superclass.whitespace : nil
end

Instance Method Details

#__built ⇒ Object

Cache of built combinators, keyed by rule name. The combinator graph for a rule is stable, so it is built once and reused (loop variables are block-local, so reusing a closure across recursive activations is safe).



88
89
90
 
# File 'lib/packrat_parser/base.rb', line 88

def __built
  @__built ||= {}
end

#__memo ⇒ Object

Per-input packrat memo table: a two-level hash, rule_name => (pos => result).



81
82
83
 
# File 'lib/packrat_parser/base.rb', line 81

def __memo
  @__memo ||= {}
end

#__skip_ws(ws, input, pos) ⇒ Object

Advance the byte offset pos past whitespace matched by the anchored regexp ws (nil when skipping is disabled). Returns the new byte offset.



141
142
143
144
 
# File 'lib/packrat_parser/base.rb', line 141

def __skip_ws(ws, input, pos)
  return pos unless ws
  input.byteindex(ws, pos) ? pos + Regexp.last_match[0].bytesize : pos
end

#parse(input, start = nil) ⇒ Object

Parse input, starting from rule start (defaults to the configured start symbol). Returns the parsed value on success; raises ParseError on failure or on leftover input. Pass start to parse from any rule, e.g. parser.parse("123", :number) or, equivalently, parser.number.parse("123").

Positions are byte offsets throughout, including the pos reported on a ParseError (see term for why matching is byte-oriented).

Raises:



159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
 
# File 'lib/packrat_parser/base.rb', line 159

def parse(input, start = nil)
  @__memo = {}
  name = start || self.class.start_symbol
  raise ParseError.new("no start symbol defined", 0) unless name

  result = send(name).call(input, 0)
  unless result.success?
    raise ParseError.new(result.message, result.pos)
  end
  # The last terminal skips only *leading* whitespace, so trailing whitespace
  # after the final token is left for parse to consume before requiring that
  # all input was used.
  ws = self.class.whitespace
  end_pos = __skip_ws(ws && /\G(?:#{ws})/, input, result.pos)
  if end_pos < input.bytesize
    raise ParseError.new("unexpected trailing input", end_pos)
  end
  result.value
end

#pure(value) ⇒ Object

A parser that succeeds with value without consuming any input (monadic unit / Scala's success).



148
149
150
 
# File 'lib/packrat_parser/base.rb', line 148

def pure(value)
  Parser.new { |_input, pos| Success.new(value, pos) }
end

#term(pattern) ⇒ Object

A terminal parser. A String matches that exact literal at the current position; a Regexp is matched anchored at the current position. The matched substring is the parser's value.

Positions are byte offsets, not character offsets: indexing a UTF-8 string by character is O(n), so matching by byte (+byteslice+ for literals, byteindex with a \G anchor for regexps) keeps each step O(match length) regardless of how far into the input we are.

When the class enables skip_whitespace, leading whitespace is consumed before the match is attempted, mirroring Scala's RegexParsers.



103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
 
# File 'lib/packrat_parser/base.rb', line 103

def term(pattern)
  ws = self.class.whitespace
  ws = /\G(?:#{ws})/ if ws
  case pattern
  when String
    bytes = pattern.bytesize
    # The failure message is constant for this terminal, and failures are
    # common (ordered choice discards them) while the message is only read if
    # the whole parse fails. Build it once and share the frozen string rather
    # than interpolating on every failed match.
    msg = "expected #{pattern.inspect}".freeze
    Parser.new do |input, pos|
      pos = __skip_ws(ws, input, pos)
      if input.byteslice(pos, bytes) == pattern
        Success.new(pattern, pos + bytes)
      else
        Failure.new(pos, msg)
      end
    end
  when Regexp
    anchored = /\G(?:#{pattern})/
    msg = "expected #{pattern.inspect}".freeze
    Parser.new do |input, pos|
      pos = __skip_ws(ws, input, pos)
      if input.byteindex(anchored, pos)
        s = Regexp.last_match[0]
        Success.new(s, pos + s.bytesize)
      else
        Failure.new(pos, msg)
      end
    end
  else
    raise ArgumentError, "term expects a String or Regexp, got #{pattern.class}"
  end
end