Class: Kumi::Parser::Token

Inherits:
Struct
  • Object
show all
Defined in:
lib/kumi/parser/token.rb

Overview

One lexical token. Unlike the old SmartTokenizer’s metadata-bag tokens, this carries only a typed ‘kind`, its literal `value`, and the byte `offset` where it starts — enough to build a location and error frame on demand via Source.

Everything the parser needs to know about a token’s role (precedence, associativity, whether it’s a type keyword, …) lives in the grammar tables in Grammar, keyed by ‘kind` — not duplicated onto every token.

Instance Attribute Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#kind ⇒ Object

Returns the value of attribute kind

Returns:

  • (Object) —

    the current value of kind



13
14
15
 
# File 'lib/kumi/parser/token.rb', line 13

def kind
  @kind
end

#offset ⇒ Object

Returns the value of attribute offset

Returns:

  • (Object) —

    the current value of offset



13
14
15
 
# File 'lib/kumi/parser/token.rb', line 13

def offset
  @offset
end

#value ⇒ Object

Returns the value of attribute value

Returns:

  • (Object) —

    the current value of value



13
14
15
 
# File 'lib/kumi/parser/token.rb', line 13

def value
  @value
end

Instance Method Details

#inspect ⇒ Object 



18
19
20
 
# File 'lib/kumi/parser/token.rb', line 18

def inspect
  to_s
end

#to_s ⇒ Object 



14
15
16
 
# File 'lib/kumi/parser/token.rb', line 14

def to_s
  "#{kind}(#{value.inspect})"
end