Class: Synthra::Parser::Token

Inherits:
Object
  • Object
show all
Defined in:
lib/synthra/parser/tokens.rb

Overview

Represents a single token from the lexer

A Token captures the type, value, and source location of a syntax element in the DSL. Tokens are produced by the Lexer and consumed by the Parser.

Examples:

Create a token

token = Token.new(:type, "uuid", line: 5, column: 10)
token.type    # => :type
token.value   # => "uuid"
token.line    # => 5
token.column  # => 10

Check token type

token.type?(:type)  # => true
token.type?(:number)  # => false

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(type, value, line:, column:) ⇒ Token

Create a new Token

Examples:

Token.new(:number, 42, line: 3, column: 15)

Parameters:

  • type (Symbol) —

    the token type (should be in Tokens::ALL)

  • value (Object) —

    the token value

  • line (Integer) —

    line number where token appears (1-based)

  • column (Integer) —

    column where token starts (1-based)



384
385
386
387
388
389
 
# File 'lib/synthra/parser/tokens.rb', line 384

def initialize(type, value, line:, column:)
  @type = type
  @value = value
  @line = line
  @column = column
end

Instance Attribute Details

#column ⇒ Object (readonly)

Returns the value of attribute column.



371
372
373
 
# File 'lib/synthra/parser/tokens.rb', line 371

def column
  @column
end

#line ⇒ Object (readonly)

Returns the value of attribute line.



365
366
367
 
# File 'lib/synthra/parser/tokens.rb', line 365

def line
  @line
end

#type ⇒ Object (readonly)

Returns the value of attribute type.



353
354
355
 
# File 'lib/synthra/parser/tokens.rb', line 353

def type
  @type
end

#value ⇒ Object (readonly)

Returns the value of attribute value.



359
360
361
 
# File 'lib/synthra/parser/tokens.rb', line 359

def value
  @value
end

Instance Method Details

#==(other) ⇒ Boolean Also known as: eql?

Check equality with another token

Two tokens are equal if they have the same type and value. Line and column are not considered for equality.

Examples:

token1 = Token.new(:type, "uuid", line: 1, column: 1)
token2 = Token.new(:type, "uuid", line: 2, column: 5)
token1 == token2  # => true (same type and value)

Parameters:

  • other (Token) —

    token to compare

Returns:

  • (Boolean) —

    true if type and value match



426
427
428
429
430
 
# File 'lib/synthra/parser/tokens.rb', line 426

def ==(other)
  return false unless other.is_a?(Token)

  type == other.type && value == other.value
end

#hash ⇒ Integer

Hash code for use in Hash keys and Sets

Based on type and value only (not line/column).

Returns:

  • (Integer) —

    hash code



442
443
444
 
# File 'lib/synthra/parser/tokens.rb', line 442

def hash
  [type, value].hash
end

#inspect ⇒ String

Inspect output (same as to_s)

Returns:

  • (String) —

    inspect representation



408
409
410
 
# File 'lib/synthra/parser/tokens.rb', line 408

def inspect
  to_s
end

#to_s ⇒ String

String representation for debugging

Examples:

token.to_s  # => "Token(type, \"uuid\", line: 5, col: 10)"

Returns:

  • (String) —

    human-readable token description



399
400
401
 
# File 'lib/synthra/parser/tokens.rb', line 399

def to_s
  "Token(#{type}, #{value.inspect}, line: #{line}, col: #{column})"
end

#type?(expected_type) ⇒ Boolean

Check if token is of a specific type

Examples:

token.type?(:type)    # => true
token.type?(:number)  # => false

Parameters:

  • expected_type (Symbol) —

    type to check against

Returns:

  • (Boolean) —

    true if token type matches



456
457
458
 
# File 'lib/synthra/parser/tokens.rb', line 456

def type?(expected_type)
  type == expected_type
end