Class: Synthra::Parser::Lexer

Inherits:
Object
  • Object
show all
Includes:
Tokens
Defined in:
lib/synthra/parser/lexer.rb

Overview

Lexer for the Synthra language

Converts DSL source text into a stream of tokens that the parser can process. The lexer is indentation-aware and tracks nesting depth to emit INDENT and DEDENT tokens.

Examples:

Tokenize DSL source

lexer = Lexer.new(source)
tokens = lexer.tokenize

Handle syntax errors

begin
  tokens = Lexer.new(source).tokenize
rescue Synthra::SyntaxError => e
  puts "Error at line #{e.line}, column #{e.column}: #{e.message}"
end

Constant Summary collapse

KEYWORDS =

Reserved keywords and their token types

These words have special meaning in the DSL and cannot be used as identifiers.

Returns:

  • (Hash<String, Symbol>) —

    keyword => token type mapping

{
  "if" => IF,           # Conditional field syntax
  "true" => TRUE_KW,    # Boolean true value
  "false" => FALSE_KW,  # Boolean false value
  "Ref" => REF,         # Cross-schema reference
  "ref" => REF          # Cross-schema reference (lowercase alias)
}.freeze
SIMULYRA_KEYWORDS =

Simulyra DSL keywords for layered architecture

These keywords define the three layers of Simulyra DSL:

  • schema: Data structure definitions
  • api: HTTP endpoint definitions
  • scenario: Response scenario definitions

Returns:

  • (Hash<String, Symbol>) —

    keyword => token type mapping

{
  "schema" => SCHEMA_KEYWORD,
  "api" => API_KEYWORD,
  "scenario" => SCENARIO_KEYWORD
}.freeze
HTTP_METHODS =

HTTP methods recognized by the lexer

These are the standard HTTP methods that can be used in API definitions.

Returns:

  • (Array<String>) —

    list of valid HTTP method names

%w[GET POST PUT PATCH DELETE OPTIONS HEAD].freeze
KNOWN_TYPES =

Known built-in type names

These are recognized as TYPE tokens when not inside parentheses. Custom types starting with uppercase are also recognized.

Returns:

  • (Array<String>) —

    list of built-in type names

%w[
  uuid ulid id_sequence
  name email text
  number boolean enum
  date past_date future_date now timestamp
  array object custom
  phone url ip ipv6
  country_code currency postal_code money
].freeze

Instance Method Summary collapse

Constructor Details

#initialize(source) ⇒ Lexer

Create a new Lexer

Examples:

lexer = Lexer.new("User:\n  name: text")

Parameters:

  • source (String) —

    the DSL source code to tokenize



135
136
137
138
139
140
141
142
143
144
145
 
# File 'lib/synthra/parser/lexer.rb', line 135

def initialize(source)
  @source = source                    # Full source text
  @lines = source.lines               # Source split into lines (for errors)
  @tokens = []                        # Accumulated tokens
  @line = 1                           # Current line number (1-based)
  @column = 1                         # Current column number (1-based)
  @pos = 0                            # Current position in source
  @indent_stack = [0]                 # Stack of indentation levels
  @at_line_start = true               # Are we at the start of a line?
  @paren_depth = 0                    # Parenthesis nesting depth
end

Instance Method Details

#advance ⇒ void (private)

This method returns an undefined value.

Advance to the next character

Updates line/column tracking when crossing newlines.



209
210
211
212
213
214
215
216
217
218
 
# File 'lib/synthra/parser/lexer.rb', line 209

def advance
  if current_char == "\n"
    @line += 1
    @column = 1
    @at_line_start = true
  else
    @column += 1
  end
  @pos += 1
end

#count_indentation ⇒ Integer (private)

Count indentation at current position

Counts spaces and tabs (tabs = 2 spaces) at current position. Advances position past the indentation. Rejects mixed tabs and spaces to prevent parsing confusion.

Returns:

  • (Integer) —

    indentation level in spaces

Raises:

  • (SyntaxError) —

    if mixed tabs and spaces are detected



349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
 
# File 'lib/synthra/parser/lexer.rb', line 349

def count_indentation
  indent = 0
  start_pos = @pos
  # :nocov: count_indentation runs only inside the `while @pos < @source.length` tokenize
  # loop, so @pos is always in bounds here — the nil (else) branch is unreachable.
  start_char = @pos < @source.length ? @source[@pos] : nil
  # :nocov:

  # Determine indentation type (spaces or tabs)
  if start_char == " "

    # Only spaces allowed - count spaces
    while @pos < @source.length && @source[@pos] == " "
      indent += 1
      @pos += 1
      @column += 1
    end

    # Reject if tabs found after spaces
    if @pos < @source.length && @source[@pos] == "\t"
      error("Mixed tabs and spaces in indentation (use either tabs or spaces, not both)")
    end

  elsif start_char == "\t"

    # Only tabs allowed - count tabs (treat as 2 spaces each)
    while @pos < @source.length && @source[@pos] == "\t"
      indent += 2
      @pos += 1
      @column += 1
    end

    # Reject if spaces found after tabs
    if @pos < @source.length && @source[@pos] == " "
      error("Mixed tabs and spaces in indentation (use either tabs or spaces, not both)")
    end
  end

  indent
end

#current_char ⇒ String? (private)

Get the current character

Returns:

  • (String, nil) —

    current character or nil if at end



188
189
190
 
# File 'lib/synthra/parser/lexer.rb', line 188

def current_char
  @source[@pos]
end

#current_line_content ⇒ String (private)

Get the content of the current source line

Used for error messages to show context.

Returns:

  • (String) —

    current line content (without trailing newline)



248
249
250
251
252
253
254
 
# File 'lib/synthra/parser/lexer.rb', line 248

def current_line_content
  # :nocov: errors only fire while @pos is within the source (the tokenize loop guards
  # `@pos < @source.length`), so @line always indexes a real line — the `&.` nil-receiver
  # else-arm (line out of range) is unreachable.
  @lines[@line - 1]&.chomp || ""
  # :nocov:
end

#emit(type, value) ⇒ void (private)

This method returns an undefined value.

Emit a token with current position

Parameters:

  • type (Symbol) —

    token type (from Tokens module)

  • value (Object) —

    token value



232
233
234
 
# File 'lib/synthra/parser/lexer.rb', line 232

def emit(type, value)
  @tokens << Token.new(type, value, line: @line, column: @column)
end

#emit_remaining_dedents ⇒ void (private)

This method returns an undefined value.

Emit DEDENT tokens for remaining indentation at end of file



423
424
425
426
427
428
 
# File 'lib/synthra/parser/lexer.rb', line 423

def emit_remaining_dedents
  while @indent_stack.length > 1
    @indent_stack.pop
    emit(DEDENT, @indent_stack.last)
  end
end

#error(message) ⇒ Object (private)

Raise a syntax error at current position

Parameters:

  • message (String) —

    error description

Raises:



262
263
264
265
266
267
268
269
 
# File 'lib/synthra/parser/lexer.rb', line 262

def error(message)
  raise SyntaxError.new(
    message,
    line: @line,
    column: @column,
    source_line: current_line_content
  )
end

#escape_char(char) ⇒ String (private)

Process an escape sequence character

Parameters:

  • char (String) —

    character after backslash

Returns:

  • (String) —

    the escaped character



949
950
951
952
953
954
955
956
 
# File 'lib/synthra/parser/lexer.rb', line 949

def escape_char(char)
  case char
  when "n" then "\n"
  when "t" then "\t"
  when "r" then "\r"
  else char
  end
end

#expect_char(expected) ⇒ void (private)

This method returns an undefined value.

Expect a specific character at current position

Parameters:

  • expected (String) —

    the expected character

Raises:



1145
1146
1147
1148
1149
 
# File 'lib/synthra/parser/lexer.rb', line 1145

def expect_char(expected)
  return if current_char == expected

  error("Expected '#{expected}', got '#{current_char}'")
end

#handle_line_start ⇒ void (private)

This method returns an undefined value.

Handle the start of a line

At line start, we need to:

  1. Skip blank lines and comments
  2. Count indentation and emit INDENT/DEDENT
  3. Check for Simulyra DSL keywords (schema, api, scenario)
  4. Check for legacy schema definitions


302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
 
# File 'lib/synthra/parser/lexer.rb', line 302

def handle_line_start

  # Skip blank lines
  if current_char == "\n"
    advance
    return
  end

  # Skip comment lines
  if current_char == "#"
    skip_comment
    return
  end

  # Count and process indentation
  indent = count_indentation
  @at_line_start = false

  # Check for Simulyra DSL keywords at any indentation level
  if looking_at_simulyra_keyword?
    process_indent(indent)
    tokenize_simulyra_keyword
    return
  end

  # Check if this is a legacy schema definition (no indent, identifier followed by :)
  if indent.zero? && looking_at_schema?
    process_indent(0)
    tokenize_schema_name
    return
  end

  # Process indentation and then tokenize line content
  process_indent(indent)
  tokenize_content
end

#looking_at_schema? ⇒ Boolean (private)

Check if we're looking at a schema definition

A schema definition is an identifier followed by a colon at the start of a line with no indentation.

Returns:

  • (Boolean) —

    true if this looks like a schema definition



443
444
445
446
447
448
449
450
451
 
# File 'lib/synthra/parser/lexer.rb', line 443

def looking_at_schema?

  # Look ahead to see if line ends with : (schema definition)
  pos = @pos
  while pos < @source.length && @source[pos] =~ /[A-Za-z0-9_]/
    pos += 1
  end
  @source[pos] == ":"
end

#looking_at_schema_reference? ⇒ Boolean (private)

Check if we're looking at a schema reference: schema(:Name)

Returns:

  • (Boolean) —

    true if this looks like a schema reference



629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
 
# File 'lib/synthra/parser/lexer.rb', line 629

def looking_at_schema_reference?
  # Check if we're at "schema(" pattern
  return false unless current_char == "s"

  pos = @pos
  word = ""

  # Read potential "schema" word
  while pos < @source.length && @source[pos] =~ /[a-z]/
    word += @source[pos]
    pos += 1
  end

  return false unless word == "schema"

  # Skip whitespace
  while pos < @source.length && @source[pos] =~ /[ \t]/
    pos += 1
  end

  # Check for opening paren
  @source[pos] == "("
end

#looking_at_simulyra_keyword? ⇒ Boolean (private)

Check if we're looking at a Simulyra DSL keyword (schema, api, scenario)

Returns:

  • (Boolean) —

    true if this looks like a Simulyra keyword



478
479
480
481
482
483
484
485
486
487
 
# File 'lib/synthra/parser/lexer.rb', line 478

def looking_at_simulyra_keyword?
  # Look ahead to see the word
  pos = @pos
  word = ""
  while pos < @source.length && @source[pos] =~ /[a-z]/
    word += @source[pos]
    pos += 1
  end
  SIMULYRA_KEYWORDS.key?(word)
end

#peek_char(offset = 1) ⇒ String? (private)

Peek at a character ahead of current position

Parameters:

  • offset (Integer) (defaults to: 1) —

    how far ahead to look (default: 1)

Returns:

  • (String, nil) —

    character at offset or nil



198
199
200
 
# File 'lib/synthra/parser/lexer.rb', line 198

def peek_char(offset = 1)
  @source[@pos + offset]
end

#process_indent(indent) ⇒ void (private)

This method returns an undefined value.

Process indentation changes

Compares current indentation to stack and emits INDENT or DEDENT tokens.

Parameters:

  • indent (Integer) —

    current indentation level



398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
 
# File 'lib/synthra/parser/lexer.rb', line 398

def process_indent(indent)
  current_indent = @indent_stack.last

  if indent > current_indent

    # Increased indentation - push new level
    @indent_stack.push(indent)
    emit(INDENT, indent)
  elsif indent < current_indent

    # Decreased indentation - pop levels until we match
    while @indent_stack.last > indent
      @indent_stack.pop
      emit(DEDENT, @indent_stack.last)
    end
  end

  # Equal indentation - no change needed
end

#read_identifier ⇒ String (private)

Read an identifier from current position

Identifiers consist of letters, digits, and underscores.

Returns:

  • (String) —

    the identifier



1129
1130
1131
1132
1133
1134
1135
1136
 
# File 'lib/synthra/parser/lexer.rb', line 1129

def read_identifier
  value = ""
  while @pos < @source.length && current_char =~ /[A-Za-z0-9_]/
    value += current_char
    advance
  end
  value
end

#read_path ⇒ String (private)

Read a URL path from current position

Paths can include:

  • Forward slashes: /
  • Identifiers: users, orders
  • Path parameters: :id, :user_id
  • Hyphens and underscores: my-api, my_api

Returns:

  • (String) —

    the path



574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
 
# File 'lib/synthra/parser/lexer.rb', line 574

def read_path
  return "" unless current_char == "/"

  path = ""
  while @pos < @source.length
    char = current_char

    # Path parameters start with :
    if char == ":"
      # Look ahead to see if this is a path parameter or the end colon
      next_char = peek_char
      if next_char =~ /[A-Za-z_]/
        # This is a path parameter like :id
        path += char
        advance
      else
        # This is the terminating colon
        break
      end
    elsif char =~ %r{[A-Za-z0-9_\-/]}
      path += char
      advance
    else
      break
    end
  end
  path
end

#skip_comment ⇒ void (private)

This method returns an undefined value.

Skip a comment (from # to end of line)



711
712
713
714
715
716
 
# File 'lib/synthra/parser/lexer.rb', line 711

def skip_comment
  while @pos < @source.length && current_char != "\n"
    advance
  end
  advance if current_char == "\n"
end

#skip_to_end_of_line ⇒ void (private)

This method returns an undefined value.

Skip to end of current line



734
735
736
737
738
739
 
# File 'lib/synthra/parser/lexer.rb', line 734

def skip_to_end_of_line
  while @pos < @source.length && current_char != "\n"
    advance
  end
  advance if current_char == "\n"
end

#skip_whitespace ⇒ void (private)

This method returns an undefined value.

Skip horizontal whitespace (spaces and tabs)



723
724
725
726
727
 
# File 'lib/synthra/parser/lexer.rb', line 723

def skip_whitespace
  while @pos < @source.length && current_char =~ /[ \t]/
    advance
  end
end

#tokenize ⇒ Array<Token>

Tokenize the entire source

Processes the source text and returns an array of Token objects. The token stream will always end with an EOF token.

Examples:

tokens = lexer.tokenize
tokens.last.type  # => :eof

Returns:

  • (Array<Token>) —

    array of tokens representing the source

Raises:

  • (SyntaxError) —

    if the source contains invalid syntax



160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
 
# File 'lib/synthra/parser/lexer.rb', line 160

def tokenize

  # Process source character by character
  while @pos < @source.length
    tokenize_line
  end

  # Emit DEDENT tokens for any remaining indentation
  emit_remaining_dedents

  # Always end with EOF token
  emit(EOF, nil)

  @tokens
end

#tokenize_after_colon ⇒ void (private)

This method returns an undefined value.

Tokenize content after a colon (field type)

After a field name and colon, the next identifier is usually a type.



857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
 
# File 'lib/synthra/parser/lexer.rb', line 857

def tokenize_after_colon
  skip_whitespace
  return if @pos >= @source.length || current_char == "\n"

  # Handle negative numbers (e.g., number(-10..10))
  if current_char == "-"
    tokenize_number
    return
  end

  # Check for schema reference: schema(:Name)
  if looking_at_schema_reference?
    tokenize_schema_reference
    return
  end

  # Read identifier (type name)
  if current_char =~ /[A-Za-z_]/
    name = read_identifier

    # Check for ? suffix (nullable type)
    has_question = current_char == "?"
    if has_question
      advance
    end

    if KEYWORDS.key?(name)
      emit(KEYWORDS[name], name)
      emit(QUESTION, "?") if has_question
    elsif KNOWN_TYPES.include?(name)
      emit(TYPE, has_question ? "#{name}?" : name)
    elsif name[0] =~ /[A-Z]/

      # Starts with uppercase - schema reference type
      emit(TYPE, has_question ? "#{name}?" : name)
    else
      emit(IDENTIFIER, name)
      emit(QUESTION, "?") if has_question
    end
  end
end

#tokenize_api_definition ⇒ void (private)

This method returns an undefined value.

Tokenize an API definition (api METHOD /path:)



538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
 
# File 'lib/synthra/parser/lexer.rb', line 538

def tokenize_api_definition
  emit(API_KEYWORD, "api")
  skip_whitespace

  # Read HTTP method
  method = read_identifier.upcase
  unless HTTP_METHODS.include?(method)
    error("Expected HTTP method (GET, POST, PUT, PATCH, DELETE, OPTIONS, HEAD), got '#{method}'")
  end
  emit(HTTP_METHOD, method.downcase.to_sym)

  skip_whitespace

  # Read path
  path = read_path
  error("Expected path starting with '/' after HTTP method") if path.empty?
  emit(PATH, path)

  skip_whitespace
  expect_char(":")
  emit(COLON, ":")
  advance
  skip_to_end_of_line
end

#tokenize_behavior ⇒ void (private)

This method returns an undefined value.

Tokenize a behavior annotation (@name)



909
910
911
912
913
 
# File 'lib/synthra/parser/lexer.rb', line 909

def tokenize_behavior
  advance  # Skip @
  name = read_identifier
  emit(AT_BEHAVIOR, name)
end

#tokenize_content ⇒ void (private)

This method returns an undefined value.

Tokenize content (not at line start)

Dispatches to specific tokenizers based on current character. Recursively processes until end of line.



754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
 
# File 'lib/synthra/parser/lexer.rb', line 754

def tokenize_content
  skip_whitespace

  return if @pos >= @source.length

  case current_char
  when "\n"
    emit(NEWLINE, "\n")
    advance
  when "#"
    skip_comment
  when "@"
    tokenize_behavior
  when ":"

    # Check if this is a symbol literal (:ar, :en, etc.) or a colon separator
    if peek_char =~ /[A-Za-z_]/
      tokenize_symbol
    else
      emit(COLON, ":")
      advance

      # Process type after colon for field definitions (not inside parens)
      tokenize_after_colon if @paren_depth.zero?
    end
  when "("
    emit(LPAREN, "(")
    @paren_depth += 1
    advance
  when ")"
    emit(RPAREN, ")")
    @paren_depth -= 1
    advance
  when ","
    emit(COMMA, ",")
    advance
  when "-"

    # Handle negative numbers (e.g., -10, -10..10)
    if peek_char =~ /[0-9]/
      tokenize_number
    else
      emit(MINUS, "-")
      advance
    end

  when "*"
    # Arithmetic operator, e.g. inside formula(a * b)
    emit(STAR, "*")
    advance

  when "+"
    # Arithmetic operator, e.g. inside formula(a + b)
    emit(PLUS, "+")
    advance

  when "/"
    # Arithmetic operator, e.g. inside formula(a / b). (Route paths are
    # tokenized separately via tokenize_behavior, so a bare "/" here was
    # previously an "Unexpected character" error.)
    emit(SLASH, "/")
    advance

  when "."
    if peek_char =~ /\d/
      # .123 style decimal
      tokenize_number
    else
      emit(DOT, ".")
      advance
    end
  when "?"
    emit(QUESTION, "?")
    advance
  when '"', "'"
    tokenize_string
  when /[0-9]/
    tokenize_number
  when /[A-Za-z_]/
    # Check for schema reference: schema(:Name)
    if looking_at_schema_reference?
      tokenize_schema_reference
    else
      tokenize_identifier_or_keyword
    end
  else
    error("Unexpected character: '#{current_char}'")
  end

  # Continue tokenizing until end of line
  # Important: Check @at_line_start because after advancing past newline,
  # we've set @at_line_start = true and should return to let handle_line_start
  # process the next line's indentation
  tokenize_content unless @pos >= @source.length || current_char == "\n" || @at_line_start
end

#tokenize_identifier_or_keyword ⇒ void (private)

This method returns an undefined value.

Tokenize an identifier or keyword

Determines if the identifier is a:

  • Keyword (if, true, false, Ref)
  • Known type (uuid, text, etc.)
  • Schema reference (starts with uppercase)
  • Field name (followed by :)
  • Regular identifier


1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
 
# File 'lib/synthra/parser/lexer.rb', line 1061

def tokenize_identifier_or_keyword
  name = read_identifier

  # Check for ? suffix (optional/nullable)
  has_question = current_char == "?"
  if has_question
    advance
  end

  # Check if followed by : (field definition)
  # But NOT if we're inside parentheses (named arguments)
  skip_whitespace
  if current_char == ":" && @paren_depth.zero?
    emit(FIELD_NAME, has_question ? "#{name}?" : name)
    return
  end

  # Handle keywords (these apply everywhere)
  if KEYWORDS.key?(name)
    emit(KEYWORDS[name], name)
    emit(QUESTION, "?") if has_question
    return
  end

  # Inside parentheses, treat as identifiers (for named args/enum values)
  if @paren_depth > 0
    emit(IDENTIFIER, name)
    emit(QUESTION, "?") if has_question
    return
  end

  # Handle known types (only at field-definition level)
  if KNOWN_TYPES.include?(name)
    emit(TYPE, has_question ? "#{name}?" : name)
    return
  end

  # Check if starts with uppercase (schema reference)
  if name[0] =~ /[A-Z]/
    emit(TYPE, has_question ? "#{name}?" : name)
    return
  end

  # Regular identifier
  emit(IDENTIFIER, name)
  emit(QUESTION, "?") if has_question
end

#tokenize_line ⇒ void (private)

This method returns an undefined value.

Process tokens from current position

Handles line-start vs content differently for indentation tracking.



283
284
285
286
287
288
289
 
# File 'lib/synthra/parser/lexer.rb', line 283

def tokenize_line
  if @at_line_start
    handle_line_start
  else
    tokenize_content
  end
end

#tokenize_number ⇒ void (private)

This method returns an undefined value.

Tokenize a number, range, or percentage

Handles:

  • Integers: 123
  • Floats: 3.14
  • Negative: -10
  • Ranges: 18..80
  • Percentages: 50%
  • Duration suffixes: 100ms, 5s, 2h


971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
 
# File 'lib/synthra/parser/lexer.rb', line 971

def tokenize_number
  start_col = @column
  value = ""

  # Handle negative numbers
  if current_char == "-"
    value += current_char
    advance
  end

  # Integer part
  while @pos < @source.length && current_char =~ /[0-9]/
    value += current_char
    advance
  end

  # Check for range (..)
  if current_char == "." && peek_char == "."
    # :nocov: `value` only holds the integer part here — the decimal part is parsed
    # after this range check — so value.include?(".") is always false (then-arm dead).
    left = value.include?(".") ? value.to_f : value.to_i
    # :nocov:
    advance  # First .
    advance  # Second .

    # Parse right side of range (may be negative)
    right_value = ""
    right_value += current_char and advance if current_char == "-"
    while @pos < @source.length && current_char =~ /[0-9.]/
      right_value += current_char
      advance
    end

    # Handle duration suffix (ms, s, m, h, d, y)
    suffix = ""
    while @pos < @source.length && current_char =~ /[mshdyMHDY]/
      suffix += current_char
      advance
    end

    right = right_value.include?(".") ? right_value.to_f : right_value.to_i
    emit(RANGE, { min: left, max: right, suffix: suffix.empty? ? nil : suffix.downcase })
    return
  end

  # Decimal part
  if current_char == "." && peek_char =~ /[0-9]/
    value += current_char
    advance
    while @pos < @source.length && current_char =~ /[0-9]/
      value += current_char
      advance
    end
  end

  # Check for percent
  if current_char == "%"
    emit(NUMBER, value.to_i)
    emit(PERCENT, "%")
    advance
    return
  end

  # Check for duration suffix
  suffix = ""
  while @pos < @source.length && current_char =~ /[mshdyMHDY]/
    suffix += current_char
    advance
  end

  num = value.include?(".") ? value.to_f : value.to_i
  if suffix.empty?
    emit(NUMBER, num)
  else
    emit(NUMBER, { value: num, suffix: suffix.downcase })
  end
end

#tokenize_scenario_definition ⇒ void (private)

This method returns an undefined value.

Tokenize a scenario definition (scenario Name:)



608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
 
# File 'lib/synthra/parser/lexer.rb', line 608

def tokenize_scenario_definition
  emit(SCENARIO_KEYWORD, "scenario")
  skip_whitespace

  # Read scenario name
  name = read_identifier
  error("Expected scenario name after 'scenario'") if name.empty?
  emit(IDENTIFIER, name)

  skip_whitespace
  expect_char(":")
  emit(COLON, ":")
  advance
  skip_to_end_of_line
end

#tokenize_schema_definition ⇒ void (private)

This method returns an undefined value.

Tokenize a schema definition (schema Name:)



517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
 
# File 'lib/synthra/parser/lexer.rb', line 517

def tokenize_schema_definition
  emit(SCHEMA_KEYWORD, "schema")
  skip_whitespace

  # Read schema name
  name = read_identifier
  error("Expected schema name after 'schema'") if name.empty?
  emit(SCHEMA_NAME, name)

  skip_whitespace
  expect_char(":")
  emit(COLON, ":")
  advance
  skip_to_end_of_line
end

#tokenize_schema_name ⇒ void (private)

This method returns an undefined value.

Tokenize a schema name (SchemaName:)



458
459
460
461
462
463
464
465
466
 
# File 'lib/synthra/parser/lexer.rb', line 458

def tokenize_schema_name
  name = read_identifier
  emit(SCHEMA_NAME, name)
  skip_whitespace
  expect_char(":")
  emit(COLON, ":")
  advance
  skip_to_end_of_line
end

#tokenize_schema_reference ⇒ void (private)

This method returns an undefined value.

Tokenize a schema reference: schema(:Name) or schema(:Name).array



658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
 
# File 'lib/synthra/parser/lexer.rb', line 658

def tokenize_schema_reference
  # Read "schema" keyword (read_identifier will consume it)
  keyword = read_identifier
  emit(SCHEMA_KEYWORD, keyword)

  skip_whitespace
  expect_char("(")
  emit(LPAREN, "(")
  advance

  expect_char(":")
  advance  # Skip the colon in schema(:Name)

  # Read the schema name
  name = read_identifier
  error("Expected schema name after 'schema(:'") if name.empty?
  emit(SCHEMA_REF, name)

  expect_char(")")
  emit(RPAREN, ")")
  advance

  # Check for .array or .optional modifier
  if current_char == "."
    emit(DOT, ".")
    advance
    modifier = read_identifier
    if modifier == "array"
      emit(IDENTIFIER, "array")
      # Check for range like .array(1..10)
      if current_char == "("
        emit(LPAREN, "(")
        @paren_depth += 1
        advance
      end
    elsif modifier == "optional"
      emit(IDENTIFIER, "optional")
    else
      emit(IDENTIFIER, modifier)
    end
  end
end

#tokenize_simulyra_keyword ⇒ void (private)

This method returns an undefined value.

Tokenize a Simulyra DSL keyword and its arguments

Handles:

  • schema Name:
  • api METHOD /path:
  • scenario Name:


499
500
501
502
503
504
505
506
507
508
509
510
 
# File 'lib/synthra/parser/lexer.rb', line 499

def tokenize_simulyra_keyword
  keyword = read_identifier

  case keyword
  when "schema"
    tokenize_schema_definition
  when "api"
    tokenize_api_definition
  when "scenario"
    tokenize_scenario_definition
  end
end

#tokenize_string ⇒ void (private)

This method returns an undefined value.

Tokenize a string literal ("..." or '...')

Supports escape sequences: \n, \t, \r, \", \'

Raises:



923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
 
# File 'lib/synthra/parser/lexer.rb', line 923

def tokenize_string
  quote = current_char
  advance  # Skip opening quote

  value = ""
  while @pos < @source.length && current_char != quote
    if current_char == "\\"
      advance
      value += escape_char(current_char)
    else
      value += current_char
    end
    advance
  end

  error("Unterminated string") if @pos >= @source.length
  advance  # Skip closing quote
  emit(STRING, value)
end

#tokenize_symbol ⇒ void (private)

This method returns an undefined value.

Tokenize a symbol literal (:identifier)

Symbols are colon-prefixed identifiers like :ar, :en, :locale



1116
1117
1118
1119
1120
 
# File 'lib/synthra/parser/lexer.rb', line 1116

def tokenize_symbol
  advance  # Skip the colon
  name = read_identifier
  emit(SYMBOL, name.to_sym)
end