Class: HeadMusic::Notation::ABC::BodyLexer

Inherits:
Object
  • Object
show all
Defined in:
lib/head_music/notation/abc/body_lexer.rb

Overview

Tokenizes the body of an ABC tune (the text after the K: header line).

Emits flat value tokens rather than a tree so the parser can decide how much structure to build. Out-of-scope-but-valid ABC constructs are emitted as :unsupported tokens (instead of raising here) so the parser can raise UnsupportedFeatureError with knowledge of what it was.

Constant Summary collapse

BAR_LINE_PATTERN =

Alternatives are ordered longest-first so the scanner never takes a short match when a longer bar line is present (e.g. “|]” before “|”).

/:\|\|:|:\|:|::|:\||\|:|\|\||\|\]|\[\||\|/
NORMALIZED_BAR_STYLES =
”:   :” and “: :” are alternate spellings of the double repeat “::”. 
{":||:" => "::", ":|:" => "::"}.freeze
NOTE_PATTERN = 
%r{(\^\^|\^|__|_|=)?([A-Ga-g])([',]*)([\d/]*)}
INLINE_FIELD_PATTERNS =

An inline field (“[K:…]”), tried closed before its unterminated fallback.

[/\[[A-Za-z]:[^\]]*\]/, /\[[A-Za-z]:[^\]]*/].freeze
REST_PATTERN = 
%r{z([\d/]*)}
VOLTA_DIGITS_PATTERN = 
/\d[\d,-]*/
UNSUPPORTED_PATTERNS =

Recognizable ABC we deliberately don’t handle: grace notes (..), decorations (!..!), tuplets, slurs, and special rests (Z, x). Ordered so a closed form is tried before its unterminated fallback.

[
  /\{[^}]*\}/, /\{[^}]*/, /![^!]*!/, /![^!]*/,
  /\(\d/, /[()~.]/, /Z\d*/, %r{x[\d/]*}
].freeze
BEAMABLE_TOKEN_TYPES =

Music tokens whose whitespace successor breaks a beam group; other tokens (bar lines, ties, etc.) already interrupt beaming on their own.

[:note, :rest, :chord].freeze
SNIPPET_LENGTH = 
20

Instance Method Summary collapse

Constructor Details

#initialize(body_text, start_line: 1) ⇒ BodyLexer

Returns a new instance of BodyLexer.



39
40
41
42
43
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 39

def initialize(body_text, start_line: 1)
  @body = body_text.to_s
  @start_line = start_line
  ensure_valid_encoding
end

Instance Method Details

#beam_break_token(scanner, line_number) ⇒ Object (private) 



123
124
125
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 123

def beam_break_token(scanner, line_number)
  Token.new(type: :beam_break, line: line_number, column: scanner.pos + 1)
end

#beamable_predecessor?(token) ⇒ Boolean (private)

A beam break only matters after a music token; whitespace elsewhere (leading, after a bar line) carries no beaming signal.

Returns:

  • (Boolean) 


119
120
121
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 119

def beamable_predecessor?(token)
  !token.nil? && BEAMABLE_TOKEN_TYPES.include?(token.type)
end

#ensure_valid_encoding ⇒ Object (private)

Raises:



51
52
53
54
55
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 51

def ensure_valid_encoding
  return if @body.valid_encoding?

  raise ParseError.new("Tune body is not valid UTF-8", line_number: @start_line)
end

#header_field_line?(line_text) ⇒ Boolean (private)

A note followed by a repeat bar (e.g. “A:|”) also puts a colon after a letter at line start, so bar-line characters after the colon disqualify.

Returns:

  • (Boolean) 


93
94
95
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 93

def header_field_line?(line_text)
  line_text.match?(/\A[A-Za-z]:/) && !["|", ":"].include?(line_text[2])
end

#lex ⇒ Object (private) 



57
58
59
60
61
62
63
64
65
66
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 57

def lex
  tokens = []
  continued = false
  @body.lines.map(&:chomp).each_with_index do |line_text, index|
    break if line_text.strip.empty?

    continued = lex_line(line_text, @start_line + index, tokens, continued)
  end
  tokens
end

#lex_line(line_text, line_number, tokens, continued) ⇒ Object (private) 



68
69
70
71
72
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 68

def lex_line(line_text, line_number, tokens, continued)
  return false if !continued && line_start_token(line_text, line_number, tokens)

  scan_line(line_text, line_number, tokens)
end

#line_start_token(line_text, line_number, tokens) ⇒ Object (private)

Handles lines that are fields rather than music: V: switches voices; any other letter-colon line is a field we don’t interpret in the body.



76
77
78
79
80
81
82
83
84
85
86
87
88
89
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 76

def line_start_token(line_text, line_number, tokens)
  if line_text.start_with?("V:")
    tokens << voice_change_token(line_text, line_number)
    return true
  end
  if header_field_line?(line_text)
    tokens << Token.new(
      type: :unsupported, line: line_number, column: 1,
      lexeme: line_text.strip[0, SNIPPET_LENGTH]
    )
    return true
  end
  false
end

#raise_unexpected_character(scanner, line_number, column) ⇒ Object (private)

Raises:



273
274
275
276
277
278
279
280
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 273

def raise_unexpected_character(scanner, line_number, column)
  character = scanner.peek(1)
  raise ParseError.new(
    "Unexpected character #{character.inspect} at column #{column}",
    line_number: line_number,
    snippet: scanner.rest[0, SNIPPET_LENGTH]
  )
end

#scan_bar_line(scanner, line_number, column, tokens) ⇒ Object (private) 



152
153
154
155
156
157
158
159
160
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 152

def scan_bar_line(scanner, line_number, column, tokens)
  lexeme = scanner.scan(BAR_LINE_PATTERN)
  return false unless lexeme

  style = NORMALIZED_BAR_STYLES.fetch(lexeme, lexeme)
  tokens << Token.new(type: :bar_line, line: line_number, column: column, style: style)
  scan_trailing_volta(scanner, line_number, tokens)
  true
end

#scan_bracket(scanner, line_number, column, tokens) ⇒ Object (private) 



172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 172

def scan_bracket(scanner, line_number, column, tokens)
  return false unless scanner.check(/\[/)

  digits = scanner.scan(/\[(\d[\d,-]*)/)
  if digits
    tokens << volta_token(scanner[1], line_number, column)
    return true
  end

  inline_field = scan_first(scanner, INLINE_FIELD_PATTERNS)
  if inline_field
    tokens << unsupported_token(inline_field, line_number, column)
    return true
  end

  return scan_chord(scanner, line_number, column, tokens) if scanner.check(ChordScanner::START_PATTERN)

  raise_unexpected_character(scanner, line_number, column)
end

#scan_broken_rhythm(scanner, line_number, column, tokens) ⇒ Object (private) 



222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 222

def scan_broken_rhythm(scanner, line_number, column, tokens)
  # Doubled marks (">>", "<<") are valid ABC (double-dotted broken
  # rhythm) but out of scope, so they surface as unsupported rather
  # than as a malformed-input error.
  doubled = scanner.scan(/[<>]{2,}/)
  if doubled
    tokens << unsupported_token(doubled, line_number, column)
    return true
  end

  lexeme = scanner.scan(/[<>]/)
  return false unless lexeme

  tokens << Token.new(
    type: :broken_rhythm, line: line_number, column: column, direction: lexeme.to_sym
  )
  true
end

#scan_chord(scanner, line_number, column, tokens) ⇒ Object (private) 



192
193
194
195
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 192

def scan_chord(scanner, line_number, column, tokens)
  tokens << ChordScanner.new(scanner, line_number, column).token
  true
end

#scan_first(scanner, patterns) ⇒ Object (private)

Returns the first pattern’s match, trying each in order.



261
262
263
264
265
266
267
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 261

def scan_first(scanner, patterns)
  patterns.each do |pattern|
    lexeme = scanner.scan(pattern)
    return lexeme if lexeme
  end
  nil
end

#scan_line(line_text, line_number, tokens) ⇒ Object (private)

Returns true when the line ends with a continuation backslash.



103
104
105
106
107
108
109
110
111
112
113
114
115
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 103

def scan_line(line_text, line_number, tokens)
  scanner = StringScanner.new(line_text)
  until scanner.eos?
    spaced = scanner.skip(/[ \t]+/)
    break if scanner.skip(/%/)
    return true if scanner.skip(/\\[ \t]*\z/)
    break if scanner.eos?

    tokens << beam_break_token(scanner, line_number) if spaced && beamable_predecessor?(tokens.last)
    scan_token(scanner, line_number, tokens)
  end
  false
end

#scan_note(scanner, line_number, column, tokens) ⇒ Object (private) 



204
205
206
207
208
209
210
211
212
213
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 204

def scan_note(scanner, line_number, column, tokens)
  return false unless scanner.scan(NOTE_PATTERN)

  tokens << Token.new(
    type: :note, line: line_number, column: column,
    accidental: scanner[1], letter: scanner[2],
    octave_marks: scanner[3], length: scanner[4]
  )
  true
end

#scan_quoted_string(scanner, line_number, column, tokens) ⇒ Object (private)

Quoted chord symbols are consumed whole so a “%” inside quotes is never mistaken for a comment.



144
145
146
147
148
149
150
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 144

def scan_quoted_string(scanner, line_number, column, tokens)
  lexeme = scanner.scan(/"[^"]*"/) || scanner.scan(/"[^"]*/)
  return false unless lexeme

  tokens << unsupported_token(lexeme, line_number, column)
  true
end

#scan_rest(scanner, line_number, column, tokens) ⇒ Object (private) 



215
216
217
218
219
220
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 215

def scan_rest(scanner, line_number, column, tokens)
  return false unless scanner.scan(REST_PATTERN)

  tokens << Token.new(type: :rest, line: line_number, column: column, length: scanner[1])
  true
end

#scan_tie(scanner, line_number, column, tokens) ⇒ Object (private)

A tie (a hyphen following a note or chord) joins it to the next note of the same pitch; the parser fuses the two into one sounding value. Ties inside a bracket chord still surface as unsupported via the chord fallback.



245
246
247
248
249
250
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 245

def scan_tie(scanner, line_number, column, tokens)
  return false unless scanner.scan("-")

  tokens << Token.new(type: :tie, line: line_number, column: column)
  true
end

#scan_token(scanner, line_number, tokens) ⇒ Object (private) 



127
128
129
130
131
132
133
134
135
136
137
138
139
140
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 127

def scan_token(scanner, line_number, tokens)
  column = scanner.pos + 1

  return if scan_quoted_string(scanner, line_number, column, tokens)
  return if scan_bar_line(scanner, line_number, column, tokens)
  return if scan_bracket(scanner, line_number, column, tokens)
  return if scan_note(scanner, line_number, column, tokens)
  return if scan_rest(scanner, line_number, column, tokens)
  return if scan_tie(scanner, line_number, column, tokens)
  return if scan_broken_rhythm(scanner, line_number, column, tokens)
  return if scan_unsupported(scanner, line_number, column, tokens)

  raise_unexpected_character(scanner, line_number, column)
end

#scan_trailing_volta(scanner, line_number, tokens) ⇒ Object (private)

After a bar line, an immediately following digit begins a volta (the “|1” / “:|2” shorthands for first and second endings).



164
165
166
167
168
169
170
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 164

def scan_trailing_volta(scanner, line_number, tokens)
  return unless scanner.check(/\d/)

  column = scanner.pos + 1
  digits = scanner.scan(VOLTA_DIGITS_PATTERN)
  tokens << volta_token(digits, line_number, column)
end

#scan_unsupported(scanner, line_number, column, tokens) ⇒ Object (private) 



252
253
254
255
256
257
258
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 252

def scan_unsupported(scanner, line_number, column, tokens)
  lexeme = scan_first(scanner, UNSUPPORTED_PATTERNS)
  return false unless lexeme

  tokens << unsupported_token(lexeme, line_number, column)
  true
end

#tokens ⇒ Object 



45
46
47
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 45

def tokens
  @tokens ||= lex
end

#unsupported_token(lexeme, line_number, column) ⇒ Object (private) 



269
270
271
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 269

def unsupported_token(lexeme, line_number, column)
  Token.new(type: :unsupported, line: line_number, column: column, lexeme: lexeme)
end

#voice_change_token(line_text, line_number) ⇒ Object (private) 



97
98
99
100
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 97

def voice_change_token(line_text, line_number)
  voice_id = line_text.delete_prefix("V:").split("%", 2).first.to_s.strip
  Token.new(type: :voice_change, line: line_number, column: 1, voice_id: voice_id)
end

#volta_token(digits, line_number, column) ⇒ Object (private) 



197
198
199
200
201
202
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 197

def volta_token(digits, line_number, column)
  Token.new(
    type: :volta, line: line_number, column: column,
    passes: VoltaPasses.parse(digits, line_number)
  )
end