Class: HeadMusic::Notation::ABC::BodyLexer

Inherits:
Object
  • Object
show all
Defined in:
lib/head_music/notation/abc/body_lexer.rb

Overview

Tokenizes the body of an ABC tune (the text after the K: header line).

Emits flat value tokens rather than a tree so the parser can decide how much structure to build. Out-of-scope-but-valid ABC constructs are emitted as :unsupported tokens (instead of raising here) so the parser can raise UnsupportedFeatureError with knowledge of what it was.

Defined Under Namespace

Classes: ChordNote, Token

Constant Summary collapse

BAR_LINE_PATTERN =

Alternatives are ordered longest-first so the scanner never takes a short match when a longer bar line is present (e.g. “|]” before “|”).

/:\|\|:|:\|:|::|:\||\|:|\|\||\|\]|\[\||\|/
NORMALIZED_BAR_STYLES =
”:   :” and “: :” are alternate spellings of the double repeat “::”. 
{":||:" => "::", ":|:" => "::"}.freeze
NOTE_PATTERN = 
%r{(\^\^|\^|__|_|=)?([A-Ga-g])([',]*)([\d/]*)}
CHORD_START_PATTERN = 
/\[(\^\^|\^|__|_|=)?[A-Ga-g]/
INLINE_FIELD_PATTERNS =

An inline field (“[K:…]”), tried closed before its unterminated fallback.

[/\[[A-Za-z]:[^\]]*\]/, /\[[A-Za-z]:[^\]]*/].freeze
CHORD_NOTE_PATTERN = 
%r{(\^\^|\^|__|_|=)?([A-Ga-g])([',]*)([\d/]*)}
REST_PATTERN = 
%r{z([\d/]*)}
VOLTA_DIGITS_PATTERN = 
/\d[\d,-]*/
UNSUPPORTED_PATTERNS =

Recognizable ABC we deliberately don’t handle: grace notes (..), decorations (!..!), tuplets, slurs, and special rests (Z, x). Ordered so a closed form is tried before its unterminated fallback.

[
  /\{[^}]*\}/, /\{[^}]*/, /![^!]*!/, /![^!]*/,
  /\(\d/, /[()~.]/, /Z\d*/, %r{x[\d/]*}
].freeze
SNIPPET_LENGTH = 
20
BEAMABLE_TOKEN_TYPES =

Music tokens whose whitespace successor breaks a beam group; other tokens (bar lines, ties, etc.) already interrupt beaming on their own.

[:note, :rest, :chord].freeze

Instance Method Summary collapse

Constructor Details

#initialize(body_text, start_line: 1) ⇒ BodyLexer

Returns a new instance of BodyLexer.



57
58
59
60
61
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 57

def initialize(body_text, start_line: 1)
  @body = body_text.to_s
  @start_line = start_line
  ensure_valid_encoding
end

Instance Method Details

#beam_break_token(scanner, line_number) ⇒ Object (private) 



147
148
149
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 147

def beam_break_token(scanner, line_number)
  Token.new(type: :beam_break, line: line_number, column: scanner.pos + 1)
end

#beamable_predecessor?(token) ⇒ Boolean (private)

A beam break only matters after a music token; whitespace elsewhere (leading, after a bar line) carries no beaming signal.

Returns:

  • (Boolean) 


143
144
145
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 143

def beamable_predecessor?(token)
  !token.nil? && BEAMABLE_TOKEN_TYPES.include?(token.type)
end

#chord_fallback(scanner, start_pos, line_number, column, tokens) ⇒ Object (private)

Non-note content inside the brackets (ties, rests, spaces, decorations) makes the whole group one unsupported token, matching how those constructs surface outside a chord.



245
246
247
248
249
250
251
252
253
254
255
256
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 245

def chord_fallback(scanner, start_pos, line_number, column, tokens)
  if scanner.eos?
    raise HeadMusic::Notation::ABC::ParseError.new(
      'Unterminated chord; expected "]"',
      line_number: line_number, snippet: chord_snippet(scanner, start_pos)
    )
  end
  scanner.pos = start_pos
  lexeme = scanner.scan(/\[[^\]]*\]/) || scanner.scan(/\[[^\]]*/)
  tokens << unsupported_token(lexeme, line_number, column)
  true
end

#chord_snippet(scanner, start_pos) ⇒ Object (private) 



258
259
260
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 258

def chord_snippet(scanner, start_pos)
  scanner.string[start_pos, SNIPPET_LENGTH]
end

#collect_chord_notes(scanner) ⇒ Object (private)

Collects the notes between the brackets, or nil when a non-note is hit (leaving the scanner where it stopped so the fallback can react).



229
230
231
232
233
234
235
236
237
238
239
240
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 229

def collect_chord_notes(scanner)
  notes = []
  until scanner.skip(/\]/)
    return nil unless scanner.scan(CHORD_NOTE_PATTERN)

    notes << ChordNote.new(
      accidental: scanner[1], letter: scanner[2],
      octave_marks: scanner[3], length: scanner[4]
    )
  end
  notes
end

#ensure_valid_encoding ⇒ Object (private)

Raises:



69
70
71
72
73
74
75
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 69

def ensure_valid_encoding
  return if @body.valid_encoding?

  raise HeadMusic::Notation::ABC::ParseError.new(
    "Tune body is not valid UTF-8", line_number: @start_line
  )
end

#expand_volta_range(part) ⇒ Object (private) 



277
278
279
280
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 277

def expand_volta_range(part)
  first, last = part.split("-", 2)
  last ? (first.to_i..last.to_i).to_a : [first.to_i]
end

#header_field_line?(line_text) ⇒ Boolean (private)

A note followed by a repeat bar (e.g. “A:|”) also puts a colon after a letter at line start, so bar-line characters after the colon disqualify.

Returns:

  • (Boolean) 


113
114
115
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 113

def header_field_line?(line_text)
  line_text.match?(/\A[A-Za-z]:/) && !["|", ":"].include?(line_text[2])
end

#lex ⇒ Object (private) 



77
78
79
80
81
82
83
84
85
86
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 77

def lex
  tokens = []
  continued = false
  @body.lines.map(&:chomp).each_with_index do |line_text, index|
    break if line_text.strip.empty?

    continued = lex_line(line_text, @start_line + index, tokens, continued)
  end
  tokens
end

#lex_line(line_text, line_number, tokens, continued) ⇒ Object (private) 



88
89
90
91
92
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 88

def lex_line(line_text, line_number, tokens, continued)
  return false if !continued && line_start_token(line_text, line_number, tokens)

  scan_line(line_text, line_number, tokens)
end

#line_start_token(line_text, line_number, tokens) ⇒ Object (private)

Handles lines that are fields rather than music: V: switches voices; any other letter-colon line is a field we don’t interpret in the body.



96
97
98
99
100
101
102
103
104
105
106
107
108
109
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 96

def line_start_token(line_text, line_number, tokens)
  if line_text.start_with?("V:")
    tokens << voice_change_token(line_text, line_number)
    return true
  end
  if header_field_line?(line_text)
    tokens << Token.new(
      type: :unsupported, line: line_number, column: 1,
      lexeme: line_text.strip[0, SNIPPET_LENGTH]
    )
    return true
  end
  false
end

#raise_unexpected_character(scanner, line_number, column) ⇒ Object (private)

Raises:



351
352
353
354
355
356
357
358
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 351

def raise_unexpected_character(scanner, line_number, column)
  character = scanner.peek(1)
  raise HeadMusic::Notation::ABC::ParseError.new(
    "Unexpected character #{character.inspect} at column #{column}",
    line_number: line_number,
    snippet: scanner.rest[0, SNIPPET_LENGTH]
  )
end

#scan_bar_line(scanner, line_number, column, tokens) ⇒ Object (private) 



176
177
178
179
180
181
182
183
184
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 176

def scan_bar_line(scanner, line_number, column, tokens)
  lexeme = scanner.scan(BAR_LINE_PATTERN)
  return false unless lexeme

  style = NORMALIZED_BAR_STYLES.fetch(lexeme, lexeme)
  tokens << Token.new(type: :bar_line, line: line_number, column: column, style: style)
  scan_trailing_volta(scanner, line_number, tokens)
  true
end

#scan_bracket(scanner, line_number, column, tokens) ⇒ Object (private) 



196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 196

def scan_bracket(scanner, line_number, column, tokens)
  return false unless scanner.check(/\[/)

  digits = scanner.scan(/\[(\d[\d,-]*)/)
  if digits
    tokens << volta_token(scanner[1], line_number, column)
    return true
  end

  inline_field = scan_first(scanner, INLINE_FIELD_PATTERNS)
  if inline_field
    tokens << unsupported_token(inline_field, line_number, column)
    return true
  end

  return scan_chord(scanner, line_number, column, tokens) if scanner.check(CHORD_START_PATTERN)

  raise_unexpected_character(scanner, line_number, column)
end

#scan_broken_rhythm(scanner, line_number, column, tokens) ⇒ Object (private) 



300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 300

def scan_broken_rhythm(scanner, line_number, column, tokens)
  # Doubled marks (">>", "<<") are valid ABC (double-dotted broken
  # rhythm) but out of scope, so they surface as unsupported rather
  # than as a malformed-input error.
  doubled = scanner.scan(/[<>]{2,}/)
  if doubled
    tokens << unsupported_token(doubled, line_number, column)
    return true
  end

  lexeme = scanner.scan(/[<>]/)
  return false unless lexeme

  tokens << Token.new(
    type: :broken_rhythm, line: line_number, column: column, direction: lexeme.to_sym
  )
  true
end

#scan_chord(scanner, line_number, column, tokens) ⇒ Object (private) 



216
217
218
219
220
221
222
223
224
225
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 216

def scan_chord(scanner, line_number, column, tokens)
  start_pos = scanner.pos
  scanner.skip(/\[/)
  notes = collect_chord_notes(scanner)
  return chord_fallback(scanner, start_pos, line_number, column, tokens) unless notes

  length = scanner.scan(%r{[\d/]*})
  tokens << Token.new(type: :chord, line: line_number, column: column, notes: notes, length: length)
  true
end

#scan_first(scanner, patterns) ⇒ Object (private)

Returns the first pattern’s match, trying each in order.



339
340
341
342
343
344
345
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 339

def scan_first(scanner, patterns)
  patterns.each do |pattern|
    lexeme = scanner.scan(pattern)
    return lexeme if lexeme
  end
  nil
end

#scan_line(line_text, line_number, tokens) ⇒ Object (private)

Returns true when the line ends with a continuation backslash.



127
128
129
130
131
132
133
134
135
136
137
138
139
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 127

def scan_line(line_text, line_number, tokens)
  scanner = StringScanner.new(line_text)
  until scanner.eos?
    spaced = scanner.skip(/[ \t]+/)
    break if scanner.skip(/%/)
    return true if scanner.skip(/\\[ \t]*\z/)
    break if scanner.eos?

    tokens << beam_break_token(scanner, line_number) if spaced && beamable_predecessor?(tokens.last)
    scan_token(scanner, line_number, tokens)
  end
  false
end

#scan_note(scanner, line_number, column, tokens) ⇒ Object (private) 



282
283
284
285
286
287
288
289
290
291
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 282

def scan_note(scanner, line_number, column, tokens)
  return false unless scanner.scan(NOTE_PATTERN)

  tokens << Token.new(
    type: :note, line: line_number, column: column,
    accidental: scanner[1], letter: scanner[2],
    octave_marks: scanner[3], length: scanner[4]
  )
  true
end

#scan_quoted_string(scanner, line_number, column, tokens) ⇒ Object (private)

Quoted chord symbols are consumed whole so a “%” inside quotes is never mistaken for a comment.



168
169
170
171
172
173
174
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 168

def scan_quoted_string(scanner, line_number, column, tokens)
  lexeme = scanner.scan(/"[^"]*"/) || scanner.scan(/"[^"]*/)
  return false unless lexeme

  tokens << unsupported_token(lexeme, line_number, column)
  true
end

#scan_rest(scanner, line_number, column, tokens) ⇒ Object (private) 



293
294
295
296
297
298
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 293

def scan_rest(scanner, line_number, column, tokens)
  return false unless scanner.scan(REST_PATTERN)

  tokens << Token.new(type: :rest, line: line_number, column: column, length: scanner[1])
  true
end

#scan_tie(scanner, line_number, column, tokens) ⇒ Object (private)

A tie (a hyphen following a note or chord) joins it to the next note of the same pitch; the parser fuses the two into one sounding value. Ties inside a bracket chord still surface as unsupported via the chord fallback.



323
324
325
326
327
328
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 323

def scan_tie(scanner, line_number, column, tokens)
  return false unless scanner.scan("-")

  tokens << Token.new(type: :tie, line: line_number, column: column)
  true
end

#scan_token(scanner, line_number, tokens) ⇒ Object (private) 



151
152
153
154
155
156
157
158
159
160
161
162
163
164
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 151

def scan_token(scanner, line_number, tokens)
  column = scanner.pos + 1

  return if scan_quoted_string(scanner, line_number, column, tokens)
  return if scan_bar_line(scanner, line_number, column, tokens)
  return if scan_bracket(scanner, line_number, column, tokens)
  return if scan_note(scanner, line_number, column, tokens)
  return if scan_rest(scanner, line_number, column, tokens)
  return if scan_tie(scanner, line_number, column, tokens)
  return if scan_broken_rhythm(scanner, line_number, column, tokens)
  return if scan_unsupported(scanner, line_number, column, tokens)

  raise_unexpected_character(scanner, line_number, column)
end

#scan_trailing_volta(scanner, line_number, tokens) ⇒ Object (private)

After a bar line, an immediately following digit begins a volta (the “|1” / “:|2” shorthands for first and second endings).



188
189
190
191
192
193
194
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 188

def scan_trailing_volta(scanner, line_number, tokens)
  return unless scanner.check(/\d/)

  column = scanner.pos + 1
  digits = scanner.scan(VOLTA_DIGITS_PATTERN)
  tokens << volta_token(digits, line_number, column)
end

#scan_unsupported(scanner, line_number, column, tokens) ⇒ Object (private) 



330
331
332
333
334
335
336
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 330

def scan_unsupported(scanner, line_number, column, tokens)
  lexeme = scan_first(scanner, UNSUPPORTED_PATTERNS)
  return false unless lexeme

  tokens << unsupported_token(lexeme, line_number, column)
  true
end

#tokens ⇒ Object 



63
64
65
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 63

def tokens
  @tokens ||= lex
end

#unsupported_token(lexeme, line_number, column) ⇒ Object (private) 



347
348
349
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 347

def unsupported_token(lexeme, line_number, column)
  Token.new(type: :unsupported, line: line_number, column: column, lexeme: lexeme)
end

#voice_change_token(line_text, line_number) ⇒ Object (private) 



117
118
119
120
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 117

def voice_change_token(line_text, line_number)
  voice_id = line_text.delete_prefix("V:").split("%", 2).first.to_s.strip
  Token.new(type: :voice_change, line: line_number, column: 1, voice_id: voice_id)
end

#volta_passes(digits) ⇒ Object (private) 



273
274
275
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 273

def volta_passes(digits)
  digits.split(",").flat_map { |part| expand_volta_range(part) }.select(&:positive?)
end

#volta_token(digits, line_number, column) ⇒ Object (private) 



262
263
264
265
266
267
268
269
270
271
 
# File 'lib/head_music/notation/abc/body_lexer.rb', line 262

def volta_token(digits, line_number, column)
  passes = volta_passes(digits)
  unless passes.uniq.length == passes.length
    raise HeadMusic::Notation::ABC::ParseError.new(
      "Volta passes must be unique",
      line_number: line_number, snippet: digits
    )
  end
  Token.new(type: :volta, line: line_number, column: column, passes: passes)
end