Module: Jekyll::FastReader::Characters

Defined in:: lib/jekyll/fast_reader/characters.rb

Overview

Single source of truth for the character classes shared by the tokenizer and the text processor. Keeping these in one place prevents the regexes from drifting apart (which previously caused inconsistent handling of the left curly quote U+2018).

Constant Summary collapse

INNER_JOINERS = Non-letter characters allowed inside a word: straight apostrophe (\x27), left/right curly quotes (U+2018/U+2019), and hyphen. Written as escapes so the source stays unambiguous in any editor.

"\\x27\\u2018\\u2019\\-".freeze

INNER_JOINERS_CLASS = Matches one such joiner, e.g. for stripping them before counting letters.

/[#{INNER_JOINERS}]/.freeze

WORD = Matches a single latin word, which must start and end with a letter and may contain joiners in between.

/[a-zA-Z](?:[a-zA-Z#{INNER_JOINERS}]*[a-zA-Z])?/.freeze