Class: Kotoshu::Language::Normalizer::Persian

Inherits:
Arabic
  • Object
show all
Defined in:
lib/kotoshu/language/normalizer/persian.rb

Overview

Normalizer for Persian (Farsi) script.

Extends Arabic with Persian-specific letter mappings. Persian shares the Arabic script but uses different codepoints for several letters that look the same to a reader but are distinct in Unicode:

  • Arabic Yeh (ي U+064A) → Persian Yeh (ی U+06CC)
  • Arabic Kaf (ك U+0643) → Persian Kaf (ک U+06A9)

Input text from Arabic-origin sources may contain either form. This normalizer maps both to the Persian form so a dictionary storing Persian codepoints matches.

Constant Summary collapse

PERSIAN_LETTER_MAP =

Arabic → Persian letter canonicalization.

{
  "ي" => "ی",  # Arabic Yeh → Persian Yeh
  "ك" => "ک"   # Arabic Kaf → Persian Kaf
}.freeze

Constants inherited from Arabic

Arabic::DIACRITICS, Arabic::PRESENTATION_FORM_MAP, Arabic::TATWEEL

Instance Method Summary collapse

Methods inherited from Arabic

#initialize, #normalize_word

Methods inherited from Base

#normalize_word, #normalized_eql?

Constructor Details

This class inherits a constructor from Kotoshu::Language::Normalizer::Arabic

Instance Method Details

#normalize(text, _options = {}) ⇒ Object 



26
27
28
29
30
31
32
 
# File 'lib/kotoshu/language/normalizer/persian.rb', line 26

def normalize(text, _options = {})
  return "" if text.nil? || text.empty?

  result = super
  PERSIAN_LETTER_MAP.each { |from, to| result = result.gsub(from, to) }
  result
end