Module: JLPT::Loanwords

Defined in:
lib/jlpt/analyzers/loanwords.rb

Overview

Japanese Word Origin and Loanword (Gairaigo 外来語) Detector.

Categorizes vocabulary into Katakana loanwords (Gairaigo), Sino-Japanese Kanji compounds (Kango), and native Japanese words (Wago).

Constant Summary collapse

KATAKANA_REGEX = 
/^[\u30A0-\u30FFー]+$/.freeze
KANJI_REGEX = 
/[\u4E00-\u9FFF]/.freeze

Class Method Summary collapse

Class Method Details

.breakdown(text) ⇒ Hash{Symbol => Array<String>}

Categorize text vocabulary by word origin (Wago, Kango, Gairaigo)

Parameters:

  • text (String) —

    Japanese text

Returns:

  • (Hash{Symbol => Array<String>}) —

    origin breakdown hash



30
31
32
33
34
35
36
37
38
39
 
# File 'lib/jlpt/analyzers/loanwords.rb', line 30

def breakdown(text)
  cleaned = Preprocessor.clean(text)
  res = { gairaigo: [], kango: [], wago: [] }
  return res if cleaned.empty?

  Tokenizer.lemmata(cleaned).uniq.each do |lemma|
    categorize_lemma(lemma, res)
  end
  res
end

.extract(text) ⇒ Array<String>

Extract all Katakana loanwords (Gairaigo) from text

Parameters:

  • text (String) —

    Japanese text

Returns:

  • (Array<String>) —

    unique array of loanword lemmata



18
19
20
21
22
23
24
 
# File 'lib/jlpt/analyzers/loanwords.rb', line 18

def extract(text)
  cleaned = Preprocessor.clean(text)
  return [] if cleaned.empty?

  lemmas = Tokenizer.lemmata(cleaned)
  lemmas.select { |lemma| lemma.match?(KATAKANA_REGEX) && lemma.length >= 2 }.uniq
end