Class: CharSplit::GermanNouns

Inherits:
Object
  • Object
show all
Includes:
Enumerable
Defined in:
lib/charsplit/german_nouns.rb

Overview

Reads the German noun CSV and turns its inflection records into words.

Constant Summary collapse

FORMS = 
%w[lemmas nominative all].freeze
GERMAN_WORD = 
/\A[A-Za-zÄÖÜäöüẞß]+(?:-[A-Za-zÄÖÜäöüẞß]+)*\z/
POS_HEADERS = 
%w[pos partofspeech wordclass wortart wortklasse wordtype type].freeze
LEMMA_HEADERS = 
%w[lemma lexeme grundform].freeze
CASE_NAMES = 
%w[nominativ genitiv dativ akkusativ nominative genitive dative accusative].freeze

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(path, forms: "lemmas") ⇒ GermanNouns

Returns a new instance of GermanNouns.

Raises:

  • (ArgumentError) 


18
19
20
21
22
23
 
# File 'lib/charsplit/german_nouns.rb', line 18

def initialize(path, forms: "lemmas")
  raise ArgumentError, "forms must be one of: #{FORMS.join(", ")}" unless FORMS.include?(forms)

  @path = path
  @forms = forms
end

Class Method Details

.normalize(value) ⇒ Object 



41
42
43
44
45
46
47
48
 
# File 'lib/charsplit/german_nouns.rb', line 41

def self.normalize(value)
  return unless value.is_a?(String) && value.valid_encoding?

  word = value.unicode_normalize(:nfc).strip
  word if GERMAN_WORD.match?(word)
rescue Encoding::CompatibilityError
  nil
end

Instance Method Details

#each ⇒ Object 



25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
 
# File 'lib/charsplit/german_nouns.rb', line 25

def each
  return enum_for(__method__) unless block_given?

  seen = Set.new
  CSV.foreach(@path, headers: true, encoding: "bom|utf-8") do |record|
    headers = header_map(record.headers)
    next unless noun?(record, headers)
    next unless lemma_row?(record)

    selected_values(record, headers).each do |value|
      word = self.class.normalize(value)
      yield word if word && seen.add?(word)
    end
  end
end