Class: Kotoshu::Dictionary::Unified

Inherits:
Object
  • Object
show all
Defined in:
lib/kotoshu/dictionary/unified.rb

Overview

Spylls-style facade over Hunspell-format .aff/.dic dictionary files. Distinct from Hunspell (which is a backend subclass of Base) and from Spellchecker (the high-level facade): this class is the low-level entry point for callers that want direct access to the underlying Lookuper / Suggester without the surrounding configuration / personal-dictionary / suggestion-generator machinery.

Was previously declared as class Kotoshu::Dictionary, which collided with the Kotoshu::Dictionary module that holds the backend namespace — Ruby raised TypeError on autoload. Renamed to Unified so it loads cleanly under the existing autoload :Unified, "kotoshu/dictionary/unified" entry.

Examples:

Loading from files

dictionary = Dictionary::Unified.from_files('/path/to/dictionary/en_US')
dictionary.lookup('spells')  # => true

Loading from zip archive

dictionary = Dictionary::Unified.from_zip('/path/to/dictionary/en_US.odt')

Loading from system

dictionary = Dictionary::Unified.from_system('en_US')

Getting suggestions

dictionary.suggest('spylls')  # => ["spells", "spills", ...]

Accessing algorithms for experimentation

dictionary.lookuper.good_forms('building') do |form|
  puts form
end

Constant Summary collapse

PATHES =

System paths to search for Hunspell dictionaries

[
  '/usr/share/hunspell',
  '/usr/share/myspell',
  '/usr/share/myspell/dicts',
  '/Library/Spelling',
  '/opt/openoffice.org/basis3.0/share/dict/ooo',
  '/usr/lib/openoffice.org/basis3.0/share/dict/ooo',
  '/opt/openoffice.org2.4/share/dict/ooo',
  '/usr/lib/openoffice.org2.4/share/dict/ooo',
  '/opt/openoffice.org2.3/share/dict/ooo',
  '/usr/lib/openoffice.org2.3/share/dict/ooo',
  '/opt/openoffice.org2.2/share/dict/ooo',
  '/usr/lib/openoffice.org2.2/share/dict/ooo',
  '/opt/openoffice.org2.1/share/dict/ooo',
  '/usr/lib/openoffice.org2.1/share/dict/ooo',
  '/opt/openoffice.org2.0/share/dict/ooo',
  '/usr/lib/openoffice.org2.0/share/dict/ooo'
].freeze
DISTRIBUTED =

Distributed dictionaries for testing

{
  'en_US' => 'en',
  'ru' => 'ru',
  'sv_SE' => 'sv'
}.freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(aff, dic_words) ⇒ Unified

Create a Dictionary from aff and dic data.

Parameters:

  • aff (Hash) —

    Aff data structure

  • dic_words (Array<Readers::Word>) —

    Dictionary word entries



83
84
85
86
87
88
89
90
91
92
93
94
95
96
 
# File 'lib/kotoshu/dictionary/unified.rb', line 83

def initialize(aff, dic_words)
  @aff = aff
  @dic_words = dic_words

  # Build the Lookuper via LookupBuilder — it owns the aff →
  # structure transformation (casing, suffixes/prefixes indexes,
  # REP patterns, etc.). The Suggester needs that processed aff
  # structure too, so we read it back off the Lookuper rather
  # than passing the raw AffReader hash.
  @lookuper = Readers::LookupBuilder.from_data(aff, dic_words).build
  @suggester = Algorithms::Suggest::Suggester.new(
    @lookuper.aff, @lookuper.dic, @lookuper
  )
end

Instance Attribute Details

#aff ⇒ Hash (readonly)

Returns Aff data structure.

Returns:

  • (Hash) —

    Aff data structure



68
69
70
 
# File 'lib/kotoshu/dictionary/unified.rb', line 68

def aff
  @aff
end

#dic_words ⇒ Array<Readers::Word> (readonly)

Returns Dic data structure.

Returns:



71
72
73
 
# File 'lib/kotoshu/dictionary/unified.rb', line 71

def dic_words
  @dic_words
end

#lookuper ⇒ Algorithms::Lookup::Lookuper (readonly)

Returns Lookuper instance for experimentation.

Returns:



74
75
76
 
# File 'lib/kotoshu/dictionary/unified.rb', line 74

def lookuper
  @lookuper
end

#suggester ⇒ Algorithms::Suggest::Suggester (readonly)

Returns Suggester instance for experimentation.

Returns:



77
78
79
 
# File 'lib/kotoshu/dictionary/unified.rb', line 77

def suggester
  @suggester
end

Class Method Details

.from_files(path) ⇒ Dictionary

Load dictionary from file path.

The path should be the base name without extension, e.g., 'en_US' for files 'en_US.aff' and 'en_US.dic'.

Examples:

Dictionary.from_files('en_US')

Parameters:

  • path (String) —

    Base path to dictionary files (without extension)

Returns:

Raises:

  • (ArgumentError) 


108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
 
# File 'lib/kotoshu/dictionary/unified.rb', line 108

def self.from_files(path)
  # Check if it's a distributed dictionary
  if DISTRIBUTED.key?(path) && !File.exist?("#{path}.aff")
    distributed_path = File.join(File.dirname(__FILE__), '../../data', DISTRIBUTED[path], path)
    if File.exist?("#{distributed_path}.aff")
      path = distributed_path
    end
  end

  aff_path = "#{path}.aff"
  dic_path = "#{path}.dic"

  raise ArgumentError, "Dictionary file not found: #{aff_path}" unless File.exist?(aff_path)
  raise ArgumentError, "Dictionary file not found: #{dic_path}" unless File.exist?(dic_path)

  # Read aff file
  aff_reader = Readers::AffReader.new(aff_path)
  aff_data = aff_reader.read

  # Read dic file
  dic_reader = Readers::DicReader.new(dic_path,
                                      flag_format: aff_data['FLAG'] || 'short',
                                      flag_synonyms: aff_data['AF'] || {})
  dic_words = dic_reader.read

  new(aff_data, dic_words)
end

.from_system(name) ⇒ Dictionary

Load dictionary from system paths.

Searches standard system locations for Hunspell dictionaries.

Examples:

Dictionary.from_system('en_US')

Parameters:

  • name (String) —

    Dictionary name (e.g., 'en_US', 'ru_RU')

Returns:

Raises:

  • (ArgumentError) —

    If dictionary not found in system paths



194
195
196
197
198
199
200
201
202
203
204
 
# File 'lib/kotoshu/dictionary/unified.rb', line 194

def self.from_system(name)
  PATHES.each do |folder|
    aff_path = File.join(folder, "#{name}.aff")
    if File.exist?(aff_path)
      base_path = aff_path.sub(/\.aff$/, '')
      return from_files(base_path)
    end
  end

  raise ArgumentError, "#{name}.aff not found in system paths: #{PATHES.inspect}"
end

.from_zip(zip_path) ⇒ Dictionary

Load dictionary from zip archive.

Supports OpenOffice/LibreOffice dictionary extensions (.odt, .oxt) and Firefox/Thunderbird dictionary extensions (.xpi).

Examples:

Dictionary.from_zip('en_US.odt')

Parameters:

  • zip_path (String) —

    Path to zip archive

Returns:



146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
 
# File 'lib/kotoshu/dictionary/unified.rb', line 146

def self.from_zip(zip_path)
  Zip::File.open(zip_path) do |zipfile|
    # Find .aff and .dic files
    aff_entry = nil
    dic_entry = nil

    zipfile.each do |entry|
      if entry.name.end_with?('.aff')
        raise ArgumentError, "Multiple .aff files found in zip" if aff_entry

        aff_entry = entry
      elsif entry.name.end_with?('.dic')
        raise ArgumentError, "Multiple .dic files found in zip" if dic_entry

        dic_entry = entry
      end
    end

    raise ArgumentError, "No .aff file found in zip" unless aff_entry
    raise ArgumentError, "No .dic file found in zip" unless dic_entry

    # Read aff file
    aff_reader = Readers::ZipReader.new(zipfile, aff_entry.name)
    aff_reader.to_a
    # Parse the raw data into proper aff structure
    Readers::AffReader.new(zip_path) # Temporary for context
    aff_data = Readers::AffReader.new(aff_entry.name).read

    # Read dic file
    dic_reader = Readers::DicReader.new(dic_entry.name,
                                        flag_format: aff_data['FLAG'] || 'short',
                                        flag_synonyms: aff_data['AF'] || {})
    dic_words = dic_reader.read

    new(aff_data, dic_words)
  end
end

Instance Method Details

#lookup(word) ⇒ Boolean

Check if a word is correct.

Examples:

dictionary.lookup('spells')  # => true
dictionary.lookup('spylls')  # => false

Parameters:

  • word (String) —

    Word to check

Returns:

  • (Boolean) —

    True if the word exists in the dictionary



214
215
216
 
# File 'lib/kotoshu/dictionary/unified.rb', line 214

def lookup(word)
  @lookuper.call(word)
end

#suggest(word) {|String| ... } ⇒ Enumerator

Generate suggestions for a misspelled word.

Returns suggestions in order of probability/similarity, with best suggestions first.

Examples:

dictionary.suggest('spylls')  # => ["spells", "spills", ...]

Parameters:

  • word (String) —

    The misspelled word

Yields:

  • (String) —

    Each suggestion

Returns:

  • (Enumerator) —

    If no block given



229
230
231
232
233
 
# File 'lib/kotoshu/dictionary/unified.rb', line 229

def suggest(word, &)
  return enum_for(:suggest, word) unless block_given?

  @suggester.suggestions(word, &)
end