Class: Kotoshu::Dictionary::Hunspell

Inherits:
Base
  • Object
show all
Defined in:
lib/kotoshu/dictionary/hunspell.rb

Overview

Hunspell dictionary backend.

This dictionary reads Hunspell-formatted dictionary files (.dic and .aff). Hunspell is the spell checker used by LibreOffice, Firefox, Chrome, and many other applications.

File format:

  • .dic: Dictionary file with word count on first line, words with optional flags
  • .aff: Affix file with prefix/suffix rules and configuration

Examples:

Creating a Hunspell dictionary

dict = Hunspell.new(
  dic_path: "en_US.dic",
  aff_path: "en_US.aff",
  language_code: "en-US"
)
dict.lookup?("hello")  # => true

Creating from GitHub cache

dict = Hunspell.from_github("de")
dict.lookup?("über")  # => true

See Also:

Instance Attribute Summary collapse

Attributes inherited from Base

#language_code, #locale, #metadata

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Base

#<<, #all_words, #contains?, #each_word, #empty?, #has_word?, #include?, load, #lookup?, #path, register_type, registry, #size, #to_s, #type, #words_matching, #words_with_prefix

Constructor Details

#initialize(dic_path:, aff_path:, language_code:, locale: nil, metadata: {}) ⇒ Hunspell

Create a new Hunspell dictionary.

Parameters:

  • dic_path (String) —

    Path or URL to the .dic file

  • aff_path (String) —

    Path or URL to the .aff file

  • language_code (String) —

    The language code

  • locale (String, nil) (defaults to: nil) —

    The locale (optional)

  • metadata (Hash) (defaults to: {}) —

    Additional metadata (optional)

Raises:



139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 139

def initialize(dic_path:, aff_path:, language_code:, locale: nil, metadata: {})
  super(language_code, locale: locale, metadata: metadata)

  @dic_path = resolve_path(dic_path)
  @aff_path = resolve_path(aff_path)

  raise DictionaryNotFoundError, @aff_path unless File.exist?(@aff_path)
  raise DictionaryNotFoundError, @dic_path unless File.exist?(@dic_path)

  # Read aff file using AffReader and cache the data
  aff_reader = Readers::AffReader.new(@aff_path)
  @aff_data = aff_reader.read
  @aff_config = @aff_data # For backward compatibility

  # Read dic file using DicReader with the same encoding as the aff file
  dic_reader = Readers::DicReader.new(@dic_path,
                                      encoding: aff_reader.encoding,
                                      flag_format: @aff_data['FLAG'] || 'short',
                                      flag_synonyms: @aff_data['AF'] || {})
  @dic_words = dic_reader.read

  # Build legacy structures for backward compatibility
  @word_index = build_word_index(@dic_words)
  @affix_rules = parse_affix_rules(@aff_config)

  # Lazy initialization of Lookuper (only created when needed)
  @lookuper = nil

  # Register this dictionary type
  self.class.register_type(:hunspell) unless Dictionary.registry.key?(:hunspell)
end

Instance Attribute Details

#aff_config ⇒ Hash (readonly)

Returns Configuration options from affix file.

Returns:

  • (Hash) —

    Configuration options from affix file



41
42
43
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 41

def aff_config
  @aff_config
end

#aff_data ⇒ Hash (readonly)

Returns Raw aff data from AffReader (cached for Lookuper).

Returns:

  • (Hash) —

    Raw aff data from AffReader (cached for Lookuper)



44
45
46
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 44

def aff_data
  @aff_data
end

#aff_path ⇒ String (readonly)

Returns Path to the .aff file.

Returns:

  • (String) —

    Path to the .aff file



35
36
37
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 35

def aff_path
  @aff_path
end

#affix_rules ⇒ Hash (readonly)

Returns Affix rules (flag => array of rules).

Returns:

  • (Hash) —

    Affix rules (flag => array of rules)



38
39
40
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 38

def affix_rules
  @affix_rules
end

#dic_path ⇒ String (readonly)

Returns Path to the .dic file.

Returns:

  • (String) —

    Path to the .dic file



32
33
34
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 32

def dic_path
  @dic_path
end

#dic_words ⇒ Array (readonly)

Returns Raw words from DicReader (cached for Lookuper).

Returns:

  • (Array) —

    Raw words from DicReader (cached for Lookuper)



47
48
49
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 47

def dic_words
  @dic_words
end

Class Method Details

.available_github_languages(cache: nil) ⇒ Array<String>

Get list of available languages on GitHub.

Parameters:

Returns:

  • (Array<String>) —

    List of supported language codes



116
117
118
119
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 116

def available_github_languages(cache: nil)
  cache ||= Cache::LanguageCache.new
  cache.available_languages
end

.available_on_github?(language_code, cache: nil) ⇒ Boolean

Check if a language is available on GitHub.

Parameters:

  • language_code (String) —

    ISO 639-1 language code

  • cache (Cache::LanguageCache, nil) (defaults to: nil) —

    Custom cache instance (optional)

Returns:

  • (Boolean) —

    True if language is supported



107
108
109
110
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 107

def available_on_github?(language_code, cache: nil)
  cache ||= Cache::LanguageCache.new
  cache.available_languages.include?(language_code)
end

.from_github(language_code, cache: nil, force_download: false) ⇒ Hunspell

Load Hunspell dictionary from GitHub cache, downloading if necessary.

This class method provides automatic dictionary management by:

  1. Checking the local cache for existing dictionaries
  2. Downloading from GitHub if not cached or expired
  3. Managing cache metadata and TTL

Examples:

Load English dictionary

dict = Hunspell.from_github("en")
dict.lookup?("hello")  # => true

Load German dictionary

dict = Hunspell.from_github("de")
dict.lookup?("über")  # => true

Force re-download

dict = Hunspell.from_github("fr", force_download: true)

Parameters:

  • language_code (String) —

    ISO 639-1 language code (e.g., 'en', 'de', 'fr')

  • cache (Cache::LanguageCache, nil) (defaults to: nil) —

    Custom cache instance (optional)

  • force_download (Boolean) (defaults to: false) —

    Force re-download even if cached

Returns:

  • (Hunspell) —

    Configured Hunspell dictionary instance

Raises:

  • (ArgumentError) —

    If language_code is not supported



85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 85

def from_github(language_code, cache: nil, force_download: false)
  cache ||= Cache::LanguageCache.new
  cached = cache.get_dictionary(language_code, force_download: force_download)

  new(
    dic_path: cached[:dic_path],
    aff_path: cached[:aff_path],
    language_code: language_code,
    metadata: {
      source: 'github',
      github_url: cached[:metadata]['url'],
      checksum: cached[:metadata]['checksum'],
      downloaded_at: cached[:metadata]['downloaded_at']
    }
  )
end

.language_info(language_code, cache: nil) ⇒ Hash

Get information about a language from GitHub.

Parameters:

  • language_code (String) —

    ISO 639-1 language code

  • cache (Cache::LanguageCache, nil) (defaults to: nil) —

    Custom cache instance (optional)

Returns:

  • (Hash) —

    Language information



126
127
128
129
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 126

def language_info(language_code, cache: nil)
  cache ||= Cache::LanguageCache.new
  cache.get_language_info(language_code)
end

Instance Method Details

#add_word(word, flags: []) ⇒ Boolean

Add a word to the dictionary.

Parameters:

  • word (String) —

    The word to add

  • flags (Array<String>) (defaults to: []) —

    Morphological flags

Returns:

  • (Boolean) —

    True if added



303
304
305
306
307
308
309
310
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 303

def add_word(word, flags: [])
  return false if word.nil? || word.empty?

  word_key = word.downcase
  @word_index[word_key] = flags

  true
end

#lookup(word) ⇒ Boolean

Check if a word exists in the dictionary.

Uses the Lookup::Lookuper algorithm for full affix and compound support.

Parameters:

  • word (String) —

    The word to look up

Returns:

  • (Boolean) —

    True if the word exists



277
278
279
280
281
282
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 277

def lookup(word)
  return false if word.nil? || word.empty?

  # Use the Lookuper for full Hunspell algorithm support
  lookuper.call(word)
end

#lookuper ⇒ Algorithms::Lookup::Lookuper

Returns The lookup algorithm instance.

Returns:



50
51
52
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 50

def lookuper
  @lookuper ||= Readers::LookupBuilder.from_data(@aff_data, @dic_words).build
end

#remove_word(word) ⇒ Boolean

Remove a word from the dictionary.

Parameters:

  • word (String) —

    The word to remove

Returns:

  • (Boolean) —

    True if removed



316
317
318
319
320
321
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 316

def remove_word(word)
  return false if word.nil? || word.empty?

  word_key = word.downcase
  !@word_index.delete(word_key).nil?
end

#suggest(word, max_suggestions: 10) ⇒ Array<String>

Generate spelling suggestions.

Uses Algorithms::Suggest::Suggester for full Hunspell-compatible suggestion generation (edits, REP, MAP, KEY, TRY, ngram, phonetic).

Parameters:

  • word (String) —

    The misspelled word

  • max_suggestions (Integer) (defaults to: 10) —

    Maximum suggestions

Returns:

  • (Array<String>) —

    List of suggested words



292
293
294
295
296
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 292

def suggest(word, max_suggestions: 10)
  return [] if word.nil? || word.empty?

  suggester.call(word).first(max_suggestions)
end

#suggester ⇒ Algorithms::Suggest::Suggester

Returns The suggestion algorithm instance.

Returns:



55
56
57
58
59
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 55

def suggester
  @suggester ||= Algorithms::Suggest::Suggester.new(
    lookuper.aff, lookuper.dic, lookuper
  )
end

#word_variants(word) ⇒ Array<String>

Get word variants using affix rules.

Parameters:

  • word (String) —

    The word

Returns:

  • (Array<String>) —

    Word variants



334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 334

def word_variants(word)
  return [] if word.nil? || word.empty?

  variants = []

  # Get flags for this word (if any)
  word_key = word.downcase
  flags = @word_index[word_key] || []

  # Generate prefix variants
  @affix_rules[:prefix].each do |flag, rules|
    next unless flags.include?(flag)

    rules.each do |rule|
      variant = rule.apply(word)
      variants << variant if variant
    end
  end

  # Generate suffix variants
  @affix_rules[:suffix].each do |flag, rules|
    next unless flags.include?(flag)

    rules.each do |rule|
      variant = rule.apply(word)
      variants << variant if variant
    end
  end

  variants
end

#words ⇒ Array<String>

Get all words in the dictionary.

Returns:

  • (Array<String>) —

    All words



326
327
328
 
# File 'lib/kotoshu/dictionary/hunspell.rb', line 326

def words
  @word_index.keys.dup
end