Class: Kotoshu::Cache::BaseCache Abstract

Inherits:
Object
  • Object
show all
Defined in:
lib/kotoshu/cache/base_cache.rb

Overview

This class is abstract.

Subclass must implement #download_resource, #load_cached

Abstract base class for all cache implementations.

Provides common functionality for:

  • HTTP downloads with metadata
  • Cache validation (exists, expired)
  • Statistics tracking (hits, misses, hit rate)
  • TTL management

Subclasses implement specific download and loading logic.

Direct Known Subclasses

FrequencyCache, LanguageCache, ModelCache

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(cache_path: nil, url_base: nil, cache_ttl: nil, github_url: nil, resource_pin: nil, manifest_url: nil, audit_log: nil, source_registry: nil, max_cache_size: nil) ⇒ BaseCache

Create a new cache.

Parameters:

  • cache_path (String) (defaults to: nil) —

    Path to cache directory

  • url_base (String) (defaults to: nil) —

    Base URL for downloads (deprecated; pass source_registry instead)

  • cache_ttl (Integer) (defaults to: nil) —

    Cache TTL in seconds

  • github_url (String) (defaults to: nil) —

    GitHub repository URL

  • resource_pin (String) (defaults to: nil) —

    Branch/tag/commit for URL templates (deprecated; use source_registry)

  • manifest_url (String, nil) (defaults to: nil) —

    Override manifest.json URL

  • audit_log (Integrity::AuditLog, nil) (defaults to: nil) —

    Override audit log

  • source_registry (Kotoshu::SourceRegistry, nil) (defaults to: nil) —

    Single source of truth for URLs/pins

  • max_cache_size (Integer, nil) (defaults to: nil) —

    Maximum cache size in bytes (default 1 GB)



58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/kotoshu/cache/base_cache.rb', line 58

def initialize(cache_path: nil, url_base: nil, cache_ttl: nil, github_url: nil,
               resource_pin: nil, manifest_url: nil, audit_log: nil,
               source_registry: nil, max_cache_size: nil)
  @cache_path = cache_path || default_cache_path
  @source_registry = source_registry || default_source_registry
  @url_base = url_base || @source_registry.base_url
  @cache_ttl = cache_ttl || default_cache_ttl
  @github_url = github_url || default_github_url
  @resource_pin = resource_pin || @source_registry.pin_for_source(:spelling)
  @manifest_url = manifest_url
  @audit_log = audit_log || default_audit_log
  @max_cache_size = max_cache_size || default_max_cache_size
  @manifest = nil
  @manifest_loaded = false
  @hits = 0
  @misses = 0

  # Ensure cache directory exists
  FileUtils.mkdir_p(@cache_path)
  FileUtils.mkdir_p(File.join(@cache_path, "tmp"))
end

Instance Attribute Details

#cache_path ⇒ String (readonly)

Returns Path to the cache directory.

Returns:

  • (String) —

    Path to the cache directory



25
26
27
 
# File 'lib/kotoshu/cache/base_cache.rb', line 25

def cache_path
  @cache_path
end

#cache_ttl ⇒ Integer (readonly)

Returns Cache TTL in seconds.

Returns:

  • (Integer) —

    Cache TTL in seconds



31
32
33
 
# File 'lib/kotoshu/cache/base_cache.rb', line 31

def cache_ttl
  @cache_ttl
end

#github_url ⇒ String (readonly)

Returns GitHub repository URL.

Returns:

  • (String) —

    GitHub repository URL



40
41
42
 
# File 'lib/kotoshu/cache/base_cache.rb', line 40

def github_url
  @github_url
end

#max_cache_size ⇒ Integer (readonly)

Returns Maximum total cache size in bytes. Enforced by #evict; the value is pulled from Kotoshu::Configuration.instance.max_cache_size by default (env: KOTOSHU_MAX_CACHE_SIZE).

Returns:

  • (Integer) —

    Maximum total cache size in bytes. Enforced by #evict; the value is pulled from Kotoshu::Configuration.instance.max_cache_size by default (env: KOTOSHU_MAX_CACHE_SIZE).



37
38
39
 
# File 'lib/kotoshu/cache/base_cache.rb', line 37

def max_cache_size
  @max_cache_size
end

#source_registry ⇒ Kotoshu::SourceRegistry (readonly)

Returns Single source of truth for per-repo URLs and pins. Subclasses MUST build URLs through this registry rather than constructing URL strings inline.

Returns:

  • (Kotoshu::SourceRegistry) —

    Single source of truth for per-repo URLs and pins. Subclasses MUST build URLs through this registry rather than constructing URL strings inline.



45
46
47
 
# File 'lib/kotoshu/cache/base_cache.rb', line 45

def source_registry
  @source_registry
end

#url_base ⇒ String (readonly)

Returns Base URL for downloading resources.

Returns:

  • (String) —

    Base URL for downloading resources



28
29
30
 
# File 'lib/kotoshu/cache/base_cache.rb', line 28

def url_base
  @url_base
end

Instance Method Details

#available?(resource_id) ⇒ Boolean

Check if a resource is available in cache.

Parameters:

  • resource_id (String) —

    The resource identifier (e.g., language code)

Returns:

  • (Boolean) —

    True if resource is cached and valid



84
85
86
87
88
89
90
91
92
 
# File 'lib/kotoshu/cache/base_cache.rb', line 84

def available?(resource_id)
  return false unless supports_resource?(resource_id)

  metadata_path = metadata_path_for(resource_id)
  return false unless File.exist?(metadata_path)
  return false if expired?(metadata_path)

  resource_files_exist?(resource_id)
end

#cached_resources ⇒ Array<String>

List all cached resources.

Returns:

  • (Array<String>) —

    List of cached resource identifiers

Raises:

  • (NotImplementedError) 


207
208
209
 
# File 'lib/kotoshu/cache/base_cache.rb', line 207

def cached_resources
  raise NotImplementedError, "Subclass must implement"
end

#clean ⇒ Hash

Clean expired cache entries.

Returns:

  • (Hash) —

    Cleanup statistics



170
171
172
173
174
175
176
177
178
 
# File 'lib/kotoshu/cache/base_cache.rb', line 170

def clean
  expired_count = clean_expired
  size_reclaimed = clean_by_size

  {
    expired_entries_removed: expired_count,
    bytes_reclaimed: size_reclaimed
  }
end

#clear(resource_id) ⇒ Boolean

Clear a specific resource from cache.

Parameters:

  • resource_id (String) —

    The resource identifier

Returns:

  • (Boolean) —

    True if cache was cleared



117
118
119
120
121
122
123
124
125
126
127
 
# File 'lib/kotoshu/cache/base_cache.rb', line 117

def clear(resource_id)
  return false unless supports_resource?(resource_id)

  resource_dir = resource_dir_for(resource_id)
  if File.exist?(resource_dir)
    FileUtils.rm_rf(resource_dir)
    return true
  end

  false
end

#clear_all ⇒ void

This method returns an undefined value.

Clear all cached resources.



132
133
134
135
136
137
138
 
# File 'lib/kotoshu/cache/base_cache.rb', line 132

def clear_all
  @hits = 0
  @misses = 0
  FileUtils.rm_rf(@cache_path)
  FileUtils.mkdir_p(@cache_path)
  FileUtils.mkdir_p(File.join(@cache_path, "tmp"))
end

#download(resource_id) ⇒ Object?

Download a resource from GitHub.

Parameters:

  • resource_id (String) —

    The resource identifier

Returns:

  • (Object, nil) —

    Downloaded resource or nil on error



223
224
225
226
227
228
229
230
231
232
233
234
235
 
# File 'lib/kotoshu/cache/base_cache.rb', line 223

def download(resource_id)
  return nil unless supports_resource?(resource_id)

  resource_dir = resource_dir_for(resource_id)
  FileUtils.mkdir_p(resource_dir)

  begin
    download_resource(resource_id, resource_dir)
  rescue StandardError => e
    warn "Error downloading #{resource_id}: #{e.message}" if $VERBOSE
    nil
  end
end

#download_resource(resource_id, dest_path) ⇒ Object

This method is abstract.

Subclass must implement

Abstract: Download a specific resource.

Parameters:

  • resource_id (String) —

    The resource identifier

  • dest_path (String) —

    Destination directory

Returns:

  • (Object) —

    Downloaded resource

Raises:

  • (NotImplementedError) 


243
244
245
 
# File 'lib/kotoshu/cache/base_cache.rb', line 243

def download_resource(resource_id, dest_path)
  raise NotImplementedError, "Subclass must implement"
end

#evict(dry_run: false) ⇒ Hash

Enforce the cache size cap by evicting the oldest entries (LRU by cached_at). When dry_run: true, returns the plan without modifying the disk; otherwise removes the evicted entries via FileUtils.rm_rf and returns the same plan.

The plan is computed by EvictionPolicy — a pure value object that performs no IO. This method is the only place that collects on-disk entries and the only place that mutates the cache directory based on the plan, keeping the policy testable in isolation.

Parameters:

  • dry_run (Boolean) (defaults to: false) —

    return the plan without writing

Returns:

  • (Hash) —

    { evict: Array, keep: Array, bytes_reclaimed: Integer } where each entry hash carries :path, :size, :cached_at



195
196
197
198
199
200
201
202
 
# File 'lib/kotoshu/cache/base_cache.rb', line 195

def evict(dry_run: false)
  policy = EvictionPolicy.new(max_size: @max_cache_size)
  plan = policy.plan(collect_eviction_entries)
  return plan if dry_run

  plan[:evict].each { |e| FileUtils.rm_rf(e[:path]) }
  plan
end

#get(resource_id, force_download: false) ⇒ Object?

Get a resource from cache or download it.

Parameters:

  • resource_id (String) —

    The resource identifier

  • force_download (Boolean) (defaults to: false) —

    Force re-download even if cached

Returns:

  • (Object, nil) —

    The cached resource or nil if not available



99
100
101
102
103
104
105
106
107
108
109
110
111
 
# File 'lib/kotoshu/cache/base_cache.rb', line 99

def get(resource_id, force_download: false)
  return nil unless supports_resource?(resource_id)

  metadata_path = metadata_path_for(resource_id)

  if !force_download && cached?(metadata_path) && !expired?(metadata_path)
    @hits += 1
    return load_cached(resource_id)
  end

  @misses += 1
  download(resource_id)
end

#load_cached(resource_id) ⇒ Object?

This method is abstract.

Subclass must implement

Abstract: Load cached resource data.

Parameters:

  • resource_id (String) —

    The resource identifier

Returns:

  • (Object, nil) —

    Loaded resource or nil

Raises:

  • (NotImplementedError) 


252
253
254
 
# File 'lib/kotoshu/cache/base_cache.rb', line 252

def load_cached(resource_id)
  raise NotImplementedError, "Subclass must implement"
end

#metadata_path_for(resource_id) ⇒ String

This method is abstract.

Subclass must implement

Abstract: Get metadata file path for a resource.

Parameters:

  • resource_id (String) —

    The resource identifier

Returns:

  • (String) —

    Metadata file path

Raises:

  • (NotImplementedError) 


261
262
263
 
# File 'lib/kotoshu/cache/base_cache.rb', line 261

def metadata_path_for(resource_id)
  raise NotImplementedError, "Subclass must implement"
end

#read_metadata(path) ⇒ Hash?

Read metadata from file.

Public because the CLI cache subcommand and other inspection paths consume it to display cache state without going through the full download-and-load pipeline.

Parameters:

  • path (String) —

    Metadata file path

Returns:

  • (Hash, nil) —

    Metadata or nil



679
680
681
682
683
684
685
 
# File 'lib/kotoshu/cache/base_cache.rb', line 679

def read_metadata(path)
  return nil unless File.exist?(path)

  JSON.parse(File.read(path))
rescue JSON::ParserError
  nil
end

#reset_stats ⇒ self

Reset statistics counters.

Returns:

  • (self) —

    Self for chaining



161
162
163
164
165
 
# File 'lib/kotoshu/cache/base_cache.rb', line 161

def reset_stats
  @hits = 0
  @misses = 0
  self
end

#resource_dir_for(resource_id) ⇒ String

This method is abstract.

Subclass must implement

Abstract: Get resource directory path.

Parameters:

  • resource_id (String) —

    The resource identifier

Returns:

  • (String) —

    Resource directory path

Raises:

  • (NotImplementedError) 


270
271
272
 
# File 'lib/kotoshu/cache/base_cache.rb', line 270

def resource_dir_for(resource_id)
  raise NotImplementedError, "Subclass must implement"
end

#resource_files_exist?(resource_id) ⇒ Boolean

This method is abstract.

Subclass must implement

Abstract: Check if all resource files exist.

Parameters:

  • resource_id (String) —

    The resource identifier

Returns:

  • (Boolean) —

    True if all files exist

Raises:

  • (NotImplementedError) 


279
280
281
 
# File 'lib/kotoshu/cache/base_cache.rb', line 279

def resource_files_exist?(resource_id)
  raise NotImplementedError, "Subclass must implement"
end

#stats ⇒ Hash

Get cache statistics.

Returns:

  • (Hash) —

    Statistics including :hits, :misses, :hit_rate, :size



143
144
145
146
147
148
149
150
151
152
153
154
155
156
 
# File 'lib/kotoshu/cache/base_cache.rb', line 143

def stats
  total = @hits + @misses
  hit_rate = total.positive? ? (@hits.to_f / total) : 0.0

  {
    hits: @hits,
    misses: @misses,
    total: total,
    hit_rate: hit_rate,
    cached_resources: cached_resources,
    size_bytes: cache_size,
    oldest_entry: oldest_entry
  }
end

#supports_resource?(resource_id) ⇒ Boolean

Check if a resource type is supported.

Parameters:

  • resource_id (String) —

    The resource identifier

Returns:

  • (Boolean) —

    True if supported

Raises:

  • (NotImplementedError) 


215
216
217
 
# File 'lib/kotoshu/cache/base_cache.rb', line 215

def supports_resource?(resource_id)
  raise NotImplementedError, "Subclass must implement"
end