Class: Factorix::Cache::FileSystem

Inherits:
Base
  • Object
show all
Defined in:
lib/factorix/cache/file_system.rb

Overview

File system based cache storage implementation.

Uses a two-level directory structure to store cached files, with file locking to handle concurrent access and TTL support for cache expiration.

Cache entries consist of:

  • Data file: the cached content (optionally compressed)

  • Metadata file (.metadata): JSON containing the logical key

  • Lock file (.lock): used for concurrent access control

Constant Summary collapse

LOCK_FILE_LIFETIME =

Maximum lifetime of lock files in seconds. Lock files older than this will be considered stale and removed

3600

Instance Attribute Summary

Attributes inherited from Base

#ttl

Instance Method Summary collapse

Constructor Details

#initialize(cache_type:, max_file_size: nil, compression_threshold: nil) ⇒ FileSystem

Initialize a new file system cache storage. Creates the cache directory if it doesn’t exist. Cache directory is auto-calculated as: factorix_cache_dir / cache_type

Parameters:

  • cache_type (Symbol) —

    cache type for directory name (e.g., :api, :download)

  • max_file_size (Integer, nil) (defaults to: nil) —

    maximum file size in bytes (nil for unlimited)

  • compression_threshold (Integer, nil) (defaults to: nil) —

    compress data larger than this size in bytes (nil: no compression, 0: always compress, N: compress if >= N bytes)

  • ttl (Integer, nil) —

    time-to-live in seconds (nil for unlimited)



46
47
48
49
50
51
52
53
 
# File 'lib/factorix/cache/file_system.rb', line 46

def initialize(cache_type:, max_file_size: nil, compression_threshold: nil, **)
  super(**)
  @cache_dir = Container[:runtime].factorix_cache_dir / cache_type.to_s
  @max_file_size = max_file_size
  @compression_threshold = compression_threshold
  @cache_dir.mkpath
  logger.info("Initializing cache", root: @cache_dir.to_s, ttl: @ttl, max_size: @max_file_size, compression_threshold: @compression_threshold)
end

Instance Method Details

#age(key) ⇒ Float?

Get the age of a cache entry in seconds. Returns nil if the entry doesn’t exist.

Parameters:

  • key (String) —

    logical cache key

Returns:

  • (Float, nil) —

    age in seconds, or nil if entry doesn’t exist



188
189
190
191
192
193
194
 
# File 'lib/factorix/cache/file_system.rb', line 188

def age(key)
  internal_key = storage_key_for(key)
  path = cache_path_for(internal_key)
  return nil unless path.exist?

  Time.now - path.mtime
end

#backend_info ⇒ Hash

Return backend-specific information.

Returns:

  • (Hash) —

    backend configuration and status



289
290
291
292
293
294
295
296
297
 
# File 'lib/factorix/cache/file_system.rb', line 289

def backend_info
  {
    type: "file_system",
    directory: @cache_dir.to_s,
    max_file_size: @max_file_size,
    compression_threshold: @compression_threshold,
    stale_locks: count_stale_locks
  }
end

#clear ⇒ void

This method returns an undefined value.

Clear all cache entries. Removes all files in the cache directory.



170
171
172
173
174
175
176
177
178
179
180
181
 
# File 'lib/factorix/cache/file_system.rb', line 170

def clear
  logger.info("Clearing cache directory", root: @cache_dir.to_s)
  count = 0
  @cache_dir.glob("**/*").each do |path|
    next unless path.file?
    next if path.extname == ".lock"

    path.delete
    count += 1
  end
  logger.info("Cache cleared", files_removed: count)
end

#delete(key) ⇒ Boolean

Delete a specific cache entry.

Parameters:

  • key (String) —

    logical cache key

Returns:

  • (Boolean) —

    true if the entry was deleted, false if it didn’t exist



153
154
155
156
157
158
159
160
161
162
163
164
 
# File 'lib/factorix/cache/file_system.rb', line 153

def delete(key)
  internal_key = storage_key_for(key)
  path = cache_path_for(internal_key)
  metadata_path = metadata_path_for(internal_key)

  return false unless path.exist?

  path.delete
  metadata_path.delete if metadata_path.exist?
  logger.debug("Deleted from cache", key:)
  true
end

#each {|key, entry| ... } ⇒ Enumerator

Enumerate cache entries.

Yields [key, entry] pairs similar to Hash#each. Skips entries without metadata files (legacy entries).

Yields:

  • (key, entry) —

    logical key and Entry object

Yield Parameters:

  • key (String) —

    logical cache key

  • entry (Entry) —

    cache entry metadata

Returns:

  • (Enumerator) —

    if no block given



264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
 
# File 'lib/factorix/cache/file_system.rb', line 264

def each
  return enum_for(__method__) unless block_given?

  @cache_dir.glob("**/*").each do |path|
    next unless path.file?
    next if path.extname == ".metadata" || path.extname == ".lock"

    metadata_path = Pathname("#{path}.metadata")
    next unless metadata_path.exist?

    logical_key = JSON.parse(metadata_path.read)["logical_key"]
    age = Time.now - path.mtime
    entry = Entry[
      size: path.size,
      age:,
      expired: @ttl ? age > @ttl : false
    ]

    yield logical_key, entry
  end
end

#exist?(key) ⇒ Boolean

Check if a cache entry exists and is not expired. A cache entry is considered to exist if its file exists and is not expired

Parameters:

  • key (String) —

    logical cache key

Returns:

  • (Boolean) —

    true if the cache entry exists and is valid, false otherwise



60
61
62
63
64
65
66
 
# File 'lib/factorix/cache/file_system.rb', line 60

def exist?(key)
  internal_key = storage_key_for(key)
  return false unless cache_path_for(internal_key).exist?
  return true if @ttl.nil?

  !expired?(key)
end

#expired?(key) ⇒ Boolean

Check if a cache entry has expired based on TTL. Returns false if TTL is not set (unlimited) or if entry doesn’t exist.

Parameters:

  • key (String) —

    logical cache key

Returns:

  • (Boolean) —

    true if expired, false otherwise



201
202
203
204
205
206
207
208
 
# File 'lib/factorix/cache/file_system.rb', line 201

def expired?(key)
  return false if @ttl.nil?

  age_seconds = age(key)
  return false if age_seconds.nil?

  age_seconds > @ttl
end

#read(key) ⇒ String?

Read a cached file as a binary string. If the cache entry doesn’t exist or is expired, returns nil. Automatically decompresses zlib-compressed cache entries.

Parameters:

  • key (String) —

    logical cache key

Returns:

  • (String, nil) —

    cached content or nil if not found/expired



105
106
107
108
109
110
111
112
113
114
 
# File 'lib/factorix/cache/file_system.rb', line 105

def read(key)
  internal_key = storage_key_for(key)
  path = cache_path_for(internal_key)
  return nil unless path.exist?
  return nil if expired?(key)

  data = path.binread
  data = Zlib.inflate(data) if zlib_compressed?(data)
  data
end

#size(key) ⇒ Integer?

Get the size of a cached file in bytes. Returns nil if the entry doesn’t exist or is expired.

Parameters:

  • key (String) —

    logical cache key

Returns:

  • (Integer, nil) —

    file size in bytes, or nil if entry doesn’t exist/expired



215
216
217
218
219
220
221
222
 
# File 'lib/factorix/cache/file_system.rb', line 215

def size(key)
  internal_key = storage_key_for(key)
  path = cache_path_for(internal_key)
  return nil unless path.exist?
  return nil if expired?(key)

  path.size
end

#store(key, src) ⇒ Boolean

Store a file in the cache. Creates necessary subdirectories and stores the file in the cache. Optionally compresses data based on compression_threshold setting. If the (possibly compressed) size exceeds max_file_size, skips caching and returns false.

Parameters:

  • key (String) —

    logical cache key

  • src (Pathname) —

    path of the file to store

Returns:

  • (Boolean) —

    true if cached successfully, false if skipped due to size limit



124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
 
# File 'lib/factorix/cache/file_system.rb', line 124

def store(key, src)
  data = src.binread
  original_size = data.bytesize

  if should_compress?(original_size)
    data = Zlib.deflate(data)
    logger.debug("Compressed data", original_size:, compressed_size: data.bytesize)
  end

  if @max_file_size && data.bytesize > @max_file_size
    logger.warn("File size exceeds cache limit, skipping", size_bytes: data.bytesize, limit_bytes: @max_file_size)
    return false
  end

  internal_key = storage_key_for(key)
  path = cache_path_for(internal_key)
  metadata_path = metadata_path_for(internal_key)

  path.dirname.mkpath
  path.binwrite(data)
  metadata_path.write(JSON.generate({logical_key: key}))
  logger.debug("Stored in cache", key:, size_bytes: data.bytesize)
  true
end

#with_lock(key) { ... } ⇒ void

This method returns an undefined value.

Executes the given block with a file lock. Uses flock for process-safe file locking and automatically removes stale locks.

Parameters:

  • key (String) —

    logical cache key

Yields:

  • Executes the block with exclusive file lock



230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
 
# File 'lib/factorix/cache/file_system.rb', line 230

def with_lock(key)
  internal_key = storage_key_for(key)
  lock_path = lock_path_for(internal_key)
  cleanup_stale_lock(lock_path)

  lock_path.dirname.mkpath
  lock_path.open(File::RDWR | File::CREAT) do |lock|
    if lock.flock(File::LOCK_EX)
      logger.debug("Acquired lock", key:)
      begin
        yield
      ensure
        lock.flock(File::LOCK_UN)
        logger.debug("Released lock", key:)
        begin
          lock_path.unlink
        rescue => e
          logger.debug("Failed to remove lock file", path: lock_path.to_s, error: e.message)
          nil
        end
      end
    end
  end
end

#write_to(key, output) ⇒ Boolean

Write cached content to a file. If the cache entry doesn’t exist or is expired, returns false without modifying the output path. Automatically decompresses zlib-compressed cache entries.

Parameters:

  • key (String) —

    logical cache key

  • output (Pathname) —

    path to write the cached content to

Returns:

  • (Boolean) —

    true if written successfully, false if not found/expired



75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
 
# File 'lib/factorix/cache/file_system.rb', line 75

def write_to(key, output)
  internal_key = storage_key_for(key)
  path = cache_path_for(internal_key)
  unless path.exist?
    logger.debug("Cache miss", key:)
    return false
  end

  if expired?(key)
    logger.debug("Cache expired", key:, age_seconds: age(key))
    return false
  end

  data = path.binread
  if zlib_compressed?(data)
    data = Zlib.inflate(data)
    output.binwrite(data)
  else
    FileUtils.cp(path, output)
  end
  logger.debug("Cache hit", key:)
  true
end