Class: Store::Digest

Inherits:
Object
  • Object
show all
Defined in:
lib/store/digest.rb,
lib/store/digest/blob.rb,
lib/store/digest/version.rb

Overview

This is a general-purpose content-addressable store that interfaces via RFC6920 addresses.

Since a content-addressable store traffics in immutable blobs of bytes, the main interface is remarkably terse:

  • #add a blob-like object or existing Entry,
  • #get an entry from the store (if it exists), if you know one of its hash URIs,
  • or, #remove it.

Digest scans and stores multiple digest algorithms at once, since clients may only have a hash for a blob in a particular algorithm, and individual algorithms may get compromised from time to time. The set of algorithms is configurable, and fixed for each store instance when it is created.

The currency of Digest, then, is the URI::NI and the Entry. There is also ReadWrapper, a small helper class capable of coercing non-IO-like objects (particulary those which one might find in a Rack message body) into something that behaves enough like an IO blob that it can be scanned. Entry objects also masquerade as blobs with additional metadata.

Defined Under Namespace

Modules: Blob, Driver, Meta, Trait Classes: Entry, Error, ReadWrapper, Stats

Constant Summary collapse

Object = 
Store::Digest::Entry
VERSION = 
"0.4.4".freeze

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(driver: Store::Digest::Driver::LMDB, blocksize: 2**16, mtimes: :preserve, ttl: 60 * 60 * 24, **options) ⇒ void

Note:

See individual drivers for driver-specific options.

Initialize a content-addressable store.

Parameters:

  • driver (Module, Symbol, #to_sym) (defaults to: Store::Digest::Driver::LMDB) —

    the driver to use

  • blocksize (Integer) (defaults to: 2**16) —

    the default block size for scanning blobs

  • mtimes (:preserve, :older, :newer) (defaults to: :preserve) —

    modification time overwrite policy

Raises:

  • (ArgumentError)

See Also:



253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
 
# File 'lib/store/digest.rb', line 253

def initialize driver: Store::Digest::Driver::LMDB,
    blocksize: 2**16, mtimes: :preserve, ttl: 60 * 60 * 24, **options
  driver ||= Store::Digest::Driver::LMDB

  @blocksize = blocksize
  @mtimes    = mtimes || :preserve
  @cache_ttl = ttl

  unless driver.is_a? Module
    # coerce to symbol
    driver = driver.to_s.to_sym
    raise ArgumentError,
      "There is no storage driver Store::Digest::Driver::#{driver}" unless
      Store::Digest::Driver.const_defined? driver
    driver = Store::Digest::Driver.const_get driver
  end

  raise ArgumentError,
    "Driver #{driver} is not a Store::Digest::Driver" unless
    driver.ancestors.include? Store::Digest::Driver

  # bolt the driver onto the instance
  extend driver

  # aaaand bootstrap it
  setup(**options)

  # warn @lmdb.info
end

Instance Attribute Details

#blocksize ⇒ Object (readonly)

Returns the value of attribute blocksize.



283
284
285
 
# File 'lib/store/digest.rb', line 283

def blocksize
  @blocksize
end

#cache_ttl ⇒ Object (readonly)

Returns the value of attribute cache_ttl.



283
284
285
 
# File 'lib/store/digest.rb', line 283

def cache_ttl
  @cache_ttl
end

#mtimes ⇒ Object (readonly)

Returns the value of attribute mtimes.



283
284
285
 
# File 'lib/store/digest.rb', line 283

def mtimes
  @mtimes
end

Instance Method Details

#add(obj, digests: nil, mtime: nil, type: nil, charset: nil, encoding: nil, language: nil, cache: false, scan: false) ⇒ Store::Digest::Entry

Note:

Already-scanned Entry instances will have to be rescanned, since the store can't trust the digests. Use #add or Store::Digest::Entry#add_to on an unscanned entry to scan only once.

Note:

:preserve will cause a noop if object metadata is identical save for :ctime and :mtime (:ctime is always ignored).

Add an object to the store. Will accept pretty much anything that makes sense to throw at it.

Parameters:

  • obj (IO, File, Pathname, String, Store::Digest::Entry) —

    the object

  • type (String) (defaults to: nil) —

    the content type

  • charset (String) (defaults to: nil) —

    the character set, if applicable

  • language (String) (defaults to: nil) —

    the language, if applicable

  • encoding (String) (defaults to: nil) —

    the encoding (eg compression) if applicable

  • mtime (Time) (defaults to: nil) —

    the modification time, if not "now"

  • cache (false, true, Numeric, Time) (defaults to: false) —

    whether the object should be treated as cache, and/or when to evict it

  • scan (false, true) (defaults to: false) —

    eagerly scan the contents

Returns:

Raises:

  • (ArgumentError) 


316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
 
# File 'lib/store/digest.rb', line 316

def add obj, digests: nil, mtime: nil, type: nil, charset: nil,
    encoding: nil, language: nil, cache: false, scan: false

  # warn "hmmmm #{obj.inspect}"

  # XXX this circumvents the integrity check
  return obj.add(self) if obj.is_a? Store::Digest::Entry

  raise ArgumentError, 'entry can\'t be nil' if obj.nil?

  # turducken-ass call graph lol
  Store::Digest::Entry.new obj, store: self, digests: digests, mtime: mtime,
    type: type, charset: charset, encoding: encoding, language: language,
    cache: cache, scan: scan
end

#can_cache? ⇒ false, true

Determine if the store is cache-aware.

Returns:

  • (false, true) 


411
412
413
 
# File 'lib/store/digest.rb', line 411

def can_cache?
  respond_to? :cache_ttl
end

#close ⇒ Object 



403
404
405
 
# File 'lib/store/digest.rb', line 403

def close
  close_internal
end

#forget(obj) ⇒ Object

Remove an object from the store and "forget" it ever existed, i.e., purge it from the metadata.



399
400
401
 
# File 'lib/store/digest.rb', line 399

def forget obj
  remove obj, forget: true
end

#get(obj, tombstone: false) ⇒ Store::Digest::Entry?

Note:

I'm not sure why you would want to #get an entry that you already had, but you can.

Retrieve an entry from the store.

Parameters:

  • obj (URI::NI, Array<URI::NI>, Hash{Symbol=>URI::NI}, Store::Digest::Entry) —

    some means of resolving an entry

Returns:



368
369
370
371
372
373
374
 
# File 'lib/store/digest.rb', line 368

def get obj, tombstone: false
  uri = coerce_uri obj

  if hash = get_raw(uri, tombstone: tombstone)
    Store::Digest::Entry.new(store: self) { hash }
  end
end

#has?(entry, tombstone: false) ⇒ false, true

Returns true if the entry is in the store.

Parameters:

  • entry (URI::NI, Store::Digest::Entry) —

    the hash address of an entry, or an entry object itself

  • tombstone (false, true) (defaults to: false) —

    whether to return "tombstone" metadata records of deleted entries

Returns:

  • (false, true) —

    whether the entry (or its tombstone) is present in the store



342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
 
# File 'lib/store/digest.rb', line 342

def has? entry, tombstone: false
  # coerce just because
  tombstone = !!tombstone

  transaction readonly: true do
    # obviously false if there's no record
    if h = get_meta(entry)
      # a metadata record is considered a tombstone if it has a dtime
      # at all if it's an ordinary entry, and in the past if it's cache
      tombstone || !deleted?(h)
    else
      false
    end
  end
end

#remove(obj, tombstone: false, forget: false) ⇒ Store::Digest::Entry?

Remove an object from the store, optionally "forgetting" it ever existed.

Parameters:

  • entry (URI::NI, Store::Digest::Entry) —

    the hash address of an entry, or an entry object itself

  • tombstone (false, true) (defaults to: false) —

    whether to return "tombstone" metadata records of deleted entries

  • forget (false, true) (defaults to: false) —

    whether to delete the metadata or just mark it as deleted

Returns:



387
388
389
390
391
392
393
394
 
# File 'lib/store/digest.rb', line 387

def remove obj, tombstone: false, forget: false
  uri = coerce_uri obj
  rm  = forget ? :forget : true

  if hash = get_raw(uri, tombstone: tombstone, remove: rm)
    Store::Digest::Entry.new { hash }
  end
end

#stats ⇒ Object

Return statistics on the store



416
417
418
 
# File 'lib/store/digest.rb', line 416

def stats
  Stats.new(**meta_get_stats)
end