File: README — Documentation by YARD 0.9.38

License Build Status RubyGems Version

Purpose

Lutaml::Store provides a store-centric database-style API for Lutaml::Model objects with model registry, polymorphic support, composite relationships, and multiple storage backends.

It offers a unified interface for storing and retrieving complex model hierarchies, batch file I/O with multiple serialization formats, HTTP-aware caching, and package-based persistence with ZIP and directory transports.

Features

DatabaseStore — high-level CRUD with model registry, polymorphism, composites
PackageStore — structured multi-model packages with ZIP and directory transport
BasicStore — low-level key-value store with optional cache, events, monitoring
CacheStore — TTL-aware cache with LRU eviction extending BasicStore
HttpCache — HTTP-aware caching with ETags, conditional requests, Cache-Control
Format-aware file I/O — YAML, YAMLS, JSON, JSONL, Marshal with layout strategies
Multiple backends — Memory, FileSystem, SQLite (all thread-safe)
Model registry — configurable key fields, polymorphic inheritance, composite relationships
Dot-notation updates — nested attribute paths with block-based updates
Ruby autoload — lazy constant loading, only loads what you use

Installation

Add this line to your application’s Gemfile:

gem 'lutaml-store'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install lutaml-store

For SQLite backend support, also add:

gem 'sqlite3'

Quick start

require 'lutaml/model'
require 'lutaml/store'

# Define your models
class Studio < Lutaml::Model::Serializable
  attribute :studio_key, :string
  attribute :name, :string
  attribute :location, :string
end

class PotteryClass < Lutaml::Model::Serializable
  attribute :studio, Studio
  attribute :class_id, :string
  attribute :description, :string
end

# Create a store with model registry
store = Lutaml::Store.new(
  adapter: :memory,
  models: [
    { model: PotteryClass, key: :class_id },
    { model: Studio, key: :studio_key }
  ]
)

# Save with composite relationships (both stored independently)
pottery = PotteryClass.new(
  class_id: "pottery_101",
  studio: Studio.new(studio_key: "main_studio", name: "Main Studio"),
  description: "Beginner pottery class"
)
store.save(pottery)

# Fetch by model and key
retrieved = store.fetch(model: PotteryClass, class_id: "pottery_101")
puts retrieved.studio.name # => "Main Studio"

# Nested update with dot notation
store.update(
  model: PotteryClass,
  class_id: "pottery_101",
  attributes: [
    { key: :description, value: "Advanced pottery class" },
    { key: "studio.location", value: "Building A" }
  ]
)

Architecture

Lutaml::Store has two layers:

DatabaseStore (via Lutaml::Store.new): High-level CRUD with model registry. Handles polymorphic dispatch, composite model decomposition, dot-notation updates, file I/O (save_all, load_all, import_all, export).
BasicStore: Low-level key-value store wrapping an adapter. Provides get, set, delete, exists?, all, keys, bulk operations, with optional caching, monitoring, and event emission.

Key classes

Class Role

Class	Role
`DatabaseStore`	High-level CRUD with model registry, composites, polymorphism
`PackageStore`	Multi-model packages with directory/ZIP transport
`PackageDefinition`	Declarative schema for package structure (models, assets, metadata)
`BasicStore`	Low-level key-value store with optional cache/events/monitoring
`CacheStore`	TTL-aware cache store extending BasicStore
`HttpCache`	HTTP-aware caching with ETags, conditional requests, Cache-Control
`ModelRegistry` / `ModelRegistration`	Register models with key fields and polymorphic config
`CompositeModelHandler`	Stores nested registered models independently, restores references
`AttributeUpdater`	Processes dot-notation paths and block-based updates
`ModelSerializer`	Serialization/deserialization with custom serializer support
`Format`	Multi-format file I/O (YAML, JSON, JSONL, Marshal) with layout strategies
`Config`	Parses and validates store configuration

DatabaseStore

High-level CRUD with model registry, composites, polymorphism

PackageStore

Multi-model packages with directory/ZIP transport

PackageDefinition

Declarative schema for package structure (models, assets, metadata)

BasicStore

Low-level key-value store with optional cache/events/monitoring

CacheStore

TTL-aware cache store extending BasicStore

HttpCache

HTTP-aware caching with ETags, conditional requests, Cache-Control

ModelRegistry / ModelRegistration

CompositeModelHandler

Stores nested registered models independently, restores references

AttributeUpdater

Processes dot-notation paths and block-based updates

ModelSerializer

Serialization/deserialization with custom serializer support

Format

Multi-format file I/O (YAML, JSON, JSONL, Marshal) with layout strategies

Config

Parses and validates store configuration

Storage adapters

All adapters inherit from Adapter::Base and provide get, set, delete, exists?, keys, all, clear, size, each_key, and bulk operations.

Adapter Type symbol Use case

Adapter	Type symbol	Use case
`Adapter::Memory`	`:memory`	Fast in-memory storage for testing, caching, temporary data
`Adapter::FileSystem`	`:filesystem`	Persistent file-based storage with integrity checks
`Adapter::Sqlite`	`:sqlite`	ACID-compliant database storage for production use

Adapter::Memory

:memory

Fast in-memory storage for testing, caching, temporary data

Adapter::FileSystem

:filesystem

Persistent file-based storage with integrity checks

Adapter::Sqlite

:sqlite

ACID-compliant database storage for production use

Model registry

Registration

store = Lutaml::Store.new(
  adapter: :memory,
  models: [
    { model: User, key: :user_id },
    { model: Post, key: :post_id }
  ]
)

Polymorphic models

For inheritance hierarchies, register the base class with a polymorphic class key:

class Studio < Lutaml::Model::Serializable
  attribute :studio_key, :string
  attribute :name, :string
  attribute :_class, :string, default: -> { "Studio" }, polymorphic_class: true
end

class CeramicStudio < Studio
  attribute :clay_type, :string
  attribute :_class, :string, default: -> { "CeramicStudio" }
end

store = Lutaml::Store.new(
  adapter: :memory,
  models: [
    { model: Studio, key: :studio_key, polymorphic_class_key: :_class }
  ]
)

store.save(CeramicStudio.new(studio_key: "cs1", name: "Clay Haus", clay_type: "Porcelain"))
retrieved = store.fetch(model: Studio, studio_key: "cs1")
puts retrieved.class.name # => "CeramicStudio"

Composite models

When registered models are nested within other registered models, they are stored independently while maintaining references:

store = Lutaml::Store.new(
  adapter: :memory,
  models: [
    { model: PotteryClass, key: :class_id },
    { model: Studio, key: :studio_key }
  ]
)

pottery = PotteryClass.new(
  class_id: "p101",
  studio: Studio.new(studio_key: "s1", name: "Main Studio")
)
store.save(pottery)

# Both accessible independently
store.fetch(model: Studio, studio_key: "s1").name  # => "Main Studio"
store.fetch(model: PotteryClass, class_id: "p101").studio.name  # => "Main Studio"

CRUD operations

Save

store.save(User.new(user_id: "u1", name: "Ada"))
store.save([user1, user2, user3])  # bulk save

Fetch

user = store.fetch(model: User, user_id: "u1")

Update

# Hash-based attributes
store.update(
  model: User, user_id: "u1",
  attributes: [
    { key: :name, value: "Grace" },
    { key: "studio.location", value: "Building A" }  # dot notation for nesting
  ]
)

# Block-based
store.update(model: User, user_id: "u1") do |user|
  user.name = "Grace"
end

Destroy

store.destroy(model: User, user_id: "u1")

PackageStore

PackageStore provides structured multi-model persistence with directory and ZIP transports. It is built on PackageDefinition, which declares the package schema declaratively.

Define a package

glossary = Lutaml::Store::PackageDefinition.new(name: "glossary") do |pkg|
  pkg.model(model: Concept, key: :term, dir: "concepts", default_format: :yaml)
  pkg.model(model: Author, key: :name, dir: "authors", default_format: :json)
  pkg.asset("glossary.yaml", type: :file)
  pkg.metadata_model = GlossaryInfo
  pkg.metadata_file = "glossary.yaml"
end

Load and save packages

# Load from directory
store = Lutaml::Store::PackageStore.load(glossary, "./my_glossary", transport: :directory)

# Load from ZIP
store = Lutaml::Store::PackageStore.load(glossary, "./glossary.zip", transport: :zip)

# Query models
concepts = store.models_for(Concept)
store.model_count(Concept)  # => 42
store.fetch_model(Concept, "API")

# Modify and save
store.add_model(Concept.new(term: "REST", definition: "..."))
store.save("./output", transport: :zip)

Package transports

Transport Symbol Description

Transport	Symbol	Description
`DirectoryTransport`	`:directory`	Filesystem directory with subdirectories per model type
`ZipTransport`	`:zip`	ZIP archive containing the same directory structure

DirectoryTransport

:directory

Filesystem directory with subdirectories per model type

ZipTransport

:zip

ZIP archive containing the same directory structure

Transports are resolved via registry (PackageTransport.resolve(:zip)), extensible without modifying existing code.

File I/O

DatabaseStore provides batch file I/O through Format handlers.

Format handlers

Format Symbol Extension Description

Format	Symbol	Extension	Description
YAML	`:yaml`	`.yaml`	Single-document YAML files
YAMLS	`:yamls`	`.yaml`	Multi-document YAML streams
JSON	`:json`	`.json`	Single JSON objects
JSONL	`:jsonl`	`.jsonl`	Line-delimited JSON
Marshal	`:marshal`	`.bin`	Ruby Marshal binary format

YAML

:yaml

.yaml

Single-document YAML files

YAMLS

:yamls

.yaml

Multi-document YAML streams

JSON

:json

.json

Single JSON objects

JSONL

:jsonl

.jsonl

Line-delimited JSON

Marshal

:marshal

.bin

Ruby Marshal binary format

save_all / load_all / import_all / export

# Write models to directory
store.save_all(concepts, path: "./data", format: :yaml, layout: :separate)

# Read models (returns array, does NOT store in backend)
models = store.load_all(Concept, path: "./data", format: :yaml, layout: :separate)

# Read AND store in backend (makes them queryable)
store.import_all(Concept, path: "./data", format: :yaml, layout: :separate)
store.fetch(model: Concept, term: "API")  # now available

# Export to a single file
store.export(all_concepts, path: "output.yaml", format: :yaml)

Layout strategies: :separate (one file per model), :grouped (models grouped by key), :flat (one file per model, no subdirectory).

HTTP caching

HttpCache provides HTTP-aware caching with ETags, conditional requests (304), Cache-Control directives, and Vary header support. It uses a storage adapter internally and serializes cache entries as Lutaml::Model objects via JSON.

cache = Lutaml::Store::HttpCache.new(
  adapter_type: "memory",
  default_ttl: 3600,
  respect_http_headers: true,
  enable_conditional_requests: true
)

# Fetch with automatic caching
response = cache.fetch("GET", "https://api.example.com/resource", {}) do |headers|
  http_client.get("https://api.example.com/resource", headers)
end

# Second call returns cached response (no HTTP request made)
cached = cache.fetch("GET", "https://api.example.com/resource", {}) { raise "shouldn't be called" }

Supports no-store, no-cache, must-revalidate, max-age, ETag-based conditional requests, query parameter normalization, and filesystem/sqlite adapters for persistent cache storage.

CacheStore

CacheStore extends BasicStore with TTL-aware caching and LRU eviction:

cache = Lutaml::Store::CacheStore.new(
  adapter: :memory,
  max_size: 1000,
  default_ttl: 3600
)

cache.set("key1", "value1", ttl: 600)
cache.get("key1")        # => "value1"
cache.fetch("key2", "default_value")  # => "default_value" (stored)
cache.fetch("key3") { expensive_compute }  # computes and caches

Storage backends

Memory

store = Lutaml::Store.new(adapter: :memory, models: [...])

Fast in-memory storage. Data lost on process exit. Thread-safe via mutex.

FileSystem

store = Lutaml::Store.new(
  adapter: { type: :filesystem, path: "./data", extension: ".json" },
  models: [...]
)

Persistent file-based storage with SHA-256 integrity checks. Files organized by key in subdirectories.

SQLite

store = Lutaml::Store.new(
  adapter: { type: :sqlite, path: "./store.db" },
  models: [...]
)

ACID-compliant database storage. Requires sqlite3 gem. Thread-safe with connection pooling.

Event system

store.on(:model_save) { |data| logger.info("Saved #{data[:model].class}") }
store.on(:model_fetch) { |data| logger.debug("Fetched #{data[:key]}") }
store.on(:model_destroy) { |data| audit_log << data }

Events: :model_save, :model_fetch, :model_update, :model_destroy, :model_save_all, :model_import, :model_export, :model_load_error, :composite_model_stored, :polymorphic_model_resolved.

Error handling

Error hierarchy:

Lutaml::Store::Error
- ConfigurationError — invalid store or adapter config
- BackendError — adapter-level failures
- ModelNotRegisteredError — operations on unregistered models
- InvalidKeyError — missing or invalid key fields
- PolymorphicUpdateError — polymorphic type conflicts
- CompositeModelError — composite model handling failures

Thread safety

All adapters use mutex-based synchronization. Safe for concurrent use across threads.

Development

bin/setup             # Install dependencies
bundle exec rake      # Run specs + rubocop
bundle exec rspec     # Run all specs
bundle exec rspec spec/lutaml/store/database_store_spec.rb  # Single spec file
bundle exec rubocop   # Lint

Ruby >= 3.1 required.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/lutaml/lutaml-store.

License

BSD-2-Clause. See the LICENSE file for details.