YamlExporter
Manage ActiveRecord data in YAML files, track it in Git.
YamlExporter serializes a record — and everything it owns — to a human-readable YAML file, and imports the file back in a single transaction. A three-method DSL (attributes, one, many) declares the mapping; yaml_export and yaml_import round-trip the data.
Useful whenever a record is really content: seed data, CMS-style entries, training material, quiz banks, fixtures — anywhere you'd rather review changes in a pull request than in the database.
Installation
Add this line to your application's Gemfile:
gem 'yaml_exporter'
And then execute:
$ bundle install
or install it yourself as:
$ gem install yaml_exporter
Hello Yaml
Imagine you are a bookstore and want to manage your books in Git.
Here is our model:
classDiagram
class Book {
- String title
- String author
- Decimal price
}
Why not create such a file here:
ruby-on-rails.yaml
title: Ruby on Rails Tutorial
author: Michael Hartl
price: 100
Then, you can configure your ActiveRecord model to be able to export and import this data from and to this file:
class Book < ActiveRecord::Base
include YamlExporter
yaml_structure do
attributes :title, :author, :price
end
end
Now you can simply import the data from the file into your ActiveRecord model:
book = Book.new
yaml_string = File.read('ruby-on-rails.yaml')
book.yaml_import(yaml_string)
And of course you can export the data from your ActiveRecord model to the file:
yaml_string = book.yaml_export
Mental model
The whole DSL fits in three methods: attributes, one, and many. Everything else is governed by two orthogonal axes and one lifecycle rule.
A YAML file describes one record and everything it owns. References (via find_by) are how you reach outside that ownership boundary to point at records that live in other trees — other files, or records managed elsewhere in code.
Axis 1 — cardinality: pick the method
| Method | What it describes |
|---|---|
attributes :col1, :col2 |
Plain columns on the current record |
one :relation, … |
Exactly one related record |
many :relations, … |
A list of related records |
Axis 2 — ownership: block vs. find_by
The arguments you pass determine both the YAML shape and who owns the record:
| Pattern | YAML shape | Ownership |
|---|---|---|
| block | hash | You own it. YamlExporter creates, updates, and destroys records to match the YAML. |
find_by: only (no block) |
bare string | Someone else owns it. YamlExporter only resolves the reference — it never creates or destroys referenced records. |
block + find_by: |
hash containing the find_by key |
You own it; identity is a stable column rather than position (for many). |
The combination table
| DSL call | YAML shape |
|---|---|
attributes :title, :price |
title: …, price: … |
one :book_detail do … end |
nested hash |
one :publisher, find_by: :slug |
bare string: publisher: some-slug |
one :responsible_editor, find_by: :slug, of: :user |
bare string: responsible_editor: some-slug (resolved via a nested association on the target) |
many :book_parts do … end |
list of hashes, matched by order |
many :book_parts, find_by: :slug do … end |
list of hashes, matched by slug |
many :book_parts, find_by: :slug, positioned_by: :position do … end |
list of hashes, matched by slug; no position: key — the column is derived from the array index |
many :authors, find_by: :slug |
list of bare strings |
many :reviewers, through: :book_reviewers, find_by: :slug |
list of bare strings (join rows managed for you; add positioned_by: to derive a join column from order) |
many :reviewers, through: :book_reviewers, find_by: :slug do … end |
list of hashes describing join-model attributes |
Lifecycle rules
These apply uniformly to every yaml_import:
- Missing key =
null= clear it. Absent attributes are reset tonil; absent owned records are destroyed. To tell the importer to ignore a field, omit it fromyaml_structure— don't leave it blank in the YAML. - One import = one transaction. If anything raises, the whole import rolls back and the record is left untouched.
- Validation failures raise. All writes go through
save!, soActiveRecord::RecordInvalidaborts the import. Unresolvable references raiseActiveRecord::RecordNotFound. The library deliberately never swallows these.
Working with relations
The examples below walk through each row of the combination table above, using the same bookstore domain.
many with a block (positional)
Your book consists of multiple parts and on the website you want to display a bit more information about each part:
classDiagram
class Book {
- String title
- String author
- Decimal price
}
class BookPart {
- String title
- String content
- Integer position
}
Book "1" -- "*" BookPart
Of course, you could create a separate yaml file for each book part and pass the book id, but actually it would be nicer to configure the parts in the book yaml file:
title: Ruby on Rails Tutorial
author: Michael Hartl
price: 100
book_parts:
- title: Chapter 1
content: |-
This is the first chapter of the book
position: 1
- title: Chapter 2
content: |-
This is the second chapter of the book
position: 2
To do so, configure your book model to have a has_many relation to book parts:
class Book < ActiveRecord::Base
has_many :book_parts, dependent: :destroy
include YamlExporter
yaml_structure do
attributes :title, :author, :price
many :book_parts do
attributes :title, :content, :position
end
end
end
Edge cases:
- If there are missing book parts in the database, they will be created.
- If the database has more book parts than in the yaml file, the extra book parts will be deleted.
- Existing rows are matched by array index — so re-ordering the list silently overwrites each row with another row's data.
Be aware that this way you need to be very careful with the order of the book parts in the yaml file. To avoid this, you can use the find_by option:
many with find_by and a block
Let us now identify the book parts by a stable column, for example, let's add a slug column to the book parts:
classDiagram
class Book {
- String title
- String author
- Decimal price
}
class BookPart {
- String title
- String content
- Integer position
- String slug
}
Book "1" -- "*" BookPart
Now you can configure your book model to identify the book parts by the slug column:
class Book < ActiveRecord::Base
has_many :book_parts, dependent: :destroy
include YamlExporter
yaml_structure do
attributes :title, :author, :price
many :book_parts, find_by: :slug do
attributes :title, :content, :position
end
end
end
The corresponding yaml file would look like this:
title: Ruby on Rails Tutorial
author: Michael Hartl
price: 100
book_parts:
- slug: chapter-1
title: Chapter 1
content: |-
This is the first chapter of the book
position: 1
- slug: chapter-2
title: Chapter 2
content: |-
This is the second chapter of the book
position: 2
Identification: YamlExporter uses book.book_parts.find_by(slug: "chapter-1") to identify each book part. Slugs do not need to be unique across all books; and if multiple book parts for the same book share a slug, only the first is matched and the rest are deleted.
Edge cases:
- Missing book parts in the database will be created.
- Extra book parts in the database (absent from the yaml) will be deleted.
- The order of the book parts in the yaml file doesn't matter for identity — but
position: 1/position: 2is still being managed by hand.
The next section shows how to make the YAML order itself drive the position column.
many with positioned_by: (derived position)
The previous example stores position as a regular column and requires the user to keep the numbers in sync with the list order. That's both tedious and a constant source of merge conflicts.
positioned_by: hands that column to YamlExporter: on import, each entry's position is set from its 1-based array index; on export, the list is sorted by the position column ASC and the column is omitted from the YAML (the order of the list is the position).
class Book < ActiveRecord::Base
has_many :book_parts, dependent: :destroy
include YamlExporter
yaml_structure do
attributes :title, :author, :price
many :book_parts, find_by: :slug, positioned_by: :position do
attributes :title, :content
end
end
end
The YAML gets tidier — no position: keys inside the entries:
title: Ruby on Rails Tutorial
author: Michael Hartl
price: 100
book_parts:
- slug: chapter-1
title: Chapter 1
content: |-
This is the first chapter of the book
- slug: chapter-2
title: Chapter 2
content: |-
This is the second chapter of the book
Reordering is now a single-line move in the YAML file, and the position column in the database tracks it automatically.
Edge cases:
- The
positioned_by:column may not also appear in the block'sattributeslist — the DSL owns it for the owned record. Declaring both raises at load time. - A
position:key (or whatever column you named) inside a YAML entry is rejected on import, for the same reason. - If the database has gaps, duplicates, or
NULLs in the position column from earlier code paths, a full import rewrites them 1..N cleanly. The YAML is always the source of truth. positioned_by:is available wherever there is an owned record to write the column onto — so with or withoutfind_by:, and also in combination withthrough:(where the column lives on the join model, same as other block attributes). It is not available onmany :authors, find_by: :slug(no block), since there is no owned record.
many with find_by and no block (reference list)
Sometimes the children of a has_many are not owned by the parent at all — they are managed elsewhere (in their own yaml files, or by the user directly), and the parent only needs to reference them. The prime example is a many-to-many relation. Say authors are their own records and a book just points at them:
classDiagram
class Book {
- String title
- Decimal price
}
class Author {
- String name
- String slug
}
Book "*" -- "*" Author
Since has_and_belongs_to_many has no join-model attributes to manage, it fits this pattern exactly — so the example below uses HABTM to demonstrate. Drop the block entirely and use find_by to declare the key:
class Book < ActiveRecord::Base
has_and_belongs_to_many :authors
include YamlExporter
yaml_structure do
attributes :title, :price
many :authors, find_by: :slug
end
end
Because the block is missing, each author appears in the YAML as a bare string (the slug), not a hash:
title: Ruby on Rails Tutorial
price: 38
authors:
- michael-hartl
- another-author
This same flavor works for has_and_belongs_to_many (as above) and for any has_many where the parent shouldn't manage the children's attributes.
Identification: YamlExporter uses Author.find_by(slug: "michael-hartl") to resolve each entry.
Edge cases:
- If a referenced author does not exist in the database,
ActiveRecord::RecordNotFoundis raised — the library never auto-creates referenced records. - If the database has more associations than the yaml file, the extra associations are removed — for HABTM only the join rows are removed, the referenced records themselves are left untouched.
- The order of the entries in the yaml file doesn't matter.
many with through: (has_many :through with join attributes)
has_many :through is not a separate DSL method — how you expose it depends on whether the join model carries its own attributes.
If the join has no extra attributes drop the block entirely — many :reviewers, through: :book_reviewers, find_by: :slug is a bare reference list, identical in shape to many :authors, find_by: :slug (a flat list of find_by values). The distinction is the presence of a block, not its contents: omit it for bare strings; pass one (even an empty one) to opt into hash-shaped entries. You don't have to expose the join association separately; YamlExporter still creates and destroys the join rows for you, it just doesn't ask the YAML for any join attributes:
yaml_structure do
attributes :title, :price
many :reviewers, through: :book_reviewers, find_by: :slug
end
title: Ruby on Rails Tutorial
price: 100
reviewers:
- alice
- bob
Add positioned_by: when the join carries an order column but nothing else — the column is derived from the YAML order, so the list stays a flat list of strings:
many :reviewers, through: :book_reviewers, find_by: :slug, positioned_by: :position
If the join carries its own attributes (e.g. finished on a book_reviewers join), declare many on the :through target and pass both through: and find_by:. The block then describes attributes of the join model:
classDiagram
class Book {
- String title
- Decimal price
}
class BookReviewer {
- String slug
- Boolean finished
}
class Reviewer {
- String name
- String slug
}
Book "*" -- "*" BookReviewer
BookReviewer "*" -- "1" Reviewer
class Book < ActiveRecord::Base
has_many :book_reviewers, dependent: :destroy
has_many :reviewers, through: :book_reviewers
include YamlExporter
yaml_structure do
attributes :title, :price
many :reviewers, through: :book_reviewers, find_by: :slug do
attributes :finished # lives on book_reviewers, the join model
end
end
end
title: Ruby on Rails Tutorial
price: 100
reviewers:
- slug: alice
finished: true
- slug: bob
finished: false
On import, each entry is processed like this:
- Find the
Reviewerby slug. (Reviewer.find_by(slug: "alice")) - Find-or-build the
book_reviewerjoin row linking this book and that reviewer — the identity of the join is(book_id, reviewer_id), so no positional matching is needed. (book.book_reviewers.find_or_create_by(reviewer_id: reviewer.id)) - Apply the block's attributes (
finished) to the join row. - Destroy any join rows whose reviewer is no longer listed.
positioned_by: works here too — the column lives on the join model, so e.g. many :reviewers, through: :book_reviewers, find_by: :slug, positioned_by: :position do … end makes the YAML order drive book_reviewers.position.
The alternative — declaring many directly on the join model itself — is also supported and useful when you want to expose the join row as its own first-class concept rather than hiding it behind the target association.
one with a block (owned)
This is the has_one pattern. Suppose every book has exactly one book_detail containing extra information like a summary and publication year:
classDiagram
class Book {
- String title
- String author
- Decimal price
}
class BookDetail {
- String summary
- Integer publication_year
}
Book "1" -- "1" BookDetail
Let's configure your models for this:
class Book < ActiveRecord::Base
has_one :book_detail, dependent: :destroy
include YamlExporter
yaml_structure do
attributes :title, :author, :price
one :book_detail do
attributes :summary, :publication_year
end
end
end
class BookDetail < ActiveRecord::Base
belongs_to :book
end
Your YAML file would look like:
title: Ruby on Rails Tutorial
author: Michael Hartl
price: 100
book_detail:
summary: |-
A practical introduction to Ruby on Rails development.
publication_year: 2022
Identification: YamlExporter uses book.book_detail to follow the association. There can only be one book detail per book.
Edge cases:
- If there is no book detail in the database, it will be created.
- If the
book_detail:key is missing from the YAML or set tonull, the existing book detail is destroyed — omission andnullare treated the same. If that violates a model-level validation (e.g.book_detailis required), the resultingActiveRecorderror propagates and the whole import is rolled back.
one with find_by (reference)
This is the belongs_to pattern. A book is published by a publisher — the book has the publisher_id foreign key, and the publisher is a first-class record managed in its own YAML file elsewhere.
classDiagram
class Book {
- String title
- String author
- Decimal price
}
class Publisher {
- String name
- String slug
}
Book "*" -- "1" Publisher
We could dump the publisher_id column to the book yaml, but that's brittle and not human-friendly. Instead we reference the publisher by a stable column:
class Book < ActiveRecord::Base
belongs_to :publisher
include YamlExporter
yaml_structure do
attributes :title, :author, :price
one :publisher, find_by: :slug
end
end
Our yaml file now looks like this:
title: Ruby on Rails Tutorial
author: Michael Hartl
price: 100
publisher: addison-wesley
Identification: YamlExporter uses Publisher.find_by(slug: "addison-wesley") to resolve the publisher. If several publishers share the slug, only the first match is used.
Edge cases:
- If no publisher matches the slug in the database,
ActiveRecord::RecordNotFoundis raised.
Two restrictions apply, both following from the ownership model:
find_byis required for a reference-flavoredone. Without it, the only alternative would be exposing the raw foreign key (publisher_id) in the YAML using theattributesmethod.- A block is not allowed when
find_byis given. The publisher owns itself — a book is just one of many records pointing at it — so the book's YAML has no business defining the publisher's attributes. Manage the publisher from its own YAML file instead.
one with find_by and of: (indirect reference)
Sometimes the record you point at does not carry the identifying column directly — a companion record does. A classic case is a two-level identity hierarchy:
classDiagram
class User {
- String name
- String slug
}
class CorporateUser {
- String name
}
class Book {
- String title
}
CorporateUser "*" -- "1" User
Book "*" -- "0..1" CorporateUser : responsible_editor
Book belongs_to :responsible_editor (a CorporateUser), but CorporateUser has no slug of its own. The slug lives on the associated User. Storing an opaque database id in the YAML is brittle; referencing the user's slug is human-readable and stable.
The of: keyword lets you do exactly that:
class Book < ActiveRecord::Base
belongs_to :responsible_editor, class_name: 'CorporateUser',
foreign_key: :responsible_editor_id, optional: true
include YamlExporter
yaml_structure do
attributes :title
one :responsible_editor, find_by: :slug, of: :user
end
end
class CorporateUser < ActiveRecord::Base
belongs_to :user
end
class User < ActiveRecord::Base
has_one :corporate_user
end
The YAML stores the user's slug:
title: Ruby on Rails Tutorial
responsible_editor: alice
On import, YamlExporter:
- Looks up
User.find_by(slug: "alice"). - Navigates back to
CorporateUserby reversing the FK (CorporateUser.find_by(user_id: alice.id)). - Assigns the result to
book.responsible_editor.
On export, book.responsible_editor.user.slug is emitted.
Restrictions:
of:requiresfind_by:— the two always appear together.- A block is not allowed when
of:is used (same ownership rule as plainfind_by:). - The
of:association must be a 1:[0,1] relation (belongs_toorhas_one). Using ahas_manyassociation raises at class-load time. - If the
of:record exists but no target record is linked to it (e.g. aUserwith noCorporateUser),ActiveRecord::RecordNotFoundis raised on import.
Putting it all together
Let us now put all together:
class Book < ActiveRecord::Base
has_many :book_parts, dependent: :destroy
has_one :book_detail, dependent: :destroy
has_and_belongs_to_many :authors
belongs_to :publisher
has_many :book_reviewers, dependent: :destroy
has_many :reviewers, through: :book_reviewers
belongs_to :responsible_editor, class_name: 'CorporateUser',
foreign_key: :responsible_editor_id, optional: true
include YamlExporter
yaml_structure do
attributes :title, :price
many :book_parts, find_by: :slug, positioned_by: :position do
attributes :title, :content
end
one :book_detail do
attributes :summary, :publication_year
end
many :authors, find_by: :slug
one :publisher, find_by: :slug
many :reviewers, through: :book_reviewers, find_by: :slug do
attributes :finished
end
one :responsible_editor, find_by: :slug, of: :user
end
end
class CorporateUser < ActiveRecord::Base
belongs_to :user
end
class User < ActiveRecord::Base
has_one :corporate_user
end
Our yaml file now looks like this:
title: Ruby on Rails Tutorial
price: 100
book_parts:
- slug: chapter-1
title: Chapter 1
content: |-
This is the first chapter of the book
- slug: chapter-2
title: Chapter 2
content: |-
This is the second chapter of the book
book_detail:
summary: |-
A practical introduction to Ruby on Rails development.
publication_year: 2022
authors:
- michael-hartl
- another-author
publisher: addison-wesley
reviewers:
- slug: alice
finished: true
- slug: bob
finished: false
responsible_editor: alice
And the resulting object graph:
classDiagram
class Book {
- String title
- Decimal price
}
class BookPart {
- String slug
- String title
- String content
- Integer position
}
class BookDetail {
- String summary
- Integer publication_year
}
class Author {
- String name
- String slug
}
class Publisher {
- String name
- String slug
}
class BookReviewer {
- Boolean finished
}
class Reviewer {
- String name
- String slug
}
class CorporateUser {
- String name
}
class User {
- String name
- String slug
}
Book "1" -- "*" BookPart
Book "1" -- "1" BookDetail
Book "*" -- "*" Author
Book "*" -- "1" Publisher
Book "1" -- "*" BookReviewer
BookReviewer "*" -- "1" Reviewer
Book "*" -- "0..1" CorporateUser : responsible_editor
CorporateUser "*" -- "1" User
Behaviors of yaml_import and yaml_export
yaml_import and yaml_export are instance methods included in the model. So if you want to import a model from a yaml file, you first find or create an object and then call yaml_import on it:
book = Book.new
book.yaml_import(File.read('ruby-on-rails.yaml'))
And of course you can export the data from your ActiveRecord model to the file:
yaml_string = book.yaml_export
We prefer to identify the objects by the filename of the file and a slug field. So if you have a directory books you can import all files in the directory with:
Dir.glob('books/*.yaml').each do |file|
book = Book.find_or_create_by(slug: File.basename(file, '.yaml'))
book.yaml_import(File.read(file))
end
Import edge cases (apply to every yaml_import, in addition to the lifecycle rules in the mental model):
- Duplicate keys inside the same YAML list — e.g. two
slug: chapter-1entries under onebook_parts:— raise an error. Eachfind_bykey must be unique within its list. - Missing /
nullclears not just plainattributescolumns, but alsoonereferences (the foreign key is cleared) andoneowned children (the child is destroyed).
Export behavior (mirrors the import rules):
- Empty values are omitted by default. A
nilcolumn, a missingonereference, an absent ownedone, and an emptymanylist are all left out of the document entirely. This is round-trip safe: import treats a missing key, an explicitnull, and an empty list the same way. To keep explicitnulls in the file — e.g. so reviewers can discover optional fields — callyaml_export(omit_nil: false). An ownedonechild that exists but has onlynilattributes is still emitted (as{}) — dropping it would destroy the child on re-import. textcolumns are written as YAML literal block scalars (|), so multi-line and long-form content stays readable and diff-friendly.string/varchar columns stay inline regardless of length, and non-string scalars (numbers, booleans, …) are unaffected. The choice follows the column type, not the value, so a model's files always look the same. Trailing whitespace on a line (and carriage returns) is stripped on export — it would otherwise force YAML to fall back to an inline double-quoted scalar, and it is almost always an accidental typo. Emoji and other astral-plane characters (codepoints ≥ U+10000) are emitted literally and stay in block style too, despite a libyaml quirk that would otherwise escape them into an inline scalar; ordinary accented and umlaut characters were never affected.- The output is a plain YAML document, without a leading
---marker. - Lists are written in a stable, diff-friendly order. The rule is: take the first rule that applies, top to bottom:
manywithpositioned_by:→ sorted by the position column ASC; the column itself is omitted from each entry's hash.manywithfind_by:(with or without a block, including thethrough:variant, withoutpositioned_by:) → sorted by thefind_bycolumn so adding or removing an entry only touches its own line in Git diffs.manywith a block only (nofind_by:) → the collection's SQL order, which is the order the children were inserted (this is also the order positional matching relies on).
API reference
All DSL methods are declared inside a yaml_structure do … end block on the model. Inside the block, attributes, one and many are resolved against a builder — they never clash with ActiveRecord's own class-level methods (e.g. attribute, has_one).
attributes(*names)
Columns of the model that are serialized and deserialized. Missing keys in the YAML reset the corresponding columns to nil on import.
one(name, find_by: nil, of: nil, &block)
A single related record. Exactly one of find_by: or a block must be given:
| Call | YAML shape | Meaning |
|---|---|---|
one :child do … end |
nested hash | Owned child (the has_one pattern). |
one :child, find_by: :slug |
bare string | Reference to a record managed elsewhere (the belongs_to pattern). |
one :child, find_by: :slug, of: :companion |
bare string | Indirect reference: the slug lives on a companion record reachable via the companion association on the target. |
of: requires find_by: and cannot be combined with a block. The of: association must be a 1:[0,1] relation (belongs_to or has_one). See one with find_by and of: for a worked example.
Passing both a block and find_by: is rejected — see the ownership reasoning in one with find_by.
many(name, positioned_by: nil, find_by: nil, through: nil, &block)
A list of related records. The flavor is picked from the combination of positioned_by:, find_by:, through: and a block:
| Call | YAML shape | Meaning |
|---|---|---|
many :children do … end |
list of hashes, matched by order | Children fully managed, identity by position. |
many :children, positioned_by: :position do … end |
same as above, without position: in each hash |
Like the previous row, but the position column is derived from the 1-based array index (and omitted on export). |
many :children, find_by: :slug do … end |
list of hashes containing the slug: key |
Children fully managed, identity by a stable column. |
many :children, find_by: :slug, positioned_by: :position do … end |
same as above, without position: in each hash |
Like the previous row, but the named column is derived from the 1-based array index (and omitted on export). |
many :children, find_by: :slug |
list of bare strings | Children referenced by key, managed elsewhere (HABTM pattern). |
many :children, through: :joins, find_by: :slug do … end |
list of hashes containing the slug: key |
has_many :through where the block describes attributes of the join model. |
many :children, through: :joins, find_by: :slug, positioned_by: :position do … |
same as above, without position: in each hash |
As above, with the position column derived on the join model. |
positioned_by: requires a block – there must be an owned record to write the column onto – and therefore cannot be used with the reference-list flavor of many.
Import / export
instance.yaml_import(yaml_string)— updatesinstancein place from the YAML, inside a single transaction. Returns the instance.instance.yaml_export(omit_nil: true)— returns a YAML string forinstancefollowing itsyaml_structure. Withomit_nil: true(the default) keys whose value is empty — anilattribute/reference, an absent owned child, or an emptymanylist — are left out. Passomit_nil: falseto keep them as explicitnulls, which is handy when you want optional fields to stay visible in the file.
Whether a string is written inline or as a literal block scalar (|) is decided by the database column type, not by yaml_export arguments: text columns always use block style, string/varchar columns always stay inline. See Export behavior.
ModelClass.yaml_schema
Returns a JSON-schema-like hash describing the YAML shape declared by yaml_structure. Useful for generating editor support or validating YAML out-of-band. Invalid YAML passed to yaml_import raises with a message pointing at the violated part of the schema.
Development
Run the test suite with:
$ bundle exec rake test