Snapshot Testing

Snapshot testing ensures your schemas generate consistent, expected output. It's perfect for catching unintended schema changes in CI pipelines.

Quick Start

1. Include the Module

# spec/spec_helper.rb
require 'fake_data_dsl/snapshot_testing'

RSpec.configure do |config|
  config.include FakeDataDSL::SnapshotTesting
end

2. Write Snapshot Tests

# spec/schemas/snapshots_spec.rb
RSpec.describe "Schema Snapshots" do
  let(:schema) { FakeDataDSL.load("schemas/user.dsl") }

  it "generates stable output" do
    expect_snapshot(schema, seed: 42)
  end
end

3. Run Tests

# First run creates snapshots
bundle exec rspec spec/schemas/snapshots_spec.rb

# Subsequent runs compare against snapshots
bundle exec rspec spec/schemas/snapshots_spec.rb

How It Works

  1. First Run: Generates data with a seed and saves it as a JSON snapshot
  2. Subsequent Runs: Generates data with the same seed and compares to the snapshot
  3. On Mismatch: Test fails, showing the difference

API Reference

expect_snapshot(schema, seed:, update:)

Main method for snapshot testing.

# Create or verify snapshot
expect_snapshot(schema, seed: 42)

# Force update snapshot (when changes are intentional)
expect_snapshot(schema, seed: 42, update: true)

Parameters:

  • schema - The FakeDataDSL schema to test
  • seed - Random seed for deterministic generation (default: 42)
  • update - If true, update the snapshot file (default: false)

match_snapshot(schema_name)

RSpec matcher for snapshot comparison.

it "matches snapshot" do
  data = schema.generate(seed: 42)
  expect(data).to match_snapshot("user")
end

SnapshotTesting.generate_all_snapshots(registry, seed:)

Generate snapshots for all schemas in a registry.

registry = FakeDataDSL::Registry.new
registry.load_dir("schemas/")

FakeDataDSL::SnapshotTesting.generate_all_snapshots(registry, seed: 42)
# Creates: spec/snapshots/fake_data_dsl/user.json
# Creates: spec/snapshots/fake_data_dsl/order.json
# etc.

SnapshotTesting.verify_all_snapshots(registry, seed:)

Verify all snapshots match.

mismatches = FakeDataDSL::SnapshotTesting.verify_all_snapshots(registry, seed: 42)

if mismatches.any?
  puts "Mismatched schemas: #{mismatches.join(', ')}"
  exit 1
end

Configuration

Snapshot Directory

Default: spec/snapshots/fake_data_dsl/

# Change snapshot directory
FakeDataDSL::SnapshotTesting.snapshot_dir = "spec/fixtures/snapshots"

Testing All Schemas

Dynamic Test Generation

RSpec.describe "All Schema Snapshots" do
  before(:all) do
    FakeDataDSL.load_schemas("schemas/")
  end

  FakeDataDSL.each_schema do |schema|
    it "#{schema.name} generates stable output" do
      expect_snapshot(schema, seed: 42)
    end
  end
end

Bulk Verification

RSpec.describe "Schema Stability" do
  it "all schemas match their snapshots" do
    registry = FakeDataDSL::Registry.new
    registry.load_dir("schemas/")

    mismatches = FakeDataDSL::SnapshotTesting.verify_all_snapshots(registry, seed: 42)

    expect(mismatches).to be_empty,
      "Schemas with mismatched snapshots: #{mismatches.join(', ')}"
  end
end

CI Integration

GitHub Actions

# .github/workflows/schema-snapshots.yml
name: Schema Snapshots

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: 3.2
          bundler-cache: true

      - name: Run snapshot tests
        run: bundle exec rspec spec/schemas/snapshots_spec.rb

GitLab CI

# .gitlab-ci.yml
schema-snapshots:
  stage: test
  script:
    - bundle exec rspec spec/schemas/snapshots_spec.rb
  only:
    - merge_requests

Updating Snapshots

When to Update

Update snapshots when you intentionally change a schema:

# In test file, temporarily set update: true
expect_snapshot(schema, seed: 42, update: true)

Batch Update

# Create a rake task
namespace :snapshots do
  task :update do
    require 'fake_data_dsl'

    registry = FakeDataDSL::Registry.new
    registry.load_dir("schemas/")

    FakeDataDSL::SnapshotTesting.generate_all_snapshots(registry, seed: 42)
    puts "Snapshots updated!"
  end
end

bundle exec rake snapshots:update

Best Practices

1. Use Consistent Seeds

Always use the same seed for deterministic output:

# Good: Consistent seed
expect_snapshot(schema, seed: 42)

# Bad: No seed (non-deterministic)
expect_snapshot(schema)

2. Commit Snapshots to Git

Snapshots should be version controlled:

git add spec/snapshots/
git commit -m "Update schema snapshots"

3. Review Snapshot Changes

Always review snapshot diffs in pull requests:

# spec/snapshots/fake_data_dsl/user.json
{
-  "email": "john@example.com",
+  "email": "john.doe@example.com",
   "name": "John Doe"
}

4. Separate Snapshot Tests

Keep snapshot tests in their own files:

spec/
├── schemas/
│   └── snapshots_spec.rb    # Snapshot tests
├── models/
│   └── user_spec.rb         # Model tests
└── snapshots/
    └── fake_data_dsl/       # Snapshot files
        ├── user.json
        └── order.json

Troubleshooting

Snapshot Mismatch

expected generated data to match snapshot for User

Solutions:

  1. If change is intentional: Update snapshot with update: true
  2. If change is unintentional: Fix the schema

Snapshot Not Found

snapshot not found: spec/snapshots/fake_data_dsl/user.json

Solution: Run tests with update: true to create initial snapshot.

Non-Deterministic Data

Some types (like now) aren't deterministic even with seeds.

Solution: Override non-deterministic fields:

it "generates stable output" do
  data = schema.generate(seed: 42, overrides: {
    "created_at" => "2024-01-01T00:00:00Z"
  })
  expect(data).to match_snapshot("user")
end

See Also