Property-Based Testing

FakeDataDSL integrates seamlessly with RSpec for property-based testing, allowing you to verify that properties hold for all generated data.

Quick Start

require 'fake_data_dsl'
include FakeDataDSL::PropertyTesting

RSpec.describe "User Registration" do
  it "accepts all valid users" do
    verify_property("User", count: 1000) do |user_data|
      user = User.create(user_data)
      expect(user).to be_valid
      expect(user.save).to be_truthy
    end
  end
end

Features

verify_property

Verifies a property holds for multiple generated records:

verify_property("User", count: 100) do |user_data|
  expect(user_data["email"]).to match(/\A[\w+\-.]+@[a-z\d\-]+(\.[a-z\d\-]+)*\.[a-z]+\z/i)
  expect(user_data["age"]).to be_between(18, 80)
end

Options:

  • count - Number of records to generate (default: 100)
  • seed - Random seed for reproducibility
  • mode - Generation mode (:random, :edge, :invalid, :hostile, :mixed)
  • registry - Custom schema registry

verify_property_stream

Memory-efficient streaming verification for large counts:

verify_property_stream("User", count: 10_000) do |user_data|
  expect(user_data["id"]).to be_a(String)
end

Records are generated lazily and garbage collected as they're processed.

shrink_counterexample

Shrinks a failing test case to find a minimal counterexample:

failing_record = schema.generate(mode: :hostile)

minimal = shrink_counterexample("User", failing_record) do |record|
  expect(record["age"]).to be > 100 # This will fail
end

# minimal contains the smallest record that still fails

Examples

Email Format Validation

verify_property("User", count: 1000) do |user_data|
  email = user_data["email"]
  expect(email).to match(/\A[\w+\-.]+@[a-z\d\-]+(\.[a-z\d\-]+)*\.[a-z]+\z/i)
end

Age Range Validation

verify_property("User", count: 1000) do |user_data|
  age = user_data["age"]
  expect(age).to be_between(18, 80)
end

UUID Format Validation

verify_property("User", count: 1000) do |user_data|
  uuid = user_data["id"]
  expect(uuid).to match(/\A[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\z/i)
end

Model Validation

verify_property("User", count: 500) do |user_data|
  user = User.new(user_data)
  expect(user).to be_valid
end

API Endpoint Testing

verify_property("Order", count: 200) do |order_data|
  response = post("/api/orders", order_data)
  expect(response.status).to eq(201)
  expect(response.json["id"]).to be_present
end

Deterministic Testing

seed = 12345

verify_property("User", count: 100, seed: seed) do |user_data|
  # Same seed = same data every time
  expect(user_data["email"]).to eq(expected_email)
end

Error Messages

When a property fails, you get detailed error information:

Property failed for record 42/100 (seed: 12345):
expected 25 to be > 100
Record: {"id"=>"550e8400...", "age"=>25, ...}

This includes:

  • Record index (which record failed)
  • Seed (for reproducibility)
  • Full error message
  • Complete record data

Best Practices

1. Use Appropriate Counts

# Quick validation (fast)
verify_property("User", count: 10) do |user_data|
  expect(user_data).to be_a(Hash)
end

# Comprehensive testing (thorough)
verify_property("User", count: 1000) do |user_data|
  expect(user_data["email"]).to match(/@/)
end

2. Use Seeds for Reproducibility

# Same seed = same data = reproducible tests
verify_property("User", count: 100, seed: 42) do |user_data|
  # Test logic
end

3. Use Streaming for Large Counts

# Memory-efficient for 10K+ records
verify_property_stream("User", count: 50_000) do |user_data|
  expect(user_data["id"]).to be_present
end

4. Test Edge Cases

# Test with edge case mode
verify_property("User", count: 100, mode: :edge) do |user_data|
  # Should handle empty strings, nulls, extremes
  expect(user_data["email"]).to be_a(String).or(be_nil)
end

5. Test Invalid Data Handling

# Test validation logic
verify_property("User", count: 100, mode: :invalid) do |user_data|
  # Should reject invalid data
  user = User.new(user_data)
  expect(user).not_to be_valid
end

Integration with RSpec

The PropertyTesting module is automatically included in RSpec when available:

# No need to include manually
RSpec.describe "User" do
  it "validates all users" do
    verify_property("User") do |user_data|
      # Test logic
    end
  end
end

Advanced Usage

Custom Registry

custom_registry = FakeDataDSL::Registry.new
custom_registry.load_dir("spec/schemas/")

verify_property("User", count: 100, registry: custom_registry) do |user_data|
  # Test logic
end

Multiple Properties

RSpec.describe "User" do
  it "validates email format" do
    verify_property("User", count: 100) do |user_data|
      expect(user_data["email"]).to match(/@/)
    end
  end

  it "validates age range" do
    verify_property("User", count: 100) do |user_data|
      expect(user_data["age"]).to be_between(18, 80)
    end
  end
end

Shrinking Counterexamples

it "finds minimal failing case" do
  failing_record = {
    "id" => "test",
    "name" => "Test",
    "email" => "test@example.com",
    "age" => 25,
    "optional_field" => "can be removed"
  }

  minimal = shrink_counterexample("User", failing_record) do |record|
    expect(record["age"]).to be > 100
  end

  # minimal may have optional_field removed if it still fails
  expect(minimal["age"]).to be <= 100
end

Performance

Property-based testing is efficient:

  • 100 records: ~0.1s
  • 1,000 records: ~1s
  • 10,000 records: ~10s (use streaming)

Use verify_property_stream for counts > 1,000 to avoid memory issues.

Limitations

  • Requires RSpec (automatically detected)
  • Shrinking is basic (removes optional fields, simplifies arrays)
  • No automatic property discovery (you must write properties)

See Also