Class: Exwiw::RowTransformer

Inherits:
Object
  • Object
show all
Defined in:
lib/exwiw/row_transformer.rb

Overview

Applies the Ruby-process-side masking modes — map and replace_with_fake_data — to the rows streamed out of a SQL adapter's #execute. Unlike replace_with/raw_sql, which compile into the SELECT and run in the database, these run in the exwiw process, so they can execute arbitrary Ruby (map) or derive deterministic fake values (fake data).

Built once per table (.build compiles the map procs, resolves seed column indexes, and pre-generates the fake-value pools), then #wrap decorates the adapter's StreamingResult: rows are transformed one at a time as the stream is drained, so the bounded-memory profile of the streaming dump is preserved. #size delegates to the underlying result, keeping the COPY path's upfront count and each_slice's allocation-hint COUNT identical to an unwrapped run.

SQL adapters only: .build returns nil for configs without a columns list (MongodbCollectionConfig has fields; rails-managed tables have no columns), and for tables where no column carries a Ruby-side mode.

Defined Under Namespace

Classes: Row, TransformedResult

Constant Summary collapse

POOL_SIZE =

Fake values are picked from a pool of this many pre-generated values. Distinct seed values beyond the pool size reuse pool entries (fine for fake data — determinism, not uniqueness, is the contract; uniqueness- sensitive types compose a 64-bit token on top, see FAKE_TYPES).

10_000
POOL_RANDOM_SEED =

Fixed Random seed for pool generation, so a pool is deterministic for a given faker gem version + locale.

715_517
FAKE_TYPES =

Supported replace_with_fake_data types. pool builds one candidate value (called POOL_SIZE times under a seeded Faker random). Types with compose are uniqueness-sensitive: the pooled base alone would collide under a unique index at scale, so the final value composes the base with a 64-bit hex token derived from the same seed digest (collision probability at 5M distinct seeds ≈ 7e-7).

{
  "human_name"   => { pool: -> { Faker::Name.name } },
  "first_name"   => { pool: -> { Faker::Name.first_name } },
  "last_name"    => { pool: -> { Faker::Name.last_name } },
  "phone_number" => { pool: -> { Faker::PhoneNumber.phone_number } },
  "address"      => { pool: -> { Faker::Address.full_address } },
  "company_name" => { pool: -> { Faker::Company.name } },
  "email"        => { pool: -> { Faker::Internet.username(specifier: 5..12) },
                      compose: ->(base, token) { "#{base}.#{token}@example.com" } },
  "username"     => { pool: -> { Faker::Internet.username(specifier: 5..12) },
                      compose: ->(base, token) { "#{base}_#{token}" } },
}.freeze

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(table) ⇒ RowTransformer

Returns a new instance of RowTransformer.



141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
 
# File 'lib/exwiw/row_transformer.rb', line 141

def initialize(table)
  @table_name = table.name
  @name_to_index = {}
  table.columns.each_with_index { |column, index| @name_to_index[column.name] = index }
  @name_to_index.freeze
  @row_accessor = Row.new(@table_name, @name_to_index)

  self.class.require_faker! if table.columns.any?(&:replace_with_fake_data)

  @transforms = table.columns.each_with_index.filter_map do |column, index|
    if column.map
      [index, compile_map(column)]
    elsif column.replace_with_fake_data
      [index, compile_fake(column, index)]
    end
  end
  @replacement_buffer = Array.new(@transforms.size)
end

Class Method Details

.build(table) ⇒ Object

-> RowTransformer | nil. nil when the table carries no Ruby-side mode, so the Runner can skip wrapping entirely (the unused path stays byte-identical and cost-free).



104
105
106
107
108
109
110
111
 
# File 'lib/exwiw/row_transformer.rb', line 104

def self.build(table)
  return nil unless table.respond_to?(:columns)

  columns = table.columns
  return nil if columns.nil? || columns.none? { |c| c.map || c.replace_with_fake_data }

  new(table)
end

.build_fake_pool(type, locale) ⇒ Object 



129
130
131
132
133
134
135
136
137
138
139
 
# File 'lib/exwiw/row_transformer.rb', line 129

def self.build_fake_pool(type, locale)
  spec = FAKE_TYPES.fetch(type)
  previous_locale = Faker::Config.locale
  previous_random = Faker::Config.random
  Faker::Config.locale = locale if locale
  Faker::Config.random = Random.new(POOL_RANDOM_SEED)
  Array.new(POOL_SIZE) { spec[:pool].call.freeze }.freeze
ensure
  Faker::Config.locale = previous_locale
  Faker::Config.random = previous_random
end

.fake_pool(type, locale) ⇒ Object

Pools are memoized per (type, locale): every column sharing a type+locale sees the same pool, which is what makes equal seed values map to equal fake values across tables and runs.



124
125
126
127
 
# File 'lib/exwiw/row_transformer.rb', line 124

def self.fake_pool(type, locale)
  @fake_pools ||= {}
  @fake_pools[[type, locale]] ||= build_fake_pool(type, locale)
end

.require_faker! ⇒ Object 



113
114
115
116
117
118
119
 
# File 'lib/exwiw/row_transformer.rb', line 113

def self.require_faker!
  require "faker"
rescue LoadError
  raise LoadError,
        "replace_with_fake_data requires the faker gem. " \
        "Add `gem \"faker\"` to your Gemfile (it is not a runtime dependency of exwiw)."
end

Instance Method Details

#transform(row) ⇒ Object

Replacements are all computed from the original row before any is written back, so a map proc / fake seed always reads the pre-transform (post-SQL-masking) value regardless of column order. Rows are mutated in place when possible (each cursor yields a fresh Array per row), but sqlite3's Statement#each yields frozen rows — those are duped first.



169
170
171
172
173
174
175
176
177
178
 
# File 'lib/exwiw/row_transformer.rb', line 169

def transform(row)
  @transforms.each_with_index do |(_, callable), k|
    @replacement_buffer[k] = callable.call(row)
  end
  row = row.dup if row.frozen?
  @transforms.each_with_index do |(index, _), k|
    row[index] = @replacement_buffer[k]
  end
  row
end

#wrap(results) ⇒ Object 



160
161
162
 
# File 'lib/exwiw/row_transformer.rb', line 160

def wrap(results)
  TransformedResult.new(results, self)
end