Class: Exwiw::Adapter::MysqlAdapter::StreamingResult

Inherits:
Object
  • Object
show all
Includes:
Enumerable
Defined in:
lib/exwiw/adapter/mysql_adapter.rb

Overview

A lazy, streaming stand-in for the materialized rows #execute used to return (‘connection.query(sql).rows`). It pulls rows off the wire one at a time (mysql2 single-row stream) instead of buffering the whole result set, so the dump’s dominant memory cost — a Ruby array as large as the table — never materializes. The Runner drives it exactly like the old Array: #size to skip empty tables and log the count, then a single streaming pass (SqlBulkInsert#write_inserts -> each_slice).

Mirrors PostgresqlAdapter::StreamingResult, with two MySQL specifics:

- #size runs a separate `SELECT COUNT(*)` of the same query. Unlike the
  pg path, it does NOT wrap the SELECT in a subquery: MySQL rejects a
  derived table with duplicate column names, which a rails-managed
  `SELECT *` joined to another table produces. Instead the projection
  is replaced by `COUNT(*)` (compile_ast(count_only: true)) — exact
  because exwiw's extraction queries have no DISTINCT/GROUP BY/LIMIT,
  so the row count is independent of the projected columns.
- the stream ties up the connection until fully drained. The Runner
  always drains it (write_inserts) before any further query
  (post_insert_sql / DELETE), and MysqlClient#stream_rows drains the
  remainder if iteration is abandoned, so the connection stays usable.

Instance Method Summary collapse

Constructor Details

#initialize(client:, data_sql:, count_sql:) ⇒ StreamingResult

Returns a new instance of StreamingResult.



33
34
35
36
37
 
# File 'lib/exwiw/adapter/mysql_adapter.rb', line 33

def initialize(client:, data_sql:, count_sql:)
  @client = client
  @data_sql = data_sql
  @count_sql = count_sql
end

Instance Method Details

#each(&block) ⇒ Object

Stream the result set row by row. Each row is an Array of String|nil (mysql2 ‘cast: false` / stringified) — identical to what `connection.query(sql).rows` produced, so the generated INSERT is unchanged.



48
49
50
51
52
53
 
# File 'lib/exwiw/adapter/mysql_adapter.rb', line 48

def each(&block)
  return enum_for(:each) { size } unless block_given?

  @client.stream_rows(@data_sql, &block)
  self
end

#size ⇒ Object Also known as: length 



39
40
41
 
# File 'lib/exwiw/adapter/mysql_adapter.rb', line 39

def size
  @size ||= @client.query(@count_sql).rows.dig(0, 0).to_i
end