Class: Exwiw::Adapter::SqliteAdapter::StreamingResult

Inherits:
Object
  • Object
show all
Includes:
Enumerable
Defined in:
lib/exwiw/adapter/sqlite_adapter.rb

Overview

A lazy, streaming stand-in for the materialized rows #execute used to return (‘connection.execute(sql)`). It walks the result one row at a time via SQLite’s statement cursor (Statement#each -> sqlite3_step) instead of buffering the whole result set, so the dump’s dominant memory cost — a Ruby array as large as the table — never materializes. The Runner drives it exactly like the old Array: #size to skip empty tables and log the count, then a single streaming pass (SqlBulkInsert#write_inserts -> each_slice) to write the INSERT.

Mirrors Mysql/PostgresqlAdapter::StreamingResult, with two SQLite specifics:

- #size runs a separate `SELECT COUNT(*)` of the same query with the
  projection replaced by COUNT(*) (compile_ast(count_only: true)) —
  exact because exwiw's extraction queries have no DISTINCT/GROUP
  BY/LIMIT, so the row count is independent of the projection. (Unlike
  MySQL, SQLite tolerates a duplicate-column subquery wrap too, but the
  count_only form is shared with MySQL and avoids the extra subquery.)
- SQLite is an embedded, single-connection engine that allows several
  active prepared statements at once, so the #size COUNT and the data
  cursor do not contend. The statement is closed in an ensure block so
  an abandoned mid-stream iteration still releases the cursor.

Instance Method Summary collapse

Constructor Details

#initialize(connection:, data_sql:, count_sql:) ⇒ StreamingResult

Returns a new instance of StreamingResult.



32
33
34
35
36
 
# File 'lib/exwiw/adapter/sqlite_adapter.rb', line 32

def initialize(connection:, data_sql:, count_sql:)
  @connection = connection
  @data_sql = data_sql
  @count_sql = count_sql
end

Instance Method Details

#each ⇒ Object

Stream the result set row by row. Each row is an Array of values in SQLite’s native type mapping — byte-identical to what ‘connection.execute(sql)` produced, so the generated INSERT is unchanged.



46
47
48
49
50
51
52
53
54
55
56
 
# File 'lib/exwiw/adapter/sqlite_adapter.rb', line 46

def each
  return enum_for(:each) { size } unless block_given?

  statement = @connection.prepare(@data_sql)
  begin
    statement.each { |row| yield row }
  ensure
    statement.close
  end
  self
end

#size ⇒ Object Also known as: length 



38
39
40
 
# File 'lib/exwiw/adapter/sqlite_adapter.rb', line 38

def size
  @size ||= @connection.execute(@count_sql).dig(0, 0).to_i
end