Module: ActiveRecord::ConnectionAdapters::Duckdb::SchemaStatements
- Included in:
- ActiveRecord::ConnectionAdapters::DuckdbAdapter
- Defined in:
- lib/active_record/connection_adapters/duckdb/schema_statements.rb
Overview
DuckDB-specific schema statement implementations Provides DuckDB-specific functionality for table, sequence, and index management
Constant Summary collapse
- DUMPABLE_DUCKLAKE_OPTIONS =
DuckLake options that should be included in schema dumps These are user-configurable options, not internal metadata like 'version' or 'created_by' See: https://ducklake.select/docs/stable/specification/tables/ducklake_metadata.html
%w[ data_inlining_row_limit target_file_size parquet_row_group_size_bytes parquet_row_group_size parquet_compression parquet_compression_level parquet_version hive_file_pattern require_commit_message rewrite_delete_threshold delete_older_than expire_older_than per_thread_output encrypted ].freeze
Instance Method Summary collapse
-
#create_sequence(sequence_name, start_with: 1, increment_by: 1, **options) ⇒ void
Creates a sequence in DuckDB.
-
#create_table(table_name, id: :primary_key, primary_key: nil, **options) ⇒ void
Creates a table with DuckDB-specific handling for sequences and primary keys.
-
#create_table_definition(name) ⇒ ActiveRecord::ConnectionAdapters::Duckdb::TableDefinition
Creates a DuckDB-specific table definition instance.
-
#data_source_sql(name = nil, type: nil) ⇒ String
Generates SQL for querying data sources (tables/views) with optional filtering.
-
#drop_sequence(sequence_name, if_exists: false) ⇒ void
Drops a sequence from the database.
-
#ducklake_metadata_schema ⇒ String?
Returns the DuckLake metadata schema name for the current database.
-
#ducklake_options ⇒ Hash?
Returns DuckLake options for schema dumping.
-
#ducklake_table_options(table_name) ⇒ Hash?
Returns DuckLake options for a specific table.
-
#execute(sql, name = nil) ⇒ DuckDB::Result
Override execute to intercept CREATE TABLE statements and inject sequence defaults.
-
#indexes(table_name) ⇒ Array<ActiveRecord::ConnectionAdapters::IndexDefinition>
Returns indexes for a specific table.
-
#internal_string_options_for_primary_key ⇒ Hash
Returns options for internal primary key columns (schema_migrations, ar_internal_metadata) DuckLake doesn't support PRIMARY KEY constraints, so we omit them in that mode.
-
#lookup_cast_type_from_column(sql_type_metadata) ⇒ ActiveRecord::Type::Value
Looks up the appropriate cast type for a column based on SQL type metadata.
-
#next_sequence_value(sequence_name) ⇒ String
Returns SQL expression to get the next value from a sequence.
-
#partition_expressions(table_name) ⇒ Array<String>?
Returns the partition expressions for a DuckLake table.
-
#quoted_scope(name = nil, type: nil) ⇒ Hash
Creates a quoted scope hash for table/schema queries with type filtering.
-
#reset_sequence!(sequence_name, value = 1) ⇒ void
Resets a sequence to a specific value.
-
#schema_creation ⇒ ActiveRecord::ConnectionAdapters::Duckdb::SchemaCreation
Returns a DuckDB-specific schema creation instance.
-
#sequence_exists?(sequence_name) ⇒ Boolean
Checks if a sequence exists in the database.
-
#sequences ⇒ Array<String>
Returns a list of all sequences in the database.
-
#set_ducklake_option(option_name, value, table_name = nil) ⇒ void
Sets a DuckLake-specific option using CALL set_option.
-
#set_partitioned_by(table_name, expressions) ⇒ void
Sets partitioning for a DuckLake table DuckLake supports time-based partitioning using SQL functions.
-
#tables ⇒ Array<String>
Returns a list of all tables in the database.
-
#type_to_sql(type, limit: nil, precision: nil, scale: nil, **options) ⇒ String
Converts ActiveRecord type to DuckDB SQL type string.
-
#views ⇒ Array<String>
Returns a list of all views in the database.
Instance Method Details
#create_sequence(sequence_name, start_with: 1, increment_by: 1, **options) ⇒ void
This method returns an undefined value.
Creates a sequence in DuckDB
88 89 90 91 92 93 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 88 def create_sequence(sequence_name, start_with: 1, increment_by: 1, **) sql = "CREATE SEQUENCE #{quote_table_name(sequence_name)}" sql << " START #{start_with.to_i}" if start_with != 1 sql << " INCREMENT #{increment_by.to_i}" if increment_by != 1 execute(sql, 'Create Sequence') end |
#create_table(table_name, id: :primary_key, primary_key: nil, **options) ⇒ void
This method returns an undefined value.
Creates a table with DuckDB-specific handling for sequences and primary keys
53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 53 def create_table(table_name, id: :primary_key, primary_key: nil, **) # Handle sequence creation for integer primary keys BEFORE table creation sequence_name = nil pk_column_name = nil needs_sequence_default = false if id != false && id != :uuid && id != :string pk_column_name = primary_key || 'id' sequence_name = "#{table_name}_#{pk_column_name}_seq" needs_sequence_default = true # Extract sequence start value from options start_with = .dig(:sequence, :start_with) || [:start_with] || 1 create_sequence_safely(sequence_name, table_name, start_with: start_with) end # Store sequence info for later use during table creation @pending_sequence_default = ({ table: table_name, column: pk_column_name, sequence: sequence_name } if needs_sequence_default && sequence_name && pk_column_name) begin # Now create the table with Rails handling the standard creation super do |td| # If block given, let user define columns yield td if block_given? end ensure # Clear the pending sequence default @pending_sequence_default = nil end end |
#create_table_definition(name) ⇒ ActiveRecord::ConnectionAdapters::Duckdb::TableDefinition
Creates a DuckDB-specific table definition instance
43 44 45 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 43 def create_table_definition(name, **) TableDefinition.new(self, name, **) end |
#data_source_sql(name = nil, type: nil) ⇒ String
Generates SQL for querying data sources (tables/views) with optional filtering
372 373 374 375 376 377 378 379 380 381 382 383 384 385 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 372 def data_source_sql(name = nil, type: nil) scope = quoted_scope(name, type: type) sql = 'SELECT table_name FROM information_schema.tables' conditions = [] conditions << "table_schema = #{scope[:schema]}" if scope[:schema] conditions << "table_name = #{scope[:name]}" if scope[:name] conditions << scope[:type] if scope[:type] # This now contains the full condition sql += " WHERE #{conditions.join(" AND ")}" if conditions.any? sql += ' ORDER BY table_name' sql end |
#drop_sequence(sequence_name, if_exists: false) ⇒ void
This method returns an undefined value.
Drops a sequence from the database
99 100 101 102 103 104 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 99 def drop_sequence(sequence_name, if_exists: false) sql = +'DROP SEQUENCE' sql << ' IF EXISTS' if if_exists sql << " #{quote_table_name(sequence_name)}" execute(sql, 'Drop Sequence') end |
#ducklake_metadata_schema ⇒ String?
Returns the DuckLake metadata schema name for the current database
210 211 212 213 214 215 216 217 218 219 220 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 210 def # Don't memoize - current_database can change after USE DATABASE # DuckLake metadata is stored in __ducklake_metadata_<database_name> result = execute('SELECT current_database()', 'Get Current Database') db_name = result.first&.first return nil if db_name.nil? || db_name.empty? "__ducklake_metadata_#{db_name}" rescue StandardError nil end |
#ducklake_options ⇒ Hash?
Returns DuckLake options for schema dumping
264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 264 def return nil unless ducklake? = return nil unless # Query the ducklake_metadata table for global options (scope IS NULL) sql = <<~SQL SELECT key, value FROM #{quote_table_name()}.ducklake_metadata WHERE scope IS NULL SQL result = execute(sql, 'Get DuckLake Options') = {} result.each do |row| key, value = row # Only include user-configurable options, not internal metadata [key] = value if DUMPABLE_DUCKLAKE_OPTIONS.include?(key) end .empty? ? nil : rescue StandardError nil end |
#ducklake_table_options(table_name) ⇒ Hash?
Returns DuckLake options for a specific table
294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 294 def (table_name) return nil unless ducklake? = return nil unless # Get the table_id first table_sql = <<~SQL SELECT table_id FROM #{quote_table_name()}.ducklake_table WHERE table_name = #{quote(table_name.to_s)} SQL table_result = execute(table_sql, 'Get Table ID') table_id = table_result.first&.first return nil unless table_id # Query the ducklake_metadata table for table-scoped options sql = <<~SQL SELECT key, value FROM #{quote_table_name()}.ducklake_metadata WHERE scope = 'table' AND scope_id = #{table_id.to_i} SQL result = execute(sql, 'Get DuckLake Table Options') = result.to_h { |key, value| DUMPABLE_DUCKLAKE_OPTIONS.include?(key) ? [key, value] : [] } .presence rescue StandardError nil end |
#execute(sql, name = nil) ⇒ DuckDB::Result
Override execute to intercept CREATE TABLE statements and inject sequence defaults
502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 502 def execute(sql, name = nil) # Check if this is a CREATE TABLE statement and we have a pending sequence default if @pending_sequence_default && sql.match?(/\A\s*CREATE TABLE/i) pending = @pending_sequence_default table_pattern = /CREATE TABLE\s+"?#{Regexp.escape(pending[:table])}"?\s*\(/i if sql.match?(table_pattern) # Find the PRIMARY KEY column definition and inject the sequence default # This pattern specifically looks for the primary key column with PRIMARY KEY constraint pk_column_pattern = /"?#{Regexp.escape(pending[:column])}"?\s+\w+\s+PRIMARY\s+KEY(?!\s+DEFAULT)/i # Only replace the first occurrence (the actual primary key) sql = sql.sub(pk_column_pattern) do |match| # Inject the sequence default before PRIMARY KEY match.sub(/(\s+)PRIMARY\s+KEY/i, "\\1DEFAULT nextval(#{quote(pending[:sequence])}) PRIMARY KEY") end end end super end |
#indexes(table_name) ⇒ Array<ActiveRecord::ConnectionAdapters::IndexDefinition>
Returns indexes for a specific table
327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 327 def indexes(table_name) indexes = [] begin result = execute("SELECT * FROM duckdb_indexes() WHERE table_name = #{quote(table_name.to_s)}", 'SCHEMA') # Store result as array immediately to avoid consumption issues result_array = result.to_a result_array.each_with_index do |index_row, _idx| # DuckDB duckdb_indexes() returns array with structure: # [database_name, database_oid, schema_name, schema_oid, index_name, index_oid, table_name, table_oid, nil, {}, is_unique, is_primary, column_names, sql] index_name = index_row[4] is_unique = index_row[10] is_primary = index_row[11] column_names_str = index_row[12] # Skip primary key indexes as they're handled separately next if is_primary # Skip if we don't have essential information next unless index_name && column_names_str # Parse column names from string format like "[name]" or "['name']" columns = parse_index_columns(column_names_str) next if columns.empty? # Clean up column names - remove extra quotes cleaned_columns = columns.map { |col| col.gsub(/^"|"$/, '') } # Create IndexDefinition with correct Rails 8.0 signature index_def = ActiveRecord::ConnectionAdapters::IndexDefinition.new( table_name.to_s, # table index_name.to_s, # name !is_unique.nil?, # unique cleaned_columns # columns ) indexes << index_def end rescue StandardError => e Rails.logger&.warn("Could not retrieve indexes for table #{table_name}: #{e.}") if defined?(Rails) end indexes end |
#internal_string_options_for_primary_key ⇒ Hash
Returns options for internal primary key columns (schema_migrations, ar_internal_metadata) DuckLake doesn't support PRIMARY KEY constraints, so we omit them in that mode
35 36 37 38 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 35 def # DuckLake doesn't support PRIMARY KEY/UNIQUE constraints ducklake? ? {} : { primary_key: true } end |
#lookup_cast_type_from_column(sql_type_metadata) ⇒ ActiveRecord::Type::Value
Looks up the appropriate cast type for a column based on SQL type metadata
390 391 392 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 390 def lookup_cast_type_from_column() lookup_cast_type(.sql_type) end |
#next_sequence_value(sequence_name) ⇒ String
Returns SQL expression to get the next value from a sequence
132 133 134 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 132 def next_sequence_value(sequence_name) "nextval(#{quote(sequence_name)})" end |
#partition_expressions(table_name) ⇒ Array<String>?
Returns the partition expressions for a DuckLake table
171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 171 def partition_expressions(table_name) return nil unless ducklake? # Find the metadata schema for the current database = return nil unless # Query partition info from DuckLake metadata tables # Schema: # ducklake_partition_column: partition_id, table_id, partition_key_index, column_id, transform # ducklake_column: column_id, table_id, column_name, ... # ducklake_table: table_id, table_name, ... sql = <<~SQL SELECT pc.partition_key_index, c.column_name, pc.transform FROM #{quote_table_name()}.ducklake_partition_column pc JOIN #{quote_table_name()}.ducklake_table t ON pc.table_id = t.table_id JOIN #{quote_table_name()}.ducklake_column c ON pc.column_id = c.column_id AND c.table_id = t.table_id WHERE t.table_name = #{quote(table_name.to_s)} ORDER BY pc.partition_key_index SQL result = execute(sql, 'Get Partition Expressions') expressions = result.map do |row| _index, column_name, transform = row if transform && !transform.to_s.empty? "#{transform}(#{column_name})" else column_name.to_s end end expressions.empty? ? nil : expressions rescue StandardError # If metadata tables don't exist or query fails, return nil nil end |
#quoted_scope(name = nil, type: nil) ⇒ Hash
Creates a quoted scope hash for table/schema queries with type filtering
398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 398 def quoted_scope(name = nil, type: nil) schema, name = extract_schema_qualified_name(name) # Default to 'main' schema if no schema specified to avoid returning # tables from information_schema, pg_catalog, or other attached databases schema ||= 'main' type_condition = case type when 'BASE TABLE' "table_type = 'BASE TABLE'" when 'VIEW' "table_type = 'VIEW'" else "table_type IN ('BASE TABLE', 'VIEW')" end { schema: quote(schema), name: name ? quote(name) : nil, type: type_condition } end |
#reset_sequence!(sequence_name, value = 1) ⇒ void
This method returns an undefined value.
Resets a sequence to a specific value
140 141 142 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 140 def reset_sequence!(sequence_name, value = 1) execute("ALTER SEQUENCE #{quote_table_name(sequence_name)} RESTART WITH #{value.to_i}", 'Reset Sequence') end |
#schema_creation ⇒ ActiveRecord::ConnectionAdapters::Duckdb::SchemaCreation
Returns a DuckDB-specific schema creation instance
14 15 16 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 14 def schema_creation SchemaCreation.new(self) end |
#sequence_exists?(sequence_name) ⇒ Boolean
Checks if a sequence exists in the database
109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 109 def sequence_exists?(sequence_name) # Try to get next value from sequence in a way that doesn't consume it # Use a transaction that we can rollback to avoid side effects transaction do execute("SELECT nextval(#{quote(sequence_name)})", 'SCHEMA') raise ActiveRecord::Rollback # Rollback to avoid consuming the sequence value end true rescue ActiveRecord::StatementInvalid, DuckDB::Error => e # If the sequence doesn't exist, nextval will fail with a specific error raise unless e..include?('does not exist') || e..include?('Catalog Error') false # Re-raise other types of errors rescue StandardError # For any other error, assume sequence doesn't exist false end |
#sequences ⇒ Array<String>
Returns a list of all sequences in the database
146 147 148 149 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 146 def sequences # For now, return empty array since DuckDB sequence introspection is limited [] end |
#set_ducklake_option(option_name, value, table_name = nil) ⇒ void
This method returns an undefined value.
Sets a DuckLake-specific option using CALL set_option
231 232 233 234 235 236 237 238 239 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 231 def set_ducklake_option(option_name, value, table_name = nil) formatted_value = value.is_a?(Integer) ? value.to_s : quote(value) sql = if table_name "CALL set_option(#{quote(option_name)}, #{formatted_value}, #{quote(table_name.to_s)})" else "CALL set_option(#{quote(option_name)}, #{formatted_value})" end execute(sql, 'Set DuckLake Option') end |
#set_partitioned_by(table_name, expressions) ⇒ void
This method returns an undefined value.
Sets partitioning for a DuckLake table DuckLake supports time-based partitioning using SQL functions
159 160 161 162 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 159 def set_partitioned_by(table_name, expressions) sql = "ALTER TABLE #{quote_table_name(table_name)} SET PARTITIONED BY (#{expressions.join(", ")})" execute(sql, 'Set Partitioned By') end |
#tables ⇒ Array<String>
Returns a list of all tables in the database
20 21 22 23 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 20 def tables result = execute(data_source_sql(type: 'BASE TABLE'), 'SCHEMA') result.to_a.map { |row| row[0] } end |
#type_to_sql(type, limit: nil, precision: nil, scale: nil, **options) ⇒ String
Converts ActiveRecord type to DuckDB SQL type string
429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 429 def type_to_sql(type, limit: nil, precision: nil, scale: nil, **) case type.to_s when 'primary_key' # Use the configured primary key type primary_key_type_definition when 'string', 'text' if limit "VARCHAR(#{limit})" else 'VARCHAR' end when 'integer' integer_to_sql(limit) when 'bigint' 'BIGINT' when 'float', 'real' 'REAL' when 'double' 'DOUBLE' when 'decimal', 'numeric' if precision && scale "DECIMAL(#{precision},#{scale})" elsif precision "DECIMAL(#{precision})" else 'DECIMAL' end when 'datetime', 'timestamp' 'TIMESTAMP' when 'time' 'TIME' when 'date' 'DATE' when 'boolean' 'BOOLEAN' when 'binary', 'blob' # TODO: Add blob size limits # Postgres has limits set on blob sized # https://github.com/rails/rails/blob/82e9029bbf63a33b69f007927979c5564a6afe9e/activerecord/lib/active_record/connection_adapters/postgresql/schema_statements.rb#L855 # Duckdb has a 4g size limit as well - https://duckdb.org/docs/stable/sql/data_types/blob 'BLOB' when 'uuid' 'UUID' # DuckDB-specific signed integer types when 'tinyint' 'TINYINT' when 'smallint' 'SMALLINT' when 'hugeint' 'HUGEINT' # DuckDB-specific unsigned integer types when 'utinyint' 'UTINYINT' when 'usmallint' 'USMALLINT' when 'uinteger' 'UINTEGER' when 'ubigint' 'UBIGINT' when 'uhugeint' 'UHUGEINT' # DuckDB interval type for time periods when 'interval' 'INTERVAL' else super end end |
#views ⇒ Array<String>
Returns a list of all views in the database
27 28 29 30 |
# File 'lib/active_record/connection_adapters/duckdb/schema_statements.rb', line 27 def views result = execute(data_source_sql(type: 'VIEW'), 'SCHEMA') result.to_a.map { |row| row[0] } end |