Module: UmbrellioUtils::ClickHouse::Backends::Base::ClickHouseDatasetMethods

Defined in:
lib/umbrellio_utils/click_house/backends/base.rb

Overview

ClickHouse-specific dataset behaviour: string escaping plus grammar Sequel's Postgres dataset doesn't know about.

Instance Method Summary collapse

Instance Method Details

#deduplicate ⇒ Object

Collapse a ReplacingMergeTree's row versions by hand instead of relying on the final setting, which merges the whole table.

This is a boundary: everything chained BEFORE it goes inside the dedup subquery, everything chained AFTER applies to the result. Only immutable selectors belong before it — filtering a mutable column first can match a superseded version and resurrect a row that FINAL would have dropped. is_deleted is applied after the boundary for the same reason.



54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
 
# File 'lib/umbrellio_utils/click_house/backends/base.rb', line 54

def deduplicate
  table_name, db_name = deduplication_source
  meta = ClickHouse.table_metadata(table_name, **db_name)

  unless meta.replacing?
    raise Sequel::Error,
          "#{table_name} is a #{meta.engine}; deduplicate needs a ReplacingMergeTree"
  end

  unless meta.version
    raise Sequel::Error,
          "#{table_name} declares no version column; deduplicate needs one"
  end

  wrapped = ClickHouse.from(wrap_source(deduplicated_source(meta)))
    .clone(ch_dedup: true)
    .then { |ds| @opts[:select] ? ds.select(*@opts[:select]) : ds }

  meta.is_deleted ? wrapped.where(meta.is_deleted => 0) : wrapped
end

#limit_by(*exprs, rows: 1) ⇒ Object

ClickHouse LIMIT n BY expr, ... — keeps the first n rows per distinct combination of the expressions, applied after ORDER BY.

Raises:

  • (Sequel::Error) 


28
29
30
31
 
# File 'lib/umbrellio_utils/click_house/backends/base.rb', line 28

def limit_by(*exprs, rows: 1)
  raise Sequel::Error, "limit_by requires at least one expression" if exprs.empty?
  clone(limit_by: { exprs:, rows: })
end

#literal_string_append(sql, str) ⇒ Object

ClickHouse uses C-style escape sequences in string literals, so backslashes must be doubled. Sequel's default (Postgres) escaping only escapes single-quotes.



22
23
24
 
# File 'lib/umbrellio_utils/click_house/backends/base.rb', line 22

def literal_string_append(sql, str)
  sql << "'" << str.gsub("\\") { "\\\\" }.gsub("'", "''") << "'"
end

#select_limit_sql(sql) ⇒ Object

LIMIT n BY precedes the regular LIMIT/OFFSET in ClickHouse.



34
35
36
37
38
39
40
41
42
43
 
# File 'lib/umbrellio_utils/click_house/backends/base.rb', line 34

def select_limit_sql(sql)
  if (limit_by = @opts[:limit_by])
    sql << " LIMIT "
    literal_append(sql, limit_by[:rows])
    sql << " BY "
    expression_list_append(sql, limit_by[:exprs])
  end

  super
end