Module: ClickHouse::HTTPClient

Defined in:
lib/clickhouse/http_client.rb,
lib/clickhouse/http_client/error.rb,
lib/clickhouse/http_client/query.rb,
lib/clickhouse/http_client/database.rb,
lib/clickhouse/http_client/response_formatter.rb,
lib/clickhouse/http_client/value_normalization.rb,
lib/clickhouse/http_client/parameter_serializer.rb

Overview

HTTP transport for ClickHouse queries: owns database configuration, HTTP POST mechanics, parameter serialization, response typecasting, logging, and instrumentation.

Queries are duck-typed: any object responding to #to_sql (SQL text with ClickHouse {name:Type} placeholders intact) and #placeholders (a Hash of placeholder names to Ruby values) is accepted, as is a plain String of trusted SQL. ClickHouse::SQL::Query satisfies this protocol; so does the HTTPClient::Query value object. An object that also responds to #to_query (e.g. ClickHouse::SQL::SelectQuery) is compiled through it first, so builder-level validations always apply.

nil placeholder values are always sent as NULL. Binding nil to a non-Nullable placeholder type fails loudly in ClickHouse for most types, but parses as an empty string for a plain String placeholder — callers (e.g. ClickHouse::SQL) are expected to validate nil against placeholder types before execution.

Derived in part from GitLab's MIT-licensed click_house-client gem; see LICENSE.

Defined Under Namespace

Modules: ParameterSerializer Classes: Database, ExecutionResult, Query

Constant Summary collapse

NOTIFICATION_NAME =

Instrumentation event emitted around every query.

"sql.click_house"
Error = 
Class.new(StandardError)
ConfigurationError = 
Class.new(Error)
DatabaseError = 
Class.new(Error)
QueryError = 
Class.new(Error)
ConnectionError =

Transport failures are re-raised as these wrappers (with the original Net::HTTP error as the cause) so callers rescue one hierarchy without knowing the underlying HTTP library. TimeoutError is the client giving up waiting; a server-side kill (max_execution_time) surfaces as a DatabaseError mentioning TIMEOUT_EXCEEDED instead.

Class.new(Error)
TimeoutError = 
Class.new(Error)

Class Attribute Summary collapse

Class Method Summary collapse

Class Attribute Details

.instrumenter ⇒ Object 



70
71
72
 
# File 'lib/clickhouse/http_client.rb', line 70

def instrumenter
  @instrumenter || NOOP_INSTRUMENTER # rubocop:disable ThreadSafety/ClassInstanceVariable
end

.logger ⇒ Object 



63
64
65
 
# File 'lib/clickhouse/http_client.rb', line 63

def logger
  @logger ||= Logger.new(IO::NULL) # rubocop:disable ThreadSafety/ClassInstanceVariable
end

Class Method Details

.database(name) ⇒ ClickHouse::HTTPClient::Database

Parameters:

  • name (Symbol) —

    Registered database name.

Returns:



83
84
85
86
87
 
# File 'lib/clickhouse/http_client.rb', line 83

def database(name)
  databases.fetch(name) do
    raise ConfigurationError, "The database '#{name}' is not registered"
  end
end

.execute(query, database, timeout: nil) ⇒ ClickHouse::HTTPClient::ExecutionResult

Executes any query without returning row data (INSERT, DELETE, DDL).

Returns:



109
110
111
112
113
114
115
116
117
118
119
120
121
 
# File 'lib/clickhouse/http_client.rb', line 109

def execute(query, database, timeout: nil)
  query = Query.wrap(query)
  body, content_type = encode_multipart(form_data(query))

  run(query, database, timeout: timeout, body: body, extra_headers: { "Content-Type" => content_type }) do |response|
    statistics = summary_statistics(response)
    result = ExecutionResult.new(
      query_id: response[QUERY_ID_HEADER],
      statistics: statistics,
    )
    [result, statistics]
  end
end

.insert_csv(query, io, database, timeout: nil) ⇒ ClickHouse::HTTPClient::ExecutionResult

Inserts a gzip-compressed CSV IO to ClickHouse.

Usage:

io = StringIO.new(gzip_compressed_csv)
ClickHouse::HTTPClient.insert_csv("INSERT INTO events (id) FORMAT CSV", io, :main)

Returns:



131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
 
# File 'lib/clickhouse/http_client.rb', line 131

def insert_csv(query, io, database, timeout: nil)
  query = Query.wrap(query)

  # The request body carries the CSV data, so there's nowhere to send
  # param_* placeholder values; ClickHouse would receive unresolved
  # {name:Type} markers in the SQL.
  unless query.placeholders.empty?
    raise QueryError, "insert_csv does not support query placeholders"
  end

  run(query, database, timeout: timeout, body: io, extra_headers: CSV_HEADERS, uri_variables: { query: query.sql }) do |response|
    statistics = summary_statistics(response)
    result = ExecutionResult.new(
      query_id: response[QUERY_ID_HEADER],
      statistics: statistics,
    )
    [result, statistics]
  end
end

.register_database(name, **args) ⇒ Object

Registers a database connection under a name.

Raises:



75
76
77
78
79
 
# File 'lib/clickhouse/http_client.rb', line 75

def register_database(name, **args)
  raise ConfigurationError, "The database '#{name}' is already registered" if databases.key?(name)

  databases[name] = Database.build(**args)
end

.select(query, database, timeout: nil, &formatter) ⇒ Object

Executes a SELECT query and returns rows as an Array of Hashes, with values typecast according to the column types in the response. Callers migrating an existing result contract can supply a formatter block, which receives the parsed response including its column metadata.



93
94
95
96
97
98
99
100
101
102
103
104
 
# File 'lib/clickhouse/http_client.rb', line 93

def select(query, database, timeout: nil, &formatter)
  query = Query.wrap(query)
  body, content_type = encode_multipart(form_data(query))

  run(query, database, timeout: timeout, body: body, extra_headers: { "Content-Type" => content_type }) do |response|
    parsed = parse_result_body(response)

    statistics = ValueNormalization.symbolize_keys(parsed["statistics"]) if parsed["statistics"]
    rows = formatter ? formatter.call(parsed) : ResponseFormatter.format_rows(parsed)
    [rows, statistics]
  end
end