Module: Tina4::DocStore

Defined in:
lib/tina4/docstore.rb

Defined Under Namespace

Classes: Cursor, DeleteResult, DocStoreDriverMissing, InsertManyResult, InsertOneResult, InvalidId, MongoCollection, MongoView, ObjectId, SqliteCollection, SqliteDatabase, UpdateResult

Constant Summary collapse

OID_RE = 
/\A[0-9a-fA-F]{24}\z/.freeze
ISO_RE = 
/\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:Z|[+-]\d{2}:\d{2})?\z/.freeze
COMPARATORS =

-- Query translation: Mongo filter Hash -> SQL WHERE over json_extract ----

{ "$gt" => ">", "$gte" => ">=", "$lt" => "<", "$lte" => "<=" }.freeze

Class Method Summary collapse

Class Method Details

.any_element(field, condition) ⇒ Object

True when the field is an ARRAY and any element satisfies condition.

MongoDB's rule for an array-valued field is that a condition matches when ANY ELEMENT matches it. json_each yields one row per element, so EXISTS is the direct translation.

The = 'array' guard is load-bearing: json_each over an OBJECT iterates its VALUES, and Mongo never matches an object field against one of its values - => "x" must NOT match => {"city" => "x"}.



217
218
219
 
# File 'lib/tina4/docstore.rb', line 217

def any_element(field, condition)
  "(#{json_type(field)} = 'array' AND EXISTS (SELECT 1 FROM #{json_each(field)} WHERE #{condition}))"
end

.apply_update(doc, update) ⇒ Object 



399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
 
# File 'lib/tina4/docstore.rb', line 399

def apply_update(doc, update)
  update = update.transform_keys(&:to_s)
  unless update.keys.any? { |k| k.start_with?("$") }
    # Full-document replace (keep the existing _id).
    new_doc = deep_string_keys(update)
    new_doc["_id"] = doc["_id"] unless new_doc.key?("_id")
    return new_doc
  end

  new_doc = doc.dup
  update.each do |op, fields|
    case op
    when "$set"
      fields.each { |k, v| set_path(new_doc, k.to_s, v) }
    when "$unset"
      fields.each_key { |k| unset_path(new_doc, k.to_s) }
    when "$inc"
      fields.each { |k, v| set_path(new_doc, k.to_s, (get_path(new_doc, k.to_s) || 0) + v) }
    else
      raise ArgumentError, "DocStore: unsupported update operator #{op.inspect}"
    end
  end
  new_doc
end

.bind(value) ⇒ Object

Bind a Ruby value for comparison against json_extract output.



326
327
328
329
330
331
332
333
334
 
# File 'lib/tina4/docstore.rb', line 326

def bind(value)
  case value
  when true then 1
  when false then 0
  when ObjectId, Time then encode_value(value)
  when Integer, Float, String, nil then value
  else JSON.generate(encode_value(value))
  end
end

.close_doc_store ⇒ Object

Close every DocStore connection: the SQLite store and all Mongo clients.



946
947
948
949
950
951
952
953
954
955
956
957
 
# File 'lib/tina4/docstore.rb', line 946

def close_doc_store
  @default_lock.synchronize do
    @mongo_clients.each_value do |client|
      client.close
    rescue StandardError
      nil # a close failure must not mask the caller's work
    end
    @mongo_clients.clear
    @default_db&.close
    @default_db = nil
  end
end

.compare(field, sql_op, operand) ⇒ Object

Compile field OP operand under Mongo's array rule -> [sql, params].

The <> 'array' guard on the scalar branch removes a measured FALSE POSITIVE: json_extract of an array returns its JSON TEXT, and SQLite sorts any text above any number, so => {"$gt" => 9} matched [1,2,3].



238
239
240
241
242
 
# File 'lib/tina4/docstore.rb', line 238

def compare(field, sql_op, operand)
  ex = extract(field)
  ["((#{json_type(field)} <> 'array' AND #{ex} #{sql_op} ?) OR #{any_element(field, "value #{sql_op} ?")})",
   [bind(operand), bind(operand)]]
end

.compile_filter(query) ⇒ Object

Compile a Mongo-style filter Hash into [sql_fragment, params]. Returns ["1=1", []] for an empty filter. Supports implicit AND across keys, $or / $and, and the per-field operator set.



247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
 
# File 'lib/tina4/docstore.rb', line 247

def compile_filter(query)
  return ["1=1", []] if query.nil? || query.empty?

  clauses = []
  params = []
  query.each do |key, value|
    key = key.to_s
    if key == "$or" || key == "$and"
      joiner = key == "$or" ? " OR " : " AND "
      subs = []
      Array(value).each do |sub|
        frag, p = compile_filter(sub)
        subs << "(#{frag})"
        params.concat(p)
      end
      clauses << "(#{subs.join(joiner)})" unless subs.empty?
      next
    end

    if value.is_a?(Hash) && !value.empty? && value.keys.all? { |k| k.to_s.start_with?("$") }
      value.each do |op, operand|
        frag, p = compile_op(key, op.to_s, operand)
        clauses << frag
        params.concat(p)
      end
    else
      # equality - the same helper $eq uses, so the array rule applies
      # whether the filter reads {"tags" => "x"} or {"tags" => {"$eq" => "x"}}
      frag, p = equality(key, value)
      clauses << frag
      params.concat(p)
    end
  end

  [clauses.empty? ? "1=1" : clauses.join(" AND "), params]
end

.compile_op(field, op, operand) ⇒ Object 



284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
 
# File 'lib/tina4/docstore.rb', line 284

def compile_op(field, op, operand)
  ex = extract(field)
  return compare(field, COMPARATORS[op], operand) if COMPARATORS.key?(op)

  case op
  when "$eq"
    equality(field, operand)
  when "$ne"
    return ["#{ex} IS NOT NULL", []] if operand.nil?

    sql, p = equality(field, operand)
    # A MISSING field satisfies $ne in Mongo, and SQL's NOT(NULL) is NULL
    # rather than true - so the IS NULL arm is required, not decoration.
    ["(NOT (#{sql}) OR #{ex} IS NULL)", p]
  when "$in"
    items = Array(operand)
    return ["0", []] if items.empty?

    placeholders = (["?"] * items.length).join(",")
    bound = items.map { |v| bind(v) }
    ["(#{ex} IN (#{placeholders}) OR #{any_element(field, "value IN (#{placeholders})")})", bound + bound]
  when "$nin"
    items = Array(operand)
    return ["1", []] if items.empty?

    placeholders = (["?"] * items.length).join(",")
    bound = items.map { |v| bind(v) }
    ["(NOT (#{ex} IN (#{placeholders}) OR #{any_element(field, "value IN (#{placeholders})")}) OR #{ex} IS NULL)",
     bound + bound]
  when "$exists"
    # json_type is NULL when the path is absent; present-but-null still has a type.
    [operand ? "#{json_type(field)} IS NOT NULL" : "#{json_type(field)} IS NULL", []]
  when "$regex"
    pattern = operand.is_a?(Hash) ? operand["$regex"].to_s : operand.to_s
    ["((#{json_type(field)} <> 'array' AND #{ex} REGEXP ?) OR #{any_element(field, "value REGEXP ?")})",
     [pattern, pattern]]
  else
    raise ArgumentError, "DocStore: unsupported query operator #{op.inspect}"
  end
end

.decode_value(value) ⇒ Object

Stored JSON value -> Ruby, rehydrating ObjectId (24-hex) and Time (ISO).



160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
 
# File 'lib/tina4/docstore.rb', line 160

def decode_value(value)
  case value
  when String
    if value.match?(OID_RE)
      ObjectId.new(value)
    elsif value.match?(ISO_RE)
      begin
        Time.parse(value)
      rescue ArgumentError
        value
      end
    else
      value
    end
  when Hash  then value.each_with_object({}) { |(k, v), h| h[k] = decode_value(v) }
  when Array then value.map { |v| decode_value(v) }
  else value
  end
end

.deep_string_keys(value) ⇒ Object 



424
425
426
427
428
429
430
 
# File 'lib/tina4/docstore.rb', line 424

def deep_string_keys(value)
  case value
  when Hash  then value.each_with_object({}) { |(k, v), h| h[k.to_s] = deep_string_keys(v) }
  when Array then value.map { |v| deep_string_keys(v) }
  else value
  end
end

.default_db ⇒ Object 



884
885
886
887
888
889
890
 
# File 'lib/tina4/docstore.rb', line 884

def default_db
  return @default_db if @default_db

  @default_lock.synchronize do
    @default_db ||= SqliteDatabase.new
  end
end

.dump_doc(document) ⇒ Object

Encode a document for storage. Pure, for the same reason as load_doc.



376
 
# File 'lib/tina4/docstore.rb', line 376

def dump_doc(document) = JSON.generate(encode_value(document))

.encode_value(value) ⇒ Object

Ruby value -> JSON-serialisable, sortable scalar form (for storage/queries).



149
150
151
152
153
154
155
156
157
 
# File 'lib/tina4/docstore.rb', line 149

def encode_value(value)
  case value
  when ObjectId then value.to_s
  when Time     then iso(value)
  when Hash     then value.each_with_object({}) { |(k, v), h| h[k.to_s] = encode_value(v) }
  when Array    then value.map { |v| encode_value(v) }
  else value
  end
end

.equality(field, operand) ⇒ Object

Compile field == operand under Mongo's array rule -> [sql, params].



222
223
224
225
226
227
228
229
230
231
 
# File 'lib/tina4/docstore.rb', line 222

def equality(field, operand)
  ex = extract(field)
  return ["#{ex} IS NULL", []] if operand.nil?

  # An Array or Hash operand compares against the WHOLE value, never
  # element-wise: {"tags" => ["x","y"]} is exact-array equality.
  return ["#{ex} = ?", [bind(operand)]] if operand.is_a?(Array) || operand.is_a?(Hash)

  ["(#{ex} = ? OR #{any_element(field, "value = ?")})", [bind(operand), bind(operand)]]
end

.extract(field) ⇒ Object 



201
 
# File 'lib/tina4/docstore.rb', line 201

def extract(field) = "json_extract(doc, '#{json_path(field)}')"

.get_collection(name) ⇒ Object

Return a collection for name.

A real Mongo driver Collection when a Mongo URI is configured (and the mongo gem is installed); otherwise a SqliteCollection backed by the local SQLite file. Same call sites either way - only the backend differs.

The Mongo collection is handed back through MongoCollection, a SimpleDelegator that adds the uniform Tina4 spellings and forwards everything else untouched (ADR-0035). What this method RETURNS is the surface a call site sees, so it is the surface the contract compares.



902
903
904
905
906
907
 
# File 'lib/tina4/docstore.rb', line 902

def get_collection(name)
  return default_db.get_collection(name) if serverless?

  db_name = ENV["TINA4_MONGO_DB"] || ENV["TINA4_SESSION_MONGO_DB"] || "tina4"
  MongoCollection.new(mongo_client(mongo_uri, db_name)[name])
end

.get_path(doc, dotted) ⇒ Object 



452
453
454
455
456
457
458
459
460
461
 
# File 'lib/tina4/docstore.rb', line 452

def get_path(doc, dotted)
  parts = dotted.split(".")
  node = doc
  parts.each do |p|
    return nil unless node.is_a?(Hash)

    node = node[p]
  end
  node
end

.id_key(value) ⇒ Object

Canonical string key for the _id column.



181
182
183
184
185
186
187
 
# File 'lib/tina4/docstore.rb', line 181

def id_key(value)
  case value
  when ObjectId then value.to_s
  when Time     then iso(value)
  else value.to_s
  end
end

.iso(time) ⇒ Object

-- Value encoding: keep scalars queryable, rehydrate types on read --------



143
144
145
146
 
# File 'lib/tina4/docstore.rb', line 143

def iso(time)
  time.getutc.strftime("%Y-%m-%dT%H:%M:%S") +
    (time.getutc.subsec.zero? ? "" : format(".%06d", time.getutc.usec)) + "Z"
end

.json_each(field) ⇒ Object

A rowset over the field: one row per element of an array, one for a scalar.



206
 
# File 'lib/tina4/docstore.rb', line 206

def json_each(field) = "json_each(doc, '#{json_path(field)}')"

.json_path(field) ⇒ Object

Field name -> a JSON path. Dotted names address nested keys.



194
195
196
197
198
199
 
# File 'lib/tina4/docstore.rb', line 194

def json_path(field)
  segments = field.to_s.split(".").map do |s|
    s.match?(/\A[A-Za-z_][A-Za-z0-9_]*\z/) ? s : "\"#{s}\""
  end
  "$." + segments.join(".")
end

.json_type(field) ⇒ Object 



203
 
# File 'lib/tina4/docstore.rb', line 203

def json_type(field) = "json_type(doc, '#{json_path(field)}')"

.load_doc(doc_text, projection = nil) ⇒ Object

Decode a stored JSON document, rehydrating ObjectId and Time values.

Module-level because it is a PURE function of its inputs. It was a public collection method only so the Cursor could reach it, which ADR-0025 corollary 1 forbids.



370
371
372
373
 
# File 'lib/tina4/docstore.rb', line 370

def load_doc(doc_text, projection = nil)
  doc = decode_value(JSON.parse(doc_text))
  projection ? project(doc, projection) : doc
end

.mongo_client(uri, db_name) ⇒ Object

Return the shared Mongo client for this (uri, database), connecting once.

MEASURED 2026-08-03 against a real MongoDB: get_collection used to build a new Mongo::Client on EVERY call and never close it, so 20 calls left 60 server connections open and the count grew without bound. It was invisible in development because the SQLite fallback has no connections at all - a resource leak that only exists AFTER the swap to the real provider.

A Mongo::Client is thread-safe and pools internally, so one per (uri, database) is the shape the driver itself expects. The double-checked lock matches default_db above: without it two threads racing the first call both build a client and one is orphaned - the same leak, just rarer.

A missing gem raises here, at provider RESOLUTION, before any socket is opened: it is a static fact and needs no network to establish.



926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
 
# File 'lib/tina4/docstore.rb', line 926

def mongo_client(uri, db_name)
  begin
    require "mongo"
  rescue LoadError
    source = mongo_uri_source || "TINA4_MONGO_URI"
    raise DocStoreDriverMissing,
          "Tina4 DocStore: #{source} is set, so the MongoDB provider is selected, but " \
          "its driver is not installed (gem 'mongo'). Install it with `gem install mongo`, " \
          "or unset #{source} to use the local SQLite store."
  end
  key = [uri, db_name]
  client = @mongo_clients[key]
  return client if client

  @default_lock.synchronize do
    @mongo_clients[key] ||= Mongo::Client.new(uri, database: db_name)
  end
end

.mongo_uri ⇒ Object

The configured Mongo URI, reusing the app-wide queue/session env vars.



864
865
866
867
 
# File 'lib/tina4/docstore.rb', line 864

def mongo_uri
  source = mongo_uri_source
  source ? ENV[source].strip : ""
end

.mongo_uri_source ⇒ Object

The env var that supplied the URI, or nil when none did.

Named separately so an error can tell the operator WHICH variable to unset without ever printing its value - a Mongo URI routinely carries user:password@.

Canonical TINA4_MONGO_URI, then the session-layer TINA4_SESSION_MONGO_URI; TINA4_SESSION_MONGO_URL is a legacy alias.



858
859
860
861
 
# File 'lib/tina4/docstore.rb', line 858

def mongo_uri_source
  %w[TINA4_MONGO_URI TINA4_SESSION_MONGO_URI TINA4_SESSION_MONGO_URL]
    .find { |name| !(ENV[name] || "").strip.empty? }
end

.project(doc, projection) ⇒ Object 



378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
 
# File 'lib/tina4/docstore.rb', line 378

def project(doc, projection)
  return doc if projection.nil? || projection.empty?

  proj = projection.transform_keys(&:to_s)
  include_keys = proj.select { |k, v| truthy?(v) && k != "_id" }.keys
  exclude_keys = proj.reject { |_k, v| truthy?(v) }.keys

  unless include_keys.empty?
    out = {}
    include_keys.each { |k| out[k] = doc[k] if doc.key?(k) }
    out["_id"] = doc["_id"] if proj.fetch("_id", 1) && doc.key?("_id") && truthy?(proj.fetch("_id", 1))
    return out
  end

  doc.reject { |k, _v| exclude_keys.include?(k) }
end

.regexp_match(pattern, value) ⇒ Object

REGEXP user function body, registered on the SQLite connection.



337
338
339
340
341
342
343
344
345
 
# File 'lib/tina4/docstore.rb', line 337

def regexp_match(pattern, value)
  return 0 if value.nil?

  begin
    Regexp.new(pattern.to_s).match?(value.to_s) ? 1 : 0
  rescue RegexpError
    0
  end
end

.reset_default_store ⇒ Object

Drop the cached default SQLite store (test helper).



960
961
962
963
964
965
 
# File 'lib/tina4/docstore.rb', line 960

def reset_default_store
  @default_lock.synchronize do
    @default_db&.close
    @default_db = nil
  end
end

.serverless? ⇒ Boolean

True when no Mongo is configured, so the SQLite fallback is in effect.

CONFIGURATION ONLY. Before 3.13.95 this also returned true when a URI was set but the mongo gem was absent, and that is precisely what made get_collection hand back the local SQLite store while the operator believed they were on Mongo. A missing driver is now an error (ADR-0033), not a second way to be serverless - otherwise an app branching on this would take the local path and never reach the raise.

Returns:

  • (Boolean) 


877
878
879
 
# File 'lib/tina4/docstore.rb', line 877

def serverless?
  mongo_uri.empty?
end

.set_path(doc, dotted, value) ⇒ Object 



432
433
434
435
436
437
438
439
440
 
# File 'lib/tina4/docstore.rb', line 432

def set_path(doc, dotted, value)
  parts = dotted.split(".")
  node = doc
  parts[0...-1].each do |p|
    node[p] = {} unless node[p].is_a?(Hash)
    node = node[p]
  end
  node[parts[-1]] = value
end

.sort_spec(key_or_list, direction = 1) ⇒ Object

Normalise the driver's three sort spellings to an array of [key, direction].

ADR-0036. Mongo::Collection::View#sort takes ONE spec document; the framework documents sort(key, direction); and a list of pairs is the third form pymongo and the Node driver both take. Measured 2026-08-04 against a real MongoDB, ALL THREE had to work on both providers before one spelling could be called portable: the two-argument form raised ArgumentError on the Ruby driver, and the pairs form reached the server as an ARRAY and came back "[14:TypeMismatch]: Expected field sort to be of type object".



356
357
358
359
360
361
 
# File 'lib/tina4/docstore.rb', line 356

def sort_spec(key_or_list, direction = 1)
  case key_or_list
  when String, Symbol then [[key_or_list.to_s, direction]]
  else key_or_list.map { |key, sort_direction| [key.to_s, sort_direction] }
  end
end

.truthy?(value) ⇒ Boolean

Returns:

  • (Boolean) 


395
396
397
 
# File 'lib/tina4/docstore.rb', line 395

def truthy?(value)
  !(value.nil? || value == false || value == 0)
end

.unset_path(doc, dotted) ⇒ Object 



442
443
444
445
446
447
448
449
450
 
# File 'lib/tina4/docstore.rb', line 442

def unset_path(doc, dotted)
  parts = dotted.split(".")
  node = doc
  parts[0...-1].each do |p|
    node = node[p]
    return unless node.is_a?(Hash)
  end
  node.delete(parts[-1]) if node.is_a?(Hash)
end