Class: LMDB::Database

Inherits:
Object
  • Object
show all
Includes:
Enumerable
Defined in:
ext/lmdb_ext/lmdb_ext.c,
lib/lmdb/database.rb,
ext/lmdb_ext/lmdb_ext.c

Overview

An LMDB Database is a table of key-value pairs. It is stored as part of the Environment.

By default, each key in a Database maps to one value. However, a Database can be configured at creation to allow duplicate keys, in which case one key will map to multiple values.

A Database stores the keys in a sorted order. The order can also be set with options when the database is created.

The basic operations on a database are to #put, #get, and #delete records. One can also iterate through the records in a database using a Cursor.

Examples:

Typical usage

env = LMDB.new "databasedir"
db = env.database "databasename"
db.put "key1", "value1"
db.put "key2", "value2"
db.get "key1"              #=> "value1"
env.close

Instance Method Summary collapse

Instance Method Details

#[](key) ⇒ Object

Retrieve the value of a record from a database

Parameters:

  • key —

    the record key to retrieve

Returns:

  • value of the record for that key, or nil if there is no record with that key

See Also:

  • #get(key)


29
30
31
 
# File 'lib/lmdb/database.rb', line 29

def [](key)
  get(key)
end

#[]=(key, value) ⇒ Object

Set (write or update) a record in a database.

Examples:

db['a'] = 'b'     #=> 'b'
db['b'] = 1234    #=> 1234
db['a']           #=> 'b'

Parameters:

  • key —

    key for the record

  • value —

    the value of the record

Returns:

  • returns the value of the record

See Also:

  • value)


42
43
44
45
 
# File 'lib/lmdb/database.rb', line 42

def []=(key, value)
  put key, value
  value
end

#cardinality(key) ⇒ Integer

Return the cardinality (number of duplicates) of a given key. Works whether :dupsort is set or not.

Parameters:

  • key (#to_s) —

    The key in question.

Returns:

  • (Integer) —

    The number of entries under the key.



100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
 
# File 'lib/lmdb/database.rb', line 100

def cardinality(key)
  ret = 0
  maybe_txn true do
  # env.transaction do
    if get key
      if dupsort?
        cursor do |c|
          c.set key
          ret = c.count
        end
      else
        ret = 1
      end
    end
  end
  ret
end

#clear ⇒ Object

Note:

The clear happens transactionally.

Empty out the database

Returns:

  • nil



1098
1099
1100
1101
1102
1103
1104
 
# File 'ext/lmdb_ext/lmdb_ext.c', line 1098

static VALUE database_clear(VALUE self) {
        DATABASE(self, database);
        if (!active_txn(database->env))
                return call_with_transaction(database->env, self, "clear", 0, 0, 0);
        check(mdb_drop(need_txn(database->env), database->dbi, 0));
        return Qnil;
}

#cursor {|cursor| ... } ⇒ Object

Create a cursor to iterate through a database. Uses current transaction, if any. Otherwise, if called with a block, creates a new transaction for the scope of the block. Otherwise, fails.

Examples:

db = env.database "abc"
db.cursor do |c|
  key, value = c.next
  puts "#{key}: #{value}"
end

Yields:

  • (cursor) —

    A block to be executed with the cursor.

Yield Parameters:

  • cursor (Cursor) —

    The cursor to be used to iterate

See Also:



1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
 
# File 'ext/lmdb_ext/lmdb_ext.c', line 1284

static VALUE database_cursor(VALUE self) {
        DATABASE(self, database);
        if (!active_txn(database->env)) {
                if (!rb_block_given_p()) {
                        rb_raise(cError, "Must call with block or active transaction.");
                }
                return call_with_transaction(database->env, self, "cursor", 0, 0, 0);
        }

        MDB_cursor* cur;
        check(mdb_cursor_open(need_txn(database->env), database->dbi, &cur));

        Cursor* cursor;
        VALUE vcur = Data_Make_Struct(cCursor, Cursor, cursor_mark, cursor_free, cursor);
        cursor->cur = cur;
        cursor->db = self;

        if (rb_block_given_p()) {
                int exception;
                VALUE ret = rb_protect(rb_yield, vcur, &exception);
                if (exception) {
                        cursor_close(vcur);
                        rb_jump_tag(exception);
                }
                cursor_close(vcur);
                return ret;
        }
        else {
                VALUE vtxn = environment_active_txn(database->env);
                if (NIL_P(vtxn)) {
                        rb_fatal("Internal error: transaction finished unexpectedly.");
                }
                else {
                        TRANSACTION(vtxn, txn);
                        rb_ary_push(txn->cursors, vcur);
                }
        }

        return vcur;
}

#delete(key, value = nil) ⇒ Object

Deletes records from the database. This function removes key/data pairs from the database. If the database does not support sorted duplicate data items (:dupsort) the value parameter is ignored. If the database supports sorted duplicates and the value parameter is nil, all of the duplicate data items for the key will be deleted. Otherwise, if the data parameter is non-nil only the matching data item will be deleted.

Parameters:

  • key —

    The key of the record to delete.

  • value —

    The optional value of the record to delete.

Raises:

  • (Error) —

    if the specified key/value pair is not in the database.



1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
 
# File 'ext/lmdb_ext/lmdb_ext.c', line 1211

static VALUE database_delete(int argc, VALUE *argv, VALUE self) {
        DATABASE(self, database);
        if (!active_txn(database->env))
                return call_with_transaction(database->env, self, "delete", argc, argv, 0);

        VALUE vkey, vval;
        rb_scan_args(argc, argv, "11", &vkey, &vval);

        vkey = StringValue(vkey);

        MDB_val key;
        key.mv_size = RSTRING_LEN(vkey);
        key.mv_data = RSTRING_PTR(vkey);

        if (NIL_P(vval)) {
                check(mdb_del(need_txn(database->env), database->dbi, &key, 0));
        } else {
                vval = StringValue(vval);
                MDB_val value;
                value.mv_size = RSTRING_LEN(vval);
                value.mv_data = RSTRING_PTR(vval);
                check(mdb_del(need_txn(database->env), database->dbi, &key, &value));
        }

        return Qnil;
}

#delete?(key, value = nil) ⇒ void

This method returns an undefined value.

Delete the key (and optional value pair) if it exists; do not complain about missing keys.

Parameters:

  • key (#to_s) —

    The key of the record

  • value (#to_s, nil) (defaults to: nil) —

    The optional value

See Also:



174
175
176
177
178
179
180
181
182
183
 
# File 'lib/lmdb/database.rb', line 174

def delete?(key, value = nil)
  env.transaction do |txn|
    begin
      delete key, value
    rescue LMDB::Error::NOTFOUND
      txn.abort
      nil
    end
  end
end

#drop ⇒ Object

Note:

The drop happens transactionally.

Remove a database from the environment.

Returns:

  • nil



1084
1085
1086
1087
1088
1089
1090
 
# File 'ext/lmdb_ext/lmdb_ext.c', line 1084

static VALUE database_drop(VALUE self) {
        DATABASE(self, database);
        if (!active_txn(database->env))
                return call_with_transaction(database->env, self, "drop", 0, 0, 0);
        check(mdb_drop(need_txn(database->env), database->dbi, 1));
        return Qnil;
}

#dupfixed? ⇒ true, false

Returns whether the database is in :dupfixed mode.

Returns:

  • (true, false) 


1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
 
# File 'ext/lmdb_ext/lmdb_ext.c', line 1067

static VALUE database_is_dupfixed(VALUE self) {
        DATABASE(self, database);
        if (!active_txn(database->env))
                return call_with_transaction(database->env, self,
                                             "dupfixed?", 0, 0, MDB_RDONLY);
        unsigned int flags;
        check(mdb_dbi_flags(need_txn(database->env), database->dbi, &flags));

        return (flags & MDB_DUPFIXED) == 0 ? Qfalse : Qtrue;
}

#dupsort? ⇒ true, false

Returns whether the database is in :dupsort mode.

Returns:

  • (true, false) 


1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
 
# File 'ext/lmdb_ext/lmdb_ext.c', line 1051

static VALUE database_is_dupsort(VALUE self) {
        DATABASE(self, database);
        if (!active_txn(database->env))
                return call_with_transaction(database->env, self,
                                             "dupsort?", 0, 0, MDB_RDONLY);
        unsigned int flags;
        check(mdb_dbi_flags(need_txn(database->env), database->dbi, &flags));

        return (flags & MDB_DUPSORT) == 0 ? Qfalse : Qtrue;
}

#each {|i| ... } ⇒ Object

Iterate through the records in a database

Examples:

db.each do |record|
  key, value = record
  puts "at #{key}: #{value}"
end

Yields:

  • (i) —

    Gives a record [key, value] to the block

Yield Parameters:

  • i (Array) —

    The key, value pair for each record



13
14
15
16
17
18
19
20
21
22
 
# File 'lib/lmdb/database.rb', line 13

def each
  maybe_txn true do
  # env.transaction do
    cursor do |c|
      while i = c.next
        yield(i)
      end
    end
  end
end

#each_key(&block) ⇒ Enumerator

Iterate over each key in the database, skipping over duplicate records.

Returns:

  • (Enumerator) —

    in lieu of a block.



57
58
59
60
61
62
63
64
65
66
67
 
# File 'lib/lmdb/database.rb', line 57

def each_key(&block)
  return enum_for :each_key unless block_given?
  maybe_txn true do
  #env.transaction do
    cursor do |c|
      while (rec = c.next true)
        yield rec.first
      end
    end
  end
end

#each_value(key, &block) ⇒ Enumerator

Iterate over the duplicate values of a given key, using an implicit cursor. Works whether :dupsort is set or not.

Parameters:

  • key (#to_s) —

    The key in question.

Returns:

  • (Enumerator) —

    in lieu of a block.



75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
 
# File 'lib/lmdb/database.rb', line 75

def each_value(key, &block)
  return enum_for :each_value, key unless block_given?

  value = get(key) or return
  unless dupsort?
    yield value
    return
  end

  maybe_txn true do
  # env.transaction do
    cursor do |c|
      rec = c.set key
      while rec
        yield rec.last
        rec = c.next_range key
      end
    end
  end
end

#empty? ⇒ Boolean

Returns whether the database is empty.

Returns:

  • (Boolean) —

    whether the database is empty



194
195
196
 
# File 'lib/lmdb/database.rb', line 194

def empty?
  stat[:entries] == 0
end

#env ⇒ Environment

Returns the environment to which this database belongs.

Returns:

  • (Environment) —

    the environment to which this database belongs.



1329
1330
1331
1332
 
# File 'ext/lmdb_ext/lmdb_ext.c', line 1329

static VALUE database_env(VALUE self) {
        DATABASE(self, database);
        return database->env;
}

#flags ⇒ Hash

Return the flags used to open the database.

Returns:

  • (Hash) —

    The flags.



1034
1035
1036
1037
1038
1039
1040
1041
1042
 
# File 'ext/lmdb_ext/lmdb_ext.c', line 1034

static VALUE database_get_flags(VALUE self) {
        DATABASE(self, database);
        if (!active_txn(database->env))
                return call_with_transaction(database->env,
                                             self, "flags", 0, 0, MDB_RDONLY);
        unsigned int flags;
        check(mdb_dbi_flags(need_txn(database->env), database->dbi, &flags));
        return flags2hash(flags);
}

#get(key) ⇒ Object

Retrieves one value associated with this key. This function retrieves key/data pairs from the database. If the database supports duplicate keys (:dupsort) then the first data item for the key will be returned. Retrieval of other items requires the use of #cursor.

Parameters:

  • key —

    The key of the record to retrieve.



1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
 
# File 'ext/lmdb_ext/lmdb_ext.c', line 1115

static VALUE database_get(VALUE self, VALUE vkey) {
        DATABASE(self, database);
        if (!active_txn(database->env))
                return call_with_transaction(database->env, self, "get", 1, &vkey, MDB_RDONLY);

        vkey = StringValue(vkey);
        MDB_val key, value;
        key.mv_size = RSTRING_LEN(vkey);
        key.mv_data = RSTRING_PTR(vkey);

        int ret = mdb_get(need_txn(database->env), database->dbi, &key, &value);
        if (ret == MDB_NOTFOUND)
                return Qnil;
        check(ret);
        return rb_str_new(value.mv_data, value.mv_size);
}

#has?(key, value = nil) ⇒ Boolean

Test if the database has a given key (or, if opened in :dupsort, value)

Returns:

  • (Boolean) 


120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
 
# File 'lib/lmdb/database.rb', line 120

def has?(key, value = nil)
  v = get(key) or return false
  return true if value.nil? or value.to_s == v
  return false unless dupsort?

  # warn "checking dupsort value `#{value.inspect}` (#{value.class})"

  ret = false
  # read-only txn was having trouble being nested inside a read-write
  maybe_txn(true) do
  # env.transaction true do
  # env.transaction do
    cursor { |c| ret = !!c.set(key, value) }
  end
  ret
end

#keys ⇒ Array

Get the keys as an array.

Returns:

  • (Array) —

    of keys.



49
50
51
 
# File 'lib/lmdb/database.rb', line 49

def keys
  each_key.to_a
end

#put(key, value, options) ⇒ Object

Stores items into a database. This function stores key/value pairs in the database. The default behavior is to enter the new key/value pair, replacing any previously existing key if duplicates are disallowed, or adding a duplicate data item if duplicates are allowed (:dupsort).

Parameters:

  • key —

    The key of the record to set

  • value —

    The value to insert for this key

Options Hash (options):

  • :nodupdata (Boolean) —

    Enter the new key/value pair only if it does not already appear in the database. This flag may only be specified if the database was opened with :dupsort. The function will raise an Error if the key/data pair already appears in the database.

  • :nooverwrite (Boolean) —

    Enter the new key/value pair only if the key does not already appear in the database. The function will raise an {Error] if the key already appears in the database, even if the database supports duplicates (:dupsort).

  • :append (Boolean) —

    Append the given key/data pair to the end of the database. No key comparisons are performed. This option allows fast bulk loading when keys are already known to be in the correct order. Loading unsorted keys with this flag will cause data corruption.

  • :appenddup (Boolean) —

    As above, but for sorted dup data.



1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
 
# File 'ext/lmdb_ext/lmdb_ext.c', line 1165

static VALUE database_put(int argc, VALUE *argv, VALUE self) {
    DATABASE(self, database);
    if (!active_txn(database->env))
        return call_with_transaction(database->env, self, "put", argc, argv, 0);

    VALUE vkey, vval, option_hash = Qnil;
#ifdef RB_SCAN_ARGS_KEYWORDS
    rb_scan_args_kw(RB_SCAN_ARGS_LAST_HASH_KEYWORDS,
                    argc, argv, "20:", &vkey, &vval, &option_hash);
#else
    rb_scan_args(argc, argv, "20:", &vkey, &vval, &option_hash);
#endif

    int flags = 0;
    if (!NIL_P(option_hash))
        rb_hash_foreach(option_hash, (int (*)(ANYARGS))database_put_flags,
                        (VALUE)&flags);

    vkey = StringValue(vkey);
    vval = StringValue(vval);

    MDB_val key, value;
    key.mv_size = RSTRING_LEN(vkey);
    key.mv_data = RSTRING_PTR(vkey);
    value.mv_size = RSTRING_LEN(vval);
    value.mv_data = RSTRING_PTR(vval);

    check(mdb_put(need_txn(database->env), database->dbi, &key, &value, flags));
    return Qnil;
}

#put?(key, value = nil, **options) ⇒ void

This method returns an undefined value.

Conditionally put a value into the database.

Parameters:

  • key (#to_s) —

    The key of the record

  • value (#to_s, nil) (defaults to: nil) —

    the (optional) value

  • options (Hash) —

    options

See Also:



147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
 
# File 'lib/lmdb/database.rb', line 147

def put?(key, value = nil, **options)
  # early bailout
  return if value.nil?

  flags = { (dupsort? ? :nodupdata : :nooverwrite) => true }

  env.transaction do |txn|
    # begin
    put(key, value, **options.merge(flags)) unless has?(key, value)
    # rescue LMDB::Error::KEYEXIST
    #   # this should never be reached lol
    #   # warn 'lol'
    #   txn.abort
    #   nil
    # end
  end
end

#size ⇒ Object

Return how many records there are in this database.

Returns:

  • the number of records in this database



188
189
190
 
# File 'lib/lmdb/database.rb', line 188

def size
  stat[:entries]
end

#stat ⇒ Hash

Return useful statistics about a database.

  • :psize Size of a database page

  • :depth Depth (height) of the B-tree

  • :branch_pages Number of internal (non-leaf) pages

  • :leaf_pages Number of leaf pages

  • :overflow_pages Number of overflow pages

  • :entries Number of data items

Returns:

  • (Hash) —

    the statistics



1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
 
# File 'ext/lmdb_ext/lmdb_ext.c', line 1018

static VALUE database_stat(VALUE self) {
        DATABASE(self, database);
        if (!active_txn(database->env))
                return call_with_transaction(database->env,
                                             self, "stat", 0, 0, MDB_RDONLY);

        MDB_stat stat;
        check(mdb_stat(need_txn(database->env), database->dbi, &stat));
        return stat2hash(&stat);
}