nim-rocksdb/examples/simple_example.nim

90 lines
2.7 KiB
Nim
Raw Permalink Normal View History

Rocksdb rewrite integration (#38) * Column families support (#34) * Update library to support column families. If not specified, uses the 'default' column family. * Added tests for column family changes. * Update library version and readme. * Updated the librocksdb c library to the latest stable version. * Started rewrite of library. * Commit library rewrite progress. * Completed initial rewrite and refactored tests. * Completed implementation of backup engine. * Added tests for new types. * Completed tests for existing features. * Remove features not supported by older versions of RocksDB to fix CI (temporary fix). * Remove flush before backup support from BackupEngine to fix CI. * Transactions support (#36) * Update library to support column families. If not specified, uses the 'default' column family. * Added tests for column family changes. * Update library version and readme. * Updated the librocksdb c library to the latest stable version. * Started rewrite of library. * Commit library rewrite progress. * Completed initial rewrite and refactored tests. * Completed implementation of backup engine. * Added tests for new types. * Completed tests for existing features. * Remove features not supported by older versions of RocksDB to fix CI (temporary fix). * Remove flush before backup support from BackupEngine to fix CI. * Implemented RocksDB iterator. * Implemented pairs iterator. * Completed implementation of WriteBatch with tests. * Fix example code. * Completed implementation of TransactionDb. * Support setting default column family. * Remove unneeded usage of var for ref type parameters. * Completed transactiondb tests. * Improve and refactor rocksdb test. * Added support for ingesting sst files using the SstFileWriter. (#37) * Create ColFamilyReadOnly and ColFamilyReadWrite types for using a specific column family. * Use inline pragma for small procs and add lock to RocksDbRef type close to prevent double free. * Added documentation for the public API. * Initial implementation of sst filewriter. * Added tests for sstfilewriter. * Documentation minor improvements.
2024-03-05 11:12:37 +08:00
import ../rocksdb/lib/librocksdb, cpuinfo
2018-01-27 17:01:14 +01:00
const
dbPath: cstring = "/tmp/rocksdb_simple_example"
dbBackupPath: cstring = "/tmp/rocksdb_simple_example_backup"
proc main() =
var
Rocksdb rewrite integration (#38) * Column families support (#34) * Update library to support column families. If not specified, uses the 'default' column family. * Added tests for column family changes. * Update library version and readme. * Updated the librocksdb c library to the latest stable version. * Started rewrite of library. * Commit library rewrite progress. * Completed initial rewrite and refactored tests. * Completed implementation of backup engine. * Added tests for new types. * Completed tests for existing features. * Remove features not supported by older versions of RocksDB to fix CI (temporary fix). * Remove flush before backup support from BackupEngine to fix CI. * Transactions support (#36) * Update library to support column families. If not specified, uses the 'default' column family. * Added tests for column family changes. * Update library version and readme. * Updated the librocksdb c library to the latest stable version. * Started rewrite of library. * Commit library rewrite progress. * Completed initial rewrite and refactored tests. * Completed implementation of backup engine. * Added tests for new types. * Completed tests for existing features. * Remove features not supported by older versions of RocksDB to fix CI (temporary fix). * Remove flush before backup support from BackupEngine to fix CI. * Implemented RocksDB iterator. * Implemented pairs iterator. * Completed implementation of WriteBatch with tests. * Fix example code. * Completed implementation of TransactionDb. * Support setting default column family. * Remove unneeded usage of var for ref type parameters. * Completed transactiondb tests. * Improve and refactor rocksdb test. * Added support for ingesting sst files using the SstFileWriter. (#37) * Create ColFamilyReadOnly and ColFamilyReadWrite types for using a specific column family. * Use inline pragma for small procs and add lock to RocksDbRef type close to prevent double free. * Added documentation for the public API. * Initial implementation of sst filewriter. * Added tests for sstfilewriter. * Documentation minor improvements.
2024-03-05 11:12:37 +08:00
db: ptr rocksdb_t
be: ptr rocksdb_backup_engine_t
2018-01-27 17:01:14 +01:00
options = rocksdb_options_create()
# Optimize RocksDB. This is the easiest way to
# get RocksDB to perform well
let cpus = countProcessors()
rocksdb_options_increase_parallelism(options, cpus.int32)
# This requires snappy - disabled because rocksdb is not always compiled with
# snappy support (for example Fedora 28, certain Ubuntu versions)
# rocksdb_options_optimize_level_style_compaction(options, 0);
2018-01-27 17:01:14 +01:00
# create the DB if it's not already present
rocksdb_options_set_create_if_missing(options, 1)
2018-01-27 17:01:14 +01:00
var # open DB
err: cstring # memory leak: example code does not free error string!
Rocksdb rewrite integration (#38) * Column families support (#34) * Update library to support column families. If not specified, uses the 'default' column family. * Added tests for column family changes. * Update library version and readme. * Updated the librocksdb c library to the latest stable version. * Started rewrite of library. * Commit library rewrite progress. * Completed initial rewrite and refactored tests. * Completed implementation of backup engine. * Added tests for new types. * Completed tests for existing features. * Remove features not supported by older versions of RocksDB to fix CI (temporary fix). * Remove flush before backup support from BackupEngine to fix CI. * Transactions support (#36) * Update library to support column families. If not specified, uses the 'default' column family. * Added tests for column family changes. * Update library version and readme. * Updated the librocksdb c library to the latest stable version. * Started rewrite of library. * Commit library rewrite progress. * Completed initial rewrite and refactored tests. * Completed implementation of backup engine. * Added tests for new types. * Completed tests for existing features. * Remove features not supported by older versions of RocksDB to fix CI (temporary fix). * Remove flush before backup support from BackupEngine to fix CI. * Implemented RocksDB iterator. * Implemented pairs iterator. * Completed implementation of WriteBatch with tests. * Fix example code. * Completed implementation of TransactionDb. * Support setting default column family. * Remove unneeded usage of var for ref type parameters. * Completed transactiondb tests. * Improve and refactor rocksdb test. * Added support for ingesting sst files using the SstFileWriter. (#37) * Create ColFamilyReadOnly and ColFamilyReadWrite types for using a specific column family. * Use inline pragma for small procs and add lock to RocksDbRef type close to prevent double free. * Added documentation for the public API. * Initial implementation of sst filewriter. * Added tests for sstfilewriter. * Documentation minor improvements.
2024-03-05 11:12:37 +08:00
db = rocksdb_open(options, dbPath, cast[cstringArray](err.addr))
doAssert err.isNil, $err
2018-01-27 17:01:14 +01:00
# open Backup Engine that we will use for backing up our database
Rocksdb rewrite integration (#38) * Column families support (#34) * Update library to support column families. If not specified, uses the 'default' column family. * Added tests for column family changes. * Update library version and readme. * Updated the librocksdb c library to the latest stable version. * Started rewrite of library. * Commit library rewrite progress. * Completed initial rewrite and refactored tests. * Completed implementation of backup engine. * Added tests for new types. * Completed tests for existing features. * Remove features not supported by older versions of RocksDB to fix CI (temporary fix). * Remove flush before backup support from BackupEngine to fix CI. * Transactions support (#36) * Update library to support column families. If not specified, uses the 'default' column family. * Added tests for column family changes. * Update library version and readme. * Updated the librocksdb c library to the latest stable version. * Started rewrite of library. * Commit library rewrite progress. * Completed initial rewrite and refactored tests. * Completed implementation of backup engine. * Added tests for new types. * Completed tests for existing features. * Remove features not supported by older versions of RocksDB to fix CI (temporary fix). * Remove flush before backup support from BackupEngine to fix CI. * Implemented RocksDB iterator. * Implemented pairs iterator. * Completed implementation of WriteBatch with tests. * Fix example code. * Completed implementation of TransactionDb. * Support setting default column family. * Remove unneeded usage of var for ref type parameters. * Completed transactiondb tests. * Improve and refactor rocksdb test. * Added support for ingesting sst files using the SstFileWriter. (#37) * Create ColFamilyReadOnly and ColFamilyReadWrite types for using a specific column family. * Use inline pragma for small procs and add lock to RocksDbRef type close to prevent double free. * Added documentation for the public API. * Initial implementation of sst filewriter. * Added tests for sstfilewriter. * Documentation minor improvements.
2024-03-05 11:12:37 +08:00
be = rocksdb_backup_engine_open(options, dbBackupPath, cast[cstringArray](err.addr))
doAssert err.isNil, $err
2018-01-27 17:01:14 +01:00
# Put key-value
var writeOptions = rocksdb_writeoptions_create()
let key = "key"
let put_value = "value"
rocksdb_put(
db,
writeOptions,
key.cstring,
key.len.csize_t,
put_value.cstring,
put_value.len.csize_t,
cast[cstringArray](err.addr),
)
doAssert err.isNil, $err
2018-01-27 17:01:14 +01:00
# Get value
var readOptions = rocksdb_readoptions_create()
2020-07-17 19:08:35 +08:00
var len: csize_t
let raw_value = rocksdb_get(
db,
readOptions,
key.cstring,
key.len.csize_t,
addr len,
cast[cstringArray](err.addr),
) # Important: rocksdb_get is not null-terminated
doAssert err.isNil, $err
# Copy it to a regular Nim string (copyMem workaround because raw value is NOT null-terminated)
2020-07-20 18:44:28 +08:00
var get_value = newString(len.int)
copyMem(addr get_value[0], unsafeAddr raw_value[0], len.int * sizeof(char))
doAssert get_value == put_value
2018-01-27 17:01:14 +01:00
# create new backup in a directory specified by DBBackupPath
Rocksdb rewrite integration (#38) * Column families support (#34) * Update library to support column families. If not specified, uses the 'default' column family. * Added tests for column family changes. * Update library version and readme. * Updated the librocksdb c library to the latest stable version. * Started rewrite of library. * Commit library rewrite progress. * Completed initial rewrite and refactored tests. * Completed implementation of backup engine. * Added tests for new types. * Completed tests for existing features. * Remove features not supported by older versions of RocksDB to fix CI (temporary fix). * Remove flush before backup support from BackupEngine to fix CI. * Transactions support (#36) * Update library to support column families. If not specified, uses the 'default' column family. * Added tests for column family changes. * Update library version and readme. * Updated the librocksdb c library to the latest stable version. * Started rewrite of library. * Commit library rewrite progress. * Completed initial rewrite and refactored tests. * Completed implementation of backup engine. * Added tests for new types. * Completed tests for existing features. * Remove features not supported by older versions of RocksDB to fix CI (temporary fix). * Remove flush before backup support from BackupEngine to fix CI. * Implemented RocksDB iterator. * Implemented pairs iterator. * Completed implementation of WriteBatch with tests. * Fix example code. * Completed implementation of TransactionDb. * Support setting default column family. * Remove unneeded usage of var for ref type parameters. * Completed transactiondb tests. * Improve and refactor rocksdb test. * Added support for ingesting sst files using the SstFileWriter. (#37) * Create ColFamilyReadOnly and ColFamilyReadWrite types for using a specific column family. * Use inline pragma for small procs and add lock to RocksDbRef type close to prevent double free. * Added documentation for the public API. * Initial implementation of sst filewriter. * Added tests for sstfilewriter. * Documentation minor improvements.
2024-03-05 11:12:37 +08:00
rocksdb_backup_engine_create_new_backup(be, db, cast[cstringArray](err.addr))
doAssert err.isNil, $err
2018-01-27 17:01:14 +01:00
rocksdb_close(db)
# If something is wrong, you might want to restore data from last backup
var restoreOptions = rocksdb_restore_options_create()
rocksdb_backup_engine_restore_db_from_latest_backup(
be, dbPath, dbPath, restoreOptions, cast[cstringArray](err.addr)
)
doAssert err.isNil, $err
2018-01-27 17:01:14 +01:00
rocksdb_restore_options_destroy(restore_options)
Rocksdb rewrite integration (#38) * Column families support (#34) * Update library to support column families. If not specified, uses the 'default' column family. * Added tests for column family changes. * Update library version and readme. * Updated the librocksdb c library to the latest stable version. * Started rewrite of library. * Commit library rewrite progress. * Completed initial rewrite and refactored tests. * Completed implementation of backup engine. * Added tests for new types. * Completed tests for existing features. * Remove features not supported by older versions of RocksDB to fix CI (temporary fix). * Remove flush before backup support from BackupEngine to fix CI. * Transactions support (#36) * Update library to support column families. If not specified, uses the 'default' column family. * Added tests for column family changes. * Update library version and readme. * Updated the librocksdb c library to the latest stable version. * Started rewrite of library. * Commit library rewrite progress. * Completed initial rewrite and refactored tests. * Completed implementation of backup engine. * Added tests for new types. * Completed tests for existing features. * Remove features not supported by older versions of RocksDB to fix CI (temporary fix). * Remove flush before backup support from BackupEngine to fix CI. * Implemented RocksDB iterator. * Implemented pairs iterator. * Completed implementation of WriteBatch with tests. * Fix example code. * Completed implementation of TransactionDb. * Support setting default column family. * Remove unneeded usage of var for ref type parameters. * Completed transactiondb tests. * Improve and refactor rocksdb test. * Added support for ingesting sst files using the SstFileWriter. (#37) * Create ColFamilyReadOnly and ColFamilyReadWrite types for using a specific column family. * Use inline pragma for small procs and add lock to RocksDbRef type close to prevent double free. * Added documentation for the public API. * Initial implementation of sst filewriter. * Added tests for sstfilewriter. * Documentation minor improvements.
2024-03-05 11:12:37 +08:00
db = rocksdb_open(options, dbPath, cast[cstringArray](err.addr))
doAssert err.isNil, $err
2018-01-27 17:01:14 +01:00
# cleanup
rocksdb_writeoptions_destroy(writeOptions)
rocksdb_readoptions_destroy(readOptions)
rocksdb_options_destroy(options)
rocksdb_backup_engine_close(be)
rocksdb_close(db)
2020-07-17 19:08:35 +08:00
main()