Commit Graph

97 Commits

Author SHA1 Message Date
Jacek Sieka 17eb247702
expose memtable flush (#72)
* expose memtable flush

This can be used to avoid log replays on startup, for example

* Add flush to RocksDbReadWriteRef.

* Remove atomic flush default.

---------

Co-authored-by: bhartnett <51288821+bhartnett@users.noreply.github.com>
2024-10-29 08:14:36 +08:00
bhartnett 7b41c5c1e5
Update nim versions and macos builder (#71) 2024-10-07 12:49:48 +08:00
bhartnett 1e0d26211f
Update RocksDb to v9.6.1 (#70)
* Update RocksDb to v9.6.1
2024-10-01 11:22:08 +08:00
web3-developer 2ae39ceb66
Bump RocksDb to v9.2.1 (#69)
* Bump RocksDb dependencies to v9.2.1

* Debug dlOpen.

* Static link dependencies.
2024-08-15 22:08:21 +08:00
Kim De Mey 965029e395
Use the make question option to avoid rebuild on macOS (#68)
There was already the earlier strange ` before the
"unity.a' is up to date message on macOS.

Turns out it might still not work on macOS because of certain
verbosity levels.

So use the question option instead of dry-run + grep as the
latter is more brittle, and the former is intended for this
purpose.
2024-07-26 16:18:18 +02:00
web3-developer 608d10a24e
Use nph v0.6.0 in nimble format and run format on project. (#66) 2024-07-22 21:05:02 +08:00
web3-developer 9a6d85e8da
Fix issue on MacOS where build is never skipped even when the build artifacts already exist. (#65) 2024-07-22 19:13:47 +08:00
web3-developer 92df0b067f
Add support for snapshots. (#64)
* Add support for snapshots.
2024-07-10 13:15:40 +08:00
web3-developer cf1267e845
Add support for optimistic transaction db, write batch with index, keyMayExist and empty keys. (#63)
* Add support for optimistic transaction db.

* Add keyMayExist to RocksDbRef.

* Add support for write batch with index.

* Allow empty keys to be used in API.
2024-07-08 22:18:30 +08:00
web3-developer 6b7de5730b
Implement missing getter and setters for option types. (#62)
* Update readme.

* Add additional opts getters and setters for ReadOptionsRef and WriteOptionsRef. Updated BackupEngineOptionsRef to use newer c library type.

* Add majority of option type getter and setters.
2024-07-03 23:46:42 +08:00
web3-developer d31b8b8a72
Build dynamic libraries for all platforms using vcpkg. (#61)
* Build dynamic libraries for all platforms using vcpkg.

* Cleanup CI.
2024-07-03 11:34:40 +08:00
web3-developer 01ced36404
Fix segfault in filterpolicy close (#60)
* Fix seq fault caused by double free. Now using API correctly.

* Add tests to cover autoClose.
2024-07-02 12:44:05 +08:00
web3-developer caedba0b3f
Implement autoclose on missed types. (#59)
* Implement autoclose on missed types.

* Cleanup using templates and add more tests.
2024-06-28 23:57:34 +08:00
web3-developer bb70e8980f
Use unity.a for building static RocksDb library. (#58)
* Use unity.a for building static RocksDb library.
2024-06-28 20:50:11 +08:00
web3-developer 03313d8068
Auto close feature to improve memory management of option types. (#57)
* Revert some changes from prior PR https://github.com/status-im/nim-rocksdb/pull/48 which enable memory leak of options when not cleaned up manually.

* Clean up, add more tests and use correct free function for listColumnFamilies.

* Close opt types when opening database fails.

* Add autoClose flag to each opt type.

* Finish autoClose changes to prevent memory leaks.
2024-06-28 10:04:37 +08:00
web3-developer ee15ce027b
Update API to support passing in ColFamilyHandleRef instead of column family string. (#52)
* Refactor API to pass in column family handle instead of string.

* Check for unknown column family in getColFamilyHandle and update tests.

* Update column family function names. Remove redundant exists check.
2024-06-27 01:31:39 +08:00
web3-developer a691d5b9d2
Format using nph (#54)
* Format entire project using nph.

* Add nph lint to CI.
2024-06-26 23:00:10 +08:00
web3-developer 45f7a9266f
Update RocksDb header file and automate regeneration of wrapper. 2024-06-21 12:08:16 +08:00
Jacek Sieka f5dcb34ae8
ignore clean failures 2024-06-19 09:42:58 +02:00
Jacek Sieka d648b10e2c
options: reclassify (#51)
Several options were mistakenly classified as db options when in fact
they are column family options - the C interface does not make this
distinction and instead puts them all under one umbrella.

This PR updates options to use nim setter/getter properties allowing
both read and write of most options - lots of options are exposed both
for reading and writing, but hey, one could always add _even more_ of
them - there's certainly no lack!
2024-06-19 08:39:15 +02:00
web3-developer 93c6df0765
If vendor RocksDb files have changed the trigger a clean rebuild. (#49) 2024-06-14 22:54:29 +08:00
Jacek Sieka 293dc0745e
max_total_wal_size is a db option (#50) 2024-06-14 15:43:49 +02:00
Jordan Hrycaj c5bbf83114
Expose function for listing existing column families (#48)
* Expose function for listing existing column families

* Free CF list after use

* Update, fix error case

* Rename listRocksDbCFs -> listColumnFamilies

* Need to free each item separately

* Handle default option objects, close after use
2024-06-13 15:43:33 +00:00
web3-developer e36f454cd4
Add helper script to cleanup build artifacts. (#47) 2024-06-13 15:47:14 +08:00
web3-developer a84cf5b896
Build windows DLLs from vendor directory (#45)
* Add vcpkg as a submodule.

* Set vendor submodules to RocksDb version v9.1.0

* Update readme.
2024-06-12 21:16:06 +08:00
Jacek Sieka 138dadac9c
expose more options (#46) 2024-06-12 11:51:03 +02:00
Jacek Sieka e34c8e825c
use `MAKE` variable when available, which can set `-j` 2024-06-04 17:14:24 +02:00
Jacek Sieka 2879227085
expose additional configuration options (#43)
* caches
* table options
* spelling fixes
2024-06-04 12:06:14 +02:00
Jordan Hrycaj eb594e33b2
Add explicit seek position method to iterator (#42) 2024-04-16 16:54:55 +00:00
web3-developer f37d7d486c
Don't set default compression as it forces us to require the compression library when linking or at runtime. Let RocksBb pick the best default. (#41) 2024-03-30 00:26:12 +08:00
web3-developer 90027f1875
Support for static linking RocksDb. (#40)
* Added rocksdb as a submodule in vendor directory.

* Add support for static linking using nimble.

* Add script to build static library dependencies.

* Disable warnings as error in deps build.

* Set recommended compression options in build and in library.

* Move static linking into wrapper code.

* Conditionally set linker using when defined(macosx).

* Add support for linking windows static libraries.

* Build rocksdb static libs using vcpkg.

* Remove lib prefix from library names for windows.

* Static linking is not supported on windows. Update documentation and CI.
2024-03-27 14:18:21 +08:00
web3-developer fc2ba4a836
Update open procs to support adding the default column family if it is missing from the columnFamilies list. (#39) 2024-03-07 09:02:24 +08:00
web3-developer 5f6282e8d4
Rocksdb rewrite integration (#38)
* Column families support (#34)

* Update library to support column families. If not specified, uses the 'default' column family.

* Added tests for column family changes.

* Update library version and readme.

* Updated the librocksdb c library to the latest stable version.

* Started rewrite of library.

* Commit library rewrite progress.

* Completed initial rewrite and refactored tests.

* Completed implementation of backup engine.

* Added tests for new types.

* Completed tests for existing features.

* Remove features not supported by older versions of RocksDB to fix CI (temporary fix).

* Remove flush before backup support from BackupEngine to fix CI.

* Transactions support (#36)

* Update library to support column families. If not specified, uses the 'default' column family.

* Added tests for column family changes.

* Update library version and readme.

* Updated the librocksdb c library to the latest stable version.

* Started rewrite of library.

* Commit library rewrite progress.

* Completed initial rewrite and refactored tests.

* Completed implementation of backup engine.

* Added tests for new types.

* Completed tests for existing features.

* Remove features not supported by older versions of RocksDB to fix CI (temporary fix).

* Remove flush before backup support from BackupEngine to fix CI.

* Implemented RocksDB iterator.

* Implemented pairs iterator.

* Completed implementation of WriteBatch with tests.

* Fix example code.

* Completed implementation of TransactionDb.

* Support setting default column family.

* Remove unneeded usage of var for ref type parameters.

* Completed transactiondb tests.

* Improve and refactor rocksdb test.

* Added support for ingesting sst files using the SstFileWriter. (#37)

* Create ColFamilyReadOnly and ColFamilyReadWrite types for using a specific column family.

* Use inline pragma for small procs and add lock to RocksDbRef type close to prevent double free.

* Added documentation for the public API.

* Initial implementation of sst filewriter.

* Added tests for sstfilewriter.

* Documentation minor improvements.
2024-03-05 11:12:37 +08:00
andri lim 5e2b026f84
Fix broken CI (#35)
* Remove appveyor and travis scripts

* Fix broken CI

* Fix set pointer field to nil with typed nil

* Add liblz4 in CI deps of linux i386
2024-02-13 20:59:41 +07:00
Adam Spitz 724b72fe10
Made del return a bool, and added clear. (#33)
* Made del return a bool, and added clear.

What I've done here feels very awkward. Maybe I'm missing something,
but it looks to me like RocksDB doesn't support these operations in
any natural way.

These changes were made in order to get our implementation of
KvStoreRef for RocksDB working again after these changes to the
KvStoreRef interface:

8f0ae55353

I don't really recommend merging this; I think I'd prefer to just stop
trying to use this common KvStoreRef interface.

Still, if we do want to keep the common interface, I think this commit
will work well enough.

* Updated the tests to expect del to return a bool too.

* Leave the new `clear` operation unimplemented.
2023-01-17 11:37:10 -05:00
Jordan Hrycaj b7f0728225
Allow access to some details of the DB driver (#31)
why:
  Playing with SST direct import
2022-08-11 09:13:10 +01:00
Ștefan Talpalaru fb3f2c30b0
Windows static lib: link required system libs (#30) 2022-03-22 18:46:47 +01:00
Ștefan Talpalaru c381daf84f
allow statically linking librocksdb (#29) 2022-03-22 10:02:39 +01:00
Ștefan Talpalaru c565aa88b9
CI: use GitHub Actions (#28)
* CI: use GitHub Actions
2022-01-11 02:39:31 +01:00
Ștefan Talpalaru eddfa91bdd
version 0.3.1 2021-11-20 01:29:06 +01:00
Ștefan Talpalaru 6f55b371e2
Nim-1.6 fix (#27) 2021-11-20 01:19:21 +01:00
Ștefan Talpalaru 6aab1e0dd6
Merge pull request #25 from xyb/removeuglycast
remove ugly cast
2020-07-20 14:45:25 +02:00
Xie Yanbo 1717743ce7 remove ugly cast 2020-07-20 18:44:28 +08:00
andri lim 11d80282c1
Merge pull request #24 from xyb/csizet
fix csize_t issue for nim 1.2.x
2020-07-20 12:34:41 +07:00
Xie Yanbo 3a54ced19d fix csize_t issue for nim 1.2.x 2020-07-20 00:30:09 +08:00
Jacek Sieka e528ee949a
get with callback (#22)
* get with callback

avoids seq copy sometimes

* comment on defect exception safety
2020-04-23 20:55:21 +02:00
Jacek Sieka 8da2f11951
nim 1.2, port to results (#21) 2020-04-18 00:18:04 +02:00
Ștefan Talpalaru 08fec021c0
CI: update script path 2019-08-24 05:01:23 +02:00
kdeme 5b1307cb1f
Merge pull request #20 from status-im/max-open-files
Allow setting max open files in RocksDbInstance init
2019-07-16 21:12:30 +02:00
kdeme a48aaf2215
Allow setting max open files in RocksDbInstance init 2019-07-16 15:51:52 +02:00