Commit Graph

33 Commits

Author SHA1 Message Date
Jacek Sieka 8f4ef19fc9
sqlite: support read-only kvstores (#563)
in a read-only database, we cannot create the table but we can still
reason about elements in it - they simply don't exist
2022-11-28 21:15:53 +01:00
Justin Traglia e7d3de6ebf
Fix some typos (#557) 2022-11-16 17:44:00 +01:00
Zahary Karadjov 3a0afa4aa2 Add sqlite3.isInsideTransaction 2022-08-22 18:09:13 +07:00
tersec 6e813353eb
make some safe cstring conversions explicit (#515) 2022-06-28 13:03:02 +02:00
KonradStaniec dacf827a86
Add custom scalar functions to sqlite (#509) 2022-06-02 14:14:15 +02:00
Jacek Sieka 41b8588ade
move rocksdb to nimbus-eth1 (#460)
sqlite ftw!
2022-04-04 13:46:57 +02:00
Jacek Sieka ce4acc168c
kvstore: simplified stmt exec iterator (#467) 2022-01-28 15:23:41 +02:00
Kim De Mey 2c236f6495
Style fixes according to --styleCheck:usages (#452)
Currently only setting `--styleCheck:hint` as there are some
dependency fixes required and the compiler seems to trip over the
findnode MessageKind, findnode Message field and the findNode
proc. Also over protocol.Protocol usage.
2021-12-20 13:14:50 +01:00
Ștefan Talpalaru 2088d7568d
CI: test with multiple Nim version (#429)
* CI: test with multiple Nim version

* clean up the testing tree a little

* replace "unittest" with "unittest2"
2021-12-11 19:12:55 +01:00
Zahary Karadjov fe1bb4c4e7
Allow Sqlite keystores to be used in read-only mode
This is useful for tools such as `ncli_db` that can work with the database
of a running Nimbus instances.
2021-11-16 13:45:46 +02:00
Ștefan Talpalaru 2dfd352fd0 metrics: initialise lock for custom collector 2021-07-06 17:46:23 +03:00
Jacek Sieka 3514ee6484
sqlite: quick exec with result (#361) 2021-05-27 11:31:34 +02:00
Jacek Sieka 68e6aadc29 sqlite3: support option type
.. and any integer when writing but don't support int32 on reads -
internally sqlite will truncate on overflow which isn't nice.
2021-05-26 13:32:31 +03:00
Jacek Sieka 2a292cfb62
sqlite: fix readonly mode (#359)
* sqlite: fix readonly mode

* document kvstore versions
2021-05-25 20:57:28 +02:00
Jacek Sieka 1995afb87e
kvstore fixes (#350)
Storing large blobs in a "WITHOUT ROWID" table turns out to be extremely
slow when the tree must be rebalanced.

* Split out keystore capability into separate interface, making each
keystore a separate instance
* Disable "WITHOUT ROWID" optimization by default
* Implement prefix lookup that allows iterating over all values with a
certain prefix in their key
2021-05-17 15:55:57 +02:00
Jacek Sieka 8890175b6a remove WITHOUT ROWID
This is a minimal performance hotfix for storing large blobs in kvstore
2021-05-14 20:04:14 +03:00
Kim De Mey 762415319c
Add build_dcli target and add it to CI (#344)
* Add build_dcli target and add it to CI

* Fix local imports for dcli

* And use local imports for all other files too

* Use local imports in tests and rlpx protocols
2021-04-06 13:33:24 +02:00
Jacek Sieka 0f48ccecc0 sqlite: manual checkpointing 2020-12-18 19:30:53 +02:00
Mamy Ratsimbazafy 3b15abab10 Address review comment 2020-11-29 11:00:09 +02:00
Mamy André-Ratsimbazafy 482ea988c0 Add support for array[N, byte] in SQLite queries 2020-11-29 11:00:09 +02:00
Zahary Karadjov 49c40c9b95
Restore a commit accidentally omitted from a merged branch 2020-11-27 20:49:08 +02:00
Zahary Karadjov 345dac702b Don't use destructors 2020-10-16 21:21:00 +03:00
Zahary Karadjov cc0d15ccac
Richer Sqlite API
* Adds a strongly typed API for creating custom SQL queries and
  executing them

* Uses destructors to simplify the error handling in the init logic
2020-10-13 21:44:42 +03:00
Zahary Karadjov c103721391
Add support for multiple keyspaces in the Sqlite keystore 2020-10-12 04:02:33 +03:00
Jacek Sieka 3ddb498f2a
close sqlite transactions earlier (#294)
If the database is locked for reading, as it is when step returns ROW,
writes cannot checkpoint the wal leading to ever-increasing wal sizes
and a long delay at shutdown.

By resetting the transaction early, writes become more independent of
reads, memory is released earlier and wal can be checkpointed.
2020-09-21 08:21:47 +02:00
tersec ea0dbb256e
support closing kvstore databases (#293)
* support closing kvstore databases

* clear MemStore databases on close
2020-09-11 13:05:52 +00:00
Dustin Brody 64d880b24d
tweak phrasing to trigger CI to run again 2020-08-24 13:34:45 +02:00
Dustin Brody 9c9b9d5e2e
WAL pragma setting for in-memory databases results in "memory", not "wal" 2020-08-24 13:08:35 +02:00
Dustin Brody e2ccbbb418
use sqlite3 write-ahead logging 2020-08-24 12:37:40 +02:00
Ștefan Talpalaru 8ded65fe0e SQLite3 metrics: cosmetic changes 2020-05-13 11:37:35 +03:00
Ștefan Talpalaru 9da92b5415 SQLite3 mem usage metric 2020-05-13 11:37:35 +03:00
Jacek Sieka f730557baa sqlite: separate contains query
Using `select 1` for contains query tells sqlite not to actually load
the data for simple contains queries
2020-05-08 22:22:55 +03:00
Jacek Sieka 205b57fe71
port kvstore from nim-beacon-chain (#230)
* port kvstore from nim-beacon-chain
* remove old database backends
* use kvstore in trie database
* add sqlite dep
* avoid template param double evaluation
* clean up heterogenous lookup todo
2020-04-27 15:16:11 +02:00