Commit Graph

25 Commits

Author SHA1 Message Date
Kim De Mey c46706ac75
Add force pruning for the storage-capacity option (#1901) 2023-11-23 11:44:57 +01:00
Kim De Mey a7bb52e5b5
Add force prune for statically set radius (#1896) 2023-11-21 16:16:15 +01:00
Kim De Mey 6338969dd5
Lower the content deletion fraction and other contentdb clean-up (#1893)
- The current setting of 0.25 was very big. Set to 0.5, which is
potentially still large. This change did expose some issues with
the current implementation and especially testing.
- General clean-up, renaming for consistency, and
re-ordering/restructuring/deletion of some code.
- Fixed several typos
- ...
2023-11-16 15:27:30 +01:00
Kim De Mey c41d531c51
Adjust storage capacity option (#1891)
- Rename storage-size to storage capacity
- Change type to uint64 to avoid potential wrap around
- Change to MB instead of KB and set default to 2GB
2023-11-10 17:16:15 +01:00
Kim De Mey 03a739ff1b
Add contentdb contentCount and contentdb renames/cleanup (#1886) 2023-11-07 19:46:26 +01:00
Kim De Mey ca61a7009d
Bump submodules and related fixes (#1729)
* Bump submodules and related fixes

* Fix some warnings due to chronos and Nim bumps

* Seems macOS and Windows have different behavior
2023-10-17 14:19:50 +02:00
andri lim 56215ed83f
Bump stint to v2.0: new array backend (#1747)
* Bump stint to v2.0: new array backend
2023-09-13 09:32:38 +07:00
Kim De Mey 9453d5bb3c
Add loop to history network to log status message on interval (#1526) 2023-03-31 18:52:21 +02:00
Kim De Mey 73e93f1f11
Reduce Nim 1.6 compiler warnings & hints
* Reduce Nim 1.6 compiler warnings/hints for Fluffy and Nimbus proxy

Mostly raises Defect removals, TaintedString removal and some
unnecessary imports.

Also updating the copyright years alongside.

* Further reduce Nim 1.6 compiler warnings/hints for Nimbus
2023-01-31 13:38:08 +01:00
Kim De Mey 8523d8b40e
Bump submodules and fix related issues (#1448)
* Updated to the latest nim-eth, nim-rocksdb, nim-web3

* Bump nimbus-eth2 module and fix related issues

Temporarily disabling Portal beacon light client network as it is
a lot of copy pasted code that did not yet take into account
forks. This will require a bigger rework and was not yet tested
in an actual network anyhow.

* More nimbus fixes after module bumps

---------

Co-authored-by: Adam Spitz <adamspitz@status.im>
Co-authored-by: jangko <jangko128@gmail.com>
2023-01-27 15:57:48 +01:00
Kim De Mey c5ecba83cd
Refactor and clean-up of history network and eth rpc (#1357)
- Switch from using Option to Opt which allows for smoother
usage with already existing Result types
- With all moved to Opt, make more use of valueOr to avoid
too many if else clause indentation and unstead have a more
clear error path at each step
- Remove dead code, char limits, style guide, etc.
- Replace getEncodedKeyForContent with ContentKey.init
and use ContentKey.init for each type
2022-12-09 17:59:36 +01:00
KonradStaniec 5374e7d37f
Initial implementation of getting range of lc updates (#1301)
* Initial implementation of getting lc updates
2022-11-18 10:00:06 +01:00
KonradStaniec d75afd9f8a
Make portal network generic in db store and db get (#1290)
* Make portal network generic in db store and db get
2022-11-08 18:31:45 +01:00
Kim De Mey 9430619272
Remove Accumulator as network content type and add it to binary (#1267)
Portal master accumulator was removed from the network specs as a
content type shared on the network, as since the merge this is
a finite accumulator (pre-merge only).
So in this PR the accumulator gets removed as network type and
gets instead baked into the library. Building it is done by
seperate tooling (eth_data_exporter).
Because of this a lot of extra code can be removed that was
located in history_network, content_db, portal_protocol, etc.

Also removed to option to build the accumulator at start-up
of fluffy as this takes several minutes making it not viable.
It can still be loaded from a provided file however.

The ssz accumulator file is for now stored in the recently
created portal-spec-tests repository.
2022-10-17 20:38:51 +02:00
Kim De Mey 50af402f59
Add verification of block data by use of accumulator (#1178) 2022-08-01 21:00:21 +02:00
KonradStaniec 0776f35e0c
Use queries with custom xor function (#1113)
* Use queries with custom xor function
2022-06-03 13:44:42 +02:00
KonradStaniec af10e8f179
Do not use vacuum when pruning (#1103)
* Do not use vacuum when pruning
2022-05-26 08:26:08 +02:00
Kim De Mey f219c69840
Rework of Portal json-rpc debug API and related functionality (#1102)
- More consistent naming in calls used by the JSON-RPC debug API
- Use storeContent everywhere to make sure content is only stored
if in range
- Remove populateHistoryDb subcommand and replace by JSON-RPC call
storeContent
- Remove some whitespace and fix some indentations
2022-05-24 13:27:22 +02:00
KonradStaniec 6a9d875fe7
Add pruning metrics (#1093) 2022-05-17 12:47:24 +02:00
KonradStaniec 4e8c5f292e
Dynamic radius adjustments (#1079)
* Add config for node radius

* Adjust radius when deleting content
2022-05-12 18:04:37 +02:00
KonradStaniec d3f231e3da
Add put method which preserves max size (#1065)
* Add put method which preserves max size
2022-05-09 17:18:57 +02:00
Kim De Mey d337806301
Improve logs and loglevels, and general cleanup (#1059) 2022-04-13 07:56:01 +02:00
KonradStaniec b3570fae6a
Add api to get n furthest elements from db (#1026)
* Add api to get n furthest elements from db
2022-04-03 15:14:44 +02:00
KonradStaniec a330f0a4ef
Add api to get db size to content db (#987)
* Add size proc to content_db

* Add way to reclaim used space
2022-03-08 14:49:41 +01:00
Kim De Mey 51626c5831
Add a basic ContentDB for portal networks (#848)
* Add a basic ContentDB for Portal networks

* Use ContentDB in StateNetwork

* Avoid probably some form of sandwich problem by re-exporting kvstore_sqlite3 from content_db
2021-09-28 19:58:41 +02:00