Commit Graph

345 Commits

Author SHA1 Message Date
Jordan Hrycaj 134fe26997
Store proved snap accounts (#1145)
* Relocated `IntervalSets` to nim-stew repo

* Accumulate accounts on temporary kv-DB

why:
  Explore the data as returned from snap/1. Will be converted to a
  `eth/db` next.

details:
  Verify and accumulate per/state-root accounts downloaded via snap.

also:
  Some unit tests

* Replace `Table` by `TrieDatabaseRef` for accounts accumulator

* update ticker statistics

details:
  mean/variance based counter update

* allow persistent db for proved accounts

* rebase, and globally activate unit test

* fix statistics
2022-07-01 12:42:17 +01:00
Kim De Mey d4d4e8c28f
Add accumulator buildProof, verifyProof and verifyHeader (#1132) 2022-06-23 21:00:59 +02:00
Jacek Sieka 9e8d2d0550
sqlite: bump 2022-06-21 16:18:41 +02:00
andri lim 5374874552
bump serialization family(serialization, json, toml) (#1135) 2022-06-20 11:32:41 +02:00
KonradStaniec 0776f35e0c
Use queries with custom xor function (#1113)
* Use queries with custom xor function
2022-06-03 13:44:42 +02:00
KonradStaniec 5c78fe64e4
Fix utp connection leak on cancel (#1107)
* Fix utp connection leak on cancel
2022-05-31 14:28:02 +02:00
KonradStaniec b975d35c84
Bump sqlite-abi (#1108) 2022-05-31 12:37:10 +02:00
jangko b03c2dca4f
bump nim-web3: engine api version alpha.9 2022-05-29 11:23:57 +07:00
Jordan Hrycaj ba940a5ce7
Snap sync simplify object inheritance (#1098)
* Reorg SnapPeerBase descriptor, notably start/stop flags

details:
  Instead of using three boolean flags startedFetch, stopped, and
  stopThisState a single enum type is used with values SyncRunningOk,
  SyncStopRequest, and SyncStopped.

* Restricting snap to eth66 and later

why:
  Id-tracked request/response wire protocol can handle overlapped
  responses when requests are sent in row.

* Align function names with source code file names

why:
  Easier to reconcile when following the implemented logic.

* Update trace logging (want file locations)

why:
  The macros previously used hid the relevant file location (when
  `chroniclesLineNumbers` turned on.) It rather printed the file
  location of the template that was wrapping `trace`.

* Use KeyedQueue table instead of sequence

why:
  Quick access, easy configuration as LRU or FIFO with max entries
  (currently LRU.)

* Dissolve `SnapPeerEx` object extension into `SnapPeer`

why;
  It is logically cleaner and more obvious not to inherit from
  `SnapPeerBase` but to specify opaque field object references of the
  merged `SnapPeer` object. These can then be locally inherited.

* Dissolve `SnapSyncEx` object extension into `SnapSync`

why;
  It is logically cleaner and more obvious not to inherit from
  `SnapSyncEx` but to specify opaque field object references of
  the `SnapPeer` object. These can then be locally inherited.

  Also, in the re-factored code here the interface descriptor
  `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO
  inheritance makes it easier to work with call back functions.)
2022-05-23 17:53:19 +01:00
jangko 41e9ea8d45
bump nim-confutils
why:
- fix Acc Quoted ident processsing bug in config-file
- fix Dot Expr processsing bug in config-file
2022-05-10 16:04:21 +07:00
Jordan Hrycaj 3b9f0e3980
Proper Uint256 parser for JSON genesis (#1071)
* Update exception tracinig

* Use lazy JSON parser

why:
  Uint246 type is not directly supported by the JSON serializer

* Json parser update for stringified UInt256 integers

why:
  Now available

* update sub-module branch reference
2022-05-06 09:02:28 +01:00
Kim De Mey 7bd9afa465
Bump nim-eth for max discv5 packet size (#1070) 2022-05-03 09:18:33 +02:00
jangko e2c9ec4f84
update submodules to accomodate styleCheck 2022-04-14 08:39:50 +07:00
Kim De Mey d337806301
Improve logs and loglevels, and general cleanup (#1059) 2022-04-13 07:56:01 +02:00
Kim De Mey 82aab2a404
Bump chronos and json-rpc and avoid http conn re-use for blockwalk (#1053)
Currently re-using http connections for the json-rpc requests
seems to slow down the requests considerably. Avoid this by
forcing a close after each requests in the blockwalk tool.
2022-04-11 19:39:45 +02:00
Jordan Hrycaj 8af5c33ef9
Facilitate http code response on websocket JWT authentication failure (#1043)
* Facilitate http code response on websocket JWT authentication failure

* Update JSON-RPC link
2022-04-07 10:37:35 +01:00
Jordan Hrycaj 737236fd6e
Enable JWT authentication for websockets (#1039)
* Enable JWT authentication for websockets

details:
  Currently, this is optional and only enabled when the jwtsecret option
  is set.

  There is a default mechanism to generate a JWT secret if it is not
  explicitly stated. This mechanism is currently unused.

* Make JWT authentication compulsory for websockets

* Fix unit test entry point + cosmetics

* Update JSON-RPC link

* Improvements as suggested by Mamy
2022-04-06 15:11:13 +01:00
Kim De Mey 16ecbeb902
Bump nim-eth for lower loglevel of rlpx and discv4 harmless errors (#1041) 2022-04-06 11:57:08 +02:00
Jacek Sieka 1d6a9951d6
move rocksdb support to eth1 (#927)
* move rocksdb support to eth1

only used here / causes unnecessary build deps
2022-04-06 07:28:19 +02:00
Jacek Sieka e9afa00f16 snappy revamp
not entirely sure why nimbus-eth1 has a copy of snappy but here's a new
version: https://github.com/status-im/nim-snappy/pull/10
2022-04-05 12:03:01 +07:00
Jacek Sieka 60032bf106
Bump chronos (#925)
Memory fixes, http fixes, fix for issue #963, etc
2022-03-31 14:55:01 +02:00
KonradStaniec 75f2c76ad0
Bump nim-eth module (#1017) 2022-03-29 21:09:49 +02:00
Jordan Hrycaj 4696a53302
Enable optional chunked RLPx messages (#1010)
* Enable optional chunked RLPx messages

why:
  Legacy feature used by Nethermind

details:
  Disable with make flag: ENABLE_CHUNKED_RLPX=0

* Rebase & bump nim-eth

* Fix default behaviour

why:
  Got lost somehow. Comments do not match GNU make code directives.
2022-03-29 10:19:32 +01:00
Ștefan Talpalaru 51bc1cf87f
dist: precompiled binaries and Docker images (#1015)
* dist: precompiled binaries and Docker images

The builds are reproducible, the binaries are portable and statically link librocksdb.

This took some patching. Upstream PR: https://github.com/facebook/rocksdb/pull/9752

32-bit ARM is missing as a target because two different GCC versions
fail with an ICE when trying to cross-compile RocksDB. Using Clang
instead is too much trouble for a platform that nobody should be using
anyway.

(Clang doesn't come with its own target headers and libraries, can't be
easily convinced to use the ones from GCC, so it needs an fs image from
a 32-bit ARM distro - at which point I stopped caring).

* CI: disable reproducibility test
2022-03-27 13:21:15 +02:00
jangko a0ba5a492c
bump some submodules
- graphql: bump version 0.2.20 to 0.2.22
- json-rpc: various bug fixes
- websock: various bug fixes
- http-utils: chronos dependancy
2022-03-25 17:16:21 +07:00
KonradStaniec ad51bd68fc
Bump nim-eth (#1004) 2022-03-21 20:18:43 +01:00
Kim De Mey b00ac490a9
Bump nim-eth and adjust API to support binding to specific address (#994)
- Fixes also a bug which would not start an EthereumNode properly
when no bootstrap nodes are provided
2022-03-16 09:01:35 +01:00
KonradStaniec 2c2bcae00e
Bump nim-eth (#986) 2022-03-04 14:36:32 +01:00
jangko 0d18b606f5
bump json-rpc 2022-02-28 10:36:35 +07:00
jangko 1f9ed449d9
bump nim-web3
update Engine API from v1-alpha6 to v1-alpha7
2022-02-27 14:37:35 +07:00
jangko b46d60f65d
bump nim-json-rpc
null value of optional param rejected by serializer
because of a bug in `jsonmarshal.nim` now fixed
2022-02-14 21:23:00 +07:00
Kim De Mey 38036966a6
Improve the tests of the local testnet (#953)
* Improve the tests of the local testnet

The local testnet test was rather flaky and would occasionally
fail. It has been made more robust by adding the ENRs directly
to the routing table instead of doing some random lookups.

Additionally, the amount of nodes were increased (=64), ip limits
configuration was added, and the bits-per-hop value was set to 1
in order to make the lookups more likely to hit the network
instead of only the local routing table.

Failure is obviously still possible to happen when sufficient
packets get lost. If this turns out to be the case with the current
amount of nodes, we might have to revise the testing strategy here.

* Disable lookup test for State network

Disable lookup test for State network due to issue with custom
distance function causing the lookup to not always converging
towards the target.
2022-02-02 22:48:33 +01:00
jangko 3e60948785 bump nim-eth
changes because of EIP-4399 preparation:
- `mixDigest` field of BlockHeader got alias `random`
2022-02-01 18:11:14 +02:00
Kim De Mey ac1196d915
Bump nim-ssz-serialization and clean-up fromSszBytes raises (#948) 2022-01-31 21:57:34 +01:00
KonradStaniec 296bf3156d
Bump nim-eth (#946)
implementation of utp fast resends
implementation of utp fast resends when processing selective acks
more logging for utp
2022-01-28 06:55:13 +01:00
Zahary Karadjov 3cbb920406
Migrate to Engine API spec version v1.0.0-alpha.5; More progress towards working M1 2022-01-25 12:52:50 +02:00
KonradStaniec 990846135a
Bump nim-eth (#941)
* Bump nim-eth
2022-01-25 10:19:16 +01:00
Zahary Karadjov 137eb97766 Initial implementation of the merge spec
Includes a simple test harness for the merge interop M1 milestone

This aims to enable connecting nimbus-eth2 to nimbus-eth1 within
the testing protocol described here:

https://github.com/status-im/nimbus-eth2/blob/amphora-merge-interop/docs/interop_merge.md

To execute the work-in-progress test, please run:

In terminal 1:
tests/amphora/launch-nimbus.sh

In terminal 2:
tests/amphora/check-merge-test-vectors.sh
2022-01-24 09:44:39 +02:00
Zahary Karadjov e7adc609f4
Bump nim-stew 2022-01-23 19:25:46 +02:00
Zahary Karadjov 8ef9c620cb
Bump nim-json-rpc 2022-01-23 19:19:24 +02:00
Jacek Sieka f47c710ada
stew: bump (#924)
`Result` changes and `endians2` performance fix
2022-01-13 21:11:54 +01:00
Kim De Mey 73b4a73423
Bump nim-eth and nim-ssz-serialization vendor modules (#926)
- nim-eth: uTP updates, ENR improvemens and extra discv5 metrics
- nim-ssz-serialization: require BitList pairs
- update github actions cache keys to invalidate cache
2022-01-12 21:41:23 +01:00
Jordan Hrycaj 1f774c01a2
Jordan/accounts cache scenario (#904)
* crash test scenario

details:
  Example code for inspecting nested block chain and accounts cache
  database transaction framework. There seems to be a pathological
  case where the system crashes after a rollback (as appeared in the
  tx-pool packer code.)

* simplified crash scenario

* Workable solution (as suggested by Andri)

details:
  Avoiding db.rollback() (db.commit() is OK) while vmState.stateDB is
  alive.

* Rename text_txcrash => test_accounts_cache

why:
  Unit tests covers part of accounts_cache handling

* comment update
2021-12-13 11:58:05 +00:00
Kim De Mey 70625bc02c
Use one bootstrap nodes argument for discv5 and Portal (#911)
Currently bootstrap nodes for discv5 and for the Portal nodes
were provided through separate cli arguments. This is however
confusing and cumbersome as typically when (currently) testing
a node will have both discv5 and the Portal networks enabled.
We merge them into one argument.

If a node happens not to support a Portal network, it will be
removed after message request failure.

Commit also contains additional clean-up and nim-eth bump.
2021-12-13 09:06:29 +01:00
jangko d5d0a44b0e
bump nim-confutils with addition features needed by nimbus-eth1
new features:
- multiple lines long description
- ignore property in addition to hidden
2021-12-12 16:38:10 +07:00
Kim De Mey 63b0945b52
Use json rpc client to run tests on portal testnet (#899)
Tests have also been added & adjusted. Original ones were actually
failing but unnoticed due to mistake in port passed in curl command.
2021-12-03 09:51:25 +01:00
jangko dcd86bb221
vendor: bump nim-json-rpc
bugfix and features:
- Switch to the Chronos HTTP client (adds support for HTTPS)
- Allow dynamic RPC method names in the 'rpc' macro
- Restore the support for using the news package
2021-11-30 11:47:11 +07:00
Kim De Mey 903350bdde
Add local testnet script and required json-rpc calls (#891)
- Add basic discv5 and portal json-rpc calls and activate them in
fluffy
- Renames in the rpc folder
- Add local testnet script and run this script in CI
- bump nim-eth
2021-11-24 08:45:55 +01:00
KonradStaniec f9ed83c221
bump nim eth (#890) 2021-11-22 12:35:08 +01:00
Kim De Mey 02afda1b45
Run everything fluffy with chronosStrictException (#889) 2021-11-18 17:52:44 +01:00