Commit Graph

2766 Commits

Author SHA1 Message Date
jangko 69a1000d77
more engine api test 2022-06-15 15:56:45 +07:00
jangko ee13e5fdae
completing 'PrevRandao Opcode Transactions' test case in engine api test 2022-06-15 07:55:14 +07:00
jangko 77be2f66d2
handle PoA block difficulty during block creation
- in vmState
- in txpool
- in sealing engine

fix #1105
2022-06-15 07:55:00 +07:00
Kim De Mey 6325712aa6
Add accumulator content keys for history network (#1123) 2022-06-14 23:38:34 +02:00
Kim De Mey 49bdaa112e
Increase wait time for content propagation to avoid CI failures (#1122) 2022-06-14 14:23:07 +02:00
jangko a37f8b17e2
fix rpc and websocket server config if they share the same port with engine api
also fixes json-rpc-engine-api server and websocket-engine-api server shutdown code,
checking if they actually created or not at startup.

fix #1119
2022-06-13 19:32:19 +07:00
KonradStaniec b23789d107
Add getting receipts from history network (#1118)
* Add getting receipts from history network
2022-06-10 12:24:53 +02:00
Nikolay Mitev c1f7503402 Use asyncSpawn instead of deprecated asyncCheck 2022-06-09 12:51:06 +03:00
Nikolay Mitev 84a89e918a Add nimbus_addPeer rpc call 2022-06-09 12:51:06 +03:00
KonradStaniec 730013bde3
Change test data and propagation format (#1115)
* Change test data and propagation format

* Add tool to download data from local node
2022-06-08 15:14:01 +02:00
Jordan Hrycaj ee77d704bc
Non-adjacent intervals set management (#1117)
why:
  extracted from snap as separate helper utility

also:
  should go to `stew`, later on
2022-06-07 15:55:42 +01:00
Jordan Hrycaj 76f6de8059
Normalise snap objects (#1114)
* Fix/recover download flag

why:
  The fetch indicator used to control the data download somehow got
  lost during re-org.

* Updated chronicles/logger topics

* Reorganised run state flags

why:
  The original code used a pair of boolean flags `(stopped,stopThisState)`
  which was translated to three states running, stoppedPending, and
  stopped. It is currently not clear whether collapsing some states was
  correct. So the original logic has been re-stored, albeit wrapped into
  directives like `isStopped()` etc.

also:
  Moving some function bodies in `worker.nim`

* Moved `reply_data.nim` and `validate_trienode.nim` to sub-directory `fetch_trie`

why:
  Only used in `fetch_trie.nim`.

* Move `fetch_*` file and directory objects to `fetch` subdirectory

why:
  Only used in `fetch.nim`

* Added start/stop and/or setup/release methods for all sub-modules

why:
  good housekeeping

also:
  updated getters/setters for ctrl states
  updated trace messages
2022-06-06 14:42:08 +01:00
KonradStaniec 0776f35e0c
Use queries with custom xor function (#1113)
* Use queries with custom xor function
2022-06-03 13:44:42 +02:00
jangko 5bd134e2f0
more engine api tests 2022-06-01 20:32:07 +07:00
Kim De Mey 6d8b25a5f5
Update header accumulator test to start from Mainnet genesis (#1111) 2022-06-01 15:21:22 +02:00
Kim De Mey 7216160ad0
Update Ultralight bootstrap nodes for testnet (#1110) 2022-06-01 08:52:54 +02:00
KonradStaniec 5c78fe64e4
Fix utp connection leak on cancel (#1107)
* Fix utp connection leak on cancel
2022-05-31 14:28:02 +02:00
KonradStaniec b975d35c84
Bump sqlite-abi (#1108) 2022-05-31 12:37:10 +02:00
jangko 0ad7ffcdbe
hive: proper rpc test 2022-05-31 15:42:01 +07:00
jangko b03c2dca4f
bump nim-web3: engine api version alpha.9 2022-05-29 11:23:57 +07:00
jangko 9843c9428f
fix related to engine api alpha.9 2022-05-29 11:23:03 +07:00
jangko 8827e04bc0
remove unused coinbase from sealing engine
header.coinbase is handled by txpool, clique, and engine api.
so the sealing engine no need to touch it anymore.
2022-05-29 10:29:56 +07:00
jangko 3f0994f80b
fix clique_sealer: reset block header coinbase
if we are not voting, coinbase should be filled with zero
because other subsystem e.g txpool can produce block header
with non zero coinbase. if that coinbase is one of the signer
and the nonce is zero, that signer will be vote out from
signer list
2022-05-29 10:26:11 +07:00
KonradStaniec af10e8f179
Do not use vacuum when pruning (#1103)
* Do not use vacuum when pruning
2022-05-26 08:26:08 +02:00
jangko 8b0d700b45
fix EVM stack.[] bug 2022-05-25 12:04:47 +07:00
Kim De Mey f219c69840
Rework of Portal json-rpc debug API and related functionality (#1102)
- More consistent naming in calls used by the JSON-RPC debug API
- Use storeContent everywhere to make sure content is only stored
if in range
- Remove populateHistoryDb subcommand and replace by JSON-RPC call
storeContent
- Remove some whitespace and fix some indentations
2022-05-24 13:27:22 +02:00
Jordan Hrycaj 96bb09457e
Snap sync rename objects (#1099)
* Disentangle `collect` module from `reply_data`

why:
  Now the module visible from `collect` for fetching data is `peer/fetch`
  only.

* Merge `SnapPeerHunt` into `collect`

why:
  This part needs to be known by `collect`, only

* rename collect => worker

* Dissolve `sync_fetch_xdesc` module into `common`

why:
  Descriptor is only used in `common` and `fetch_trie`

* rename `snap/peer` directory => `snap/worker`

* rename `SnapSync` -> `Worker`, `SnapPeer` -> `WorkerBuddy`

* moved `snap/base_desc.nim` -> `snap/worker/worker_desc.nim`

* Unified opaque object ref naming in `worker_desc.nim`

details:
  indicated my inheriting module (exactly one, always)
2022-05-24 09:07:39 +01:00
Kim De Mey 553be51217
Change radius-config option to radius and allow also logRadius (#1100) 2022-05-23 23:23:24 +02:00
Jordan Hrycaj ba940a5ce7
Snap sync simplify object inheritance (#1098)
* Reorg SnapPeerBase descriptor, notably start/stop flags

details:
  Instead of using three boolean flags startedFetch, stopped, and
  stopThisState a single enum type is used with values SyncRunningOk,
  SyncStopRequest, and SyncStopped.

* Restricting snap to eth66 and later

why:
  Id-tracked request/response wire protocol can handle overlapped
  responses when requests are sent in row.

* Align function names with source code file names

why:
  Easier to reconcile when following the implemented logic.

* Update trace logging (want file locations)

why:
  The macros previously used hid the relevant file location (when
  `chroniclesLineNumbers` turned on.) It rather printed the file
  location of the template that was wrapping `trace`.

* Use KeyedQueue table instead of sequence

why:
  Quick access, easy configuration as LRU or FIFO with max entries
  (currently LRU.)

* Dissolve `SnapPeerEx` object extension into `SnapPeer`

why;
  It is logically cleaner and more obvious not to inherit from
  `SnapPeerBase` but to specify opaque field object references of the
  merged `SnapPeer` object. These can then be locally inherited.

* Dissolve `SnapSyncEx` object extension into `SnapSync`

why;
  It is logically cleaner and more obvious not to inherit from
  `SnapSyncEx` but to specify opaque field object references of
  the `SnapPeer` object. These can then be locally inherited.

  Also, in the re-factored code here the interface descriptor
  `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO
  inheritance makes it easier to work with call back functions.)
2022-05-23 17:53:19 +01:00
Kim De Mey a69b16abff
Cleanup some imports and fix some warnings in the process (#1096) 2022-05-22 22:44:15 +02:00
Kim De Mey 6e05c7588e
Add Header Accumulator for the Header Gossip Network (#1094) 2022-05-18 22:49:35 +02:00
Jordan Hrycaj 575c69e6ba
Objects inheritance reorg for snap sync (#1091)
* new: time_helper, types

* new: path_desc

* new: base_desc

* Re-organised objects inheritance

why:
  Previous code used macros to instantiate opaque object references. This
  has been re-implemented with OO inheritance based logic.

* Normalised trace macros

* Using distinct types for Hash256 aliases

why:
  Better control of the meaning of the hashes, all or the same format

caveat:
  The protocol handler DSL used by eth66.nim and snap1.nim uses the
  underlying type Hash256 and cannot handle the distinct alias in
  rlp and chronicles/log macros. So Hash256 is used directly (does
  not change readability as the type is clear by parameter names.)
2022-05-17 12:09:49 +01:00
KonradStaniec 6a9d875fe7
Add pruning metrics (#1093) 2022-05-17 12:47:24 +02:00
Kim De Mey 56c3e31efc
Refactor and clean-up of Portal wire protocol test (#1089) 2022-05-16 13:17:42 +02:00
Jordan Hrycaj 62d31d6f1d
Normalise sync handler prototypes (#1087)
* Use type name eth and snap (rather than snap1)

* Prettified snap/eth handler trace messages

* Regrouped sync sources

details:
  Snap storage related sources are moved to common directory.
  Option --new-sync renamed to --snap-sync

also:
  Normalised logging for secondary/non-protocol handlers.

* Merge protocol wrapper files => protocol.nim

details:
  Merge wrapper sync/protocol_ethxx.nim and sync/protocol_snapxx.nim
  into single file snap/protocol.nim

* Comments cosmetics

* Similar start logic for blockchain_sync.nim and sync/snap.nim

* Renamed p2p/blockchain_sync.nim -> sync/fast.nim
2022-05-13 17:30:10 +01:00
KonradStaniec 4e8c5f292e
Dynamic radius adjustments (#1079)
* Add config for node radius

* Adjust radius when deleting content
2022-05-12 18:04:37 +02:00
jangko 41e9ea8d45
bump nim-confutils
why:
- fix Acc Quoted ident processsing bug in config-file
- fix Dot Expr processsing bug in config-file
2022-05-10 16:04:21 +07:00
Jordan Hrycaj 569d426ea8
Restore not waiting for fastBlockchainSync() to succeed on start up (#1077)
why:
  Accidentally wrapped into waitFor() directive with reviving jl/sync
  branch.

also:
  Decorate eth/66 and snap/1 protocol trace messages with protocol
  type and version
2022-05-10 09:02:34 +01:00
KonradStaniec d3f231e3da
Add put method which preserves max size (#1065)
* Add put method which preserves max size
2022-05-09 17:18:57 +02:00
Jordan Hrycaj 58e0543920
Squashed snap-sync-preview patch (#1076)
* Squashed snap-sync-preview patch

why:
  Providing end results makes it easier to have an overview.

  Collected patch set comments are available as nimbus/sync/ChangeLog.md
  in chronological order, oldest first.

* Removed some cruft and obsolete imports, normalised logging
2022-05-09 15:04:48 +01:00
Kim De Mey 69366e1880
Improve selecting of nodes in neighborhood gossip (#1072)
Allow also concurrent neighborhood gossip jobs when seeding data
into the network.
Update Grafana dashboard for two additional metrics regarding
lookups in neighborhood gossip.
2022-05-07 13:50:16 +02:00
Jordan Hrycaj 5467abed8f
update docu (#1074)
why:
  extract_consensus_data.nim has been integrated into consensus_sim.nim
2022-05-06 16:49:48 +01:00
Jordan Hrycaj 3b9f0e3980
Proper Uint256 parser for JSON genesis (#1071)
* Update exception tracinig

* Use lazy JSON parser

why:
  Uint246 type is not directly supported by the JSON serializer

* Json parser update for stringified UInt256 integers

why:
  Now available

* update sub-module branch reference
2022-05-06 09:02:28 +01:00
Kim De Mey 7bd9afa465
Bump nim-eth for max discv5 packet size (#1070) 2022-05-03 09:18:33 +02:00
jangko dd339a1fb0
add simulators ci 2022-04-21 12:27:03 +07:00
jangko 6fcd63cb58
modify hive simulators to run in CI 2022-04-21 12:01:18 +07:00
Kim De Mey 99eea47dff
Fix clean-up socket in FindContent on cancel (#1067) 2022-04-20 18:46:00 +02:00
Kim De Mey 991edc1711
Fix log for block body network fetching (#1066) 2022-04-15 16:10:05 +02:00
jangko f2f204293e
first step into styleCheck fixes 2022-04-14 08:39:50 +07:00
jangko e2c9ec4f84
update submodules to accomodate styleCheck 2022-04-14 08:39:50 +07:00