Commit Graph

253 Commits

Author SHA1 Message Date
tersec 5cab17dc1a
database state storage benchmarking via ncli_db (#2312)
* database state storage benchmarking via ncli_db

* more cleanups from immutable validator state branch

* unexport some eth2_network constants and remove unused variables/templates

* make two PeerScore constants public
2021-02-15 17:40:00 +01:00
Giovanni Petrantoni 72b01161c1
populate gossipsub scores (#2091)
* force pushing to fix unstable base

* increase attestation/aggregate queue sizes

when there are many validators, many aggregates and attestations arrive
every slot - increase the queue size a bit - also do batches on each
idle loop iteration since it's fairly quick

* don't score subnets for now

* wrapping up

* refactor and cleanups

* gossip parameters fixes

* comment fix

Co-authored-by: Jacek Sieka <jacek@status.im>
2021-02-09 10:20:55 +01:00
Giovanni Petrantoni 2cd627f21a
hotfix gossip scoring (#2286)
* hotfix gossip scoring

* skip gossip scoring parameters validation as they violate for now (but does not matter cos we don't score)

* workaround again gossip validation
2021-02-02 15:59:18 +01:00
Dmitriy Ryajov 1117993a24
Libp2p connection limits (#2272)
* max peers gets set as connection limits

* pass max connections
2021-01-28 07:31:29 +01:00
tersec 4de9875cdf
Revert "pass max peers to libp2p (#2265)" (#2270)
This reverts commit 64e4d74f87.
2021-01-26 17:53:52 +01:00
Dmitriy Ryajov 64e4d74f87
pass max peers to libp2p (#2265) 2021-01-26 10:35:22 +01:00
Zahary Karadjov 7571e74dbd Fix #2252 (consistent typing for the 'delay' log property) 2021-01-22 15:09:21 +02:00
tersec 55ecb61c3a cycle attestation subnets every slot (#2240)
Cycle attestation subnets every slot
2021-01-19 19:44:03 +02:00
Kim De Mey 6fabefa76f
Fix discv5 loop in case of no nodes (#2243)
* Fix discovery loop in case of no peers in routing table

* local testnet: Stop searching when amount of peers in testnet is reached
2021-01-18 14:13:26 +01:00
Giovanni Petrantoni 295e3c9c73
Topics validation and direct peers (#2237)
* pick the right libp2p branch

* add topics validation
2021-01-15 04:17:06 +00:00
Kim De Mey 66d8f317cd
Use async queryRandom instead of inefficient randomNodes to discover nodes (#2211) 2021-01-14 08:58:13 +01:00
Giovanni Petrantoni a3a651b565
always enable topic and aggreate metric topics (#2229) 2021-01-12 04:27:09 +01:00
Giovanni Petrantoni ed24f60f70
remove async from sub/unsub (#2197)
* remove await/async from sub/unsub

* fix unsubscribe wrong key (missed _snappy)

* use the right libp2p commit hash

* remove unused async

* fix inspector

* fix subnet calculation in RPC and insert broadcast attestations into node's pool

* unify codepaths to ensure only mostly-checked-to-be-valid attestations enter the pool, even from node's own broadcasts

* update attestation pool tests for new validateAttestation param

Co-authored-by: Dustin Brody <tersec@users.noreply.github.com>
2020-12-24 09:48:52 +01:00
Jacek Sieka 452042b17b
msgid formatting, libp2p bump (#2198)
* msgid formatting, libp2p bump

* fix libp2p commit
2020-12-18 18:45:27 +00:00
Kim De Mey 8cc7effe52
Fix ENR attnets update to only hold persistent subnets (#2193)
* Fix ENR attnets update to only hold persistent subnets

* Use only stability subnet in metadata and enr
2020-12-18 09:50:29 +01:00
Jacek Sieka 4e191a06ac use stew/leb128
* 3 varint parsers -> 1
* 1/3 memory usage / allocations when writing stuff to libp2p
2020-12-15 17:06:15 +02:00
Giovanni Petrantoni c91091197f
Lower gossipFactor since we default to 160 peers (#2169)
And it might potentially send a bit more IHAVE messages then wanted.
2020-12-11 12:15:34 +01:00
Sacha Saint-Leger 94c66f15a9
Wanted peers log message fix (#2114)
* Edit Peer count low wanted peers log

* remove prometheus folder

* remove prometheus.yml

* Add current peers to low peer count log

* change to 25% peer count notice, 12.5% warning

Co-authored-by: kdeme <kim.demey@gmail.com>
2020-12-02 10:47:23 +01:00
Jacek Sieka a2364ce1bc
add getGossipSubPeers RPC (#2107) 2020-11-29 20:07:20 +01:00
Jacek Sieka c3bfe15a35
increase default max peers (#2106)
* increase default max peers

also avoid reconnection when opening stream as this might induce
a loop

* Use dial without addresses

* dial back max peers a little
2020-11-29 14:43:41 +01:00
Zahary Karadjov bf2673abc4
Restore the ETH display in the status bar 2020-11-28 20:53:51 +02:00
Eugene Kabanov db030eb944
Setup agent/proto version identifiers in libp2p. (#2079)
* Supply `nim-libp2p` with `agent-string` and `proto-string` as configured parameters.

* Remove protoString and rng check.

* Fix compilation problem.
2020-11-28 08:00:36 +01:00
Jacek Sieka d16e127daf
Peer stuff (#2084)
* Revert "Revert "Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065)" (#2082)"

This reverts commit 7cc3dc8027.

* fix nil disconnectedFut crash

* fixes

don't resetPeer, it causes peer miscounts

* disconnect disconnecting peers

...when there's a race.

* avoid connection spamming

* never decrease SeenTable timeout
* only recover ENR for known peers

* seen only when really disconnected
2020-11-26 20:23:45 +01:00
Giovanni Petrantoni 9b9aa93cd4
Seenttl by spec (#2083)
* add rpc call to get chronos futures at runtime

* add seenTTL gossip parameter

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-11-26 13:08:22 +01:00
Jacek Sieka 7cc3dc8027
Revert "Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065)" (#2082)
This reverts commit d041287a4f.
2020-11-26 09:05:23 +01:00
Eugene Kabanov d041287a4f
Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065)
* Initial commit.

* Fix log lines and compilation error.

* Add get_v1_node_peers() implementation.

* Fix peer's lifetime states.

* Use the most recent multiaddress.

* Fix assign NewPeerScore again.
Fix compilation error with last seen address.
Fix Peer upgraded log line place.

* syncing, health, peer_count, peer_id and fixes for identity.

* Fix compilation problems.

* Move object declaration to callsigs.
Fix identity addresses fields.

* Finish node RPC calls.

* Avoid leak of lifetime future.

* Bump chronos.

* Fix json generator problem.
2020-11-26 08:09:59 +01:00
kdeme e69b5ff473 Add a record create and print command 2020-11-25 18:32:59 +02:00
Giovanni Petrantoni 6f476ca380
Gossip11 params (#2047)
* add rpc call to get chronos futures at runtime

* add eth2 spec gossip parameters

* bump libp2p

* fix url, remove 1.0

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-11-23 10:57:03 +01:00
tersec 1d7fb2ed0c
remove {.inline.} pragmas (#2033)
* remove {.inline.} pragmas

* re-add inline on bitseqs functions and tweak inlining threshold

* remove macOS/LLVM inlining setting; revert non-init/module-local/tests inline pragma removal
2020-11-20 11:00:22 +01:00
Zahary Karadjov ebfacf597c
Review and fix some usages of Future.cancel; Bump Chronos 2020-11-18 00:27:21 +02:00
Jacek Sieka 4479c0a9f1
fix some libp2p channel hangs (#2032)
* fix some libp2p channel hangs

* make sure no concurrent reads are issues by awaiting cancellations
* bumps with fixes

* merge bump
2020-11-17 16:07:05 +01:00
tersec 9e716b32bd
address some XDeclaredButNotUsed hints (#2028) 2020-11-17 11:14:53 +01:00
Jacek Sieka a6b188bfd4
misc fixes (#2027)
* log when database is loading (to avoid confusion)
* generate network keys later during startup
* fix quarantine not scheduling chain of parents for download and
increase size to one epoch
* log validator count, enr and peerid more clearly on startup
2020-11-16 20:15:43 +01:00
tersec 21c4ce8fd4
remove superfluous TODOs/not-really-TODOs, type conversion, imports (#2025) 2020-11-16 17:10:51 +01:00
cheatfate 19707e89c2 Change random network key log line and include PeerID. 2020-11-16 17:41:23 +02:00
tersec a0621d52f6
bump remaining rc.0 spec ref (#2003) 2020-11-12 13:58:41 +00:00
Jacek Sieka 43c6c5a0b3
Gossip message id should decompress data (#1998)
* also fix missing return on empty message
2020-11-12 11:45:28 +01:00
Jacek Sieka c5710d97fc
default to gossipsub 1.1 (#1992) 2020-11-11 13:30:18 +01:00
Zahary Karadjov 389c11743a
Review TODO items and self-assign the most important ones 2020-11-10 20:41:04 +02:00
tersec 7eaae89263
keep AllTests-mainnet.md consistent; ensure don't send overly large messages aren't sent (#1973) 2020-11-09 18:11:09 +00:00
Zahary Karadjov bd1047b715 0.6.0 release fixes
* Updated README
* No double v in nimbus_beacon_node --version
* No md5 checksums
2020-11-09 17:09:49 +02:00
Sacha Saint-Leger 36b54fe091 Update eth2_network.nim
More informative and friendlier log message (from the user's perspective). Also helps avoid the potential for confusion with the validator key.
2020-11-03 23:23:32 +02:00
Jacek Sieka 937decb0e9
Revert "Gossip11 post audit (#1917)" (#1947)
This reverts commit 63173ab2c1.

It appears the cluster is having trouble staying connected - since the culprit is unknown, this is a first step on the way to what was stable.

Notably, this does not fully revert libp2p itself, merely the gossip version.
2020-11-03 11:45:40 +01:00
Giovanni Petrantoni 63173ab2c1
Gossip11 post audit (#1917)
* bump libp2p gossip

* use gossip 1.1

* use proper libp2p commit hash

* switch to master libp2p, has audit fixes now
2020-10-30 14:26:35 +00:00
Jacek Sieka ee2ebc96a8
move rpc api to own folder, mimic upstream structure (#1905)
also implements a few more endpoints
2020-10-27 10:00:57 +01:00
Kim De Mey 756b62fdf5
Bump nim-eth to get UseDiscv51 flag (#1895)
* Bump nim-eth to get UseDiscv51 flag

* Switch medalla to discovery v5.1, other targets to v5.0

* Bump nim-eth for better discv5.1 logging

* Bump eth2-testnets for updated medalla bootnodes
2020-10-23 18:59:58 +02:00
tersec decfd66a78
bump libp2p: seq[byte] message IDs and improve reject/ignore handling (#1898)
* bump libp2p: use seq[byte] message IDs and improve reject/ignore handling

* bump libp2p to pick up debug message details
2020-10-21 20:03:12 +00:00
tersec a136c2e95a
bump libp2p; integrate pubsub.ValidationResult into extended validation (#1893) 2020-10-20 12:31:20 +00:00
tersec e87e0ef179
update to 1.0.0-rc.0 message ID; unexport funcs; enable test in mainnet; remove unused subscribe-with-nondummy-handler function; fix gossip received metric (#1884) 2020-10-20 08:54:11 +00:00
Zahary Karadjov e69af00e3a Don't penalize flooding peers; Just rate limit them 2020-10-15 20:15:51 +03:00