Commit Graph

408 Commits

Author SHA1 Message Date
Jamie Lokier 9a28ed7ef5 RLPx: Protocol names have never been limited to 3 characters
Don't treat 3 characters as special in `cmp`.  `cmp` for `ProtocolInfo` was
wrong because it ignored all characters after the first 3.

In the wild we have seen protocol names longer than 3 characters.  `snap`,
`hive`, `istanbul`, `bzzeth`, `bzz-stream`, `bzz-retrieve`, `dbix`, `opera`,
`pchain`, `pchain_child_0`, `sero`, `smilobft`, `spock`.

There was never a 3 character limit in the [specification]
(https://github.com/ethereum/devp2p/blob/master/rlpx.md).

It always said "short ASCII name", until recently on 2021-02-25 it was changed
to an 8 characters limit.

Also `pi.nameStr` can be removed.  Nothing uses it, and it has the same actual
effect as just copying the string `pi.name`.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-08-10 14:08:35 +03:00
Jamie Lokier 5234e30f8b Remove `{.rlpInline.}` which was never really implemented
For a long time this caused invalid RLP parsing of `NewBlock` messages in the
`eth` protocol.

The `rlpInline` pragma was accepted but had no effect.  We could implemented
it, but it doesn't seem worth doing, with tests etc, as there's only one user
which has been fixed another way.

With `NewBlock`, whenever a peer sent us `NewBlock`, we'd get an RLP decoding
error, and disconnected the peer thinking it was the peer's error.

These messages are sent often by good peers, so whenever we connected to a
really good peer, we'd end up disconnecting within a minute due to this.  This
went unnoticed for years, as we stayed connected to old peers which have no new
blocks, and we weren't looking at peer quality, disconnect reasons or real-time
blockchain updates anyway.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-08-10 14:08:35 +03:00
Kim De Mey 9bc4fa366a
Let talkreq directly return the seq[byte] for easier API (#384) 2021-07-30 16:04:14 +02:00
Kim De Mey dd02d1be23
Remove unused lastUpdated from buckets (#382) 2021-07-29 21:58:10 +02:00
Kim De Mey a8d11dd30b
Add top level push raises Defect to p2p code (#374) 2021-07-16 21:44:30 +02:00
Kim De Mey 2557fd35c6
Use aesKeySize const for aes key instead if ivSize (same values) (#375) 2021-07-16 14:55:52 +02:00
Kim De Mey eb0908e33f
Push raises Defect to rlpx and accompanying changes (#373) 2021-07-14 10:35:35 +02:00
Kim De Mey 79911ed5d8
Log distance to uint16 and add public neighbours calls (#371)
* Use uint16 instead of uint32 for discv5 log distance

* Make neighboursAtDistances and neighbours calls available
2021-07-13 10:05:46 +02:00
Kim De Mey 41127eaee8
Remove portal wire code which was moved to nimbus-eth1 repo (#370) 2021-07-09 22:14:31 +02:00
kdeme 1c400e3f0e
Improve Portal message tracing 2021-06-09 14:57:35 +02:00
kdeme 880b753ad2
Add Portal wire readme doc 2021-06-09 14:57:35 +02:00
kdeme e2e30247bf
Add implementation of Portal wire protocol 2021-06-09 14:57:30 +02:00
Kim De Mey d18ebaa570
Slightly improved logging traces for error on message responses (#364) 2021-06-09 14:55:00 +02:00
Kim De Mey 8abe6b7144
Add support for discv5 talk protocols (#357) 2021-05-20 09:49:46 +02:00
Kim De Mey d05cb5d3bd
Fix raw Exceptions in hexary caused by forward declarations (#349)
* Fix raw Exceptions in hexary caused by forward declarations

* Fix raw Exceptions in trie/db caused by forward declarations

* And now we can remove those db Proc CatchableError raises
2021-05-11 17:32:47 +02:00
kdeme 00a45a7b91
Remove inline pragmas 2021-05-11 09:59:58 +02:00
kdeme 81f0a56ebd
Add/update bunch of license headers 2021-05-11 09:37:33 +02:00
kdeme 755729c6a1
Fix several compiler warnings
Mostly replacing deprecated calls
2021-05-11 09:24:23 +02:00
kdeme bcb58216d1
Add CatchableErrors where needed because of db backends used
In nim-eth this will not fail, as they are base, not implemented
methods. In for example Nimbus-eth1 it will.
2021-05-07 16:28:48 +02:00
kdeme 90b4724492
Adjust for chronosStrictException usage in rest of eth/p2p 2021-05-06 17:20:54 +02:00
kdeme e10ef19f81
Move push raises to top and add/update license info where needed 2021-04-28 16:20:05 +02:00
kdeme b0474c0d40
Add raises annotations to discovery.nim
And add push raises Defect, remove unneeded gcsafe and remove
all usage of inline.
2021-04-27 21:11:54 +02:00
kdeme a1da5d5e59
Use asyncSpawn instead of asyncCheck so chronos strict makes sense
And additional cleanup:
- Push raises Defect at top
- remove inlines
- remove unneeded gcsafe
- remove usage of deprecated calls
2021-04-27 11:30:08 +02:00
kdeme 9fed10de88
Allow for discv4 chronos strict usage
And group p2p tests that can be run with strict usage along the way.
2021-04-27 10:09:54 +02:00
Jamie Lokier 0f3bb61678
Fix import syntax error in `les_protocol.nim` added by PR #344
`les_protocol.nim` failed to build, due to very silly Nim bugs
nim-lang/Nim#8792 and nim-lang/Nim#17102.

    import
      ../../[rlp, keys], ../../common/eth_types,
      ../[rlpx, kademlia, blockchain_utils], ../private/p2p_types,

The silly part is `../` has to be quoted if it's before a group of files, but
not before a single file.  Most places in PR #344 / 7624153 use the workaround
`".."/` but it was missed in `les_protocol.nim`:

    nimbus-eth1/vendor/nim-eth/eth/p2p/rlpx_protocols/les_protocol.nim(14, 3)
        Error: cannot open file: ../../[rlp,keys]

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-07 18:07:00 +01:00
Jamie Lokier 209171d97d
discv4: Refactor findNode so it can be called directly from a test
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-06 18:41:47 +01:00
Kim De Mey 762415319c
Add build_dcli target and add it to CI (#344)
* Add build_dcli target and add it to CI

* Fix local imports for dcli

* And use local imports for all other files too

* Use local imports in tests and rlpx protocols
2021-04-06 13:33:24 +02:00
Jamie Lokier e4b4b7f4af
discv4: Fix Kademlia crash when trying to sync (#342)
Fixes status-im/nim-eth#341, status-im/nimbus-eth1#489.

When using discv4 (Kademlia) to find peers, there is a crash after a few
minutes.  It occurs for most of us on Eth1 mainnet, and everyone on Ropsten.

The cause is `findNodes` being called twice in succession to the same peer,
within about 5 seconds of each other.  ("About" 5 seconds, because Chronos does
not guarantee to run the timeout branch at a particular time, due to queuing
and clock reading delays.)

Then `findNodes` sends a duplicate message to the peer and calls
`waitNeighbours` to listen for the reply.  There's already a `waitNeighbours`
callback in a shared table, so that function hits an assert failure.

Ignoring the assert would be wrong as it would break timeout logic, and sending
`FindNodes` twice in rapid succession also makes us a bad peer.

As a simple workaround, just skip `findNodes` in this state and return a fake
empty `Neighbours` reply.  This is a bit of a hack as `findNodes` should not be
called like this; there's a logic error at a higher level.  But it works.

Tested for about 4 days constant operation on Ropsten.  The crash which occured
every few minutes no longer occurs, and discv4 keeps working.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-02 23:29:02 +02:00
Kim De Mey c5dace27ca
Use chronos http server for dcli metrics and remove insecure compile flag (#343)
And add cfg file to default have runtime log filtering on
2021-04-02 17:29:38 +02:00
Kim De Mey ae4177851f
Remove no longer needed try/except (#339) 2021-03-26 17:49:03 +01:00
Jacek Sieka 698b3bea88
Exception tracking (#334)
* Exception tracking

https://github.com/status-im/nim-chronos/pull/166

* Remove redundant raises annotation

Co-authored-by: kdeme <kim.demey@gmail.com>
2021-03-25 18:12:11 +01:00
Kim De Mey 8c27f291f5
Add raises annotation to the FilterMsgHandler proc type (#337) 2021-03-25 15:06:12 +01:00
Kim De Mey 0be863d2de
Add raises annotations to make exception tracking work (#336)
See https://github.com/status-im/nim-chronos/pull/166
2021-03-24 12:52:09 +01:00
Jacek Sieka 16802c0e52
Fix minor exception effect issues (#335)
Makes code compatible with
https://github.com/status-im/nim-chronos/pull/166 without requiring it.
2021-03-23 07:47:55 +01:00
Kim De Mey be5e088b21
Move parsing of nat cli option out of setupAddress (#331)
* Move parsing of nat cli option out of setupAddress

* Change some proc to func
2021-03-05 21:23:54 +01:00
Kim De Mey 0700ec770f
Improve network address discovery / NAT setup (#323)
* Add search for best route and refactor setupNat to setupAddress

* Update setupAddress and make enr ports in discovery optional

* Add specific error log when no route is found
* Use bindIP if it is public
* Adjust some log levels
2021-03-02 17:13:29 +01:00
jangko dda2bec8aa turn networkId into distinct uint 2021-02-13 17:43:17 +07:00
Kim De Mey a339944bcf
Some discv5 cleanup / refactor (#326)
* Use Address in HandshakeKey

* Refactor + comments + dcli use queryRandom

* Rename types.nim to messages.nim
2021-02-02 22:47:21 +01:00
kdeme 4e58eb48ce Add comments on enrAutoUpdate and add metric 2021-02-01 20:30:55 +02:00
kdeme bfbad64d83 Add update of ENR on newly discovered address 2021-02-01 20:30:55 +02:00
kdeme 689eef2c11 Move IpAddress into the pong message 2021-02-01 20:30:55 +02:00
kdeme d33d27c2ee Small improvements and cleanup 2021-02-01 20:30:55 +02:00
kdeme e43ee6ef9c Add ip address voting through pong responses 2021-02-01 20:30:55 +02:00
kdeme 181bbadcbc
Add more message related metrics 2021-01-13 21:44:17 +01:00
kdeme aa7442c9ff
dcli: actually use the provided listen address 2021-01-11 15:01:02 +01:00
kdeme 9cedbc0cc8 Move code into seedTable and populateTable proc 2021-01-07 10:20:40 +01:00
kdeme ac58a1f35c Only do discovery queries to refresh the table 2021-01-07 10:20:40 +01:00
kdeme 12ec608efd
Put a limit on the nodes returned on a findnode request 2020-12-17 17:06:42 +01:00
kdeme 21423fca42
Use of discv5.1 findnode request with multiple distances 2020-12-17 15:42:04 +01:00
kdeme a506739b23
Add query proc and use this for recurrent queries
This query proc is similar to the original (faulty) lookup proc.
But as we don't need to look for specific targets, it can be
used still as it gives a quicker and broader search resulting
in more nodes.
2020-12-17 15:23:02 +01:00
kdeme e1acc1ae2d
Avoid sending more requests to node that timed out + request metrics 2020-12-16 00:07:49 +01:00
kdeme 68c9b7b3ad
Fix lookup to sort and query closest nodes 2020-12-15 13:24:57 +01:00
Kim De Mey 0820dbba46
Use one instead of deprecated oneIndex (#317) 2020-12-15 12:41:53 +01:00
Kim De Mey d024ea8e85
Change invalid distance warning to debug (#316)
And also:

* move connected_peers metric so it does get added where it
shouldn't.

* imports cleanup
2020-12-14 12:21:03 +01:00
Kim De Mey b4c1391be9
Discv5 ip limits for routing table (#308)
* Add ip limits to routing table and routing table buckets

* Fix order of ip limit check and duplicate check for replacement

* Fix ip limit for node with updated ip in ENR

* Fix bug where address wouldn't update on ENR update

and update some comments

* Reuse some add/remove code in routing table

* Fix seen bug on ENR update in routing table

* Rework addNode to make sure to do address check always

and adjust some logs.

* More documentation on the ip limits in routing table [skip ci]
2020-11-26 18:20:15 +01:00
Kim De Mey b88fef203b
Add info log line in case of no external IP (#313) 2020-11-26 18:19:13 +01:00
kdeme 18430ef127
Add empty talkreq command to dcli 2020-11-13 12:45:39 +01:00
kdeme 44637cdd8e
Remove support for discovery v5.0 2020-11-13 12:33:07 +01:00
Kim De Mey 71d6dbd1b4
Discv5 small adjustments (#307)
* Use checkedEnumAssign in discv5.1

* Some renaming according to discv5 spec nomenclature
2020-10-29 22:04:37 +01:00
Kim De Mey 3626755529
Improve discovery v5.1 logging (#306) 2020-10-23 16:41:44 +02:00
Kim De Mey 6bdf1b4b0f
Simplify UseDiscv51 compile flag (#305) 2020-10-19 21:04:53 +02:00
kdeme cf586c363f Random message size and message size checks 2020-10-16 21:20:47 +03:00
kdeme 9b971a0e14 Improve error logging messages
As we don't need to differentate between errors with an enum
anymore thanks to refactor.
2020-10-16 21:20:47 +03:00
kdeme c28cba3589 Limit request id to maximum 8 bytes 2020-10-16 21:20:47 +03:00
kdeme 301067c869 Fix RequestId encoding/decoding and improved tests
- RequestId to variable rlp byte array
- Do encoding and decoding steps in the message tests
- Do sign and verify in the nonce sign test
2020-10-16 21:20:47 +03:00
kdeme 76c10c9b79 Minimum size of random message fix for interop 2020-10-16 21:20:47 +03:00
kdeme 17ef0b25e0 Update to support the latest discv5.1 specification 2020-10-16 21:20:47 +03:00
kdeme 820a73f96f Store session secrets only on succesful decodeHandshakePacket 2020-10-16 21:20:47 +03:00
kdeme a10ee4cc42 Fix topicquery value 2020-10-16 21:20:47 +03:00
kdeme 44b30ef0cb Add all message types and a basic implementation of talkreq/talkresp 2020-10-16 21:20:47 +03:00
kdeme ce2cd2323c Support findnode with multiple distances in discv5.1 2020-10-16 21:20:47 +03:00
kdeme 8042d72711 Add discovery v5.1 implementation 2020-10-16 21:20:47 +03:00
Clyybber 13089cad4c Fix compilation for 1.4
Since proc params are now turned into syms during sem
we need to "refresh" them for use in another context
2020-10-16 20:06:59 +03:00
Zahary Karadjov de2d43a7e7 Remove some unused code from NBC by making it RLPx-specific 2020-10-05 17:28:58 +03:00
Kim De Mey 6b17531d48
Improvements on dropping of challenges and handling of too large distance (#296)
- drop handshake challenge on invalid handshake
- send empty nodes reponse when distance is > 256
- misc
2020-09-30 09:43:51 +02:00
Kim De Mey c9caafb2a4
Replace memory db with lrucache for temporary storage of sessions (#292) 2020-09-10 14:49:48 +02:00
Zahary Karadjov 73c1bb817c
Adapt to latest snappy; Avoid decompression bombs 2020-08-19 14:14:59 +03:00
kdeme f0638eb3f1
Add rlp_decode fuzz test + some corpus generate code 2020-07-20 17:09:15 +02:00
kdeme 4e4836a0fe
Add discv5 fuzzing test targets 2020-07-20 14:17:44 +02:00
kdeme ec260dcfe4
Add & update tests 2020-07-17 22:48:14 +02:00
kdeme 1eae8f93f9
Fix empty ENR list in auth-response properly + potentially request new ENR on revalidation 2020-07-17 16:18:50 +02:00
kdeme 84fd39a8f8
Update enrs in buckets and replacement caches and fix empty ENR list in AuthResponse 2020-07-17 11:14:08 +02:00
kdeme c7ac9f54b3
Refactor the findNode code + add tests 2020-07-15 15:27:22 +02:00
kdeme 33f72e77bc
Add duplicate and distance checks in Nodes message 2020-07-15 11:15:50 +02:00
Kim De Mey 7febbec673
Use brHmacDrbgGenerate for all random operations in discovery (#272) 2020-07-13 14:34:53 +02:00
Kim De Mey f3de959261
Discv5 readme (#278)
* Add discovery v5 readme + test_discv5 task

* Move hkdf tests to test file and add to nimble task
2020-07-12 23:53:27 +02:00
Kim De Mey 0888667ac0
Comments and cleanup (#276)
* Fix dcli + add more comments

* Fix pong enr sequence number + varia cleanup

* Send randomData in case no handshake was done yet. Fix #277
2020-07-12 17:25:18 +02:00
Kim De Mey 28a8d52308
Fix enr deserialization crash + more tests + fuzz test (#274)
* Fix enr deserialization crash + more tests + fuzz test

* CI: Install depsOnly
2020-07-11 01:28:53 +02:00
Eugene Kabanov e64efc5dbb
Fix LSWAP problem. (#275) 2020-07-10 23:30:34 +02:00
Kim De Mey 7f77ba2308
Fix formatFloat issue (#273) 2020-07-10 14:17:06 +02:00
kdeme eeb958e834
Add comments [skip ci] 2020-07-08 14:56:56 +02:00
kdeme 4f3df3c9b2
Change update back to use var parameter of Record 2020-07-08 14:28:54 +02:00
kdeme d3db83fa0a
Extra test on newProtocol 2020-07-08 13:16:07 +02:00
kdeme 95a09fdf7f
Get rid of some duplicate code 2020-07-08 12:14:00 +02:00
kdeme 0fb21e72d7
Add updateEnr and allow for table constructor usage in newProtocol 2020-07-08 11:45:58 +02:00
kdeme 72420d7f17
Allow for passing in previous enr at discovery protocol creation 2020-07-07 23:39:32 +02:00
kdeme 57302fcf52
Allow for multiple enr fields to be inserted + enr update proc 2020-07-07 22:48:26 +02:00
kdeme 3bbe757fe3
Add record size check on init + add insertFieldPair call + tests 2020-07-07 17:19:15 +02:00
Jacek Sieka 484fbcab1b
use bearssl rng throughout (#265)
* use bearssl rng throughout

* seeder can fail

* imports and exports

* modules, sigh

* one more try

* move var

* even fewer thread vars

* remove out-of-date genrated files
2020-07-07 10:56:26 +02:00
kdeme ffb7923370
Use different try/except construct to enforce returning on error 2020-06-30 21:58:23 +02:00