Commit Graph

584 Commits

Author SHA1 Message Date
Jacek Sieka 8890175b6a remove WITHOUT ROWID
This is a minimal performance hotfix for storing large blobs in kvstore
2021-05-14 20:04:14 +03:00
Kim De Mey d05cb5d3bd
Fix raw Exceptions in hexary caused by forward declarations (#349)
* Fix raw Exceptions in hexary caused by forward declarations

* Fix raw Exceptions in trie/db caused by forward declarations

* And now we can remove those db Proc CatchableError raises
2021-05-11 17:32:47 +02:00
Kim De Mey 2d75e73bab
Merge pull request #348 from status-im/p2p-chronos-strict
Adjust for chronosStrictException usage in rest of eth/p2p
2021-05-11 11:03:43 +02:00
kdeme 00a45a7b91
Remove inline pragmas 2021-05-11 09:59:58 +02:00
kdeme 81f0a56ebd
Add/update bunch of license headers 2021-05-11 09:37:33 +02:00
kdeme 755729c6a1
Fix several compiler warnings
Mostly replacing deprecated calls
2021-05-11 09:24:23 +02:00
kdeme bcb58216d1
Add CatchableErrors where needed because of db backends used
In nim-eth this will not fail, as they are base, not implemented
methods. In for example Nimbus-eth1 it will.
2021-05-07 16:28:48 +02:00
kdeme 90b4724492
Adjust for chronosStrictException usage in rest of eth/p2p 2021-05-06 17:20:54 +02:00
Kim De Mey 04f641c923
Merge pull request #347 from status-im/discv4-strict
Discv4 with chronos strict usage
2021-04-29 17:28:47 +02:00
kdeme e10ef19f81
Move push raises to top and add/update license info where needed 2021-04-28 16:20:05 +02:00
kdeme b0474c0d40
Add raises annotations to discovery.nim
And add push raises Defect, remove unneeded gcsafe and remove
all usage of inline.
2021-04-27 21:11:54 +02:00
kdeme a1da5d5e59
Use asyncSpawn instead of asyncCheck so chronos strict makes sense
And additional cleanup:
- Push raises Defect at top
- remove inlines
- remove unneeded gcsafe
- remove usage of deprecated calls
2021-04-27 11:30:08 +02:00
kdeme 9fed10de88
Allow for discv4 chronos strict usage
And group p2p tests that can be run with strict usage along the way.
2021-04-27 10:09:54 +02:00
Jamie Lokier 0f3bb61678
Fix import syntax error in `les_protocol.nim` added by PR #344
`les_protocol.nim` failed to build, due to very silly Nim bugs
nim-lang/Nim#8792 and nim-lang/Nim#17102.

    import
      ../../[rlp, keys], ../../common/eth_types,
      ../[rlpx, kademlia, blockchain_utils], ../private/p2p_types,

The silly part is `../` has to be quoted if it's before a group of files, but
not before a single file.  Most places in PR #344 / 7624153 use the workaround
`".."/` but it was missed in `les_protocol.nim`:

    nimbus-eth1/vendor/nim-eth/eth/p2p/rlpx_protocols/les_protocol.nim(14, 3)
        Error: cannot open file: ../../[rlp,keys]

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-07 18:07:00 +01:00
Jamie Lokier a48687d277
discv4: Add test associated with fix to Kademlia crash (#341)
It's been confirmed that the test fails if the Kademlia crash workaround isn't
in `findNode`, and passes with it there.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-06 18:46:32 +01:00
Jamie Lokier 209171d97d
discv4: Refactor findNode so it can be called directly from a test
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-06 18:41:47 +01:00
Kim De Mey 762415319c
Add build_dcli target and add it to CI (#344)
* Add build_dcli target and add it to CI

* Fix local imports for dcli

* And use local imports for all other files too

* Use local imports in tests and rlpx protocols
2021-04-06 13:33:24 +02:00
Jamie Lokier e4b4b7f4af
discv4: Fix Kademlia crash when trying to sync (#342)
Fixes status-im/nim-eth#341, status-im/nimbus-eth1#489.

When using discv4 (Kademlia) to find peers, there is a crash after a few
minutes.  It occurs for most of us on Eth1 mainnet, and everyone on Ropsten.

The cause is `findNodes` being called twice in succession to the same peer,
within about 5 seconds of each other.  ("About" 5 seconds, because Chronos does
not guarantee to run the timeout branch at a particular time, due to queuing
and clock reading delays.)

Then `findNodes` sends a duplicate message to the peer and calls
`waitNeighbours` to listen for the reply.  There's already a `waitNeighbours`
callback in a shared table, so that function hits an assert failure.

Ignoring the assert would be wrong as it would break timeout logic, and sending
`FindNodes` twice in rapid succession also makes us a bad peer.

As a simple workaround, just skip `findNodes` in this state and return a fake
empty `Neighbours` reply.  This is a bit of a hack as `findNodes` should not be
called like this; there's a logic error at a higher level.  But it works.

Tested for about 4 days constant operation on Ropsten.  The crash which occured
every few minutes no longer occurs, and discv4 keeps working.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-02 23:29:02 +02:00
Kim De Mey c5dace27ca
Use chronos http server for dcli metrics and remove insecure compile flag (#343)
And add cfg file to default have runtime log filtering on
2021-04-02 17:29:38 +02:00
Kim De Mey 207b783253
Nimble file cleanup and group discv5 tests to compile once (#340) 2021-03-29 15:26:34 +02:00
Kim De Mey ae4177851f
Remove no longer needed try/except (#339) 2021-03-26 17:49:03 +01:00
Kim De Mey ac8aa0e037
Run discv5 tests with chronosStrictException on (#338) 2021-03-26 15:20:03 +01:00
Jacek Sieka 698b3bea88
Exception tracking (#334)
* Exception tracking

https://github.com/status-im/nim-chronos/pull/166

* Remove redundant raises annotation

Co-authored-by: kdeme <kim.demey@gmail.com>
2021-03-25 18:12:11 +01:00
Kim De Mey 8c27f291f5
Add raises annotation to the FilterMsgHandler proc type (#337) 2021-03-25 15:06:12 +01:00
Kim De Mey 0be863d2de
Add raises annotations to make exception tracking work (#336)
See https://github.com/status-im/nim-chronos/pull/166
2021-03-24 12:52:09 +01:00
Jacek Sieka 16802c0e52
Fix minor exception effect issues (#335)
Makes code compatible with
https://github.com/status-im/nim-chronos/pull/166 without requiring it.
2021-03-23 07:47:55 +01:00
Kim De Mey be5e088b21
Move parsing of nat cli option out of setupAddress (#331)
* Move parsing of nat cli option out of setupAddress

* Change some proc to func
2021-03-05 21:23:54 +01:00
Kim De Mey 0700ec770f
Improve network address discovery / NAT setup (#323)
* Add search for best route and refactor setupNat to setupAddress

* Update setupAddress and make enr ports in discovery optional

* Add specific error log when no route is found
* Use bindIP if it is public
* Adjust some log levels
2021-03-02 17:13:29 +01:00
Kim De Mey e8fbd3a83e
Add github actions timeout of 60 minutes (#328) 2021-02-17 15:43:05 +01:00
Kim De Mey be92a68d4c
Update discv5.md
Update some documentation in the discv5 docs.
2021-02-16 11:44:03 +01:00
jangko dda2bec8aa turn networkId into distinct uint 2021-02-13 17:43:17 +07:00
Zahary Karadjov e8c9691b35
awaitWithTimeout for Future[void] 2021-02-04 16:45:50 +02:00
Kim De Mey a339944bcf
Some discv5 cleanup / refactor (#326)
* Use Address in HandshakeKey

* Refactor + comments + dcli use queryRandom

* Rename types.nim to messages.nim
2021-02-02 22:47:21 +01:00
kdeme 4e58eb48ce Add comments on enrAutoUpdate and add metric 2021-02-01 20:30:55 +02:00
kdeme bfbad64d83 Add update of ENR on newly discovered address 2021-02-01 20:30:55 +02:00
kdeme 689eef2c11 Move IpAddress into the pong message 2021-02-01 20:30:55 +02:00
kdeme d33d27c2ee Small improvements and cleanup 2021-02-01 20:30:55 +02:00
kdeme e43ee6ef9c Add ip address voting through pong responses 2021-02-01 20:30:55 +02:00
Jacek Sieka 51a8795e56
one upnp is enough (#324)
we shouldn't have to recreate the upnp interface every time
2021-01-26 14:12:04 +01:00
Kim De Mey ab969f72c1
Update discv5.md 2021-01-14 09:26:19 +01:00
Kim De Mey e25500c9f0
Merge pull request #322 from status-im/refresh-only
Only do discovery queries to refresh the table
2021-01-13 22:40:35 +01:00
kdeme 181bbadcbc
Add more message related metrics 2021-01-13 21:44:17 +01:00
kdeme aa7442c9ff
dcli: actually use the provided listen address 2021-01-11 15:01:02 +01:00
kdeme 9cedbc0cc8 Move code into seedTable and populateTable proc 2021-01-07 10:20:40 +01:00
kdeme ac58a1f35c Only do discovery queries to refresh the table 2021-01-07 10:20:40 +01:00
andri lim 8de6bd4f48
Merge pull request #321 from status-im/github_action
add github action
2021-01-06 17:31:39 +07:00
jangko f50a68a992
add github action 2021-01-05 18:54:53 +07:00
Jacek Sieka 0f48ccecc0 sqlite: manual checkpointing 2020-12-18 19:30:53 +02:00
Kim De Mey a945eacfd3
Merge pull request #318 from status-im/lookup-adjust
Fix lookup to sort and query closest nodes
2020-12-18 14:24:50 +01:00
kdeme 12ec608efd
Put a limit on the nodes returned on a findnode request 2020-12-17 17:06:42 +01:00