Commit Graph

350 Commits

Author SHA1 Message Date
Viktor Trón b536460f8e
Merge pull request #17231 from ethersphere/develop
swarm: client-side MRU signatures ; BMT fixes ; network simulation tests
2018-07-24 08:44:43 +02:00
Wenbiao Zheng fe6a9473dc p2p: token is useless in xxxEncHandshake (#17230) 2018-07-23 17:36:08 +02:00
Janoš Guljaš dcaaa3c804 swarm: network simulation for swarm tests (#769)
* cmd/swarm: minor cli flag text adjustments

* cmd/swarm, swarm/storage, swarm: fix  mingw on windows test issues

* cmd/swarm: support for smoke tests on the production swarm cluster

* cmd/swarm/swarm-smoke: simplify cluster logic as per suggestion

* changed colour of landing page

* landing page reacts to enter keypress

* swarm/api/http: sticky footer for swarm landing page using flex

* swarm/api/http: sticky footer for error pages and fix for multiple choices

* swarm: propagate ctx to internal apis (#754)

* swarm/simnet: add basic node/service functions

* swarm/netsim: add buckets for global state and kademlia health check

* swarm/netsim: Use sync.Map as bucket and provide cleanup function for...

* swarm, swarm/netsim: adjust SwarmNetworkTest

* swarm/netsim: fix tests

* swarm: added visualization option to sim net redesign

* swarm/netsim: support multiple services per node

* swarm/netsim: remove redundant return statement

* swarm/netsim: add comments

* swarm: shutdown HTTP in Simulation.Close

* swarm: sim HTTP server timeout

* swarm/netsim: add more simulation methods and peer events examples

* swarm/netsim: add WaitKademlia example

* swarm/netsim: fix comments

* swarm/netsim: terminate peer events goroutines on simulation done

* swarm, swarm/netsim: naming updates

* swarm/netsim: return not healthy kademlias on WaitTillHealthy

* swarm: fix WaitTillHealthy call in testSwarmNetwork

* swarm/netsim: allow bucket to have any type for a key

* swarm: Added snapshots to new netsim

* swarm/netsim: add more tests for bucket

* swarm/netsim: move http related things into separate files

* swarm/netsim: add AddNodeWithService option

* swarm/netsim: add more tests and Start* methods

* swarm/netsim: add peer events and kademlia tests

* swarm/netsim: fix some tests flakiness

* swarm/netsim: improve random nodes selection, fix TestStartStop* tests

* swarm/netsim: remove time measurement from TestClose to avoid flakiness

* swarm/netsim: builder pattern for netsim HTTP server (#773)

* swarm/netsim: add connect related tests

* swarm/netsim: add comment for TestPeerEvents

* swarm: rename netsim package to network/simulation
2018-07-23 15:33:25 +02:00
jkcomment 65c91ad5e7 p2p: correct comments typo (#17184) 2018-07-18 10:41:18 +03:00
Anton Evangelatov 7c9314f231 swarm: integrate OpenTracing; propagate ctx to internal APIs (#17169)
* swarm: propagate ctx, enable opentracing

* swarm/tracing: log error when tracing is misconfigured
2018-07-13 17:40:28 +02:00
Felix Lange c73b654fd1 p2p/discover: move bond logic from table to transport (#17048)
* p2p/discover: move bond logic from table to transport

This commit moves node endpoint verification (bonding) from the table to
the UDP transport implementation. Previously, adding a node to the table
entailed pinging the node if needed. With this change, the ping-back
logic is embedded in the packet handler at a lower level.

It is easy to verify that the basic protocol is unchanged: we still
require a valid pong reply from the node before findnode is accepted.

The node database tracked the time of last ping sent to the node and
time of last valid pong received from the node. Node endpoints are
considered verified when a valid pong is received and the time of last
pong was called 'bond time'. The time of last ping sent was unused. In
this commit, the last ping database entry is repurposed to mean last
ping _received_. This entry is now used to track whether the node needs
to be pinged back.

The other big change is how nodes are added to the table. We used to add
nodes in Table.bond, which ran when a remote node pinged us or when we
encountered the node in a neighbors reply. The transport now adds to the
table directly after the endpoint is verified through ping. To ensure
that the Table can't be filled just by pinging the node repeatedly, we
retain the isInitDone check. During init, only nodes from neighbors
replies are added.

* p2p/discover: reduce findnode failure counter on success

* p2p/discover: remove unused parameter of loadSeedNodes

* p2p/discover: improve ping-back check and comments

* p2p/discover: add neighbors reply nodes always, not just during init
2018-07-03 16:24:12 +03:00
ethersphere e187711c65 swarm: network rewrite merge 2018-06-21 21:10:31 +02:00
Elad 1836366ac1 all: library changes for swarm-network-rewrite (#16898)
This commit adds all changes needed for the merge of swarm-network-rewrite.
The changes:

- build: increase linter timeout
- contracts/ens: export ensNode
- log: add Output method and enable fractional seconds in format
- metrics: relax test timeout
- p2p: reduced some log levels, updates to simulation packages
- rpc: increased maxClientSubscriptionBuffer to 20000
2018-06-14 11:21:17 +02:00
Felix Lange 0255951587 crypto: replace ToECDSAPub with error-checking func UnmarshalPubkey (#16932)
ToECDSAPub was unsafe because it returned a non-nil key with nil X, Y in
case of invalid input. This change replaces ToECDSAPub with
UnmarshalPubkey across the codebase.
2018-06-12 15:26:08 +02:00
Dmitry Shulyak 7677ec1f34 p2p/discv5: add egress/ingress traffic metrics to discv5 udp transport (#16369) 2018-05-29 13:46:09 +02:00
Felix Lange 6286c255f1
p2p/enr: updates for discovery v4 compatibility (#16679)
This applies spec changes from ethereum/EIPs#1049 and adds support for
pluggable identity schemes.

Some care has been taken to make the "v4" scheme standalone. It uses
public APIs only and could be moved out of package enr at any time.

A couple of minor changes were needed to make identity schemes work:

- The sequence number is now updated in Set instead of when signing.
- Record is now copy-safe, i.e. calling Set on a shallow copy doesn't
  modify the record it was copied from.
2018-05-17 15:11:27 +02:00
Guilherme Salgado c60f6f6214 p2p: don't discard reason set by Disconnect (#16559)
Peer.run was discarding the reason for disconnection sent to the disc
channel by Disconnect.
2018-05-09 01:20:20 +02:00
Ivan Daniluk c4a4613d95 p2p/simulations/adapters: fix websocket log line parsing in exec adapter (#16667) 2018-05-08 17:05:27 +02:00
kiel barry 864e80a48f p2p: fix some golint warnings (#16577) 2018-05-08 13:08:43 +02:00
GagziW f2447bd4c3 p2p: changed if-else blocks to conform with golint (#16660) 2018-05-03 11:33:39 +03:00
thomasmodeneis ba1030b6b8 build: enable goimports and varcheck linters (#16446) 2018-04-18 00:53:50 +02:00
David Huie 23ac783332 ecies: drop randomness parameter from `PrivateKey.Decrypt` (#16374)
The parameter `rand` is unused in `PrivateKey.Decrypt`. Decryption in
the ECIES encryption scheme is deterministic, so randomness isn't
needed.
2018-03-26 13:46:18 +03:00
JU HYEONG PARK 61c9730b2d p2p: fix doEncHandshake documentation (#16184) 2018-02-26 17:22:46 +01:00
Anton Evangelatov ae9f97221a metrics: pull library and introduce ResettingTimer and InfluxDB reporter (#15910)
* go-metrics: fork library and introduce ResettingTimer and InfluxDB reporter.

* vendor: change nonsense/go-metrics to ethersphere/go-metrics

* go-metrics: add tests. move ResettingTimer logic from reporter to type.

* all, metrics: pull in metrics package in go-ethereum

* metrics/test: make sure metrics are enabled for tests

* metrics: apply gosimple rules

* metrics/exp, internal/debug: init expvar endpoint when starting pprof server

* internal/debug: tiny comment formatting fix
2018-02-23 11:56:08 +02:00
Ivan Daniluk 8522b31221 p2p: remove unused code (#16158)
* p2p: remove unused code

* p2p: remove unused imports
2018-02-22 19:20:28 +02:00
Balint Gabor 221486a291
Merge pull request #15919 from ethersphere/p2p-protocols-pr
p2p/protocols, p2p/testing: protocol abstraction and testing
2018-02-22 15:02:51 +01:00
Anton Evangelatov 1e457b6599 p2p: don't send DiscReason when using net.Pipe (#16004) 2018-02-22 11:41:06 +01:00
Felix Lange 28b20cff4b p2p/protocols: gofmt -w -s 2018-02-22 11:37:57 +01:00
Dmitry Shulyak 14c76371ba p2p: when peer is removed remove it also from dial history (#16060)
This change removes a peer information from dialing history
when peer is removed from static list. It allows to force a
server to re-dial concrete peer if it is needed.

In our case we are running geth node on mobile devices, and
it is common for a network connection to flap on mobile.
Almost every time it flaps or network connection is changed
from cellular to wifi peers are disconnected with read
timeout. And usually it takes 30 seconds (default expiration
timeout) to recover connection with static peers after
connectivity is restored.

This change allows us to reconnect with peers almost
immediately and it seems harmless enough.
2018-02-21 15:03:26 +01:00
Janos Guljas e07603bbc4 p2p/testing: check for all expectations in TestExchanges
Handle all expectations in ProtocolSession.TestExchanges in any
order that are received.
2018-02-17 23:42:28 +01:00
Felix Lange aeedec4078 p2p/discover: s/lastPong/bondTime/, update TestUDP_findnode
I forgot to change the check in udp.go when I changed Table.bond to be
based on lastPong instead of node presence in db. Rename lastPong to
bondTime and add hasBond so it's clearer what this DB key is used for
now.
2018-02-16 21:29:20 +01:00
Péter Szilágyi 32301a4d6b
p2p/discover: validate bond against lastpong, not db presence 2018-02-16 17:05:08 +02:00
Felix Lange a5c0bbb4f4
all: update license information (#16089) 2018-02-14 13:49:11 +01:00
Péter Szilágyi 20797348ca
p2p/discover: fix out-of-bounds issue 2018-02-13 20:59:43 +02:00
Martin Holst Swende 589b603a9b rpc: dns rebind protection (#15962)
* cmd,node,rpc: add allowedHosts to prevent dns rebinding attacks

* p2p,node: Fix bug with dumpconfig introduced in r54aeb8e4c0bb9f0e7a6c67258af67df3b266af3d

* rpc: add wildcard support for rpcallowedhosts + go fmt

* cmd/geth, cmd/utils, node, rpc: ignore direct ip(v4/6) addresses in rpc virtual hostnames check

* http, rpc, utils: make vhosts into map, address review concerns

* node: change log messages to use geth standard (not sprintf)

* rpc: fix spelling
2018-02-12 14:52:07 +02:00
Felix Lange 9123eceb0f p2p, p2p/discover: misc connectivity improvements (#16069)
* p2p: add DialRatio for configuration of inbound vs. dialed connections

* p2p: add connection flags to PeerInfo

* p2p/netutil: add SameNet, DistinctNetSet

* p2p/discover: improve revalidation and seeding

This changes node revalidation to be periodic instead of on-demand. This
should prevent issues where dead nodes get stuck in closer buckets
because no other node will ever come along to replace them.

Every 5 seconds (on average), the last node in a random bucket is
checked and moved to the front of the bucket if it is still responding.
If revalidation fails, the last node is replaced by an entry of the
'replacement list' containing recently-seen nodes.

Most close buckets are removed because it's very unlikely we'll ever
encounter a node that would fall into any of those buckets.

Table seeding is also improved: we now require a few minutes of table
membership before considering a node as a potential seed node. This
should make it less likely to store short-lived nodes as potential
seeds.

* p2p/discover: fix nits in UDP transport

We would skip sending neighbors replies if there were fewer than
maxNeighbors results and CheckRelayIP returned an error for the last
one. While here, also resolve a TODO about pong reply tokens.
2018-02-12 14:36:09 +02:00
Felföldi Zsolt c4712bf96b p2p/discv5: fix multiple discovery issues (#16036)
* p2p/discv5: add query delay, fix node address update logic, retry refresh if empty

* p2p/discv5: remove unnecessary ping before topic query

* p2p/discv5: do not filter local address from topicNodes

* p2p/discv5: remove canQuery()

* p2p/discv5: gofmt
2018-02-08 19:06:31 +02:00
Felföldi Zsolt 6198c53e28 p2p/discv5: fix removeTicketRef cached ticket removal (#15995) 2018-01-30 18:01:22 +02:00
Felföldi Zsolt 397c6cde1e p2p/discv5: fix topic register panic at shutdown (#15946) 2018-01-23 12:53:09 +02:00
Martin Holst Swende 48641d7308
p2p/discv5: logs info about discv5 node info at bind time 2018-01-23 08:50:11 +01:00
Felföldi Zsolt 92580d69d3 p2p, p2p/discover, p2p/discv5: implement UDP port sharing (#15200)
This commit affects p2p/discv5 "topic discovery" by running it on
the same UDP port where the old discovery works. This is realized
by giving an "unhandled" packet channel to the old v4 discovery
packet handler where all invalid packets are sent. These packets
are then processed by v5. v5 packets are always invalid when
interpreted by v4 and vice versa. This is ensured by adding one
to the first byte of the packet hash in v5 packets.

DiscoveryV5Bootnodes is also changed to point to new bootnodes
that are implementing the changed packet format with modified
hash. Existing and new v5 bootnodes are both running on different
ports ATM.
2018-01-22 13:38:34 +01:00
zelig 407339085f p2p/protocols, p2p/testing: protocol abstraction and testing 2018-01-18 10:53:47 +01:00
Felix Lange 5c2f1e0014 all: update generated code (#15808)
* core/types, core/vm, eth, tests: regenerate gencodec files

* Makefile: update devtools target

Install protoc-gen-go and print reminders about npm, solc and protoc.
Also switch to github.com/kevinburke/go-bindata because it's more
maintained.

* contracts/ens: update contracts and regenerate with solidity v0.4.19

The newer upstream version of the FIFSRegistrar contract doesn't set the
resolver anymore. The resolver is now deployed separately.

* contracts/release: regenerate with solidity v0.4.19

* contracts/chequebook: fix fallback and regenerate with solidity v0.4.19

The contract didn't have a fallback function, payments would be rejected
when compiled with newer solidity. References to 'mortal' and 'owned'
use the local file system so we can compile without network access.

* p2p/discv5: regenerate with recent stringer

* cmd/faucet: regenerate

* dashboard: regenerate

* eth/tracers: regenerate

* internal/jsre/deps: regenerate

* dashboard: avoid sed -i because it's not portable

* accounts/usbwallet/internal/trezor: fix go generate warnings
2018-01-08 14:15:57 +02:00
ferhat elmas 5866626b08 core, p2p/discv5: use time.NewTicker instead of time.Tick (#15747) 2018-01-02 12:50:46 +01:00
Anton Evangelatov 36a10875c8 p2p/enr: initial implementation (#15585)
Initial implementation of ENR according to ethereum/EIPs#778
2017-12-29 21:18:51 +01:00
Péter Szilágyi c15d76a40f p2p/discv5: fix reg lookup, polish code, use logger (#15737) 2017-12-28 14:17:03 +01:00
ferhat elmas afa3c72c40 p2p/discover: fix leaked goroutine in data expiration 2017-12-18 09:16:54 +01:00
Felix Lange 3654aeaa4f
p2p/simulations: fix gosimple nit (#15661) 2017-12-13 03:15:27 +01:00
holisticode fd777bb210 p2p/simulations: add mocker functionality (#15207)
This commit adds mocker functionality to p2p/simulations. A
mocker allows to starting/stopping of nodes via the HTTP API.
2017-12-12 19:10:41 +01:00
Zach 3da1bf8ca1 all: use gometalinter.v2, fix new gosimple issues (#15650) 2017-12-12 19:05:47 +01:00
ferhat elmas 1d06e41f04 p2p, swarm/network/kademlia: use IsZero to check for zero time (#15603) 2017-12-04 11:07:10 +01:00
Lewis Marshall 54aeb8e4c0 p2p/simulations: various stability fixes (#15198)
p2p/simulations: introduce dialBan

- Refactor simulations/network connection getters to support
  avoiding simultaneous dials between two peers If two peers dial
  simultaneously, the connection will be dropped to help avoid
  that, we essentially lock the connection object with a
  timestamp which serves as a ban on dialing for a period of time
  (dialBanTimeout).

- The connection getter InitConn can be wrapped and passed to the
  nodes via adapters.NodeConfig#Reachable field and then used by
  the respective services when they initiate connections. This
  massively stablise the emerging connectivity when running with
  hundreds of nodes bootstrapping a network.

p2p: add Inbound public method to p2p.Peer

p2p/simulations: Add server id to logs to support debugging
in-memory network simulations when multiple peers are logging.

p2p: SetupConn now returns error. The dialer checks the error and
only calls resolve if the actual TCP dial fails.
2017-12-01 12:49:04 +01:00
ferhat elmas 86f6568f66 build: enable unconvert linter (#15456)
* build: enable unconvert linter

 - fixes #15453
 - update code base for failing cases

* cmd/puppeth: replace syscall.Stdin with os.Stdin.Fd() for unconvert linter
2017-11-10 19:06:45 +02:00
Darrel Herbst d54e3539d4 p2p/nat: delete port mapping before adding (#15222)
Fixes #1024
2017-10-06 13:39:47 +02:00
Péter Szilágyi 2ee885958b p2p: snappy encoding for devp2p (version bump to 5) (#15106)
* p2p: snappy encoding for devp2p (version bump to 5)

* p2p: remove lazy decompression, enforce 16MB limit
2017-09-26 16:54:49 +03:00