Commit Graph

208 Commits

Author SHA1 Message Date
Steven Allen c39da30592 chore: update deps 2021-03-08 13:35:26 -08:00
vyzo 2a69faae62 use transient connections in identify streams 2021-02-25 08:45:16 +02:00
Aarsh Shah 69916ed465
Emit event for User's NAT Type i.e. Hard NAT or Easy NAT (#1042)
* emit NAT device type
2021-02-19 15:14:03 +05:30
Aarsh Shah 1c850e1293
Timeout all Identify stream reads (#1032)
* fix: read timeouts on Identify protocols

* fixed tests

* review and go fmt
2021-01-14 11:47:05 +05:30
Marten Seemann 34eb40850b use a context when opening streams 2020-12-19 12:05:19 +07:00
Steven Allen 7a98f28856 feat: update to go-libp2p-core 0.7.0 interface changes
This patch updates go-libp2p for the stream interface changes in go-libp2p-core
0.7.0. This is a _significant_ breaking change to streams and all users should
read https://github.com/libp2p/go-libp2p-core/releases/tag/v0.7.0. In practice,
this change should remove a significant footgun.

TL;DR:

* `Stream.Close` now behaves like `net.TCPConn.Close`.
* There is a new `Stream.CloseWrite` (send an EOF) and `Stream.CloseRead` (close
  for reading), behaving like their counterparts in `net.TCPConn`.
2020-11-11 09:35:37 -08:00
Steven Allen de10591dd8 fix tests
1. Fix tests on go 1.15 by avoiding `string(i)`.
2. Fix the autorelay test by making relays announce fake "public" addresses. I'm
not sure how this worked before.
2020-08-19 11:03:55 -07:00
Raúl Kripalani a5786f0415
upgrade deps + interoperable uvarint delimited writer/reader. (#985) 2020-07-30 12:42:57 +01:00
Steven Allen 34e67e019e Fix peer handler race
Alternative to #963. Instead of using atomics, this patch ensures that all state
changes are performed on the main thread. It also:

* Threads the context through explicitly to make sure fields changed by the main
thread are never read by the handler goroutines. This isn't strictly necessary
but it makes it easier to reason about.
* Uses a callback to signal "on done". There may be cleaner ways to do this (I
hate callbacks), but this nicely decouples the peer handler from the main loop.

fixes #963
2020-06-05 10:20:28 -07:00
Steven Allen 5ae0888629 fix(id): document observation struct 2020-06-04 14:57:07 -07:00
Steven Allen a03403fa64 fix: optimize numInbound count
We call this _very_ frequently when computing our local addresses.
2020-06-04 14:55:17 -07:00
Steven Allen 1e4ce46ca7 test(id): fix nits 2020-06-04 14:55:17 -07:00
Steven Allen 1dd7a8c4d4 fix: count observations with at least one inbound connection as inbound
We prefer addresses from inbound connections. We don't want outbound connections
to hide these perfectly good addresses.
2020-06-04 14:55:13 -07:00
Steven Allen a0813d7292 fix: make address observation types private
We don't use and/or expose these anyways. Making them private makes it easier to
reason about their state.
2020-06-04 14:55:10 -07:00
Steven Allen b22add866a fix: remove more field
We no longer need it now that we keep reading till the stream closes.
2020-06-03 15:41:08 -07:00
Aarsh Shah 513e13d0d2 changes as per review 2020-06-03 15:27:03 -07:00
Aarsh Shah c4a1e220ac test large push msgs 2020-06-03 15:27:02 -07:00
Aarsh Shah 60b52f10ae more protocol strings 2020-06-03 15:27:02 -07:00
Aarsh Shah 1b36fd8697 first draft 2020-06-03 15:27:01 -07:00
Steven Allen 294ba3dfac
Merge pull request #953 from libp2p/fix/remove-addresses-immediately
fix: remove old addresses in identify immediately
2020-05-27 19:14:33 -07:00
Steven Allen be70fa2460 test: fix test for immediately forgetting addresses 2020-05-27 16:58:53 -07:00
Steven Allen 59062c6d6d fix: remove old addresses in identify immediately
Previously, we'd keep addresses discovered through the DHT for up to 2
minutes (temporary TTL) and previously seen addresses (recently connected) for
up to 10 minutes (the TTL).

1. Make sure to downgrade both connected and recently connected addresses to the
"temporary" ttl before adding new addresses.
2. Finally, downgrade addresses with the temporary TTL to 0.

This could be more efficient with a better peerstore abstraction, but this is
better than nothing.
2020-05-27 16:22:39 -07:00
Aarsh Shah 0dd65a5998
fix flaky test (#952)
Fix flaky TestIdentifyPushOnAddrChange
2020-05-21 12:02:29 +05:30
Steven Allen 6ef5f5dacf fix: group observations by zeroing port
In #917, we started dropping additional address observations if we had multiple
for the same transport set. However, on further consideration, this isn't quite
correct. We _want_ to keep additional observations for multiple IP addresses.
The real issue is many observations for different ports.

So this patch simply changes the key with which we group observations from
"address protocols" to "address without the port" (well, with the port set to
0).
2020-05-20 12:13:25 -07:00
Aarsh Shah ba118a4d14 create subscriber first 2020-05-20 15:10:04 +05:30
Aarsh Shah 79ead33e32
Filter observed addresses (#917)
* filter observed addrs
2020-05-20 11:39:45 +05:30
Steven Allen 01da330ef2 fix: don't try to marshal a nil record
fixes #939
2020-05-19 18:15:33 -07:00
Steven Allen e3a983a355 fix: don't leak a goroutine if a peer connects and immediately disconnects 2020-05-19 13:37:52 -07:00
Aarsh Shah a7671efe6b
no signed peer records for mocknets (#934)
* Disable generation of Signed peer record for Mockenets.
2020-05-19 13:16:54 +05:30
Steven Allen b42ba0faf3
various identify fixes and nits (#922)
* various identify fixes and nits

Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
2020-05-14 17:24:10 +05:30
Aarsh Shah 973933ad7d
Remove race between ID, Push & Delta (#907)
* Solve races between Identify, Push and Delta.
2020-05-13 10:50:37 +05:30
Raúl Kripalani d9d18512ca
fix a compilation error introduced in 077a818. (#919) 2020-05-12 17:24:58 +01:00
Yusef Napora 077a81814f
exchange signed routing records in identify (#747)
*  Exchange signed routing records in identify


Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
2020-04-30 11:06:31 +05:30
Steven Allen 84b923c7bf fix: address code review 2020-04-24 23:54:07 -07:00
Steven Allen 06d9fc7987 fix: keep observed addrs alive as long as their associated connections are alive
Otherwise, if we're not creating new connections, we'll eventually forget them.

Up until now, this wasn't _too_ much of an issue because our peers would still
remember our addresses. However, we now _tell_ our connected peers when our
addresses change. That means we'll tell our peers to forget where we are,
preventing anyone from finding us.
2020-04-24 22:55:16 -07:00
Steven Allen d3b7b3b907 test: fix a flaky test 2020-04-24 19:33:19 -07:00
Aarsh Shah 3d676b688c skip test local addr filtering 2020-04-24 19:05:23 -07:00
Steven Allen bd040ba050 fix: refactor logic for identifying connections
0. NEVER call `peerstore.SetProtocols(p)` (clear the protocol set). Given the
   new identify events, if someone looked in the peerstore at the wrong time, they
   could decide that the peer no longer speaks some protocol.
1. Reliably wait for identify before trying to open a stream. The old logic was
   _really_ racy.
2. Avoids potentially calling identify on the same connection multiple times.
3. Calls identify as early as possible. Previously, we'd invoke identify on
   inbound connections using an event that was only invoked _after_ all `Connected`
   event handlers completed. Now we invoke identify from a `Connected` handler.
2020-04-24 19:05:22 -07:00
Aarsh Shah 615f125342
Remove goprocess from Host (#865)
* remove goprocess from Host
2020-04-06 11:34:45 +05:30
Steven Allen 6cdc404180 fix: remove an unnecessary goroutine
The stream handler runs in a goroutine anyways. We might as well block it.
2020-03-02 15:51:36 -08:00
Steven Allen 7ba322244e emit identify events and avoid advertising localhost
* fix(identify): announce localhost as long as one side of the connection is local (#742)
* identify: emit events on completion/failure. (#660)
* identify: Update addr advertise logic to exclude localhost addrs… (#657)

This is a rollup of the stabilize fixes.

Co-authored-by: bigs <cole@protocol.ai>
Co-authored-by: Raúl Kripalani <raul@protocol.ai>
2020-02-21 12:45:25 -05:00
Hector Sanjuan 7e49c66219 Identify: Make activation threshold configurable
Also: fix the comments and try to explain how the activation/aliveness of
addresses happen (and write the correct defaults).
2019-11-05 12:18:55 +01:00
Steven Allen 94d7efc58e chore(dep): update libp2p-core and protobuf
* Enforces 2048 bit RSA keys when using OpenSSL
* Updates go-multiaddr-dns for /dns support.
* Update gogo/protobuf and regenerate all protobufs.
2019-10-28 16:41:54 -07:00
Raúl Kripalani 9cf6af474b add godocs. 2019-08-28 11:50:26 +01:00
Steven Allen e337633824 set a default client-version using go modules
This should help us improve network stats on who's using libp2p.
2019-08-12 19:13:22 -07:00
Steven Allen d73a51776e new: allow overriding the user-agent with an option
Instead of using a global variable.

This also:

* Adds an option to the identify service to set the user agent.
* Removes the ability to pass an identify service to NewHost as any reasonable
  Identify service already needs to be constructed with an instance of the host.
2019-08-12 19:13:21 -07:00
Christian Muehlhaeuser 169f2d4139 Avoid unnecessary conversions (#685)
No need to convert these types.
2019-07-22 13:46:15 +01:00
Christian Muehlhaeuser 40cb489097 Fixed typos (#680)
Just a few nit-picky typo fixes.
2019-07-22 13:44:24 +01:00
Raúl Kripalani e69d17141c
integrate the event bus, handle protocol update events, make identify emit deltas (#659) 2019-06-24 14:44:47 +01:00
Raúl Kripalani d87f89314c
Consolidate abstractions and core types into go-libp2p-core (#601) 2019-05-26 22:55:46 +01:00
Steven Allen d0ab45164c ping: simplify ping loop 2019-05-07 15:47:55 -07:00
Steven Allen d3d5351d81 ping: return a stream of results
Otherwise, we can't return errors. This is a breaking change but unlikely to
have a large impact on anyone but go-ipfs.

Part of https://github.com/ipfs/go-ipfs/issues/6298
2019-05-07 15:47:52 -07:00
vyzo 3697552406 handle misbehaving peers in identify push 2019-04-19 20:03:53 +03:00
vyzo 8fa257cdf0 track context given in identify constructor and use it to gate identify push 2019-04-19 20:03:53 +03:00
vyzo bf651ca7c7 use a single, NoDial context in identify push 2019-04-19 20:03:53 +03:00
vyzo a88ae79f6f use transientTTL for invalidated addrs instead of 0 2019-04-16 21:09:15 +03:00
vyzo 06391d4f2f replace peer addresses in identify 2019-04-16 11:49:32 +03:00
vyzo a4776e6174 fix panic in observed address activation check
The activated check cleans up, which leads to panics with concurrent reads.
This moves the seenBy set clean up into gc, where it belongs
2019-04-09 21:52:40 +03:00
vyzo efdc140264 raise activation channel capacity to 16 2019-04-09 20:35:36 +03:00
vyzo fe7ba05e38 delete empty address sets on observed address set gc 2019-04-09 14:00:03 +03:00
vyzo 17001b24dd use a read/write lock for observed address set 2019-04-09 13:40:46 +03:00
vyzo f4af8d493d fix identify tests 2019-04-09 13:20:38 +03:00
vyzo 9b8192b7d2 initialize ttl in ObservedAddrSet 2019-04-09 13:19:54 +03:00
vyzo b8a152f593 use a background worker for updating/gcing the observed address set 2019-04-09 12:44:13 +03:00
vyzo e76c8a9c38 don't eagerly update the observed address set in Addrs
allocates and eats a lot of cpu time under the lock;
let a background worker gc periodically.
2019-04-09 12:13:30 +03:00
Steven Allen 316f0867f6 identify: avoid parsing/printing multiaddrs 2019-04-08 18:43:11 -07:00
Steven Allen aebe405680 test: disable TestAddAddrsProfile when the race detector is enabled 2019-04-08 18:30:59 -07:00
whyrusleeping a7eb2efe4b move things outside of the lock in obsaddr 2019-04-08 16:14:05 -07:00
Steven Allen bcbf7a59c1 identify: only record observed addresses that match an address we're announcing
This is should prevent us from, e.g., announcing relay addresses _just_ because
a peer tells us we're reachable through a relay.
2019-04-08 10:15:42 -07:00
Steven Allen 544c412193 identify: only store _reported_ multiaddrs
We still tell the remote host about the observed addr but we don't store it.
That way, we give them a chance to decide if they want to actually use and
advertise it.

Ideally, we'd distinguish between local information and signed routing
information but we don't do that yet.

This should reduce the address explosion issue where peers learn about
multiple (bad) observed addresses for the same peer. It should also give peers
more control over how they can be dialed.
2019-04-05 13:57:50 -07:00
lnykww 5da711dc6c Fix UDP listen on a Unspecified Address and Dial from the Unspecified Address
LocalMultiaddr May return the Unspecified Address, if so, we need to
check if we listen on the Unspecified address and they were equal.
2019-03-14 10:57:19 +08:00
Steven Allen 3617750071 better nat mapping
1. Update to work with https://github.com/libp2p/go-libp2p-nat/pull/14.
2. Avoid observed addrs when our NAT tells us about external addrs.
3. Ignore bad addrs reported by our NAT. Substitute with observed addrs.
4. Fix https://github.com/libp2p/go-libp2p/issues/428.
2019-03-06 10:56:49 -08:00
Jakub Sztandera 533d9866c6 gx publish 6.0.34 2019-02-18 16:01:43 +01:00
Abhishek Upperwal 36a2645306 removing redundant if condition 2018-12-02 01:00:46 +05:30
Abhishek Upperwal 3ae2d75e46 added comments for clarity 2018-12-02 00:56:46 +05:30
Abhishek Upperwal 88d98287ff suppressing error msg for NoSecurity option 2018-11-30 16:54:25 +05:30
Steven Allen 00ba6a165a deflake identify test
This was probably failing (rarely) due to the fact that we're shrinking the
timeout asynchronously (I think?).
2018-11-08 10:49:57 -08:00
vyzo 7fabe39290 parallel identify push 2018-11-04 10:52:29 +02:00
vyzo 4ea04c23b3 identify: implement identify push protocol 2018-11-04 10:52:29 +02:00
Steven Allen 7eaeeaa9fe
Merge pull request #461 from libp2p/fix/remove-protocol-version-check
don't disconnect on protocol version mismatch
2018-10-24 07:35:23 -07:00
Steven Allen dd7868a1a7 don't disconnect on protocol version mismatch
Sending a protocol version is nice. However, this "disconnect if our versions
are different" logic makes the version entirely useless (because we can't change
it).

Really, each indevidual protocol is versioned so let's just leave it at that. If
we make a breaking change that requires a protocol bump, we can do that and
then switch on the other side's version. However, we'll have to wait for the
entire network to upgrade for that to work.
2018-10-24 07:00:42 -07:00
vyzo 48e0effd6a resolve import cycle in ping test 2018-10-19 12:47:08 +03:00
vyzo a768598c50 free standing Ping function 2018-10-19 12:22:13 +03:00
Can ZHANG 92ec4b2d18 Updates due to code review 2018-09-29 20:16:09 +08:00
Can ZHANG 96df62bd5b Changes on discussion
- map internal -> []{external -> { observer -> [time, direction] } }
- some cleaning
2018-09-26 16:52:46 +08:00
Can ZHANG b4e4d496ea Fix test 2018-09-26 16:52:46 +08:00
Can ZHANG d859fb0dd7 Track more info for observed addresses 2018-09-26 16:52:46 +08:00
QYuan 38abbc9128
Delete extra comments 2018-09-21 10:33:15 +08:00
Steven Allen e436bcc391 always remove connection from identify service map
fixes #419

Also call FullClose in a goroutine; no need to block this.

(not happy with that but I'm starting to think we need to rethink stream
closing, again...)
2018-09-10 15:51:09 -07:00
Steven Allen 7d6f9521c2
Merge pull request #380 from libp2p/fix/reset-ping-stream
always reset ping streams when done
2018-08-21 05:22:22 +00:00
Steven Allen f0eba35c6e
Merge pull request #388 from libp2p/fix/id-reset
reset when we fail in identify
2018-08-16 17:43:59 +00:00
Steven Allen f6ad991bb5 update gogo protobuf
and regenerate protobuf files
2018-08-08 18:08:13 -07:00
Steven Allen 2e13245085 reset when we fail in identify 2018-08-08 11:48:45 -07:00
Steven Allen 72df25af76 always reset ping streams when done
Currently, we leak a stream. We could use FullClose but, IMO, it's not worth it.
We might as well just reset and walk away.
2018-07-27 10:54:34 -07:00
Steven Allen b4428fda58 reduce error to warning and log the error
This happens all the time in tests where we intentionally use fake keys for
performance.

Anyways, users probably don't want their logs spammed with errors they can't do
anything about.
2018-06-15 14:27:22 -07:00
Steven Allen 41c6834850 refactor for transport changes
Also, make the libp2p constructor fully useful. There should now be no need to
manually construct a swarm/host.
2018-06-04 21:22:24 -07:00
Steven Allen 5f75373b53 make the identify test less racy 2018-01-27 21:14:08 -08:00
Steven Allen d7c4c0a202 add test for forgetting address records 2018-01-19 22:37:14 -08:00
Steven Allen ea95a94e78 register identify service netNotify
This way, we actually process disconnect notifications (and reduce the lifetime
on peer addr records).
2018-01-19 22:36:33 -08:00
Steven Allen 096a2c8303 fix peerstore apocalypse redux
This commit prevents us from repeatedly extending the lifetimes of all observed
addresses if a peer keeps on reconnecting.

It also fixes two race conditions:

1. We may end up processing a disconnect after a re-connect and may
accidentally give the addresses associated with that peer a
RecentlyConnectedAddrTTL instead of a ConnectedAddrTTL.

2. We may end up processing a connect after the associated disconnect storing
the associated peer addresses indefinitely.
2018-01-19 21:52:22 -08:00