Matt Joiner
3d3052b8bd
Merge branch 'request-strategy-experiments'
...
This should bring in significant performance improvements that fix issues with the peer-requesting that existed from v1.34.0.
2021-12-12 18:38:33 +11:00
Matt Joiner
04beb8937f
More consistent cancellation management
2021-12-12 18:35:02 +11:00
Matt Joiner
30375615b3
Remove unused request strategy code
2021-12-12 18:35:02 +11:00
Matt Joiner
1bc84e316f
Don't group Torrent piece request orders without a storage capacity together
2021-12-12 18:35:02 +11:00
Matt Joiner
8c5df03e14
Minimize allocations due to heap escapes
2021-12-12 18:35:01 +11:00
Matt Joiner
8c9a308c9a
Only steal an odd request if the stealer more recently received a chunk
...
This helps break the stealing cycle during endgame, and lets us trickle the request to the peer conn with the best record. It might not be sufficient but works nice in testing so far.
2021-12-12 18:35:01 +11:00
Matt Joiner
7de7b986c9
Only use last requested when both requests are already pending
2021-12-12 18:35:01 +11:00
Matt Joiner
ddf41fc2f1
Add Torrent-level request cancel for consistency
2021-12-12 18:35:01 +11:00
Matt Joiner
aca22a0896
Update peer cancel assumptions and return
2021-12-12 18:35:01 +11:00
Matt Joiner
1b92249f2a
Improve panic message
2021-12-12 18:35:01 +11:00
Matt Joiner
117ae28b38
Try request stealing
2021-12-12 18:35:01 +11:00
Matt Joiner
73be571f50
Don't try to request anything without the torrent info
2021-12-12 18:35:01 +11:00
Matt Joiner
99a64bcf63
Wait between duplicate requests
2021-12-12 18:35:01 +11:00
Matt Joiner
8025d15155
Stop iterating pieces when storage is exhausted
2021-12-12 18:35:01 +11:00
Matt Joiner
135f21fb64
Use interfaces to lazily expose the bare minimum inputs to GetRequestablePieces
2021-12-12 18:35:01 +11:00
Matt Joiner
4088e847f3
Provide mapping from infohash to Torrent in Input
2021-12-12 18:35:01 +11:00
Matt Joiner
94bb5d40ba
Implement piece request ordering with retained state
2021-12-12 18:35:01 +11:00
Matt Joiner
b99dd505b5
Avoid reflection sorting request pieces
2021-12-12 18:35:01 +11:00
Matt Joiner
de2bea502c
Use indices to lookup pieces for comparison
2021-12-12 18:35:01 +11:00
Matt Joiner
231301f5d7
Cache piece request orderings
2021-12-12 18:35:01 +11:00
Matt Joiner
8e99558e11
Retract last few minor versions with peer-requesting issues
...
(cherry picked from commit 85a436a5ca75a47a52dcab9648d85f3c3fb53e35)
2021-12-12 18:34:31 +11:00
Matt Joiner
cd9a3a14ad
Revert "Link to godocs.io for documentation"
...
This reverts commit c2ed60f9f2d100ddd15a38a6c260121a9ad6e893.
2021-12-12 17:20:02 +11:00
Matt Joiner
67ed5d0032
Partition piece request strategy by storage capacity key
2021-12-12 16:56:01 +11:00
Matt Joiner
ad082bc644
Do webseed request updates asynchronously
...
Fixes a bug where received chunks are immediately requested while still being processed.
2021-12-12 16:56:01 +11:00
Matt Joiner
6625b05da4
Add more identifying information to PeerConn.String
...
Some logs for unexpected protocol violations could be helped by them.
2021-12-12 16:56:01 +11:00
Matt Joiner
1d787c3923
Add generalized decodeJsonByteString and a fuzz target for it
2021-12-12 16:56:01 +11:00
Matt Joiner
14ee69aaeb
Add failing bencode fuzz input
2021-12-12 16:56:01 +11:00
Matt Joiner
96ef73aeb6
Transmission does not reject cancelled requests
2021-12-12 16:56:01 +11:00
Matt Joiner
388bfafa01
Don't log webseed request context cancellation results
2021-12-12 16:56:01 +11:00
Matt Joiner
cd7e6dcd5c
bencode: Avoid allocating interface string up front
...
This should fix crashes during fuzzing. It's not a complete fix, we really want to limit the amount a given Decode can allocate. Maybe Go isn't the right language for this.
2021-12-12 16:56:01 +11:00
Matt Joiner
79511abe6c
bencode: Fix decoding integer with leading +
2021-12-12 16:56:01 +11:00
Matt Joiner
8764456d23
bencode: Enforce dict key ordering
...
Fix bencode dict key ordering in HTTP tracker test
2021-12-12 16:56:01 +11:00
Matt Joiner
78e48f71dd
bencode incorrectly parsed integers with leading zeroes
2021-12-12 16:56:01 +11:00
Matt Joiner
42dca16753
Add bencode FuzzInterfaceRoundTrip
2021-12-12 16:56:01 +11:00
Matt Joiner
4cfdc2f497
Shuffle duplicate requests
...
Add missing import
2021-12-12 16:56:00 +11:00
Matt Joiner
4913f17c01
Generate full sintel magnet link in fs test
2021-12-12 15:01:50 +11:00
Matt Joiner
e16c799638
Add sintel.torrent
2021-12-12 15:01:50 +11:00
Matt Joiner
8dc897d4a2
Fix races using resources on Close
2021-12-12 15:01:50 +11:00
Matt Joiner
69f3b6064e
Don't use the Client's IP blocklist after the Client is closed
...
Fixes a segfault due to using a memory mapped blocklist in confluence.
2021-12-12 15:01:50 +11:00
Matt Joiner
54d529aac0
Remove unused torrent_pending_pieces.go
...
Unfortunately github.com/elliotchance/orderedmap is still used in ./analysis, so no deps are removed.
2021-12-12 15:01:50 +11:00
Matt Joiner
ba592028e5
Filter update requests on piece priority change by peer choking and allowed fast
2021-12-12 15:01:50 +11:00
Matt Joiner
a9485e8796
Ensure peerRequests.torrentStrategyInput is set
2021-12-12 15:01:50 +11:00
Yaroslav Kolomiiets
102f3182fc
PEX: remove workaround for krpc marshaller
2021-12-11 19:44:42 +00:00
Yaroslav Kolomiiets
bdb218a698
PEX: use new NodeAddr search methods in krpc
2021-12-11 19:18:30 +00:00
Yaroslav Kolomiiets
4a06517856
PEX: fluid event log
2021-12-12 00:27:28 +11:00
Matt Joiner
7197c5f493
Fix wasm build
...
This was broken in the move to zombiezen sqlite, which still doesn't work on WASM/JS.
2021-12-07 13:08:06 +11:00
Matt Joiner
22053c9238
Don't fail-fast on go versions
2021-12-07 12:42:35 +11:00
Matt Joiner
a2c50ea2bd
Handle 503 returns from webseed peer endpoints
2021-12-06 15:24:28 +11:00
Matt Joiner
32d9ec900c
Record an observation about why webseed peers are doing most of the work in recent testing
2021-12-06 15:22:06 +11:00
Matt Joiner
c653cf2070
Do webseed request parts sequentially
...
This means we can treat the number of bytes in the result with enough accuracy to decide if we should count it as a wasted chunk. Also I'm not sure why it was a good idea to do parts of a request in parallel anyway, it could just lead to spikes in outstanding requests to the webseed, rather than sticking to the predictable maxRequests limit.
2021-12-06 15:22:00 +11:00
