f235c8ea51
Reuse piece order state for request ordering
2022-05-12 10:34:28 +10:00
23a0e7f7d3
Remove redundant type conversions
2022-05-09 12:51:02 +10:00
a54d2d81e4
Retain peer local request ordering
2022-05-09 12:05:50 +10:00
9a9c7dee00
Add and use typed roaring bitmap
2022-05-09 12:05:50 +10:00
a1a820d3c9
Avoid heap allocation in GetRequestablePieces ( #734 )
...
The calculation of whether we should ignore a piece in
GetRequestablePieces ends up doing an allocation for every piece, when
all we really need to do is query the index in the torrent. Provide an
IgnorePiece function instead, which avoids the need for a temporary
allocation.
Observed to cut out 40% of object allocations in some workloads (large
download, lots of seeds).
2022-04-11 14:22:05 +10:00
934944e498
Add AjwernerBtree as an alternative btree backing for piece request order
...
Also add some scans to benchmarks. Make a few changes to reduce allocations using piece request order.
2022-01-22 18:40:33 +11:00
c54dc22592
Benchmark PieceRequestOrder with varying styles of path hint usage
...
Add per-piece piece request order path hints
2022-01-22 18:39:36 +11:00
62060fb897
Use tidwall/btree generics for piece request ordering
2022-01-22 18:39:36 +11:00
1bae62fd22
Dynamic outbound max requests
...
This might be one solution to https://github.com/anacrolix/torrent/issues/698 .
2021-12-24 08:55:57 +11:00
506ff8d037
Use relative availabilities to determine piece request order
...
Most overhead comes from peers that connect that have everything, and we just increment every single piece's availability. There may be some unresolved determinism with torrents that share the same ordering.
2021-12-23 14:00:00 +11:00
1668a18859
Some optimizations in PieceRequestOrder.Update and item comparisons
2021-12-15 19:18:16 +11:00
04beb8937f
More consistent cancellation management
2021-12-12 18:35:02 +11:00
30375615b3
Remove unused request strategy code
2021-12-12 18:35:02 +11:00
1bc84e316f
Don't group Torrent piece request orders without a storage capacity together
2021-12-12 18:35:02 +11:00
8c5df03e14
Minimize allocations due to heap escapes
2021-12-12 18:35:01 +11:00
1b92249f2a
Improve panic message
2021-12-12 18:35:01 +11:00
8025d15155
Stop iterating pieces when storage is exhausted
2021-12-12 18:35:01 +11:00
135f21fb64
Use interfaces to lazily expose the bare minimum inputs to GetRequestablePieces
2021-12-12 18:35:01 +11:00
4088e847f3
Provide mapping from infohash to Torrent in Input
2021-12-12 18:35:01 +11:00
94bb5d40ba
Implement piece request ordering with retained state
2021-12-12 18:35:01 +11:00
b99dd505b5
Avoid reflection sorting request pieces
2021-12-12 18:35:01 +11:00
de2bea502c
Use indices to lookup pieces for comparison
2021-12-12 18:35:01 +11:00
231301f5d7
Cache piece request orderings
2021-12-12 18:35:01 +11:00
67ed5d0032
Partition piece request strategy by storage capacity key
2021-12-12 16:56:01 +11:00
8e707c9b15
Remove unused peer stuff in request strategy
2021-11-29 10:14:16 +11:00
c6ee03f449
gofumpt
2021-11-08 14:47:01 +11:00
7d0be0ac65
Optimize chunk calculations in request strategy
2021-10-19 14:08:56 +11:00
0f53cbf07e
Change peer requesting to spread requests out evenly
2021-10-19 14:08:13 +11:00
eab111dd84
Switch Peer.PieceAllowedFast and several request strategy inputs to raw roaring.Bitmaps
...
This is in preparation to support encoding request strategy run inputs for benchmarking.
2021-10-05 20:06:23 +11:00
1d2d1a9cde
Store peer requests in a bitmap
2021-09-19 15:16:37 +10:00
21358ba458
Do peer requests separately for each peer
2021-09-18 20:34:14 +10:00
cfb23e271f
Export request_strategy.GetRequestablePieces
2021-09-18 18:57:50 +10:00
c6fb0535d4
requesting: Remove some obsoleted condition checks
...
These shouldn't be necessary since peers were pruned for the ability to allocate chunks to the current piece.
2021-09-18 12:53:22 +10:00
dd52a1440f
request_strategy.Peer.canRequestPiece performance: Reorder conditions
2021-09-18 12:53:22 +10:00
b757b62cf4
requesting: Filter peers that can't be allocated chunks
2021-09-16 10:47:27 +10:00
a4697497aa
Performance improvements in piece requesting
2021-09-16 00:12:24 +10:00
2b443d695b
Change torrent capacity to not return a pointer
...
It's an unnecessary complication for a storage implementer.
2021-09-15 23:33:17 +10:00
a8db640c62
Drop bradfitz/iter dependency ( #605 )
...
* Drop bradfitz/iter dependency
`range iter.N` looks nice and doesn't allocate, but unfortunately using a `range` expression blocks a function from being inlined wherever it's used (for now). It's not that we need inlining in all cases, but I do think a C-style for loop looks just as nice and is probably clearer to the majority. There also aren't any clear disadvantages to changing (unless you just happen to dislike the look of C)
* Update misc_test.go
* Update rlreader_test.go
* Update torrent_test.go
* Update bench_test.go
* Update client_test.go
* Update iplist_test.go
* Update mse_test.go
* Update peerconn_test.go
* Update peerconn.go
* Update order_test.go
* Update decoder_test.go
* Update main.go
* Update bench-piece-mark-complete.go
* Update main.go
* Update torrent.go
* Update iplist_test.go
* Update main.go
2021-09-14 13:46:50 +10:00
6c9a7fec39
Optimize peersForPieces allocations for make and sorting
2021-09-10 23:07:10 +10:00
e9193968a3
Disable ensureValidSortedPeersForPieceRequests
...
Should approximately half the contribution of the parent function to overhead.
2021-09-10 23:07:10 +10:00
016bf1b07c
Optimize sortPeersForPiece in allocatePendingChunks
2021-09-10 23:07:10 +10:00
7096da9f03
Minimize allocations and resizing in request computation
2021-08-01 22:02:40 +10:00
f50b8fc2fa
Fix issue with duplicate preallocated requests
...
Likely introduced by allowing actual and next request states to differ.
2021-06-07 13:01:40 +10:00
1a27e14391
Don't always force piece reallocation
...
The balancing for unforced pieces may not be exactly what we want yet.
2021-06-07 13:01:39 +10:00
ad298364aa
Add client-level max unverified bytes
2021-06-07 13:01:39 +10:00
63b0e42731
Get max unverified bytes working
2021-06-07 13:01:39 +10:00
307d6d178f
Prepare to implement max unverified bytes
2021-06-07 13:01:39 +10:00
8e9cb9f2be
Do checks for preallocated requests too
...
Otherwise we reserve requests with the assumption that they can be assigned later, and they actually might not be.
2021-06-07 13:01:39 +10:00
9dd85e5bd2
Fix non-deterministic test
2021-06-07 13:01:39 +10:00
b80e2b08b5
Get request rebalancing working optimally!
2021-06-07 13:01:39 +10:00
Matt Joiner
Jonathan McDowell
YenForYang