f235c8ea51
Reuse piece order state for request ordering
2022-05-12 10:34:28 +10:00
23a0e7f7d3
Remove redundant type conversions
2022-05-09 12:51:02 +10:00
9a9c7dee00
Add and use typed roaring bitmap
2022-05-09 12:05:50 +10:00
a1a820d3c9
Avoid heap allocation in GetRequestablePieces ( #734 )
...
The calculation of whether we should ignore a piece in
GetRequestablePieces ends up doing an allocation for every piece, when
all we really need to do is query the index in the torrent. Provide an
IgnorePiece function instead, which avoids the need for a temporary
allocation.
Observed to cut out 40% of object allocations in some workloads (large
download, lots of seeds).
2022-04-11 14:22:05 +10:00
934944e498
Add AjwernerBtree as an alternative btree backing for piece request order
...
Also add some scans to benchmarks. Make a few changes to reduce allocations using piece request order.
2022-01-22 18:40:33 +11:00
62060fb897
Use tidwall/btree generics for piece request ordering
2022-01-22 18:39:36 +11:00
506ff8d037
Use relative availabilities to determine piece request order
...
Most overhead comes from peers that connect that have everything, and we just increment every single piece's availability. There may be some unresolved determinism with torrents that share the same ordering.
2021-12-23 14:00:00 +11:00
1668a18859
Some optimizations in PieceRequestOrder.Update and item comparisons
2021-12-15 19:18:16 +11:00
30375615b3
Remove unused request strategy code
2021-12-12 18:35:02 +11:00
8c5df03e14
Minimize allocations due to heap escapes
2021-12-12 18:35:01 +11:00
8025d15155
Stop iterating pieces when storage is exhausted
2021-12-12 18:35:01 +11:00
135f21fb64
Use interfaces to lazily expose the bare minimum inputs to GetRequestablePieces
2021-12-12 18:35:01 +11:00
4088e847f3
Provide mapping from infohash to Torrent in Input
2021-12-12 18:35:01 +11:00
94bb5d40ba
Implement piece request ordering with retained state
2021-12-12 18:35:01 +11:00
b99dd505b5
Avoid reflection sorting request pieces
2021-12-12 18:35:01 +11:00
de2bea502c
Use indices to lookup pieces for comparison
2021-12-12 18:35:01 +11:00
231301f5d7
Cache piece request orderings
2021-12-12 18:35:01 +11:00
67ed5d0032
Partition piece request strategy by storage capacity key
2021-12-12 16:56:01 +11:00
8e707c9b15
Remove unused peer stuff in request strategy
2021-11-29 10:14:16 +11:00
c6ee03f449
gofumpt
2021-11-08 14:47:01 +11:00
7d0be0ac65
Optimize chunk calculations in request strategy
2021-10-19 14:08:56 +11:00
eab111dd84
Switch Peer.PieceAllowedFast and several request strategy inputs to raw roaring.Bitmaps
...
This is in preparation to support encoding request strategy run inputs for benchmarking.
2021-10-05 20:06:23 +11:00
1d2d1a9cde
Store peer requests in a bitmap
2021-09-19 15:16:37 +10:00
21358ba458
Do peer requests separately for each peer
2021-09-18 20:34:14 +10:00
cfb23e271f
Export request_strategy.GetRequestablePieces
2021-09-18 18:57:50 +10:00
c6fb0535d4
requesting: Remove some obsoleted condition checks
...
These shouldn't be necessary since peers were pruned for the ability to allocate chunks to the current piece.
2021-09-18 12:53:22 +10:00
b757b62cf4
requesting: Filter peers that can't be allocated chunks
2021-09-16 10:47:27 +10:00
a4697497aa
Performance improvements in piece requesting
2021-09-16 00:12:24 +10:00
2b443d695b
Change torrent capacity to not return a pointer
...
It's an unnecessary complication for a storage implementer.
2021-09-15 23:33:17 +10:00
6c9a7fec39
Optimize peersForPieces allocations for make and sorting
2021-09-10 23:07:10 +10:00
e9193968a3
Disable ensureValidSortedPeersForPieceRequests
...
Should approximately half the contribution of the parent function to overhead.
2021-09-10 23:07:10 +10:00
016bf1b07c
Optimize sortPeersForPiece in allocatePendingChunks
2021-09-10 23:07:10 +10:00
7096da9f03
Minimize allocations and resizing in request computation
2021-08-01 22:02:40 +10:00
f50b8fc2fa
Fix issue with duplicate preallocated requests
...
Likely introduced by allowing actual and next request states to differ.
2021-06-07 13:01:40 +10:00
1a27e14391
Don't always force piece reallocation
...
The balancing for unforced pieces may not be exactly what we want yet.
2021-06-07 13:01:39 +10:00
ad298364aa
Add client-level max unverified bytes
2021-06-07 13:01:39 +10:00
63b0e42731
Get max unverified bytes working
2021-06-07 13:01:39 +10:00
307d6d178f
Prepare to implement max unverified bytes
2021-06-07 13:01:39 +10:00
8e9cb9f2be
Do checks for preallocated requests too
...
Otherwise we reserve requests with the assumption that they can be assigned later, and they actually might not be.
2021-06-07 13:01:39 +10:00
b80e2b08b5
Get request rebalancing working optimally!
2021-06-07 13:01:39 +10:00
4e9f707aeb
Move piece allocation to its own func
2021-06-07 13:01:39 +10:00
07ba6e9210
New tests and fixes for them
...
Not complete. There's still a request-stealing balancing issue, but it's functional for now.
2021-06-07 13:01:39 +10:00
26837ee73c
Add a working request strategy test
2021-06-07 13:01:39 +10:00
0d4e566fc0
Extract request strategy stuff into a separate module
2021-06-07 13:01:39 +10:00
Matt Joiner
Jonathan McDowell