status-im
/
codex-research
mirror of https://github.com/status-im/codex-research.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

Add queue pausing and prevention of small availabilities from clearing the queue (#188) 

Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
This commit is contained in:
Eric 2024-07-02 22:21:55 +10:00 committed by GitHub
parent 64c91dd792
commit 560a3c40fa
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 29 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
design/sales.md
View File

@ -170,10 +170,11 @@ state is validated.
### Sort order ### Sort order
Slots in the queue should be sorted in the following order: Slots in the queue should be sorted in the following order:
1. Profit (descending)<sup>1</sup> 1. Seen flag (`true` flag should be lower than `false`)
2. Collateral required (ascending) 2. Profit (descending)<sup>1</sup>
3. Time before expiry (descending) 3. Collateral required (ascending)
4. Dataset size (ascending) 4. Time before expiry (descending)
5. Dataset size (ascending)
<sup>1</sup> While profit cannot yet be calculated correctly as this calculation will <sup>1</sup> While profit cannot yet be calculated correctly as this calculation will
involve bandwidth incentives, profit can be estimated as `duration * reward` involve bandwidth incentives, profit can be estimated as `duration * reward`
@ -208,6 +209,30 @@ mechanism](https://github.com/codex-storage/codex-research/blob/master/design/ma
If the host is not allowed to fill the slot, the sales process will exit and the If the host is not allowed to fill the slot, the sales process will exit and the
host will process the top slot in the queue. host will process the top slot in the queue.
#### Preventing continual processing when there are small availabilities
If the processed slot cannot continue because there are no availabilities, the
slot should be marked as `seen` and put back into the queue. This flag will
cause the slot to be ordered lower in the heap queue. If, upon processing
a slot, the slot item already has a `seen` flag set, the queue should be
paused.
This serves to prevent availabilities that are small (in available bytes) from
emptying the queue.
#### Pausing the queue
When availabilities are modified or removed, and there are no availabilities
left, the queue should be paused.
A paused queue will wait until it is unpaused before continuing to process items
in the queue. This prevents unnecessarily popping items off the queue.
#### Unpausing the queue
When availabilities are modified or added, the queue should be unpaused if it
was paused and any slots in the queue should have their `seen` flag cleared.
Additionally, when slots are pushed to the queue, the queue should be unpaused
if it was paused, however the `seen` flags of existing queue items should not be
cleared.
#### Queue workers #### Queue workers
Each time an item in the queue is processed, it is assigned to a workers. The Each time an item in the queue is processed, it is assigned to a workers. The
number of allowed workers can be specified during queue creation. Specifying a number of allowed workers can be specified during queue creation. Specifying a