From d0602aa38887d9820cdde08a85eaa382e7d3bbdc Mon Sep 17 00:00:00 2001 From: Eric Mastro Date: Mon, 5 Jun 2023 17:20:27 +1000 Subject: [PATCH 1/3] add request queue design --- design/sales.md | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/design/sales.md b/design/sales.md index 7cc1f00..a330b37 100644 --- a/design/sales.md +++ b/design/sales.md @@ -140,6 +140,47 @@ the state is kept on local disk by the Repo and the Datastore. How much space is reserved to be sold is persisted on disk by the Repo. The availabilities are persisted on disk by the Datastore. +Request queue +---------- +Once a new request for storage is created on chain, all hosts will receive a +contract event announcing the storage request and decide if they want to act on +the request by matching their availabilities with the incoming request. Because +there will be many requests being announced over time, each host will create a +queue of matching requests, adding each new storage request to the queue. +Requests in the queue will be sorted in the following order: +1. Profit (descending, yet to be determined how this will be calculated) +2. Collateral required (ascending) +3. Time before expiry (descending) +4. Data size (ascending) + +Queue processing will be started only once, when the sales module starts and will +process requests continuously, in order, until the queue is empty. If the +queue is empty, processing of the queue will resume once items have been added +to the queue. If the queue is not empty, but there are no availabilities, queue +processing should be parked until availabilites have been added. + +When a request is processed, it first is checked to ensure there is a matching +availabilty, as these availabilities will have changed over time. Then, a random +slot will be chosen, and the sales process will begin. The start of the sales +process should ensure that the random slot chosen is indeed available (slot +state is "free") before continuing. If it is not available, the sales process +will exit and the host will process the top request in the queue. Note: it may +be that the top request in the queue is the same request that was just +processed, however if the request remains in the queue, it means that there are +still slots to be filled (request hasn't started or expired yet). This +re-processing of the same request allows for the possibility that the random +slot index chosen had already been filled by another host, and gives the host a +chance to fill a different slot index of the same request. + +Hosts will also recieve contract events for when any contract is started, +failed, or cancelled. In all of these cases, requests in the queue should be +removed as they are no longer fillable by the host. + +Request queue implementations should keep in mind that requests will likely need +to be accessed randomly (by key, eg request id) and by index (for sorting), so +implemented structures should handle these types of operations in a little time +as possible. + Repo ---- From c3516a775a66e0396f00dff81c153cd0bec9da2a Mon Sep 17 00:00:00 2001 From: Eric Mastro Date: Tue, 6 Jun 2023 13:28:57 +1000 Subject: [PATCH 2/3] Add repair case, clarification, formatting MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1. Add repair case — when a slot is freed in a contract 2. Clarify re-processing of requests 3. Format the document in sectional manner using Title, Heading 1, Heading 2 4. Organise the Request Queue design into sections --- design/sales.md | 75 +++++++++++++++++++++++++++++++------------------ 1 file changed, 47 insertions(+), 28 deletions(-) diff --git a/design/sales.md b/design/sales.md index a330b37..da9fa82 100644 --- a/design/sales.md +++ b/design/sales.md @@ -1,5 +1,4 @@ -Sales module -============ +# Sales module The sales module is responsible for selling a node's available storage in the [marketplace](./marketplace.md). In order to do so it needs to know how much @@ -39,8 +38,7 @@ price to sell it for. They represent storage that is for sale, but not yet sold. This is information local to the node that can be altered without affecting global state. -Adding availability -------------------- +## Adding availability When a user adds availability, then the reservations module will check whether there is enough space available in the Repo. If there is enough space, then it @@ -59,8 +57,7 @@ node is restarted. | | persist availability | | |------------------------------>| -Selling storage ---------------- +## Selling storage When a request for storage is submitted on chain, the sales module decides whether or not it wants to act on it. First, it tries to find an availability @@ -105,8 +102,7 @@ updated to match the duration of the storage request. |------------------->| update time-to-live | | |----------------------------------->| -Ending a request ----------------- +## Ending a request When a storage request comes to an end, then the content can be removed from the repo and the storage space can be made available for sale again. The same should @@ -129,8 +125,7 @@ for the request can be re-added to the reservations module. |-------------------->| | | | | -Persisting state ----------------- +## Persisting state The sales module keeps state in a number of places. Most state is kept on chain, this includes the slots that a host is filling and the state of each slot. This @@ -140,49 +135,74 @@ the state is kept on local disk by the Repo and the Datastore. How much space is reserved to be sold is persisted on disk by the Repo. The availabilities are persisted on disk by the Datastore. -Request queue ----------- +## Request queue + Once a new request for storage is created on chain, all hosts will receive a contract event announcing the storage request and decide if they want to act on the request by matching their availabilities with the incoming request. Because there will be many requests being announced over time, each host will create a queue of matching requests, adding each new storage request to the queue. -Requests in the queue will be sorted in the following order: + +### Adding requests to the queue + +Requests will be added to the queue when request for storage events are received +from the contracts. Additionally, when slots are freed, a contract event will +also be received, and the request (and slot index) will be added to the queue. +If the request of the freed slot already exists in the queue, the entry will be +updated to include the slot index (multiple slot indices should be allowed). + +### Removing requests from the queue + +Hosts will also recieve contract events for when any contract is started, +failed, or cancelled. In all of these cases, requests in the queue should be +removed as they are no longer fillable by the host. Note: expired requests will +be checked when a request is processed and its state is validated. + +### Sort order + +Requests in the queue should be sorted in the following order: 1. Profit (descending, yet to be determined how this will be calculated) 2. Collateral required (ascending) 3. Time before expiry (descending) -4. Data size (ascending) +4. Dataset size (ascending) -Queue processing will be started only once, when the sales module starts and will -process requests continuously, in order, until the queue is empty. If the +Note: datset size may eventually be included in the profit algorithm and may not +need to be included in the future. Additionally, data dispersal may also impact +the datset size to be downloaded by the host, and consequently the profitability +of servicing a storage request, which will need to be considered in the future +once profitability can be calculated. + +### Queue processing + +Queue processing will be started only once, when the sales module starts and +will process requests continuously, in order, until the queue is empty. If the queue is empty, processing of the queue will resume once items have been added to the queue. If the queue is not empty, but there are no availabilities, queue -processing should be parked until availabilites have been added. +processing will resume once availabilites have been added. When a request is processed, it first is checked to ensure there is a matching availabilty, as these availabilities will have changed over time. Then, a random -slot will be chosen, and the sales process will begin. The start of the sales +slot will be chosen, and the sales process will begin. If the request being +processed contains one or more slot indices (as in the case of repair), the +random slot should be chosen from that sample of indices. The start of the sales process should ensure that the random slot chosen is indeed available (slot state is "free") before continuing. If it is not available, the sales process will exit and the host will process the top request in the queue. Note: it may be that the top request in the queue is the same request that was just -processed, however if the request remains in the queue, it means that there are -still slots to be filled (request hasn't started or expired yet). This +processed, indicating that the request still has slots to be filled, as the +request was had not yet been removed due to being started/failed/cancelled. This re-processing of the same request allows for the possibility that the random slot index chosen had already been filled by another host, and gives the host a chance to fill a different slot index of the same request. -Hosts will also recieve contract events for when any contract is started, -failed, or cancelled. In all of these cases, requests in the queue should be -removed as they are no longer fillable by the host. +### Implementation tips Request queue implementations should keep in mind that requests will likely need to be accessed randomly (by key, eg request id) and by index (for sorting), so -implemented structures should handle these types of operations in a little time +implemented structures should handle these types of operations in as little time as possible. -Repo ----- +## Repo The Repo exposes the following functions that allow the reservations module to query the amount of available storage, to update the amount of reserved @@ -194,8 +214,7 @@ space, and to store data for a guaranteed amount of time. function release(amount) function setTtl(cid, ttl) -Datastore ---------- +## Datastore The Datastore is a generic key-value store that is used to persist the state of the Reservations module, so that it survives node restarts. From 19e3ccb4101a07532884d494567e5fbd50c496e4 Mon Sep 17 00:00:00 2001 From: Eric Mastro Date: Wed, 7 Jun 2023 17:32:41 +1000 Subject: [PATCH 3/3] add consideration for sliding window mechanism --- design/sales.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/design/sales.md b/design/sales.md index da9fa82..1e8413a 100644 --- a/design/sales.md +++ b/design/sales.md @@ -187,13 +187,18 @@ processed contains one or more slot indices (as in the case of repair), the random slot should be chosen from that sample of indices. The start of the sales process should ensure that the random slot chosen is indeed available (slot state is "free") before continuing. If it is not available, the sales process -will exit and the host will process the top request in the queue. Note: it may -be that the top request in the queue is the same request that was just -processed, indicating that the request still has slots to be filled, as the -request was had not yet been removed due to being started/failed/cancelled. This -re-processing of the same request allows for the possibility that the random -slot index chosen had already been filled by another host, and gives the host a -chance to fill a different slot index of the same request. +will exit and the host will process the top request in the queue. The start of +the sales process should also check to ensure the host is allowed to fill the +slot, due to the [sliding window +mechanism](https://github.com/codex-storage/codex-research/blob/master/design/marketplace.md#dispersal). +If the host is not allowed to fill the slot, the sales process will exit and the +host will process the top request in the queue. Note: it may be that the top +request in the queue is the same request that was just processed, indicating +that the request still has slots to be filled, as the request was had not yet +been removed due to being started/failed/cancelled. This re-processing of the +same request allows for the possibility that the random slot index chosen had +already been filled by another host, and gives the host a chance to fill a +different slot index of the same request. ### Implementation tips