logos-storage/logos-storage-spec
1
0
Fork 0
mirror of https://github.com/logos-storage/logos-storage-spec.git synced 2026-01-02 13:33:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update marketplace.md

Browse Source
This commit is contained in:
Jimmy Debe 2024-10-01 18:47:33 -04:00 committed by GitHub
parent 8c49a44b63
commit b05a1d4bde
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 29 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

58
specs/marketplace.md
View File

@ -58,15 +58,15 @@ more roles described below.
### Roles
A node can assume one of the three main roles in the network: the client, SP, and validator.
A node can assume one of the three main roles in the network: the client, SP node, and validator.
A client is a potentially short-lived node in the network with the purpose of persisting its data in the Codex persistence layer.
An SP is a long-lived node providing storage for clients in exchange for profit.
An SP node is a long-lived node providing storage for clients in exchange for profit.
To ensure a reliable, robust service for clients,
SPs are required to periodically provide proofs that they are persisting the data.
SP nodes are required to periodically provide proofs that they are persisting the data.
A validator ensures that SPs have submitted valid proofs each period where the smart contract required a proof to be submitted for slots filled by the SP.
A validator ensures that SP nodes have submitted valid proofs each period where the smart contract required a proof to be submitted for slots filled by the SP node.
A validator may be a long-lived or short-lived node.
This role does not get penalized if deciding not to be a long-lived node.
@ -92,7 +92,7 @@ When a user prompts the client node to create a storage request, the client node
To create a request to persist a dataset on the Codex network, client nodes MUST split the dataset into data chunks, $(c_1, c_2, c_3, \ldots, c_{n})$.
The data chunks are then encoded with the erasure coding method and the provided input parameters.
The applied erasure coding method, used on data being persisted all SPs,
The applied erasure coding method, used on data being persisted all SP nodes,
MUST be the [Reed-Solomon algorithm](https://hackmd.io/FB58eZQoTNm-dnhu0Y1XnA).
When dataset is encoded with different algorithm, the dataset will be inaccessible.
@ -139,9 +139,9 @@ The the table below provides the description of the `Request` and the associated
| `content` | `Content` | The dataset that will be hosted with the storage request. |
| `expiry` | `uint256` | Timeout in seconds during which all the slots have to be filled, otherwise request will get cancelled. The final deadline timestamp is calculated at the moment the transaction is mined. |
| `nonce` | `byte32` | Random value to differentiate from other requests of same parameters. It SHOULD be a random byte array. |
| `reward` | `uint256` | Amount of tokens that will be awarded to SPs for finishing the storage request. It MUST be an amount of tokens offered per slot per second. The Ethereum address that submits the `requestStorage()` transaction MUST have [approval](https://docs.openzeppelin.com/contracts/2.x/api/token/erc20#IERC20-approve-address-uint256-) for the transfer of at least an equivalent amount in tokens. |
| `collateral` | `uint256` | The amount of tokens that SP nodes MUST submit when they fill slots. Collateral is then slashed or forfeited if SPs fail to provide the service requested by the storage request (more information in the [Slashing](#slashing) section). |
| `proofProbability` | `uint256` | Determines the average frequency that a proof is required within a period: $\frac{1}{proofProbability}$. SPs are required to provide proofs of storage to the marketplace smart contract when challenged by the smart contract. To prevent hosts from only coming online when proofs are required, the frequency at which proofs are requested from SP nodes is stochastic and is influenced by the `proofProbability` parameter. |
| `reward` | `uint256` | Amount of tokens that will be awarded to SP nodes for finishing the storage request. It MUST be an amount of tokens offered per slot per second. The Ethereum address that submits the `requestStorage()` transaction MUST have [approval](https://docs.openzeppelin.com/contracts/2.x/api/token/erc20#IERC20-approve-address-uint256-) for the transfer of at least an equivalent amount in tokens. |
| `collateral` | `uint256` | The amount of tokens that SP nodes MUST submit when they fill slots. Collateral is then slashed or forfeited if SP nodes fail to provide the service requested by the storage request (more information in the [Slashing](#slashing) section). |
| `proofProbability` | `uint256` | Determines the average frequency that a proof is required within a period: $\frac{1}{proofProbability}$. SP nodes are required to provide proofs of storage to the marketplace smart contract when challenged by the smart contract. To prevent hosts from only coming online when proofs are required, the frequency at which proofs are requested from SP nodes is stochastic and is influenced by the `proofProbability` parameter. |
| `duration` | `uint256` | Total duration of the storage request in seconds. |
| `slots` | `uint64` | The number of requested slots. The slots will all have the same size. |
| `slotSize` | `uint256` | Amount of storage per slot in bytes. |
@ -153,7 +153,7 @@ The the table below provides the description of the `Request` and the associated
It should be noted that the marketplace does not support extending requests. It is REQUIRED that if the user wants to extend the duration of a request, a new request with the same CID must be [created](#creating-storage-requests) **before the original request completes**.
This ensures that the data will continue to persist in the network at the time when the new (or existing) SPs need to retrieve the complete dataset to fill the slots of the new request.
This ensures that the data will continue to persist in the network at the time when the new (or existing) SP nodes need to retrieve the complete dataset to fill the slots of the new request.
### Withdrawing Funds
@ -168,7 +168,7 @@ the client node SHOULD initiate the withdrawal of the remaining funds from the s
## Storage Provider Role
A Codex node acting as an SP persists data across the network by hosting slots requested by clients in their storage requests.
A Codex node acting as an SP node persists data across the network by hosting slots requested by clients in their storage requests.
The following tasks need to be considered when hosting a slot:
@ -230,13 +230,13 @@ When a new request is created, the `StorageRequested(requestId, ask, expiry)` ev
Based on the emitted parameters and node's operator configuration,
SP nodes decide whether it participate in the request by attempting to fill its slot(s).
Note that one SP can fill more than one slot.
Note that one SP node can fill more than one slot.
If the SP node decides to ignore the request, no further action is required.
However, if the SP node decides to fill a slot, it MUST follow the remaining steps described below.
An SP MUST decide which slot, specified by the slot index, it wants to fill. The SP MAY attempt to fill more than one slot. To fill a slot, the SP MUST first download the slot data using the CID of the manifest (**TODO: Manifest RFC**) and the slot index. The CID is specified in `request.content.cid`, which can be retrieved from the smart contract using `getRequest(requestId)`. Then, the node MUST generate a proof over the downloaded data (**TODO: Proving RFC**).
An SP node MUST decide which slot, specified by the slot index, it wants to fill. The SP node MAY attempt to fill more than one slot. To fill a slot, the SP node MUST first download the slot data using the CID of the manifest (**TODO: Manifest RFC**) and the slot index. The CID is specified in `request.content.cid`, which can be retrieved from the smart contract using `getRequest(requestId)`. Then, the node MUST generate a proof over the downloaded data (**TODO: Proving RFC**).
When the proof is ready, the SP MUST make a transaction calling `fillSlot()` on the smart contract with the following REQUIRED parameters:
When the proof is ready, the SP node MUST make a transaction calling `fillSlot()` on the smart contract with the following REQUIRED parameters:
- `requestId` - the ID of the request.
- `slotIndex` - the slot index that the node wants to fill.
@ -245,20 +245,20 @@ When the proof is ready, the SP MUST make a transaction calling `fillSlot()` on
The Ethereum address, of the SP node from which the transaction originates, MUST have [approval](https://docs.openzeppelin.com/contracts/2.x/api/token/erc20#IERC20-approve-address-uint256-) for the transfer of at least the amount of required tokens.
If the proof delivered by the SP is invalid or the slot was already filled by another SP,
then the transaction will revert. Otherwise, a `SlotFilled(requestId, slotIndex)` event is emitted. If the transaction is successful, the SP SHOULD transition into the __proving__ state, where it will need to submit proof of data possession when challenged by the smart contract.
If the proof delivered by the SP node is invalid or the slot was already filled by another SP node,
then the transaction will revert. Otherwise, a `SlotFilled(requestId, slotIndex)` event is emitted. If the transaction is successful, the SP node SHOULD transition into the __proving__ state, where it will need to submit proof of data possession when challenged by the smart contract.
It should be noted that if the SP node observes a `SlotFilled` event for the slot it is currently downloading
the dataset for or generating the proof for,
it means that the slot has been filled by another node in the meantime.
In response, the SP SHOULD stop its current operation and attempt to fill a different, unfilled slot.
In response, the SP node SHOULD stop its current operation and attempt to fill a different, unfilled slot.
### Proving
Once an SP fills a slot, it MUST submit proofs to the smart contract when a challenge is issued by the contract. SPs SHOULD detect that a proof is required for the current period using the `isProofRequired(slotId)` function,
Once an SP node fills a slot, it MUST submit proofs to the smart contract when a challenge is issued by the contract. SP nodes SHOULD detect that a proof is required for the current period using the `isProofRequired(slotId)` function,
or that it will be required using the `willProofBeRequired(slotId)` function in the case that the [pointer is in downtime](https://github.com/codex-storage/codex-research/blob/41c4b4409d2092d0a5475aca0f28995034e58d14/design/storage-proof-timing.md).
Once an SP knows it has to provide a proof,
Once an SP node knows it has to provide a proof,
it MUST get the proof challenge using `getChallenge(slotId)`
which then MUST be incorporated into the proof generation,
as described in Proving RFC (**TODO: Proving RFC**).
@ -267,7 +267,7 @@ When the proof is generated, it MUST be submitted by calling the `submitProof(sl
#### Slashing
There is a slashing scheme orchestrated by the smart contract to incentivize correct behavior and proper proof submissions by SPs. This scheme is configured at the smart contract level and applies uniformly to all participants in the network. The configuration of the slashing scheme can be obtained via the `getConfig()` contract call.
There is a slashing scheme orchestrated by the smart contract to incentivize correct behavior and proper proof submissions by SP nodes. This scheme is configured at the smart contract level and applies uniformly to all participants in the network. The configuration of the slashing scheme can be obtained via the `getConfig()` contract call.
The slashing works as follows:
@ -276,20 +276,20 @@ The slashing works as follows:
(hence the slashing amount is always the same for a given request).
- If the number of slashes exceeds `config.collateral.maxNumberOfSlashes`, the slot is freed, the remaining collateral is burned, and the slot is offered to other nodes for repair. The smart contract also emits the `SlotFreed(requestId, slotIndex)` event.
If, at any time, the number of freed slots exceeds the value specified by the `request.ask.maxSlotLoss` parameter, the dataset is considered lost, and the request is deemed _failed_. The collateral of all SPs that hosted the slots associated with the storage request is burned, and the `RequestFailed(requestId)` event is emitted.
If, at any time, the number of freed slots exceeds the value specified by the `request.ask.maxSlotLoss` parameter, the dataset is considered lost, and the request is deemed _failed_. The collateral of all SP nodes that hosted the slots associated with the storage request is burned, and the `RequestFailed(requestId)` event is emitted.
### Repair
When a slot is freed due to too many missed proofs, which SHOULD be detected by listening to the `SlotFreed(requestId, slotIndex)` event, an SP node can decide whether to participate in repairing the slot. Similar to filling a slot, the node SHOULD consider the operator's configuration when making this decision. The SP that originally hosted the slot but failed to comply with proving requirements MAY also participate in the repair. However, by refilling the slot, the SP **will not** recover its original collateral and must submit new collateral using the `fillSlot()` call.
When a slot is freed due to too many missed proofs, which SHOULD be detected by listening to the `SlotFreed(requestId, slotIndex)` event, an SP node can decide whether to participate in repairing the slot. Similar to filling a slot, the node SHOULD consider the operator's configuration when making this decision. The SP node that originally hosted the slot but failed to comply with proving requirements MAY also participate in the repair. However, by refilling the slot, the SP node **will not** recover its original collateral and must submit new collateral using the `fillSlot()` call.
The repair process is similar to filling slots. If the original slot dataset is no longer present in the network, the SP MAY use erasure coding to reconstruct the dataset. Reconstructing the original slot dataset requires retrieving other pieces of the dataset stored in other slots belonging to the request. For this reason, the node that successfully repairs a slot is entitled to an additional reward. (**TODO: Implementation**)
The repair process is similar to filling slots. If the original slot dataset is no longer present in the network, the SP node MAY use erasure coding to reconstruct the dataset. Reconstructing the original slot dataset requires retrieving other pieces of the dataset stored in other slots belonging to the request. For this reason, the node that successfully repairs a slot is entitled to an additional reward. (**TODO: Implementation**)
The repair process proceeds as follows:
1. The SP observes the `SlotFreed` event and decides to repair the slot.
2. The SP MUST download the chunks of data required to reconstruct the freed slot's data. The node MUST use the [Reed-Solomon algorithm](https://hackmd.io/FB58eZQoTNm-dnhu0Y1XnA) to reconstruct the missing data.
3. The SP MUST generate proof over the reconstructed data.
4. The SP MUST call the `fillSlot()` smart contract function with the same parameters and collateral allowance as described in the [Filling Slots](#filling-slots) section.
1. The SP node observes the `SlotFreed` event and decides to repair the slot.
2. The SP node MUST download the chunks of data required to reconstruct the freed slot's data. The node MUST use the [Reed-Solomon algorithm](https://hackmd.io/FB58eZQoTNm-dnhu0Y1XnA) to reconstruct the missing data.
3. The SP node MUST generate proof over the reconstructed data.
4. The SP node MUST call the `fillSlot()` smart contract function with the same parameters and collateral allowance as described in the [Filling Slots](#filling-slots) section.
### Collecting Funds
@ -311,11 +311,11 @@ For each of the states listed above, different funds are handled as follows:
## Validator Role
In a blockchain, a contract cannot change its state without a transaction and gas initiating the state change. Therefore, our smart contract requires an external trigger to periodically check and confirm that a storage proof has been delivered by the SP. This is where the validator role is essential.
In a blockchain, a contract cannot change its state without a transaction and gas initiating the state change. Therefore, our smart contract requires an external trigger to periodically check and confirm that a storage proof has been delivered by the SP node. This is where the validator role is essential.
The validator role is fulfilled by nodes that help to verify that SPs have submitted the required storage proofs.
The validator role is fulfilled by nodes that help to verify that SP nodes have submitted the required storage proofs.
It is the smart contract that checks if the proof requested from an SP has been delivered. The validator only triggers the decision-making function in the smart contract. To incentivize validators, they receive a reward each time they correctly mark a proof as missing.
It is the smart contract that checks if the proof requested from an SP node has been delivered. The validator only triggers the decision-making function in the smart contract. To incentivize validators, they receive a reward each time they correctly mark a proof as missing.
Each time a validator observes the `SlotFilled` event, it SHOULD add the slot reported in the `SlotFilled` event to the validator's list of watched slots. Then, after the end of each period, a validator has up to `config.proofs.timeout` seconds (a configuration parameter retrievable with `getConfig()`) to validate all the slots. If a slot lacks the required proof, the validator SHOULD call the `markProofAsMissing(slotId, period)` function on the smart contract. This function validates the correctness of the claim, and if right, will send a reward to the validator.