diff --git a/design/marketplace.md b/design/marketplace.md new file mode 100644 index 0000000..79d4b62 --- /dev/null +++ b/design/marketplace.md @@ -0,0 +1,219 @@ +A marketplace for storage durability +==================================== + +We present a new design for a storage marketplace, that is both simpler and +includes incentives for repair. + +Context +------- + +Our current storage marketplace is designed around the notion of sending out +requests for storage, waiting for hosts to offer storage, and then choosing a +selection from these hosts to start a storage contract with. It requires +separate contracts for each of these hosts, active participation of the client +during the negotiation phase, and does not yet have any provisions for repairing +storage when hosts fail to deliver on their contracts. + +In this document we describe a new design that is simpler, requires less +interactions, and has repair incentives built in. + +A new design +------------ + +We propose to create new type of storage contract, containing a number of slots. +Each of these slots represents an agreement with a storage host to store a part +of the content. When a client wants store data on the network with durability +guarantees, it posts a storage contract on the blockchain. Hosts that want to +offer storage can fill a slot in the contract. + + + -------- + ---- fill slot --- | Host | + | -------- + | + v + -------------- + ---------- | | -------- + | Client | --- contract ---> | Blockchain | <--- fill slot --- | Host | + ---------- | | -------- + -------------- + ^ + | + | -------- + ---- fill slot --- | Host | + -------- + + +The storage contract contains the content identifier, so that hosts can locate +and download the content. It also contains the reward that hosts receive for +storing the data and the collateral that hosts are expected to deposit. It +contains parameters pertaining to storage proofs and erasure coding. And +finally, it contains the amount of hosts that are expected to store the content, +including a small amount of host losses that can be tolerated. + + + StorageContract + + cid # content identifier + + reward # tokens payed per second per filled slot + collateral # amount of collateral required per host + + proof probability # frequency at which proofs are required + proof parameters # proof of retrievability parameters + erasure coding # erasure coding parameters + + hosts # amount of storage hosts (including loss) + loss # number of allowed host losses + + slots # assigned host slots + + timeout # slots need to be filled before timeout + +Slots +----- + +Initially all host slots are empty. An empty slot can be filled by anyone by +submitting a correct storage proof together with collateral. + + + collateral collateral + proof proof missed proof missed + | | | | | + v v v v v + ------------------------------------------------------------------- + slot: |///////////////////////| |////////////////////| + ------------------------------------------------------------------- + | | + v v + collateral collateral + burned burned + + + + ---------------- time ----------------> + + +The time interval that a slot is filled by a host determines the host payout; +for every second of the interval a certain amount of tokens are awarded to the +host. Hosts that fill a slot are required to submit frequent proofs of storage. + +When a proof is missed, the collateral associated with a slot is mostly burned, +except for a fee for the one who marked the proof as missing. The slot is now +considered empty again until another host submits a correct proof together with +collateral. + +To incentivize repair, we allow the host that takes over a slot to count the +interval between the missed proof and the new proof towards its own profits. +This means that when a proof is missed, the incentive to repair increases over +time. + +The situation where the payout on repair becomes larger than the required +collateral is an interesting one. On the one hand, this could be an opportunity +for zero-cash entry into the network. On the other hand, it might lead to +undesired behavior because the loss of collateral is no longer that important +for a node. This is probably worth looking further into. + +Contract lifecycle +------------------ + +A contract starts when all slots are filled. Regular storage proofs will be +required from the hosts that filled the slots. + +Some contracts may not attract the required amount of hosts, for instance +because the payment is insufficient or the storage demands on the network are +too high. To ensure that such contracts end, we add a timeout to the contract. +If the contract failed to attract sufficient hosts before the timeout is +reached, it is considered cancelled, and the hosts that filled any of the slots +are able to withdraw their collateral. They are also paid for the time interval +before the timeout. The client is able to withdraw the rest of the tokens in the +contract. + +A contract ends when the money that was paid upfront runs out. The end time can +be calculated from the amount of tokens that are paid out per second. Note that +in our scheme this amount does not change during the lifetime of the contract, + even when proofs are missed and repair happens. This is a desirable property +for hosts; they can be sure of a steady source of income, and a predetermined +contract length. When a contract ends, the hosts may withdraw their collateral. + +When too many hosts fail to submit storage proofs, and no other hosts take over +the slots that they vacate, then the content can be considered lost. The +contract is considered failed. The collateral of every host in the contract is +burned as an additional incentive for the network hosts to avoid this scenario. +The client is able to retrieve any funds that are left in the contract. + + | + | create + | + v + ----------- timeout ------------- + | created | ------------------> | cancelled | + ----------- ------------- + | + | all slots filled + | + v + ----------- too many losses ---------- + | started | -------------------> | failed | + ----------- ---------- + | + | money runs out + | + v + --------- + | ended | + --------- + + +Renewal +------- + +When a contract is about to end, and someone in the network wants the contract +to continue for longer, then they can post a new contract with the same content +identifier. + +We've chosen not to allow top-ups of existing contracts with new funds. Even +though this has many advantages (it's a very simple way to extend the lifetime +of the contract, it allows people to easily chip in to host content, etc.) it +has one big disadvantage: hosts no longer know for how long they'll be kept to +the contract. When a contract is continuously topped up, they cannot leave the +contract without losing their collateral. + +Conclusion +---------- + +The design that we presented here deviates significantly from the previous +marketplace design. + +There is no explicit negotiation phase for storage contracts. Clients are no +longer able to choose which hosts will be responsible for keeping the content on +the network. This removes the selection step that was required in the old +design. Instead a host presents the network with an opportunity to earn money by +storing content. Hosts can decide whether or not they want to take part in the +contract, and if they do they are expected to keep to their part of the deal +lest they lose their collateral. + +The first hosts that download the content and provide initial storage proofs are +awarded slots in the contract. This removes the explicit contract start (and its +associated timeout behavior) that was required in the old design. It also adds +an incentive to quickly start storing the content while slots are available in +the contract. + +Instead of receiving a payout at the end of a contract in the old design, now +hosts earn money while the contract is running. This could be used to pay for +running costs on longer contracts. + +While the old design required separate negotiations per host, this design +ensures that either the single contract starts with all hosts, or is cancelled. +This is a significant reduction in the amount of interactions required. + +The old design required new negotiations when a host is not able to fulfill its +obligations, and a separately designed repair protocol. In this design we +managed to include repair incentives and a repair protocol that is nearly +identical to contract start. + +In the old design we had a single collateral per host that could be used to +cover many contracts. Here we decided to include collateral per contract. This +is done to simplify collateral handling, but it is not a requirement of the new +design. The new design can also be made to work with a single collateral per +host.