nim-dagger/codex/contracts
Eric Mastro ccf349bd14
[marketplace] Add Reservations Module (#340)
* [marketplace] reservations module

- add de/serialization for Availability
- add markUsed/markUnused in persisted availability
- add query for unused
- add reserve/release
- reservation module tests
- split ContractInteractions into client contracts and host contracts
- remove reservations start/stop as the repo start/stop is being managed by the node
- remove dedicated reservations metadata store and use the metadata store from the repo instead
- Split ContractInteractions into:
  - ClientInteractions (with purchasing)
  - HostInteractions (with sales and proving)
- compilation fix for nim 1.2

[repostore] fix started flag, add tests

[marketplace] persist slot index
For loading the sales state from chain, the slot index was not previously persisted in the contract. Will retrieve the slot index from the contract when the sales state is loaded.

* Revert repostore changes

In favour of separate PR https://github.com/status-im/nim-codex/pull/374.

* remove warnings

* clean up

* tests: stop repostore during teardown

* change constructor type identifier

Change Contracts constructor to accept Contracts type instead of ContractInteractions.

* change constructor return type to Result instead of Option

* fix and split interactions tests

* clean up, fix tests

* find availability by slot id

* remove duplication in host/client interactions

* add test for finding availability by slotId

* log instead of raiseAssert when failed to mark availability as unused

* move to SaleErrored state instead of raiseAssert

* remove unneeded reverse

It appears that order is not preserved in the repostore, so reversing does not have the intended effect here.

* update open api spec for potential rest endpoint errors

* move functions about available bytes to repostore

* WIP: reserve and release availabilities as needed

WIP: not tested yet

Availabilities are marked as used when matched (just before downloading starts) so that future matching logic does not match an availability currently in use.

As the download progresses, batches of blocks are written to disk, and the equivalent bytes are released from the reservation module. The size of the availability is reduced as well.

During a reserve or release operation, availability updates occur after the repo is manipulated. If the availability update operation fails, the reserve or release is rolled back to maintain correct accounting of bytes.

Finally, once download completes, or if an error occurs, the availability is marked as unused so future matching can occur.

* delete availability when all bytes released

* fix tests + cleanup

* remove availability from SalesContext callbacks

Availability is no longer used past the SaleDownloading state in the state machine. Cleanup of Availability (marking unused) is handled directly in the SaleDownloading state, and no longer in SaleErrored or SaleFinished. Likewise, Availabilities shouldn’t need to be handled on node restart.

Additionally, Availability was being passed in SalesContext callbacks, and now that Availability is only used temporarily in the SaleDownloading state, Availability is contextually irrelevant to the callbacks, except in OnStore possibly, though it was not being consumed.

* test clean up

* - remove availability from callbacks and constructors from previous commit that needed to be removed (oopsie)
- fix integration test that checks availabilities
  - there was a bug fixed that crashed the node due to a missing `return success` in onStore
  - the test was fixed by ensuring that availabilities are remaining on the node, and the size has been reduced
- change Availability back to non-ref object and constructor back to init
- add trace logging of all state transitions in state machine
- add generally useful trace logging

* fixes after rebase

1. Fix onProve callbacks
2. Use Slot type instead of tuple for retrieving active slot.
3. Bump codex-contracts-eth that exposes getActivceSlot call.

* swap contracts branch to not support slot collateral

Slot collateral changes in the contracts require further changes in the client code, so we’ll skip those changes for now and add in a separate commit.

* modify Interactions and Deployment constructors

- `HostInteractions` and `ClientInteractions` constructors were simplified to take a contract address and no overloads
- `Interactions` prepared simplified so there are no overloads
- `Deployment` constructor updated so that it takes an optional string parameter, instead `Option[string]`

* Move `batchProc` declaration

`batchProc` needs to be consumed by both `node` and `salescontext`, and they can’t reference each other as it creates a circular dependency.

* [reservations] rename `available` to `hasAvailable`

* [reservations] default error message to inner error msg

* add SaleIngored state

When a storage request is handled but the request does match availabilities, the sales agent machine is sent to the SaleIgnored state. In addition, the agent is constructed in a way that if the request is ignored, the sales agent is removed from the list of active agents being tracked in the sales module.
2023-04-04 17:05:16 +10:00
..
interactions [marketplace] Add Reservations Module (#340) 2023-04-04 17:05:16 +10:00
Readme.md fix: approving tokens transfer when creating request (#385) 2023-03-30 11:34:38 +02:00
clock.nim [contracts] Use Duration instead of float for clock offset 2022-10-03 15:17:03 +02:00
config.nim [marketplace] Load sales state from chain (#306) 2023-03-08 14:34:26 +01:00
deployment.nim [marketplace] Add Reservations Module (#340) 2023-04-04 17:05:16 +10:00
interactions.nim [marketplace] Add Reservations Module (#340) 2023-04-04 17:05:16 +10:00
market.nim [marketplace] Add Reservations Module (#340) 2023-04-04 17:05:16 +10:00
marketplace.nim [marketplace] Add Reservations Module (#340) 2023-04-04 17:05:16 +10:00
proofs.nim [marketplace] Load sales state from chain (#306) 2023-03-08 14:34:26 +01:00
requests.nim [marketplace] Add Reservations Module (#340) 2023-04-04 17:05:16 +10:00

Readme.md

Codex Contracts in Nim

Nim API for the Codex smart contracts.

Usage

For a global overview of the steps involved in starting and fulfilling a storage contract, see Codex Contracts.

Smart contract

Connecting to the smart contract on an Ethereum node:

import codex/contracts
import ethers

let address = # fill in address where the contract was deployed
let provider = JsonRpcProvider.new("ws://localhost:8545")
let marketplace = Marketplace.new(address, provider)

Setup client and host so that they can sign transactions; here we use the first two accounts on the Ethereum node:

let accounts = await provider.listAccounts()
let client = provider.getSigner(accounts[0])
let host = provider.getSigner(accounts[1])

Collateral

Hosts need to put up collateral before participating in storage contracts.

A host can learn about the amount of collateral that is required:

let config = await marketplace.config()
let collateral = config.collateral.initialAmount

After preparing the payment, the host can deposit collateral:

await storage
  .connect(host)
  .deposit(collateral)

When a host is not participating in storage offers or contracts, it can withdraw its collateral:

await storage
  .connect(host)
  .withdraw()

Storage requests

Creating a request for storage:

let request : StorageRequest = (
  client:           # address of the client requesting storage
  duration:         # duration of the contract in seconds
  size:             # size in bytes
  contentHash:      # SHA256 hash of the content that's going to be stored
  proofProbability: # require a storage proof roughly once every N periods
  maxPrice:         # maximum price the client is willing to pay
  expiry:           # expiration time of the request (in unix time)
  nonce:            # random nonce to differentiate between similar requests
)

When a client wants to submit this request to the network, it needs to pay the maximum price to the smart contract in advance. The difference between the maximum price and the offered price will be reimbursed later.

Once the payment has been prepared, the client can submit the request to the network:

await storage
  .connect(client)
  .requestStorage(request)

Storage offers

Creating a storage offer:

let offer: StorageOffer = (
  host:       # address of the host that is offering storage
  requestId:  request.id,
  price:      # offered price (in number of tokens)
  expiry:     # expiration time of the offer (in unix time)
)

Hosts submits an offer:

await storage
  .connect(host)
  .offerStorage(offer)

Client selects an offer:

await storage
  .connect(client)
  .selectOffer(offer.id)

Starting and finishing a storage contract

The host whose offer got selected can start the storage contract once it received the data that needs to be stored:

await storage
  .connect(host)
  .startContract(offer.id)

Once the storage contract is finished, the host can release payment:

await storage
  .connect(host)
  .finishContract(id)

Storage proofs

Time is divided into periods, and each period a storage proof may be required from the host. The odds of requiring a storage proof are negotiated through the storage request. For more details about the timing of storage proofs, please refer to the design document.

At the start of each period of time, the host can check whether a storage proof is required:

let isProofRequired = await storage.isProofRequired(offer.id)

If a proof is required, the host can submit it before the end of the period:

await storage
  .connect(host)
  .submitProof(id, proof)

If a proof is not submitted, then a validator can mark a proof as missing:

await storage
  .connect(validator)
  .markProofAsMissing(id, period)