[marketplace] Load sales state from chain (#306)

* [marketplace] get active slots from chain

# Conflicts:
#	codex/contracts/market.nim

* [marketplace] make on chain event callbacks async

# Conflicts:
#	tests/codex/helpers/mockmarket.nim

* [marketplace] make availability optional for node restart

# Conflicts:
#	tests/codex/testsales.nim

* [marketplace] add async state machine

Allows for `enterAsync` to be cancelled.

* [marketplace] move sale process to async state machine

* [marketplace] sales state machine tests

* bump dagger-contracts

* [marketplace] fix ci issue with chronicles output

* PR comments

- add slotIndex to `SalesAgent` constructor
- remove `SalesAgent.init`
- rename `SalesAgent.init` to `start` and `SalesAgent.deinit` to `stop`.
- rename `SalesAgent. populateRequest` to `SalesAgent.retreiveRequest`.
- move availability removal to the downloading state. once availability is persisted to disk, it should survive node restarts.
-

* [marketplace] handle slot filled by other host

Handle the case when in the downloading, proving, or filling states, that another host fills the slot.

* [marketplace] use requestId for mySlots

* [marketplace] infer slot index from slotid

prevents reassigning a random slot index when restoring state from chain

* [marketplace] update to work with latest contracts

* [marketplace] clean up

* [marketplace] align with contract changes

- getState / state > requestState
- getSlot > getRequestFromSlotId
- support MarketplaceConfig
- support slotState, remove unneeded Slot type
- collateral > config.collateral.initialAmount
- remove proofPeriod contract call
- Revert reason “Slot empty” > “Slot is free”
- getProofEnd > read SlotState

Tests for changes

* [marketplace] add missing file

* [marketplace] bump codex-contracts-eth

* [config] remove unused imports

* [sales] cleanup

* [sales] fix: do not crash when fetching state fails

* [sales] make slotIndex non-optional

* Rebase and update NBS commit

Rebase on top of main and update NBS commit to the CI fix.

* [marketplace] use async subscription event handlers

* [marketplace] support slotIndex no longer optional

Previously, SalesAgent.slotIndex had been moved to not optional. However, there were still many places where optionality was assumed. This commit removes those assumuptions.

* [marketplace] sales state machine: use slotState

Use `slotState` instead of `requestState` for sales state machine.

* [marketplace] clean up

* [statemachine] adds a statemachine for async workflows

Allows events to be scheduled synchronously.

See https://github.com/status-im/nim-codex/pull/344

Co-Authored-By: Ben Bierens <thatbenbierens@gmail.com>
Co-Authored-By: Eric Mastro <eric.mastro@gmail.com>

* [market] make market callbacks synchronous

* [statemachine] export Event

* [statemachine] ensure that no errors are raised

* [statemachine] add machine parameter to run method

* [statemachine] initialize queue on start

* [statemachine] check futures before cancelling them

* [sales] use new async state machine

- states use new run() method and event mechanism
- StartState starts subscriptions and loads request

* [statemachine] fix unsusbscribe before subscribe

* [sales] replace old state transition tests

* [sales] separate state machine from sales data

* [sales] remove reference from SalesData to Sales

* [sales] separate sales context from sales

* [sales] move decoupled types into their own modules

* [sales] move retrieveRequest to SalesData

* [sales] move subscription logic into SalesAgent

* [sales] unsubscribe when finished or errored

* [build] revert back to released version of nim-ethers

* [sales] remove SaleStart state

* [sales] add missing base method

* [sales] move asyncSpawn helper to utils

* [sales] fix imports

* [sales] remove unused variables

* [sales statemachine] add async state machine error handling (#349)

* [statemachine] add error handling to asyncstatemachine

- add error handling to catch errors during state.run
- Sales: add ErrorState to identify which state to transition to during an error. This had to be added to SalesAgent constructor due to circular dependency issues, otherwise it would have been added directly to SalesAgent.
- Sales: when an error during run is encountered, the SaleErrorState is constructed with the error, and by default (base impl) will return the error state, so the machine can transition to it. This can be overridden by individual states if needed.

* [sales] rename onSaleFailed to onSaleErrored

Because there is already a state named SaleFailed which is meant to react to an onchain RequestFailed event and also because this callback is called from SaleErrored, renaming to onSaleErrored prevents ambiguity and confusion as to what has happened at the callback callsite.

* [statemachine] forward error to state directly

without going through a machine method first

* [statemachine] remove unnecessary error handling

AsyncQueueFullError is already handled in schedule()

* [statemachine] test that cancellation ignores onError

* [sales] simplify error handling in states

Rely on the state machine error handling instead
of catching errors in the state run method

---------

Co-authored-by: Mark Spanbroek <mark@spanbroek.net>

* [statemachine] prevent memory leaks

prevent memory leaks and nil access defects by:
- allowing multiple subscribe/unsubscribes of salesagent
- disallowing individual salesagent subscription calls to be made externally (requires the .subscribed check)
- allowing mutiple start/stops of asyncstatemachine
- disregard asyncstatemachine schedules if machine not yet started

* [salesagent] add salesagent-specific tests

1. test multiple subscribe/unsubscribes
2. test scheduling machine without being started
3. test subscriptions are working correctly with external events
4. test errors can be overridden at the state level for ErrorHandlingStates.

---------

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
Co-authored-by: Mark Spanbroek <mark@spanbroek.net>
Co-authored-by: Ben Bierens <thatbenbierens@gmail.com>

codex/contracts/Readme.md

@@ -39,7 +39,8 @@ Hosts need to put up collateral before participating in storage contracts.
 
 A host can learn about the amount of collateral that is required:
 ```nim
-let collateral = await marketplace.collateral()
+let config = await marketplace.config()
+let collateral = config.collateral.initialAmount
 ```
 
 The host then needs to prepare a payment to the smart contract by calling the

codex/contracts/config.nim

@@ -0,0 +1,71 @@
+import pkg/contractabi
+import pkg/ethers/fields
+import pkg/questionable/results
+
+export contractabi
+
+type
+  MarketplaceConfig* = object
+    collateral*: CollateralConfig
+    proofs*: ProofConfig
+  CollateralConfig* = object
+    initialAmount*: UInt256 # amount of collateral necessary to fill a slot
+    minimumAmount*: UInt256 # frees slot when collateral drops below this minimum
+    slashCriterion*: UInt256 # amount of proofs missed that lead to slashing
+    slashPercentage*: UInt256 # percentage of the collateral that is slashed
+  ProofConfig* = object
+    period*: UInt256 # proofs requirements are calculated per period (in seconds)
+    timeout*: UInt256 # mark proofs as missing before the timeout (in seconds)
+    downtime*: uint8 # ignore this much recent blocks for proof requirements
+
+
+func fromTuple(_: type ProofConfig, tupl: tuple): ProofConfig =
+  ProofConfig(
+    period: tupl[0],
+    timeout: tupl[1],
+    downtime: tupl[2]
+  )
+
+func fromTuple(_: type CollateralConfig, tupl: tuple): CollateralConfig =
+  CollateralConfig(
+    initialAmount: tupl[0],
+    minimumAmount: tupl[1],
+    slashCriterion: tupl[2],
+    slashPercentage: tupl[3]
+  )
+
+func fromTuple(_: type MarketplaceConfig, tupl: tuple): MarketplaceConfig =
+  MarketplaceConfig(
+    collateral: tupl[0],
+    proofs: tupl[1]
+  )
+
+func solidityType*(_: type ProofConfig): string =
+  solidityType(ProofConfig.fieldTypes)
+
+func solidityType*(_: type CollateralConfig): string =
+  solidityType(CollateralConfig.fieldTypes)
+
+func solidityType*(_: type MarketplaceConfig): string =
+  solidityType(CollateralConfig.fieldTypes)
+
+func encode*(encoder: var AbiEncoder, slot: ProofConfig) =
+  encoder.write(slot.fieldValues)
+
+func encode*(encoder: var AbiEncoder, slot: CollateralConfig) =
+  encoder.write(slot.fieldValues)
+
+func encode*(encoder: var AbiEncoder, slot: MarketplaceConfig) =
+  encoder.write(slot.fieldValues)
+
+func decode*(decoder: var AbiDecoder, T: type ProofConfig): ?!T =
+  let tupl = ?decoder.read(ProofConfig.fieldTypes)
+  success ProofConfig.fromTuple(tupl)
+
+func decode*(decoder: var AbiDecoder, T: type CollateralConfig): ?!T =
+  let tupl = ?decoder.read(CollateralConfig.fieldTypes)
+  success CollateralConfig.fromTuple(tupl)
+
+func decode*(decoder: var AbiDecoder, T: type MarketplaceConfig): ?!T =
+  let tupl = ?decoder.read(MarketplaceConfig.fieldTypes)
+  success MarketplaceConfig.fromTuple(tupl)

codex/contracts/market.nim

@@ -31,6 +31,9 @@ method getSigner*(market: OnChainMarket): Future[Address] {.async.} =
 method myRequests*(market: OnChainMarket): Future[seq[RequestId]] {.async.} =
   return await market.contract.myRequests
 
+method mySlots*(market: OnChainMarket): Future[seq[SlotId]] {.async.} =
+  return await market.contract.mySlots()
+
 method requestStorage(market: OnChainMarket, request: StorageRequest){.async.} =
   await market.contract.requestStorage(request)
 
@@ -43,15 +46,19 @@ method getRequest(market: OnChainMarket,
       return none StorageRequest
     raise e
 
-method getState*(market: OnChainMarket,
+method requestState*(market: OnChainMarket,
                  requestId: RequestId): Future[?RequestState] {.async.} =
   try:
-    return some await market.contract.state(requestId)
+    return some await market.contract.requestState(requestId)
   except ProviderError as e:
     if e.revertReason.contains("Unknown request"):
       return none RequestState
     raise e
 
+method slotState*(market: OnChainMarket,
+                  slotId: SlotId): Future[SlotState] {.async.} =
+  return await market.contract.slotState(slotId)
+
 method getRequestEnd*(market: OnChainMarket,
                       id: RequestId): Future[SecondsSince1970] {.async.} =
   return await market.contract.requestEnd(id)
@@ -66,6 +73,15 @@ method getHost(market: OnChainMarket,
   else:
     return none Address
 
+method getRequestFromSlotId*(market: OnChainMarket,
+                             slotId: SlotId): Future[?StorageRequest] {.async.} =
+  try:
+    return some await market.contract.getRequestFromSlotId(slotId)
+  except ProviderError as e:
+    if e.revertReason.contains("Slot is free"):
+      return none StorageRequest
+    raise e
+
 method fillSlot(market: OnChainMarket,
                 requestId: RequestId,
                 slotIndex: UInt256,
@@ -119,7 +135,7 @@ method subscribeRequestFailed*(market: OnChainMarket,
                               requestId: RequestId,
                               callback: OnRequestFailed):
                             Future[MarketSubscription] {.async.} =
-  proc onEvent(event: RequestFailed) {.upraises:[].} =
+  proc onEvent(event: RequestFailed) {.upraises:[]} =
     if event.requestId == requestId:
       callback(event.requestId)
   let subscription = await market.contract.subscribe(RequestFailed, onEvent)

codex/contracts/marketplace.nim

@@ -4,9 +4,11 @@ import pkg/stint
 import pkg/chronos
 import ../clock
 import ./requests
+import ./config
 
 export stint
 export ethers
+export config
 
 type
   Marketplace* = ref object of Contract
@@ -28,7 +30,7 @@ type
     proof*: seq[byte]
 
 
-proc collateral*(marketplace: Marketplace): UInt256 {.contract, view.}
+proc config*(marketplace: Marketplace): MarketplaceConfig {.contract, view.}
 proc slashMisses*(marketplace: Marketplace): UInt256 {.contract, view.}
 proc slashPercentage*(marketplace: Marketplace): UInt256 {.contract, view.}
 proc minCollateralThreshold*(marketplace: Marketplace): UInt256 {.contract, view.}
@@ -43,12 +45,14 @@ proc withdrawFunds*(marketplace: Marketplace, requestId: RequestId) {.contract.}
 proc freeSlot*(marketplace: Marketplace, id: SlotId) {.contract.}
 proc getRequest*(marketplace: Marketplace, id: RequestId): StorageRequest {.contract, view.}
 proc getHost*(marketplace: Marketplace, id: SlotId): Address {.contract, view.}
+proc getRequestFromSlotId*(marketplace: Marketplace, id: SlotId): StorageRequest {.contract, view.}
 
 proc myRequests*(marketplace: Marketplace): seq[RequestId] {.contract, view.}
-proc state*(marketplace: Marketplace, requestId: RequestId): RequestState {.contract, view.}
+proc mySlots*(marketplace: Marketplace): seq[SlotId] {.contract, view.}
+proc requestState*(marketplace: Marketplace, requestId: RequestId): RequestState {.contract, view.}
+proc slotState*(marketplace: Marketplace, slotId: SlotId): SlotState {.contract, view.}
 proc requestEnd*(marketplace: Marketplace, requestId: RequestId): SecondsSince1970 {.contract, view.}
 
-proc proofPeriod*(marketplace: Marketplace): UInt256 {.contract, view.}
 proc proofTimeout*(marketplace: Marketplace): UInt256 {.contract, view.}
 
 proc proofEnd*(marketplace: Marketplace, id: SlotId): UInt256 {.contract, view.}

codex/contracts/proofs.nim

@@ -21,7 +21,8 @@ proc new*(_: type OnChainProofs, marketplace: Marketplace): OnChainProofs =
   OnChainProofs(marketplace: marketplace, pollInterval: DefaultPollInterval)
 
 method periodicity*(proofs: OnChainProofs): Future[Periodicity] {.async.} =
-  let period = await proofs.marketplace.proofPeriod()
+  let config = await proofs.marketplace.config()
+  let period = config.proofs.period
   return Periodicity(seconds: period)
 
 method isProofRequired*(proofs: OnChainProofs,
@@ -29,7 +30,7 @@ method isProofRequired*(proofs: OnChainProofs,
   try:
     return await proofs.marketplace.isProofRequired(id)
   except ProviderError as e:
-    if e.revertReason.contains("Slot empty"):
+    if e.revertReason.contains("Slot is free"):
       return false
     raise e
 
@@ -38,18 +39,13 @@ method willProofBeRequired*(proofs: OnChainProofs,
   try:
     return await proofs.marketplace.willProofBeRequired(id)
   except ProviderError as e:
-    if e.revertReason.contains("Slot empty"):
+    if e.revertReason.contains("Slot is free"):
       return false
     raise e
 
-method getProofEnd*(proofs: OnChainProofs,
+method slotState*(proofs: OnChainProofs,
-                    id: SlotId): Future[UInt256] {.async.} =
+                  id: SlotId): Future[SlotState] {.async.} =
-  try:
+  return await proofs.marketplace.slotState(id)
-    return await proofs.marketplace.proofEnd(id)
-  except ProviderError as e:
-    if e.revertReason.contains("Slot empty"):
-      return 0.u256
-    raise e
 
 method submitProof*(proofs: OnChainProofs,
                     id: SlotId,

codex/contracts/requests.nim

@@ -39,6 +39,12 @@ type
     Cancelled
     Finished
     Failed
+  SlotState* {.pure.} = enum
+    Free
+    Filled
+    Finished
+    Failed
+    Paid
 
 proc `==`*(x, y: Nonce): bool {.borrow.}
 proc `==`*(x, y: RequestId): bool {.borrow.}

codex/market.nim

@@ -28,15 +28,22 @@ method requestStorage*(market: Market,
 method myRequests*(market: Market): Future[seq[RequestId]] {.base, async.} =
   raiseAssert("not implemented")
 
+method mySlots*(market: Market): Future[seq[SlotId]] {.base, async.} =
+  raiseAssert("not implemented")
+
 method getRequest*(market: Market,
                    id: RequestId):
                   Future[?StorageRequest] {.base, async.} =
   raiseAssert("not implemented")
 
-method getState*(market: Market,
+method requestState*(market: Market,
                  requestId: RequestId): Future[?RequestState] {.base, async.} =
   raiseAssert("not implemented")
 
+method slotState*(market: Market,
+                  slotId: SlotId): Future[SlotState] {.base, async.} =
+  raiseAssert("not implemented")
+
 method getRequestEnd*(market: Market,
                       id: RequestId): Future[SecondsSince1970] {.base, async.} =
   raiseAssert("not implemented")
@@ -46,6 +53,10 @@ method getHost*(market: Market,
                 slotIndex: UInt256): Future[?Address] {.base, async.} =
   raiseAssert("not implemented")
 
+method getRequestFromSlotId*(market: Market,
+                             slotId: SlotId): Future[?StorageRequest] {.base, async.} =
+  raiseAssert("not implemented")
+
 method fillSlot*(market: Market,
                  requestId: RequestId,
                  slotIndex: UInt256,

codex/node.nim

@@ -333,7 +333,7 @@ proc start*(node: CodexNodeRef) {.async.} =
     # TODO: remove Sales callbacks, pass BlockStore and StorageProofs instead
     contracts.sales.onStore = proc(request: StorageRequest,
                                    slot: UInt256,
-                                   availability: Availability) {.async.} =
+                                   availability: ?Availability) {.async.} =
       ## store data in local storage
       ##
 
@@ -354,7 +354,7 @@ proc start*(node: CodexNodeRef) {.async.} =
       if fetchRes.isErr:
         raise newException(CodexError, "Unable to retrieve blocks")
 
-    contracts.sales.onClear = proc(availability: Availability,
+    contracts.sales.onClear = proc(availability: ?Availability,
                                    request: StorageRequest,
                                    slotIndex: UInt256) =
       # TODO: remove data from local storage

codex/proving.nim

@@ -38,7 +38,8 @@ proc removeEndedContracts(proving: Proving) {.async.} =
   let now = proving.clock.now().u256
   var ended: HashSet[SlotId]
   for id in proving.slots:
-    if now >= (await proving.proofs.getProofEnd(id)):
+    let state = await proving.proofs.slotState(id)
+    if state != SlotState.Filled:
       ended.incl(id)
   proving.slots.excl(ended)
 

codex/purchasing/states/started.nim

@@ -21,8 +21,10 @@ method enterAsync*(state: PurchaseStarted) {.async.} =
   try:
     let fut = await one(ended, failed)
     if fut.id == failed.id:
+      ended.cancel()
       state.switch(PurchaseFailed())
     else:
+      failed.cancel()
       state.switch(PurchaseFinished())
     await subscription.unsubscribe()
   except CatchableError as error:

codex/purchasing/states/unknown.nim

@@ -14,7 +14,7 @@ method enterAsync(state: PurchaseUnknown) {.async.} =
 
   try:
     if (request =? await purchase.market.getRequest(purchase.requestId)) and
-       (requestState =? await purchase.market.getState(purchase.requestId)):
+       (requestState =? await purchase.market.requestState(purchase.requestId)):
 
       purchase.request = some request
 

codex/sales.nim

@@ -9,6 +9,12 @@ import ./market
 import ./clock
 import ./proving
 import ./contracts/requests
+import ./sales/salescontext
+import ./sales/salesagent
+import ./sales/availability
+import ./sales/statemachine
+import ./sales/states/downloading
+import ./sales/states/unknown
 
 ## Sales holds a list of available storage that it may sell.
 ##
@@ -29,55 +35,36 @@ import ./contracts/requests
 ##     |                          | ---- storage proof ---> |
 
 export stint
+export availability
 
 type
   Sales* = ref object
-    market: Market
+    context*: SalesContext
-    clock: Clock
+    subscription*: ?market.Subscription
-    subscription: ?market.Subscription
+    available: seq[Availability]
-    available*: seq[Availability]
+    agents*: seq[SalesAgent]
-    onStore: ?OnStore
-    onProve: ?OnProve
-    onClear: ?OnClear
-    onSale: ?OnSale
-    proving: Proving
-  Availability* = object
-    id*: array[32, byte]
-    size*: UInt256
-    duration*: UInt256
-    minPrice*: UInt256
-  SalesAgent = ref object
-    sales: Sales
-    requestId: RequestId
-    ask: StorageAsk
-    availability: Availability
-    request: ?StorageRequest
-    slotIndex: ?UInt256
-    subscription: ?market.Subscription
-    running: ?Future[void]
-    waiting: ?Future[void]
-    finished: bool
-  OnStore = proc(request: StorageRequest,
-                 slot: UInt256,
-                 availability: Availability): Future[void] {.gcsafe, upraises: [].}
-  OnProve = proc(request: StorageRequest,
-                 slot: UInt256): Future[seq[byte]] {.gcsafe, upraises: [].}
-  OnClear = proc(availability: Availability,
-                 request: StorageRequest,
-                 slotIndex: UInt256) {.gcsafe, upraises: [].}
-  OnSale = proc(availability: Availability,
-                request: StorageRequest,
-                slotIndex: UInt256) {.gcsafe, upraises: [].}
 
-func new*(_: type Sales,
+proc `onStore=`*(sales: Sales, onStore: OnStore) =
-          market: Market,
+  sales.context.onStore = some onStore
-          clock: Clock,
+
-          proving: Proving): Sales =
+proc `onProve=`*(sales: Sales, onProve: OnProve) =
-  Sales(
+  sales.context.onProve = some onProve
-    market: market,
+
-    clock: clock,
+proc `onClear=`*(sales: Sales, onClear: OnClear) =
-    proving: proving
+  sales.context.onClear = some onClear
-  )
+
+proc `onSale=`*(sales: Sales, callback: OnSale) =
+  sales.context.onSale = some callback
+
+proc onStore*(sales: Sales): ?OnStore = sales.context.onStore
+
+proc onProve*(sales: Sales): ?OnProve = sales.context.onProve
+
+proc onClear*(sales: Sales): ?OnClear = sales.context.onClear
+
+proc onSale*(sales: Sales): ?OnSale = sales.context.onSale
+
+proc available*(sales: Sales): seq[Availability] = sales.available
 
 proc init*(_: type Availability,
           size: UInt256,
@@ -87,140 +74,95 @@ proc init*(_: type Availability,
   doAssert randomBytes(id) == 32
   Availability(id: id, size: size, duration: duration, minPrice: minPrice)
 
-proc `onStore=`*(sales: Sales, onStore: OnStore) =
-  sales.onStore = some onStore
-
-proc `onProve=`*(sales: Sales, onProve: OnProve) =
-  sales.onProve = some onProve
-
-proc `onClear=`*(sales: Sales, onClear: OnClear) =
-  sales.onClear = some onClear
-
-proc `onSale=`*(sales: Sales, callback: OnSale) =
-  sales.onSale = some callback
-
 func add*(sales: Sales, availability: Availability) =
-  sales.available.add(availability)
+  if not sales.available.contains(availability):
+    sales.available.add(availability)
+  # TODO: add to disk (persist), serialise to json.
 
 func remove*(sales: Sales, availability: Availability) =
   sales.available.keepItIf(it != availability)
+  # TODO: remove from disk availability, mark as in use by assigning
+  # a slotId, so that it can be used for restoration (node restart)
 
-func findAvailability(sales: Sales, ask: StorageAsk): ?Availability =
+func new*(_: type Sales,
+          market: Market,
+          clock: Clock,
+          proving: Proving): Sales =
+
+  let sales = Sales(context: SalesContext(
+    market: market,
+    clock: clock,
+    proving: proving
+  ))
+
+  proc onSaleErrored(availability: Availability) =
+    sales.add(availability)
+
+  sales.context.onSaleErrored = some onSaleErrored
+  sales
+
+func findAvailability*(sales: Sales, ask: StorageAsk): ?Availability =
   for availability in sales.available:
     if ask.slotSize <= availability.size and
        ask.duration <= availability.duration and
        ask.pricePerSlot >= availability.minPrice:
       return some availability
 
-proc finish(agent: SalesAgent, success: bool) =
+proc randomSlotIndex(numSlots: uint64): UInt256 =
-  if agent.finished:
-    return
-
-  agent.finished = true
-
-  if subscription =? agent.subscription:
-    asyncSpawn subscription.unsubscribe()
-
-  if running =? agent.running:
-    running.cancel()
-
-  if waiting =? agent.waiting:
-    waiting.cancel()
-
-  if success:
-    if request =? agent.request and
-       slotIndex =? agent.slotIndex:
-      agent.sales.proving.add(request.slotId(slotIndex))
-
-      if onSale =? agent.sales.onSale:
-        onSale(agent.availability, request, slotIndex)
-  else:
-    if onClear =? agent.sales.onClear and
-       request =? agent.request and
-       slotIndex =? agent.slotIndex:
-      onClear(agent.availability, request, slotIndex)
-    agent.sales.add(agent.availability)
-
-proc selectSlot(agent: SalesAgent)  =
   let rng = Rng.instance
-  let slotIndex = rng.rand(agent.ask.slots - 1)
+  let slotIndex = rng.rand(numSlots - 1)
-  agent.slotIndex = some slotIndex.u256
+  return slotIndex.u256
 
-proc onSlotFilled(agent: SalesAgent,
+proc findSlotIndex(numSlots: uint64,
-                  requestId: RequestId,
+                   requestId: RequestId,
-                  slotIndex: UInt256) {.async.} =
+                   slotId: SlotId): ?UInt256 =
-  try:
+  for i in 0..<numSlots:
-    let market = agent.sales.market
+    if slotId(requestId, i.u256) == slotId:
-    let host = await market.getHost(requestId, slotIndex)
+      return some i.u256
-    let me = await market.getSigner()
-    agent.finish(success = (host == me.some))
-  except CatchableError:
-    agent.finish(success = false)
 
-proc subscribeSlotFilled(agent: SalesAgent, slotIndex: UInt256) {.async.} =
+  return none UInt256
-  proc onSlotFilled(requestId: RequestId,
-                    slotIndex: UInt256) {.gcsafe, upraises:[].} =
-    asyncSpawn agent.onSlotFilled(requestId, slotIndex)
-  let market = agent.sales.market
-  let subscription = await market.subscribeSlotFilled(agent.requestId,
-                                                      slotIndex,
-                                                      onSlotFilled)
-  agent.subscription = some subscription
 
-proc waitForExpiry(agent: SalesAgent) {.async.} =
+proc handleRequest(sales: Sales,
-  without request =? agent.request:
+                   requestId: RequestId,
-    return
+                   ask: StorageAsk) =
-  await agent.sales.clock.waitUntil(request.expiry.truncate(int64))
-  agent.finish(success = false)
-
-proc start(agent: SalesAgent) {.async.} =
-  try:
-    let sales = agent.sales
-    let market = sales.market
-    let availability = agent.availability
-
-    without onStore =? sales.onStore:
-      raiseAssert "onStore callback not set"
-
-    without onProve =? sales.onProve:
-      raiseAssert "onProve callback not set"
-
-    sales.remove(availability)
-
-    agent.selectSlot()
-    without slotIndex =? agent.slotIndex:
-      raiseAssert "no slot selected"
-
-    await agent.subscribeSlotFilled(slotIndex)
-
-    agent.request = await market.getRequest(agent.requestId)
-    without request =? agent.request:
-      agent.finish(success = false)
-      return
-
-    agent.waiting = some agent.waitForExpiry()
-
-    await onStore(request, slotIndex, availability)
-    let proof = await onProve(request, slotIndex)
-    await market.fillSlot(request.id, slotIndex, proof)
-  except CancelledError:
-    raise
-  except CatchableError as e:
-    error "SalesAgent failed", msg = e.msg
-    agent.finish(success = false)
-
-proc handleRequest(sales: Sales, requestId: RequestId, ask: StorageAsk) =
   without availability =? sales.findAvailability(ask):
     return
-
+  sales.remove(availability)
-  let agent = SalesAgent(
+  # TODO: check if random slot is actually available (not already filled)
-    sales: sales,
+  let slotIndex = randomSlotIndex(ask.slots)
-    requestId: requestId,
+  let agent = newSalesAgent(
-    ask: ask,
+    sales.context,
-    availability: availability
+    requestId,
+    slotIndex,
+    some availability,
+    none StorageRequest
   )
+  agent.start(SaleDownloading())
+  sales.agents.add agent
 
-  agent.running = some agent.start()
+proc load*(sales: Sales) {.async.} =
+  let market = sales.context.market
+
+  # TODO: restore availability from disk
+  let slotIds = await market.mySlots()
+
+  for slotId in slotIds:
+    # TODO: this needs to be optimised
+    if request =? await market.getRequestFromSlotId(slotId):
+      let availability = sales.findAvailability(request.ask)
+      without slotIndex =? findSlotIndex(request.ask.slots,
+                                          request.id,
+                                          slotId):
+        raiseAssert "could not find slot index"
+
+      let agent = newSalesAgent(
+        sales.context,
+        request.id,
+        slotIndex,
+        availability,
+        some request)
+      agent.start(SaleUnknown())
+      sales.agents.add agent
 
 proc start*(sales: Sales) {.async.} =
   doAssert sales.subscription.isNone, "Sales already started"
@@ -229,7 +171,7 @@ proc start*(sales: Sales) {.async.} =
     sales.handleRequest(requestId, ask)
 
   try:
-    sales.subscription = some await sales.market.subscribeRequests(onRequest)
+    sales.subscription = some await sales.context.market.subscribeRequests(onRequest)
   except CatchableError as e:
     error "Unable to start sales", msg = e.msg
 
@@ -240,3 +182,6 @@ proc stop*(sales: Sales) {.async.} =
       await subscription.unsubscribe()
     except CatchableError as e:
       warn "Unsubscribe failed", msg = e.msg
+
+  for agent in sales.agents:
+    await agent.stop()

codex/sales/availability.nim

@@ -0,0 +1,8 @@
+import pkg/stint
+
+type
+  Availability* = object
+    id*: array[32, byte]
+    size*: UInt256
+    duration*: UInt256
+    minPrice*: UInt256

codex/sales/salesagent.nim

@@ -0,0 +1,123 @@
+import pkg/chronos
+import pkg/upraises
+import pkg/stint
+import ../contracts/requests
+import ../utils/asyncspawn
+import ./statemachine
+import ./salescontext
+import ./salesdata
+import ./availability
+
+type SalesAgent* = ref object of Machine
+  context*: SalesContext
+  data*: SalesData
+  subscribed: bool
+
+proc newSalesAgent*(context: SalesContext,
+                    requestId: RequestId,
+                    slotIndex: UInt256,
+                    availability: ?Availability,
+                    request: ?StorageRequest): SalesAgent =
+  SalesAgent(
+    context: context,
+    data: SalesData(
+      requestId: requestId,
+      availability: availability,
+      slotIndex: slotIndex,
+      request: request))
+
+proc retrieveRequest*(agent: SalesAgent) {.async.} =
+  let data = agent.data
+  let market = agent.context.market
+  if data.request.isNone:
+    data.request = await market.getRequest(data.requestId)
+
+proc subscribeCancellation(agent: SalesAgent) {.async.} =
+  let data = agent.data
+  let market = agent.context.market
+  let clock = agent.context.clock
+
+  proc onCancelled() {.async.} =
+    without request =? data.request:
+      return
+
+    await clock.waitUntil(request.expiry.truncate(int64))
+    if not data.fulfilled.isNil:
+      asyncSpawn data.fulfilled.unsubscribe(), ignore = CatchableError
+    agent.schedule(cancelledEvent(request))
+
+  data.cancelled = onCancelled()
+
+  proc onFulfilled(_: RequestId) =
+    data.cancelled.cancel()
+
+  data.fulfilled =
+    await market.subscribeFulfillment(data.requestId, onFulfilled)
+
+proc subscribeFailure(agent: SalesAgent) {.async.} =
+  let data = agent.data
+  let market = agent.context.market
+
+  proc onFailed(_: RequestId) =
+    without request =? data.request:
+      return
+    asyncSpawn data.failed.unsubscribe(), ignore = CatchableError
+    agent.schedule(failedEvent(request))
+
+  data.failed =
+    await market.subscribeRequestFailed(data.requestId, onFailed)
+
+proc subscribeSlotFilled(agent: SalesAgent) {.async.} =
+  let data = agent.data
+  let market = agent.context.market
+
+  proc onSlotFilled(requestId: RequestId, slotIndex: UInt256) =
+    asyncSpawn data.slotFilled.unsubscribe(), ignore = CatchableError
+    agent.schedule(slotFilledEvent(requestId, data.slotIndex))
+
+  data.slotFilled =
+    await market.subscribeSlotFilled(data.requestId,
+                                     data.slotIndex,
+                                     onSlotFilled)
+
+proc subscribe*(agent: SalesAgent) {.async.} =
+  if agent.subscribed:
+    return
+
+  await agent.subscribeCancellation()
+  await agent.subscribeFailure()
+  await agent.subscribeSlotFilled()
+  agent.subscribed = true
+
+proc unsubscribe*(agent: SalesAgent) {.async.} =
+  if not agent.subscribed:
+    return
+
+  let data = agent.data
+  try:
+    if not data.fulfilled.isNil:
+      await data.fulfilled.unsubscribe()
+      data.fulfilled = nil
+  except CatchableError:
+    discard
+  try:
+    if not data.failed.isNil:
+      await data.failed.unsubscribe()
+      data.failed = nil
+  except CatchableError:
+    discard
+  try:
+    if not data.slotFilled.isNil:
+      await data.slotFilled.unsubscribe()
+      data.slotFilled = nil
+  except CatchableError:
+    discard
+  if not data.cancelled.isNil:
+    await data.cancelled.cancelAndWait()
+    data.cancelled = nil
+
+  agent.subscribed = false
+
+proc stop*(agent: SalesAgent) {.async.} =
+  procCall Machine(agent).stop()
+  await agent.unsubscribe()

codex/sales/salescontext.nim

@@ -0,0 +1,28 @@
+import pkg/upraises
+import ../market
+import ../clock
+import ../proving
+import ./availability
+
+type
+  SalesContext* = ref object
+    market*: Market
+    clock*: Clock
+    onStore*: ?OnStore
+    onProve*: ?OnProve
+    onClear*: ?OnClear
+    onSale*: ?OnSale
+    onSaleErrored*: ?OnSaleErrored
+    proving*: Proving
+  OnStore* = proc(request: StorageRequest,
+                  slot: UInt256,
+                  availability: ?Availability): Future[void] {.gcsafe, upraises: [].}
+  OnProve* = proc(request: StorageRequest,
+                  slot: UInt256): Future[seq[byte]] {.gcsafe, upraises: [].}
+  OnClear* = proc(availability: ?Availability,# TODO: when availability changes introduced, make availability non-optional (if we need to keep it at all)
+                  request: StorageRequest,
+                  slotIndex: UInt256) {.gcsafe, upraises: [].}
+  OnSale* = proc(availability: ?Availability, # TODO: when availability changes introduced, make availability non-optional (if we need to keep it at all)
+                 request: StorageRequest,
+                 slotIndex: UInt256) {.gcsafe, upraises: [].}
+  OnSaleErrored* = proc(availability: Availability) {.gcsafe, upraises: [].}

codex/sales/salesdata.nim

@@ -0,0 +1,16 @@
+import pkg/chronos
+import ../contracts/requests
+import ../market
+import ./availability
+
+type
+  SalesData* = ref object
+    requestId*: RequestId
+    ask*: StorageAsk
+    availability*: ?Availability # TODO: when availability persistence is added, change this to not optional
+    request*: ?StorageRequest
+    slotIndex*: UInt256
+    failed*: market.Subscription
+    fulfilled*: market.Subscription
+    slotFilled*: market.Subscription
+    cancelled*: Future[void]

codex/sales/statemachine.nim

@@ -0,0 +1,42 @@
+import pkg/questionable
+import pkg/upraises
+import ../errors
+import ../utils/asyncstatemachine
+import ../market
+import ../clock
+import ../proving
+import ../contracts/requests
+
+export market
+export clock
+export asyncstatemachine
+export proving
+
+type
+  SaleState* = ref object of State
+  SaleError* = ref object of CodexError
+
+method `$`*(state: SaleState): string {.base.} =
+  raiseAssert "not implemented"
+
+method onCancelled*(state: SaleState, request: StorageRequest): ?State {.base, upraises:[].} =
+  discard
+
+method onFailed*(state: SaleState, request: StorageRequest): ?State {.base, upraises:[].} =
+  discard
+
+method onSlotFilled*(state: SaleState, requestId: RequestId,
+                     slotIndex: UInt256): ?State {.base, upraises:[].} =
+  discard
+
+proc cancelledEvent*(request: StorageRequest): Event =
+  return proc (state: State): ?State =
+    SaleState(state).onCancelled(request)
+
+proc failedEvent*(request: StorageRequest): Event =
+  return proc (state: State): ?State =
+    SaleState(state).onFailed(request)
+
+proc slotFilledEvent*(requestId: RequestId, slotIndex: UInt256): Event =
+  return proc (state: State): ?State =
+    SaleState(state).onSlotFilled(requestId, slotIndex)

codex/sales/states/cancelled.nim

@@ -0,0 +1,14 @@
+import ../statemachine
+import ./errorhandling
+import ./errored
+
+type
+  SaleCancelled* = ref object of ErrorHandlingState
+  SaleCancelledError* = object of CatchableError
+  SaleTimeoutError* = object of SaleCancelledError
+
+method `$`*(state: SaleCancelled): string = "SaleCancelled"
+
+method run*(state: SaleCancelled, machine: Machine): Future[?State] {.async.} =
+  let error = newException(SaleTimeoutError, "Sale cancelled due to timeout")
+  return some State(SaleErrored(error: error))

codex/sales/states/downloading.nim

@@ -0,0 +1,42 @@
+import ../../market
+import ../salesagent
+import ../statemachine
+import ./errorhandling
+import ./cancelled
+import ./failed
+import ./filled
+import ./proving
+
+type
+  SaleDownloading* = ref object of ErrorHandlingState
+    failedSubscription: ?market.Subscription
+    hasCancelled: ?Future[void]
+
+method `$`*(state: SaleDownloading): string = "SaleDownloading"
+
+method onCancelled*(state: SaleDownloading, request: StorageRequest): ?State =
+  return some State(SaleCancelled())
+
+method onFailed*(state: SaleDownloading, request: StorageRequest): ?State =
+  return some State(SaleFailed())
+
+method onSlotFilled*(state: SaleDownloading, requestId: RequestId,
+                     slotIndex: UInt256): ?State =
+  return some State(SaleFilled())
+
+method run*(state: SaleDownloading, machine: Machine): Future[?State] {.async.} =
+  let agent = SalesAgent(machine)
+  let data = agent.data
+  let context = agent.context
+
+  await agent.retrieveRequest()
+  await agent.subscribe()
+
+  without onStore =? context.onStore:
+    raiseAssert "onStore callback not set"
+
+  without request =? data.request:
+    raiseAssert "no sale request"
+
+  await onStore(request, data.slotIndex, data.availability)
+  return some State(SaleProving())

codex/sales/states/errored.nim

@@ -0,0 +1,34 @@
+import pkg/upraises
+import pkg/chronicles
+import ../statemachine
+import ../salesagent
+
+type SaleErrored* = ref object of SaleState
+  error*: ref CatchableError
+
+method `$`*(state: SaleErrored): string = "SaleErrored"
+
+method onError*(state: SaleState, err: ref CatchableError): ?State {.upraises:[].} =
+  error "error during SaleErrored run", error = err.msg
+
+method run*(state: SaleErrored, machine: Machine): Future[?State] {.async.} =
+  let agent = SalesAgent(machine)
+  let data = agent.data
+  let context = agent.context
+
+  if onClear =? context.onClear and
+      request =? data.request and
+      slotIndex =? data.slotIndex:
+    onClear(data.availability, request, slotIndex)
+
+  # TODO: when availability persistence is added, change this to not optional
+  # NOTE: with this in place, restoring state for a restarted node will
+  # never free up availability once finished. Persisting availability
+  # on disk is required for this.
+  if onSaleErrored =? context.onSaleErrored and
+     availability =? data.availability:
+    onSaleErrored(availability)
+
+    await agent.unsubscribe()
+
+  error "Sale error", error=state.error.msg

codex/sales/states/errorhandling.nim

@@ -0,0 +1,9 @@
+import pkg/questionable
+import ../statemachine
+import ./errored
+
+type
+  ErrorHandlingState* = ref object of SaleState
+
+method onError*(state: ErrorHandlingState, error: ref CatchableError): ?State =
+  some State(SaleErrored(error: error))

codex/sales/states/failed.nim

@@ -0,0 +1,13 @@
+import ../statemachine
+import ./errorhandling
+import ./errored
+
+type
+  SaleFailed* = ref object of ErrorHandlingState
+  SaleFailedError* = object of SaleError
+
+method `$`*(state: SaleFailed): string = "SaleFailed"
+
+method run*(state: SaleFailed, machine: Machine): Future[?State] {.async.} =
+  let error = newException(SaleFailedError, "Sale failed")
+  return some State(SaleErrored(error: error))

codex/sales/states/filled.nim

@@ -0,0 +1,32 @@
+import pkg/questionable
+import ../statemachine
+import ../salesagent
+import ./errorhandling
+import ./errored
+import ./finished
+import ./cancelled
+import ./failed
+
+type
+  SaleFilled* = ref object of ErrorHandlingState
+  HostMismatchError* = object of CatchableError
+
+method onCancelled*(state: SaleFilled, request: StorageRequest): ?State =
+  return some State(SaleCancelled())
+
+method onFailed*(state: SaleFilled, request: StorageRequest): ?State =
+  return some State(SaleFailed())
+
+method `$`*(state: SaleFilled): string = "SaleFilled"
+
+method run*(state: SaleFilled, machine: Machine): Future[?State] {.async.} =
+  let data = SalesAgent(machine).data
+  let market = SalesAgent(machine).context.market
+
+  let host = await market.getHost(data.requestId, data.slotIndex)
+  let me = await market.getSigner()
+  if host == me.some:
+    return some State(SaleFinished())
+  else:
+    let error = newException(HostMismatchError, "Slot filled by other host")
+    return some State(SaleErrored(error: error))

codex/sales/states/filling.nim

@@ -0,0 +1,30 @@
+import ../../market
+import ../statemachine
+import ../salesagent
+import ./errorhandling
+import ./filled
+import ./errored
+import ./cancelled
+import ./failed
+
+type
+  SaleFilling* = ref object of ErrorHandlingState
+    proof*: seq[byte]
+
+method `$`*(state: SaleFilling): string = "SaleFilling"
+
+method onCancelled*(state: SaleFilling, request: StorageRequest): ?State =
+  return some State(SaleCancelled())
+
+method onFailed*(state: SaleFilling, request: StorageRequest): ?State =
+  return some State(SaleFailed())
+
+method onSlotFilled*(state: SaleFilling, requestId: RequestId,
+                     slotIndex: UInt256): ?State =
+  return some State(SaleFilled())
+
+method run(state: SaleFilling, machine: Machine): Future[?State] {.async.} =
+  let data = SalesAgent(machine).data
+  let market = SalesAgent(machine).context.market
+
+  await market.fillSlot(data.requestId, data.slotIndex, state.proof)

codex/sales/states/finished.nim

@@ -0,0 +1,37 @@
+import pkg/chronos
+import ../statemachine
+import ../salesagent
+import ./errorhandling
+import ./cancelled
+import ./failed
+
+type
+  SaleFinished* = ref object of ErrorHandlingState
+
+method `$`*(state: SaleFinished): string = "SaleFinished"
+
+method onCancelled*(state: SaleFinished, request: StorageRequest): ?State =
+  return some State(SaleCancelled())
+
+method onFailed*(state: SaleFinished, request: StorageRequest): ?State =
+  return some State(SaleFailed())
+
+method run*(state: SaleFinished, machine: Machine): Future[?State] {.async.} =
+  let agent = SalesAgent(machine)
+  let data = agent.data
+  let context = agent.context
+
+  if request =? data.request and
+      slotIndex =? data.slotIndex:
+    context.proving.add(request.slotId(slotIndex))
+
+    if onSale =? context.onSale:
+      onSale(data.availability, request, slotIndex)
+
+  # TODO: Keep track of contract completion using local clock. When contract
+  # has finished, we need to add back availability to the sales module.
+  # This will change when the state machine is updated to include the entire
+  # sales process, as well as when availability is persisted, so leaving it
+  # as a TODO for now.
+
+  await agent.unsubscribe()

codex/sales/states/proving.nim

@@ -0,0 +1,35 @@
+import ../statemachine
+import ../salesagent
+import ./errorhandling
+import ./filling
+import ./cancelled
+import ./failed
+import ./filled
+
+type
+  SaleProving* = ref object of ErrorHandlingState
+
+method `$`*(state: SaleProving): string = "SaleProving"
+
+method onCancelled*(state: SaleProving, request: StorageRequest): ?State =
+  return some State(SaleCancelled())
+
+method onFailed*(state: SaleProving, request: StorageRequest): ?State =
+  return some State(SaleFailed())
+
+method onSlotFilled*(state: SaleProving, requestId: RequestId,
+                     slotIndex: UInt256): ?State =
+  return some State(SaleFilled())
+
+method run*(state: SaleProving, machine: Machine): Future[?State] {.async.} =
+  let data = SalesAgent(machine).data
+  let context = SalesAgent(machine).context
+
+  without request =? data.request:
+    raiseAssert "no sale request"
+
+  without onProve =? context.onProve:
+    raiseAssert "onProve callback not set"
+
+  let proof = await onProve(request, data.slotIndex)
+  return some State(SaleFilling(proof: proof))

codex/sales/states/unknown.nim

@@ -0,0 +1,46 @@
+import ../statemachine
+import ../salesagent
+import ./filled
+import ./finished
+import ./failed
+import ./errored
+import ./cancelled
+
+type
+  SaleUnknown* = ref object of SaleState
+  SaleUnknownError* = object of CatchableError
+  UnexpectedSlotError* = object of SaleUnknownError
+
+method `$`*(state: SaleUnknown): string = "SaleUnknown"
+
+method onCancelled*(state: SaleUnknown, request: StorageRequest): ?State =
+  return some State(SaleCancelled())
+
+method onFailed*(state: SaleUnknown, request: StorageRequest): ?State =
+  return some State(SaleFailed())
+
+method run*(state: SaleUnknown, machine: Machine): Future[?State] {.async.} =
+  let agent = SalesAgent(machine)
+  let data = agent.data
+  let market = agent.context.market
+
+  await agent.retrieveRequest()
+  await agent.subscribe()
+
+  let slotId = slotId(data.requestId, data.slotIndex)
+
+  without slotState =? await market.slotState(slotId):
+    let error = newException(SaleUnknownError, "cannot retrieve slot state")
+    return some State(SaleErrored(error: error))
+
+  case slotState
+  of SlotState.Free:
+    let error = newException(UnexpectedSlotError,
+      "slot state on chain should not be 'free'")
+    return some State(SaleErrored(error: error))
+  of SlotState.Filled:
+    return some State(SaleFilled())
+  of SlotState.Finished, SlotState.Paid:
+    return some State(SaleFinished())
+  of SlotState.Failed:
+    return some State(SaleFailed())

codex/storageproofs/timing/proofs.nim

@@ -1,5 +1,6 @@
 import pkg/chronos
 import pkg/stint
+import pkg/questionable
 import pkg/upraises
 import ./periods
 import ../../contracts/requests
@@ -26,8 +27,8 @@ method willProofBeRequired*(proofs: Proofs,
                             id: SlotId): Future[bool] {.base, async.} =
   raiseAssert("not implemented")
 
-method getProofEnd*(proofs: Proofs,
+method slotState*(proofs: Proofs,
-                    id: SlotId): Future[UInt256] {.base, async.} =
+                  id: SlotId): Future[SlotState] {.base, async.} =
   raiseAssert("not implemented")
 
 method submitProof*(proofs: Proofs,

codex/utils/asyncspawn.nim

@@ -0,0 +1,10 @@
+import pkg/chronos
+
+proc asyncSpawn*(future: Future[void], ignore: type CatchableError) =
+  proc ignoringError {.async.} =
+    try:
+      await future
+    except ignore:
+      discard
+  asyncSpawn ignoringError()
+

codex/utils/asyncstatemachine.nim

@@ -0,0 +1,86 @@
+import pkg/questionable
+import pkg/chronos
+import pkg/chronicles
+import pkg/upraises
+
+push: {.upraises:[].}
+
+type
+  Machine* = ref object of RootObj
+    state: State
+    running: Future[void]
+    scheduled: AsyncQueue[Event]
+    scheduling: Future[void]
+    started: bool
+  State* = ref object of RootObj
+  Event* = proc(state: State): ?State {.gcsafe, upraises:[].}
+
+proc transition(_: type Event, previous, next: State): Event =
+  return proc (state: State): ?State =
+    if state == previous:
+      return some next
+
+proc schedule*(machine: Machine, event: Event) =
+  if not machine.started:
+    return
+
+  try:
+    machine.scheduled.putNoWait(event)
+  except AsyncQueueFullError:
+    raiseAssert "unlimited queue is full?!"
+
+method run*(state: State, machine: Machine): Future[?State] {.base, async.} =
+  discard
+
+method onError*(state: State, error: ref CatchableError): ?State {.base.} =
+  raise (ref Defect)(msg: "error in state machine: " & error.msg, parent: error)
+
+proc onError(machine: Machine, error: ref CatchableError): Event =
+  return proc (state: State): ?State =
+    state.onError(error)
+
+proc run(machine: Machine, state: State) {.async.} =
+  try:
+    if next =? await state.run(machine):
+      machine.schedule(Event.transition(state, next))
+  except CancelledError:
+    discard
+
+proc scheduler(machine: Machine) {.async.} =
+  proc onRunComplete(udata: pointer) {.gcsafe.} =
+    var fut = cast[FutureBase](udata)
+    if fut.failed():
+      machine.schedule(machine.onError(fut.error))
+
+  try:
+    while true:
+      let event = await machine.scheduled.get()
+      if next =? event(machine.state):
+        if not machine.running.isNil:
+          await machine.running.cancelAndWait()
+        machine.state = next
+        machine.running = machine.run(machine.state)
+        machine.running.addCallback(onRunComplete)
+  except CancelledError:
+    discard
+
+proc start*(machine: Machine, initialState: State) =
+  if machine.started:
+    return
+
+  if machine.scheduled.isNil:
+    machine.scheduled = newAsyncQueue[Event]()
+  machine.scheduling = machine.scheduler()
+  machine.started = true
+  machine.schedule(Event.transition(machine.state, initialState))
+
+proc stop*(machine: Machine) =
+  if not machine.started:
+    return
+
+  if not machine.scheduling.isNil:
+    machine.scheduling.cancel()
+  if not machine.running.isNil:
+    machine.running.cancel()
+
+  machine.started = false

codex/utils/statemachine.nim

@@ -1,3 +1,5 @@
+import std/typetraits
+import pkg/chronicles
 import pkg/questionable
 import pkg/chronos
 import ./optionalcast
@@ -62,6 +64,9 @@ type
   State* = ref object of RootObj
     context: ?StateMachine
 
+method `$`*(state: State): string {.base.} =
+  (typeof state).name
+
 method enter(state: State) {.base.} =
   discard
 
@@ -88,6 +93,8 @@ proc switch*(oldState, newState: State) =
 
 type
   AsyncState* = ref object of State
+    activeTransition: ?Future[void]
+  StateMachineAsync* =  ref object of StateMachine
 
 method enterAsync(state: AsyncState) {.base, async.} =
   discard
@@ -100,3 +107,24 @@ method enter(state: AsyncState) =
 
 method exit(state: AsyncState) =
   asyncSpawn state.exitAsync()
+
+proc switchAsync*(machine: StateMachineAsync, newState: AsyncState) {.async.} =
+  if state =? (machine.state as AsyncState):
+    trace "Switching sales state", `from` = $state, to = $newState
+    if activeTransition =? state.activeTransition and
+        not activeTransition.completed:
+      await activeTransition.cancelAndWait()
+    # should wait for exit before switch. could add a transition option during
+    # switch if we don't need to wait
+    await state.exitAsync()
+    state.context = none StateMachine
+  else:
+    trace "Switching state", `from` = "no state", to = $newState
+
+  machine.state = some State(newState)
+  newState.context = some StateMachine(machine)
+  newState.activeTransition = some newState.enterAsync()
+
+proc switchAsync*(oldState, newState: AsyncState) {.async.} =
+  if context =? oldState.context:
+    await StateMachineAsync(context).switchAsync(newState)

tests/codex/helpers/mockmarket.nim

@@ -2,6 +2,8 @@ import std/sequtils
 import std/tables
 import std/hashes
 import pkg/codex/market
+import pkg/codex/contracts/requests
+import pkg/codex/contracts/config
 
 export market
 export tables
@@ -9,23 +11,26 @@ export tables
 type
   MockMarket* = ref object of Market
     activeRequests*: Table[Address, seq[RequestId]]
+    activeSlots*: Table[Address, seq[SlotId]]
     requested*: seq[StorageRequest]
     requestEnds*: Table[RequestId, SecondsSince1970]
-    state*: Table[RequestId, RequestState]
+    requestState*: Table[RequestId, RequestState]
+    slotState*: Table[SlotId, SlotState]
     fulfilled*: seq[Fulfillment]
-    filled*: seq[Slot]
+    filled*: seq[MockSlot]
     withdrawn*: seq[RequestId]
     signer: Address
     subscriptions: Subscriptions
+    config: MarketplaceConfig
   Fulfillment* = object
     requestId*: RequestId
     proof*: seq[byte]
     host*: Address
-  Slot* = object
+  MockSlot* = object
     requestId*: RequestId
+    host*: Address
     slotIndex*: UInt256
     proof*: seq[byte]
-    host*: Address
   Subscriptions = object
     onRequest: seq[RequestSubscription]
     onFulfillment: seq[FulfillmentSubscription]
@@ -60,7 +65,20 @@ proc hash*(requestId: RequestId): Hash =
   hash(requestId.toArray)
 
 proc new*(_: type MockMarket): MockMarket =
-  MockMarket(signer: Address.example)
+  let config = MarketplaceConfig(
+    collateral: CollateralConfig(
+      initialAmount: 100.u256,
+      minimumAmount: 40.u256,
+      slashCriterion: 3.u256,
+      slashPercentage: 10.u256
+    ),
+    proofs: ProofConfig(
+      period: 10.u256,
+      timeout: 5.u256,
+      downtime: 64.uint8
+    )
+  )
+  MockMarket(signer: Address.example, config: config)
 
 method getSigner*(market: MockMarket): Future[Address] {.async.} =
   return market.signer
@@ -74,6 +92,9 @@ method requestStorage*(market: MockMarket, request: StorageRequest) {.async.} =
 method myRequests*(market: MockMarket): Future[seq[RequestId]] {.async.} =
   return market.activeRequests[market.signer]
 
+method mySlots*(market: MockMarket): Future[seq[SlotId]] {.async.} =
+  return market.activeSlots[market.signer]
+
 method getRequest(market: MockMarket,
                   id: RequestId): Future[?StorageRequest] {.async.} =
   for request in market.requested:
@@ -81,15 +102,30 @@ method getRequest(market: MockMarket,
       return some request
   return none StorageRequest
 
-method getState*(market: MockMarket,
+method getRequestFromSlotId*(market: MockMarket,
+                             slotId: SlotId): Future[?StorageRequest] {.async.} =
+  for slot in market.filled:
+    if slotId(slot.requestId, slot.slotIndex) == slotId:
+      return await market.getRequest(slot.requestId)
+  return none StorageRequest
+
+method requestState*(market: MockMarket,
                  requestId: RequestId): Future[?RequestState] {.async.} =
-  return market.state.?[requestId]
+  return market.requestState.?[requestId]
+
+method slotState*(market: MockMarket,
+                  slotId: SlotId): Future[SlotState] {.async.} =
+  if market.slotState.hasKey(slotId):
+    return market.slotState[slotId]
+  else:
+    return SlotState.Free
+
 
 method getRequestEnd*(market: MockMarket,
                       id: RequestId): Future[SecondsSince1970] {.async.} =
   return market.requestEnds[id]
 
-method getHost(market: MockMarket,
+method getHost*(market: MockMarket,
                requestId: RequestId,
                slotIndex: UInt256): Future[?Address] {.async.} =
   for slot in market.filled:
@@ -130,7 +166,7 @@ proc fillSlot*(market: MockMarket,
                slotIndex: UInt256,
                proof: seq[byte],
                host: Address) =
-  let slot = Slot(
+  let slot = MockSlot(
     requestId: requestId,
     slotIndex: slotIndex,
     proof: proof,

tests/codex/helpers/mockproofs.nim

@@ -11,6 +11,7 @@ type
     proofsToBeRequired: HashSet[SlotId]
     proofEnds: Table[SlotId, UInt256]
     subscriptions: seq[MockSubscription]
+    slotStates: Table[SlotId, SlotState]
   MockSubscription* = ref object of Subscription
     proofs: MockProofs
     callback: OnProofSubmitted
@@ -49,13 +50,6 @@ method willProofBeRequired*(mock: MockProofs,
 proc setProofEnd*(mock: MockProofs, id: SlotId, proofEnd: UInt256) =
   mock.proofEnds[id] = proofEnd
 
-method getProofEnd*(mock: MockProofs,
-                    id: SlotId): Future[UInt256] {.async.} =
-  if mock.proofEnds.hasKey(id):
-    return mock.proofEnds[id]
-  else:
-    return UInt256.high
-
 method submitProof*(mock: MockProofs,
                     id: SlotId,
                     proof: seq[byte]) {.async.} =
@@ -71,3 +65,10 @@ method subscribeProofSubmission*(mock: MockProofs,
 
 method unsubscribe*(subscription: MockSubscription) {.async, upraises:[].} =
   subscription.proofs.subscriptions.keepItIf(it != subscription)
+
+method slotState*(mock: MockProofs,
+                  slotId: SlotId): Future[SlotState] {.async.} =
+  return mock.slotStates[slotId]
+
+proc setSlotState*(mock: MockProofs, slotId: SlotId, state: SlotState) =
+  mock.slotStates[slotId] = state

tests/codex/sales/states/testdownloading.nim

@@ -0,0 +1,29 @@
+import std/unittest
+import pkg/questionable
+import pkg/codex/contracts/requests
+import pkg/codex/sales/states/downloading
+import pkg/codex/sales/states/cancelled
+import pkg/codex/sales/states/failed
+import pkg/codex/sales/states/filled
+import ../../examples
+
+suite "sales state 'downloading'":
+
+  let request = StorageRequest.example
+  let slotIndex = (request.ask.slots div 2).u256
+  var state: SaleDownloading
+
+  setup:
+    state = SaleDownloading.new()
+
+  test "switches to cancelled state when request expires":
+    let next = state.onCancelled(request)
+    check !next of SaleCancelled
+
+  test "switches to failed state when request fails":
+    let next = state.onFailed(request)
+    check !next of SaleFailed
+
+  test "switches to filled state when slot is filled":
+    let next = state.onSlotFilled(request.id, slotIndex)
+    check !next of SaleFilled

tests/codex/sales/states/testfilled.nim

@@ -0,0 +1,46 @@
+import pkg/asynctest
+import pkg/codex/contracts/requests
+import pkg/codex/sales
+import pkg/codex/sales/salesagent
+import pkg/codex/sales/salescontext
+import pkg/codex/sales/states/filled
+import pkg/codex/sales/states/errored
+import pkg/codex/sales/states/finished
+import ../../helpers/mockmarket
+import ../../examples
+
+suite "sales state 'filled'":
+
+  let request = StorageRequest.example
+  let slotIndex = (request.ask.slots div 2).u256
+
+  var market: MockMarket
+  var slot: MockSlot
+  var agent: SalesAgent
+  var state: SaleFilled
+
+  setup:
+    market = MockMarket.new()
+    slot = MockSlot(requestId: request.id,
+                    host: Address.example,
+                    slotIndex: slotIndex,
+                    proof: @[])
+    let context = SalesContext(market: market)
+    agent = newSalesAgent(context,
+                          request.id,
+                          slotIndex,
+                          Availability.none,
+                          StorageRequest.none)
+    state = SaleFilled.new()
+
+  test "switches to finished state when slot is filled by me":
+    slot.host = await market.getSigner()
+    market.filled = @[slot]
+    let next = await state.run(agent)
+    check !next of SaleFinished
+
+  test "switches to error state when slot is filled by another host":
+    slot.host = Address.example
+    market.filled = @[slot]
+    let next = await state.run(agent)
+    check !next of SaleErrored

tests/codex/sales/states/testfilling.nim

@@ -0,0 +1,29 @@
+import std/unittest
+import pkg/questionable
+import pkg/codex/contracts/requests
+import pkg/codex/sales/states/filling
+import pkg/codex/sales/states/cancelled
+import pkg/codex/sales/states/failed
+import pkg/codex/sales/states/filled
+import ../../examples
+
+suite "sales state 'filling'":
+
+  let request = StorageRequest.example
+  let slotIndex = (request.ask.slots div 2).u256
+  var state: SaleFilling
+
+  setup:
+    state = SaleFilling.new()
+
+  test "switches to cancelled state when request expires":
+    let next = state.onCancelled(request)
+    check !next of SaleCancelled
+
+  test "switches to failed state when request fails":
+    let next = state.onFailed(request)
+    check !next of SaleFailed
+
+  test "switches to filled state when slot is filled":
+    let next = state.onSlotFilled(request.id, slotIndex)
+    check !next of SaleFilled

tests/codex/sales/states/testfinished.nim

@@ -0,0 +1,23 @@
+import std/unittest
+import pkg/questionable
+import pkg/codex/contracts/requests
+import pkg/codex/sales/states/finished
+import pkg/codex/sales/states/cancelled
+import pkg/codex/sales/states/failed
+import ../../examples
+
+suite "sales state 'finished'":
+
+  let request = StorageRequest.example
+  var state: SaleFinished
+
+  setup:
+    state = SaleFinished.new()
+
+  test "switches to cancelled state when request expires":
+    let next = state.onCancelled(request)
+    check !next of SaleCancelled
+
+  test "switches to failed state when request fails":
+    let next = state.onFailed(request)
+    check !next of SaleFailed

tests/codex/sales/states/testproving.nim

@@ -0,0 +1,30 @@
+import std/unittest
+import pkg/questionable
+import pkg/codex/contracts/requests
+import pkg/codex/sales/states/proving
+import pkg/codex/sales/states/cancelled
+import pkg/codex/sales/states/failed
+import pkg/codex/sales/states/filled
+import ../../examples
+
+suite "sales state 'proving'":
+
+  let request = StorageRequest.example
+  let slotIndex = (request.ask.slots div 2).u256
+  var state: SaleProving
+
+  setup:
+    state = SaleProving.new()
+
+  test "switches to cancelled state when request expires":
+    let next = state.onCancelled(request)
+    check !next of SaleCancelled
+
+  test "switches to failed state when request fails":
+    let next = state.onFailed(request)
+    check !next of SaleFailed
+
+  test "switches to filled state when slot is filled":
+    let next = state.onSlotFilled(request.id, slotIndex)
+    check !next of SaleFilled
+

tests/codex/sales/states/testunknown.nim

@@ -0,0 +1,61 @@
+import pkg/asynctest
+import pkg/codex/contracts/requests
+import pkg/codex/sales
+import pkg/codex/sales/salesagent
+import pkg/codex/sales/salescontext
+import pkg/codex/sales/states/unknown
+import pkg/codex/sales/states/errored
+import pkg/codex/sales/states/filled
+import pkg/codex/sales/states/finished
+import pkg/codex/sales/states/failed
+import ../../helpers/mockmarket
+import ../../examples
+
+suite "sales state 'unknown'":
+
+  let request = StorageRequest.example
+  let slotIndex = (request.ask.slots div 2).u256
+  let slotId = slotId(request.id, slotIndex)
+
+  var market: MockMarket
+  var agent: SalesAgent
+  var state: SaleUnknown
+
+  setup:
+    market = MockMarket.new()
+    let context = SalesContext(market: market)
+    agent = newSalesAgent(context,
+                          request.id,
+                          slotIndex,
+                          Availability.none,
+                          StorageRequest.none)
+    state = SaleUnknown.new()
+
+  test "switches to error state when on chain state cannot be fetched":
+    let next = await state.run(agent)
+    check !next of SaleErrored
+
+  test "switches to error state when on chain state is 'free'":
+    market.slotState[slotId] = SlotState.Free
+    let next = await state.run(agent)
+    check !next of SaleErrored
+
+  test "switches to filled state when on chain state is 'filled'":
+    market.slotState[slotId] = SlotState.Filled
+    let next = await state.run(agent)
+    check !next of SaleFilled
+
+  test "switches to finished state when on chain state is 'finished'":
+    market.slotState[slotId] = SlotState.Finished
+    let next = await state.run(agent)
+    check !next of SaleFinished
+
+  test "switches to finished state when on chain state is 'paid'":
+    market.slotState[slotId] = SlotState.Paid
+    let next = await state.run(agent)
+    check !next of SaleFinished
+
+  test "switches to failed state when on chain state is 'failed'":
+    market.slotState[slotId] = SlotState.Failed
+    let next = await state.run(agent)
+    check !next of SaleFailed

tests/codex/sales/testsales.nim

@@ -0,0 +1,290 @@
+import std/sets
+import std/sequtils
+import std/sugar
+import std/times
+import pkg/asynctest
+import pkg/chronos
+import pkg/codex/sales
+import pkg/codex/sales/salesdata
+import pkg/codex/proving
+import ../helpers/mockmarket
+import ../helpers/mockclock
+import ../helpers/eventually
+import ../examples
+
+suite "Sales":
+
+  let availability = Availability.init(
+    size=100.u256,
+    duration=60.u256,
+    minPrice=600.u256
+  )
+  var request = StorageRequest(
+    ask: StorageAsk(
+      slots: 4,
+      slotSize: 100.u256,
+      duration: 60.u256,
+      reward: 10.u256,
+    ),
+    content: StorageContent(
+      cid: "some cid"
+    ),
+    expiry: (getTime() + initDuration(hours=1)).toUnix.u256
+  )
+  let proof = exampleProof()
+
+  var sales: Sales
+  var market: MockMarket
+  var clock: MockClock
+  var proving: Proving
+
+  setup:
+    market = MockMarket.new()
+    clock = MockClock.new()
+    proving = Proving.new()
+    sales = Sales.new(market, clock, proving)
+    sales.onStore = proc(request: StorageRequest,
+                         slot: UInt256,
+                         availability: ?Availability) {.async.} =
+      discard
+    sales.onProve = proc(request: StorageRequest,
+                         slot: UInt256): Future[seq[byte]] {.async.} =
+      return proof
+    await sales.start()
+    request.expiry = (clock.now() + 42).u256
+
+  teardown:
+    await sales.stop()
+
+  test "has no availability initially":
+    check sales.available.len == 0
+
+  test "can add available storage":
+    let availability1 = Availability.example
+    let availability2 = Availability.example
+    sales.add(availability1)
+    check sales.available.contains(availability1)
+    sales.add(availability2)
+    check sales.available.contains(availability1)
+    check sales.available.contains(availability2)
+
+  test "can remove available storage":
+    sales.add(availability)
+    sales.remove(availability)
+    check sales.available.len == 0
+
+  test "generates unique ids for storage availability":
+    let availability1 = Availability.init(1.u256, 2.u256, 3.u256)
+    let availability2 = Availability.init(1.u256, 2.u256, 3.u256)
+    check availability1.id != availability2.id
+
+  test "makes storage unavailable when matching request comes in":
+    sales.add(availability)
+    await market.requestStorage(request)
+    check eventually sales.available.len == 0
+
+  test "ignores request when no matching storage is available":
+    sales.add(availability)
+    var tooBig = request
+    tooBig.ask.slotSize = request.ask.slotSize + 1
+    await market.requestStorage(tooBig)
+    await sleepAsync(1.millis)
+    check eventually sales.available == @[availability]
+
+  test "ignores request when reward is too low":
+    sales.add(availability)
+    var tooCheap = request
+    tooCheap.ask.reward = request.ask.reward - 1
+    await market.requestStorage(tooCheap)
+    await sleepAsync(1.millis)
+    check eventually sales.available == @[availability]
+
+  test "retrieves and stores data locally":
+    var storingRequest: StorageRequest
+    var storingSlot: UInt256
+    var storingAvailability: Availability
+    sales.onStore = proc(request: StorageRequest,
+                         slot: UInt256,
+                         availability: ?Availability) {.async.} =
+      storingRequest = request
+      storingSlot = slot
+      check availability.isSome
+      storingAvailability = !availability
+    sales.add(availability)
+    await market.requestStorage(request)
+    check eventually storingRequest == request
+    check storingSlot < request.ask.slots.u256
+    check storingAvailability == availability
+
+  test "handles errors during state run":
+    var saleFailed = false
+    sales.onStore = proc(request: StorageRequest,
+                         slot: UInt256,
+                         availability: ?Availability) {.async.} =
+      # raise an exception so machine.onError is called
+      raise newException(ValueError, "some error")
+
+    # onSaleErrored is called in SaleErrored.run
+    proc onSaleErrored(availability: Availability) =
+      saleFailed = true
+    sales.context.onSaleErrored = some onSaleErrored
+    sales.add(availability)
+    await market.requestStorage(request)
+    check eventually saleFailed
+
+  test "makes storage available again when data retrieval fails":
+    let error = newException(IOError, "data retrieval failed")
+    sales.onStore = proc(request: StorageRequest,
+                         slot: UInt256,
+                         availability: ?Availability) {.async.} =
+      raise error
+    sales.add(availability)
+    await market.requestStorage(request)
+    await sleepAsync(1.millis)
+    check eventually sales.available == @[availability]
+
+  test "generates proof of storage":
+    var provingRequest: StorageRequest
+    var provingSlot: UInt256
+    sales.onProve = proc(request: StorageRequest,
+                         slot: UInt256): Future[seq[byte]] {.async.} =
+      provingRequest = request
+      provingSlot = slot
+    sales.add(availability)
+    await market.requestStorage(request)
+    check eventually provingRequest == request
+    check provingSlot < request.ask.slots.u256
+
+  test "fills a slot":
+    sales.add(availability)
+    await market.requestStorage(request)
+    check eventually market.filled.len == 1
+    check market.filled[0].requestId == request.id
+    check market.filled[0].slotIndex < request.ask.slots.u256
+    check market.filled[0].proof == proof
+    check market.filled[0].host == await market.getSigner()
+
+  test "calls onSale when slot is filled":
+    var soldAvailability: Availability
+    var soldRequest: StorageRequest
+    var soldSlotIndex: UInt256
+    sales.onSale = proc(availability: ?Availability,
+                        request: StorageRequest,
+                        slotIndex: UInt256) =
+      if a =? availability:
+        soldAvailability = a
+      soldRequest = request
+      soldSlotIndex = slotIndex
+    sales.add(availability)
+    await market.requestStorage(request)
+    check eventually soldAvailability == availability
+    check soldRequest == request
+    check soldSlotIndex < request.ask.slots.u256
+
+  test "calls onClear when storage becomes available again":
+    # fail the proof intentionally to trigger `agent.finish(success=false)`,
+    # which then calls the onClear callback
+    sales.onProve = proc(request: StorageRequest,
+                         slot: UInt256): Future[seq[byte]] {.async.} =
+      raise newException(IOError, "proof failed")
+    var clearedAvailability: Availability
+    var clearedRequest: StorageRequest
+    var clearedSlotIndex: UInt256
+    sales.onClear = proc(availability: ?Availability,
+                         request: StorageRequest,
+                         slotIndex: UInt256) =
+      if a =? availability:
+        clearedAvailability = a
+      clearedRequest = request
+      clearedSlotIndex = slotIndex
+    sales.add(availability)
+    await market.requestStorage(request)
+    check eventually clearedAvailability == availability
+    check clearedRequest == request
+    check clearedSlotIndex < request.ask.slots.u256
+
+  test "makes storage available again when other host fills the slot":
+    let otherHost = Address.example
+    sales.onStore = proc(request: StorageRequest,
+                         slot: UInt256,
+                         availability: ?Availability) {.async.} =
+      await sleepAsync(chronos.hours(1))
+    sales.add(availability)
+    await market.requestStorage(request)
+    await sleepAsync(1.millis)
+    for slotIndex in 0..<request.ask.slots:
+      market.fillSlot(request.id, slotIndex.u256, proof, otherHost)
+    await sleepAsync(chronos.seconds(2))
+    check sales.available == @[availability]
+
+  test "makes storage available again when request expires":
+    sales.onStore = proc(request: StorageRequest,
+                         slot: UInt256,
+                         availability: ?Availability) {.async.} =
+      await sleepAsync(chronos.hours(1))
+    sales.add(availability)
+    await market.requestStorage(request)
+    await sleepAsync(1.millis)
+    clock.set(request.expiry.truncate(int64))
+    check eventually (sales.available == @[availability])
+
+  test "adds proving for slot when slot is filled":
+    var soldSlotIndex: UInt256
+    sales.onSale = proc(availability: ?Availability,
+                        request: StorageRequest,
+                        slotIndex: UInt256) =
+      soldSlotIndex = slotIndex
+    check proving.slots.len == 0
+    sales.add(availability)
+    await market.requestStorage(request)
+    check eventually proving.slots.len == 1
+    check proving.slots.contains(request.slotId(soldSlotIndex))
+
+  test "loads active slots from market":
+    let me = await market.getSigner()
+
+    request.ask.slots = 2
+    market.requested = @[request]
+    market.requestState[request.id] = RequestState.New
+
+    proc fillSlot(slotIdx: UInt256 = 0.u256) {.async.} =
+      let address = await market.getSigner()
+      let slot = MockSlot(requestId: request.id,
+                          slotIndex: slotIdx,
+                          proof: proof,
+                          host: address)
+      market.filled.add slot
+      market.slotState[slotId(request.id, slotIdx)] = SlotState.Filled
+
+    let slot0 = MockSlot(requestId: request.id,
+                     slotIndex: 0.u256,
+                     proof: proof,
+                     host: me)
+    await fillSlot(slot0.slotIndex)
+
+    let slot1 = MockSlot(requestId: request.id,
+                     slotIndex: 1.u256,
+                     proof: proof,
+                     host: me)
+    await fillSlot(slot1.slotIndex)
+    market.activeSlots[me] = @[request.slotId(0.u256), request.slotId(1.u256)]
+    market.requested = @[request]
+    market.activeRequests[me] = @[request.id]
+
+    await sales.load()
+    let expected = SalesData(requestId: request.id,
+                             availability: none Availability,
+                             request: some request)
+    # because sales.load() calls agent.start, we won't know the slotIndex
+    # randomly selected for the agent, and we also won't know the value of
+    # `failed`/`fulfilled`/`cancelled` futures, so we need to compare
+    # the properties we know
+    # TODO: when calling sales.load(), slot index should be restored and not
+    # randomly re-assigned, so this may no longer be needed
+    proc `==` (data0, data1: SalesData): bool =
+      return data0.requestId == data1.requestId and
+             data0.availability == data1.availability and
+             data0.request == data1.request
+
+    check eventually sales.agents.len == 2
+    check sales.agents.all(agent => agent.data == expected)

tests/codex/sales/testsalesagent.nim

@@ -0,0 +1,165 @@
+import std/sets
+import std/sequtils
+import std/sugar
+import std/times
+import pkg/asynctest
+import pkg/chronos
+import pkg/codex/sales
+import pkg/codex/sales/salesagent
+import pkg/codex/sales/salescontext
+import pkg/codex/sales/statemachine
+import pkg/codex/sales/states/errorhandling
+import pkg/codex/proving
+import ../helpers/mockmarket
+import ../helpers/mockclock
+import ../helpers/eventually
+import ../examples
+
+var onCancelCalled = false
+var onFailedCalled = false
+var onSlotFilledCalled = false
+var onErrorCalled = false
+
+type
+  MockState = ref object of SaleState
+  MockErrorState = ref object of ErrorHandlingState
+
+method `$`*(state: MockState): string = "MockState"
+
+method onCancelled*(state: MockState, request: StorageRequest): ?State =
+  onCancelCalled = true
+
+method onFailed*(state: MockState, request: StorageRequest): ?State =
+  onFailedCalled = true
+
+method onSlotFilled*(state: MockState, requestId: RequestId,
+                    slotIndex: UInt256): ?State =
+  onSlotFilledCalled = true
+
+method onError*(state: MockErrorState, err: ref CatchableError): ?State =
+  onErrorCalled = true
+
+method run*(state: MockErrorState, machine: Machine): Future[?State] {.async.} =
+  raise newException(ValueError, "failure")
+
+suite "Sales agent":
+
+  let availability = Availability.init(
+    size=100.u256,
+    duration=60.u256,
+    minPrice=600.u256
+  )
+  var request = StorageRequest(
+    ask: StorageAsk(
+      slots: 4,
+      slotSize: 100.u256,
+      duration: 60.u256,
+      reward: 10.u256,
+    ),
+    content: StorageContent(
+      cid: "some cid"
+    ),
+    expiry: (getTime() + initDuration(hours=1)).toUnix.u256
+  )
+
+  var agent: SalesAgent
+  var context: SalesContext
+  var slotIndex: UInt256
+  var market: MockMarket
+  var clock: MockClock
+
+  setup:
+    market = MockMarket.new()
+    clock = MockClock.new()
+    context = SalesContext(market: market, clock: clock)
+    slotIndex = 0.u256
+    onCancelCalled = false
+    onFailedCalled = false
+    onSlotFilledCalled = false
+    agent = newSalesAgent(context,
+                          request.id,
+                          slotIndex,
+                          some availability,
+                          some request)
+    request.expiry = (getTime() + initDuration(hours=1)).toUnix.u256
+
+  teardown:
+    await agent.stop()
+
+  test "can retrieve request":
+    agent = newSalesAgent(context,
+                          request.id,
+                          slotIndex,
+                          some availability,
+                          none StorageRequest)
+    market.requested = @[request]
+    await agent.retrieveRequest()
+    check agent.data.request == some request
+
+  test "subscribe assigns subscriptions/futures":
+    await agent.subscribe()
+    check not agent.data.cancelled.isNil
+    check not agent.data.failed.isNil
+    check not agent.data.fulfilled.isNil
+    check not agent.data.slotFilled.isNil
+
+  test "unsubscribe deassigns subscriptions/futures":
+    await agent.subscribe()
+    await agent.unsubscribe()
+    check agent.data.cancelled.isNil
+    check agent.data.failed.isNil
+    check agent.data.fulfilled.isNil
+    check agent.data.slotFilled.isNil
+
+  test "subscribe can be called multiple times, without overwriting subscriptions/futures":
+    await agent.subscribe()
+    let cancelled = agent.data.cancelled
+    let failed = agent.data.failed
+    let fulfilled = agent.data.fulfilled
+    let slotFilled = agent.data.slotFilled
+    await agent.subscribe()
+    check cancelled == agent.data.cancelled
+    check failed == agent.data.failed
+    check fulfilled == agent.data.fulfilled
+    check slotFilled == agent.data.slotFilled
+
+  test "unsubscribe can be called multiple times":
+    await agent.subscribe()
+    await agent.unsubscribe()
+    await agent.unsubscribe()
+
+  test "subscribe can be called when request expiry has lapsed":
+    # succeeds when agent.data.fulfilled.isNil
+    request.expiry = (getTime() - initDuration(seconds=1)).toUnix.u256
+    agent.data.request = some request
+    check agent.data.fulfilled.isNil
+    await agent.subscribe()
+
+  test "current state onCancelled called when cancel emitted":
+    let state = MockState.new()
+    agent.start(state)
+    await agent.subscribe()
+    clock.set(request.expiry.truncate(int64))
+    check eventually onCancelCalled
+
+  test "cancelled future is finished (cancelled) when fulfillment emitted":
+    agent.start(MockState.new())
+    await agent.subscribe()
+    market.emitRequestFulfilled(request.id)
+    check eventually agent.data.cancelled.cancelled()
+
+  test "current state onFailed called when failed emitted":
+    agent.start(MockState.new())
+    await agent.subscribe()
+    market.emitRequestFailed(request.id)
+    check eventually onFailedCalled
+
+  test "current state onSlotFilled called when slot filled emitted":
+    agent.start(MockState.new())
+    await agent.subscribe()
+    market.emitSlotFilled(request.id, slotIndex)
+    check eventually onSlotFilledCalled
+
+  test "ErrorHandlingState.onError can be overridden at the state level":
+    agent.start(MockErrorState.new())
+    check eventually onErrorCalled

tests/codex/sales/teststates.nim

@@ -0,0 +1,8 @@
+import ./states/testunknown
+import ./states/testdownloading
+import ./states/testfilling
+import ./states/testfinished
+import ./states/testproving
+import ./states/testfilled
+
+{.warning[UnusedImport]: off.}

tests/codex/testproving.nim

@@ -47,6 +47,7 @@ suite "Proving":
     proc onProofRequired(id: SlotId) =
       called = true
     proving.onProofRequired = onProofRequired
+    proofs.setSlotState(id, SlotState.Filled)
     proofs.setProofRequired(id, true)
     await proofs.advanceToNextPeriod()
     check eventually called
@@ -59,6 +60,8 @@ suite "Proving":
     proc onProofRequired(id: SlotId) =
       callbackIds.add(id)
     proving.onProofRequired = onProofRequired
+    proofs.setSlotState(id1, SlotState.Filled)
+    proofs.setSlotState(id2, SlotState.Filled)
     proofs.setProofRequired(id1, true)
     await proofs.advanceToNextPeriod()
     check eventually callbackIds == @[id1]
@@ -76,6 +79,7 @@ suite "Proving":
     proving.onProofRequired = onProofRequired
     proofs.setProofRequired(id, false)
     proofs.setProofToBeRequired(id, true)
+    proofs.setSlotState(id, SlotState.Filled)
     await proofs.advanceToNextPeriod()
     check eventually called
 
@@ -90,6 +94,7 @@ suite "Proving":
     proving.onProofRequired = onProofRequired
     proofs.setProofRequired(id, true)
     await proofs.advanceToNextPeriod()
+    proofs.setSlotState(id, SlotState.Finished)
     check eventually (not proving.slots.contains(id))
     check not called
 

tests/codex/testpurchasing.nim

@@ -141,11 +141,11 @@ suite "Purchasing state machine":
     let request1, request2, request3, request4, request5 = StorageRequest.example
     market.requested = @[request1, request2, request3, request4, request5]
     market.activeRequests[me] = @[request1.id, request2.id, request3.id, request4.id, request5.id]
-    market.state[request1.id] = RequestState.New
+    market.requestState[request1.id] = RequestState.New
-    market.state[request2.id] = RequestState.Started
+    market.requestState[request2.id] = RequestState.Started
-    market.state[request3.id] = RequestState.Cancelled
+    market.requestState[request3.id] = RequestState.Cancelled
-    market.state[request4.id] = RequestState.Finished
+    market.requestState[request4.id] = RequestState.Finished
-    market.state[request5.id] = RequestState.Failed
+    market.requestState[request5.id] = RequestState.Failed
 
     # ensure the started state doesn't error, giving a false positive test result
     market.requestEnds[request2.id] = clock.now() - 1
@@ -162,7 +162,7 @@ suite "Purchasing state machine":
     let request = StorageRequest.example
     let purchase = Purchase.new(request, market, clock)
     market.requested = @[request]
-    market.state[request.id] = RequestState.New
+    market.requestState[request.id] = RequestState.New
     purchase.switch(PurchaseUnknown())
     check (purchase.state as PurchaseSubmitted).isSome
 
@@ -171,7 +171,7 @@ suite "Purchasing state machine":
     let purchase = Purchase.new(request, market, clock)
     market.requestEnds[request.id] = clock.now() + request.ask.duration.truncate(int64)
     market.requested = @[request]
-    market.state[request.id] = RequestState.Started
+    market.requestState[request.id] = RequestState.Started
     purchase.switch(PurchaseUnknown())
     check (purchase.state as PurchaseStarted).isSome
 
@@ -179,7 +179,7 @@ suite "Purchasing state machine":
     let request = StorageRequest.example
     let purchase = Purchase.new(request, market, clock)
     market.requested = @[request]
-    market.state[request.id] = RequestState.Cancelled
+    market.requestState[request.id] = RequestState.Cancelled
     purchase.switch(PurchaseUnknown())
     check (purchase.state as PurchaseErrored).isSome
     check purchase.error.?msg == "Purchase cancelled due to timeout".some
@@ -188,7 +188,7 @@ suite "Purchasing state machine":
     let request = StorageRequest.example
     let purchase = Purchase.new(request, market, clock)
     market.requested = @[request]
-    market.state[request.id] = RequestState.Finished
+    market.requestState[request.id] = RequestState.Finished
     purchase.switch(PurchaseUnknown())
     check (purchase.state as PurchaseFinished).isSome
 
@@ -196,7 +196,7 @@ suite "Purchasing state machine":
     let request = StorageRequest.example
     let purchase = Purchase.new(request, market, clock)
     market.requested = @[request]
-    market.state[request.id] = RequestState.Failed
+    market.requestState[request.id] = RequestState.Failed
     purchase.switch(PurchaseUnknown())
     check (purchase.state as PurchaseErrored).isSome
     check purchase.error.?msg == "Purchase failed".some
@@ -206,7 +206,7 @@ suite "Purchasing state machine":
     let request = StorageRequest.example
     market.requested = @[request]
     market.activeRequests[me] = @[request.id]
-    market.state[request.id] = RequestState.Started
+    market.requestState[request.id] = RequestState.Started
     market.requestEnds[request.id] = clock.now() + request.ask.duration.truncate(int64)
     await purchasing.load()
 
@@ -226,7 +226,7 @@ suite "Purchasing state machine":
     let request = StorageRequest.example
     market.requested = @[request]
     market.activeRequests[me] = @[request.id]
-    market.state[request.id] = RequestState.Started
+    market.requestState[request.id] = RequestState.Started
     market.requestEnds[request.id] = clock.now() + request.ask.duration.truncate(int64)
     await purchasing.load()
 
@@ -234,7 +234,7 @@ suite "Purchasing state machine":
     clock.advance(request.ask.duration.truncate(int64))
 
     # now check the result
-    proc getState: ?PurchaseState =
+    proc requestState: ?PurchaseState =
       purchasing.getPurchase(PurchaseId(request.id)).?state as PurchaseState
 
-    check eventually (getState() as PurchaseFinished).isSome
+    check eventually (requestState() as PurchaseFinished).isSome

tests/codex/testsales.nim

@@ -1,212 +1,4 @@
-import std/sets
+import ./sales/testsales
-import pkg/asynctest
+import ./sales/teststates
-import pkg/chronos
-import pkg/codex/contracts/requests
-import pkg/codex/proving
-import pkg/codex/sales
-import ./helpers/mockmarket
-import ./helpers/mockclock
-import ./helpers/eventually
-import ./examples
 
-suite "Sales":
+{.warning[UnusedImport]: off.}
-
-  let availability = Availability.init(
-    size=100.u256,
-    duration=60.u256,
-    minPrice=600.u256
-  )
-  var request = StorageRequest(
-    ask: StorageAsk(
-      slots: 4,
-      slotSize: 100.u256,
-      duration: 60.u256,
-      reward: 10.u256,
-    ),
-    content: StorageContent(
-      cid: "some cid"
-    )
-  )
-  let proof = exampleProof()
-
-  var sales: Sales
-  var market: MockMarket
-  var clock: MockClock
-  var proving: Proving
-
-  setup:
-    market = MockMarket.new()
-    clock = MockClock.new()
-    proving = Proving.new()
-    sales = Sales.new(market, clock, proving)
-    sales.onStore = proc(request: StorageRequest,
-                         slot: UInt256,
-                         availability: Availability) {.async.} =
-      discard
-    sales.onProve = proc(request: StorageRequest,
-                         slot: UInt256): Future[seq[byte]] {.async.} =
-      return proof
-    await sales.start()
-    request.expiry = (clock.now() + 42).u256
-
-  teardown:
-    await sales.stop()
-
-  test "has no availability initially":
-    check sales.available.len == 0
-
-  test "can add available storage":
-    let availability1 = Availability.example
-    let availability2 = Availability.example
-    sales.add(availability1)
-    check sales.available.contains(availability1)
-    sales.add(availability2)
-    check sales.available.contains(availability1)
-    check sales.available.contains(availability2)
-
-  test "can remove available storage":
-    sales.add(availability)
-    sales.remove(availability)
-    check sales.available.len == 0
-
-  test "generates unique ids for storage availability":
-    let availability1 = Availability.init(1.u256, 2.u256, 3.u256)
-    let availability2 = Availability.init(1.u256, 2.u256, 3.u256)
-    check availability1.id != availability2.id
-
-  test "makes storage unavailable when matching request comes in":
-    sales.add(availability)
-    await market.requestStorage(request)
-    check sales.available.len == 0
-
-  test "ignores request when no matching storage is available":
-    sales.add(availability)
-    var tooBig = request
-    tooBig.ask.slotSize = request.ask.slotSize + 1
-    await market.requestStorage(tooBig)
-    check sales.available == @[availability]
-
-  test "ignores request when reward is too low":
-    sales.add(availability)
-    var tooCheap = request
-    tooCheap.ask.reward = request.ask.reward - 1
-    await market.requestStorage(tooCheap)
-    check sales.available == @[availability]
-
-  test "retrieves and stores data locally":
-    var storingRequest: StorageRequest
-    var storingSlot: UInt256
-    var storingAvailability: Availability
-    sales.onStore = proc(request: StorageRequest,
-                         slot: UInt256,
-                         availability: Availability) {.async.} =
-      storingRequest = request
-      storingSlot = slot
-      storingAvailability = availability
-    sales.add(availability)
-    await market.requestStorage(request)
-    check storingRequest == request
-    check storingSlot < request.ask.slots.u256
-    check storingAvailability == availability
-
-  test "makes storage available again when data retrieval fails":
-    let error = newException(IOError, "data retrieval failed")
-    sales.onStore = proc(request: StorageRequest,
-                         slot: UInt256,
-                         availability: Availability) {.async.} =
-      raise error
-    sales.add(availability)
-    await market.requestStorage(request)
-    check sales.available == @[availability]
-
-  test "generates proof of storage":
-    var provingRequest: StorageRequest
-    var provingSlot: UInt256
-    sales.onProve = proc(request: StorageRequest,
-                         slot: UInt256): Future[seq[byte]] {.async.} =
-      provingRequest = request
-      provingSlot = slot
-    sales.add(availability)
-    await market.requestStorage(request)
-    check provingRequest == request
-    check provingSlot < request.ask.slots.u256
-
-  test "fills a slot":
-    sales.add(availability)
-    await market.requestStorage(request)
-    check market.filled.len == 1
-    check market.filled[0].requestId == request.id
-    check market.filled[0].slotIndex < request.ask.slots.u256
-    check market.filled[0].proof == proof
-    check market.filled[0].host == await market.getSigner()
-
-  test "calls onSale when slot is filled":
-    var soldAvailability: Availability
-    var soldRequest: StorageRequest
-    var soldSlotIndex: UInt256
-    sales.onSale = proc(availability: Availability,
-                        request: StorageRequest,
-                        slotIndex: UInt256) =
-      soldAvailability = availability
-      soldRequest = request
-      soldSlotIndex = slotIndex
-    sales.add(availability)
-    await market.requestStorage(request)
-    check soldAvailability == availability
-    check soldRequest == request
-    check soldSlotIndex < request.ask.slots.u256
-
-  test "calls onClear when storage becomes available again":
-    # fail the proof intentionally to trigger `agent.finish(success=false)`,
-    # which then calls the onClear callback
-    sales.onProve = proc(request: StorageRequest,
-                         slot: UInt256): Future[seq[byte]] {.async.} =
-      raise newException(IOError, "proof failed")
-    var clearedAvailability: Availability
-    var clearedRequest: StorageRequest
-    var clearedSlotIndex: UInt256
-    sales.onClear = proc(availability: Availability,
-                         request: StorageRequest,
-                         slotIndex: UInt256) =
-      clearedAvailability = availability
-      clearedRequest = request
-      clearedSlotIndex = slotIndex
-    sales.add(availability)
-    await market.requestStorage(request)
-    check clearedAvailability == availability
-    check clearedRequest == request
-    check clearedSlotIndex < request.ask.slots.u256
-
-  test "makes storage available again when other host fills the slot":
-    let otherHost = Address.example
-    sales.onStore = proc(request: StorageRequest,
-                         slot: UInt256,
-                         availability: Availability) {.async.} =
-      await sleepAsync(1.hours)
-    sales.add(availability)
-    await market.requestStorage(request)
-    for slotIndex in 0..<request.ask.slots:
-      market.fillSlot(request.id, slotIndex.u256, proof, otherHost)
-    check sales.available == @[availability]
-
-  test "makes storage available again when request expires":
-    sales.onStore = proc(request: StorageRequest,
-                         slot: UInt256,
-                         availability: Availability) {.async.} =
-      await sleepAsync(1.hours)
-    sales.add(availability)
-    await market.requestStorage(request)
-    clock.set(request.expiry.truncate(int64))
-    check eventually (sales.available == @[availability])
-
-  test "adds proving for slot when slot is filled":
-    var soldSlotIndex: UInt256
-    sales.onSale = proc(availability: Availability,
-                        request: StorageRequest,
-                        slotIndex: UInt256) =
-      soldSlotIndex = slotIndex
-    check proving.slots.len == 0
-    sales.add(availability)
-    await market.requestStorage(request)
-    check proving.slots.len == 1
-    check proving.slots.contains(request.slotId(soldSlotIndex))

tests/codex/testutils.nim

@@ -1,5 +1,7 @@
 import ./utils/teststatemachine
+import ./utils/teststatemachineasync
 import ./utils/testoptionalcast
 import ./utils/testkeyutils
+import ./utils/testasyncstatemachine
 
 {.warning[UnusedImport]: off.}

tests/codex/utils/testasyncstatemachine.nim

@@ -0,0 +1,110 @@
+import pkg/asynctest
+import pkg/questionable
+import pkg/chronos
+import pkg/upraises
+import codex/utils/asyncstatemachine
+import ../helpers/eventually
+
+type
+  State1 = ref object of State
+  State2 = ref object of State
+  State3 = ref object of State
+  State4 = ref object of State
+
+var runs, cancellations, errors = [0, 0, 0, 0]
+
+method run(state: State1, machine: Machine): Future[?State] {.async.} =
+  inc runs[0]
+  return some State(State2.new())
+
+method run(state: State2, machine: Machine): Future[?State] {.async.} =
+  inc runs[1]
+  try:
+    await sleepAsync(1.hours)
+  except CancelledError:
+    inc cancellations[1]
+    raise
+
+method run(state: State3, machine: Machine): Future[?State] {.async.} =
+  inc runs[2]
+
+method run(state: State4, machine: Machine): Future[?State] {.async.} =
+  inc runs[3]
+  raise newException(ValueError, "failed")
+
+method onMoveToNextStateEvent*(state: State): ?State {.base, upraises:[].} =
+  discard
+
+method onMoveToNextStateEvent(state: State2): ?State =
+  some State(State3.new())
+
+method onMoveToNextStateEvent(state: State3): ?State =
+  some State(State1.new())
+
+method onError(state: State1, error: ref CatchableError): ?State =
+  inc errors[0]
+
+method onError(state: State2, error: ref CatchableError): ?State =
+  inc errors[1]
+
+method onError(state: State3, error: ref CatchableError): ?State =
+  inc errors[2]
+
+method onError(state: State4, error: ref CatchableError): ?State =
+  inc errors[3]
+  some State(State2.new())
+
+suite "async state machines":
+  var machine: Machine
+
+  proc moveToNextStateEvent(state: State): ?State =
+    state.onMoveToNextStateEvent()
+
+  setup:
+    runs = [0, 0, 0, 0]
+    cancellations = [0, 0, 0, 0]
+    errors = [0, 0, 0, 0]
+    machine = Machine.new()
+
+  test "should call run on start state":
+    machine.start(State1.new())
+    check eventually runs[0] == 1
+
+  test "moves to next state when run completes":
+    machine.start(State1.new())
+    check eventually runs == [1, 1, 0, 0]
+
+  test "state2 moves to state3 on event":
+    machine.start(State2.new())
+    machine.schedule(moveToNextStateEvent)
+    check eventually runs == [0, 1, 1, 0]
+
+  test "state transition will cancel the running state":
+    machine.start(State2.new())
+    machine.schedule(moveToNextStateEvent)
+    check eventually cancellations == [0, 1, 0, 0]
+
+  test "scheduled events are handled one after the other":
+    machine.start(State2.new())
+    machine.schedule(moveToNextStateEvent)
+    machine.schedule(moveToNextStateEvent)
+    check eventually runs == [1, 2, 1, 0]
+
+  test "stops scheduling and current state":
+    machine.start(State2.new())
+    await sleepAsync(1.millis)
+    machine.stop()
+    machine.schedule(moveToNextStateEvent)
+    await sleepAsync(1.millis)
+    check runs == [0, 1, 0, 0]
+    check cancellations == [0, 1, 0, 0]
+
+  test "forwards errors to error handler":
+    machine.start(State4.new())
+    check eventually errors == [0, 0, 0, 1] and runs == [0, 1, 0, 1]
+
+  test "error handler ignores CancelledError":
+    machine.start(State2.new())
+    machine.schedule(moveToNextStateEvent)
+    check eventually cancellations == [0, 1, 0, 0]
+    check errors == [0, 0, 0, 0]

tests/codex/utils/teststatemachineasync.nim

@@ -0,0 +1,30 @@
+import pkg/asynctest
+import pkg/chronos
+import pkg/questionable
+import codex/utils/statemachine
+
+type
+  AsyncMachine = ref object of StateMachineAsync
+  LongRunningStart = ref object of AsyncState
+  LongRunningFinish = ref object of AsyncState
+  LongRunningError = ref object of AsyncState
+  Callback = proc(): Future[void] {.gcsafe.}
+
+proc triggerIn(time: Duration, cb: Callback) {.async.} =
+  await sleepAsync(time)
+  await cb()
+
+method enterAsync(state: LongRunningStart) {.async.} =
+  proc cb() {.async.} =
+    await state.switchAsync(LongRunningFinish())
+  asyncSpawn triggerIn(500.milliseconds, cb)
+  await sleepAsync(1.seconds)
+  await state.switchAsync(LongRunningError())
+
+suite "async state machines":
+
+  test "can cancel a state":
+    let am = AsyncMachine()
+    await am.switchAsync(LongRunningStart())
+    await sleepAsync(2.seconds)
+    check (am.state as LongRunningFinish).isSome

tests/contracts/testContracts.nim

@@ -34,8 +34,9 @@ ethersuite "Marketplace contracts":
     await token.mint(await client.getAddress(), 1_000_000_000.u256)
     await token.mint(await host.getAddress(), 1000_000_000.u256)
 
-    collateral = await marketplace.collateral()
+    let config = await marketplace.config()
-    periodicity = Periodicity(seconds: await marketplace.proofPeriod())
+    collateral = config.collateral.initialAmount
+    periodicity = Periodicity(seconds: config.proofs.period)
 
     request = StorageRequest.example
     request.client = await client.getAddress()

tests/contracts/testMarket.nim

@@ -1,6 +1,7 @@
 import std/options
 import pkg/chronos
 import pkg/ethers/testing
+import pkg/stew/byteutils
 import codex/contracts
 import codex/contracts/testtoken
 import codex/storageproofs
@@ -24,12 +25,13 @@ ethersuite "On-Chain Market":
     token = TestToken.new(!deployment.address(TestToken), provider.getSigner())
     await token.mint(accounts[0], 1_000_000_000.u256)
 
-    let collateral = await marketplace.collateral()
+    let config = await marketplace.config()
+    let collateral = config.collateral.initialAmount
     await token.approve(marketplace.address, collateral)
     await marketplace.deposit(collateral)
 
     market = OnChainMarket.new(marketplace)
-    periodicity = Periodicity(seconds: await marketplace.proofPeriod())
+    periodicity = Periodicity(seconds: config.proofs.period)
 
     request = StorageRequest.example
     request.client = accounts[0]
@@ -187,14 +189,13 @@ ethersuite "On-Chain Market":
     for slotIndex in 0..request.ask.maxSlotLoss:
       let slotId = request.slotId(slotIndex.u256)
       while true:
-        try:
+        let slotState = await marketplace.slotState(slotId)
-          await waitUntilProofRequired(slotId)
+        if slotState == SlotState.Free:
-          let missingPeriod = periodicity.periodOf(await provider.currentTime())
+          break
-          await provider.advanceTime(periodicity.seconds)
+        await waitUntilProofRequired(slotId)
-          await marketplace.markProofAsMissing(slotId, missingPeriod)
+        let missingPeriod = periodicity.periodOf(await provider.currentTime())
-        except ProviderError as e:
+        await provider.advanceTime(periodicity.seconds)
-          if e.revertReason == "Slot empty":
+        await marketplace.markProofAsMissing(slotId, missingPeriod)
-            break
     check receivedIds == @[request.id]
     await subscription.unsubscribe()
 
@@ -230,9 +231,45 @@ ethersuite "On-Chain Market":
     await market.requestStorage(request2)
     check (await market.myRequests()) == @[request.id, request2.id]
 
+  test "retrieves correct request state when request is unknown":
+    check (await market.requestState(request.id)) == none RequestState
+
   test "can retrieve request state":
     await token.approve(marketplace.address, request.price)
     await market.requestStorage(request)
     for slotIndex in 0..<request.ask.slots:
       await market.fillSlot(request.id, slotIndex.u256, proof)
-    check (await market.getState(request.id)) == some RequestState.Started
+    check (await market.requestState(request.id)) == some RequestState.Started
+
+  test "can retrieve active slots":
+    await token.approve(marketplace.address, request.price)
+    await market.requestStorage(request)
+    await market.fillSlot(request.id, slotIndex - 1, proof)
+    await market.fillSlot(request.id, slotIndex, proof)
+    let slotId1 = request.slotId(slotIndex - 1)
+    let slotId2 = request.slotId(slotIndex)
+    check (await market.mySlots()) == @[slotId1, slotId2]
+
+  test "returns none when slot is empty":
+    await token.approve(marketplace.address, request.price)
+    await market.requestStorage(request)
+    let slotId = request.slotId(slotIndex)
+    check (await market.getRequestFromSlotId(slotId)) == none StorageRequest
+
+  test "can retrieve request details from slot id":
+    await token.approve(marketplace.address, request.price)
+    await market.requestStorage(request)
+    await market.fillSlot(request.id, slotIndex, proof)
+    let slotId = request.slotId(slotIndex)
+    check (await market.getRequestFromSlotId(slotId)) == some request
+
+  test "retrieves correct slot state when request is unknown":
+    let slotId = request.slotId(slotIndex)
+    check (await market.slotState(slotId)) == SlotState.Free
+
+  test "retrieves correct slot state once filled":
+    await token.approve(marketplace.address, request.price)
+    await market.requestStorage(request)
+    await market.fillSlot(request.id, slotIndex, proof)
+    let slotId = request.slotId(slotIndex)
+    check (await market.slotState(slotId)) == SlotState.Filled

tests/contracts/testProofs.nim

@@ -1,4 +1,5 @@
 import codex/contracts
+import codex/contracts/testtoken
 import ../ethertest
 import ./examples
 import ./time
@@ -18,7 +19,8 @@ ethersuite "On-Chain Proofs":
 
   test "can retrieve proof periodicity":
     let periodicity = await proofs.periodicity()
-    let periodLength = await marketplace.proofPeriod()
+    let config = await marketplace.config()
+    let periodLength = config.proofs.period
     check periodicity.seconds == periodLength
 
   test "supports checking whether proof is required now":
@@ -27,8 +29,8 @@ ethersuite "On-Chain Proofs":
   test "supports checking whether proof is required soon":
     check (await proofs.willProofBeRequired(contractId)) == false
 
-  test "retrieves proof end time":
+  test "retrieves correct slot state when request is unknown":
-    check (await proofs.getProofEnd(contractId)) == 0.u256
+    check (await proofs.slotState(SlotId.example)) == SlotState.Free
 
   test "submits proofs":
     await proofs.submitProof(contractId, proof)
@@ -55,6 +57,3 @@ ethersuite "On-Chain Proofs":
 
   test "proof will not be required when slot is empty":
     check not await proofs.willProofBeRequired(contractId)
-
-  test "proof end is zero when slot is empty":
-    check (await proofs.getProofEnd(contractId)) == 0.u256

tests/integration/tokens.nim

@@ -12,4 +12,5 @@ proc mint*(signer: Signer, amount = 1_000_000.u256) {.async.} =
 proc deposit*(signer: Signer) {.async.} =
   ## Deposits sufficient collateral into the Marketplace contract.
   let marketplace = Marketplace.new(!deployment().address(Marketplace), signer)
-  await marketplace.deposit(await marketplace.collateral())
+  let config = await marketplace.config()
+  await marketplace.deposit(config.collateral.initialAmount)

vendor/codex-contracts-eth

@@ -1 +1 @@
-Subproject commit ce335d0568b4d056def6570f7d5d5c0d6a840083
+Subproject commit cde543626236bd48188354d842cbe1513052c560