nim-codex/tests/contracts/testMarket.nim

321 lines
12 KiB
Nim
Raw Permalink Normal View History

import std/options
import pkg/chronos
[marketplace] Load sales state from chain (#306) * [marketplace] get active slots from chain # Conflicts: # codex/contracts/market.nim * [marketplace] make on chain event callbacks async # Conflicts: # tests/codex/helpers/mockmarket.nim * [marketplace] make availability optional for node restart # Conflicts: # tests/codex/testsales.nim * [marketplace] add async state machine Allows for `enterAsync` to be cancelled. * [marketplace] move sale process to async state machine * [marketplace] sales state machine tests * bump dagger-contracts * [marketplace] fix ci issue with chronicles output * PR comments - add slotIndex to `SalesAgent` constructor - remove `SalesAgent.init` - rename `SalesAgent.init` to `start` and `SalesAgent.deinit` to `stop`. - rename `SalesAgent. populateRequest` to `SalesAgent.retreiveRequest`. - move availability removal to the downloading state. once availability is persisted to disk, it should survive node restarts. - * [marketplace] handle slot filled by other host Handle the case when in the downloading, proving, or filling states, that another host fills the slot. * [marketplace] use requestId for mySlots * [marketplace] infer slot index from slotid prevents reassigning a random slot index when restoring state from chain * [marketplace] update to work with latest contracts * [marketplace] clean up * [marketplace] align with contract changes - getState / state > requestState - getSlot > getRequestFromSlotId - support MarketplaceConfig - support slotState, remove unneeded Slot type - collateral > config.collateral.initialAmount - remove proofPeriod contract call - Revert reason “Slot empty” > “Slot is free” - getProofEnd > read SlotState Tests for changes * [marketplace] add missing file * [marketplace] bump codex-contracts-eth * [config] remove unused imports * [sales] cleanup * [sales] fix: do not crash when fetching state fails * [sales] make slotIndex non-optional * Rebase and update NBS commit Rebase on top of main and update NBS commit to the CI fix. * [marketplace] use async subscription event handlers * [marketplace] support slotIndex no longer optional Previously, SalesAgent.slotIndex had been moved to not optional. However, there were still many places where optionality was assumed. This commit removes those assumuptions. * [marketplace] sales state machine: use slotState Use `slotState` instead of `requestState` for sales state machine. * [marketplace] clean up * [statemachine] adds a statemachine for async workflows Allows events to be scheduled synchronously. See https://github.com/status-im/nim-codex/pull/344 Co-Authored-By: Ben Bierens <thatbenbierens@gmail.com> Co-Authored-By: Eric Mastro <eric.mastro@gmail.com> * [market] make market callbacks synchronous * [statemachine] export Event * [statemachine] ensure that no errors are raised * [statemachine] add machine parameter to run method * [statemachine] initialize queue on start * [statemachine] check futures before cancelling them * [sales] use new async state machine - states use new run() method and event mechanism - StartState starts subscriptions and loads request * [statemachine] fix unsusbscribe before subscribe * [sales] replace old state transition tests * [sales] separate state machine from sales data * [sales] remove reference from SalesData to Sales * [sales] separate sales context from sales * [sales] move decoupled types into their own modules * [sales] move retrieveRequest to SalesData * [sales] move subscription logic into SalesAgent * [sales] unsubscribe when finished or errored * [build] revert back to released version of nim-ethers * [sales] remove SaleStart state * [sales] add missing base method * [sales] move asyncSpawn helper to utils * [sales] fix imports * [sales] remove unused variables * [sales statemachine] add async state machine error handling (#349) * [statemachine] add error handling to asyncstatemachine - add error handling to catch errors during state.run - Sales: add ErrorState to identify which state to transition to during an error. This had to be added to SalesAgent constructor due to circular dependency issues, otherwise it would have been added directly to SalesAgent. - Sales: when an error during run is encountered, the SaleErrorState is constructed with the error, and by default (base impl) will return the error state, so the machine can transition to it. This can be overridden by individual states if needed. * [sales] rename onSaleFailed to onSaleErrored Because there is already a state named SaleFailed which is meant to react to an onchain RequestFailed event and also because this callback is called from SaleErrored, renaming to onSaleErrored prevents ambiguity and confusion as to what has happened at the callback callsite. * [statemachine] forward error to state directly without going through a machine method first * [statemachine] remove unnecessary error handling AsyncQueueFullError is already handled in schedule() * [statemachine] test that cancellation ignores onError * [sales] simplify error handling in states Rely on the state machine error handling instead of catching errors in the state run method --------- Co-authored-by: Mark Spanbroek <mark@spanbroek.net> * [statemachine] prevent memory leaks prevent memory leaks and nil access defects by: - allowing multiple subscribe/unsubscribes of salesagent - disallowing individual salesagent subscription calls to be made externally (requires the .subscribed check) - allowing mutiple start/stops of asyncstatemachine - disregard asyncstatemachine schedules if machine not yet started * [salesagent] add salesagent-specific tests 1. test multiple subscribe/unsubscribes 2. test scheduling machine without being started 3. test subscriptions are working correctly with external events 4. test errors can be overridden at the state level for ErrorHandlingStates. --------- Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Mark Spanbroek <mark@spanbroek.net> Co-authored-by: Ben Bierens <thatbenbierens@gmail.com>
2023-03-08 13:34:26 +00:00
import pkg/stew/byteutils
2022-05-19 19:56:03 +00:00
import codex/contracts
import ../ethertest
import ./examples
import ./time
ethersuite "On-Chain Market":
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
let proof = exampleProof()
var market: OnChainMarket
var marketplace: Marketplace
2022-03-29 07:47:49 +00:00
var request: StorageRequest
var slotIndex: UInt256
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
var periodicity: Periodicity
setup:
[marketplace] Add Reservations Module (#340) * [marketplace] reservations module - add de/serialization for Availability - add markUsed/markUnused in persisted availability - add query for unused - add reserve/release - reservation module tests - split ContractInteractions into client contracts and host contracts - remove reservations start/stop as the repo start/stop is being managed by the node - remove dedicated reservations metadata store and use the metadata store from the repo instead - Split ContractInteractions into: - ClientInteractions (with purchasing) - HostInteractions (with sales and proving) - compilation fix for nim 1.2 [repostore] fix started flag, add tests [marketplace] persist slot index For loading the sales state from chain, the slot index was not previously persisted in the contract. Will retrieve the slot index from the contract when the sales state is loaded. * Revert repostore changes In favour of separate PR https://github.com/status-im/nim-codex/pull/374. * remove warnings * clean up * tests: stop repostore during teardown * change constructor type identifier Change Contracts constructor to accept Contracts type instead of ContractInteractions. * change constructor return type to Result instead of Option * fix and split interactions tests * clean up, fix tests * find availability by slot id * remove duplication in host/client interactions * add test for finding availability by slotId * log instead of raiseAssert when failed to mark availability as unused * move to SaleErrored state instead of raiseAssert * remove unneeded reverse It appears that order is not preserved in the repostore, so reversing does not have the intended effect here. * update open api spec for potential rest endpoint errors * move functions about available bytes to repostore * WIP: reserve and release availabilities as needed WIP: not tested yet Availabilities are marked as used when matched (just before downloading starts) so that future matching logic does not match an availability currently in use. As the download progresses, batches of blocks are written to disk, and the equivalent bytes are released from the reservation module. The size of the availability is reduced as well. During a reserve or release operation, availability updates occur after the repo is manipulated. If the availability update operation fails, the reserve or release is rolled back to maintain correct accounting of bytes. Finally, once download completes, or if an error occurs, the availability is marked as unused so future matching can occur. * delete availability when all bytes released * fix tests + cleanup * remove availability from SalesContext callbacks Availability is no longer used past the SaleDownloading state in the state machine. Cleanup of Availability (marking unused) is handled directly in the SaleDownloading state, and no longer in SaleErrored or SaleFinished. Likewise, Availabilities shouldn’t need to be handled on node restart. Additionally, Availability was being passed in SalesContext callbacks, and now that Availability is only used temporarily in the SaleDownloading state, Availability is contextually irrelevant to the callbacks, except in OnStore possibly, though it was not being consumed. * test clean up * - remove availability from callbacks and constructors from previous commit that needed to be removed (oopsie) - fix integration test that checks availabilities - there was a bug fixed that crashed the node due to a missing `return success` in onStore - the test was fixed by ensuring that availabilities are remaining on the node, and the size has been reduced - change Availability back to non-ref object and constructor back to init - add trace logging of all state transitions in state machine - add generally useful trace logging * fixes after rebase 1. Fix onProve callbacks 2. Use Slot type instead of tuple for retrieving active slot. 3. Bump codex-contracts-eth that exposes getActivceSlot call. * swap contracts branch to not support slot collateral Slot collateral changes in the contracts require further changes in the client code, so we’ll skip those changes for now and add in a separate commit. * modify Interactions and Deployment constructors - `HostInteractions` and `ClientInteractions` constructors were simplified to take a contract address and no overloads - `Interactions` prepared simplified so there are no overloads - `Deployment` constructor updated so that it takes an optional string parameter, instead `Option[string]` * Move `batchProc` declaration `batchProc` needs to be consumed by both `node` and `salescontext`, and they can’t reference each other as it creates a circular dependency. * [reservations] rename `available` to `hasAvailable` * [reservations] default error message to inner error msg * add SaleIngored state When a storage request is handled but the request does match availabilities, the sales agent machine is sent to the SaleIgnored state. In addition, the agent is constructed in a way that if the request is ignored, the sales agent is removed from the list of active agents being tracked in the sales module.
2023-04-04 07:05:16 +00:00
let deployment = Deployment.init()
marketplace = Marketplace.new(!deployment.address(Marketplace), provider.getSigner())
[marketplace] Load sales state from chain (#306) * [marketplace] get active slots from chain # Conflicts: # codex/contracts/market.nim * [marketplace] make on chain event callbacks async # Conflicts: # tests/codex/helpers/mockmarket.nim * [marketplace] make availability optional for node restart # Conflicts: # tests/codex/testsales.nim * [marketplace] add async state machine Allows for `enterAsync` to be cancelled. * [marketplace] move sale process to async state machine * [marketplace] sales state machine tests * bump dagger-contracts * [marketplace] fix ci issue with chronicles output * PR comments - add slotIndex to `SalesAgent` constructor - remove `SalesAgent.init` - rename `SalesAgent.init` to `start` and `SalesAgent.deinit` to `stop`. - rename `SalesAgent. populateRequest` to `SalesAgent.retreiveRequest`. - move availability removal to the downloading state. once availability is persisted to disk, it should survive node restarts. - * [marketplace] handle slot filled by other host Handle the case when in the downloading, proving, or filling states, that another host fills the slot. * [marketplace] use requestId for mySlots * [marketplace] infer slot index from slotid prevents reassigning a random slot index when restoring state from chain * [marketplace] update to work with latest contracts * [marketplace] clean up * [marketplace] align with contract changes - getState / state > requestState - getSlot > getRequestFromSlotId - support MarketplaceConfig - support slotState, remove unneeded Slot type - collateral > config.collateral.initialAmount - remove proofPeriod contract call - Revert reason “Slot empty” > “Slot is free” - getProofEnd > read SlotState Tests for changes * [marketplace] add missing file * [marketplace] bump codex-contracts-eth * [config] remove unused imports * [sales] cleanup * [sales] fix: do not crash when fetching state fails * [sales] make slotIndex non-optional * Rebase and update NBS commit Rebase on top of main and update NBS commit to the CI fix. * [marketplace] use async subscription event handlers * [marketplace] support slotIndex no longer optional Previously, SalesAgent.slotIndex had been moved to not optional. However, there were still many places where optionality was assumed. This commit removes those assumuptions. * [marketplace] sales state machine: use slotState Use `slotState` instead of `requestState` for sales state machine. * [marketplace] clean up * [statemachine] adds a statemachine for async workflows Allows events to be scheduled synchronously. See https://github.com/status-im/nim-codex/pull/344 Co-Authored-By: Ben Bierens <thatbenbierens@gmail.com> Co-Authored-By: Eric Mastro <eric.mastro@gmail.com> * [market] make market callbacks synchronous * [statemachine] export Event * [statemachine] ensure that no errors are raised * [statemachine] add machine parameter to run method * [statemachine] initialize queue on start * [statemachine] check futures before cancelling them * [sales] use new async state machine - states use new run() method and event mechanism - StartState starts subscriptions and loads request * [statemachine] fix unsusbscribe before subscribe * [sales] replace old state transition tests * [sales] separate state machine from sales data * [sales] remove reference from SalesData to Sales * [sales] separate sales context from sales * [sales] move decoupled types into their own modules * [sales] move retrieveRequest to SalesData * [sales] move subscription logic into SalesAgent * [sales] unsubscribe when finished or errored * [build] revert back to released version of nim-ethers * [sales] remove SaleStart state * [sales] add missing base method * [sales] move asyncSpawn helper to utils * [sales] fix imports * [sales] remove unused variables * [sales statemachine] add async state machine error handling (#349) * [statemachine] add error handling to asyncstatemachine - add error handling to catch errors during state.run - Sales: add ErrorState to identify which state to transition to during an error. This had to be added to SalesAgent constructor due to circular dependency issues, otherwise it would have been added directly to SalesAgent. - Sales: when an error during run is encountered, the SaleErrorState is constructed with the error, and by default (base impl) will return the error state, so the machine can transition to it. This can be overridden by individual states if needed. * [sales] rename onSaleFailed to onSaleErrored Because there is already a state named SaleFailed which is meant to react to an onchain RequestFailed event and also because this callback is called from SaleErrored, renaming to onSaleErrored prevents ambiguity and confusion as to what has happened at the callback callsite. * [statemachine] forward error to state directly without going through a machine method first * [statemachine] remove unnecessary error handling AsyncQueueFullError is already handled in schedule() * [statemachine] test that cancellation ignores onError * [sales] simplify error handling in states Rely on the state machine error handling instead of catching errors in the state run method --------- Co-authored-by: Mark Spanbroek <mark@spanbroek.net> * [statemachine] prevent memory leaks prevent memory leaks and nil access defects by: - allowing multiple subscribe/unsubscribes of salesagent - disallowing individual salesagent subscription calls to be made externally (requires the .subscribed check) - allowing mutiple start/stops of asyncstatemachine - disregard asyncstatemachine schedules if machine not yet started * [salesagent] add salesagent-specific tests 1. test multiple subscribe/unsubscribes 2. test scheduling machine without being started 3. test subscriptions are working correctly with external events 4. test errors can be overridden at the state level for ErrorHandlingStates. --------- Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Mark Spanbroek <mark@spanbroek.net> Co-authored-by: Ben Bierens <thatbenbierens@gmail.com>
2023-03-08 13:34:26 +00:00
let config = await marketplace.config()
2022-03-29 07:47:49 +00:00
market = OnChainMarket.new(marketplace)
[marketplace] Load sales state from chain (#306) * [marketplace] get active slots from chain # Conflicts: # codex/contracts/market.nim * [marketplace] make on chain event callbacks async # Conflicts: # tests/codex/helpers/mockmarket.nim * [marketplace] make availability optional for node restart # Conflicts: # tests/codex/testsales.nim * [marketplace] add async state machine Allows for `enterAsync` to be cancelled. * [marketplace] move sale process to async state machine * [marketplace] sales state machine tests * bump dagger-contracts * [marketplace] fix ci issue with chronicles output * PR comments - add slotIndex to `SalesAgent` constructor - remove `SalesAgent.init` - rename `SalesAgent.init` to `start` and `SalesAgent.deinit` to `stop`. - rename `SalesAgent. populateRequest` to `SalesAgent.retreiveRequest`. - move availability removal to the downloading state. once availability is persisted to disk, it should survive node restarts. - * [marketplace] handle slot filled by other host Handle the case when in the downloading, proving, or filling states, that another host fills the slot. * [marketplace] use requestId for mySlots * [marketplace] infer slot index from slotid prevents reassigning a random slot index when restoring state from chain * [marketplace] update to work with latest contracts * [marketplace] clean up * [marketplace] align with contract changes - getState / state > requestState - getSlot > getRequestFromSlotId - support MarketplaceConfig - support slotState, remove unneeded Slot type - collateral > config.collateral.initialAmount - remove proofPeriod contract call - Revert reason “Slot empty” > “Slot is free” - getProofEnd > read SlotState Tests for changes * [marketplace] add missing file * [marketplace] bump codex-contracts-eth * [config] remove unused imports * [sales] cleanup * [sales] fix: do not crash when fetching state fails * [sales] make slotIndex non-optional * Rebase and update NBS commit Rebase on top of main and update NBS commit to the CI fix. * [marketplace] use async subscription event handlers * [marketplace] support slotIndex no longer optional Previously, SalesAgent.slotIndex had been moved to not optional. However, there were still many places where optionality was assumed. This commit removes those assumuptions. * [marketplace] sales state machine: use slotState Use `slotState` instead of `requestState` for sales state machine. * [marketplace] clean up * [statemachine] adds a statemachine for async workflows Allows events to be scheduled synchronously. See https://github.com/status-im/nim-codex/pull/344 Co-Authored-By: Ben Bierens <thatbenbierens@gmail.com> Co-Authored-By: Eric Mastro <eric.mastro@gmail.com> * [market] make market callbacks synchronous * [statemachine] export Event * [statemachine] ensure that no errors are raised * [statemachine] add machine parameter to run method * [statemachine] initialize queue on start * [statemachine] check futures before cancelling them * [sales] use new async state machine - states use new run() method and event mechanism - StartState starts subscriptions and loads request * [statemachine] fix unsusbscribe before subscribe * [sales] replace old state transition tests * [sales] separate state machine from sales data * [sales] remove reference from SalesData to Sales * [sales] separate sales context from sales * [sales] move decoupled types into their own modules * [sales] move retrieveRequest to SalesData * [sales] move subscription logic into SalesAgent * [sales] unsubscribe when finished or errored * [build] revert back to released version of nim-ethers * [sales] remove SaleStart state * [sales] add missing base method * [sales] move asyncSpawn helper to utils * [sales] fix imports * [sales] remove unused variables * [sales statemachine] add async state machine error handling (#349) * [statemachine] add error handling to asyncstatemachine - add error handling to catch errors during state.run - Sales: add ErrorState to identify which state to transition to during an error. This had to be added to SalesAgent constructor due to circular dependency issues, otherwise it would have been added directly to SalesAgent. - Sales: when an error during run is encountered, the SaleErrorState is constructed with the error, and by default (base impl) will return the error state, so the machine can transition to it. This can be overridden by individual states if needed. * [sales] rename onSaleFailed to onSaleErrored Because there is already a state named SaleFailed which is meant to react to an onchain RequestFailed event and also because this callback is called from SaleErrored, renaming to onSaleErrored prevents ambiguity and confusion as to what has happened at the callback callsite. * [statemachine] forward error to state directly without going through a machine method first * [statemachine] remove unnecessary error handling AsyncQueueFullError is already handled in schedule() * [statemachine] test that cancellation ignores onError * [sales] simplify error handling in states Rely on the state machine error handling instead of catching errors in the state run method --------- Co-authored-by: Mark Spanbroek <mark@spanbroek.net> * [statemachine] prevent memory leaks prevent memory leaks and nil access defects by: - allowing multiple subscribe/unsubscribes of salesagent - disallowing individual salesagent subscription calls to be made externally (requires the .subscribed check) - allowing mutiple start/stops of asyncstatemachine - disregard asyncstatemachine schedules if machine not yet started * [salesagent] add salesagent-specific tests 1. test multiple subscribe/unsubscribes 2. test scheduling machine without being started 3. test subscriptions are working correctly with external events 4. test errors can be overridden at the state level for ErrorHandlingStates. --------- Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Mark Spanbroek <mark@spanbroek.net> Co-authored-by: Ben Bierens <thatbenbierens@gmail.com>
2023-03-08 13:34:26 +00:00
periodicity = Periodicity(seconds: config.proofs.period)
2022-03-29 07:47:49 +00:00
request = StorageRequest.example
request.client = accounts[0]
slotIndex = (request.ask.slots div 2).u256
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
proc advanceToNextPeriod() {.async.} =
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
let currentPeriod = periodicity.periodOf(await provider.currentTime())
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
await provider.advanceTimeTo(periodicity.periodEnd(currentPeriod) + 1)
proc waitUntilProofRequired(slotId: SlotId) {.async.} =
await advanceToNextPeriod()
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
while not (
(await marketplace.isProofRequired(slotId)) and
(await marketplace.getPointer(slotId)) < 250
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
):
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
await advanceToNextPeriod()
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
test "fails to instantiate when contract does not have a signer":
let storageWithoutSigner = marketplace.connect(provider)
expect AssertionDefect:
discard OnChainMarket.new(storageWithoutSigner)
2022-07-04 12:14:56 +00:00
test "knows signer address":
check (await market.getSigner()) == (await provider.getSigner().getAddress())
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
test "can retrieve proof periodicity":
let periodicity = await market.periodicity()
let config = await marketplace.config()
let periodLength = config.proofs.period
check periodicity.seconds == periodLength
test "can retrieve proof timeout":
let proofTimeout = await market.proofTimeout()
let config = await marketplace.config()
check proofTimeout == config.proofs.timeout
test "supports marketplace requests":
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
await market.requestStorage(request)
2022-03-29 07:47:49 +00:00
2022-06-15 12:12:34 +00:00
test "can retrieve previously submitted requests":
check (await market.getRequest(request.id)) == none StorageRequest
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
await market.requestStorage(request)
let r = await market.getRequest(request.id)
check (r) == some request
test "supports withdrawing of funds":
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
await market.requestStorage(request)
await provider.advanceTimeTo(request.expiry)
await market.withdrawFunds(request.id)
2022-06-15 12:12:34 +00:00
test "supports request subscriptions":
var receivedIds: seq[RequestId]
var receivedAsks: seq[StorageAsk]
proc onRequest(id: RequestId, ask: StorageAsk) =
receivedIds.add(id)
receivedAsks.add(ask)
2022-03-29 07:47:49 +00:00
let subscription = await market.subscribeRequests(onRequest)
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
await market.requestStorage(request)
check receivedIds == @[request.id]
check receivedAsks == @[request.ask]
await subscription.unsubscribe()
2022-03-29 07:47:49 +00:00
test "supports filling of slots":
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
await market.requestStorage(request)
await market.fillSlot(request.id, slotIndex, proof, request.ask.collateral)
2022-03-29 07:47:49 +00:00
test "can retrieve host that filled slot":
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
await market.requestStorage(request)
check (await market.getHost(request.id, slotIndex)) == none Address
await market.fillSlot(request.id, slotIndex, proof, request.ask.collateral)
check (await market.getHost(request.id, slotIndex)) == some accounts[0]
2022-07-04 10:11:18 +00:00
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
test "supports freeing a slot":
await market.requestStorage(request)
await market.fillSlot(request.id, slotIndex, proof, request.ask.collateral)
await market.freeSlot(slotId(request.id, slotIndex))
check (await market.getHost(request.id, slotIndex)) == none Address
test "supports checking whether proof is required now":
check (await market.isProofRequired(slotId(request.id, slotIndex))) == false
test "supports checking whether proof is required soon":
check (await market.willProofBeRequired(slotId(request.id, slotIndex))) == false
test "submits proofs":
await market.submitProof(slotId(request.id, slotIndex), proof)
test "marks a proof as missing":
let slotId = slotId(request, slotIndex)
await market.requestStorage(request)
await market.fillSlot(request.id, slotIndex, proof, request.ask.collateral)
await waitUntilProofRequired(slotId)
let missingPeriod = periodicity.periodOf(await provider.currentTime())
await advanceToNextPeriod()
await market.markProofAsMissing(slotId, missingPeriod)
check (await marketplace.missingProofs(slotId)) == 1
test "can check whether a proof can be marked as missing":
let slotId = slotId(request, slotIndex)
await market.requestStorage(request)
await market.fillSlot(request.id, slotIndex, proof, request.ask.collateral)
await waitUntilProofRequired(slotId)
let missingPeriod = periodicity.periodOf(await provider.currentTime())
await advanceToNextPeriod()
check (await market.canProofBeMarkedAsMissing(slotId, missingPeriod)) == true
test "supports slot filled subscriptions":
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
await market.requestStorage(request)
var receivedIds: seq[RequestId]
var receivedSlotIndices: seq[UInt256]
proc onSlotFilled(id: RequestId, slotIndex: UInt256) =
receivedIds.add(id)
receivedSlotIndices.add(slotIndex)
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
let subscription = await market.subscribeSlotFilled(onSlotFilled)
await market.fillSlot(request.id, slotIndex, proof, request.ask.collateral)
check receivedIds == @[request.id]
check receivedSlotIndices == @[slotIndex]
await subscription.unsubscribe()
test "subscribes only to a certain slot":
var otherSlot = slotIndex - 1
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
await market.requestStorage(request)
var receivedSlotIndices: seq[UInt256]
proc onSlotFilled(requestId: RequestId, slotIndex: UInt256) =
receivedSlotIndices.add(slotIndex)
let subscription = await market.subscribeSlotFilled(request.id, slotIndex, onSlotFilled)
await market.fillSlot(request.id, otherSlot, proof, request.ask.collateral)
check receivedSlotIndices.len == 0
await market.fillSlot(request.id, slotIndex, proof, request.ask.collateral)
check receivedSlotIndices == @[slotIndex]
await subscription.unsubscribe()
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
test "supports slot freed subscriptions":
await market.requestStorage(request)
await market.fillSlot(request.id, slotIndex, proof, request.ask.collateral)
var receivedIds: seq[SlotId]
proc onSlotFreed(id: SlotId) =
receivedIds.add(id)
let subscription = await market.subscribeSlotFreed(onSlotFreed)
await market.freeSlot(slotId(request.id, slotIndex))
check receivedIds == @[slotId(request.id, slotIndex)]
await subscription.unsubscribe()
test "support fulfillment subscriptions":
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
await market.requestStorage(request)
var receivedIds: seq[RequestId]
proc onFulfillment(id: RequestId) =
receivedIds.add(id)
let subscription = await market.subscribeFulfillment(request.id, onFulfillment)
for slotIndex in 0..<request.ask.slots:
await market.fillSlot(request.id, slotIndex.u256, proof, request.ask.collateral)
check receivedIds == @[request.id]
2022-03-29 09:20:07 +00:00
await subscription.unsubscribe()
test "subscribes only to fulfillment of a certain request":
2022-03-29 09:20:07 +00:00
var otherRequest = StorageRequest.example
otherRequest.client = accounts[0]
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
await market.requestStorage(request)
await market.requestStorage(otherRequest)
2022-03-29 09:20:07 +00:00
var receivedIds: seq[RequestId]
proc onFulfillment(id: RequestId) =
receivedIds.add(id)
2022-03-29 09:20:07 +00:00
let subscription = await market.subscribeFulfillment(request.id, onFulfillment)
2022-03-29 09:20:07 +00:00
for slotIndex in 0..<request.ask.slots:
await market.fillSlot(request.id, slotIndex.u256, proof, request.ask.collateral)
for slotIndex in 0..<otherRequest.ask.slots:
await market.fillSlot(otherRequest.id, slotIndex.u256, proof, otherRequest.ask.collateral)
2022-03-29 09:20:07 +00:00
check receivedIds == @[request.id]
2022-03-29 09:20:07 +00:00
await subscription.unsubscribe()
test "support request cancelled subscriptions":
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
await market.requestStorage(request)
var receivedIds: seq[RequestId]
proc onRequestCancelled(id: RequestId) =
receivedIds.add(id)
let subscription = await market.subscribeRequestCancelled(request.id, onRequestCancelled)
await provider.advanceTimeTo(request.expiry)
await market.withdrawFunds(request.id)
check receivedIds == @[request.id]
await subscription.unsubscribe()
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
test "support request failed subscriptions":
await market.requestStorage(request)
var receivedIds: seq[RequestId]
proc onRequestFailed(id: RequestId) =
receivedIds.add(id)
let subscription = await market.subscribeRequestFailed(request.id, onRequestFailed)
for slotIndex in 0..<request.ask.slots:
await market.fillSlot(request.id, slotIndex.u256, proof, request.ask.collateral)
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
for slotIndex in 0..request.ask.maxSlotLoss:
let slotId = request.slotId(slotIndex.u256)
while true:
[marketplace] Load sales state from chain (#306) * [marketplace] get active slots from chain # Conflicts: # codex/contracts/market.nim * [marketplace] make on chain event callbacks async # Conflicts: # tests/codex/helpers/mockmarket.nim * [marketplace] make availability optional for node restart # Conflicts: # tests/codex/testsales.nim * [marketplace] add async state machine Allows for `enterAsync` to be cancelled. * [marketplace] move sale process to async state machine * [marketplace] sales state machine tests * bump dagger-contracts * [marketplace] fix ci issue with chronicles output * PR comments - add slotIndex to `SalesAgent` constructor - remove `SalesAgent.init` - rename `SalesAgent.init` to `start` and `SalesAgent.deinit` to `stop`. - rename `SalesAgent. populateRequest` to `SalesAgent.retreiveRequest`. - move availability removal to the downloading state. once availability is persisted to disk, it should survive node restarts. - * [marketplace] handle slot filled by other host Handle the case when in the downloading, proving, or filling states, that another host fills the slot. * [marketplace] use requestId for mySlots * [marketplace] infer slot index from slotid prevents reassigning a random slot index when restoring state from chain * [marketplace] update to work with latest contracts * [marketplace] clean up * [marketplace] align with contract changes - getState / state > requestState - getSlot > getRequestFromSlotId - support MarketplaceConfig - support slotState, remove unneeded Slot type - collateral > config.collateral.initialAmount - remove proofPeriod contract call - Revert reason “Slot empty” > “Slot is free” - getProofEnd > read SlotState Tests for changes * [marketplace] add missing file * [marketplace] bump codex-contracts-eth * [config] remove unused imports * [sales] cleanup * [sales] fix: do not crash when fetching state fails * [sales] make slotIndex non-optional * Rebase and update NBS commit Rebase on top of main and update NBS commit to the CI fix. * [marketplace] use async subscription event handlers * [marketplace] support slotIndex no longer optional Previously, SalesAgent.slotIndex had been moved to not optional. However, there were still many places where optionality was assumed. This commit removes those assumuptions. * [marketplace] sales state machine: use slotState Use `slotState` instead of `requestState` for sales state machine. * [marketplace] clean up * [statemachine] adds a statemachine for async workflows Allows events to be scheduled synchronously. See https://github.com/status-im/nim-codex/pull/344 Co-Authored-By: Ben Bierens <thatbenbierens@gmail.com> Co-Authored-By: Eric Mastro <eric.mastro@gmail.com> * [market] make market callbacks synchronous * [statemachine] export Event * [statemachine] ensure that no errors are raised * [statemachine] add machine parameter to run method * [statemachine] initialize queue on start * [statemachine] check futures before cancelling them * [sales] use new async state machine - states use new run() method and event mechanism - StartState starts subscriptions and loads request * [statemachine] fix unsusbscribe before subscribe * [sales] replace old state transition tests * [sales] separate state machine from sales data * [sales] remove reference from SalesData to Sales * [sales] separate sales context from sales * [sales] move decoupled types into their own modules * [sales] move retrieveRequest to SalesData * [sales] move subscription logic into SalesAgent * [sales] unsubscribe when finished or errored * [build] revert back to released version of nim-ethers * [sales] remove SaleStart state * [sales] add missing base method * [sales] move asyncSpawn helper to utils * [sales] fix imports * [sales] remove unused variables * [sales statemachine] add async state machine error handling (#349) * [statemachine] add error handling to asyncstatemachine - add error handling to catch errors during state.run - Sales: add ErrorState to identify which state to transition to during an error. This had to be added to SalesAgent constructor due to circular dependency issues, otherwise it would have been added directly to SalesAgent. - Sales: when an error during run is encountered, the SaleErrorState is constructed with the error, and by default (base impl) will return the error state, so the machine can transition to it. This can be overridden by individual states if needed. * [sales] rename onSaleFailed to onSaleErrored Because there is already a state named SaleFailed which is meant to react to an onchain RequestFailed event and also because this callback is called from SaleErrored, renaming to onSaleErrored prevents ambiguity and confusion as to what has happened at the callback callsite. * [statemachine] forward error to state directly without going through a machine method first * [statemachine] remove unnecessary error handling AsyncQueueFullError is already handled in schedule() * [statemachine] test that cancellation ignores onError * [sales] simplify error handling in states Rely on the state machine error handling instead of catching errors in the state run method --------- Co-authored-by: Mark Spanbroek <mark@spanbroek.net> * [statemachine] prevent memory leaks prevent memory leaks and nil access defects by: - allowing multiple subscribe/unsubscribes of salesagent - disallowing individual salesagent subscription calls to be made externally (requires the .subscribed check) - allowing mutiple start/stops of asyncstatemachine - disregard asyncstatemachine schedules if machine not yet started * [salesagent] add salesagent-specific tests 1. test multiple subscribe/unsubscribes 2. test scheduling machine without being started 3. test subscriptions are working correctly with external events 4. test errors can be overridden at the state level for ErrorHandlingStates. --------- Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Mark Spanbroek <mark@spanbroek.net> Co-authored-by: Ben Bierens <thatbenbierens@gmail.com>
2023-03-08 13:34:26 +00:00
let slotState = await marketplace.slotState(slotId)
if slotState == SlotState.Free:
break
await waitUntilProofRequired(slotId)
let missingPeriod = periodicity.periodOf(await provider.currentTime())
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
await advanceToNextPeriod()
[marketplace] Load sales state from chain (#306) * [marketplace] get active slots from chain # Conflicts: # codex/contracts/market.nim * [marketplace] make on chain event callbacks async # Conflicts: # tests/codex/helpers/mockmarket.nim * [marketplace] make availability optional for node restart # Conflicts: # tests/codex/testsales.nim * [marketplace] add async state machine Allows for `enterAsync` to be cancelled. * [marketplace] move sale process to async state machine * [marketplace] sales state machine tests * bump dagger-contracts * [marketplace] fix ci issue with chronicles output * PR comments - add slotIndex to `SalesAgent` constructor - remove `SalesAgent.init` - rename `SalesAgent.init` to `start` and `SalesAgent.deinit` to `stop`. - rename `SalesAgent. populateRequest` to `SalesAgent.retreiveRequest`. - move availability removal to the downloading state. once availability is persisted to disk, it should survive node restarts. - * [marketplace] handle slot filled by other host Handle the case when in the downloading, proving, or filling states, that another host fills the slot. * [marketplace] use requestId for mySlots * [marketplace] infer slot index from slotid prevents reassigning a random slot index when restoring state from chain * [marketplace] update to work with latest contracts * [marketplace] clean up * [marketplace] align with contract changes - getState / state > requestState - getSlot > getRequestFromSlotId - support MarketplaceConfig - support slotState, remove unneeded Slot type - collateral > config.collateral.initialAmount - remove proofPeriod contract call - Revert reason “Slot empty” > “Slot is free” - getProofEnd > read SlotState Tests for changes * [marketplace] add missing file * [marketplace] bump codex-contracts-eth * [config] remove unused imports * [sales] cleanup * [sales] fix: do not crash when fetching state fails * [sales] make slotIndex non-optional * Rebase and update NBS commit Rebase on top of main and update NBS commit to the CI fix. * [marketplace] use async subscription event handlers * [marketplace] support slotIndex no longer optional Previously, SalesAgent.slotIndex had been moved to not optional. However, there were still many places where optionality was assumed. This commit removes those assumuptions. * [marketplace] sales state machine: use slotState Use `slotState` instead of `requestState` for sales state machine. * [marketplace] clean up * [statemachine] adds a statemachine for async workflows Allows events to be scheduled synchronously. See https://github.com/status-im/nim-codex/pull/344 Co-Authored-By: Ben Bierens <thatbenbierens@gmail.com> Co-Authored-By: Eric Mastro <eric.mastro@gmail.com> * [market] make market callbacks synchronous * [statemachine] export Event * [statemachine] ensure that no errors are raised * [statemachine] add machine parameter to run method * [statemachine] initialize queue on start * [statemachine] check futures before cancelling them * [sales] use new async state machine - states use new run() method and event mechanism - StartState starts subscriptions and loads request * [statemachine] fix unsusbscribe before subscribe * [sales] replace old state transition tests * [sales] separate state machine from sales data * [sales] remove reference from SalesData to Sales * [sales] separate sales context from sales * [sales] move decoupled types into their own modules * [sales] move retrieveRequest to SalesData * [sales] move subscription logic into SalesAgent * [sales] unsubscribe when finished or errored * [build] revert back to released version of nim-ethers * [sales] remove SaleStart state * [sales] add missing base method * [sales] move asyncSpawn helper to utils * [sales] fix imports * [sales] remove unused variables * [sales statemachine] add async state machine error handling (#349) * [statemachine] add error handling to asyncstatemachine - add error handling to catch errors during state.run - Sales: add ErrorState to identify which state to transition to during an error. This had to be added to SalesAgent constructor due to circular dependency issues, otherwise it would have been added directly to SalesAgent. - Sales: when an error during run is encountered, the SaleErrorState is constructed with the error, and by default (base impl) will return the error state, so the machine can transition to it. This can be overridden by individual states if needed. * [sales] rename onSaleFailed to onSaleErrored Because there is already a state named SaleFailed which is meant to react to an onchain RequestFailed event and also because this callback is called from SaleErrored, renaming to onSaleErrored prevents ambiguity and confusion as to what has happened at the callback callsite. * [statemachine] forward error to state directly without going through a machine method first * [statemachine] remove unnecessary error handling AsyncQueueFullError is already handled in schedule() * [statemachine] test that cancellation ignores onError * [sales] simplify error handling in states Rely on the state machine error handling instead of catching errors in the state run method --------- Co-authored-by: Mark Spanbroek <mark@spanbroek.net> * [statemachine] prevent memory leaks prevent memory leaks and nil access defects by: - allowing multiple subscribe/unsubscribes of salesagent - disallowing individual salesagent subscription calls to be made externally (requires the .subscribed check) - allowing mutiple start/stops of asyncstatemachine - disregard asyncstatemachine schedules if machine not yet started * [salesagent] add salesagent-specific tests 1. test multiple subscribe/unsubscribes 2. test scheduling machine without being started 3. test subscriptions are working correctly with external events 4. test errors can be overridden at the state level for ErrorHandlingStates. --------- Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Mark Spanbroek <mark@spanbroek.net> Co-authored-by: Ben Bierens <thatbenbierens@gmail.com>
2023-03-08 13:34:26 +00:00
await marketplace.markProofAsMissing(slotId, missingPeriod)
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
check receivedIds == @[request.id]
await subscription.unsubscribe()
test "subscribes only to a certain request cancellation":
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
var otherRequest = request
otherRequest.nonce = Nonce.example
await market.requestStorage(request)
await market.requestStorage(otherRequest)
var receivedIds: seq[RequestId]
proc onRequestCancelled(requestId: RequestId) =
receivedIds.add(requestId)
let subscription = await market.subscribeRequestCancelled(request.id, onRequestCancelled)
await provider.advanceTimeTo(request.expiry) # shares expiry with otherRequest
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
await market.withdrawFunds(otherRequest.id)
check receivedIds.len == 0
await market.withdrawFunds(request.id)
check receivedIds == @[request.id]
await subscription.unsubscribe()
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
test "supports proof submission subscriptions":
var receivedIds: seq[SlotId]
var receivedProofs: seq[seq[byte]]
proc onProofSubmission(id: SlotId, proof: seq[byte]) =
receivedIds.add(id)
receivedProofs.add(proof)
let subscription = await market.subscribeProofSubmission(onProofSubmission)
await market.submitProof(slotId(request.id, slotIndex), proof)
check receivedIds == @[slotId(request.id, slotIndex)]
check receivedProofs == @[proof]
await subscription.unsubscribe()
test "request is none when unknown":
check isNone await market.getRequest(request.id)
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
test "can retrieve active requests":
await market.requestStorage(request)
var request2 = StorageRequest.example
request2.client = accounts[0]
await market.requestStorage(request2)
check (await market.myRequests()) == @[request.id, request2.id]
[marketplace] Load sales state from chain (#306) * [marketplace] get active slots from chain # Conflicts: # codex/contracts/market.nim * [marketplace] make on chain event callbacks async # Conflicts: # tests/codex/helpers/mockmarket.nim * [marketplace] make availability optional for node restart # Conflicts: # tests/codex/testsales.nim * [marketplace] add async state machine Allows for `enterAsync` to be cancelled. * [marketplace] move sale process to async state machine * [marketplace] sales state machine tests * bump dagger-contracts * [marketplace] fix ci issue with chronicles output * PR comments - add slotIndex to `SalesAgent` constructor - remove `SalesAgent.init` - rename `SalesAgent.init` to `start` and `SalesAgent.deinit` to `stop`. - rename `SalesAgent. populateRequest` to `SalesAgent.retreiveRequest`. - move availability removal to the downloading state. once availability is persisted to disk, it should survive node restarts. - * [marketplace] handle slot filled by other host Handle the case when in the downloading, proving, or filling states, that another host fills the slot. * [marketplace] use requestId for mySlots * [marketplace] infer slot index from slotid prevents reassigning a random slot index when restoring state from chain * [marketplace] update to work with latest contracts * [marketplace] clean up * [marketplace] align with contract changes - getState / state > requestState - getSlot > getRequestFromSlotId - support MarketplaceConfig - support slotState, remove unneeded Slot type - collateral > config.collateral.initialAmount - remove proofPeriod contract call - Revert reason “Slot empty” > “Slot is free” - getProofEnd > read SlotState Tests for changes * [marketplace] add missing file * [marketplace] bump codex-contracts-eth * [config] remove unused imports * [sales] cleanup * [sales] fix: do not crash when fetching state fails * [sales] make slotIndex non-optional * Rebase and update NBS commit Rebase on top of main and update NBS commit to the CI fix. * [marketplace] use async subscription event handlers * [marketplace] support slotIndex no longer optional Previously, SalesAgent.slotIndex had been moved to not optional. However, there were still many places where optionality was assumed. This commit removes those assumuptions. * [marketplace] sales state machine: use slotState Use `slotState` instead of `requestState` for sales state machine. * [marketplace] clean up * [statemachine] adds a statemachine for async workflows Allows events to be scheduled synchronously. See https://github.com/status-im/nim-codex/pull/344 Co-Authored-By: Ben Bierens <thatbenbierens@gmail.com> Co-Authored-By: Eric Mastro <eric.mastro@gmail.com> * [market] make market callbacks synchronous * [statemachine] export Event * [statemachine] ensure that no errors are raised * [statemachine] add machine parameter to run method * [statemachine] initialize queue on start * [statemachine] check futures before cancelling them * [sales] use new async state machine - states use new run() method and event mechanism - StartState starts subscriptions and loads request * [statemachine] fix unsusbscribe before subscribe * [sales] replace old state transition tests * [sales] separate state machine from sales data * [sales] remove reference from SalesData to Sales * [sales] separate sales context from sales * [sales] move decoupled types into their own modules * [sales] move retrieveRequest to SalesData * [sales] move subscription logic into SalesAgent * [sales] unsubscribe when finished or errored * [build] revert back to released version of nim-ethers * [sales] remove SaleStart state * [sales] add missing base method * [sales] move asyncSpawn helper to utils * [sales] fix imports * [sales] remove unused variables * [sales statemachine] add async state machine error handling (#349) * [statemachine] add error handling to asyncstatemachine - add error handling to catch errors during state.run - Sales: add ErrorState to identify which state to transition to during an error. This had to be added to SalesAgent constructor due to circular dependency issues, otherwise it would have been added directly to SalesAgent. - Sales: when an error during run is encountered, the SaleErrorState is constructed with the error, and by default (base impl) will return the error state, so the machine can transition to it. This can be overridden by individual states if needed. * [sales] rename onSaleFailed to onSaleErrored Because there is already a state named SaleFailed which is meant to react to an onchain RequestFailed event and also because this callback is called from SaleErrored, renaming to onSaleErrored prevents ambiguity and confusion as to what has happened at the callback callsite. * [statemachine] forward error to state directly without going through a machine method first * [statemachine] remove unnecessary error handling AsyncQueueFullError is already handled in schedule() * [statemachine] test that cancellation ignores onError * [sales] simplify error handling in states Rely on the state machine error handling instead of catching errors in the state run method --------- Co-authored-by: Mark Spanbroek <mark@spanbroek.net> * [statemachine] prevent memory leaks prevent memory leaks and nil access defects by: - allowing multiple subscribe/unsubscribes of salesagent - disallowing individual salesagent subscription calls to be made externally (requires the .subscribed check) - allowing mutiple start/stops of asyncstatemachine - disregard asyncstatemachine schedules if machine not yet started * [salesagent] add salesagent-specific tests 1. test multiple subscribe/unsubscribes 2. test scheduling machine without being started 3. test subscriptions are working correctly with external events 4. test errors can be overridden at the state level for ErrorHandlingStates. --------- Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Mark Spanbroek <mark@spanbroek.net> Co-authored-by: Ben Bierens <thatbenbierens@gmail.com>
2023-03-08 13:34:26 +00:00
test "retrieves correct request state when request is unknown":
check (await market.requestState(request.id)) == none RequestState
Load purchase state from chain (#283) * [purchasing] Simplify test * [utils] Move StorageRequest.example up one level * [purchasing] Load purchases from market * [purchasing] load purchase states * Implement myRequest() and getState() methods for OnChainMarket * [proofs] Fix intermittently failing tests Ensures that examples of proofs in tests are never of length 0; these are considered invalid proofs by the smart contract logic. * [contracts] Fix failing test With the new solidity contracts update, a contract can only be paid out after it started. * [market] Add method to get request end time * [purchasing] wait until purchase is finished Purchase.wait() would previously wait until purchase was started, now we wait until it is finished. * [purchasing] Handle 'finished' and 'failed' states * [marketplace] move to failed state once request fails - Add support for subscribing to request failure events. - Add supporting contract tests for subscribing to request failure events. - Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted - Add supporting tests for moving from PurchaseStarted to PurchaseFailure - Add state transition tests for PurchaseUnknown. * [marketplace] Fix test with longer sleepAsync * [integration] Add function to restart a codex node * [purchasing] Set client address before requesting storage To prevent the purchase id (which equals the request id) from changing once it's been submitted. * [contracts] Fix: OnChainMarket.getState() Had the wrong method signature before * [purchasing] Load purchases on node start * [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored' Allows for an exception type called 'PurchaseError' * [purchasing] Load purchases in background No longer calls market.getRequest() for every purchase on node start. * [contracts] Add `$` for RequestId, SlotId and Nonce To aid with debugging * [purchasing] Add Purchasing.stop() To ensure that all contract interactions have both a start() and a stop() for * [tests] Remove sleepAsync where possible Use `eventually` loop instead, to make sure that we're not waiting unnecessarily. * [integration] Fix: handle non-json response in test * [purchasing] Add purchase state to json * [integration] Ensure that purchase is submitted before restart Fixes test failure on slower CI * [purchasing] re-implement `description` as method Allows description to be set in the same module where the state type is defined. Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [contracts] fix typo Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [market] Use more generic error type Should we decide to change the provider type later Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 07:10:17 +00:00
test "can retrieve request state":
await market.requestStorage(request)
for slotIndex in 0..<request.ask.slots:
await market.fillSlot(request.id, slotIndex.u256, proof, request.ask.collateral)
[marketplace] Load sales state from chain (#306) * [marketplace] get active slots from chain # Conflicts: # codex/contracts/market.nim * [marketplace] make on chain event callbacks async # Conflicts: # tests/codex/helpers/mockmarket.nim * [marketplace] make availability optional for node restart # Conflicts: # tests/codex/testsales.nim * [marketplace] add async state machine Allows for `enterAsync` to be cancelled. * [marketplace] move sale process to async state machine * [marketplace] sales state machine tests * bump dagger-contracts * [marketplace] fix ci issue with chronicles output * PR comments - add slotIndex to `SalesAgent` constructor - remove `SalesAgent.init` - rename `SalesAgent.init` to `start` and `SalesAgent.deinit` to `stop`. - rename `SalesAgent. populateRequest` to `SalesAgent.retreiveRequest`. - move availability removal to the downloading state. once availability is persisted to disk, it should survive node restarts. - * [marketplace] handle slot filled by other host Handle the case when in the downloading, proving, or filling states, that another host fills the slot. * [marketplace] use requestId for mySlots * [marketplace] infer slot index from slotid prevents reassigning a random slot index when restoring state from chain * [marketplace] update to work with latest contracts * [marketplace] clean up * [marketplace] align with contract changes - getState / state > requestState - getSlot > getRequestFromSlotId - support MarketplaceConfig - support slotState, remove unneeded Slot type - collateral > config.collateral.initialAmount - remove proofPeriod contract call - Revert reason “Slot empty” > “Slot is free” - getProofEnd > read SlotState Tests for changes * [marketplace] add missing file * [marketplace] bump codex-contracts-eth * [config] remove unused imports * [sales] cleanup * [sales] fix: do not crash when fetching state fails * [sales] make slotIndex non-optional * Rebase and update NBS commit Rebase on top of main and update NBS commit to the CI fix. * [marketplace] use async subscription event handlers * [marketplace] support slotIndex no longer optional Previously, SalesAgent.slotIndex had been moved to not optional. However, there were still many places where optionality was assumed. This commit removes those assumuptions. * [marketplace] sales state machine: use slotState Use `slotState` instead of `requestState` for sales state machine. * [marketplace] clean up * [statemachine] adds a statemachine for async workflows Allows events to be scheduled synchronously. See https://github.com/status-im/nim-codex/pull/344 Co-Authored-By: Ben Bierens <thatbenbierens@gmail.com> Co-Authored-By: Eric Mastro <eric.mastro@gmail.com> * [market] make market callbacks synchronous * [statemachine] export Event * [statemachine] ensure that no errors are raised * [statemachine] add machine parameter to run method * [statemachine] initialize queue on start * [statemachine] check futures before cancelling them * [sales] use new async state machine - states use new run() method and event mechanism - StartState starts subscriptions and loads request * [statemachine] fix unsusbscribe before subscribe * [sales] replace old state transition tests * [sales] separate state machine from sales data * [sales] remove reference from SalesData to Sales * [sales] separate sales context from sales * [sales] move decoupled types into their own modules * [sales] move retrieveRequest to SalesData * [sales] move subscription logic into SalesAgent * [sales] unsubscribe when finished or errored * [build] revert back to released version of nim-ethers * [sales] remove SaleStart state * [sales] add missing base method * [sales] move asyncSpawn helper to utils * [sales] fix imports * [sales] remove unused variables * [sales statemachine] add async state machine error handling (#349) * [statemachine] add error handling to asyncstatemachine - add error handling to catch errors during state.run - Sales: add ErrorState to identify which state to transition to during an error. This had to be added to SalesAgent constructor due to circular dependency issues, otherwise it would have been added directly to SalesAgent. - Sales: when an error during run is encountered, the SaleErrorState is constructed with the error, and by default (base impl) will return the error state, so the machine can transition to it. This can be overridden by individual states if needed. * [sales] rename onSaleFailed to onSaleErrored Because there is already a state named SaleFailed which is meant to react to an onchain RequestFailed event and also because this callback is called from SaleErrored, renaming to onSaleErrored prevents ambiguity and confusion as to what has happened at the callback callsite. * [statemachine] forward error to state directly without going through a machine method first * [statemachine] remove unnecessary error handling AsyncQueueFullError is already handled in schedule() * [statemachine] test that cancellation ignores onError * [sales] simplify error handling in states Rely on the state machine error handling instead of catching errors in the state run method --------- Co-authored-by: Mark Spanbroek <mark@spanbroek.net> * [statemachine] prevent memory leaks prevent memory leaks and nil access defects by: - allowing multiple subscribe/unsubscribes of salesagent - disallowing individual salesagent subscription calls to be made externally (requires the .subscribed check) - allowing mutiple start/stops of asyncstatemachine - disregard asyncstatemachine schedules if machine not yet started * [salesagent] add salesagent-specific tests 1. test multiple subscribe/unsubscribes 2. test scheduling machine without being started 3. test subscriptions are working correctly with external events 4. test errors can be overridden at the state level for ErrorHandlingStates. --------- Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Mark Spanbroek <mark@spanbroek.net> Co-authored-by: Ben Bierens <thatbenbierens@gmail.com>
2023-03-08 13:34:26 +00:00
check (await market.requestState(request.id)) == some RequestState.Started
test "can retrieve active slots":
await market.requestStorage(request)
await market.fillSlot(request.id, slotIndex - 1, proof, request.ask.collateral)
await market.fillSlot(request.id, slotIndex, proof, request.ask.collateral)
[marketplace] Load sales state from chain (#306) * [marketplace] get active slots from chain # Conflicts: # codex/contracts/market.nim * [marketplace] make on chain event callbacks async # Conflicts: # tests/codex/helpers/mockmarket.nim * [marketplace] make availability optional for node restart # Conflicts: # tests/codex/testsales.nim * [marketplace] add async state machine Allows for `enterAsync` to be cancelled. * [marketplace] move sale process to async state machine * [marketplace] sales state machine tests * bump dagger-contracts * [marketplace] fix ci issue with chronicles output * PR comments - add slotIndex to `SalesAgent` constructor - remove `SalesAgent.init` - rename `SalesAgent.init` to `start` and `SalesAgent.deinit` to `stop`. - rename `SalesAgent. populateRequest` to `SalesAgent.retreiveRequest`. - move availability removal to the downloading state. once availability is persisted to disk, it should survive node restarts. - * [marketplace] handle slot filled by other host Handle the case when in the downloading, proving, or filling states, that another host fills the slot. * [marketplace] use requestId for mySlots * [marketplace] infer slot index from slotid prevents reassigning a random slot index when restoring state from chain * [marketplace] update to work with latest contracts * [marketplace] clean up * [marketplace] align with contract changes - getState / state > requestState - getSlot > getRequestFromSlotId - support MarketplaceConfig - support slotState, remove unneeded Slot type - collateral > config.collateral.initialAmount - remove proofPeriod contract call - Revert reason “Slot empty” > “Slot is free” - getProofEnd > read SlotState Tests for changes * [marketplace] add missing file * [marketplace] bump codex-contracts-eth * [config] remove unused imports * [sales] cleanup * [sales] fix: do not crash when fetching state fails * [sales] make slotIndex non-optional * Rebase and update NBS commit Rebase on top of main and update NBS commit to the CI fix. * [marketplace] use async subscription event handlers * [marketplace] support slotIndex no longer optional Previously, SalesAgent.slotIndex had been moved to not optional. However, there were still many places where optionality was assumed. This commit removes those assumuptions. * [marketplace] sales state machine: use slotState Use `slotState` instead of `requestState` for sales state machine. * [marketplace] clean up * [statemachine] adds a statemachine for async workflows Allows events to be scheduled synchronously. See https://github.com/status-im/nim-codex/pull/344 Co-Authored-By: Ben Bierens <thatbenbierens@gmail.com> Co-Authored-By: Eric Mastro <eric.mastro@gmail.com> * [market] make market callbacks synchronous * [statemachine] export Event * [statemachine] ensure that no errors are raised * [statemachine] add machine parameter to run method * [statemachine] initialize queue on start * [statemachine] check futures before cancelling them * [sales] use new async state machine - states use new run() method and event mechanism - StartState starts subscriptions and loads request * [statemachine] fix unsusbscribe before subscribe * [sales] replace old state transition tests * [sales] separate state machine from sales data * [sales] remove reference from SalesData to Sales * [sales] separate sales context from sales * [sales] move decoupled types into their own modules * [sales] move retrieveRequest to SalesData * [sales] move subscription logic into SalesAgent * [sales] unsubscribe when finished or errored * [build] revert back to released version of nim-ethers * [sales] remove SaleStart state * [sales] add missing base method * [sales] move asyncSpawn helper to utils * [sales] fix imports * [sales] remove unused variables * [sales statemachine] add async state machine error handling (#349) * [statemachine] add error handling to asyncstatemachine - add error handling to catch errors during state.run - Sales: add ErrorState to identify which state to transition to during an error. This had to be added to SalesAgent constructor due to circular dependency issues, otherwise it would have been added directly to SalesAgent. - Sales: when an error during run is encountered, the SaleErrorState is constructed with the error, and by default (base impl) will return the error state, so the machine can transition to it. This can be overridden by individual states if needed. * [sales] rename onSaleFailed to onSaleErrored Because there is already a state named SaleFailed which is meant to react to an onchain RequestFailed event and also because this callback is called from SaleErrored, renaming to onSaleErrored prevents ambiguity and confusion as to what has happened at the callback callsite. * [statemachine] forward error to state directly without going through a machine method first * [statemachine] remove unnecessary error handling AsyncQueueFullError is already handled in schedule() * [statemachine] test that cancellation ignores onError * [sales] simplify error handling in states Rely on the state machine error handling instead of catching errors in the state run method --------- Co-authored-by: Mark Spanbroek <mark@spanbroek.net> * [statemachine] prevent memory leaks prevent memory leaks and nil access defects by: - allowing multiple subscribe/unsubscribes of salesagent - disallowing individual salesagent subscription calls to be made externally (requires the .subscribed check) - allowing mutiple start/stops of asyncstatemachine - disregard asyncstatemachine schedules if machine not yet started * [salesagent] add salesagent-specific tests 1. test multiple subscribe/unsubscribes 2. test scheduling machine without being started 3. test subscriptions are working correctly with external events 4. test errors can be overridden at the state level for ErrorHandlingStates. --------- Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Mark Spanbroek <mark@spanbroek.net> Co-authored-by: Ben Bierens <thatbenbierens@gmail.com>
2023-03-08 13:34:26 +00:00
let slotId1 = request.slotId(slotIndex - 1)
let slotId2 = request.slotId(slotIndex)
check (await market.mySlots()) == @[slotId1, slotId2]
test "returns none when slot is empty":
await market.requestStorage(request)
let slotId = request.slotId(slotIndex)
[marketplace] Add Reservations Module (#340) * [marketplace] reservations module - add de/serialization for Availability - add markUsed/markUnused in persisted availability - add query for unused - add reserve/release - reservation module tests - split ContractInteractions into client contracts and host contracts - remove reservations start/stop as the repo start/stop is being managed by the node - remove dedicated reservations metadata store and use the metadata store from the repo instead - Split ContractInteractions into: - ClientInteractions (with purchasing) - HostInteractions (with sales and proving) - compilation fix for nim 1.2 [repostore] fix started flag, add tests [marketplace] persist slot index For loading the sales state from chain, the slot index was not previously persisted in the contract. Will retrieve the slot index from the contract when the sales state is loaded. * Revert repostore changes In favour of separate PR https://github.com/status-im/nim-codex/pull/374. * remove warnings * clean up * tests: stop repostore during teardown * change constructor type identifier Change Contracts constructor to accept Contracts type instead of ContractInteractions. * change constructor return type to Result instead of Option * fix and split interactions tests * clean up, fix tests * find availability by slot id * remove duplication in host/client interactions * add test for finding availability by slotId * log instead of raiseAssert when failed to mark availability as unused * move to SaleErrored state instead of raiseAssert * remove unneeded reverse It appears that order is not preserved in the repostore, so reversing does not have the intended effect here. * update open api spec for potential rest endpoint errors * move functions about available bytes to repostore * WIP: reserve and release availabilities as needed WIP: not tested yet Availabilities are marked as used when matched (just before downloading starts) so that future matching logic does not match an availability currently in use. As the download progresses, batches of blocks are written to disk, and the equivalent bytes are released from the reservation module. The size of the availability is reduced as well. During a reserve or release operation, availability updates occur after the repo is manipulated. If the availability update operation fails, the reserve or release is rolled back to maintain correct accounting of bytes. Finally, once download completes, or if an error occurs, the availability is marked as unused so future matching can occur. * delete availability when all bytes released * fix tests + cleanup * remove availability from SalesContext callbacks Availability is no longer used past the SaleDownloading state in the state machine. Cleanup of Availability (marking unused) is handled directly in the SaleDownloading state, and no longer in SaleErrored or SaleFinished. Likewise, Availabilities shouldn’t need to be handled on node restart. Additionally, Availability was being passed in SalesContext callbacks, and now that Availability is only used temporarily in the SaleDownloading state, Availability is contextually irrelevant to the callbacks, except in OnStore possibly, though it was not being consumed. * test clean up * - remove availability from callbacks and constructors from previous commit that needed to be removed (oopsie) - fix integration test that checks availabilities - there was a bug fixed that crashed the node due to a missing `return success` in onStore - the test was fixed by ensuring that availabilities are remaining on the node, and the size has been reduced - change Availability back to non-ref object and constructor back to init - add trace logging of all state transitions in state machine - add generally useful trace logging * fixes after rebase 1. Fix onProve callbacks 2. Use Slot type instead of tuple for retrieving active slot. 3. Bump codex-contracts-eth that exposes getActivceSlot call. * swap contracts branch to not support slot collateral Slot collateral changes in the contracts require further changes in the client code, so we’ll skip those changes for now and add in a separate commit. * modify Interactions and Deployment constructors - `HostInteractions` and `ClientInteractions` constructors were simplified to take a contract address and no overloads - `Interactions` prepared simplified so there are no overloads - `Deployment` constructor updated so that it takes an optional string parameter, instead `Option[string]` * Move `batchProc` declaration `batchProc` needs to be consumed by both `node` and `salescontext`, and they can’t reference each other as it creates a circular dependency. * [reservations] rename `available` to `hasAvailable` * [reservations] default error message to inner error msg * add SaleIngored state When a storage request is handled but the request does match availabilities, the sales agent machine is sent to the SaleIgnored state. In addition, the agent is constructed in a way that if the request is ignored, the sales agent is removed from the list of active agents being tracked in the sales module.
2023-04-04 07:05:16 +00:00
check (await market.getActiveSlot(slotId)) == none Slot
[marketplace] Load sales state from chain (#306) * [marketplace] get active slots from chain # Conflicts: # codex/contracts/market.nim * [marketplace] make on chain event callbacks async # Conflicts: # tests/codex/helpers/mockmarket.nim * [marketplace] make availability optional for node restart # Conflicts: # tests/codex/testsales.nim * [marketplace] add async state machine Allows for `enterAsync` to be cancelled. * [marketplace] move sale process to async state machine * [marketplace] sales state machine tests * bump dagger-contracts * [marketplace] fix ci issue with chronicles output * PR comments - add slotIndex to `SalesAgent` constructor - remove `SalesAgent.init` - rename `SalesAgent.init` to `start` and `SalesAgent.deinit` to `stop`. - rename `SalesAgent. populateRequest` to `SalesAgent.retreiveRequest`. - move availability removal to the downloading state. once availability is persisted to disk, it should survive node restarts. - * [marketplace] handle slot filled by other host Handle the case when in the downloading, proving, or filling states, that another host fills the slot. * [marketplace] use requestId for mySlots * [marketplace] infer slot index from slotid prevents reassigning a random slot index when restoring state from chain * [marketplace] update to work with latest contracts * [marketplace] clean up * [marketplace] align with contract changes - getState / state > requestState - getSlot > getRequestFromSlotId - support MarketplaceConfig - support slotState, remove unneeded Slot type - collateral > config.collateral.initialAmount - remove proofPeriod contract call - Revert reason “Slot empty” > “Slot is free” - getProofEnd > read SlotState Tests for changes * [marketplace] add missing file * [marketplace] bump codex-contracts-eth * [config] remove unused imports * [sales] cleanup * [sales] fix: do not crash when fetching state fails * [sales] make slotIndex non-optional * Rebase and update NBS commit Rebase on top of main and update NBS commit to the CI fix. * [marketplace] use async subscription event handlers * [marketplace] support slotIndex no longer optional Previously, SalesAgent.slotIndex had been moved to not optional. However, there were still many places where optionality was assumed. This commit removes those assumuptions. * [marketplace] sales state machine: use slotState Use `slotState` instead of `requestState` for sales state machine. * [marketplace] clean up * [statemachine] adds a statemachine for async workflows Allows events to be scheduled synchronously. See https://github.com/status-im/nim-codex/pull/344 Co-Authored-By: Ben Bierens <thatbenbierens@gmail.com> Co-Authored-By: Eric Mastro <eric.mastro@gmail.com> * [market] make market callbacks synchronous * [statemachine] export Event * [statemachine] ensure that no errors are raised * [statemachine] add machine parameter to run method * [statemachine] initialize queue on start * [statemachine] check futures before cancelling them * [sales] use new async state machine - states use new run() method and event mechanism - StartState starts subscriptions and loads request * [statemachine] fix unsusbscribe before subscribe * [sales] replace old state transition tests * [sales] separate state machine from sales data * [sales] remove reference from SalesData to Sales * [sales] separate sales context from sales * [sales] move decoupled types into their own modules * [sales] move retrieveRequest to SalesData * [sales] move subscription logic into SalesAgent * [sales] unsubscribe when finished or errored * [build] revert back to released version of nim-ethers * [sales] remove SaleStart state * [sales] add missing base method * [sales] move asyncSpawn helper to utils * [sales] fix imports * [sales] remove unused variables * [sales statemachine] add async state machine error handling (#349) * [statemachine] add error handling to asyncstatemachine - add error handling to catch errors during state.run - Sales: add ErrorState to identify which state to transition to during an error. This had to be added to SalesAgent constructor due to circular dependency issues, otherwise it would have been added directly to SalesAgent. - Sales: when an error during run is encountered, the SaleErrorState is constructed with the error, and by default (base impl) will return the error state, so the machine can transition to it. This can be overridden by individual states if needed. * [sales] rename onSaleFailed to onSaleErrored Because there is already a state named SaleFailed which is meant to react to an onchain RequestFailed event and also because this callback is called from SaleErrored, renaming to onSaleErrored prevents ambiguity and confusion as to what has happened at the callback callsite. * [statemachine] forward error to state directly without going through a machine method first * [statemachine] remove unnecessary error handling AsyncQueueFullError is already handled in schedule() * [statemachine] test that cancellation ignores onError * [sales] simplify error handling in states Rely on the state machine error handling instead of catching errors in the state run method --------- Co-authored-by: Mark Spanbroek <mark@spanbroek.net> * [statemachine] prevent memory leaks prevent memory leaks and nil access defects by: - allowing multiple subscribe/unsubscribes of salesagent - disallowing individual salesagent subscription calls to be made externally (requires the .subscribed check) - allowing mutiple start/stops of asyncstatemachine - disregard asyncstatemachine schedules if machine not yet started * [salesagent] add salesagent-specific tests 1. test multiple subscribe/unsubscribes 2. test scheduling machine without being started 3. test subscriptions are working correctly with external events 4. test errors can be overridden at the state level for ErrorHandlingStates. --------- Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Mark Spanbroek <mark@spanbroek.net> Co-authored-by: Ben Bierens <thatbenbierens@gmail.com>
2023-03-08 13:34:26 +00:00
test "can retrieve request details from slot id":
await market.requestStorage(request)
await market.fillSlot(request.id, slotIndex, proof, request.ask.collateral)
[marketplace] Load sales state from chain (#306) * [marketplace] get active slots from chain # Conflicts: # codex/contracts/market.nim * [marketplace] make on chain event callbacks async # Conflicts: # tests/codex/helpers/mockmarket.nim * [marketplace] make availability optional for node restart # Conflicts: # tests/codex/testsales.nim * [marketplace] add async state machine Allows for `enterAsync` to be cancelled. * [marketplace] move sale process to async state machine * [marketplace] sales state machine tests * bump dagger-contracts * [marketplace] fix ci issue with chronicles output * PR comments - add slotIndex to `SalesAgent` constructor - remove `SalesAgent.init` - rename `SalesAgent.init` to `start` and `SalesAgent.deinit` to `stop`. - rename `SalesAgent. populateRequest` to `SalesAgent.retreiveRequest`. - move availability removal to the downloading state. once availability is persisted to disk, it should survive node restarts. - * [marketplace] handle slot filled by other host Handle the case when in the downloading, proving, or filling states, that another host fills the slot. * [marketplace] use requestId for mySlots * [marketplace] infer slot index from slotid prevents reassigning a random slot index when restoring state from chain * [marketplace] update to work with latest contracts * [marketplace] clean up * [marketplace] align with contract changes - getState / state > requestState - getSlot > getRequestFromSlotId - support MarketplaceConfig - support slotState, remove unneeded Slot type - collateral > config.collateral.initialAmount - remove proofPeriod contract call - Revert reason “Slot empty” > “Slot is free” - getProofEnd > read SlotState Tests for changes * [marketplace] add missing file * [marketplace] bump codex-contracts-eth * [config] remove unused imports * [sales] cleanup * [sales] fix: do not crash when fetching state fails * [sales] make slotIndex non-optional * Rebase and update NBS commit Rebase on top of main and update NBS commit to the CI fix. * [marketplace] use async subscription event handlers * [marketplace] support slotIndex no longer optional Previously, SalesAgent.slotIndex had been moved to not optional. However, there were still many places where optionality was assumed. This commit removes those assumuptions. * [marketplace] sales state machine: use slotState Use `slotState` instead of `requestState` for sales state machine. * [marketplace] clean up * [statemachine] adds a statemachine for async workflows Allows events to be scheduled synchronously. See https://github.com/status-im/nim-codex/pull/344 Co-Authored-By: Ben Bierens <thatbenbierens@gmail.com> Co-Authored-By: Eric Mastro <eric.mastro@gmail.com> * [market] make market callbacks synchronous * [statemachine] export Event * [statemachine] ensure that no errors are raised * [statemachine] add machine parameter to run method * [statemachine] initialize queue on start * [statemachine] check futures before cancelling them * [sales] use new async state machine - states use new run() method and event mechanism - StartState starts subscriptions and loads request * [statemachine] fix unsusbscribe before subscribe * [sales] replace old state transition tests * [sales] separate state machine from sales data * [sales] remove reference from SalesData to Sales * [sales] separate sales context from sales * [sales] move decoupled types into their own modules * [sales] move retrieveRequest to SalesData * [sales] move subscription logic into SalesAgent * [sales] unsubscribe when finished or errored * [build] revert back to released version of nim-ethers * [sales] remove SaleStart state * [sales] add missing base method * [sales] move asyncSpawn helper to utils * [sales] fix imports * [sales] remove unused variables * [sales statemachine] add async state machine error handling (#349) * [statemachine] add error handling to asyncstatemachine - add error handling to catch errors during state.run - Sales: add ErrorState to identify which state to transition to during an error. This had to be added to SalesAgent constructor due to circular dependency issues, otherwise it would have been added directly to SalesAgent. - Sales: when an error during run is encountered, the SaleErrorState is constructed with the error, and by default (base impl) will return the error state, so the machine can transition to it. This can be overridden by individual states if needed. * [sales] rename onSaleFailed to onSaleErrored Because there is already a state named SaleFailed which is meant to react to an onchain RequestFailed event and also because this callback is called from SaleErrored, renaming to onSaleErrored prevents ambiguity and confusion as to what has happened at the callback callsite. * [statemachine] forward error to state directly without going through a machine method first * [statemachine] remove unnecessary error handling AsyncQueueFullError is already handled in schedule() * [statemachine] test that cancellation ignores onError * [sales] simplify error handling in states Rely on the state machine error handling instead of catching errors in the state run method --------- Co-authored-by: Mark Spanbroek <mark@spanbroek.net> * [statemachine] prevent memory leaks prevent memory leaks and nil access defects by: - allowing multiple subscribe/unsubscribes of salesagent - disallowing individual salesagent subscription calls to be made externally (requires the .subscribed check) - allowing mutiple start/stops of asyncstatemachine - disregard asyncstatemachine schedules if machine not yet started * [salesagent] add salesagent-specific tests 1. test multiple subscribe/unsubscribes 2. test scheduling machine without being started 3. test subscriptions are working correctly with external events 4. test errors can be overridden at the state level for ErrorHandlingStates. --------- Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Mark Spanbroek <mark@spanbroek.net> Co-authored-by: Ben Bierens <thatbenbierens@gmail.com>
2023-03-08 13:34:26 +00:00
let slotId = request.slotId(slotIndex)
[marketplace] Add Reservations Module (#340) * [marketplace] reservations module - add de/serialization for Availability - add markUsed/markUnused in persisted availability - add query for unused - add reserve/release - reservation module tests - split ContractInteractions into client contracts and host contracts - remove reservations start/stop as the repo start/stop is being managed by the node - remove dedicated reservations metadata store and use the metadata store from the repo instead - Split ContractInteractions into: - ClientInteractions (with purchasing) - HostInteractions (with sales and proving) - compilation fix for nim 1.2 [repostore] fix started flag, add tests [marketplace] persist slot index For loading the sales state from chain, the slot index was not previously persisted in the contract. Will retrieve the slot index from the contract when the sales state is loaded. * Revert repostore changes In favour of separate PR https://github.com/status-im/nim-codex/pull/374. * remove warnings * clean up * tests: stop repostore during teardown * change constructor type identifier Change Contracts constructor to accept Contracts type instead of ContractInteractions. * change constructor return type to Result instead of Option * fix and split interactions tests * clean up, fix tests * find availability by slot id * remove duplication in host/client interactions * add test for finding availability by slotId * log instead of raiseAssert when failed to mark availability as unused * move to SaleErrored state instead of raiseAssert * remove unneeded reverse It appears that order is not preserved in the repostore, so reversing does not have the intended effect here. * update open api spec for potential rest endpoint errors * move functions about available bytes to repostore * WIP: reserve and release availabilities as needed WIP: not tested yet Availabilities are marked as used when matched (just before downloading starts) so that future matching logic does not match an availability currently in use. As the download progresses, batches of blocks are written to disk, and the equivalent bytes are released from the reservation module. The size of the availability is reduced as well. During a reserve or release operation, availability updates occur after the repo is manipulated. If the availability update operation fails, the reserve or release is rolled back to maintain correct accounting of bytes. Finally, once download completes, or if an error occurs, the availability is marked as unused so future matching can occur. * delete availability when all bytes released * fix tests + cleanup * remove availability from SalesContext callbacks Availability is no longer used past the SaleDownloading state in the state machine. Cleanup of Availability (marking unused) is handled directly in the SaleDownloading state, and no longer in SaleErrored or SaleFinished. Likewise, Availabilities shouldn’t need to be handled on node restart. Additionally, Availability was being passed in SalesContext callbacks, and now that Availability is only used temporarily in the SaleDownloading state, Availability is contextually irrelevant to the callbacks, except in OnStore possibly, though it was not being consumed. * test clean up * - remove availability from callbacks and constructors from previous commit that needed to be removed (oopsie) - fix integration test that checks availabilities - there was a bug fixed that crashed the node due to a missing `return success` in onStore - the test was fixed by ensuring that availabilities are remaining on the node, and the size has been reduced - change Availability back to non-ref object and constructor back to init - add trace logging of all state transitions in state machine - add generally useful trace logging * fixes after rebase 1. Fix onProve callbacks 2. Use Slot type instead of tuple for retrieving active slot. 3. Bump codex-contracts-eth that exposes getActivceSlot call. * swap contracts branch to not support slot collateral Slot collateral changes in the contracts require further changes in the client code, so we’ll skip those changes for now and add in a separate commit. * modify Interactions and Deployment constructors - `HostInteractions` and `ClientInteractions` constructors were simplified to take a contract address and no overloads - `Interactions` prepared simplified so there are no overloads - `Deployment` constructor updated so that it takes an optional string parameter, instead `Option[string]` * Move `batchProc` declaration `batchProc` needs to be consumed by both `node` and `salescontext`, and they can’t reference each other as it creates a circular dependency. * [reservations] rename `available` to `hasAvailable` * [reservations] default error message to inner error msg * add SaleIngored state When a storage request is handled but the request does match availabilities, the sales agent machine is sent to the SaleIgnored state. In addition, the agent is constructed in a way that if the request is ignored, the sales agent is removed from the list of active agents being tracked in the sales module.
2023-04-04 07:05:16 +00:00
let expected = Slot(request: request, slotIndex: slotIndex)
check (await market.getActiveSlot(slotId)) == some expected
[marketplace] Load sales state from chain (#306) * [marketplace] get active slots from chain # Conflicts: # codex/contracts/market.nim * [marketplace] make on chain event callbacks async # Conflicts: # tests/codex/helpers/mockmarket.nim * [marketplace] make availability optional for node restart # Conflicts: # tests/codex/testsales.nim * [marketplace] add async state machine Allows for `enterAsync` to be cancelled. * [marketplace] move sale process to async state machine * [marketplace] sales state machine tests * bump dagger-contracts * [marketplace] fix ci issue with chronicles output * PR comments - add slotIndex to `SalesAgent` constructor - remove `SalesAgent.init` - rename `SalesAgent.init` to `start` and `SalesAgent.deinit` to `stop`. - rename `SalesAgent. populateRequest` to `SalesAgent.retreiveRequest`. - move availability removal to the downloading state. once availability is persisted to disk, it should survive node restarts. - * [marketplace] handle slot filled by other host Handle the case when in the downloading, proving, or filling states, that another host fills the slot. * [marketplace] use requestId for mySlots * [marketplace] infer slot index from slotid prevents reassigning a random slot index when restoring state from chain * [marketplace] update to work with latest contracts * [marketplace] clean up * [marketplace] align with contract changes - getState / state > requestState - getSlot > getRequestFromSlotId - support MarketplaceConfig - support slotState, remove unneeded Slot type - collateral > config.collateral.initialAmount - remove proofPeriod contract call - Revert reason “Slot empty” > “Slot is free” - getProofEnd > read SlotState Tests for changes * [marketplace] add missing file * [marketplace] bump codex-contracts-eth * [config] remove unused imports * [sales] cleanup * [sales] fix: do not crash when fetching state fails * [sales] make slotIndex non-optional * Rebase and update NBS commit Rebase on top of main and update NBS commit to the CI fix. * [marketplace] use async subscription event handlers * [marketplace] support slotIndex no longer optional Previously, SalesAgent.slotIndex had been moved to not optional. However, there were still many places where optionality was assumed. This commit removes those assumuptions. * [marketplace] sales state machine: use slotState Use `slotState` instead of `requestState` for sales state machine. * [marketplace] clean up * [statemachine] adds a statemachine for async workflows Allows events to be scheduled synchronously. See https://github.com/status-im/nim-codex/pull/344 Co-Authored-By: Ben Bierens <thatbenbierens@gmail.com> Co-Authored-By: Eric Mastro <eric.mastro@gmail.com> * [market] make market callbacks synchronous * [statemachine] export Event * [statemachine] ensure that no errors are raised * [statemachine] add machine parameter to run method * [statemachine] initialize queue on start * [statemachine] check futures before cancelling them * [sales] use new async state machine - states use new run() method and event mechanism - StartState starts subscriptions and loads request * [statemachine] fix unsusbscribe before subscribe * [sales] replace old state transition tests * [sales] separate state machine from sales data * [sales] remove reference from SalesData to Sales * [sales] separate sales context from sales * [sales] move decoupled types into their own modules * [sales] move retrieveRequest to SalesData * [sales] move subscription logic into SalesAgent * [sales] unsubscribe when finished or errored * [build] revert back to released version of nim-ethers * [sales] remove SaleStart state * [sales] add missing base method * [sales] move asyncSpawn helper to utils * [sales] fix imports * [sales] remove unused variables * [sales statemachine] add async state machine error handling (#349) * [statemachine] add error handling to asyncstatemachine - add error handling to catch errors during state.run - Sales: add ErrorState to identify which state to transition to during an error. This had to be added to SalesAgent constructor due to circular dependency issues, otherwise it would have been added directly to SalesAgent. - Sales: when an error during run is encountered, the SaleErrorState is constructed with the error, and by default (base impl) will return the error state, so the machine can transition to it. This can be overridden by individual states if needed. * [sales] rename onSaleFailed to onSaleErrored Because there is already a state named SaleFailed which is meant to react to an onchain RequestFailed event and also because this callback is called from SaleErrored, renaming to onSaleErrored prevents ambiguity and confusion as to what has happened at the callback callsite. * [statemachine] forward error to state directly without going through a machine method first * [statemachine] remove unnecessary error handling AsyncQueueFullError is already handled in schedule() * [statemachine] test that cancellation ignores onError * [sales] simplify error handling in states Rely on the state machine error handling instead of catching errors in the state run method --------- Co-authored-by: Mark Spanbroek <mark@spanbroek.net> * [statemachine] prevent memory leaks prevent memory leaks and nil access defects by: - allowing multiple subscribe/unsubscribes of salesagent - disallowing individual salesagent subscription calls to be made externally (requires the .subscribed check) - allowing mutiple start/stops of asyncstatemachine - disregard asyncstatemachine schedules if machine not yet started * [salesagent] add salesagent-specific tests 1. test multiple subscribe/unsubscribes 2. test scheduling machine without being started 3. test subscriptions are working correctly with external events 4. test errors can be overridden at the state level for ErrorHandlingStates. --------- Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Mark Spanbroek <mark@spanbroek.net> Co-authored-by: Ben Bierens <thatbenbierens@gmail.com>
2023-03-08 13:34:26 +00:00
test "retrieves correct slot state when request is unknown":
let slotId = request.slotId(slotIndex)
check (await market.slotState(slotId)) == SlotState.Free
test "retrieves correct slot state once filled":
await market.requestStorage(request)
await market.fillSlot(request.id, slotIndex, proof, request.ask.collateral)
[marketplace] Load sales state from chain (#306) * [marketplace] get active slots from chain # Conflicts: # codex/contracts/market.nim * [marketplace] make on chain event callbacks async # Conflicts: # tests/codex/helpers/mockmarket.nim * [marketplace] make availability optional for node restart # Conflicts: # tests/codex/testsales.nim * [marketplace] add async state machine Allows for `enterAsync` to be cancelled. * [marketplace] move sale process to async state machine * [marketplace] sales state machine tests * bump dagger-contracts * [marketplace] fix ci issue with chronicles output * PR comments - add slotIndex to `SalesAgent` constructor - remove `SalesAgent.init` - rename `SalesAgent.init` to `start` and `SalesAgent.deinit` to `stop`. - rename `SalesAgent. populateRequest` to `SalesAgent.retreiveRequest`. - move availability removal to the downloading state. once availability is persisted to disk, it should survive node restarts. - * [marketplace] handle slot filled by other host Handle the case when in the downloading, proving, or filling states, that another host fills the slot. * [marketplace] use requestId for mySlots * [marketplace] infer slot index from slotid prevents reassigning a random slot index when restoring state from chain * [marketplace] update to work with latest contracts * [marketplace] clean up * [marketplace] align with contract changes - getState / state > requestState - getSlot > getRequestFromSlotId - support MarketplaceConfig - support slotState, remove unneeded Slot type - collateral > config.collateral.initialAmount - remove proofPeriod contract call - Revert reason “Slot empty” > “Slot is free” - getProofEnd > read SlotState Tests for changes * [marketplace] add missing file * [marketplace] bump codex-contracts-eth * [config] remove unused imports * [sales] cleanup * [sales] fix: do not crash when fetching state fails * [sales] make slotIndex non-optional * Rebase and update NBS commit Rebase on top of main and update NBS commit to the CI fix. * [marketplace] use async subscription event handlers * [marketplace] support slotIndex no longer optional Previously, SalesAgent.slotIndex had been moved to not optional. However, there were still many places where optionality was assumed. This commit removes those assumuptions. * [marketplace] sales state machine: use slotState Use `slotState` instead of `requestState` for sales state machine. * [marketplace] clean up * [statemachine] adds a statemachine for async workflows Allows events to be scheduled synchronously. See https://github.com/status-im/nim-codex/pull/344 Co-Authored-By: Ben Bierens <thatbenbierens@gmail.com> Co-Authored-By: Eric Mastro <eric.mastro@gmail.com> * [market] make market callbacks synchronous * [statemachine] export Event * [statemachine] ensure that no errors are raised * [statemachine] add machine parameter to run method * [statemachine] initialize queue on start * [statemachine] check futures before cancelling them * [sales] use new async state machine - states use new run() method and event mechanism - StartState starts subscriptions and loads request * [statemachine] fix unsusbscribe before subscribe * [sales] replace old state transition tests * [sales] separate state machine from sales data * [sales] remove reference from SalesData to Sales * [sales] separate sales context from sales * [sales] move decoupled types into their own modules * [sales] move retrieveRequest to SalesData * [sales] move subscription logic into SalesAgent * [sales] unsubscribe when finished or errored * [build] revert back to released version of nim-ethers * [sales] remove SaleStart state * [sales] add missing base method * [sales] move asyncSpawn helper to utils * [sales] fix imports * [sales] remove unused variables * [sales statemachine] add async state machine error handling (#349) * [statemachine] add error handling to asyncstatemachine - add error handling to catch errors during state.run - Sales: add ErrorState to identify which state to transition to during an error. This had to be added to SalesAgent constructor due to circular dependency issues, otherwise it would have been added directly to SalesAgent. - Sales: when an error during run is encountered, the SaleErrorState is constructed with the error, and by default (base impl) will return the error state, so the machine can transition to it. This can be overridden by individual states if needed. * [sales] rename onSaleFailed to onSaleErrored Because there is already a state named SaleFailed which is meant to react to an onchain RequestFailed event and also because this callback is called from SaleErrored, renaming to onSaleErrored prevents ambiguity and confusion as to what has happened at the callback callsite. * [statemachine] forward error to state directly without going through a machine method first * [statemachine] remove unnecessary error handling AsyncQueueFullError is already handled in schedule() * [statemachine] test that cancellation ignores onError * [sales] simplify error handling in states Rely on the state machine error handling instead of catching errors in the state run method --------- Co-authored-by: Mark Spanbroek <mark@spanbroek.net> * [statemachine] prevent memory leaks prevent memory leaks and nil access defects by: - allowing multiple subscribe/unsubscribes of salesagent - disallowing individual salesagent subscription calls to be made externally (requires the .subscribed check) - allowing mutiple start/stops of asyncstatemachine - disregard asyncstatemachine schedules if machine not yet started * [salesagent] add salesagent-specific tests 1. test multiple subscribe/unsubscribes 2. test scheduling machine without being started 3. test subscriptions are working correctly with external events 4. test errors can be overridden at the state level for ErrorHandlingStates. --------- Co-authored-by: Eric Mastro <eric.mastro@gmail.com> Co-authored-by: Mark Spanbroek <mark@spanbroek.net> Co-authored-by: Ben Bierens <thatbenbierens@gmail.com>
2023-03-08 13:34:26 +00:00
let slotId = request.slotId(slotIndex)
check (await market.slotState(slotId)) == SlotState.Filled