nim-dagger/tests/integration/testmarketplace.nim

177 lines
6.7 KiB
Nim
Raw Normal View History

refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
import pkg/stew/byteutils
import pkg/codex/units
import ../examples
import ../contracts/time
import ../contracts/deployment
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
import ./marketplacesuite
import ./twonodes
Prover CLI updates (#735) * rework cli to accept circuit params * check circom files extension * adding new required cli changes * don't use ufcs * persistence is a command now * use `nimOldCaseObjects` switch for nim confutils compat * misc * Update cli integration tests * Fix: simulateProofFailures option is not for validator * moving circom params under `prover` command * update tests * Use circuit assets from codex-contract-eth in tests * Add "prover" cli command to tests * use correct stores * make `verifier` a cmd option * update circuit artifacts path * fix cli tests * Update integration tests to use cli commands Integration tests have been updated to use the new cli commands. The api for usage in the integration tests has also changed a bit. Proofs tests have been updated to use 5 nodes and 8 blocks of data. The remaining integration tests also need to be updated. * remove parsedCli from CodexConfig Instead, parse the cli args on the fly when needed * remove unneeded gcsafes * graceful shutdowns Where possible, do not raise assert, as other nodes in the test may already be running. Instead, raise exceptions, catch them in multinodes.nim, and attempt to do a teardown before failing the test. `abortOnError` is set to true so that `fail()` will quit immediately, after teardown has been run. * update testmarketplace to new api, with valid EC params --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com> Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2024-03-12 09:57:13 +00:00
import ./nodeconfigs
twonodessuite "Marketplace", debug1 = false, debug2 = false:
setup:
# Our Hardhat configuration does use automine, which means that time tracked by `ethProvider.currentTime()` is not
# advanced until blocks are mined and that happens only when transaction is submitted.
# As we use in tests ethProvider.currentTime() which uses block timestamp this can lead to synchronization issues.
await ethProvider.advanceTime(1.u256)
test "nodes negotiate contracts on the marketplace":
let size = 0xFFFFFF.u256
let data = await RandomChunker.example(blocks=8)
# client 2 makes storage available
let availability = client2.postAvailability(totalSize=size, duration=20*60.u256, minPrice=300.u256, maxCollateral=300.u256).get
# client 1 requests storage
let cid = client1.upload(data).get
let id = client1.requestStorage(
cid,
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
duration=20*60.u256,
reward=400.u256,
proofProbability=3.u256,
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
expiry=10*60,
collateral=200.u256,
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
nodes = 3,
tolerance = 1).get
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
check eventually(client1.purchaseStateIs(id, "started"), timeout=10*60*1000)
let purchase = client1.getPurchase(id).get
check purchase.error == none string
let availabilities = client2.getAvailabilities().get
check availabilities.len == 1
let newSize = availabilities[0].freeSize
check newSize > 0 and newSize < size
let reservations = client2.getAvailabilityReservations(availability.id).get
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
check reservations.len == 3
check reservations[0].requestId == purchase.requestId
test "node slots gets paid out and rest of tokens are returned to client":
let size = 0xFFFFFF.u256
let data = await RandomChunker.example(blocks = 8)
let marketplace = Marketplace.new(Marketplace.address, ethProvider.getSigner())
let tokenAddress = await marketplace.token()
let token = Erc20Token.new(tokenAddress, ethProvider.getSigner())
let reward = 400.u256
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
let duration = 20*60.u256
let nodes = 3'u
# client 2 makes storage available
let startBalanceHost = await token.balanceOf(account2)
discard client2.postAvailability(totalSize=size, duration=20*60.u256, minPrice=300.u256, maxCollateral=300.u256).get
# client 1 requests storage
let cid = client1.upload(data).get
let id = client1.requestStorage(
cid,
duration=duration,
reward=reward,
proofProbability=3.u256,
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
expiry=10*60,
collateral=200.u256,
nodes = nodes,
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
tolerance = 1).get
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
check eventually(client1.purchaseStateIs(id, "started"), timeout=10*60*1000)
let purchase = client1.getPurchase(id).get
check purchase.error == none string
let clientBalanceBeforeFinished = await token.balanceOf(account1)
# Proving mechanism uses blockchain clock to do proving/collect/cleanup round
# hence we must use `advanceTime` over `sleepAsync` as Hardhat does mine new blocks
# only with new transaction
await ethProvider.advanceTime(duration)
# Checking that the hosting node received reward for at least the time between <expiry;end>
check eventually (await token.balanceOf(account2)) - startBalanceHost >= (duration-5*60)*reward*nodes.u256
# Checking that client node receives some funds back that were not used for the host nodes
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
check eventually(
(await token.balanceOf(account1)) - clientBalanceBeforeFinished > 0,
timeout = 10*1000 # give client a bit of time to withdraw its funds
)
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
marketplacesuite "Marketplace payouts":
test "expired request partially pays out for stored time",
NodeConfigs(
# Uncomment to start Hardhat automatically, typically so logs can be inspected locally
Prover CLI updates (#735) * rework cli to accept circuit params * check circom files extension * adding new required cli changes * don't use ufcs * persistence is a command now * use `nimOldCaseObjects` switch for nim confutils compat * misc * Update cli integration tests * Fix: simulateProofFailures option is not for validator * moving circom params under `prover` command * update tests * Use circuit assets from codex-contract-eth in tests * Add "prover" cli command to tests * use correct stores * make `verifier` a cmd option * update circuit artifacts path * fix cli tests * Update integration tests to use cli commands Integration tests have been updated to use the new cli commands. The api for usage in the integration tests has also changed a bit. Proofs tests have been updated to use 5 nodes and 8 blocks of data. The remaining integration tests also need to be updated. * remove parsedCli from CodexConfig Instead, parse the cli args on the fly when needed * remove unneeded gcsafes * graceful shutdowns Where possible, do not raise assert, as other nodes in the test may already be running. Instead, raise exceptions, catch them in multinodes.nim, and attempt to do a teardown before failing the test. `abortOnError` is set to true so that `fail()` will quit immediately, after teardown has been run. * update testmarketplace to new api, with valid EC params --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com> Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2024-03-12 09:57:13 +00:00
hardhat: HardhatConfig.none,
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
clients:
Prover CLI updates (#735) * rework cli to accept circuit params * check circom files extension * adding new required cli changes * don't use ufcs * persistence is a command now * use `nimOldCaseObjects` switch for nim confutils compat * misc * Update cli integration tests * Fix: simulateProofFailures option is not for validator * moving circom params under `prover` command * update tests * Use circuit assets from codex-contract-eth in tests * Add "prover" cli command to tests * use correct stores * make `verifier` a cmd option * update circuit artifacts path * fix cli tests * Update integration tests to use cli commands Integration tests have been updated to use the new cli commands. The api for usage in the integration tests has also changed a bit. Proofs tests have been updated to use 5 nodes and 8 blocks of data. The remaining integration tests also need to be updated. * remove parsedCli from CodexConfig Instead, parse the cli args on the fly when needed * remove unneeded gcsafes * graceful shutdowns Where possible, do not raise assert, as other nodes in the test may already be running. Instead, raise exceptions, catch them in multinodes.nim, and attempt to do a teardown before failing the test. `abortOnError` is set to true so that `fail()` will quit immediately, after teardown has been run. * update testmarketplace to new api, with valid EC params --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com> Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2024-03-12 09:57:13 +00:00
CodexConfigs.init(nodes=1)
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
# .debug() # uncomment to enable console log output.debug()
# .withLogFile() # uncomment to output log file to tests/integration/logs/<start_datetime> <suite_name>/<test_name>/<node_role>_<node_idx>.log
Prover CLI updates (#735) * rework cli to accept circuit params * check circom files extension * adding new required cli changes * don't use ufcs * persistence is a command now * use `nimOldCaseObjects` switch for nim confutils compat * misc * Update cli integration tests * Fix: simulateProofFailures option is not for validator * moving circom params under `prover` command * update tests * Use circuit assets from codex-contract-eth in tests * Add "prover" cli command to tests * use correct stores * make `verifier` a cmd option * update circuit artifacts path * fix cli tests * Update integration tests to use cli commands Integration tests have been updated to use the new cli commands. The api for usage in the integration tests has also changed a bit. Proofs tests have been updated to use 5 nodes and 8 blocks of data. The remaining integration tests also need to be updated. * remove parsedCli from CodexConfig Instead, parse the cli args on the fly when needed * remove unneeded gcsafes * graceful shutdowns Where possible, do not raise assert, as other nodes in the test may already be running. Instead, raise exceptions, catch them in multinodes.nim, and attempt to do a teardown before failing the test. `abortOnError` is set to true so that `fail()` will quit immediately, after teardown has been run. * update testmarketplace to new api, with valid EC params --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com> Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2024-03-12 09:57:13 +00:00
# .withLogTopics("node", "erasure")
.some,
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
providers:
Prover CLI updates (#735) * rework cli to accept circuit params * check circom files extension * adding new required cli changes * don't use ufcs * persistence is a command now * use `nimOldCaseObjects` switch for nim confutils compat * misc * Update cli integration tests * Fix: simulateProofFailures option is not for validator * moving circom params under `prover` command * update tests * Use circuit assets from codex-contract-eth in tests * Add "prover" cli command to tests * use correct stores * make `verifier` a cmd option * update circuit artifacts path * fix cli tests * Update integration tests to use cli commands Integration tests have been updated to use the new cli commands. The api for usage in the integration tests has also changed a bit. Proofs tests have been updated to use 5 nodes and 8 blocks of data. The remaining integration tests also need to be updated. * remove parsedCli from CodexConfig Instead, parse the cli args on the fly when needed * remove unneeded gcsafes * graceful shutdowns Where possible, do not raise assert, as other nodes in the test may already be running. Instead, raise exceptions, catch them in multinodes.nim, and attempt to do a teardown before failing the test. `abortOnError` is set to true so that `fail()` will quit immediately, after teardown has been run. * update testmarketplace to new api, with valid EC params --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com> Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2024-03-12 09:57:13 +00:00
CodexConfigs.init(nodes=1)
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
# .debug() # uncomment to enable console log output
# .withLogFile() # uncomment to output log file to tests/integration/logs/<start_datetime> <suite_name>/<test_name>/<node_role>_<node_idx>.log
Prover CLI updates (#735) * rework cli to accept circuit params * check circom files extension * adding new required cli changes * don't use ufcs * persistence is a command now * use `nimOldCaseObjects` switch for nim confutils compat * misc * Update cli integration tests * Fix: simulateProofFailures option is not for validator * moving circom params under `prover` command * update tests * Use circuit assets from codex-contract-eth in tests * Add "prover" cli command to tests * use correct stores * make `verifier` a cmd option * update circuit artifacts path * fix cli tests * Update integration tests to use cli commands Integration tests have been updated to use the new cli commands. The api for usage in the integration tests has also changed a bit. Proofs tests have been updated to use 5 nodes and 8 blocks of data. The remaining integration tests also need to be updated. * remove parsedCli from CodexConfig Instead, parse the cli args on the fly when needed * remove unneeded gcsafes * graceful shutdowns Where possible, do not raise assert, as other nodes in the test may already be running. Instead, raise exceptions, catch them in multinodes.nim, and attempt to do a teardown before failing the test. `abortOnError` is set to true so that `fail()` will quit immediately, after teardown has been run. * update testmarketplace to new api, with valid EC params --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com> Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2024-03-12 09:57:13 +00:00
# .withLogTopics("node", "marketplace", "sales", "reservations", "node", "proving", "clock")
.some,
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
):
let reward = 400.u256
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
let duration = 20.periods
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
let collateral = 200.u256
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
let expiry = 10.periods
let data = await RandomChunker.example(blocks=8)
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
let client = clients()[0]
let provider = providers()[0]
let clientApi = client.client
let providerApi = provider.client
let startBalanceProvider = await token.balanceOf(provider.ethAccount)
let startBalanceClient = await token.balanceOf(client.ethAccount)
# provider makes storage available
discard providerApi.postAvailability(
# make availability size small enough that we can't fill all the slots,
# thus causing a cancellation
totalSize=(data.len div 2).u256,
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
duration=duration.u256,
minPrice=reward,
maxCollateral=collateral)
let cid = clientApi.upload(data).get
var slotIdxFilled = none UInt256
proc onSlotFilled(event: SlotFilled) =
slotIdxFilled = some event.slotIndex
let subscription = await marketplace.subscribe(SlotFilled, onSlotFilled)
# client requests storage but requires multiple slots to host the content
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
let id = await clientApi.requestStorage(
cid,
duration=duration,
reward=reward,
expiry=expiry,
collateral=collateral,
nodes=3,
tolerance=1
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
)
# wait until one slot is filled
check eventually(slotIdxFilled.isSome, timeout=expiry.int * 1000)
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
let slotId = slotId(!clientApi.requestId(id), !slotIdxFilled)
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
# wait until sale is cancelled
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
await ethProvider.advanceTime(expiry.u256)
check eventually providerApi.saleStateIs(slotId, "SaleCancelled")
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
check eventually (
let endBalanceProvider = (await token.balanceOf(provider.ethAccount));
endBalanceProvider > startBalanceProvider and
endBalanceProvider < startBalanceProvider + expiry.u256*reward
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
)
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
check eventually(
(
let endBalanceClient = (await token.balanceOf(client.ethAccount));
let endBalanceProvider = (await token.balanceOf(provider.ethAccount));
(startBalanceClient - endBalanceClient) == (endBalanceProvider - startBalanceProvider)
),
timeout = 10*1000 # give client a bit of time to withdraw its funds
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
)
await subscription.unsubscribe()