nim-codex/tests/testIntegration.nim

import ./integration/testcli
import ./integration/testrestapi
import ./integration/testupdownload
import ./integration/testsales
import ./integration/testpurchasing
import ./integration/testblockexpiration
import ./integration/testmarketplace
import ./integration/testslotreservations
import ./integration/testproofs
import ./integration/testvalidator
import ./integration/testecbug

{.warning[UnusedImport]:off.}
Taiko L2 (#483) * [docs] fix two client scenario: add missing collateral * [integration] separate step to wait for node to be started * [cli] add option to specify ethereum private key * Remove unused imports * Fix warnings * [integration] move type definitions to correct place * [integration] wait a bit longer for a node to start in debug mode When e.g. running against Taiko test net rpc, the node start takes longer * [integration] simplify handling of codex node and client * [integration] add Taiko integration test * [contracts] await token approval confirmation before next tx * [contracts] deployment address of marketplace on Taiko * [cli] --eth-private-key now takes a file name Instead of supplying the private key on the command line, expect the private key to be in a file with the correct permissions. * [utils] Fixes undeclared `activeChroniclesStream` on Windows * [build] update nim-ethers to include PR #52 Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [cli] Better error messages when reading eth private key Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [integration] simplify reading of cmd line arguments Co-authored-by: Eric Mastro <eric.mastro@gmail.com> * [build] update to latest version of nim-ethers * [contracts] updated contract address for Taiko L2 * [build] update codex contracts to latest version --------- Co-authored-by: Eric Mastro <eric.mastro@gmail.com> 2023-09-13 16:17:56 +02:00			`import ./integration/testcli`
Cleanup integration tests (#757) * integration: move REST API tests into their own module * integration: move upload and download tests into their own module * integration: move purchasing tests into their own module * integration: move marketplace tests to the right module * integration: mine a block after starting nodes To ensure that tests involving multiple nodes do not start with out-of-sync clocks * Fix: do not swallow CancellationErrors * integration: avoid underflow in UInt256 * network: remove unnecessary error handling No Exceptions can occur, only Defects, because everything is asyncSpawned. Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com> * network: do not raise in asyncSpawned proc Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com> --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com> 2024-05-23 17:29:30 +02:00			`import ./integration/testrestapi`
			`import ./integration/testupdownload`
			`import ./integration/testsales`
			`import ./integration/testpurchasing`
Blockstore maintenance (#347) * setting up * Implements timer utility * applies async and cancellation to timer loop * Sets up mocktimer and mockblockstore to set up first maintenance module test * wip: first test that calls blockChecker * wip: adding user type to timer callback * Chronicles doesn't like type-arguments? Disables logging in timer module for now * Implementing block check test for blockMaintainer module * Sets up additional tests for blockmaintainer * Removes generic from timer module. Implements numberOfBlocks per interval in blockMaintainer. * Implements blockMaintainer * Sets up tests for blockChecker * Some comments by Mark * Cleanup repostore tests * Setting up the use of std/times for block TTL tracking * repostore adds expiration timestamp * Defaults the repostore clock to the system clock * Applies updates to repostore interface. * Implements retrieving of block expiration information from repostore * Sets up tests for maintenance module behavior * Implements block maintenance module * Wires maintenance module into codex. Sets up integration tests for block expiration * Sets up test for missing behavior: removing timestamp metadata on block delete * Implements removing of expiration metadata in repoStore * Fixes integration tests for block expiration * Adds block expiration tests to integration test run * Handled some comments by Dmitriy * Review comment by Dmitriy: Removes seq[cid] from runBlockCheck * Review comment by Dmitriy: Moves key formatting methods to keyutils. * Review comment by Dmitriy: Encodes durations using chronos * Fixes conversion of TTL type in conf. * Review comments by Dmitriy * Adds unit tests for keyUtils. * Adds test coverage for exception in maintenance module 2023-03-08 16:04:54 +01:00			`import ./integration/testblockexpiration`
refactor: multinode integration test refactor (#662) * refactor multi node test suite Refactor the multinode test suite into the marketplace test suite. - Arbitrary number of nodes can be started with each test: clients, providers, validators - Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file. - Log files for all nodes can be persisted on disk, with configuration at the test-level - Log files, if persisted (as specified in the test), will be persisted to a CI artifact - Node config is specified at the test-level instead of the suite-level - Node/Hardhat process starting/stopping is now async, and runs much faster - Per-node config includes: - simulating proof failures - logging to file - log level - log topics - storage quota - debug (print logs to stdout) - Tests find next available ports when starting nodes, as closing ports on Windows can lag - Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests). - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively. - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively. * additional logging for debug purposes * address PR feedback - fix spelling - revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR. - remove method label from raiseAssert - remove unused import * Use API instead of command exec to test for free port Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs. * clean up * remove upraises annotations from tests * Update tests to work with updated erasure coding slot sizes * update dataset size, nodes, tolerance to match valid ec params Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage. Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params). All integration tests pass when the async `clock.now` changes are reverted. * dont use async clock for now * fix workflow * move integration logs uplod to reusable --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com> 2024-02-19 15:55:39 +11:00			`import ./integration/testmarketplace`
Add integration test that checks for "nonce too high" in logs 2024-10-29 17:47:33 +11:00			`import ./integration/testslotreservations`
Generate proofs when required (#383) * [maintenance] speedup integration test * [rest api] add proofProbability parameter to storage requests * [integration] negotiation test ends when contract starts * [integration] reusable 2 node setup for tests * [integration] introduce CodexClient for tests * [node] submit storage proofs when required * [contracts] Add Slot type * [proving] replace onProofRequired & submitProof with onProve Removes duplication between Sales.onProve() and Proving.onProofRequired() 2023-03-27 15:47:25 +02:00			`import ./integration/testproofs`
Validator historical state restoration (#922) * adds a new overload of queryPastEvents allowing to query past events based on timestamp in the past * adds state restoration to validator * refactors a bit to get the tests back to work * replaces deprecated generic methods from Market with methods for specific event types * Refactors binary search * adds market tests for querying past SlotFilled events and binary search * Takes into account that <<earliest>> block available is not necessarily the genesis block * Adds more logging and makes testing earliest block boundary more reliable * adds validation tests for historical state restoration * adds mockprovider to simplify and improve testing of the edge conditions * adds slot reservation to the new tests after rebasing * adds validation groups and group index in logs of validator * adds integration test with two validators * adds comment on how to enable logging in integration test executable itself * testIntegration: makes list is running nodes injected and available in the body of the test * validation: adds integration test for historical state * adds more logging to validator * integration test: validator only looks 30 days back for historical state * adds logging of the slotState when removing slots during validation * review and refactor validator integration tests * adds validation to the set of integration tests * Fixes mistyped name of the mock provider module in testMarket * Fixes a typo in the name of the validation suite in integration tests * Makes validation unit test a bit easier to follow * better use of logScopes to reduce duplication * improves timing and clarifies the test conditions * uses http as default RPC provider for nodes running in integration tests as a workaround for dropped subscriptions * simplifies the validation integration tests by waiting for failed request instead of tracking slots * adds config option allowing selectively to set different provider url * Brings back the default settings for RPC provider in integration tests * use http RPC provider for clients in validation integration tests * fine-tune the tests * Makes validator integration test more robust - adds extra tracking * brings tracking of marketplace event back to validator integration test * refactors integration tests * deletes tmp file * adds <<return>> after forcing integration test to fail preliminarily * re-enables all integration tests and matrix * stops debug output in CI * allows to choose a different RPC provider for a given integration test suite * fixes signature of <<getBlock>> method in mockProvider * adds missing import which seem to be braking integration tests on windows * makes sure that clients, SPs, and validators use the same provider url * makes validator integration tests using http at 127.0.0.1:8545 * testvalidator: stop resubscribing as we are now using http polling as rpc provider * applying review comments * groups queryPastStorage overrides together (review comment) * groups the historical validation tests into a sub suite * removes the temporary extensions in marketplacesuite and multinodesuite allowing to specify provider url * simplifies validation integration tests * Removes debug logs when waiting for request to fail * Renaming waitForRequestFailed => waitForRequestToFail * renames blockNumberForBlocksAgo to pastBlockTag and makes it private * removes redundant debugging logs * refines logging in validation * removes dev logging from mockmarket * improves exception handling in provider helper procs and prepares for extraction to a separate module * Uses chronos instead of std/times for Duration * extracts provider and binary search helpers to a separate module * removes redundant log entry params from validator * unifies the notation to consistently use method call syntax * reuses ProviderError from nim-ethers in the provider extension * clarifies the comment in multinodesuite * uses == operator to check the predefined tags and raises exception when `BlockTag.pending` is requested. * when waiting for request to fail, we break on any request state that is not Started * removes tests that were moved to testProvider from testMarket * extracts tests that use MockProvider to a separate async suite * improves performance of the historical state restoration * removing redundant log messages in validator (groupIndex and groups) * adds testProvider to testContracts group * removes unused import in testMarket 2024-12-14 06:07:55 +01:00			`import ./integration/testvalidator`
Fix verifiable manifest constructor (#844) * Fix verifiable manifest constructor * Add integration test for verifiable manifest download Add integration test for testing download of verifiable dataset after creating request for storage * add missing import * add testecbug to integration suite * Remove hardhat instance from integration test * change description, drop echo --------- Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com> Co-authored-by: gmega <giuliano.mega@gmail.com> 2024-06-26 22:02:39 +02:00			`import ./integration/testecbug`
[test] Add integration test for sales and purchasing 2022-05-18 14:31:45 +02:00
Fix warnings (drops Nim 1.2) (#348) * [build] disable XCannotRaiseY hint There are too many {.raises:[Defect].} in the libraries that we use, drowning out all other warnings and hints * [build] disable BareExcept warning Not yet enabled in a released version of Nim, so libraries that we depend on have not fixed this yet, drowning out our own hints and warnings * [build] disable DotLikeOps warning dot-like ops were an experiment that is not going land in Nim * [build] compile log statements in tests When running tests, all log statements are compiled. They are filtered out at runtime during a test run. * [build] do not build executable when running unit test It's already built in the integration test * [build] Fix warnings - remove unused code - remove unused imports - stop using deprecated stuff * [build] Put compiler flags behind nim version checks * [CI] remove Nim 1.2 compatibility 2023-03-09 12:23:45 +01:00			`{.warning[UnusedImport]:off.}`