Commit Graph

393 Commits

Author SHA1 Message Date
Mark Spanbroek fa205677fd
[ci] Use nimble instead of nimbus-build-system 2022-11-14 11:51:30 +01:00
Mark Spanbroek 3c32da5854
[build] set correct lrucache requirement in nimble file 2022-11-14 11:27:52 +01:00
Mark Spanbroek 508c44349e
[build] Update nim-ethers to 0.2.4 2022-11-14 11:22:45 +01:00
Mark Spanbroek 39d98ae1e1
[ci] Cancel all other jobs when one fails
Allows for faster feedback loops; a next attempt to run
the CI will start sooner.
2022-11-09 16:38:29 +01:00
Mark Spanbroek 6dc2055ba2
[ci] Ensure that unit tests are not dependent on Ethereum
Start Ethereum node *after* the unit tests are run.
2022-11-09 16:38:29 +01:00
Mark Spanbroek a0f6ef54bb
[PoR] Test PoR network with a small block size
To get a reasonable test execution time.
2022-11-09 16:38:29 +01:00
Mark Spanbroek 72f67abfb1
[PoR] Test PoR store with a small block size
To get a reasonable test execution time.
2022-11-09 16:38:29 +01:00
Mark Spanbroek 51f25c7655
[erasure] fix typo 2022-11-09 16:38:28 +01:00
Mark Spanbroek 5c2041c0d7
[erasure] Test erasure coding with a small block size
To get a reasonable test execution time.
2022-11-09 16:38:28 +01:00
Mark Spanbroek d5a8d110a5
[build] update to latest nimbus-build-system
Requires an update to latest nim-nat-traversal
2022-11-09 16:38:28 +01:00
Mark Spanbroek cf6d5c1640
[ci] Set cache_nonce for all jobs, not just those in the matrix 2022-11-09 16:38:28 +01:00
Mark Spanbroek b890b9d387
[ci] Update caches on every run
Will automatically update caches when a new minor version
of Nim is released.

See also:
https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache
2022-11-09 16:38:28 +01:00
Mark Spanbroek e19cc548c9
[ci] Checkout submodules recursively
So that nimbus-build-system doesn't have to
do it afterwards.
2022-11-09 16:38:28 +01:00
Mark Spanbroek 9df32625d9
[ci] Set NIM_COMMIT environment variable
To prevent subsequent `make` command to build a
different version of Nim.
2022-11-09 16:38:27 +01:00
Mark Spanbroek 67832a39e7
[PoR] Test proofs of retrievability with a small block size
To get a reasonable test execution time.
2022-11-09 16:38:27 +01:00
Mark Spanbroek e31f405b90
[ci] Calculate coverage on unit tests only
This enables faster development cycles at
the expense of code coverage accuracy.
2022-11-09 10:35:32 +01:00
Mark Spanbroek 8962e14cea
[ci] Separate build job for Nim 1.2 2022-11-09 10:35:32 +01:00
Mark Spanbroek c031fbd2a4
[ci] Separate build job for coverage that is allowed to fail 2022-11-09 10:35:32 +01:00
Mark Spanbroek 91e184aef8
[ci] Separate steps for unit, contract and integration tests 2022-11-09 10:35:32 +01:00
Mark Spanbroek e9681fcaae
[ci] Use amd64 by default 2022-11-09 10:35:31 +01:00
Mark Spanbroek 8a98ea4a3d
[ci] Cleanup NodeJS setup 2022-11-09 10:35:05 +01:00
Mark Spanbroek fa2fa3574f
[ci] Extract setup of nimbus build system into separate file 2022-11-08 16:10:34 +01:00
markspanbroek 4175689745
Load purchase state from chain (#283)
* [purchasing] Simplify test

* [utils] Move StorageRequest.example up one level

* [purchasing] Load purchases from market

* [purchasing] load purchase states

* Implement myRequest() and getState() methods for OnChainMarket

* [proofs] Fix intermittently failing tests

Ensures that examples of proofs in tests are never of length 0;
these are considered invalid proofs by the smart contract logic.

* [contracts] Fix failing test

With the new solidity contracts update, a contract can only
be paid out after it started.

* [market] Add method to get request end time

* [purchasing] wait until purchase is finished

Purchase.wait() would previously wait until purchase
was started, now we wait until it is finished.

* [purchasing] Handle 'finished' and 'failed' states

* [marketplace] move to failed state once request fails

- Add support for subscribing to request failure events.
- Add supporting contract tests for subscribing to request failure events.
- Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted
- Add supporting tests for moving from PurchaseStarted to PurchaseFailure
- Add state transition tests for PurchaseUnknown.

* [marketplace] Fix test with longer sleepAsync

* [integration] Add function to restart a codex node

* [purchasing] Set client address before requesting storage

To prevent the purchase id (which equals the request id)
from changing once it's been submitted.

* [contracts] Fix: OnChainMarket.getState()

Had the wrong method signature before

* [purchasing] Load purchases on node start

* [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored'

Allows for an exception type called 'PurchaseError'

* [purchasing] Load purchases in background

No longer calls market.getRequest() for every purchase
on node start.

* [contracts] Add `$` for RequestId, SlotId and Nonce

To aid with debugging

* [purchasing] Add Purchasing.stop()

To ensure that all contract interactions have both a
start() and a stop() for

* [tests] Remove sleepAsync where possible

Use `eventually` loop instead, to make sure that we're
not waiting unnecessarily.

* [integration] Fix: handle non-json response in test

* [purchasing] Add purchase state to json

* [integration] Ensure that purchase is submitted before restart

Fixes test failure on slower CI

* [purchasing] re-implement `description` as method

Allows description to be set in the same module where the
state type is defined.

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>

* [contracts] fix typo

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>

* [market] Use more generic error type

Should we decide to change the provider type later

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 08:10:17 +01:00
markspanbroek be32b9619b
Fixes loading of private key on Windows (#299)
Unix permissions don't work on Windows; 
adds check for correct ACL settings in Windows.
2022-11-07 15:54:24 +01:00
Michael Bradley c893b1f0cb
track latest nim-libp2p's master branch (#248)
* [build] track nim-libp2p's unstable branch

* cid formatIt change

* track nim-libp2p-unstable

* don't stringify cid

* Fixed testblockexc.nim

1. Assign price to blocks
2. Delete on node1 cached blocks from node2 before buying them again

* add trace logging

* init pricing info

* remove duplicate price setting

* cid serialization in trace logs

* bumping dht to latest main

* bump

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Co-authored-by: Bulat-Ziganshin <bulat.ziganshin@gmail.com>
2022-11-02 11:40:28 -06:00
Dmitriy Ryajov 0ecbfcec9f
setup and persist private key (#292)
* setup and persist private key

* return dht record spr

* helper to remap multiaddr ip and port

* set/update discovery and announce addrs

* add nat and discovery IPs

* allow for announce and DHT addresses separatelly

* update tests

* check for nat or discoveryIp

* fix integration tests

* misc align

* don't share data dirs and and set bootstrap node

* add log scope

* remap announceAddrs after node start

* simplify discovery initialization

* make nat and disc-ip required

* add log scope don't init dht spr in constructor

* bump dht

* dissallow `0.0.0.0` for `--nat`
2022-11-01 18:58:41 -06:00
Dmitriy Ryajov ae46f4dc2f
add support for logging topics (#293)
* add support for logging topics

* add top level codex topic
2022-11-01 14:05:40 -06:00
Dmitriy Ryajov 6e4a8b86ab
Rework discovery (#288)
* use multiaddrs instead of ip/port

* rework to support updating spr

* fix tests

* fix compilation

* use base for base methods
2022-10-27 07:44:56 -06:00
Dmitriy Ryajov e50ea88411
Remove protobuf serialization (#289)
* add format for cid

* cid formatIt change

* track nim-libp2p-unstable

* rework probuf serialization for por

* add missing include

* removing nim protobuf serialization

* rollback to dht to main

* remove protobuf serialization import
2022-10-27 07:41:34 -06:00
Mark Spanbroek 92eecb0702 [purchasing] Rework purchasing as a state machine 2022-10-25 15:10:35 +11:00
Mark Spanbroek 7841a9692c [utils] Add AsyncState to state machine 2022-10-25 15:10:35 +11:00
Mark Spanbroek b52d291785 [utils] Add state machine implementation 2022-10-25 15:10:35 +11:00
Mark Spanbroek e648c26340 [utils] Add operator `as` for optional casting 2022-10-25 15:10:35 +11:00
Mark Spanbroek bc3734b1a8 [purchasing] move Purchase and PurchaseId into separate modules 2022-10-25 15:10:35 +11:00
Eric Mastro 16024f21e9 bump nim-ethers (to main) 2022-10-25 15:10:35 +11:00
Eric Mastro 1f50612b68 bump dagger-contracts (to main) 2022-10-25 15:10:35 +11:00
Eric Mastro 25df599c08 Update to new reverts API, add tests
1. use `ProviderError` to catch contract errors instaed of `JsonRpcProviderError`.
2. Add tests for `isProofRequired`, `willProofBeRequired`, and `getProofEnd` when slot is empty (and is call is reverted)
3. bump nim-ethers to branch that has new `reverts` API and `ProviderError` support.
4. Update to new `reverts` API.
2022-10-25 15:10:35 +11:00
Eric Mastro a9cea802ff bump dagger-contracts 2022-10-25 15:10:35 +11:00
Eric Mastro a7277e90fb update market test 2022-10-25 15:10:35 +11:00
Eric Mastro 38643c9709 fix revert check on storage test 2022-10-25 15:10:35 +11:00
Eric Mastro 5eb65336fd catch and check revert reasons
Ensure specific contract reverts are handled and returned as booleans or default values

# Conflicts:
#	codex/contracts/proofs.nim
2022-10-25 15:10:35 +11:00
Eric Mastro 10d4dcd70d bump nim-ethers, fix tests
# Conflicts:
#	vendor/nim-ethers
2022-10-25 15:10:35 +11:00
Eric Mastro dab65eb6bd catch contract errors to return bool
# Conflicts:
#	codex/contracts/proofs.nim
#	vendor/nim-ethers
2022-10-25 15:10:35 +11:00
Eric Mastro ae9467b780 simplify getRequest 2022-10-25 15:10:35 +11:00
Eric Mastro eb14ca6166 bump dagger-contracts
# Conflicts:
#	vendor/dagger-contracts
2022-10-25 15:10:35 +11:00
Eric Mastro b9e53e182f [marketplace] support freeing of slots and failed state
Support storage contract parameters to free slots after too many proofs missed, as well as fail a contract after too many slots have been freed.

# Conflicts:
#	vendor/dagger-contracts
2022-10-25 15:10:35 +11:00
Eric Mastro 8dd110765f [marketplace] Catch slot empty, return default values 2022-10-25 15:10:35 +11:00
Eric Mastro 9d218c88a5 [marketplace] Remove `isSlotCancelled` and `isCancelled`
Remove `isSlotCancelled` and `isCancelled` from proving, as it did not fit in the module.

Update the proving module to not rely on checking the contract to understand if a request was cancelled. Instead, `proofEnd` was modified in `dagger-contracts` such that it returns a past timestamp when the contract is cancelled. This successfully removes
2022-10-25 15:10:35 +11:00
Eric Mastro b4a14e00f7 [marketplace] address more PR comments
- remove RequestState as no longer being used
- remove RequestState check on purchase timeout
- add tests for `withdrawFunds` and `subscribeRequestCancelled`
- update `dagger-contracts` so that `RequestCancelled.requestId` property is indexed in the event
2022-10-25 15:10:35 +11:00
Eric Mastro 2db0131fa4 [fix] fix tests 2022-10-25 15:10:35 +11:00