267 Commits

Author SHA1 Message Date
Mark Spanbroek
1067908b02
clock: fix on-chain clock for hardhat
Only use 'latest' block for updates
Only update the first time you see a block
2024-03-03 06:46:17 +01:00
Mark Spanbroek
5e9307204a
Merge remote-tracking branch 'origin/master' into node-wire-prover 2024-02-28 16:36:55 +01:00
Mark Spanbroek
733a3bf37d
clock: remove unused field 2024-02-28 15:45:42 +01:00
Mark Spanbroek
42f8d65c89
sales: stop cancellation loop when request started, finished or failed 2024-02-28 15:45:42 +01:00
Mark Spanbroek
fb1de95a3c
sales: do not raise in proving loop when slot is cancelled
Allow the onCancelled callback to handle cancellation, and
the onFailed callback to handle failed requests.
2024-02-27 10:34:47 +01:00
Eric
e2cb91ef8f
Proofs tests updates
Contains changes to get the proving tests working reliably.
2024-02-26 18:59:52 +11:00
Mark Spanbroek
b8dd2543dc
sales: wait for stable proof challenge
When the block pointer is nearing the
wrap-around point, we wait another period
before calculating a proof.
2024-02-22 16:37:48 +01:00
Giuliano Mega
457567531f
Fixes active cancellation for pending want requests (#714)
* add block cancellation support + tests

* tie issueCancellations into resolveBlocks for proper exception tracking, address comments

* pull cancellation as separate primitive in BlockExcNetwork

* use allFutures, rename issueBlockCancellations -> cancelBlocks

* use trc instead of wrn to register send error

* do not log peer IDs
2024-02-22 14:54:45 +00:00
Mark Spanbroek
0aa7590c96
sales: calculate initial proof at start of period
reason: this ensures that the period (and therefore
the challenge) doesn't change while we're calculating
the proof
2024-02-22 10:07:11 +01:00
Mark Spanbroek
2a1ef5d9e6
clock: use wall clock in integration tests
reason: we'll need to wait for the next period in
integration tests, and we can't do that if the
time doesn't advance
2024-02-22 09:30:50 +01:00
Mark Spanbroek
d442cc80d7
clock: add 1 second leeway before acting on timeouts 2024-02-22 09:30:48 +01:00
Mark Spanbroek
ed47edfe69
market: use pending blocktag when querying onchain state 2024-02-22 06:58:50 +01:00
Mark Spanbroek
906779fba0
contracts: cleanup 2024-02-21 14:23:26 +01:00
Mark Spanbroek
1c04c4d1ff
contracts: update G2Point definition 2024-02-21 14:23:26 +01:00
Eric
8dd0a420c6
Update integration tests to use cli commands
Integration tests have been updated to use the new cli commands. The api for usage in the integration tests has also changed a bit.

Proofs tests have been updated to use 5 nodes and 8 blocks of data. The remaining integration tests also need to be updated.
2024-02-21 16:09:43 +11:00
Dmitriy Ryajov
23275b67ba
merge master 2024-02-19 13:34:46 -06:00
Dmitriy Ryajov
cba998fc12
make verifier a cmd option 2024-02-19 13:34:46 -06:00
Dmitriy Ryajov
d1b2b96383
use correct stores 2024-02-19 13:34:22 -06:00
Dmitriy Ryajov
42d4a5c234
use push raises 2024-02-19 13:33:37 -06:00
Dmitriy Ryajov
d75a33af49
styling 2024-02-19 13:33:37 -06:00
Dmitriy Ryajov
4c69d3f43c
styling 2024-02-19 13:33:37 -06:00
Dmitriy Ryajov
b7c736b34c
use parens around =? (await...) calls 2024-02-19 13:33:37 -06:00
Dmitriy Ryajov
3ce68f696e
make errors more searchable 2024-02-19 13:33:37 -06:00
Dmitriy Ryajov
90acd52d1d
use correct dataset geometry in erasure 2024-02-19 13:33:36 -06:00
Dmitriy Ryajov
947b3dadc0
misc 2024-02-19 13:33:36 -06:00
Dmitriy Ryajov
3996d4fb81
misc 2024-02-19 13:33:36 -06:00
Dmitriy Ryajov
b76fe82f79
make block and manifest object instead of ref 2024-02-19 13:33:36 -06:00
Dmitriy Ryajov
65a1f19626
add circom compat to solidity groth16 convertion 2024-02-19 13:33:36 -06:00
Dmitriy Ryajov
5ec9269c05
moving circom params under prover command 2024-02-19 13:33:35 -06:00
Mark Spanbroek
6b301bddb6
Fix: simulateProofFailures option is not for validator 2024-02-19 13:33:34 -06:00
Eric
53f6eae9f6
format challenges in logs 2024-02-19 13:33:34 -06:00
Dmitriy Ryajov
78b7dc93dd
adding usefull traces 2024-02-19 13:33:24 -06:00
Dmitriy Ryajov
5b58a137b1
don't use ufcs 2024-02-19 13:33:07 -06:00
Dmitriy Ryajov
83c3aa07d2
use a real verifying contract address 2024-02-19 13:33:07 -06:00
Dmitriy Ryajov
1f7861b39a
misc renames 2024-02-19 13:32:40 -06:00
Dmitriy Ryajov
1b2ac806b1
return correct proof 2024-02-19 13:32:11 -06:00
Dmitriy Ryajov
f6e7f12c1a
merge rework-cli into node-wire-prover 2024-02-19 13:31:05 -06:00
Dmitriy Ryajov
4deeed64ef
add circom params (temprorary) 2024-02-19 13:31:05 -06:00
Dmitriy Ryajov
e271875f6a
wire prover into node 2024-02-19 13:31:03 -06:00
Dmitriy Ryajov
6cbadc6079
check circom files extension 2024-02-19 13:29:35 -06:00
Dmitriy Ryajov
2e0f087389
rework cli to accept circuit params 2024-02-19 13:29:32 -06:00
Dmitriy Ryajov
9e13d2251a
Prover works with zkey (#711)
* rework backend to instantiate key at initialization

* add groth16 convertes for solidity

* prover taks num samples on construction

* add zkey file

* rework helpers

* rename types

* update tests

* reworked test helpers

* rename types

* rework test

* test all slots artifacts

* bump to latest version
2024-02-19 18:58:39 +00:00
Dmitriy Ryajov
3e884430c1
Node use correct stores (#710)
* don't pass erasure

* use correct stores and construct erasure inside the node

* fix tests to match new constructor

* remove prover argument

* review commets

* revert failing on no-prover for now

* small cleanup

* comment out invalid proofs broken test
2024-02-19 10:12:10 -08:00
Eric
d70ab59004
refactor: multinode integration test refactor (#662)
* refactor multi node test suite

Refactor the multinode test suite into the marketplace test suite.

- Arbitrary number of nodes can be started with each test: clients, providers, validators
- Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file.
- Log files for all nodes can be persisted on disk, with configuration at the test-level
- Log files, if persisted (as specified in the test), will be persisted to a CI artifact
- Node config is specified at the test-level instead of the suite-level
- Node/Hardhat process starting/stopping is now async, and runs much faster
- Per-node config includes:
  - simulating proof failures
  - logging to file
  - log level
  - log topics
  - storage quota
  - debug (print logs to stdout)
- Tests find next available ports when starting nodes, as closing ports on Windows can lag
- Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests).
  - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively.
  - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively.

* additional logging for debug purposes

* address PR feedback

- fix spelling
- revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR.
- remove method label from raiseAssert
- remove unused import

* Use API instead of command exec to test for free port

Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs.

* clean up

* remove upraises annotations from tests

* Update tests to work with updated erasure coding slot sizes

* update dataset size, nodes, tolerance to match valid ec params

Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage.

Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params).

All integration tests pass when the async `clock.now` changes are reverted.

* dont use async clock for now

* fix workflow

* move integration logs uplod to reusable

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
Dmitriy Ryajov
0497114e44
Use correct blocks count in builder (#709) 2024-02-19 02:19:59 +00:00
Dmitriy Ryajov
3fce267352
Block and manifest as objects (#708)
* make block and manifest objects

* use unsafeAddr since block is an object type now

* make object var
2024-02-19 02:10:16 +00:00
Eric
61ec4275c8
Revert "remove call to waitFor from clock.now and make it async (#691)" (#707)
This reverts commit 591be9446a5cee1ce559c87e496fa4d7dab92924.
2024-02-16 22:12:16 +00:00
Dmitriy Ryajov
6fab252bf1
safer backend release (#701) 2024-02-10 23:17:11 +00:00
Dmitriy Ryajov
e23159b065
wire in circom backend (#698)
* wire in circom backend

* should contain leafs

* adding circom compad and circuits deps

* update windows build

* fix windows build

* improve test names

* move proving defaults to codextypes

* remove unnedded inmports and move defaults to codextypes

* capture error code on backend failure
2024-02-09 21:40:30 +00:00
Dmitriy Ryajov
825766eea0
Rework builder & sampler (#697)
* add proof initializer for poseidon2 and `$`

* fix padding and block selection

* fix sample selection and input construction

* fix sample selection & more descriptive names

* add concrete types for sampler & builder

* add missing digest calls

* use concrete types

* add sample test fixtures

* use concrete types and don't fetch dummy blocks
2024-02-08 02:27:11 +00:00