Commit Graph

667 Commits

Author SHA1 Message Date
benbierens 1524803fb0
Merge branch 'master' into feature/ceremony-files 2024-04-02 20:12:00 +02:00
Giuliano Mega 53809129a5
Disable Discv5 bucket and routing table IP limits (#764)
* disable Discv5 bucket and routing table IP limits

* actually configure the discovery protocol
2024-04-02 12:31:52 +00:00
Jaremy Creechley 0a34c37c87
Add Node SPR endpoint (#760)
* Create a node api section, add spr and peerid endpoints, and update twoclient

* tweak docs
2024-04-02 11:13:49 +00:00
benbierens f2f1dd51ef
Create circuit dir if not exists 2024-04-01 10:16:43 +02:00
benbierens 644c83bc6e
Merge branch 'master' into feature/ceremony-files 2024-04-01 08:04:06 +02:00
Ben Bierens 91bb972e6f
use integer instead of number (#762) 2024-03-29 18:18:41 +00:00
Eric dca9f2ee76
Use circuit artifacts in codex circuit fixtures for tests (#759) 2024-03-29 17:15:21 +00:00
Ben 4bd14a498c
Adds separate path for circuit files 2024-03-29 16:03:18 +01:00
Ben Bierens c97f973410
Adds two traces and bumps nim-ethers (#761) 2024-03-29 11:51:17 +00:00
Ben 61ea685d4a
Merge branch 'fix/wait-for-sync' into feature/ceremony-files 2024-03-28 14:03:09 +01:00
Ben 445c232259
Adds two traces and bumps nim-ethers 2024-03-28 13:28:22 +01:00
Ben cd280d468d
Merge branch 'master' into feature/ceremony-files 2024-03-27 08:09:51 +01:00
Ben Bierens 8378ac4dac
Openapi mismatches (#754)
* Aligns response with openapi spec

* Fixes json serialization for local content response

* Review comments by Mark

* Fixes missed rename

* Removes array type for dataList
2024-03-26 15:12:15 +00:00
Ben 455b95dbc9
Merge branch 'master' into feature/ceremony-files
# Conflicts:
#	docker/codex.Dockerfile
2024-03-26 08:28:06 +01:00
Ben Bierens 4038eed3c9
Copies openapi yaml into image (#751) 2024-03-25 15:23:30 +00:00
Ben c219a5f0a3
Merge branch 'master' into feature/ceremony-files
# Conflicts:
#	codex/codex.nim
2024-03-25 09:32:03 +01:00
Tomasz Bekas 59d9439ae9
Scheduling erasure coding on another thread (#716)
* Scheduling erasure coding on another thread

* Code review fixes

* Fix for review comments

* Fix missing import

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-03-23 09:56:35 +00:00
Ben 1b6244c34b
sets local proof verification for dist-test images 2024-03-22 14:56:45 +01:00
Ben 2a34c480c3
Merge branch 'master' into feature/ceremony-files 2024-03-22 11:04:35 +01:00
Adam Uhlíř de1714ed06
feat(api): availabilities patch endpoint (#692)
* feat(api): availabilities patch and reservations endpoints

* test: fixing tests and writing

* test: test reservations endpoint

* chore: feedback implementation

* chore: feedback implementation

* test: fix integration tests
2024-03-21 10:53:45 +00:00
Eric 43f0bf1c1c
refactor: add MarketError (#670)
* Add MarketError

Add MarketError and convert all EthersErrors (ProviderError, SignerError) to MarketError

* Include token contract call in conversion of ethers error
2024-03-21 00:30:04 +00:00
Ben e4ddb942a1
Merge branch 'master' into feature/ceremony-files
# Conflicts:
#	.gitmodules
2024-03-20 11:38:32 +01:00
Eric 916e4834d0
Remove codex_use_hardhat compilation flag (#741)
This flag was originally used to change OnChainClock behavior when using hardhat as an ethereum chain source due to a very strange bug which would mark the timestamp of new blocks as one second off the timestamp they should have been.

The issue has since been worked around in another manner, and thus this flag is no longer needed.
2024-03-19 11:08:54 +00:00
Eric f567f4ec15
refactor: use serde for json de/serialization instead of utils/json (#704)
* json > nim-serde bump

Should wait until serde is integrated into nim-ethers before making these changes as there will be less import exceptions required.

* bump nim-serde

* change func to proc due to chronicles side effects

* import serde into utils/json, use as proxy

import nim-serde into utils/json and use utils/json as a proxy for serde functions, including overloading `%` and `fromJson` for application types.

* update tests to use serde

* bump serde to latest

* remove testjson -- no longer needed

* bump serde in nimble

* updates to reconcile rebase with master
2024-03-19 03:25:13 +00:00
Giuliano Mega b4de53f436
update path for libp2p (#743) 2024-03-18 22:41:22 +00:00
Ben Bierens 8f65f578db
Pulls in latest contracts. (#742) 2024-03-18 13:41:25 +00:00
Ben 4379461b5e
Merge branch 'master' into feature/ceremony-files 2024-03-18 09:06:19 +01:00
Giuliano Mega d33804f700
fixes double lookups when block does not exist (#739)
* fixes double lookups when block does not exist

* handle timeouts on requestBlock

* fix indentation which was causing an integration test to fail
2024-03-15 21:50:56 +00:00
Ben ba0e7d9e2a
Fixes issue where inner-scope values are lost before returning 2024-03-15 15:45:26 +01:00
benbierens 50ab23d4e0
removes integration cli tests for r1cs, wasm, and zkey file arguments. 2024-03-12 19:15:53 +01:00
benbierens 09796a7075
pulls in updated contracts 2024-03-12 19:15:52 +01:00
benbierens 97bc1f8676
includes libzip-dev 2024-03-12 19:15:29 +01:00
benbierens 3d008c71fc
Sets up tests for backendfactory 2024-03-12 19:13:07 +01:00
benbierens 8ff6b50c37
Fixes testprover.nim 2024-03-12 19:13:07 +01:00
benbierens 983da3c24f
functional implementation 2024-03-12 19:13:06 +01:00
benbierens 7934d415cd
Wires up downloading and unzipping 2024-03-12 19:13:06 +01:00
benbierens c67c2e7cd7
Implements loading backend from cli files or previously downloaded local files 2024-03-12 19:13:06 +01:00
benbierens 83e1347038
Extracts backend initialization to backend-factory 2024-03-12 19:13:06 +01:00
benbierens 63ab4c5064
sets up three paths for backend initialization 2024-03-12 19:13:06 +01:00
benbierens 828484a646
Moves backend creation into start method 2024-03-12 19:13:05 +01:00
benbierens cfca6b4111
Introduces a start method to prover 2024-03-12 19:12:52 +01:00
markspanbroek 4d069599c9
Use real verifier (#737)
* use a real verifying contract address

* contracts: cleanup

* marketplacesuite: set correct circuit files, interval mining

* Proofs tests updates

Contains changes to get the proving tests working reliably.

* integration: use correct circom artifacts for creating proofs

* integration: cleanup

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2024-03-12 14:20:42 +00:00
markspanbroek b3e57a37e2
Wire up prover (#736)
* wire prover into node

* stricter case object checks

* return correct proof

* misc renames

* adding usefull traces

* fix nodes and tolerance to match expected params

* format challenges in logs

* add circom compat to solidity groth16 convertion

* update

* bump time to give nodes time to load with all circom artifacts

* misc

* misc

* use correct dataset geometry in erasure

* make errors more searchable

* use parens around `=? (await...)` calls

* styling

* styling

* use push raises

* fix to match constructor arguments

* merge master

* merge master

* integration: fix proof parameters for a test

Increased times due to ZK proof generation.
Increased storage requirement because we're now hosting
5 slots instead of 1.

* sales: calculate initial proof at start of period

reason: this ensures that the period (and therefore
the challenge) doesn't change while we're calculating
the proof

* integration: fix proof parameters for tests

Increased times due to waiting on next period.
Fixed data to be of right size.
Updated expected payout due to hosting 5 slots.

* sales: wait for stable proof challenge

When the block pointer is nearing the
wrap-around point, we wait another period
before calculating a proof.

* fix merge conflict

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2024-03-12 12:10:14 +00:00
markspanbroek 293c676f22
Prover CLI updates (#735)
* rework cli to accept circuit params

* check circom files extension

* adding new required cli changes

* don't use ufcs

* persistence is a command now

* use `nimOldCaseObjects` switch for nim confutils compat

* misc

* Update cli integration tests

* Fix: simulateProofFailures option is not for validator

* moving circom params under `prover` command

* update tests

* Use circuit assets from codex-contract-eth in tests

* Add "prover" cli command to tests

* use correct stores

* make `verifier` a cmd option

* update circuit artifacts path

* fix cli tests

* Update integration tests to use cli commands

Integration tests have been updated to use the new cli commands. The api for usage in the integration tests has also changed a bit.

Proofs tests have been updated to use 5 nodes and 8 blocks of data. The remaining integration tests also need to be updated.

* remove parsedCli from CodexConfig

Instead, parse the cli args on the fly when needed

* remove unneeded gcsafes

* graceful shutdowns

Where possible, do not raise assert, as other nodes in the test may already be running. Instead, raise exceptions, catch them in multinodes.nim, and attempt to do a teardown before failing the test.

`abortOnError` is set to true so that `fail()` will quit immediately, after teardown has been run.

* update testmarketplace to new api, with valid EC params

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2024-03-12 09:57:13 +00:00
markspanbroek 8589e63d34
Update contract (#734)
* Update codex-contracts-eth

* contracts: update G2Point definition

* integration: disable automatic advancing of time

reason: makes reasoning about timing in tests harder,
because the period is set to 60 seconds in the
marketplace configuration, but this code switches to
a new period every 500 milliseconds

* integration: fix parameters of marketplace payouts test

* integration: update test settings

* integration: fix typo

* integration: workaround for hardhat issue

Subscriptions expire after 5 minutes when using
websockets. Use http and polling instead.

* integration: remove origDatasetSizeInBlocks

* integration: fix proof parameters for test

* integration: do not log output by default

* integration: fix failure rate in test

* integration: fix warning

* integration: include clock in logs

* integration: allow for more periods

5 periods was cutting it close, if we get too much
pointer downtime, then the test would fail
2024-03-12 08:18:25 +00:00
markspanbroek e654e93c71
Fix sales agent (#733)
* clock: add 1 second leeway before acting on timeouts

* sales: do not raise in proving loop when slot is cancelled

Allow the onCancelled callback to handle cancellation, and
the onFailed callback to handle failed requests.

* sales: cleanup proving tests

* sales: fix sales agent tests

* sales: stop cancellation loop when request started, finished or failed

* sales: fix flaky test

* sales: fix another flaky test

* clock: add comment explaining the + 1 second

Co-Authored-By: benbierens <thatbenbierens@gmail.com>

---------

Co-authored-by: benbierens <thatbenbierens@gmail.com>
2024-03-12 06:41:03 +00:00
markspanbroek d1658d7b77
Fix on-chain clock (#732)
* market: use `pending` blocktag when querying onchain state

* clock: use wall clock in integration tests

reason: we'll need to wait for the next period in
integration tests, and we can't do that if the
time doesn't advance

* clock: remove unused field

* integration: use pending block time to get current time

* clock: fix on-chain clock for hardhat

Only use 'latest' block for updates
Only update the first time you see a block

* integration: do not start tests with a very outdated block

* integration: allow for longer expiry period
2024-03-11 16:57:20 +00:00
markspanbroek 1fc0510f2b
build: update nim-ethers to latest chronosv3 branch (#731)
Fixes validator, which uses transaction simulation to
check whether a proof can be marked as missing.

Fixes gas estimations, so they are done on the pending
block, instead of the latest block.
2024-03-11 15:25:41 +00:00
Giuliano Mega 74679e9e2e
abort Codex when startup sequence fails (#738) 2024-03-11 12:31:17 +00:00
Slava 6ee1162989
Update Docker files and building Codex docs (#722)
* Make Build Codex copy/paste friendly

* Add auto-export for env var from files

* Use latest Ubuntu LTS image and install rust via package manager
2024-03-01 12:23:05 +00:00