Commit Graph

599 Commits

Author SHA1 Message Date
gmega 13d4e1453f
drop non-ref objects of RootObj to avoid triggering broken sink/move in refc 2024-02-23 11:43:17 -03:00
gmega fca28f672b
wip 2024-02-22 11:52:45 -03:00
gmega 4f3802b1ec
Merge branch 'master' into feat/chronos-v4 2024-02-22 11:18:26 -03:00
Slava 1eebaa4ce3
ci: use latest amd64 macos runner (#718)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-02-21 22:10:37 +00:00
Slava c99085c396
Set environment variables from files in Docker (#717)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-02-21 20:50:57 +00:00
gmega a6d6464efd
Merge branch 'master' into feat/chronos-v4 2024-02-20 18:47:27 -03:00
Dmitriy Ryajov 9e13d2251a
Prover works with zkey (#711)
* rework backend to instantiate key at initialization

* add groth16 convertes for solidity

* prover taks num samples on construction

* add zkey file

* rework helpers

* rename types

* update tests

* reworked test helpers

* rename types

* rework test

* test all slots artifacts

* bump to latest version
2024-02-19 18:58:39 +00:00
Slava 6b86601597
ci: update actions to the latest major versions (#712)
* ci: update actions to the latest major versions

* ci: add OS to the integration tests logs name

* ci: add CPU and Nim version to the integration tests logs name

* ci: upload integrations tests logs only when they run
2024-02-19 18:55:49 +00:00
Dmitriy Ryajov 3e884430c1
Node use correct stores (#710)
* don't pass erasure

* use correct stores and construct erasure inside the node

* fix tests to match new constructor

* remove prover argument

* review commets

* revert failing on no-prover for now

* small cleanup

* comment out invalid proofs broken test
2024-02-19 10:12:10 -08:00
Eric d70ab59004
refactor: multinode integration test refactor (#662)
* refactor multi node test suite

Refactor the multinode test suite into the marketplace test suite.

- Arbitrary number of nodes can be started with each test: clients, providers, validators
- Hardhat can also be started locally with each test, usually for the purpose of saving and inspecting its log file.
- Log files for all nodes can be persisted on disk, with configuration at the test-level
- Log files, if persisted (as specified in the test), will be persisted to a CI artifact
- Node config is specified at the test-level instead of the suite-level
- Node/Hardhat process starting/stopping is now async, and runs much faster
- Per-node config includes:
  - simulating proof failures
  - logging to file
  - log level
  - log topics
  - storage quota
  - debug (print logs to stdout)
- Tests find next available ports when starting nodes, as closing ports on Windows can lag
- Hardhat is no longer required to be running prior to starting the integration tests (as long as Hardhat is configured to run in the tests).
  - If Hardhat is already running, a snapshot will be taken and reverted before and after each test, respectively.
  - If Hardhat is not already running and configured to run at the test-level, a Hardhat process will be spawned and torn down before and after each test, respectively.

* additional logging for debug purposes

* address PR feedback

- fix spelling
- revert change from catching ProviderError to SignerError -- this should be handled more consistently in the Market abstraction, and will be handled in another PR.
- remove method label from raiseAssert
- remove unused import

* Use API instead of command exec to test for free port

Use chronos `createStreamServer` API to test for free port by binding localhost address and port. Use `ServerFlags.ReuseAddr` to enable reuse of same IP/Port on multiple test runs.

* clean up

* remove upraises annotations from tests

* Update tests to work with updated erasure coding slot sizes

* update dataset size, nodes, tolerance to match valid ec params

Integration tests now have valid dataset sizes (blocks), tolerances, and number of nodes, to work with valid ec params. These values are validated when requested storage.

Print the rest api failure message (via doAssert) when a rest api call fails (eg the rest api may validate some ec params).

All integration tests pass when the async `clock.now` changes are reverted.

* dont use async clock for now

* fix workflow

* move integration logs uplod to reusable

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2024-02-19 04:55:39 +00:00
Dmitriy Ryajov 0497114e44
Use correct blocks count in builder (#709) 2024-02-19 02:19:59 +00:00
Dmitriy Ryajov 3fce267352
Block and manifest as objects (#708)
* make block and manifest objects

* use unsafeAddr since block is an object type now

* make object var
2024-02-19 02:10:16 +00:00
Eric 61ec4275c8
Revert "remove call to `waitFor` from clock.now and make it async (#691)" (#707)
This reverts commit 591be9446a.
2024-02-16 22:12:16 +00:00
Slava 79af5c0e64
Fix Docker entrypoint NAT helper variables (#706)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-02-15 13:57:50 +00:00
Dmitriy Ryajov 6fab252bf1
safer backend release (#701) 2024-02-10 23:17:11 +00:00
Slava 64e6a602fd
Install Rust during Docker builds (#703)
* Install Rust during Docker builds

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Stop CI matrix on error

---------

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-02-10 17:48:40 +00:00
Dmitriy Ryajov e23159b065
wire in circom backend (#698)
* wire in circom backend

* should contain leafs

* adding circom compad and circuits deps

* update windows build

* fix windows build

* improve test names

* move proving defaults to codextypes

* remove unnedded inmports and move defaults to codextypes

* capture error code on backend failure
2024-02-09 21:40:30 +00:00
Dmitriy Ryajov 825766eea0
Rework builder & sampler (#697)
* add proof initializer for poseidon2 and `$`

* fix padding and block selection

* fix sample selection and input construction

* fix sample selection & more descriptive names

* add concrete types for sampler & builder

* add missing digest calls

* use concrete types

* add sample test fixtures

* use concrete types and don't fetch dummy blocks
2024-02-08 02:27:11 +00:00
Dmitriy Ryajov afec86b3cf
cleanup manifest and wire in strategy (#696) 2024-02-07 20:54:57 +00:00
Slava 1e957da109
CI update (#695)
* fix: update codecov action

* fix: remove nim v2 from Nim-matrix
2024-02-07 19:47:40 +00:00
Dmitriy Ryajov 2fc538337b
rework indexing strategy to use an enum (#694) 2024-02-07 11:13:27 -08:00
markspanbroek 2cf892c467
Smart contracts update: Groth16Proof instead of bytes (#683)
* Smart contracts update: Groth16Proof instead of bytes

* Use dummy verifier for now, until we can create ZK proofs

* Fix tests: submit proof only when slot is filled

* Submit dummy proofs for now

* More detailed log when proof submission failed

* Use dummy verifier for integration tests

For now at least

* Fix mistake in blanket renaming to ethProvider

* Update to latest codex-contracts-eth

* feat: zkey-hash from chain

* Fix zkeyHash

---------

Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
2024-02-07 07:50:35 +01:00
gmega f92d069d82
Merge branch 'master' into feat/chronos-v4
* add special repo_version version to the Nim compiler which picks up what's in the repo
2024-02-06 18:27:27 -03:00
Slava 403b9baf9f
Add Nim-matrix workflow to run on merge queue (#693)
* Add Nim-matrix workflow to run on merge queue

* Use reusable workflows for CI and Nim-matrix
2024-02-06 12:56:27 +02:00
Giuliano Mega 591be9446a
remove call to `waitFor` from clock.now and make it async (#691) 2024-02-05 08:36:28 -03:00
Giuliano Mega d1409f9399
Merge branch 'master' into feat/chronos-v4 2024-02-02 11:25:35 -03:00
Slava ca0660b07e
Use default short commit length for Codex revision (#690) 2024-01-30 19:58:55 +02:00
gmega 3802ba3fd6
bump Chronos to the official 4.0.0 release 2024-01-30 09:51:44 -03:00
gmega 0f4b4a0eb9
"fix" export (no idea why it broke in the first place) 2024-01-30 09:51:22 -03:00
gmega 92fc374fe3
Merge branch 'master' into feat/chronos-v4 2024-01-29 20:57:32 -03:00
markspanbroek fd3c566b25
update asynctest to 0.5.1 (#671)
* update asynctest to 0.5.0

Co-Authored-By: gmega <giuliano.mega@gmail.com>

* update remaining package imports

* add asynctest package wrapper

---------

Co-authored-by: gmega <giuliano.mega@gmail.com>
2024-01-29 17:03:51 -03:00
gmega 1f2ede3120
fix typo 2024-01-29 16:59:38 -03:00
gmega 979a7f37c8
Merge branch 'master' into feat/chronos-v4 2024-01-29 16:36:37 -03:00
Slava 95a4484de0
fix: update codex help output (#687) 2024-01-29 19:31:29 +00:00
Slava ae61c29b3f
ci: fix docker workflow after actions update (#689)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-01-29 20:12:07 +02:00
Slava 34095461ae
ci: update actions to lates major versions (#688) 2024-01-29 17:21:52 +02:00
Ben Bierens 0b8811c98d
Debug/sampling (#681)
* Extra logging in sampler

* wip: Fixing sampling issue in padded slot cells

* Cleanup

* Restores tests
2024-01-25 15:34:38 -08:00
gmega e343170c91
remove duplicate import 2024-01-23 10:24:28 -03:00
gmega 56d7b089d3
bump nimbus-build-system 2024-01-23 10:13:23 -03:00
gmega be7ab15064
update env to reflect latest feat/build-improvements 2024-01-23 09:55:07 -03:00
gmega ed8551b23c
bump nimbus-build-system 2024-01-23 09:55:06 -03:00
gmega 13375768d0
add default expansion for NIM_REPO 2024-01-23 09:55:06 -03:00
gmega e8ad522799
bring feat/chronos-v4 up-to-date with feat/build-improvements 2024-01-23 09:55:06 -03:00
gmega 7540cf3b18
add explicit calls to env script :-( 2024-01-23 09:55:06 -03:00
gmega 3c0445daef
source env before running coverage script 2024-01-23 09:55:05 -03:00
gmega 22449fd0a8
rebase, update new files to asynctest 0.5.0, indicies -> indices :-) 2024-01-23 09:55:03 -03:00
gmega fd78727978
backport changes from https://github.com/codex-storage/nim-codex/pull/680 so I can use a custom compiler in dist tests without too much hassle 2024-01-23 09:54:37 -03:00
gmega 7aa922b90f
change signatures to async and add raw: true 2024-01-23 09:54:37 -03:00
gmega ec9a9125d3
change input name to avoid env var clash in Windows; add correct version args into coverage 2024-01-23 09:54:36 -03:00
gmega e5586f3970
bump nimbus-build-system 2024-01-23 09:54:36 -03:00