114 Commits

Author SHA1 Message Date
Arnaud
93997df6a7
Enable temporary artifacts generation on push 2025-12-19 06:38:59 +01:00
Arnaud
23233dc8e2
Fix cache 2025-12-19 06:38:59 +01:00
Arnaud
728845b5f4
Add artifacts upload for c binding 2025-12-19 06:38:58 +01:00
Arnaud
887991cab3
Remove cbinding from reusable ci 2025-12-19 06:38:58 +01:00
Arnaud
6156082400
Add cache for cbinding 2025-12-19 06:38:58 +01:00
Arnaud
6d7f8fab98
Separatre cbinding into another task 2025-12-19 06:38:57 +01:00
Arnaud
993674fc8d
Continue on error on CI 2025-12-19 06:38:57 +01:00
Arnaud
1fd1cc10bc
Add tests to CI 2025-12-19 06:38:57 +01:00
Arnaud
60861d6af8
chore: rename codex to logos storage (#1359) 2025-12-18 17:23:09 +00:00
Eric
49e801803f
ci: remove dist tests and devnet deployment (#1338) 2025-12-17 06:03:59 +00:00
Eric
6147a751f1
fix(ci): Remove macos amd release build (#1337) 2025-11-13 05:37:43 +00:00
Eric
7aca2f0e61
fix(ci): Move conventional commits job to workflow (#1340) 2025-11-02 04:00:55 +00:00
Slava
d1f2e2399b
ci: validate pr title to adhere conventional commits (#1254) 2025-08-12 08:51:41 +00:00
Slava
8cd10edb69
ci: auto deploy codex on devnet (#1302) 2025-07-28 10:02:19 +00:00
Slava
6cf99e255c
ci: release master builds and upload them to the cloud (#1298) 2025-07-10 11:17:11 +00:00
Slava
9ef9258720
chore(ci): bump node to v22 (#1285) 2025-06-26 01:11:00 +00:00
markspanbroek
01615354af
refactor(ci): run integration tests in parallel by spinning up more runners (#1287) 2025-06-25 08:56:16 +00:00
markspanbroek
4d44154a40
fix(ci): remove "update" to gcc-14 on windows (#1288) 2025-06-24 09:00:56 +00:00
Arnaud
7b660e3554
chore(marketplace): use hardhat ignition (#1195) 2025-06-20 15:55:00 +00:00
Slava
93e4e0f177
ci(docker): add stable tag for dist-tests images (#1273) 2025-06-16 16:22:09 +00:00
Adam Uhlíř
f267d99ea8
ci: docker stable tag (#1265) 2025-06-11 13:49:39 +00:00
Giuliano Mega
d59c5b023c
chore: bump Nim to 2.2.4 (#1242)
* chore: bump Nim to 2.2.4

* fix: resolve symbol ambiguity and drop auto type

* fix: use reference to task instead of pointer or the compiler will deallocate `task` before the encoding/decoding is done

* fix: convention that maxCollateralPerByte equals totalRemainingCollateral when freeSize is 0 to avoid DivByZeroDefect

* fix: bump compiler version in CI pipeline as well
2025-05-29 16:37:38 -07:00
Slava
13811825b3
ci: use macos arm runners (#1174)
* ci: use inputs instead of matrix in a ccache key

* ci: switch to arm runners for macos

* ci: use node 20

* ci: pass cpu to a composite action
2025-05-29 10:17:46 +00:00
Adam Uhlíř
d220e53fe1
ci: trigger python generator upon release (#1208) 2025-04-22 14:46:03 +00:00
Slava
22f5150d1d
ci: add compatible contracts image for nim-codex dist-tests docker image (#1204) 2025-04-18 14:21:24 +00:00
Adam Uhlíř
7c7871ac75
ci: add compatible contracts image for nim-codex docker image (#1186)
* ci: add compatible contracts image for nim-codex docker image

* ci: with submodules

* ci: with submodules on correct place

* ci: remove double dash

* ci: avoiding artifact conflicts

* ci: add labels to arch images

* ci: correct way to add label to arch images

* ci: correct contract label

* ci: avoid building contracts image and use contracts commit hash

* refactor: better way to get the hash
2025-04-15 13:52:19 +00:00
Slava
1213377ac4
ci: switch out from ubuntu 20.04 (#1184)
* ci: use ubuntu-latest for coverage (#1141)

* ci: pass --keep-going to lcov and genhtml (#1141)

* ci: use ubuntu-22.04 for release workflow (#1141)

* ci: install gcc-14 on linux (#1141)

* chore: bump nim-leveldbstatic to 0.2.1
2025-04-02 09:09:43 +00:00
Arnaud
eb09e610d5
fix(ci): handle coverage as a string to enable gcc 14 on linux (#1140)
* Handle coverage as a string not a boolean

* Update ubuntu version to latest
2025-03-05 08:35:46 +00:00
Slava
20f6fef7ab
fix: use ubuntu-24.04 runners for docker workflows (#1102)
Co-authored-by: Giuliano Mega <giuliano.mega@gmail.com>
2025-02-11 21:49:37 +00:00
Marcin Czenko
11888e78d7
deploy openapi spec only when tagged (#1106) 2025-02-11 15:16:45 +00:00
Csaba Kiraly
e62a09d9b1
add ccache and sccache to speed up CI (#1074)
* add ccache and sccache to speed up CI

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* include testname and nim version in cache separation

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* Make sure ccache has precedence over custom clang/llvm

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* enable ccache for windows

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* ccache: evict old files

Make sure old unused cache files are not lingering around for long

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

---------

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2025-02-06 21:36:14 +00:00
Adam Uhlíř
e5df8c50d3
style: nph formatting (#1067)
* style: nph setup

* chore: formates codex/ and tests/ folder with nph 0.6.1
2025-01-21 20:54:46 +00:00
Slava
833e253baa
feat: switch to github arm runners (#1077)
* feat: switch to github arm runners

* chore: fabiocaccamo/create-matrix-action
2025-01-20 14:42:01 +00:00
Arnaud
f25c555d59
Chore/update nim version (#1052)
* Move to version 2.0.6

* Update nim-confutils submodule to latest version

* Update dependencies

* Update Nim version to 2.0.12

* Add gcsafe pragma

* Add missing import

* Update specific conf for Nim 2.x

* Fix method signatures

* Revert erasure coding attempt to fix bug

* More gcsafe pragma

* Duplicate code from libp2p because it is not exported anymore

* Fix camelcase function names

* Use alreadySeen because need is not a bool anymore

* newLPStreamReadError does not exist anymore so use another error

* Replace ValidIpAddress by IpAddress

* Add gcsafe pragma

* Restore maintenance parameter deleted by mistake when removing esasure coding fix attempt code

* Update method signatures

* Copy LPStreamReadError code from libp2p which was removed

* Fix camel case

* Fix enums in tests

* Fix camel case

* Extract node components to a variable to make Nim 2 happy

* Update the tests using ValidIpAddress to IpAddress

* Fix cast for value which is already an option

* Set nim version to 2.0.x for CI

* Set nim version to 2.0.x for CI

* Move to miniupnp version 2.2.4 to avoid symlink error

* Set core.symlinks to false for Windows for miniupnp >= 2.2.5 support

* Update to Nim 2.0.14

* Update CI nim versions to 2.0.14

* Try with GCC 14

* Replace apt-fast by apt-get

* Update ubuntu runner to latest

* Use Ubuntu 20.04 for coverage

* Disable CI cache for coverage

* Add coverage property description

* Remove commented test

* Check the node value of seen instead of using alreadySeen

* Fix the merge. The taskpool work was reverted.

* Update nim-ethers submodule

* Remove deprecated ValidIpAddress. Fix missing case and imports.

* Fix a weird issue where nim-confutils cannot find NatAny

* Fix tests and remove useless static keyword
2025-01-10 14:12:37 +00:00
Slava
6d415b0ace
ci: split nim-matrix workflow (#1041) 2024-12-15 23:31:55 +00:00
Marcin Czenko
92a0eda79a
Validator historical state restoration (#922)
* adds a new overload of queryPastEvents allowing to query past events based on timestamp in the past

* adds state restoration to validator

* refactors a bit to get the tests back to work

* replaces deprecated generic methods from Market with methods for specific event types

* Refactors binary search

* adds market tests for querying past SlotFilled events and binary search

* Takes into account that <<earliest>> block available is not necessarily the genesis block

* Adds more logging and makes testing earliest block boundary more reliable

* adds validation tests for historical state restoration

* adds mockprovider to simplify and improve testing of the edge conditions

* adds slot reservation to the new tests after rebasing

* adds validation groups and group index in logs of validator

* adds integration test with two validators

* adds comment on how to enable logging in integration test executable itself

* testIntegration: makes list is running nodes injected and available in the body of the test

* validation: adds integration test for historical state

* adds more logging to validator

* integration test: validator only looks 30 days back for historical state

* adds logging of the slotState when removing slots during validation

* review and refactor validator integration tests

* adds validation to the set of integration tests

* Fixes mistyped name of the mock provider module in testMarket

* Fixes a typo in the name of the validation suite in integration tests

* Makes validation unit test a bit easier to follow

* better use of logScopes to reduce duplication

* improves timing and clarifies the test conditions

* uses http as default RPC provider for nodes running in integration tests as a workaround for dropped subscriptions

* simplifies the validation integration tests by waiting for failed request instead of tracking slots

* adds config option allowing selectively to set different provider url

* Brings back the default settings for RPC provider in integration tests

* use http RPC provider for clients in validation integration tests

* fine-tune the tests

* Makes validator integration test more robust - adds extra tracking

* brings tracking of marketplace event back to validator integration test

* refactors integration tests

* deletes tmp file

* adds <<return>> after forcing integration test to fail preliminarily

* re-enables all integration tests and matrix

* stops debug output in CI

* allows to choose a different RPC provider for a given integration test suite

* fixes signature of <<getBlock>> method in mockProvider

* adds missing import which seem to be braking integration tests on windows

* makes sure that clients, SPs, and validators use the same provider url

* makes validator integration tests using http at 127.0.0.1:8545

* testvalidator: stop resubscribing as we are now using http polling as rpc provider

* applying review comments

* groups queryPastStorage overrides together (review comment)

* groups the historical validation tests into a sub suite

* removes the temporary extensions in marketplacesuite and multinodesuite allowing to specify provider url

* simplifies validation integration tests

* Removes debug logs when waiting for request to fail

* Renaming waitForRequestFailed => waitForRequestToFail

* renames blockNumberForBlocksAgo to pastBlockTag and makes it private

* removes redundant debugging logs

* refines logging in validation

* removes dev logging from mockmarket

* improves exception handling in provider helper procs and prepares for extraction to a separate module

* Uses chronos instead of std/times for Duration

* extracts provider and binary search helpers to a separate module

* removes redundant log entry params from validator

* unifies the notation to consistently use method call syntax

* reuses ProviderError from nim-ethers in the provider extension

* clarifies the comment in multinodesuite

* uses == operator to check the predefined tags and raises exception when `BlockTag.pending` is requested.

* when waiting for request to fail, we break on any request state that is not Started

* removes tests that were moved to testProvider from testMarket

* extracts tests that use MockProvider to a separate async suite

* improves performance of the historical state restoration

* removing redundant log messages in validator (groupIndex and groups)

* adds testProvider to testContracts group

* removes unused import in testMarket
2024-12-14 05:07:55 +00:00
Slava
921159f87f
Run release tests for docker images (#1017)
* Refactor Docker reusable workflow and add release tests support

https://github.com/codex-storage/cs-codex-dist-tests/issues/108

* Add an option to run release tests for Docker images

https://github.com/codex-storage/cs-codex-dist-tests/issues/108

* Use bigger instance for arm builds

https://github.com/codex-storage/cs-codex-dist-tests/issues/108

* Pass repository and branch to release tests workflow

https://github.com/codex-storage/cs-codex-dist-tests/issues/108

* Do not use computation job because run_release_tests is a string

https://github.com/codex-storage/cs-codex-dist-tests/issues/108
2024-12-04 11:52:51 +00:00
Slava
d12de20868
ci: use rust 1.7.9 for release workflow and dockerfile (#999)
* ci: use rust 1.7.9 for release workflow (#996)

* docker: use rust 1.7.9 for docker (#996)
2024-11-25 16:13:14 +00:00
markspanbroek
29433bad9a
Fix concurrency issues (#993)
* Use http subscriptions instead of websocket for tests

To work around this issue when subscriptions are
inactive for more than 5 minutes:
https://github.com/NomicFoundation/hardhat/issues/2053

Use 100 millisecond polling; default polling interval
of 4 seconds is too close to the 5 second timeout for
`check eventually`.

* use .confirm(1) instead of confirm(0)

confirm(0) doesn't wait at all, confirm(1) waits
for the transaction to be mined

* speed up partial payout integration test

* update nim-ethers to version 0.10.0

includes fixes for http polling and .confirm()

* fix timing of marketplace tests

allow for a bit more time to withdraw funds

* use .confirm(1) in marketplace tests

to ensure that the transaction has been processed
before continuing with the test

* fix timing issue in validation unit test

* fix proof integration test

there were two logic errors in this test:
- a slot is freed anyway at the end of the contract
- when starting the request takes a long time, the
  first slot can already be freed because there were
  too many missing proofs

* fix intermittent error in contract tests

currentTime() doesn't always correctly reflect
the time of the next transaction

* reduce number of slots in integration test

otherwise the windows runner in the CI won't
be able to start the request before it expires

* fix timing in purchasing test

allow for a bit more time for a request to
be submitted

* fix timing of request submission in test

windows ci is so slow, it can take up to 40 seconds
just to submit a storage request to hardhat

* increase proof period to 90 seconds

* adjust timing of integration tests

reason: with the increased period length of 90 seconds, it
can take longer to wait for a stable challenge at the
beginning of a period.

* increase CI timeout to 2 hours

* Fix slow builds on windows

apparently it takes windows 2-3 seconds to
resolve "localhost" to 127.0.0.1 for every
json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
Slava
6038fb456e
ci: split linux and macos tests (#997)
* Split Linux and macOS tests

* Make jobs names more readable
2024-11-22 12:05:00 +00:00
Slava
71b8a95d12
ci: install rust 1.7.9 as required by packages (#996)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-11-22 06:34:10 +00:00
Slava
a2ac7453fa
Build Postman Collection (#973)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-10-28 13:53:41 +00:00
Adam Uhlíř
7c33473c88
ci: linux ci runs on ubuntu-20.04 (#953)
* ci: linux ci runs uses ubuntu-20.04

* ci: use ubuntu-20.04 for nim-matrix

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

---------

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
Co-authored-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-10-14 11:24:53 +00:00
Slava
91e4e368de
Use Ubuntu 20.04 for Linux amd64 releases (#939)
* Use Ubuntu 20.04 for Linux amd64 releases (#932)

* Accept branches with the slash in the name for release workflow (#932)

* Increase artifacts retention-days for release workflow (#932)
2024-10-04 11:07:44 +00:00
Slava
e1a02c8b76
Use CLI args when passed for cirdl in Docker entrypoint (#927)
* Use CLI args when passed for cirdl in Docker entrypoint

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Increase CI timeout

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

---------

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-10-01 13:05:22 +00:00
Ben Bierens
4e8630791a
Rework circuit downloader (#882)
* Introduces a start method to prover

* Moves backend creation into start method

* sets up three paths for backend initialization

* Extracts backend initialization to backend-factory

* Implements loading backend from cli files or previously downloaded local files

* Wires up downloading and unzipping

* functional implementation

* Fixes testprover.nim

* Sets up tests for backendfactory

* includes libzip-dev

* pulls in updated contracts

* removes integration cli tests for r1cs, wasm, and zkey file arguments.

* Fixes issue where inner-scope values are lost before returning

* sets local proof verification for dist-test images

* Adds two traces and bumps nim-ethers

* Adds separate path for circuit files

* Create circuit dir if not exists

* fix: make sure requestStorage is mined

* fix: correct place to plug confirm

* test: fixing contracts tests

* Restores gitmodules

* restores nim-datastore reference

* Sets up downloader exe

* sets up tool skeleton

* implements getting of circuit hash

* Implements downloader tool

* sets up test skeleton

* Implements test for cirdl

* includes testTools in testAll

* Cleanup building.md

* cleans up previous downloader implementation

* cleans up testbackendfactory

* moves start of prover into node.nim

* Fills in arguments in example command

* Initializes backend in prover constructor

* Restores tests

* Restores tests for cli instructions

* Review comments by Dmitriy, part 1

* Quotes path in download instruction.

* replaces curl with chronos http session

* Moves cirdl build output to 'build' folder.

* Fixes chronicles log output

* Add cirdl support to the codex Dockerfile

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Add cirdl support to the docker entrypoint

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Add cirdl support to the release workflow

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Disable verify_circuit flag for releases

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Removes backendFactory placeholder type

* wip

* Replaces zip library with status-im/zippy library (which supports zip and tar)

* Updates cirdl to not change circuitdir folder

* Switches from zip to tar.gz

* Review comments by Dmitriy

* updates codex-contracts-eth

* Adds testTools to CI

* Adds check for access to config.circuitdir

* Update fixture circuit zkey

* Update matrix to run tools tests on Windows

* Adds 'deps' dependency for cirdl

* Adjust docker-entrypoint.sh to use CODEX_CIRCUIT_DIR env var

* Review comments by Giuliano

---------

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
Co-authored-by: Veaceslav Doina <20563034+veaceslavdoina@users.noreply.github.com>
2024-09-23 14:37:17 +00:00
Slava
b7934fc686
Downgrade to gcc 13 on Windows (#874)
* Downgrade to gcc 13 on Windows

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Increase build job timeout to 90 minutes

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

---------

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-08-10 09:50:05 +00:00
Slava
8f740b42e6
Update Release workflow (#858)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-07-10 18:14:28 +00:00
Giuliano Mega
b004ca75f6
Bump Nim to 1.6.21 (#851)
* bump Nim to 1.6.21 (range type reset fixes)

* remove incompatible versions from compiler matrix
2024-06-27 19:58:27 +00:00
Slava
ad53066131
ci: add verify_circuit=true to the releases (#840) 2024-06-25 18:31:50 +00:00