783 Commits

Author SHA1 Message Date
Eric
aea6c99a69
increase port distance per test run 2025-02-19 15:41:46 +11:00
Eric
7f058deece
add constructor param for continuous status update in terminal
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:45 +11:00
Eric
749f65b134
fix nodeprocess not exiting 2025-02-19 15:41:45 +11:00
Eric
8501220fda
ensure timeouts work
Work around the AsyncProcess timeout not working correctly
2025-02-19 15:41:45 +11:00
Eric
92513e083e
continually show test status updates
allows user to know which tests are still running
2025-02-19 15:41:45 +11:00
Eric
2ef044dfcb
cleanup
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:44 +11:00
Eric
a46bf719ef
lock hardhat start to avoid intermittent start failures 2025-02-19 15:41:44 +11:00
Eric
ff8ac244bd
clean up 2025-02-19 15:41:44 +11:00
Eric
beb270de3d
print hardhat testmanager errors and hardhat stdout when there is an error starting hardhat 2025-02-19 15:41:43 +11:00
Eric
f4712684ba
hang manager off IntegrationTest, cleanup
- add a TestManager property to IntegrationTest, so manager does not need to be passed into all functions
- cleanup:
  - remove unneeded stopHardhat function
  - add hardhat instance to manager outside of startHardhat
2025-02-19 15:41:43 +11:00
Eric
62e538a596
pass hardhat port to ethersuite
# Conflicts:
#	tests/ethertest.nim
2025-02-19 15:41:43 +11:00
Eric
f55831a2c8
reinstate removed types 2025-02-19 15:41:43 +11:00
Eric
0a8487618a
Add additional logging info print outs
# Conflicts:
#	tests/integration/testcli.nim
2025-02-19 15:41:42 +11:00
Eric
34f445993d
expect process exit code
prevents showing error in the logs when an expected process exit code is encountered

# Conflicts:
#	tests/integration/testcli.nim
2025-02-19 15:41:42 +11:00
Eric
6a18238205
stop hardhat and show logs
# Conflicts:
#	tests/integration/hardhatprocess.nim
2025-02-19 15:41:42 +11:00
Eric
e4c035651f
default test name to the filename of the integration test
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:42 +11:00
Eric
cf9134861e
print start/finish stderr/stdout markers
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:41 +11:00
Eric
1400848f86
clean up 2025-02-19 15:41:41 +11:00
Eric
ce4b1d439c
stdout formatting for test results
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:41 +11:00
Eric
7252a32579
prevent errors during node shutdown 2025-02-19 15:41:40 +11:00
Eric
40142fa684
test summary formatting 2025-02-19 15:41:40 +11:00
Eric
93d3de315e
parallel tests running
- need to test with longer tests to ensure the parallelisation is truly happening
- is the +10 hardhat port needed?
- try with more integration tests

# Conflicts:
#	tests/integration/hardhatprocess.nim
#	tests/integration/multinodes.nim
#	tests/integration/testcli.nim
#	tests/testIntegration.nim
2025-02-19 15:41:40 +11:00
Eric
14a223fb36
fix after rebase 2025-02-19 15:41:40 +11:00
Eric
66683a8606
clean up 2025-02-19 15:41:39 +11:00
Eric
3aed1ead53
chore(integration): simplify block expiration integration test 2025-02-19 15:41:32 +11:00
Eric
87590f43ce
fix(statemachine): do not raise from state.run (#1115)
* fix(statemachine): do not raise from state.run

* fix rebase

* fix exception handling in SaleProvingSimulated.prove

- re-raise CancelledError
- don't return State on CatchableError
- expect the Proofs_InvalidProof custom error instead of checking a string

* asyncSpawn salesagent.onCancelled

This was swallowing a KeyError in one of the tests (fixed in the previous commit)

* remove error handling states in asyncstatemachine

* revert unneeded changes

* formatting

* PR feedback, logging updates
2025-02-19 00:18:45 +00:00
Adam Uhlíř
1052dad30c
feat: request duration limit (#1057)
* feat: request duration limit

* Fix tests and duration type

* Add custom error

* Remove merge issue

* Update codex contracts eth

* Update market config and fix test

* Fix SlotReservationsConfig syntax

* Update dependencies

* test: remove doubled test

* chore: update contracts repo

---------

Co-authored-by: Arnaud <arnaud@status.im>
2025-02-18 19:41:54 +00:00
Arnaud
2298a0bf81
Use results instead of stew/results (#1112) 2025-02-18 10:17:05 +00:00
Arnaud
0107eb06fe
chore(marketplace): cid should be bytes (#1073)
* Change cid format from string to bytes for the marketplace

* refactor: marketplace custom errors handling

* chore: update contracts repo

* chore: update contracts submodule

* Update contracts submodule

* Initialize the Cid using init function

* Restorage serialize pragma

* Use Cid object instead of buffer

* Simplify cid usage

* Simplify cid usage

* bump codex-contracts-eth after PR merge, formatting

* fix rebase

* collateralPerByte => collateralPerSlot

---------

Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2025-02-18 05:47:47 +00:00
Arnaud
6e73338425
Remove deprecated function (#1111)
Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2025-02-17 22:04:04 +00:00
Eric
5af3477793
chore(deps): bump ethers to propagate cancellations (#1116)
* chore(deps): bump ethers to propagate cancellations

Ethers was swallowing canellations and turning them into EthersErrors, which was causing the sales statemachine to error when it should have been simply cancelling the current state's run. Hopefully fixes the intermittently failing marketplace integration test.

* Add missing errors in async raises pragma

* bump to version of ethers that supports cancellations

---------

Co-authored-by: Arnaud <arnaud@status.im>
2025-02-17 22:00:52 +00:00
Arnaud
dc08ff8840
chore(marketplace): add a cache for storage requests (#1090)
* Add cache to for requests

* Change request cache description message and use const as default value

* Set request cache size configuration hidden
2025-02-17 10:34:42 +00:00
Giuliano Mega
25c84f4e0e
Fix/repostore deletes for non-orphan blocks (#1109)
* fix: fix deletion of non-orphan blocks

* feat: improve error feedback for illegal direct block deletes

* chore: minor rewording of test header
2025-02-14 13:34:17 +00:00
munna0908
c65148822e
feat: multithreading support for erasure coding (#1087)
* implement async encode

* implement async decode

* cleanup code

* add num-threads flag

* fix tests

* code cleanup

* improve return types and exception handling for async proc

* add validation check for numThreads flag

* modify encode method

* add new tests for aync encoding

* modify decode method

* cleanup test cases

* add new cli flag for threadCount

* test cleanup

* add new tests

* fix decodeAsync exception handling

* code cleanup

* chore: cosmetic changes
2025-02-12 17:56:26 +00:00
Dmitriy Ryajov
45e97513a7
remove uploadedAt from manifest (#1091)
* remove uploadedAt from manifest

* fix test
2025-02-12 10:48:58 +00:00
Slava
20f6fef7ab
fix: use ubuntu-24.04 runners for docker workflows (#1102)
Co-authored-by: Giuliano Mega <giuliano.mega@gmail.com>
2025-02-11 21:49:37 +00:00
Giuliano Mega
bbe1f09cd7
Purging of local datasets (#1103)
* feat(codex-node): add dataset deletion API to Codex node

* feat(api): add deletion of local datasets to API

* fix: logging, remove garbage, drop some CORS headers from DELETE request

* fix: change empty response return code to 204 instead of 200

* fix: add time-based idling to avoid locking up the node during deletes, fix API status code

* fix: uncomment commented tests committed by accident

* fix: return correct code when missing CID is a Manifest CID; add back CORS headers

* fix: remove lingering echo
2025-02-11 19:00:05 +00:00
Marcin Czenko
11888e78d7
deploy openapi spec only when tagged (#1106) 2025-02-11 15:16:45 +00:00
Marcin Czenko
8880ad9cd4
fix linting in "codex/blockexchange/engine/engine.nim" (#1107) 2025-02-11 10:47:25 +00:00
Eric
dfa90a9981
fix(build): compilation on macos when including nim-nat-traversal (#1084)
* fix(build): compilation on macos when including nim-nat-traversal

- removes the `VERSION` rename to `VERSION_temp` in the Makefile
- instead, relies on `-iqoute` to include the `nim-nat-traversal/vendor/libnatpmp-upstream` directory in the search paths. `-iquote` will match the `vendor/libnatpmp-upstream/VERSION` file for `#include "version"` and not `#include <version>`, the latter being what is included by the macos sdk and was causing issues with `-I`. The [gcc 14.2 docs](https://gcc.gnu.org/onlinedocs/gcc-14.2.0/cpp/Invocation.html#index-I) describe how `-iquote` alleviates this issue:
> Directories specified with -iquote apply only to the quote form of the directive, #include "file". Directories specified with -I, -isystem, or -idirafter apply to lookup for both the #include "file" and #include <file> directives.

For more info, please see https://github.com/status-im/nim-nat-traversal/pull/34.

* bump nim-nat-traversal

Now that https://github.com/status-im/nim-nat-traversal/pull/34 has been merged, change back to master commit
2025-02-07 01:18:00 +00:00
Dmitriy Ryajov
17d3f99f45
use a case-of instead of if for better readability (#1063) 2025-02-06 21:36:35 +00:00
Csaba Kiraly
e62a09d9b1
add ccache and sccache to speed up CI (#1074)
* add ccache and sccache to speed up CI

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* include testname and nim version in cache separation

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* Make sure ccache has precedence over custom clang/llvm

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* enable ccache for windows

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* ccache: evict old files

Make sure old unused cache files are not lingering around for long

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

---------

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2025-02-06 21:36:14 +00:00
Marcin Czenko
c05eec422c
fix dataset and slot size calculations in integration tests (#1095)
* fixes datasetSize and slotSize helpers (and also RandomChunker.example)

* adds overload for <<upload>> for seq[byte]

* changes RandomChunker.example to return seq[byte]

* fixes restapi tests after correcting RandomChunker.example

* review: use string.fromBytes from nim-stew to convert seq[byte] to string
2025-02-06 15:21:12 +00:00
Slava
54d499be41
docker: add BOOTSTRAP_NODE_URL to Docker entrypoint (#1098) 2025-02-04 21:22:34 +00:00
Giuliano Mega
0595723f66
Minor improvements to download API (#1092)
* chore: improve error messages in upload API

* chore: remove unreachable (dead) code

* fix: API integration test
2025-02-04 16:01:14 +00:00
Arnaud
58a962add8
Update contracts and add custom errors (#1088) 2025-01-28 11:16:24 +00:00
Marcin Czenko
962fc1cd95
Feat: price per byte (#1078)
* openAPI: StorageRequestCreation: reward => pricePerByte, collateral => collateralPerByte

* purchasing: reward => pricePerByte, collateral => collateralPerByte

* Updates availabilities and reservations to use totalCollateral, minPricePerByte, and maxCollateralPerByte

* Uses correct div operator when operating on UInt256

* proposal updating totalCollateral in availability

* makes sure that reading currentCollateral happens before freeing slot

* Updates naming

* fixes tests: unit and contracts

* uses feat/price-per-byte branch for codex-contracts-eth

* temporarily disables integration tests on CI

* introduces high level <<totalCollateral>> property for a cleaner external interface

* updates integration tests

* Applies review comments

* Updates description of totalCollateral in SalesAvailability

* updates codex-contracts-eth (price-per-byte)
2025-01-24 17:18:00 +00:00
markspanbroek
f6c792de79
fix slotqueue worker starvation (#1081)
* fix slotqueue worker starvation

* improve slotqueue tests

Co-Authored-By: Marcin Czenko <marcin.czenko@pm.me>

* slotqueue nph formatting

---------

Co-authored-by: Marcin Czenko <marcin.czenko@pm.me>
2025-01-23 09:28:14 +00:00
Adam Uhlíř
1c4184f29c
chore: ignoring style formatting commit for git blame (#1085) 2025-01-22 09:05:47 +00:00
Adam Uhlíř
e5df8c50d3
style: nph formatting (#1067)
* style: nph setup

* chore: formates codex/ and tests/ folder with nph 0.6.1
2025-01-21 20:54:46 +00:00