793 Commits

Author SHA1 Message Date
Arnaud
e558fd8f94
Enable http pipeline 2025-02-27 10:48:59 +01:00
Arnaud
f06b7ffcae
Add raises errors to async pragram and remove useless try except 2025-02-25 13:58:31 +01:00
Arnaud
42ee85b2f3
Remove test related to timing issue 2025-02-25 08:49:10 +01:00
Arnaud
c1b120cbf7
Merge branch 'master' into feat/persistent-availabilities
# Conflicts:
#	codex/rest/json.nim
#	codex/sales/reservations.nim
#	codex/sales/states/finished.nim
#	codex/sales/states/preparing.nim
#	tests/codex/examples.nim
#	tests/codex/sales/testreservations.nim
#	tests/codex/sales/testsales.nim
#	tests/integration/codexclient.nim
#	tests/integration/testsales.nim
2025-02-24 15:40:34 +01:00
Arnaud
d851b3a9ca
Fix indent 2025-02-24 15:07:50 +01:00
Arnaud
03c1b3331f
Apply suggestion changes: reorganize tests 2025-02-24 11:48:37 +01:00
Arnaud
a18e879705
Apply suggestion changes for until dates 2025-02-24 11:48:37 +01:00
Arnaud
b64aa5042b
Apply suggestion changes: return the reservation module error 2025-02-24 11:48:36 +01:00
Arnaud
c437ccf9a7
Move until validation to reservations module 2025-02-24 11:48:32 +01:00
Arnaud
5b564b14e1
Clean up and fix tests 2025-02-24 11:46:36 +01:00
Arnaud
ff9b01a640
Add until parameter 2025-02-24 11:45:37 +01:00
Arnaud
26079af3d4
Return bytes to availability when finished 2025-02-24 11:41:29 +01:00
Arnaud
2dd0cb4e6a
Add availability enabled parameter 2025-02-24 11:40:57 +01:00
Eric
fc971feecf
bump contracts to master (#1122) 2025-02-24 11:24:50 +01:00
Adam Uhlíř
d2d8f64989
perf: contract storage optimizations (#1094)
* perf: contract storage optimizations

* Apply optimization changes

* Apply optimizing parameters sizing

* Update codex-contracts-eth

* bump latest changes in contracts branch

* Change requestDurationLimit to uint64

* fix tests

* fix tests

---------

Co-authored-by: Arnaud <arnaud@status.im>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2025-02-24 11:24:47 +01:00
Eric
b294af57ba
chore(integration): simplify block expiration integration test (#1100)
* chore(integration): simplify block expiration integration test

* clean up

* fix after rebase
2025-02-24 11:01:38 +01:00
Eric
1220ada0d6
fix(statemachine): do not raise from state.run (#1115)
* fix(statemachine): do not raise from state.run

* fix rebase

* fix exception handling in SaleProvingSimulated.prove

- re-raise CancelledError
- don't return State on CatchableError
- expect the Proofs_InvalidProof custom error instead of checking a string

* asyncSpawn salesagent.onCancelled

This was swallowing a KeyError in one of the tests (fixed in the previous commit)

* remove error handling states in asyncstatemachine

* revert unneeded changes

* formatting

* PR feedback, logging updates
2025-02-24 11:01:38 +01:00
Adam Uhlíř
84b948278a
feat: request duration limit (#1057)
* feat: request duration limit

* Fix tests and duration type

* Add custom error

* Remove merge issue

* Update codex contracts eth

* Update market config and fix test

* Fix SlotReservationsConfig syntax

* Update dependencies

* test: remove doubled test

* chore: update contracts repo

---------

Co-authored-by: Arnaud <arnaud@status.im>
2025-02-24 10:56:56 +01:00
Arnaud
a93fa3b7bc
Use results instead of stew/results (#1112) 2025-02-24 10:56:56 +01:00
Arnaud
d045a7e6e6
Add openapi documentation 2025-02-21 17:42:45 +01:00
Arnaud
dba109fda1
Remove until validation in restapi when creating an availability 2025-02-21 17:09:34 +01:00
Arnaud
2b02d38b45
Call onAvailabilityAdded callabck when the availability is enabled in sales 2025-02-21 17:03:41 +01:00
Arnaud
ada1a6f865
Move until validation to reservations module 2025-02-21 17:03:09 +01:00
Eric
f6aee4ff6e
bump contracts to master (#1122) 2025-02-21 11:02:36 +00:00
Arnaud
c62a023191
Merge branch 'feat/persistent-availabilities' of github.com:codex-storage/nim-codex into feat/persistent-availabilities 2025-02-20 10:09:51 +01:00
Adam Uhlíř
44981d24d0
perf: contract storage optimizations (#1094)
* perf: contract storage optimizations

* Apply optimization changes

* Apply optimizing parameters sizing

* Update codex-contracts-eth

* bump latest changes in contracts branch

* Change requestDurationLimit to uint64

* fix tests

* fix tests

---------

Co-authored-by: Arnaud <arnaud@status.im>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2025-02-20 07:11:06 +00:00
Eric
04327a3986
chore(integration): simplify block expiration integration test (#1100)
* chore(integration): simplify block expiration integration test

* clean up

* fix after rebase
2025-02-20 05:52:51 +00:00
Arnaud
d2ccc4778a
Avoid swallowing CancelledError 2025-02-19 09:38:19 +01:00
Eric
87590f43ce
fix(statemachine): do not raise from state.run (#1115)
* fix(statemachine): do not raise from state.run

* fix rebase

* fix exception handling in SaleProvingSimulated.prove

- re-raise CancelledError
- don't return State on CatchableError
- expect the Proofs_InvalidProof custom error instead of checking a string

* asyncSpawn salesagent.onCancelled

This was swallowing a KeyError in one of the tests (fixed in the previous commit)

* remove error handling states in asyncstatemachine

* revert unneeded changes

* formatting

* PR feedback, logging updates
2025-02-19 00:18:45 +00:00
Adam Uhlíř
1052dad30c
feat: request duration limit (#1057)
* feat: request duration limit

* Fix tests and duration type

* Add custom error

* Remove merge issue

* Update codex contracts eth

* Update market config and fix test

* Fix SlotReservationsConfig syntax

* Update dependencies

* test: remove doubled test

* chore: update contracts repo

---------

Co-authored-by: Arnaud <arnaud@status.im>
2025-02-18 19:41:54 +00:00
Arnaud
f9e5342637
Merge branch 'master' into feat/persistent-availabilities 2025-02-18 11:19:16 +01:00
Arnaud
2298a0bf81
Use results instead of stew/results (#1112) 2025-02-18 10:17:05 +00:00
Arnaud
0107eb06fe
chore(marketplace): cid should be bytes (#1073)
* Change cid format from string to bytes for the marketplace

* refactor: marketplace custom errors handling

* chore: update contracts repo

* chore: update contracts submodule

* Update contracts submodule

* Initialize the Cid using init function

* Restorage serialize pragma

* Use Cid object instead of buffer

* Simplify cid usage

* Simplify cid usage

* bump codex-contracts-eth after PR merge, formatting

* fix rebase

* collateralPerByte => collateralPerSlot

---------

Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2025-02-18 05:47:47 +00:00
Arnaud
6e73338425
Remove deprecated function (#1111)
Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2025-02-17 22:04:04 +00:00
Eric
5af3477793
chore(deps): bump ethers to propagate cancellations (#1116)
* chore(deps): bump ethers to propagate cancellations

Ethers was swallowing canellations and turning them into EthersErrors, which was causing the sales statemachine to error when it should have been simply cancelling the current state's run. Hopefully fixes the intermittently failing marketplace integration test.

* Add missing errors in async raises pragma

* bump to version of ethers that supports cancellations

---------

Co-authored-by: Arnaud <arnaud@status.im>
2025-02-17 22:00:52 +00:00
Arnaud
dc08ff8840
chore(marketplace): add a cache for storage requests (#1090)
* Add cache to for requests

* Change request cache description message and use const as default value

* Set request cache size configuration hidden
2025-02-17 10:34:42 +00:00
Giuliano Mega
25c84f4e0e
Fix/repostore deletes for non-orphan blocks (#1109)
* fix: fix deletion of non-orphan blocks

* feat: improve error feedback for illegal direct block deletes

* chore: minor rewording of test header
2025-02-14 13:34:17 +00:00
munna0908
c65148822e
feat: multithreading support for erasure coding (#1087)
* implement async encode

* implement async decode

* cleanup code

* add num-threads flag

* fix tests

* code cleanup

* improve return types and exception handling for async proc

* add validation check for numThreads flag

* modify encode method

* add new tests for aync encoding

* modify decode method

* cleanup test cases

* add new cli flag for threadCount

* test cleanup

* add new tests

* fix decodeAsync exception handling

* code cleanup

* chore: cosmetic changes
2025-02-12 17:56:26 +00:00
Dmitriy Ryajov
45e97513a7
remove uploadedAt from manifest (#1091)
* remove uploadedAt from manifest

* fix test
2025-02-12 10:48:58 +00:00
Slava
20f6fef7ab
fix: use ubuntu-24.04 runners for docker workflows (#1102)
Co-authored-by: Giuliano Mega <giuliano.mega@gmail.com>
2025-02-11 21:49:37 +00:00
Giuliano Mega
bbe1f09cd7
Purging of local datasets (#1103)
* feat(codex-node): add dataset deletion API to Codex node

* feat(api): add deletion of local datasets to API

* fix: logging, remove garbage, drop some CORS headers from DELETE request

* fix: change empty response return code to 204 instead of 200

* fix: add time-based idling to avoid locking up the node during deletes, fix API status code

* fix: uncomment commented tests committed by accident

* fix: return correct code when missing CID is a Manifest CID; add back CORS headers

* fix: remove lingering echo
2025-02-11 19:00:05 +00:00
Marcin Czenko
11888e78d7
deploy openapi spec only when tagged (#1106) 2025-02-11 15:16:45 +00:00
Marcin Czenko
8880ad9cd4
fix linting in "codex/blockexchange/engine/engine.nim" (#1107) 2025-02-11 10:47:25 +00:00
Eric
dfa90a9981
fix(build): compilation on macos when including nim-nat-traversal (#1084)
* fix(build): compilation on macos when including nim-nat-traversal

- removes the `VERSION` rename to `VERSION_temp` in the Makefile
- instead, relies on `-iqoute` to include the `nim-nat-traversal/vendor/libnatpmp-upstream` directory in the search paths. `-iquote` will match the `vendor/libnatpmp-upstream/VERSION` file for `#include "version"` and not `#include <version>`, the latter being what is included by the macos sdk and was causing issues with `-I`. The [gcc 14.2 docs](https://gcc.gnu.org/onlinedocs/gcc-14.2.0/cpp/Invocation.html#index-I) describe how `-iquote` alleviates this issue:
> Directories specified with -iquote apply only to the quote form of the directive, #include "file". Directories specified with -I, -isystem, or -idirafter apply to lookup for both the #include "file" and #include <file> directives.

For more info, please see https://github.com/status-im/nim-nat-traversal/pull/34.

* bump nim-nat-traversal

Now that https://github.com/status-im/nim-nat-traversal/pull/34 has been merged, change back to master commit
2025-02-07 01:18:00 +00:00
Dmitriy Ryajov
17d3f99f45
use a case-of instead of if for better readability (#1063) 2025-02-06 21:36:35 +00:00
Csaba Kiraly
e62a09d9b1
add ccache and sccache to speed up CI (#1074)
* add ccache and sccache to speed up CI

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* include testname and nim version in cache separation

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* Make sure ccache has precedence over custom clang/llvm

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* enable ccache for windows

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* ccache: evict old files

Make sure old unused cache files are not lingering around for long

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

---------

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2025-02-06 21:36:14 +00:00
Marcin Czenko
c05eec422c
fix dataset and slot size calculations in integration tests (#1095)
* fixes datasetSize and slotSize helpers (and also RandomChunker.example)

* adds overload for <<upload>> for seq[byte]

* changes RandomChunker.example to return seq[byte]

* fixes restapi tests after correcting RandomChunker.example

* review: use string.fromBytes from nim-stew to convert seq[byte] to string
2025-02-06 15:21:12 +00:00
Arnaud
ffd149fba8
Update documentations and cleanup 2025-02-06 15:17:06 +01:00
Arnaud
9764d3f133
Clean up and fix tests 2025-02-06 14:54:04 +01:00
Arnaud
ac24f35250
Remove debug message 2025-02-06 08:30:29 +01:00