472 Commits

Author SHA1 Message Date
E M
04f7c6cfd5
feat(cli)!: change --listen-addrs to --listen-port
This allows specifying the port only for the listening address, the address that is announced on the DHT for peer connections. It makes the developer experience less confusing, and easier to change for things like port forwarding.

# Conflicts:
#	storage/storage.nim
#	tests/integration/multinodes.nim
2026-02-19 16:47:45 +11:00
Eric
ab413bdfcf
chore!: Finish renaming Codex to Logos Storage (#1399) 2026-02-19 04:59:15 +00:00
Eric
4068bcb2ed
fix: bump nph and refactor build process in makefile (#1410) 2026-02-19 02:12:45 +00:00
Eric
fef46aee35
feat(testing): local libstorage c bindings test (#1407) 2026-02-18 04:29:59 +00:00
Giuliano Mega
1a7cf2d335
fix: propagate callback cancellation (#1400) 2026-02-05 14:33:23 +00:00
Eric
44ad291b16
refactor!: move merkletree to own repo (#1390) 2026-02-02 22:43:14 +00:00
Eric
2a1a548341
refactor!: remove unused modules (#1362)
Signed-off-by: Chrysostomos Nanakos <chris@include.gr>
Co-authored-by: Chrysostomos Nanakos <chris@include.gr>
Co-authored-by: gmega <giuliano.mega@gmail.com>
2026-01-16 19:03:54 +00:00
Giuliano Mega
e894fb03fa
feat: primitives for sampling with and without replacement (#1125) 2026-01-16 13:49:04 -03:00
Eric
1acedcf71c
fix(ci): introduce a number of integration test fixes (#1342)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
Co-authored-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
Co-authored-by: Arnaud <arnaud@status.im>
Co-authored-by: gmega <giuliano.mega@gmail.com>
2026-01-16 10:47:59 +00:00
Jacek Sieka
cce002fcbf
feat: async tree building v2 (#1360)
Signed-off-by: Giuliano Mega <giuliano.mega@gmail.com>
Co-authored-by: munna0908 <munnasitu0908@gmail.com>
Co-authored-by: gmega <giuliano.mega@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-01-15 18:13:14 +00:00
Arnaud
60861d6af8
chore: rename codex to logos storage (#1359) 2025-12-18 17:23:09 +00:00
Jacek Sieka
6765beee2c
chore: assorted bumps (#1351) 2025-12-11 21:03:36 +00:00
Jacek Sieka
9ac9f6ff3c
chore: drop usage of upraises (#1348) 2025-12-11 09:03:43 +00:00
Arnaud
bd36032251
feat: add c binding (#1322)
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2025-11-13 07:34:09 +00:00
Chrysostomos Nanakos
be759baf4d
feat: Block exchange optimizations (#1325)
Signed-off-by: Giuliano Mega <giuliano.mega@gmail.com>
Signed-off-by: Chrysostomos Nanakos <chris@include.gr>
Co-authored-by: gmega <giuliano.mega@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2025-11-13 05:47:02 +00:00
Eric
ee47ca8760
feat(libs): Use libp2p multiformats extensions instead of a rolling branch (#1329) 2025-11-13 04:48:33 +00:00
Eric
f791a960f2
fix(ci): Windows SIGILL in CI (#1339) 2025-11-03 11:45:02 +00:00
Arnaud
db8f866db4
feat: check if CID exists in local store (#1331) 2025-11-02 04:32:47 +00:00
Eric
072bff5cab
fix: ci integration tests (#1335) 2025-10-30 19:38:11 +11:00
Arnaud
af55a761e6
chore: skip marketplace and long integration tests (#1326) 2025-10-22 19:22:33 +11:00
markspanbroek
01615354af
refactor(ci): run integration tests in parallel by spinning up more runners (#1287) 2025-06-25 08:56:16 +00:00
markspanbroek
e1c397e112
fix(tests): auto import all tests files and fix forgotten tests (#1281) 2025-06-23 11:18:59 +00:00
Arnaud
7b660e3554
chore(marketplace): use hardhat ignition (#1195) 2025-06-20 15:55:00 +00:00
Arnaud
d443df441d
chore: improve marketplace integration tests (#1268) 2025-06-19 06:36:10 +00:00
Chrysostomos Nanakos
b305e00160
Add support for slot reconstruction on unavailable slot detection (#1235)
Co-authored-by: Arnaud <arnaud@status.im>
2025-06-12 22:19:42 +00:00
markspanbroek
27d807a841
fix(sales): fix marketplace block expiry (#1258) 2025-06-11 11:27:09 +00:00
Ben Bierens
85823342e9
Improves logging in maintenance module and erasure. (#1264) 2025-06-10 13:27:52 +00:00
Adam Uhlíř
7502b9ad2c
feat(cirdl): auto-discovery of marketplace contract (#1259) 2025-06-09 10:04:15 +00:00
Eric
1bea94c390
fix(tests): fetching past contract events test (#1255) 2025-06-04 20:36:09 -07:00
markspanbroek
2dd436bfb7
fix(sales): do not crash when retrieving request fails (#1248) 2025-06-04 11:22:14 +00:00
Arnaud
45ade0e3c1
chore(marketplace): use canMarkProofAsMissing (#1188)
* Add canProofBeMarkedAsMissing

* Add more tests

* Update contracts submodule
2025-06-03 09:08:57 +00:00
Arnaud
ca869f6dce
fix(availabilities): use totalRemainingCollateral instead of totalCollateral for comparaison (#1229)
* Use totalRemainingCollateral instead of totalCollateral to compare the availability changes

* Update test to use totalRemainingCollateral instead of totalCollateral when testing OnAvailabilitySaved

* Reduce poll interval

* Fix flaky test

* Fix format
2025-06-02 16:47:12 +00:00
Giuliano Mega
d59c5b023c
chore: bump Nim to 2.2.4 (#1242)
* chore: bump Nim to 2.2.4

* fix: resolve symbol ambiguity and drop auto type

* fix: use reference to task instead of pointer or the compiler will deallocate `task` before the encoding/decoding is done

* fix: convention that maxCollateralPerByte equals totalRemainingCollateral when freeSize is 0 to avoid DivByZeroDefect

* fix: bump compiler version in CI pipeline as well
2025-05-29 16:37:38 -07:00
Arnaud
28a83db69e
chore: returns the collateral when a slot is reserved but not filled (#1216)
* Change token allowance method because increaseAllowance does not exist anymore

* Returns collateral when a reservation is deleted and not only a slot is filled

* Remove the returnedCollateral when the slot is not filled by the host

* Add returnedCollateral when the sale is ignored

* Add returnsCollateral variable for ignored state

* Rebase the contracts submodule on the master

* Add integration test

* Fix duration

* Remove unnecessary teardown function

* Remove misleading comment

* Get returned collateral from the request

* Enable logs to debug on CI

* Fix test

* Increase test timeout

* Fix typo

* Fix rebase
2025-05-29 14:47:37 +00:00
Arnaud
c689542579
fix: sales cleanup cancellation (#1234)
* fix(sales): handle cancellation of slot cleanup

Ensures that processing slots from the slot queue
continues even when cleanup of a slot is cancelled.

Co-Authored-By: Eric <5089238+emizzle@users.noreply.github.com>

* chore(reservations): add more `raises` annotations

* Fix cleanup cancellation

* Add remove-agent to trackedfutures instead of the cleanup function

* Increase the timeout to match the request expiry

* Enable logs to debug on CI

* Remove useless except and do not return when add item back to slot queue fails

* Reduce poll interval to detect sale cancelled state

* Avoid cancelling cleanup routine

* Do not cancel creating reservation in order to avoid inconsistent state

* Remove useless try except

---------

Co-authored-by: Mark Spanbroek <mark@spanbroek.net>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2025-05-29 06:57:05 +00:00
markspanbroek
25a8077e80
fix(integration): fix api validation test (#1201)
* integration: shutdown codex node at end of test

On Windows the codex node did not shut down properly after this test
finished.

* contracts: fix flaky test
2025-05-26 16:49:53 +00:00
Arnaud
f7d06cd0e8
chore(marketplace): switch to websocket (#1166)
* Switch to websocket

* Create resubscribe future

* Resubscribe websocket events after 5 minutes

* Remove the subscribe workaround and use define the resubscribe symbol

* Use localhost for ws url

* Define 240 seconds for resubscription interval

* Ensute that updates are sync when using ws
2025-05-23 14:13:19 +00:00
Marcin Czenko
748830570a
checked exceptions in stores (#1179)
* checked exceptions in stores

* makes asynciter as much exception safe as it gets

* introduce "SafeAsyncIter" that uses Results and limits exceptions to cancellations

* adds {.push raises: [].} to errors

* uses SafeAsyncIter in "listBlocks" and in "getBlockExpirations"

* simplifies safeasynciter (magic of auto)

* gets rid of ugly casts

* tiny fix in hte way we create raising futures in tests of safeasynciter

* Removes two more casts caused by using checked exceptions

* adds an extended explanation of one more complex SafeAsyncIter test

* adds missing "finishOnErr" param in slice constructor of SafeAsyncIter

* better fix for "Error: Exception can raise an unlisted exception: Exception" error.

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2025-05-21 21:17:04 +00:00
markspanbroek
bde98738c2
fix(slotqueue): simplify slot queue workers (#1224)
* fix(slotqueue): simplify slot queue workers

- worker is now just an async running loop
- instead of passing a "done" Future, use an
  AsyncEvent to signal completion

* chore(slotqueue): address review comments

Co-Authored-By: Eric <5089238+emizzle@users.noreply.github.com>
Co-Authored-By: Dmitriy Ryajov <dryajov@gmail.com>

---------

Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2025-05-15 13:02:04 +00:00
Adam Uhlíř
f144099377
fix(api): availability creation validation (#1212) 2025-05-14 08:46:16 +00:00
Eric
0f152d333c
chore: bump contracts to master (#1197)
Bump contracts to master branch.

There was a change that allowed hardhat to have multiple blocks with the same timestamp, so this needed to be reflected in two tests.
2025-04-17 06:13:43 +00:00
Marcin Czenko
5ec3b2b027
make sure we do not call "get" on unverified Result while fetching in batches (#1169)
* makes sure we do not call "get" on unverified result

* make handling of failed blocks in fetchBatched even more explicit

* simplifies allFinishedValues and makes it independent from allFinishedFailed

* only sleep if not iter.finished in fetchBatched
2025-03-31 04:57:55 +00:00
Marcin Czenko
0ec52abc98
fixes RandomChunker not respecting padding (#1170) 2025-03-31 04:48:22 +00:00
Arnaud
0032e60398
fix(marketplace): catch Marketplace_SlotIsFree and continue the cancelled process (#1139)
* Catch Marketplace_SlotIsFree and continue the cancelled process

* Add log message when the slot if free during failed state

* Reduce log level to debug for slot free error

* Separate slot mock errors

* Initialize variable in setyp

* Improve tests

* Remove non-meaningful checks and rename test

* Remove the Option in the error setters

* Return collateral when the state is cancelled only if the slot is filled by the host

* Do not propagate AsyncLockError

* Wrap contract error into specific error type

* Remove debug message

* Catch only SlotStateMismatchError in cancelled

* Fix error

* Remove returnBytesWas

* Use MarketError after raises pragma were defined

* Fix typo

* Fix lint
2025-03-26 15:17:39 +00:00
Arnaud
7deeb7d2b3
feat(marketplace): persistent availabilities (#1099)
* Add availability enabled parameter

* Return bytes to availability when finished

* Add until parameter

* Remove debug message

* Clean up and fix tests

* Update documentations and cleanup

* Avoid swallowing CancelledError

* Move until validation to reservations module

* Call onAvailabilityAdded callabck when the availability is enabled in sales

* Remove until validation in restapi when creating an availability

* Add openapi documentation

* Use results instead of stew/results (#1112)

* feat: request duration limit (#1057)

* feat: request duration limit

* Fix tests and duration type

* Add custom error

* Remove merge issue

* Update codex contracts eth

* Update market config and fix test

* Fix SlotReservationsConfig syntax

* Update dependencies

* test: remove doubled test

* chore: update contracts repo

---------

Co-authored-by: Arnaud <arnaud@status.im>

* fix(statemachine): do not raise from state.run (#1115)

* fix(statemachine): do not raise from state.run

* fix rebase

* fix exception handling in SaleProvingSimulated.prove

- re-raise CancelledError
- don't return State on CatchableError
- expect the Proofs_InvalidProof custom error instead of checking a string

* asyncSpawn salesagent.onCancelled

This was swallowing a KeyError in one of the tests (fixed in the previous commit)

* remove error handling states in asyncstatemachine

* revert unneeded changes

* formatting

* PR feedback, logging updates

* chore(integration): simplify block expiration integration test (#1100)

* chore(integration): simplify block expiration integration test

* clean up

* fix after rebase

* perf: contract storage optimizations (#1094)

* perf: contract storage optimizations

* Apply optimization changes

* Apply optimizing parameters sizing

* Update codex-contracts-eth

* bump latest changes in contracts branch

* Change requestDurationLimit to uint64

* fix tests

* fix tests

---------

Co-authored-by: Arnaud <arnaud@status.im>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>

* bump contracts to master (#1122)

* Add availability enabled parameter

* Return bytes to availability when finished

* Add until parameter

* Clean up and fix tests

* Move until validation to reservations module

* Apply suggestion changes: return the reservation module error

* Apply suggestion changes for until dates

* Apply suggestion changes: reorganize tests

* Fix indent

* Remove test related to timing issue

* Add raises errors to async pragram and remove useless try except

* Update open api documentation

* Fix wording

* Remove the httpClient restart statements

* Use market.getRequestEnd to set validUntil

* Remove returnBytes

* Use clock.now in testing

* Move the api validation file to the right file

---------

Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2025-03-26 11:45:22 +00:00
Arnaud
60b6996eb0
chore(marketplace): define raises for async pragma (#1165)
* Define raises for async pragma

* Update nim ethers

* Replace CatchableError by MarketError
2025-03-26 08:06:37 +00:00
Arnaud
a0d6fbaf02
chore(marketplace) - fix the http error codes when validating the availability requests (#1104)
* Use 422 http code when there is a validation error

* Update the open api description

* Fix typo

* Add more tests for total size

* Catch CancelledError because TrackedFuture raise no error

* Split rest api validation test to a new file

* Change the way of testing negative numbers

* Rename client variable and fix test status code

* Try to reduce the number of requests in CI when asserting in tests

* Fix rebase and remove safeEventually
2025-03-24 15:47:05 +00:00
Arnaud
709a8648fd
chore: add request validations (#1144)
* Add request validations

* Define expiry as required field in storage request params and fix tests

* Fix error messages

* Enable logs to figure out the issue with recurring failing test on macos

* Add custom errors raised by contract

* Remove custom error non existing anymore

* Update asynctest module

* Update timer tests after updating asynctest
2025-03-24 11:53:34 +00:00
Arnaud
9d7b521519
chore: add missing custom errors (#1134)
* Add missing custom errors

* Separate mock state errors

* Remove the Option in the error setters

* Wrap the contract errors in MarketError

* Remove async raises (needs to address it in another PR)

* Wrap contract errors into specific error types

* Rename SlotNotFreeError to SlotStateMismatchError
2025-03-18 07:06:46 +00:00
Giuliano Mega
54177e9fbf
feat(integration): use async client instead of standard Nim HTTP client (#1159)
* WiP: migrating CodexClient to chronos http client

* fix(api): fixes #1163

* feat: fully working API integration tests

* convert most of the tests in testupdownload

* feat: working updownload tests on async client

* feat: make testsales work with async codexclient

* feat: make testpurchasing work with async codexclient

* feat: make testblockexpiration work with async codexclient

* feat: make marketplacesuite work with async codexclient

* make testproofs work with async codexclient

* chore: refactor client to express higher level in terms of lower level operations

* fix: set correct content-length for erasure-coded datasets

* feat: make testecbug work with async client

* feat: make testvalidator work with async client

* refactor: simplify request aliases, add close operation

* wire back client.close at node shutdown

* refactor: remove unused exception

* fix: use await instead of waitFor on async call sites
2025-03-17 20:08:24 +00:00