Commit Graph

704 Commits

Author SHA1 Message Date
Arnaud daadffe30a
Replace ValidIpAddress by IpAddress 2024-12-18 17:17:49 +01:00
Arnaud 83b4943c17
newLPStreamReadError does not exist anymore so use another error 2024-12-18 16:51:12 +01:00
Arnaud 902c6dc692
Use alreadySeen because need is not a bool anymore 2024-12-18 16:50:26 +01:00
Arnaud a2f09b2296
Fix camelcase function names 2024-12-18 16:49:48 +01:00
Arnaud a2f0528dc4
Duplicate code from libp2p because it is not exported anymore 2024-12-18 16:48:56 +01:00
Arnaud 599e868bc1
More gcsafe pragma 2024-12-18 16:48:15 +01:00
Arnaud a1c8c94ada
Revert erasure coding attempt to fix bug 2024-12-18 16:47:21 +01:00
Arnaud 4935494c29
Fix method signatures 2024-12-18 16:45:25 +01:00
Arnaud 0ecf3f7a78
Update specific conf for Nim 2.x 2024-12-18 16:43:38 +01:00
Arnaud 0e00f0c7cb
Add missing import 2024-12-18 16:41:42 +01:00
Arnaud 7c9b607e38
Add gcsafe pragma 2024-12-18 16:37:12 +01:00
Arnaud 34f5e0071c
Update Nim version to 2.0.12 2024-12-18 16:28:49 +01:00
Arnaud ef3274b290
Update dependencies 2024-12-18 16:25:52 +01:00
Arnaud ca2b63c3b1
Update nim-confutils submodule to latest version 2024-11-28 09:48:12 +01:00
Arnaud b5af96f068
Move to version 2.0.6 2024-11-28 09:47:59 +01:00
Ben Bierens 21249968d4
Requesting the same CID sometimes causes a worker to discard the request if it's already inflight by another worker. (#1002) 2024-11-27 07:13:34 +00:00
Ben Bierens d47ce38894
Fixes race in testsales (#995)
* Adds isWaiting to mockClock to remove sleep in testsales

* Review comments by Eric. Also replaced two more sleeps with check-eventually
2024-11-26 10:48:52 +00:00
Slava 5c6bbb0cee
chore: update testnet marketplace address (#1000) (#1001)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-11-26 08:16:41 +00:00
Ben Bierens 024c75e4f9
Bumps leveldbstatic to v0.1.6 (#998) 2024-11-25 17:15:05 +00:00
Slava d12de20868
ci: use rust 1.7.9 for release workflow and dockerfile (#999)
* ci: use rust 1.7.9 for release workflow (#996)

* docker: use rust 1.7.9 for docker (#996)
2024-11-25 16:13:14 +00:00
markspanbroek 29433bad9a
Fix concurrency issues (#993)
* Use http subscriptions instead of websocket for tests

To work around this issue when subscriptions are
inactive for more than 5 minutes:
https://github.com/NomicFoundation/hardhat/issues/2053

Use 100 millisecond polling; default polling interval
of 4 seconds is too close to the 5 second timeout for
`check eventually`.

* use .confirm(1) instead of confirm(0)

confirm(0) doesn't wait at all, confirm(1) waits
for the transaction to be mined

* speed up partial payout integration test

* update nim-ethers to version 0.10.0

includes fixes for http polling and .confirm()

* fix timing of marketplace tests

allow for a bit more time to withdraw funds

* use .confirm(1) in marketplace tests

to ensure that the transaction has been processed
before continuing with the test

* fix timing issue in validation unit test

* fix proof integration test

there were two logic errors in this test:
- a slot is freed anyway at the end of the contract
- when starting the request takes a long time, the
  first slot can already be freed because there were
  too many missing proofs

* fix intermittent error in contract tests

currentTime() doesn't always correctly reflect
the time of the next transaction

* reduce number of slots in integration test

otherwise the windows runner in the CI won't
be able to start the request before it expires

* fix timing in purchasing test

allow for a bit more time for a request to
be submitted

* fix timing of request submission in test

windows ci is so slow, it can take up to 40 seconds
just to submit a storage request to hardhat

* increase proof period to 90 seconds

* adjust timing of integration tests

reason: with the increased period length of 90 seconds, it
can take longer to wait for a stable challenge at the
beginning of a period.

* increase CI timeout to 2 hours

* Fix slow builds on windows

apparently it takes windows 2-3 seconds to
resolve "localhost" to 127.0.0.1 for every
json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
Slava 6038fb456e
ci: split linux and macos tests (#997)
* Split Linux and macOS tests

* Make jobs names more readable
2024-11-22 12:05:00 +00:00
Slava 71b8a95d12
ci: install rust 1.7.9 as required by packages (#996)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-11-22 06:34:10 +00:00
Arnaud 9b7f3f4aaf
chore(openapi): update the openapi spec for the manifest object (#972)
* Update the manifest openapi spec

* Make metadata nullable

Signed-off-by: Arnaud <arnaud@status.im>

---------

Signed-off-by: Arnaud <arnaud@status.im>
Co-authored-by: Giuliano Mega <giuliano.mega@gmail.com>
2024-11-15 13:21:18 +00:00
Slava d7ae8b734a
makefile: Use do not use -mssse3 instructions on arm (#940) (#990) 2024-11-09 11:39:39 +00:00
markspanbroek a6f0311b50
change default proof period to 2 minutes (#989)
on hardhat the proof period remains 1 minute
2024-11-08 08:03:16 +00:00
Marko Burčul 2151e02838
nix-flake: Add shell definition (#954)
Initialized flake file and added development shell definition.
Exporting of default compiler flags is moved to makefile.

Referenced issue: https://github.com/codex-storage/nim-codex/issues/940

Signed-off-by: markoburcul <marko@status.im>
2024-11-04 07:46:22 +00:00
Slava 86257054ee
chore: update testnet marketplace address (#983) (#984)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-11-04 00:41:03 +00:00
Slava 96459188c9
Add ETH_PRIVATE_KEY to Docker entrypoint (#982)
* Add ETH_PRIVATE_KEY to Docker entrypoint

* Add deprecation warning for PRIV_KEY variable

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

---------

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-11-03 06:51:57 +00:00
Ben Bierens b8dd68063f
fix: bumps ethers to fix missing nonce error (#980)
* fix: bumps ethers to fix missing nonce error

* fix was merged in nim-ethers

---------

Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2024-11-01 17:23:43 +00:00
Eric 2b5a40559e
chore: update dependencies, especially nim-ethers to chronos v4 compatible version (#968)
* chore: bump dependencies, including nim-ethers with chronos v4 support

Bumps the following dependencies:
- nim-ethers to commit 507ac6a4cc71cec9be7693fa393db4a49b52baf9 which contains a pinned nim-eth version. This is to be replaced by a versioned library, so it will be pinned to a particular version. There is a crucial fix in this version of ethers that fixes nonce management which is causing issues in the Codex testnet.
- nim-json-rpc to v0.4.4
- nim-json-serialization to v0.2.8
- nim-serde to v1.2.2
- nim-serialization to v0.2.4

Currently, one of the integration tests is failing.

* fix integration test

- When a state's run was cancelled, it was being caught as an error due to catching all CatchableErrors. This caused a state transition to SaleErrored, however cancellation of run was not actually an error. Handling this correctly fixed the issue.
- Stopping of the clock was moved to after `HostInteractions` (sales) which avoided an assertion around getting time when the clock was not started.

* bump ethers to include nonce fix and filter not found fix

* bump ethers: fixes missing symbol not exported in ethers

* Fix cirdl test imports/exports

* Debugging in ci

* Handle CancelledErrors for state.run in one place only

* Rename `config` to `configuration`

There was a symbol clash preventing compilation and it was easiest to rename `config` to `configuration` in the contracts. Not even remotely ideal, but it was the only way.

* bump ethers to latest

Prevents an issue were `JsonNode.items` symbol could not be found

* More changes to support `config` > `configuration`

* cleanup

* testing to see if this fixes failure in ci

* bumps contracts

- ensures slot is free before allowing reservation
- renames config to configuration to avoid symbol clash
2024-10-30 10:40:17 +00:00
Arnaud 942f940c92
Move the upload headers to the POST method (#978) 2024-10-29 14:40:43 +00:00
Slava a2ac7453fa
Build Postman Collection (#973)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-10-28 13:53:41 +00:00
Arnaud 2fb7031ec6
feat: add metadata to the manifest (#960)
* Add metadata to the manifest

* Remove useless import

* Fix the openapi documentation

* Use optional fields instead of default values

* Remove testRestApi target

* Return failure when the protobuf cannot get the field

* Set download headers and fix cors headers when an error is returned

* Add tests to verify the download headers

* Try to adjust the content length header

* Fix convertion to string

* Remove the content length header

* Remove testRestApi target

* Removing debug messages
2024-10-25 13:43:19 +00:00
Arnaud bcc1468130
Remove duplicated header (#970) 2024-10-25 09:23:35 +00:00
Arnaud 40068512a6
Complete documentation for debug endpoint (#969) 2024-10-25 07:20:00 +00:00
Eric 0157ca4c57
fix(slot-reservations): Avoid slot filled cancellations (#963)
* Avoid cancelling states when slot is filled

* improve logging

Improves logging for situations where a Sale should be ignored instead of being considered an error, including when reservation is not allowed and when a slot was filled by another host.

* remove onSlotFilled unit tests from states
2024-10-24 05:56:12 +00:00
Slava 3a2d0926f1
chore: new marketplace address for testnet (#961)
https://github.com/codex-storage/infra-codex/issues/248

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-10-21 13:01:56 +03:00
Ben Bierens 562e4329e2
Download API upgrade (#955)
* Adds API for fetching manifest only and downloading dataset without stream

* Updates openapi.yaml

* Adds tests for downloading manifest-only and without stream.

* review comments by Giuliano

* updates test clients
2024-10-17 16:54:28 +00:00
Adam Uhlíř 436baef20a
docs: openapi node fix (#950) 2024-10-14 17:26:58 +00:00
Adam Uhlíř 7c33473c88
ci: linux ci runs on ubuntu-20.04 (#953)
* ci: linux ci runs uses ubuntu-20.04

* ci: use ubuntu-20.04 for nim-matrix

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

---------

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
Co-authored-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-10-14 11:24:53 +00:00
Ben Bierens 93960033f4
Adds log for cirdl download URL (#948) 2024-10-10 12:05:03 +00:00
Adam Uhlíř ffa203b04f
feat: partial rewards and withdraws (#880)
* feat: partial rewards and withdraws

* test: missing reserve slot

* test: fix contracts
2024-10-10 11:53:33 +00:00
Ben Bierens 3699601393
Handles LPStreamError in chunker (#947)
* Handles LPStreamError in chunker

* Adds test for lpstream exception

* Adds tests for other stream exceptions. Cleanup.
2024-10-10 11:22:36 +00:00
Arnaud 1fe3abfd03
fix(restapi): Add cors headers when the request is returning errors (#942)
* Add cors headers when the request is returning errors

* Prevent nim presto to send multiple cors headers
2024-10-10 08:25:07 +00:00
Eric 7e0ec3c233
Support enforcement of slot reservations before filling slot (#934) 2024-10-09 04:44:07 +00:00
Slava 44f21b8a68
Update Codex Testnet marketplace contract address (#944)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-10-08 12:10:42 +03:00
Arnaud f02de34f77
docs(openapi): provider better documentation for space endpoint parameters (#921)
* Trying to improve documentation

* Update openapi.yaml

Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
Signed-off-by: Arnaud <arno.deville@gmail.com>

* Update openapi.yaml

Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
Signed-off-by: Arnaud <arno.deville@gmail.com>

* Update openapi.yaml

Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
Signed-off-by: Arnaud <arno.deville@gmail.com>

---------

Signed-off-by: Arnaud <arno.deville@gmail.com>
Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
2024-10-08 06:37:10 +00:00
Ben Bierens 17f0988fc7
Fix: null-ref in networkPeer (#937)
* Fixes nullref in networkPeer

* Removes inflight semaphore

* Revert "Removes inflight semaphore"

This reverts commit 26ec15c6f7.
2024-10-07 08:50:54 +00:00
Slava 0ea8cfb085
Remove moved docs (#935)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-10-07 08:33:10 +00:00