129 Commits

Author SHA1 Message Date
Eric
dd1a761a5e
Back off port inc in the hopes that this is causing issues with CI 2025-02-19 15:43:21 +11:00
Eric
9ebc95fe89
formatting
trying to make nph happy
2025-02-19 15:43:20 +11:00
Eric
a99949228b
formatting 2025-02-19 15:43:20 +11:00
Eric
8e8cbc511f
clean up 2025-02-19 15:43:20 +11:00
Eric
7132983e1d
use tracked futures for showContinuousStatusUpdates 2025-02-19 15:43:19 +11:00
Eric
b290ee758e
formatting 2025-02-19 15:43:19 +11:00
Eric
583792f52c
fix rebase, formatting 2025-02-19 15:41:47 +11:00
Eric
849fd00a2c
fix rebase 2025-02-19 15:41:47 +11:00
Eric
ca3aeb28cf
Use absolute path for binary when checking for existence 2025-02-19 15:41:47 +11:00
Eric
567a8444cb
fail early with message when hardhat binary not found
# Conflicts:
#	tests/integration/hardhatprocess.nim
2025-02-19 15:41:47 +11:00
Eric
cc8f00e707
remove unneeded sleep 2025-02-19 15:41:46 +11:00
Eric
9b0eca0815
make onoutputline callback gcsafe 2025-02-19 15:41:46 +11:00
Eric
aea6c99a69
increase port distance per test run 2025-02-19 15:41:46 +11:00
Eric
7f058deece
add constructor param for continuous status update in terminal
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:45 +11:00
Eric
749f65b134
fix nodeprocess not exiting 2025-02-19 15:41:45 +11:00
Eric
8501220fda
ensure timeouts work
Work around the AsyncProcess timeout not working correctly
2025-02-19 15:41:45 +11:00
Eric
92513e083e
continually show test status updates
allows user to know which tests are still running
2025-02-19 15:41:45 +11:00
Eric
2ef044dfcb
cleanup
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:44 +11:00
Eric
a46bf719ef
lock hardhat start to avoid intermittent start failures 2025-02-19 15:41:44 +11:00
Eric
ff8ac244bd
clean up 2025-02-19 15:41:44 +11:00
Eric
beb270de3d
print hardhat testmanager errors and hardhat stdout when there is an error starting hardhat 2025-02-19 15:41:43 +11:00
Eric
f4712684ba
hang manager off IntegrationTest, cleanup
- add a TestManager property to IntegrationTest, so manager does not need to be passed into all functions
- cleanup:
  - remove unneeded stopHardhat function
  - add hardhat instance to manager outside of startHardhat
2025-02-19 15:41:43 +11:00
Eric
f55831a2c8
reinstate removed types 2025-02-19 15:41:43 +11:00
Eric
0a8487618a
Add additional logging info print outs
# Conflicts:
#	tests/integration/testcli.nim
2025-02-19 15:41:42 +11:00
Eric
34f445993d
expect process exit code
prevents showing error in the logs when an expected process exit code is encountered

# Conflicts:
#	tests/integration/testcli.nim
2025-02-19 15:41:42 +11:00
Eric
6a18238205
stop hardhat and show logs
# Conflicts:
#	tests/integration/hardhatprocess.nim
2025-02-19 15:41:42 +11:00
Eric
e4c035651f
default test name to the filename of the integration test
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:42 +11:00
Eric
cf9134861e
print start/finish stderr/stdout markers
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:41 +11:00
Eric
1400848f86
clean up 2025-02-19 15:41:41 +11:00
Eric
ce4b1d439c
stdout formatting for test results
# Conflicts:
#	tests/testIntegration.nim
2025-02-19 15:41:41 +11:00
Eric
7252a32579
prevent errors during node shutdown 2025-02-19 15:41:40 +11:00
Eric
40142fa684
test summary formatting 2025-02-19 15:41:40 +11:00
Eric
93d3de315e
parallel tests running
- need to test with longer tests to ensure the parallelisation is truly happening
- is the +10 hardhat port needed?
- try with more integration tests

# Conflicts:
#	tests/integration/hardhatprocess.nim
#	tests/integration/multinodes.nim
#	tests/integration/testcli.nim
#	tests/testIntegration.nim
2025-02-19 15:41:40 +11:00
Eric
14a223fb36
fix after rebase 2025-02-19 15:41:40 +11:00
Eric
66683a8606
clean up 2025-02-19 15:41:39 +11:00
Eric
3aed1ead53
chore(integration): simplify block expiration integration test 2025-02-19 15:41:32 +11:00
Adam Uhlíř
1052dad30c
feat: request duration limit (#1057)
* feat: request duration limit

* Fix tests and duration type

* Add custom error

* Remove merge issue

* Update codex contracts eth

* Update market config and fix test

* Fix SlotReservationsConfig syntax

* Update dependencies

* test: remove doubled test

* chore: update contracts repo

---------

Co-authored-by: Arnaud <arnaud@status.im>
2025-02-18 19:41:54 +00:00
Arnaud
0107eb06fe
chore(marketplace): cid should be bytes (#1073)
* Change cid format from string to bytes for the marketplace

* refactor: marketplace custom errors handling

* chore: update contracts repo

* chore: update contracts submodule

* Update contracts submodule

* Initialize the Cid using init function

* Restorage serialize pragma

* Use Cid object instead of buffer

* Simplify cid usage

* Simplify cid usage

* bump codex-contracts-eth after PR merge, formatting

* fix rebase

* collateralPerByte => collateralPerSlot

---------

Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2025-02-18 05:47:47 +00:00
Dmitriy Ryajov
45e97513a7
remove uploadedAt from manifest (#1091)
* remove uploadedAt from manifest

* fix test
2025-02-12 10:48:58 +00:00
Giuliano Mega
bbe1f09cd7
Purging of local datasets (#1103)
* feat(codex-node): add dataset deletion API to Codex node

* feat(api): add deletion of local datasets to API

* fix: logging, remove garbage, drop some CORS headers from DELETE request

* fix: change empty response return code to 204 instead of 200

* fix: add time-based idling to avoid locking up the node during deletes, fix API status code

* fix: uncomment commented tests committed by accident

* fix: return correct code when missing CID is a Manifest CID; add back CORS headers

* fix: remove lingering echo
2025-02-11 19:00:05 +00:00
Marcin Czenko
c05eec422c
fix dataset and slot size calculations in integration tests (#1095)
* fixes datasetSize and slotSize helpers (and also RandomChunker.example)

* adds overload for <<upload>> for seq[byte]

* changes RandomChunker.example to return seq[byte]

* fixes restapi tests after correcting RandomChunker.example

* review: use string.fromBytes from nim-stew to convert seq[byte] to string
2025-02-06 15:21:12 +00:00
Giuliano Mega
0595723f66
Minor improvements to download API (#1092)
* chore: improve error messages in upload API

* chore: remove unreachable (dead) code

* fix: API integration test
2025-02-04 16:01:14 +00:00
Marcin Czenko
962fc1cd95
Feat: price per byte (#1078)
* openAPI: StorageRequestCreation: reward => pricePerByte, collateral => collateralPerByte

* purchasing: reward => pricePerByte, collateral => collateralPerByte

* Updates availabilities and reservations to use totalCollateral, minPricePerByte, and maxCollateralPerByte

* Uses correct div operator when operating on UInt256

* proposal updating totalCollateral in availability

* makes sure that reading currentCollateral happens before freeing slot

* Updates naming

* fixes tests: unit and contracts

* uses feat/price-per-byte branch for codex-contracts-eth

* temporarily disables integration tests on CI

* introduces high level <<totalCollateral>> property for a cleaner external interface

* updates integration tests

* Applies review comments

* Updates description of totalCollateral in SalesAvailability

* updates codex-contracts-eth (price-per-byte)
2025-01-24 17:18:00 +00:00
Adam Uhlíř
e5df8c50d3
style: nph formatting (#1067)
* style: nph setup

* chore: formates codex/ and tests/ folder with nph 0.6.1
2025-01-21 20:54:46 +00:00
Arnaud
f25c555d59
Chore/update nim version (#1052)
* Move to version 2.0.6

* Update nim-confutils submodule to latest version

* Update dependencies

* Update Nim version to 2.0.12

* Add gcsafe pragma

* Add missing import

* Update specific conf for Nim 2.x

* Fix method signatures

* Revert erasure coding attempt to fix bug

* More gcsafe pragma

* Duplicate code from libp2p because it is not exported anymore

* Fix camelcase function names

* Use alreadySeen because need is not a bool anymore

* newLPStreamReadError does not exist anymore so use another error

* Replace ValidIpAddress by IpAddress

* Add gcsafe pragma

* Restore maintenance parameter deleted by mistake when removing esasure coding fix attempt code

* Update method signatures

* Copy LPStreamReadError code from libp2p which was removed

* Fix camel case

* Fix enums in tests

* Fix camel case

* Extract node components to a variable to make Nim 2 happy

* Update the tests using ValidIpAddress to IpAddress

* Fix cast for value which is already an option

* Set nim version to 2.0.x for CI

* Set nim version to 2.0.x for CI

* Move to miniupnp version 2.2.4 to avoid symlink error

* Set core.symlinks to false for Windows for miniupnp >= 2.2.5 support

* Update to Nim 2.0.14

* Update CI nim versions to 2.0.14

* Try with GCC 14

* Replace apt-fast by apt-get

* Update ubuntu runner to latest

* Use Ubuntu 20.04 for coverage

* Disable CI cache for coverage

* Add coverage property description

* Remove commented test

* Check the node value of seen instead of using alreadySeen

* Fix the merge. The taskpool work was reverted.

* Update nim-ethers submodule

* Remove deprecated ValidIpAddress. Fix missing case and imports.

* Fix a weird issue where nim-confutils cannot find NatAny

* Fix tests and remove useless static keyword
2025-01-10 14:12:37 +00:00
Ben Bierens
caed3c07a3
Fix sending of WantBlocks messages and tracking of peerWants (#1019)
* sends wantBlock to peers with block. wantHave to everyone else

* Cleanup cheapestPeer. Fixes test for peers lists

* Fixes issue where peerWants are only stored for type wantBlock.

* Review comments by Dmitriy

* consistent logging of addresses

* prevents duplicate scheduling. Fixes cancellation

* fast

* Marks cancel-presence situation with todo comment.

* fixtest: testsales enable logging

* Review by Dmitriy: Remember peerWants only if we don't have them.

* rework `wantListHandler` handling

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2025-01-09 22:44:02 +00:00
munna0908
74c46b3651
network: support for uPnP and PMP nat traversal (#1050)
* add support for uPnP and PMP nat traversal

* update cli flags

* merge with master

* use listener address from switch

* update build script to rename version file

* fix styling issues

* update docker files

- remove Disc_ip env
- update CODE_NAT parsing logic

* code cleanup

* move nat flag parsing logic to conf.nim
2025-01-09 18:11:22 +00:00
Eric
6ff4d30b43
fix(tests/integration): fix bootstrap node spr (#1048)
After a change in PR #1031, bootstrap node sprs may not work when Hardhat nodes are started with the tests. This fixes it by appending all started client's and provider's SPR to a sequence, and using that sequence of SPRs to start the next node. This means all subsequently started nodes will be connected to its previously started peers.

This also fixes the case when bootstrap SPRs would not be present if no clients were started.
2024-12-18 08:10:36 +00:00
Eric
8645d336ff
refactor(trackedfutures): remove return of future from tracked futures api (#1046)
- cleans up all instances of `.track` to use the `module.trackedfutures.track(future)` procedure, for better readability
- removes the `track` override that is no longer used in the codebase
2024-12-18 07:39:03 +00:00
Ben Bierens
20bb5e5a38
Applies multinodessuite in twonodessuite (#1031)
* Sets timeout for codexclient httpClient. Adds reliable transfer test.

* disable new test to check timeout setting in CI

* restores new test

* adds heartbeat log and logfile to ci output

* fixes suite

* fixes blocked output stream by switching to multinode fixture

* new twonodessuite based on multinodesuite

* Applies updated twonodessuite

* removes heartbeat log

* applies multinodesuite in testsales

* applies multinodesuite in testmarketplace

* fixes account fetch for host and client in testmarketplace

* adds waitTillNextPeriod at end of marketplace test

* Uses marketplacesuite in testmarketplace
2024-12-17 13:01:41 +00:00