760 Commits

Author SHA1 Message Date
Arnaud
43f6a9ec0d
Use configSync to avoid async pragma in onStorageRequested 2025-01-31 17:17:28 +01:00
Arnaud
68565a16a5
Let slotCollateral determines the slot sate 2025-01-30 12:41:56 +01:00
Arnaud
b7e720a762
Use raiseMarketError helper 2025-01-30 12:40:00 +01:00
Arnaud
dcd06adb88
Cosmetic change 2025-01-30 12:39:42 +01:00
Arnaud
751da2fd27
Add exceptions to async pragma 2025-01-30 12:39:02 +01:00
Arnaud
d6275cf3c3
Use raiseMarketError helper 2025-01-30 12:38:47 +01:00
Arnaud
97368230fd
Merge branch 'master' into feat/indicate-slot-reparation 2025-01-30 09:53:20 +01:00
Arnaud
bf715b3c5d
Cosmetic change 2025-01-28 17:14:53 +01:00
Arnaud
3ec9b476ff
Remove unused code 2025-01-28 17:10:18 +01:00
Arnaud
6da4ef6fce
Use override func to optimise calls when the slot state is known 2025-01-28 17:06:04 +01:00
Arnaud
f7b3f51985
Rename SlotQueueItem collateral and required in init 2025-01-28 15:53:26 +01:00
Arnaud
11ea1735df
Move cache request to another PR 2025-01-28 14:39:13 +01:00
Arnaud
58a962add8
Update contracts and add custom errors (#1088) 2025-01-28 11:16:24 +00:00
Arnaud
a104e0fcc7
Minor fixes 2025-01-28 09:38:51 +01:00
Arnaud
75ba6ab45f
Simplify StorageAsk parameter 2025-01-28 09:22:06 +01:00
Arnaud
69e1eb83bc
Update with pricePerByte 2025-01-28 09:02:31 +01:00
Arnaud
5bc5f85db0
Merge branch 'master' into feat/indicate-slot-reparation 2025-01-27 17:06:11 +01:00
Arnaud
fe3d9bc977
Add slotCollateral calculation with getRequest cache and remove populationItem function 2025-01-27 17:01:15 +01:00
Marcin Czenko
962fc1cd95
Feat: price per byte (#1078)
* openAPI: StorageRequestCreation: reward => pricePerByte, collateral => collateralPerByte

* purchasing: reward => pricePerByte, collateral => collateralPerByte

* Updates availabilities and reservations to use totalCollateral, minPricePerByte, and maxCollateralPerByte

* Uses correct div operator when operating on UInt256

* proposal updating totalCollateral in availability

* makes sure that reading currentCollateral happens before freeing slot

* Updates naming

* fixes tests: unit and contracts

* uses feat/price-per-byte branch for codex-contracts-eth

* temporarily disables integration tests on CI

* introduces high level <<totalCollateral>> property for a cleaner external interface

* updates integration tests

* Applies review comments

* Updates description of totalCollateral in SalesAvailability

* updates codex-contracts-eth (price-per-byte)
2025-01-24 17:18:00 +00:00
markspanbroek
f6c792de79
fix slotqueue worker starvation (#1081)
* fix slotqueue worker starvation

* improve slotqueue tests

Co-Authored-By: Marcin Czenko <marcin.czenko@pm.me>

* slotqueue nph formatting

---------

Co-authored-by: Marcin Czenko <marcin.czenko@pm.me>
2025-01-23 09:28:14 +00:00
Arnaud
e9b1de5ce7
Calculate the repair collateral when adding the item into the queue 2025-01-23 10:24:45 +01:00
Adam Uhlíř
1c4184f29c
chore: ignoring style formatting commit for git blame (#1085) 2025-01-22 09:05:47 +00:00
Arnaud
00b31809a9
Apply nph 2025-01-22 08:58:27 +01:00
Arnaud
8b502086cd
Merge branch 'master' into feat/indicate-slot-reparation
# Conflicts:
#	codex/node.nim
#	codex/sales.nim
#	codex/sales/salescontext.nim
#	codex/sales/slotqueue.nim
#	codex/sales/states/downloading.nim
#	tests/codex/helpers/mockslotqueueitem.nim
#	tests/codex/sales/testsales.nim
2025-01-22 08:55:06 +01:00
Adam Uhlíř
e5df8c50d3
style: nph formatting (#1067)
* style: nph setup

* chore: formates codex/ and tests/ folder with nph 0.6.1
2025-01-21 20:54:46 +00:00
Arnaud
1441615a18
Fix tests 2025-01-21 18:00:28 +01:00
Arnaud
8611a6ad19
Indicate that slot is being repaired when trying to download 2025-01-21 17:28:19 +01:00
markoburcul
d114e6e942 nix: build nim compiler
Using nimbus build system since the codex now uses version of Nim >2.
Tested with nim versions(>2) in nixpkgs but none of them work thus the
removal of USE_SYSTEM_NIM and its logic.
Signed-off-by: markoburcul <marko@status.im>
2025-01-21 16:45:33 +01:00
markoburcul
893f6d02ab jenkins: create initial pipeline
Referenced issue: https://github.com/codex-storage/nim-codex/issues/940

Signed-off-by: markoburcul <marko@status.im>
2025-01-21 16:45:33 +01:00
Arnaud
389ab59aa7
Use custom error message (#1079) 2025-01-20 16:04:50 +00:00
Giuliano Mega
ac12de37b2
chore: delete leftover code from old threaded erasure coding backend (#1080) 2025-01-20 14:50:33 +00:00
Slava
833e253baa
feat: switch to github arm runners (#1077)
* feat: switch to github arm runners

* chore: fabiocaccamo/create-matrix-action
2025-01-20 14:42:01 +00:00
Arnaud
2ad7c31c85
Remove duplicate header (#971)
Co-authored-by: Giuliano Mega <giuliano.mega@gmail.com>
2025-01-16 13:51:07 +00:00
Arnaud
4606726e27
Add default Content-Disposition header for download (#981) 2025-01-16 13:25:26 +00:00
Adam Uhlíř
68ad804f9e
refactor: marketplace custom errors handling (#1061)
* refactor: marketplace custom errors handling

* chore: update contracts repo

* chore: update contracts submodule
2025-01-16 09:34:44 +00:00
Marko Burčul
39e8e6e6fa
nix: update readme (#1064)
Include the instructions for running nim-codex as a systemd service on
NixOS.

Signed-off-by: markoburcul <marko@status.im>
2025-01-14 08:21:11 +00:00
markoburcul
0cffa02748
nix: add codex service definition
Referenced issue: https://github.com/codex-storage/nim-codex/issues/940

Signed-off-by: markoburcul <marko@status.im>
2025-01-10 17:49:20 +01:00
markoburcul
3dc7224330
nix: update nix packages and dependencies in flake
Signed-off-by: markoburcul <marko@status.im>
2025-01-10 17:49:18 +01:00
Arnaud
f25c555d59
Chore/update nim version (#1052)
* Move to version 2.0.6

* Update nim-confutils submodule to latest version

* Update dependencies

* Update Nim version to 2.0.12

* Add gcsafe pragma

* Add missing import

* Update specific conf for Nim 2.x

* Fix method signatures

* Revert erasure coding attempt to fix bug

* More gcsafe pragma

* Duplicate code from libp2p because it is not exported anymore

* Fix camelcase function names

* Use alreadySeen because need is not a bool anymore

* newLPStreamReadError does not exist anymore so use another error

* Replace ValidIpAddress by IpAddress

* Add gcsafe pragma

* Restore maintenance parameter deleted by mistake when removing esasure coding fix attempt code

* Update method signatures

* Copy LPStreamReadError code from libp2p which was removed

* Fix camel case

* Fix enums in tests

* Fix camel case

* Extract node components to a variable to make Nim 2 happy

* Update the tests using ValidIpAddress to IpAddress

* Fix cast for value which is already an option

* Set nim version to 2.0.x for CI

* Set nim version to 2.0.x for CI

* Move to miniupnp version 2.2.4 to avoid symlink error

* Set core.symlinks to false for Windows for miniupnp >= 2.2.5 support

* Update to Nim 2.0.14

* Update CI nim versions to 2.0.14

* Try with GCC 14

* Replace apt-fast by apt-get

* Update ubuntu runner to latest

* Use Ubuntu 20.04 for coverage

* Disable CI cache for coverage

* Add coverage property description

* Remove commented test

* Check the node value of seen instead of using alreadySeen

* Fix the merge. The taskpool work was reverted.

* Update nim-ethers submodule

* Remove deprecated ValidIpAddress. Fix missing case and imports.

* Fix a weird issue where nim-confutils cannot find NatAny

* Fix tests and remove useless static keyword
2025-01-10 14:12:37 +00:00
Ben Bierens
caed3c07a3
Fix sending of WantBlocks messages and tracking of peerWants (#1019)
* sends wantBlock to peers with block. wantHave to everyone else

* Cleanup cheapestPeer. Fixes test for peers lists

* Fixes issue where peerWants are only stored for type wantBlock.

* Review comments by Dmitriy

* consistent logging of addresses

* prevents duplicate scheduling. Fixes cancellation

* fast

* Marks cancel-presence situation with todo comment.

* fixtest: testsales enable logging

* Review by Dmitriy: Remember peerWants only if we don't have them.

* rework `wantListHandler` handling

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2025-01-09 22:44:02 +00:00
munna0908
74c46b3651
network: support for uPnP and PMP nat traversal (#1050)
* add support for uPnP and PMP nat traversal

* update cli flags

* merge with master

* use listener address from switch

* update build script to rename version file

* fix styling issues

* update docker files

- remove Disc_ip env
- update CODE_NAT parsing logic

* code cleanup

* move nat flag parsing logic to conf.nim
2025-01-09 18:11:22 +00:00
Adam Uhlíř
407f77871f
chore: warning cleanup (#1055)
* chore: warning cleanup

* chore: fix proper disabling of warning

* chore: ignore the import when not needed
2025-01-08 11:30:54 +00:00
Arnaud
145aa5d84a
chore: remove old fixme (#1060)
* Remove FIXME

* Fix indentation
2025-01-08 07:45:15 +00:00
Arnaud
0badcb662a
chore(openapi): add announceAddresses for debug (#994)
Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
2025-01-08 07:34:54 +00:00
Adam Uhlíř
4b99b58645
fix: unhide log-format cli flag (#1049) 2024-12-19 12:03:50 +00:00
Eric
6ff4d30b43
fix(tests/integration): fix bootstrap node spr (#1048)
After a change in PR #1031, bootstrap node sprs may not work when Hardhat nodes are started with the tests. This fixes it by appending all started client's and provider's SPR to a sequence, and using that sequence of SPRs to start the next node. This means all subsequently started nodes will be connected to its previously started peers.

This also fixes the case when bootstrap SPRs would not be present if no clients were started.
2024-12-18 08:10:36 +00:00
Eric
8645d336ff
refactor(trackedfutures): remove return of future from tracked futures api (#1046)
- cleans up all instances of `.track` to use the `module.trackedfutures.track(future)` procedure, for better readability
- removes the `track` override that is no longer used in the codebase
2024-12-18 07:39:03 +00:00
Ben Bierens
20bb5e5a38
Applies multinodessuite in twonodessuite (#1031)
* Sets timeout for codexclient httpClient. Adds reliable transfer test.

* disable new test to check timeout setting in CI

* restores new test

* adds heartbeat log and logfile to ci output

* fixes suite

* fixes blocked output stream by switching to multinode fixture

* new twonodessuite based on multinodesuite

* Applies updated twonodessuite

* removes heartbeat log

* applies multinodesuite in testsales

* applies multinodesuite in testmarketplace

* fixes account fetch for host and client in testmarketplace

* adds waitTillNextPeriod at end of marketplace test

* Uses marketplacesuite in testmarketplace
2024-12-17 13:01:41 +00:00
Eric
c498e2f53b
fix(nodeprocess): asyncspawn capture output (#1045)
- Ensures no exceptions are raised from `captureOutput`
- Asyncspawns the future to ensure errors are not silently swallowed
2024-12-17 09:51:38 +00:00
Eric
f0f04ddf1d
refactor(then): removes then util (#1047)
- removes then util as it is no longer being used in the codebase
2024-12-17 09:51:05 +00:00