30 Commits

Author SHA1 Message Date
Slava
484124db09
Release v0.1.4 (#912)
* fix: createReservation lock (#825)

* fix: createReservation lock

* fix: additional locking places

* fix: acquire lock

* chore: feedback

Co-authored-by: markspanbroek <mark@spanbroek.net>
Signed-off-by: Adam Uhlíř <adam@uhlir.dev>

* feat: withLock template and fixed tests

* fix: use proc for MockReservations constructor

* chore: feedback

Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
Signed-off-by: Adam Uhlíř <adam@uhlir.dev>

* chore: feedback implementation

---------

Signed-off-by: Adam Uhlíř <adam@uhlir.dev>
Co-authored-by: markspanbroek <mark@spanbroek.net>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>

* Block deletion with ref count & repostore refactor (#631)

* Fix StoreStream so it doesn't return parity bytes  (#838)

* fix storestream so it doesn\'t return parity bits for protected/verifiable manifests

* use Cid.example instead of creating a mock manually

* Fix verifiable manifest initialization (#839)

* fix verifiable manifest initialization

* fix linearstrategy, use verifiableStrategy to select blocks for slots

* check for both strategies in attribute inheritance test

* ci: add verify_circuit=true to the releases (#840)

* provisional fix so EC errors do not crash the node on download (#841)

* prevent node crashing with `not val.isNil` (#843)

* bump nim-leopard to handle no parity data (#845)

* Fix verifiable manifest constructor (#844)

* Fix verifiable manifest constructor

* Add integration test for verifiable manifest download

Add integration test for testing download of verifiable dataset after creating request for storage

* add missing import

* add testecbug to integration suite

* Remove hardhat instance from integration test

* change description, drop echo

---------

Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
Co-authored-by: gmega <giuliano.mega@gmail.com>

* Bump Nim to 1.6.21 (#851)

* bump Nim to 1.6.21 (range type reset fixes)

* remove incompatible versions from compiler matrix

* feat(rest): adds erasure coding constraints when requesting storage (#848)

* Rest API: add erasure coding constraints when requesting storage

* clean up

* Make error message for "dataset too small" more informative.

* fix API integration test

---------

Co-authored-by: gmega <giuliano.mega@gmail.com>

* Prover workshop band-aid (#853)

* add prover bandaid

* Improve error message text

Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
Signed-off-by: Giuliano Mega <giuliano.mega@gmail.com>

---------

Signed-off-by: Giuliano Mega <giuliano.mega@gmail.com>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>

* Bandaid for failing erasure coding (#855)

* Update Release workflow (#858)

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Fixes prover behavior with singleton proof trees (#859)

* add logs and test

* add Merkle proof checks

* factor out Circom input normalization, fix proof input serialization

* add test and update existing ones

* update circuit assets

* add back trace message

* switch contracts to fix branch

* update codex-contracts-eth to latest

* do not expose prove with prenormalized inputs

* Chronos v4 Update (v3 Compat Mode) (#814)

* add changes to use chronos v4 in compat mode

* switch chronos to compat fix branch

* use nimbus-build-system with configurable Nim repo

* add missing imports

* add missing await

* bump compat

* pin nim version in Makefile

* add await instead of asyncSpawn to advertisement queue loop

* bump DHT to v0.5.0

* allow error state of `onBatch` to propagate upwards in test code

* pin Nim compiler commit to avoid fetching stale branch

* make CI build against branch head instead of merge

* fix handling of return values in testslotqueue

* Downgrade to gcc 13 on Windows (#874)

* Downgrade to gcc 13 on Windows

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Increase build job timeout to 90 minutes

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

---------

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Add MIT/Apache licenses (#861)

* Add MIT/Apache licenses

* Center "Apache License"

Signed-off-by: Giuliano Mega <giuliano.mega@gmail.com>

* remove wrong legal entity; rename apache license file

---------

Signed-off-by: Giuliano Mega <giuliano.mega@gmail.com>

* Add OPTIONS endpoint to allow the content-type header for the upload endpoint (#869)

* Add OPTIONS endpoint to allow the content-type header
exec git commit --amend --no-edit -S

* Remove useless header "Access-Control-Headers" and add cache

Signed-off-by: Arnaud <arnaud@status.im>

---------

Signed-off-by: Arnaud <arnaud@status.im>
Co-authored-by: Giuliano Mega <giuliano.mega@gmail.com>

* chore: add `downtimeProduct` config parameter (#867)

* chore: add `downtimeProduct` config parameter

* bump codex-contracts-eth to master

* Support CORS preflight requests when the storage request api returns an error  (#878)

* Add CORS headers when the REST API is returning an error

* Use the allowedOrigin instead of the wilcard when setting the origin

Signed-off-by: Arnaud <arnaud@status.im>

---------

Signed-off-by: Arnaud <arnaud@status.im>

* refactor(marketplace): generic querying of historical marketplace events (#872)

* refactor(marketplace): move marketplace events to the Market abstraction

Move marketplace contract events to the Market abstraction so the types can be shared across all modules that call the Market abstraction.

* Remove unneeded conversion

* Switch to generic implementation of event querying

* change parent type to MarketplaceEvent

* Remove extra license file (#876)

* remove extra license

* center "apache license"

* Update advertising (#862)

* Setting up advertiser

* Wires up advertiser

* cleanup

* test compiles

* tests pass

* setting up test for advertiser

* Finishes advertiser tests

* fixes commonstore tests

* Review comments by Giuliano

* Race condition found by Giuliano

* Review comment by Dmitriy

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Signed-off-by: Ben Bierens <39762930+benbierens@users.noreply.github.com>

* fixes tests

---------

Signed-off-by: Ben Bierens <39762930+benbierens@users.noreply.github.com>
Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>

* feat: add `--payout-address` (#870)

* feat: add `--payout-address`

Allows SPs to be paid out to a separate address, keeping their profits secure.
Supports https://github.com/codex-storage/codex-contracts-eth/pull/144 in the nim-codex client.

* Remove optional payoutAddress

Change --payout-address so that it is no longer optional. There is no longer an overload in `Marketplace.sol` for `fillSlot` accepting no `payoutAddress`.

* Update integration tests to include --payout-address

* move payoutAddress from fillSlot to freeSlot

* Update integration tests to use required payoutAddress

- to make payoutAddress required, the integration tests needed to avoid building the cli params until just before starting the node, otherwise if cli params were added ad-hoc, there would be an error after a non-required parameter was added before a required parameter.

* support client payout address

- withdrawFunds requires a withdrawAddress parameter, directs payouts for withdrawing of client funds (for a cancelled request) to go to that address.

* fix integration test

adds --payout-address to validators

* refactor: support withdrawFunds and freeSlot optional parameters

- withdrawFunds has an optional parameter for withdrawRecipient
- freeSlot has optional parameters for rewardRecipient and collateralRecipient
- change --payout-address to --reward-recipient to match contract signature naming

* Revert "Update integration tests to include --payout-address"

This reverts commit 8f9535cf35b0f2b183ac4013a7ed11b246486964.
There are some valid improvements to the integration tests, but they can be handled in a separate PR.

* small fix

* bump contracts to fix marketplace spec

* bump codex-contracts-eth, now rebased on master

* bump codex-contracts-eth

now that feat/reward-address has been merged to master

* clean up, comments

* Rework circuit downloader (#882)

* Introduces a start method to prover

* Moves backend creation into start method

* sets up three paths for backend initialization

* Extracts backend initialization to backend-factory

* Implements loading backend from cli files or previously downloaded local files

* Wires up downloading and unzipping

* functional implementation

* Fixes testprover.nim

* Sets up tests for backendfactory

* includes libzip-dev

* pulls in updated contracts

* removes integration cli tests for r1cs, wasm, and zkey file arguments.

* Fixes issue where inner-scope values are lost before returning

* sets local proof verification for dist-test images

* Adds two traces and bumps nim-ethers

* Adds separate path for circuit files

* Create circuit dir if not exists

* fix: make sure requestStorage is mined

* fix: correct place to plug confirm

* test: fixing contracts tests

* Restores gitmodules

* restores nim-datastore reference

* Sets up downloader exe

* sets up tool skeleton

* implements getting of circuit hash

* Implements downloader tool

* sets up test skeleton

* Implements test for cirdl

* includes testTools in testAll

* Cleanup building.md

* cleans up previous downloader implementation

* cleans up testbackendfactory

* moves start of prover into node.nim

* Fills in arguments in example command

* Initializes backend in prover constructor

* Restores tests

* Restores tests for cli instructions

* Review comments by Dmitriy, part 1

* Quotes path in download instruction.

* replaces curl with chronos http session

* Moves cirdl build output to 'build' folder.

* Fixes chronicles log output

* Add cirdl support to the codex Dockerfile

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Add cirdl support to the docker entrypoint

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Add cirdl support to the release workflow

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Disable verify_circuit flag for releases

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Removes backendFactory placeholder type

* wip

* Replaces zip library with status-im/zippy library (which supports zip and tar)

* Updates cirdl to not change circuitdir folder

* Switches from zip to tar.gz

* Review comments by Dmitriy

* updates codex-contracts-eth

* Adds testTools to CI

* Adds check for access to config.circuitdir

* Update fixture circuit zkey

* Update matrix to run tools tests on Windows

* Adds 'deps' dependency for cirdl

* Adjust docker-entrypoint.sh to use CODEX_CIRCUIT_DIR env var

* Review comments by Giuliano

---------

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
Co-authored-by: Veaceslav Doina <20563034+veaceslavdoina@users.noreply.github.com>

* Support CORS for POST and PATCH availability endpoints (#897)

* Adds testnet marketplace address to known deployments (#911)

* API tweaks for OpenAPI, errors and endpoints (#886)

* All sort of tweaks

* docs: availability's minPrice doc

* Revert changes to the two node test example

* Change default EC params in REST API

Change default EC params in REST API to 3 nodes and 1 tolerance.

Adjust integration tests to honour these settings.

---------

Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>

---------

Signed-off-by: Adam Uhlíř <adam@uhlir.dev>
Signed-off-by: Giuliano Mega <giuliano.mega@gmail.com>
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
Signed-off-by: Arnaud <arnaud@status.im>
Signed-off-by: Ben Bierens <39762930+benbierens@users.noreply.github.com>
Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
Co-authored-by: markspanbroek <mark@spanbroek.net>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
Co-authored-by: Tomasz Bekas <tomasz.bekas@gmail.com>
Co-authored-by: Giuliano Mega <giuliano.mega@gmail.com>
Co-authored-by: Arnaud <arno.deville@gmail.com>
Co-authored-by: Ben Bierens <39762930+benbierens@users.noreply.github.com>
Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
Co-authored-by: Arnaud <arnaud@status.im>
2024-09-24 13:19:58 +03:00
Slava
15ff87a8bb
Merge latest master into release (#842)
* fix: createReservation lock (#825)

* fix: createReservation lock

* fix: additional locking places

* fix: acquire lock

* chore: feedback

Co-authored-by: markspanbroek <mark@spanbroek.net>
Signed-off-by: Adam Uhlíř <adam@uhlir.dev>

* feat: withLock template and fixed tests

* fix: use proc for MockReservations constructor

* chore: feedback

Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
Signed-off-by: Adam Uhlíř <adam@uhlir.dev>

* chore: feedback implementation

---------

Signed-off-by: Adam Uhlíř <adam@uhlir.dev>
Co-authored-by: markspanbroek <mark@spanbroek.net>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>

* Block deletion with ref count & repostore refactor (#631)

* Fix StoreStream so it doesn't return parity bytes  (#838)

* fix storestream so it doesn\'t return parity bits for protected/verifiable manifests

* use Cid.example instead of creating a mock manually

* Fix verifiable manifest initialization (#839)

* fix verifiable manifest initialization

* fix linearstrategy, use verifiableStrategy to select blocks for slots

* check for both strategies in attribute inheritance test

* ci: add verify_circuit=true to the releases (#840)

* provisional fix so EC errors do not crash the node on download (#841)

---------

Signed-off-by: Adam Uhlíř <adam@uhlir.dev>
Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
Co-authored-by: markspanbroek <mark@spanbroek.net>
Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
Co-authored-by: Tomasz Bekas <tomasz.bekas@gmail.com>
Co-authored-by: Giuliano Mega <giuliano.mega@gmail.com>
2024-06-26 05:38:04 +03:00
Ben Bierens
8378ac4dac
Openapi mismatches (#754)
* Aligns response with openapi spec

* Fixes json serialization for local content response

* Review comments by Mark

* Fixes missed rename

* Removes array type for dataList
2024-03-26 15:12:15 +00:00
Adam Uhlíř
de1714ed06
feat(api): availabilities patch endpoint (#692)
* feat(api): availabilities patch and reservations endpoints

* test: fixing tests and writing

* test: test reservations endpoint

* chore: feedback implementation

* chore: feedback implementation

* test: fix integration tests
2024-03-21 10:53:45 +00:00
Eric
f567f4ec15
refactor: use serde for json de/serialization instead of utils/json (#704)
* json > nim-serde bump

Should wait until serde is integrated into nim-ethers before making these changes as there will be less import exceptions required.

* bump nim-serde

* change func to proc due to chronicles side effects

* import serde into utils/json, use as proxy

import nim-serde into utils/json and use utils/json as a proxy for serde functions, including overloading `%` and `fromJson` for application types.

* update tests to use serde

* bump serde to latest

* remove testjson -- no longer needed

* bump serde in nimble

* updates to reconcile rebase with master
2024-03-19 03:25:13 +00:00
Eric
de88fd2c53
feat: create logging proxy (#663)
* implement a logging proxy

The logging proxy:
- prevents the need to import chronicles (as well as export except toJson),
- prevents the need to override `writeValue` or use or import nim-json-seralization elsewhere in the codebase, allowing for sole use of utils/json for de/serialization,
- and handles json formatting correctly in chronicles json sinks

* Rename logging -> logutils to avoid ambiguity with common names

* clean up

* add setProperty for JsonRecord, remove nim-json-serialization conflict

* Allow specifying textlines and json format separately

Not specifying a LogFormat will apply the formatting to both textlines and json sinks.

Specifying a LogFormat will apply the formatting to only that sink.

* remove unneeded usages of std/json

We only need to import utils/json instead of std/json

* move serialization from rest/json to utils/json so it can be shared

* fix NoColors ambiguity

Was causing unit tests to fail on Windows.

* Remove nre usage to fix Windows error

Windows was erroring with `could not load: pcre64.dll`. Instead of fixing that error, remove the pcre usage :)

* Add logutils module doc

* Shorten logutils.formatIt for `NBytes`

Both json and textlines formatIt were not needed, and could be combined into one formatIt

* remove debug integration test config

debug output and logformat of json for integration test logs

* Use ## module doc to support docgen

* bump nim-poseidon2 to export fromBytes

Before the changes in this branch, fromBytes was likely being resolved by nim-stew, or other dependency. With the changes in this branch, that dependency was removed and fromBytes could no longer be resolved. By exporting fromBytes from nim-poseidon, the correct resolution is now happening.

* fixes to get compiling after rebasing master

* Add support for Result types being logged using formatIt
2024-01-22 23:35:03 -08:00
Eric
f293082ae9
Reverts logging-proxy, commit 27f585eb6f380a6de2fbbef721fdf1a16bd5f600 (#660) 2023-12-20 13:24:40 +11:00
Eric
27f585eb6f
feat: create logging proxy (#653)
* implement a logging proxy

The logging proxy:
- prevents the need to import chronicles (as well as export except toJson),
- prevents the need to override `writeValue` or use or import nim-json-seralization elsewhere in the codebase, allowing for sole use of utils/json for de/serialization,
- and handles json formatting correctly in chronicles json sinks

* Rename logging -> logutils to avoid ambiguity with common names

* clean up

* add setProperty for JsonRecord, remove nim-json-serialization conflict

* Allow specifying textlines and json format separately

Not specifying a LogFormat will apply the formatting to both textlines and json sinks.

Specifying a LogFormat will apply the formatting to only that sink.

* remove unneeded usages of std/json

We only need to import utils/json instead of std/json

* move serialization from rest/json to utils/json so it can be shared

* fix NoColors ambiguity

Was causing unit tests to fail on Windows.

* Remove nre usage to fix Windows error

Windows was erroring with `could not load: pcre64.dll`. Instead of fixing that error, remove the pcre usage :)

* Add logutils module doc

* Shorten logutils.formatIt for `NBytes`

Both json and textlines formatIt were not needed, and could be combined into one formatIt

* remove debug integration test config

debug output and logformat of json for integration test logs

* Use ## module doc to support docgen
2023-12-19 22:12:47 +00:00
Ben Bierens
f1b1fe9152
Adds endpoint to expose quota information (#652) 2023-12-14 10:57:16 +00:00
Eric
3907ca4095
Add get active slot /slots/{slotId} to REST api, use utils/json (#645)
* Add get active slot /slots/{slotId} to REST api, use utils/json

- Add endpoint /slots/{slotId} to get an active SalesAgent from the Sales module. Used in integration tests to test when a sale has reached a certain state. Those integration test changes will be included in a larger PR, coming later.
- Add OpenAPI changes for new endpoint and associated components
- Use utils/json instead of nim-json-serialization. Required exemption of imports from several packages that export nim-json-serialization by default.

* Only except `toJson` from import/export of chronicles
2023-12-07 12:16:36 +11:00
Ben Bierens
40d7714a67
Fixes json formatting for NodeId (#614) 2023-11-15 13:29:17 +00:00
Ben Bierens
80e2ef4eb5
Debug/info shows announce addresses (#610)
* Shows announce addresses in debug/info endpoint.

* Shows announce addresses in debug/info endpoint.
2023-11-12 09:58:40 +01:00
Ben Bierens
cb02962231
Adds endpoint for listing files (manifests) in node. Useful for demo UI. (#599)
* Adds endpoint for listing files (manifests) in node. Useful for demo UI.

* Moves upload/download/files into content API calls.

* Cleans up json serialization for manifest

* Cleans up some more json serialization

* Moves block iteration and decoding to node.nim

* Moves api methods into their own init procs.

* Applies RestContent api object.

* Replaces format methods with Rest objects in json.nim

* Unused import

* Review comments by Adam

* Fixes issue where content/local endpoint clashes with content/cid.

* faulty merge resolution

* Renames content API to data.

* Fixes faulty rebase

* Adds test for data/local API

* Renames local and download api.
2023-11-09 08:47:09 +00:00
Eric
37b3d99c3d
Improve integration testing client (CodexClient) and json serialization (#514)
* Improve integration testing client (CodexClient) and json serialization

The current client used for integration testing against the REST endpoints for Codex accepts and passes primitive types. This caused a hard to diagnose bug where a `uint` was not being deserialized correctly.

In addition, the json de/serializing done between the CodexClient and REST client was not easy to read and was not tested.

These changes bring non-primitive types to most of the CodexClient functions, allowing us to lean on the compiler to ensure we're providing correct typings. More importantly, a json de/serialization util was created as a drop-in replacement for the std/json lib, with the main two differences being that field serialization is opt-in (instead of opt-out as in the case of json_serialization) and serialization errors are captured and logged, making debugging serialization issues much easier.

* Update integration test to use nodes=2 and tolerance=1

* clean up
2023-09-01 15:44:41 +10:00
Jaremy Creechley
e47b38af11
Improving proc/func formatting consistency (#454)
* Fixes/workarounds for nimsuggest failures in codex.nim.

* remove rng prefix - it appears to work now

* format new's to be more consistent

* making proc formatting a bit more consistent
2023-06-22 08:11:18 -07:00
Eric Mastro
2a92dc9702
[marketplace] Simulate invalid proof submissions (#393) 2023-06-22 20:32:18 +10:00
Adam Uhlíř
cfd2cf9302
feat: bigint uses decimal over hex encoding (#452) 2023-06-21 07:46:18 +02:00
Adam Uhlíř
4cd8dd2e05
feat: slots rest api (#443)
Co-authored-by: markspanbroek <mark@spanbroek.net>
2023-06-20 14:52:15 +02:00
markspanbroek
3181361658
Cleanup purchasing state machine (#422)
* [state machine] Allow querying of state properties

* [purchasing] use new state machine

* [state machine] remove old state machine implementation

* [purchasing] remove duplication in error handling
2023-06-05 10:48:06 +02:00
Adam Uhlíř
131d003a0c
feat: collateral per slot (#390)
Co-authored-by: Eric Mastro <github@egonat.me>
2023-04-14 11:04:17 +02:00
Eric Mastro
ccf349bd14
[marketplace] Add Reservations Module (#340)
* [marketplace] reservations module

- add de/serialization for Availability
- add markUsed/markUnused in persisted availability
- add query for unused
- add reserve/release
- reservation module tests
- split ContractInteractions into client contracts and host contracts
- remove reservations start/stop as the repo start/stop is being managed by the node
- remove dedicated reservations metadata store and use the metadata store from the repo instead
- Split ContractInteractions into:
  - ClientInteractions (with purchasing)
  - HostInteractions (with sales and proving)
- compilation fix for nim 1.2

[repostore] fix started flag, add tests

[marketplace] persist slot index
For loading the sales state from chain, the slot index was not previously persisted in the contract. Will retrieve the slot index from the contract when the sales state is loaded.

* Revert repostore changes

In favour of separate PR https://github.com/status-im/nim-codex/pull/374.

* remove warnings

* clean up

* tests: stop repostore during teardown

* change constructor type identifier

Change Contracts constructor to accept Contracts type instead of ContractInteractions.

* change constructor return type to Result instead of Option

* fix and split interactions tests

* clean up, fix tests

* find availability by slot id

* remove duplication in host/client interactions

* add test for finding availability by slotId

* log instead of raiseAssert when failed to mark availability as unused

* move to SaleErrored state instead of raiseAssert

* remove unneeded reverse

It appears that order is not preserved in the repostore, so reversing does not have the intended effect here.

* update open api spec for potential rest endpoint errors

* move functions about available bytes to repostore

* WIP: reserve and release availabilities as needed

WIP: not tested yet

Availabilities are marked as used when matched (just before downloading starts) so that future matching logic does not match an availability currently in use.

As the download progresses, batches of blocks are written to disk, and the equivalent bytes are released from the reservation module. The size of the availability is reduced as well.

During a reserve or release operation, availability updates occur after the repo is manipulated. If the availability update operation fails, the reserve or release is rolled back to maintain correct accounting of bytes.

Finally, once download completes, or if an error occurs, the availability is marked as unused so future matching can occur.

* delete availability when all bytes released

* fix tests + cleanup

* remove availability from SalesContext callbacks

Availability is no longer used past the SaleDownloading state in the state machine. Cleanup of Availability (marking unused) is handled directly in the SaleDownloading state, and no longer in SaleErrored or SaleFinished. Likewise, Availabilities shouldn’t need to be handled on node restart.

Additionally, Availability was being passed in SalesContext callbacks, and now that Availability is only used temporarily in the SaleDownloading state, Availability is contextually irrelevant to the callbacks, except in OnStore possibly, though it was not being consumed.

* test clean up

* - remove availability from callbacks and constructors from previous commit that needed to be removed (oopsie)
- fix integration test that checks availabilities
  - there was a bug fixed that crashed the node due to a missing `return success` in onStore
  - the test was fixed by ensuring that availabilities are remaining on the node, and the size has been reduced
- change Availability back to non-ref object and constructor back to init
- add trace logging of all state transitions in state machine
- add generally useful trace logging

* fixes after rebase

1. Fix onProve callbacks
2. Use Slot type instead of tuple for retrieving active slot.
3. Bump codex-contracts-eth that exposes getActivceSlot call.

* swap contracts branch to not support slot collateral

Slot collateral changes in the contracts require further changes in the client code, so we’ll skip those changes for now and add in a separate commit.

* modify Interactions and Deployment constructors

- `HostInteractions` and `ClientInteractions` constructors were simplified to take a contract address and no overloads
- `Interactions` prepared simplified so there are no overloads
- `Deployment` constructor updated so that it takes an optional string parameter, instead `Option[string]`

* Move `batchProc` declaration

`batchProc` needs to be consumed by both `node` and `salescontext`, and they can’t reference each other as it creates a circular dependency.

* [reservations] rename `available` to `hasAvailable`

* [reservations] default error message to inner error msg

* add SaleIngored state

When a storage request is handled but the request does match availabilities, the sales agent machine is sent to the SaleIgnored state. In addition, the agent is constructed in a way that if the request is ignored, the sales agent is removed from the list of active agents being tracked in the sales module.
2023-04-04 17:05:16 +10:00
markspanbroek
4ffe7b8e06
Generate proofs when required (#383)
* [maintenance] speedup integration test

* [rest api] add proofProbability parameter to storage requests

* [integration] negotiation test ends when contract starts

* [integration] reusable 2 node setup for tests

* [integration] introduce CodexClient for tests

* [node] submit storage proofs when required

* [contracts] Add Slot type

* [proving] replace onProofRequired & submitProof with onProve

Removes duplication between Sales.onProve() and
Proving.onProofRequired()
2023-03-27 15:47:25 +02:00
Ben Bierens
da79660f8e
Enable stylecheck (#353)
* applying styleCheck

* stuck on vendor folder

* Applies style check

* Turns styleCheck back off

* switches to stylecheck:usages

* Fixes empty template casing

* rolls up nim-blscurve, nim-datastore, nim-ethers, nim-leopard, and nim-taskpools.

* bumps nim-confutils and removes unused import from fileutils.nim

* Unused using in fileutils.nim is required by CI

* Reverts bump of nim-confutils module
2023-03-10 08:02:54 +01:00
markspanbroek
4175689745
Load purchase state from chain (#283)
* [purchasing] Simplify test

* [utils] Move StorageRequest.example up one level

* [purchasing] Load purchases from market

* [purchasing] load purchase states

* Implement myRequest() and getState() methods for OnChainMarket

* [proofs] Fix intermittently failing tests

Ensures that examples of proofs in tests are never of length 0;
these are considered invalid proofs by the smart contract logic.

* [contracts] Fix failing test

With the new solidity contracts update, a contract can only
be paid out after it started.

* [market] Add method to get request end time

* [purchasing] wait until purchase is finished

Purchase.wait() would previously wait until purchase
was started, now we wait until it is finished.

* [purchasing] Handle 'finished' and 'failed' states

* [marketplace] move to failed state once request fails

- Add support for subscribing to request failure events.
- Add supporting contract tests for subscribing to request failure events.
- Allow the PurchaseStarted state to move to PurchaseFailure once a request failure event is emitted
- Add supporting tests for moving from PurchaseStarted to PurchaseFailure
- Add state transition tests for PurchaseUnknown.

* [marketplace] Fix test with longer sleepAsync

* [integration] Add function to restart a codex node

* [purchasing] Set client address before requesting storage

To prevent the purchase id (which equals the request id)
from changing once it's been submitted.

* [contracts] Fix: OnChainMarket.getState()

Had the wrong method signature before

* [purchasing] Load purchases on node start

* [purchasing] Rename state 'PurchaseError' to 'PurchaseErrored'

Allows for an exception type called 'PurchaseError'

* [purchasing] Load purchases in background

No longer calls market.getRequest() for every purchase
on node start.

* [contracts] Add `$` for RequestId, SlotId and Nonce

To aid with debugging

* [purchasing] Add Purchasing.stop()

To ensure that all contract interactions have both a
start() and a stop() for

* [tests] Remove sleepAsync where possible

Use `eventually` loop instead, to make sure that we're
not waiting unnecessarily.

* [integration] Fix: handle non-json response in test

* [purchasing] Add purchase state to json

* [integration] Ensure that purchase is submitted before restart

Fixes test failure on slower CI

* [purchasing] re-implement `description` as method

Allows description to be set in the same module where the
state type is defined.

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>

* [contracts] fix typo

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>

* [market] Use more generic error type

Should we decide to change the provider type later

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>
2022-11-08 08:10:17 +01:00
Eric Mastro
8f116cd01e [chore] make RequestId, SlotId, Nonce, PurcahseId distinct
Make RequestId, SlotId, Nonce, PurcahseId distinct types.

Add/modify conversions to support the distinct type (ABI encoding/decoding, JSON encoding, REST decoding).

Update tests
2022-08-26 13:29:09 +10:00
Mark Spanbroek
7f5310d640 [rest api] Allow user to specify amount of hosts 2022-08-08 14:00:55 +02:00
Mark Spanbroek
698baeef35 [purchasing] Remove Purchase.selected
There no longer exists a single host that
is selected for a contract.
2022-08-08 14:00:55 +02:00
Mark Spanbroek
360c03e7c4 [contracts] Rename: maxPrice -> reward 2022-08-08 14:00:55 +02:00
Mark Spanbroek
96ca1de768 Cleanup: remove StorageOffer everywhere 2022-07-20 13:43:20 +02:00
Michael Bradley
209343087c
Change every dagger to codex (#102) 2022-05-19 13:56:03 -06:00