256 Commits

Author SHA1 Message Date
Dmitriy Ryajov
2e8b39cf7c
fix random failing test (#77) 2022-04-07 17:08:43 -06:00
Tanguy
aa50127c14
Compiler dagger.nim in the CI (#71) 2022-04-06 14:00:25 +02:00
Dmitriy Ryajov
ffa9b624f1
Integrate erasure (#73)
* wip: adding request for storage endpoint

* wire in erasure coding

* fix tests for erasure coding

* put type definitions into separate file

* integrate erasure coding

* change run/shutdown to start/stop

* temporary sleep, otherwise the fsstore blocks
2022-04-05 18:34:29 -06:00
Dmitriy Ryajov
49c41e27b7
Erasure coding dataset (#72)
* add erasure coding support

* add leopard support

* erasure coding tests
2022-04-05 13:12:59 -06:00
Dmitriy Ryajov
3222f436cc
Protected manifest (#69)
* add protected manifes type

* encode protected manifest

* add empty block type handling

* add tests for protected manifest
2022-04-04 18:46:13 -06:00
Mark Spanbroek
befebcf325 [sales] Make storage available again when offer expires 2022-04-04 11:29:32 +02:00
Mark Spanbroek
e46c9816fa [sales] Make storage available again when offer declined 2022-04-04 11:29:32 +02:00
Mark Spanbroek
11f7b4b4d8 [sales] Clean up tests 2022-04-04 11:29:32 +02:00
Mark Spanbroek
3810889fcb [sales] Do not call onSale() when someone else's offer is selected 2022-04-04 11:29:32 +02:00
Mark Spanbroek
71f25d40c7 [market] offerStorage() returns submitted offer 2022-04-04 11:29:32 +02:00
Mark Spanbroek
94e34e7d28 [market] requestStorage() returns submitted request 2022-04-04 11:29:32 +02:00
Mark Spanbroek
2ffde32772 [sales] call onSale() when offer has been selected 2022-04-04 11:29:32 +02:00
Mark Spanbroek
71baedb5ec [sales] Set offer expiry timestamp 2022-04-04 11:29:32 +02:00
Mark Spanbroek
76be728f61 [sales] Availability.new -> Availability.init 2022-04-04 11:29:32 +02:00
Mark Spanbroek
291c506e0b [market] Fix timing of test for slow CI 2022-04-04 11:29:32 +02:00
Mark Spanbroek
dd89a55351 [market] Add Market.getTime() 2022-04-04 11:29:32 +02:00
Mark Spanbroek
061b32296a Introduce Sales 2022-04-04 11:29:32 +02:00
Mark Spanbroek
75ec8c0bfd [market] wait until request/offer expires 2022-04-04 10:03:46 +02:00
Mark Spanbroek
8ff748eff8 [market] select a storage offer 2022-04-04 10:03:46 +02:00
Mark Spanbroek
5ea64522d8 [market] subscribe to offers for a certain request 2022-04-04 10:03:46 +02:00
Mark Spanbroek
9f76e714a2 [market] submit storage offers 2022-04-04 10:03:46 +02:00
Mark Spanbroek
e52153e2ba [market] Require a storage contract with a signer 2022-04-04 10:03:46 +02:00
Mark Spanbroek
e78a0fb43e [contracts] Wrap contracts in Market abstraction 2022-04-04 10:03:46 +02:00
Mark Spanbroek
bd8f4d5d74 [tests] Extract common examples into separate module 2022-04-04 10:03:46 +02:00
Mark Spanbroek
5b5f3335d6 [purchasing] Ignore offers that are about to expire 2022-04-04 10:03:46 +02:00
Mark Spanbroek
dcfd6be1c6 [purchasing] Ignore expired offers 2022-04-04 10:03:46 +02:00
Mark Spanbroek
fe23cb89d7 [purchasing] Select cheapest offer 2022-04-04 10:03:46 +02:00
Mark Spanbroek
04c291b05c [purchasing] rename requests -> requested 2022-04-04 10:03:46 +02:00
Mark Spanbroek
7c9c244836 [purchasing] Remove PurchaseRequest; use StorageRequest 2022-04-04 10:03:46 +02:00
Mark Spanbroek
9ade3fdd37 [contracts] StorageRequest/Offer is object instead of tuple 2022-04-04 10:03:46 +02:00
Mark Spanbroek
5f10549f19 Introduce Purchasing and Market 2022-04-04 10:03:46 +02:00
Mark Spanbroek
a3fdd35f73 [contracts] Split marketplace.nim into requests.nim and offers.nim 2022-04-04 10:03:46 +02:00
Dmitriy Ryajov
43cea1743a
Node storestream (#62)
* use storestream for retrieval

* use `len` prop on manifest

* remove read pattern

* make blocksize and chunksize consistent

* fix tests

* fix chunker tests - now padded by default
2022-03-29 20:43:35 -06:00
Mark Spanbroek
42edbf5076 [contracts] Remove obsolete marketplace tests 2022-03-28 11:59:01 +02:00
Mark Spanbroek
efb4f5c375 [contracts] Update to new marketplace design 2022-03-28 11:59:01 +02:00
Mark Spanbroek
87bd947d3d [contracts] Update to new collateral design 2022-03-28 11:59:01 +02:00
Mark Spanbroek
d07a5cd19e [contracts] Run contract tests in CI
Introduce new nimble 'test' task that runs *all* tests.
Renames testAll to testDagger, because testAll didn't
run all tests, it forgot to run the contract tests.
2022-03-28 11:59:01 +02:00
Dmitriy Ryajov
e965f5e0de
Seekable stream (#58)
* cleanup manifest

* adding a "seekable" base stream
2022-03-21 12:09:59 -06:00
Dmitriy Ryajov
d204b518e8
Make block a ref (#56)
* cleanup manifest

* make block a ref type
2022-03-18 13:50:53 -06:00
Dmitriy Ryajov
472e1e6fc3
Store stream (#55)
* make manifest a ref object

* capitalize tests

* add storestream and tests

* fix tests

* run storestream tests
2022-03-17 07:56:46 -06:00
Dmitriy Ryajov
f2aa945991
Manifest block size (#53)
* add `blockSize` and `size` to manifest

* fix tests

* fix decoding/encoding

* re-export coders

* style
2022-03-15 12:47:31 -06:00
Dmitriy Ryajov
f414c695be
Cleanup-manifest (#52)
* adding nim-blscurve

* adding taskpools

* cleanup manifest
2022-03-14 10:06:36 -06:00
Eric Mastro
70cbdff901
feat: introduce LRU cache (#50)
* feat: introduce LRU cache

Replace `MemoryStore` with LRU caching mechanism.

`lrucache` library was forked to https://github.com/status-im/lrucache.nim.

Co-authored-by: Eric Mastro <eric.mastro@gmail.com>

# Conflicts:
#	dagger/dagger.nim
#	dagger/stores.nim
#	dagger/stores/manager.nim
#	tests/dagger/blockexc/testengine.nim
#	tests/dagger/helpers/nodeutils.nim
#	tests/dagger/testnode.nim
#	tests/dagger/teststores.nim

* feat: introduce --cache-size CLI option

Allow for a value of `0` to disable the cache.

# Conflicts:
#	dagger/dagger.nim

* allow dynamic block size in cache

allow block size to be variable/dynamic

update lrucache to use updated lrucache dep

Using removeLru proc, added tests

* Refactor CacheStore init block

Co-authored-by: Michael Bradley, Jr <michaelsbradleyjr@gmail.com>
2022-03-02 10:30:42 -06:00
Eric Mastro
2e5c28781c feat: integrate dagger contracts
Integrate dagger contracts from `nim-dagger-contracts` repo.

Add `dagger-contracts`, `nim-web3`, and all of `nim-web3`’s transitive deps as submodule deps to `nim-dagger`. Note: `nim-web3` and its transitive deps may no longer be needed when we switch to `nim-ethers`.

Add a `testContracts` nimble task to test all of the contracts functionality. Namely, this spins up an ethereum simulator, deploys the contracts (in `dagger-contracts`), runs the contract tests, and finally, regardless of success/error, kills the ethereum sim processes. The nimble task can be run with `./env.sh nimble testContracts`.

We also tested `nim-dagger-contracts` as a submodule dep of `nim-dagger`, and while the tests run as expected, the preference is to merge `nim-dagger-contracts` inside of `nim-dagger` for ease of parallel development. There’s also a high probability that `nim-dagger-contracts` is not being used as a dep by other projects. Are there any strong objections to this?

Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
2022-02-04 15:34:56 +11:00
Dmitriy Ryajov
5f48de6a44
cleanup manifest (#44)
* cleanup manifest

* Attach decode to `Manifest`

* move manifest container to it's own file

* rebase main

* update header license year
2022-01-12 19:55:51 -06:00
Dmitriy Ryajov
68a45d8f57
use tryGet, which is more informative than get (#45) 2022-01-12 18:42:18 -06:00
Dmitriy Ryajov
2188f69273
Minor cleanup and update deps (#42)
* abort download if atleast one block failed

* make chunker a regular object

* updating questionable

* updating vendor deps

* cleanup unused imports

* Remove mention of Waku
2022-01-12 16:19:37 -06:00
Dmitriy Ryajov
15ef6c78eb
fix treehash generation (#39) 2022-01-10 21:04:00 -06:00
Dmitriy Ryajov
47a0d4ef7a
Move empty digest (#38)
* improve empty digest initialization

* cleanup warnings
2022-01-10 20:25:27 -06:00
Dmitriy Ryajov
a9721dd65f
Block.init should return Result (#37)
* Block.init should return Result

* use get() instead of tryGet()
2022-01-10 20:25:13 -06:00