ec66e42e73
fix(rest): download unknown cid ( #48 )
...
Downloading a CID that is unknown to the network would wait for the blockexchange to try to get the data. However, a subsequent API call, eg upload for that CID, or a blockexchange timeout would cause the dagger node to crash, due to an attempt to complete a future that had already been completed.
We were able to narrow this down to an attempt to call `resp.finish()` when the download block had already returned an `RestApiResponse` (and never sent any chunks to the response from the libp2p stream).
Change the HTTP response code for an unknown cid from 400 to 404.
Fix spelling mistake “cunk”.
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
2022-02-03 08:02:18 -06:00
5f48de6a44
cleanup manifest ( #44 )
...
* cleanup manifest
* Attach decode to `Manifest`
* move manifest container to it's own file
* rebase main
* update header license year
2022-01-12 19:55:51 -06:00
68a45d8f57
use tryGet, which is more informative than get ( #45 )
2022-01-12 18:42:18 -06:00
b315c759bb
refreshing stew ( #43 )
2022-01-12 16:52:09 -06:00
2188f69273
Minor cleanup and update deps ( #42 )
...
* abort download if atleast one block failed
* make chunker a regular object
* updating questionable
* updating vendor deps
* cleanup unused imports
* Remove mention of Waku
2022-01-12 16:19:37 -06:00
15ef6c78eb
fix treehash generation ( #39 )
2022-01-10 21:04:00 -06:00
47a0d4ef7a
Move empty digest ( #38 )
...
* improve empty digest initialization
* cleanup warnings
2022-01-10 20:25:27 -06:00
a9721dd65f
Block.init should return Result ( #37 )
...
* Block.init should return Result
* use get() instead of tryGet()
2022-01-10 20:25:13 -06:00
9d17c91d4c
adding basic comments to rest api ( #36 )
...
* adding basic comments to rest api
* Update dagger/rest/api.nim
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
2022-01-10 14:35:52 -06:00
fbe161a073
Node setup ( #32 )
...
* settup basic nim node
* adding http utils
* adding confutils
* rough rest api proto
* adding missing deps
* turn tls emulation off
* adding toml serialization
* wip
* adding missing deps
* make sure to clean old state in teardown
* adding file upload rest endpoint
* renaming blockexchange to networkstore
* updating nim-presto
* updating libp2p
* wip adding streaming upload
* reworked chunking
* bump to latest unstable
* adding asyncfutures stream
* make streamable
* deleting unused files
* reworking stores api
* use new stores api
* rework blockset and remove blockstream
* don't return option from constructor
* rework chunker
* wip implement upload
* fix tests
* move unrelated logic to engine
* don't print entire message
* logging
* basic encode/decode to/from dag-pb
* add basic upload/download support
* fix tests
* renaming blockset to manifest
* don't pass config to node
* remove config and use new manifest
* wip: make endpoints more reliable
* wip: adding node tests
* include correct manifest test
* removing asyncfutures
* proper chunking of files
* simplify stream reading
* test with encoding/decoding with many blocks
* add block storing tests
* adding retrieval test
* add logging
* tidy up chunker
* tidy up manifest and node
* use default chunk size
* fix tests
* fix tests
* make sure Eof is set properly
* wip
* minor cleanup
* add file utils
* cleanup config
* splitout DaggerServer and "main"
* remove events since they are not used
* add broadcast method to network peer
* add and wire localstore
* use localstore in the node
* wip
* logging
* move file utils
* use the constant
* updating deps
* fix memstore
* use latest libp2p unstable
* fix tests
* rework block streaming
* don't fail storing if the block already exists
* add helper info endpoint
* correct comment
* rename localstore to fsstore
* fix tests
* remove unused tests
* add test to retrieve one block
* move some test files around
* consolidate setup
* Update dagger/blockexchange/engine.nim
Co-authored-by: Tanguy <tanguy@status.im>
* typo
* better block path handling
* don't inherit rootobj
* remove useless template
* Update tests/dagger/blockexc/testblockexc.nim
Co-authored-by: markspanbroek <mark@spanbroek.net>
* use isMainModule
* use proper flag for starter/stoped
* cleanup optional use
* wrap in isMainModule
* use `cancelAndAwait`
* remove unused imports
* wip
* don't use optional
* use functional error api
* rework store tests and add fs tests
* Block.new() to Block.init()
* don't use optional for engine blocks
* use result instead of optional for getBlock
* remove unused imports
* move stopping servers to `shutdown`
* use result instead of optional
* rework with results
* fix tests
* use waitFor in signal handlers
* error helper
* use `?` and mapFailure where possible
* remove unnecesary `=?`
* improve empty cid digest initialization
Co-authored-by: Tanguy <tanguy@status.im>
Co-authored-by: markspanbroek <mark@spanbroek.net>
2022-01-10 09:32:56 -06:00
a7628e204a
add env.sh shim to project root ( #34 )
...
`env.sh` is copied verbatim from:
https://github.com/status-im/nimbus-build-system#envsh
2021-12-20 13:12:18 -06:00
0f7f5237ad
readding gitignore rules
2021-11-25 09:37:12 +01:00
1663c1d71d
don't run win i386 tests
2021-11-25 09:37:12 +01:00
aa5260df73
update binaries cache
2021-11-25 09:37:12 +01:00
879a76fa41
make test
2021-11-25 09:37:12 +01:00
4d136a2e4c
make ci use new build system
2021-11-25 09:37:12 +01:00
ff498f4280
wip
2021-11-25 09:37:12 +01:00
eb8ee2dcdf
adding gitignore
2021-11-25 09:37:12 +01:00
2c460c575f
adding submodules
2021-11-25 09:37:12 +01:00
4c2344cbd9
wip
2021-11-25 09:37:12 +01:00
8522f516df
remove unused file
2021-11-25 09:37:12 +01:00
623cfe2e02
adding nimbus build system
2021-11-25 09:37:12 +01:00
90a6131d7d
Fix occasional test failure due to timeout
2021-11-16 16:51:40 +01:00
13f88096ef
Disable ObservableStores warning
...
Rationale: In Nim 1.2.x this warning was triggered
too easily, this was fixed in Nim 1.4.x. Because
1.2.x is still used, we disable this warning to ensure
that the long list of ObservableStores warnings does not
obfuscates other warnings.
2021-11-16 16:51:24 +01:00
ef5fa96c7d
Fix unused import warnings
2021-11-16 16:51:24 +01:00
536b2eaad5
Disable LockLevel warnings
...
Rationale: we don't use this experimental Nim feature,
and its warnings obfuscate other warnings that may
be relevant.
2021-11-16 16:51:24 +01:00
ce66e43440
fix compilation issues with latest libp2p ( #27 )
...
* fix compilation issues with latest libp2p
* change package description
* fix CI compiler
* try with nim 1.4.2
* try with 1.4.2
* use more standardized ci config
* set reasonable log level
* remove nim version dependency
* reverting to 1.4.6 as otherwise it fails on amd64
2021-10-29 13:30:52 -06:00
2fb39ca4a3
Rename bitswap ( #25 )
...
* use PeerInfo in event handlers
* use CidV1 and raw multicodec as default
* add block stream abstraction
* raises defect
* adding dataset abstraction
* move blockstream into own dir
* reorg files and fix tests
* rename dataset to blockset
* wip
* wip
* adding basic test for treehash algo
* run blockset tests along with with the rest
* remove obsolete contents
* fix chunker tests
* rename bitswap and move to stores
* rename bitwsap to blockexc and move to stores
* moare project structure reorg
2021-08-30 13:25:20 -06:00
b049e54d9e
Update CI script
...
Copied from lib-p2p master; fixes error with windows dependencies.
2021-06-07 13:21:34 +02:00
6bd4260911
Replace pricing exchange by account exchange
...
Rationale: price is no longer set per peer, but per chunk.
Only the Ethereum accounts of the peers needs to be exchanged.
2021-06-07 09:50:17 +02:00
d4dd15734e
Remove double bookkeeping in peerHave and peerPrices
2021-06-07 09:50:17 +02:00
e3b68f96d8
Remove debt ratio
...
Rationale: selecting peers based on debt ratio in bytes
is no longer needed when the bandwidth for each chunk is
compensated.
2021-06-07 09:50:17 +02:00
fa05bffeac
Pay per-block price instead of per-peer price
2021-06-07 09:50:17 +02:00
f69827d582
Send block prices
2021-06-07 09:50:17 +02:00
98d8571ceb
Simplify
2021-06-07 09:50:17 +02:00
0a28338136
Simplify test
2021-06-07 09:50:17 +02:00
c57ecde046
Update to version 0.9.1 of questionable
2021-06-07 09:50:17 +02:00
0e34b69d7d
Add block prices to peer context
2021-06-07 09:50:17 +02:00
0ac876dbc3
Add pricing to block presence messages
2021-06-07 09:50:17 +02:00
11da2339de
Hard-code asset address
...
We're only going to support a single asset for now.
2021-06-07 09:50:17 +02:00
811b29fb5e
Receive payments for blocks that were sent
2021-06-07 09:50:17 +02:00
5160b267b6
Use reference semantics for wallets
2021-06-07 09:50:17 +02:00
2d4a735e5f
Update to latest versions of nitro and questionable
2021-06-07 09:50:17 +02:00
753bde7775
Fix build failure on 32 bit platforms
2021-06-07 09:50:17 +02:00
ee978f354e
Engine sends payments for received blocks
2021-06-07 09:50:17 +02:00
9512bbc50b
Add proc to engine that pays peers for bytes
2021-06-07 09:50:17 +02:00
ad48f55aaa
Update to latest versions of nitro and questionable
2021-06-07 09:50:17 +02:00
d9e5bc934f
Move peer context into its own module
2021-06-07 09:50:17 +02:00
4ce3f6d3da
Add nitro wallet to BitswapEngine
2021-06-07 09:50:17 +02:00
22f4c277dd
Revert "Add nitro wallet to BitswapNetwork"
...
This reverts commit e55ac4e9de
2021-06-07 09:50:17 +02:00
Eric Mastro
Dmitriy Ryajov
Michael Bradley
Mark Spanbroek