* rework discovery with async queues
* increase max message size for large manifests
* increase sleep time to 100 millis
* pass config
* check for nil on start/stop
* fix tests and split out discovery tests
* don't auto mount network
* add discovery tests
* rework moc discovery
* move discovery moc to disc dir
* don't force logging syncs
* don't force moc discovery on all tests
* rework discovery with methods
* add top level utils file
* don't use asyncCheck
* don't pass entire blocks to list blocks calback
* spelling
* - don't send want reqs to peers reporting the cid
- Don't request blocks directly on presense update, use `requestBlock`
* bug, nodes should not have blocks in local store
* Add failing test
* prefetch blocks so that download isn't serial
* if request already pending, return the handle
* fire discovery if no peers report block as have
* only query discovery if not enough nodes for cid
* wrap async req in template
* use non awaiting version of queue routines
* rework E2E tests as unittest
* re-add chronicles sinks
Co-authored-by: Tanguy <tanguy@status.im>
* rework discovery with async queues
* misc style changes
* increase max message size for large manifests
* use upraises and avoid exceptions on key access
* increase sleep time to 100 millis
* pass config
* make list blocks trigger a callback on each block
* check for nil on start/stop
* fix tests and split out discovery tests
* don't auto mount network
* add list block tests
* add discovery tests
* rework moc discovery
* move discovery moc to disc dir
* don't force logging syncs
* don't force moc discovery on all tests
* rework discovery with methods
* add top level utils file
* don't use asyncCheck
* don't pass entire blocks to list blocks calback
* spelling
* wip: adding request for storage endpoint
* wire in erasure coding
* fix tests for erasure coding
* put type definitions into separate file
* integrate erasure coding
* change run/shutdown to start/stop
* temporary sleep, otherwise the fsstore blocks
* use storestream for retrieval
* use `len` prop on manifest
* remove read pattern
* make blocksize and chunksize consistent
* fix tests
* fix chunker tests - now padded by default
This PR achieves the following:
1. Improves the exception handling when dialling a peer fails or an unknown error occurs.
2. Add a `Content-Type` header to the `/download` endpoint of `application/octet-stream`, which is [defined by MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types#types) as meant to be used for "generic binary data (or binary data whose true type is unknown)".
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
Downloading a CID that is unknown to the network would wait for the blockexchange to try to get the data. However, a subsequent API call, eg upload for that CID, or a blockexchange timeout would cause the dagger node to crash, due to an attempt to complete a future that had already been completed.
We were able to narrow this down to an attempt to call `resp.finish()` when the download block had already returned an `RestApiResponse` (and never sent any chunks to the response from the libp2p stream).
Change the HTTP response code for an unknown cid from 400 to 404.
Fix spelling mistake “cunk”.
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
* adding basic comments to rest api
* Update dagger/rest/api.nim
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
Co-authored-by: Michael Bradley <michaelsbradleyjr@gmail.com>
Mark Spanbroek
Dmitriy Ryajov
Tanguy
Eric Mastro
Eric Mastro