Increase blocksize from ~8KiB to ~64KiB and remove useless unbounded prefetching on download
* increased upload/download speed on my Win10 Zen3 box to 50/150 MB/s (with FSStore)
* made manifest files 8x smaller
* two more changes in the tests to make them pass
Closes#263
* upgrading nimbus build system to latest
* use new `updateRecord`
* bump nim-libp2p-dht
* bump nimbus build system
* bumping deps
* update to latest dht
Support latest version of nim-ethers which relies on nim-eth.
NOTE: This commit points to PR #31 which is a branch of nim-ethers. That branch should probably be merged first, and this PR updated to point to the commit on main.
Make RequestId, SlotId, Nonce, PurcahseId distinct types.
Add/modify conversions to support the distinct type (ABI encoding/decoding, JSON encoding, REST decoding).
Update tests
- rename `ContractId` to `SlotId`
- add `RequestId`, `PurchaseId`, `Nonce` types as aliases of `array[32, byte]`
- rename `Proving.contracts` to `Proving.slots`
- change signatures of `isSlotCancelled` and `isCancelled` to use `SlotId` and `RequestId` types, respectively.
- change all references to `RequestId`, `SlotId`, and `PurchaseId`
The initial goal of this patch was to allow to download of a file via REST API in exactly the same size as it was uploaded, which required adding fields Chunker.offset and Manifest.originalBytes to keep that size. On top of that, we added more integrity checks to operations on Manifest, and reorganized TestNode.nim to test the actual interaction between node.store and node.retrieve operations.
Note that the wire format of Manifest was changed, so we need to recreate all BlockStores.
* Download without padding
* Fixed chunker tests
* Chunker: get rid of RabinChunker
* Verify offset in the chunker tests
* Use manifest.originalBytesPadded in StoreStream.size
* StoreStream: replace emptyBlock with zeroMem
* Manifest.bytes: compute how many bytes corresponding StoreStream(Manifest, pad) will return
* Manifest: verify originalBytes and originalLen on new/encode/decode
Also set originalBytes in each Manifest creation/update scenario
* Manifest: comments, split code into sections
* Reordered parameters to deal with int64 size in 32-bit builds
* TestNode.nim: combine Store and Retrieve tests
1. Instead of copy-pasting code from node.nim, new test calls node.store() and node.retrieve() in order to check that they can correctly store and then retrieve data
2. New test compares only file contents, manifest contents considered an implementation detail
3. New test chunks at odd chunkSize=BlockSize/1.618 in order to ensure that data retrieved correctly even when buffer sizes mismatch
* TestNode.nim: code refactoring
* Manifest.add: one more test
* Manifest.verify: return Result instead of raising Defect
* Node.store: added blockSize parameter
change return type for `method getBlock` from `Future[?!(?Block)]` to
`Future[?!Block]`
use `type BlockNotFoundError = object of CodexError` to differentiate between
"block not found in the store" and other errors
also make some logic and error handling/messages more consistent across
BlockStore implementations
closes#177closes#182closes#210
alternative to #205, #209
ECC decoding can be done with any encoded.K blocks, and now we start it once any K blocks arrived.
Previous code essentially waited for all blocks to arrive (or fail to arrive), that may mean unnecessary delays.
New code will sometimes ECC-decode when parity blocks were retrieved faster that data blocks (instead of waiting for data blocks from network), but we expect that it's OK due to extremely fast Leopard decoding (~~500 MB/s)
* [contracts] ContractInteractions.new() now requires account parameter
* [cli] Only perform Ethereum interactions when --eth-account is specified
* [cli] Add --persistence option that is disabled by default
* [cli] Use Option for ethDeployment parameter
* [node] Better error reporting when Ethereum node cannot be reached
also fix a discrepancy where cli option `--cache-size` is documented as
`0 disables the cache` in `codex/conf.nim`, but previously the value `0` would
result in a cache being constructed with default parameter values for
`CacheStore.new()`
Closes#180
Bulat-Ziganshin
Dmitriy Ryajov
Mark Spanbroek
Eric Mastro
Michael Bradley