Commit Graph

7 Commits

Author SHA1 Message Date
Bulat-Ziganshin f24ded0f76
Download files without padding (#218)
The initial goal of this patch was to allow to download of a file via REST API in exactly the same size as it was uploaded, which required adding fields Chunker.offset and Manifest.originalBytes to keep that size. On top of that, we added more integrity checks to operations on Manifest, and reorganized TestNode.nim to test the actual interaction between node.store and node.retrieve operations.

Note that the wire format of Manifest was changed, so we need to recreate all BlockStores.

* Download without padding
* Fixed chunker tests
* Chunker: get rid of RabinChunker
* Verify offset in the chunker tests
* Use manifest.originalBytesPadded in StoreStream.size
* StoreStream: replace emptyBlock with zeroMem
* Manifest.bytes: compute how many bytes corresponding StoreStream(Manifest, pad) will return
* Manifest: verify originalBytes and originalLen on new/encode/decode
Also set originalBytes in each Manifest creation/update scenario
* Manifest: comments, split code into sections
* Reordered parameters to deal with int64 size in 32-bit builds
* TestNode.nim: combine Store and Retrieve tests
1. Instead of copy-pasting code from node.nim, new test calls node.store() and node.retrieve() in order to check that they can correctly store and then retrieve data
2. New test compares only file contents, manifest contents considered an implementation detail
3. New test chunks at odd chunkSize=BlockSize/1.618 in order to ensure that data retrieved correctly even when buffer sizes mismatch
* TestNode.nim: code refactoring
* Manifest.add: one more test
* Manifest.verify: return Result instead of raising Defect
* Node.store: added blockSize parameter
2022-08-24 15:15:59 +03:00
Michael Bradley, Jr 3d823dcbc6 [stores] update getBlock return type
change return type for `method getBlock` from `Future[?!(?Block)]` to
`Future[?!Block]`

use `type BlockNotFoundError = object of CodexError` to differentiate between
"block not found in the store" and other errors

also make some logic and error handling/messages more consistent across
BlockStore implementations

closes #177
closes #182
closes #210

alternative to #205, #209
2022-08-22 20:52:51 -05:00
markspanbroek 097a4cfd67
Better command line options for Ethereum (#181)
* [contracts] ContractInteractions.new() now requires account parameter

* [cli] Only perform Ethereum interactions when --eth-account is specified

* [cli] Add --persistence option that is disabled by default

* [cli] Use Option for ethDeployment parameter

* [node] Better error reporting when Ethereum node cannot be reached
2022-08-09 06:29:06 +02:00
Dmitriy Ryajov 625facff4a
Better batching (#170)
* add cleaner batching

* pass block instead of cid

* cleanup manifest fetching

* bug: reference correct cid version

* add batch tests

* make sure batch is always <= batchSize
2022-07-29 14:04:12 -06:00
Dmitriy Ryajov 7ccde112f8
Contract store (#161)
* split out manifest and block retrieval

* adding test for `fetchManifest`

* raise exceptions in onStore
2022-07-28 11:44:59 -06:00
Bulat-Ziganshin 0bfe26440e
WIP: Harmonize BlockStore API (remaining functions) (#123) (#130)
BlockStore API got new return types (rationale in https://github.com/status-im/nim-codex/issues/123#issuecomment-1163797753):
- getBlock: Future[?! (?Block)]
- putBlock/delBlock/listBlocks: Future[?!void]
- hasBlock: Future[?!bool]

Plus refactored readOnce(StoreStream) and check received data in its tests.

And replaced local use of AsyncHeapQueue with seq.sort.
2022-07-28 03:39:17 +03:00
Michael Bradley 209343087c
Change every dagger to codex (#102) 2022-05-19 13:56:03 -06:00