nim-codex/codex
Bulat-Ziganshin f24ded0f76
Download files without padding (#218)
The initial goal of this patch was to allow to download of a file via REST API in exactly the same size as it was uploaded, which required adding fields Chunker.offset and Manifest.originalBytes to keep that size. On top of that, we added more integrity checks to operations on Manifest, and reorganized TestNode.nim to test the actual interaction between node.store and node.retrieve operations.

Note that the wire format of Manifest was changed, so we need to recreate all BlockStores.

* Download without padding
* Fixed chunker tests
* Chunker: get rid of RabinChunker
* Verify offset in the chunker tests
* Use manifest.originalBytesPadded in StoreStream.size
* StoreStream: replace emptyBlock with zeroMem
* Manifest.bytes: compute how many bytes corresponding StoreStream(Manifest, pad) will return
* Manifest: verify originalBytes and originalLen on new/encode/decode
Also set originalBytes in each Manifest creation/update scenario
* Manifest: comments, split code into sections
* Reordered parameters to deal with int64 size in 32-bit builds
* TestNode.nim: combine Store and Retrieve tests
1. Instead of copy-pasting code from node.nim, new test calls node.store() and node.retrieve() in order to check that they can correctly store and then retrieve data
2. New test compares only file contents, manifest contents considered an implementation detail
3. New test chunks at odd chunkSize=BlockSize/1.618 in order to ensure that data retrieved correctly even when buffer sizes mismatch
* TestNode.nim: code refactoring
* Manifest.add: one more test
* Manifest.verify: return Result instead of raising Defect
* Node.store: added blockSize parameter
2022-08-24 15:15:59 +03:00
..
blockexchange Adding metrics (#203) 2022-08-23 10:11:21 -06:00
contracts Better command line options for Ethereum (#181) 2022-08-09 06:29:06 +02:00
erasure [stores] update getBlock return type 2022-08-22 20:52:51 -05:00
manifest Download files without padding (#218) 2022-08-24 15:15:59 +03:00
rest [rest api] Allow user to specify amount of hosts 2022-08-08 14:00:55 +02:00
storageproofs Upload authenticators (#108) 2022-05-25 20:29:31 -06:00
stores [stores] update getBlock return type 2022-08-22 20:52:51 -05:00
streams Download files without padding (#218) 2022-08-24 15:15:59 +03:00
utils Change every dagger to codex (#102) 2022-05-19 13:56:03 -06:00
blockexchange.nim Change every dagger to codex (#102) 2022-05-19 13:56:03 -06:00
blocktype.nim [stores] update getBlock return type 2022-08-22 20:52:51 -05:00
chunker.nim Download files without padding (#218) 2022-08-24 15:15:59 +03:00
clock.nim [purchasing] Update to latest dagger-contracts 2022-07-20 13:43:20 +02:00
codex.nim [build] update nim-datastore 2022-08-10 13:13:07 -05:00
conf.nim Better command line options for Ethereum (#181) 2022-08-09 06:29:06 +02:00
contracts.nim Cleanup: remove StorageOffer everywhere 2022-07-20 13:43:20 +02:00
discovery.nim Upload authenticators (#108) 2022-05-25 20:29:31 -06:00
erasure.nim Change every dagger to codex (#102) 2022-05-19 13:56:03 -06:00
errors.nim Change every dagger to codex (#102) 2022-05-19 13:56:03 -06:00
manifest.nim Change every dagger to codex (#102) 2022-05-19 13:56:03 -06:00
market.nim [market] remove fulfillRequest() 2022-08-08 14:00:55 +02:00
node.nim Download files without padding (#218) 2022-08-24 15:15:59 +03:00
proving.nim Por serialize (#106) 2022-05-23 23:24:15 -06:00
purchasing.nim [purchasing] Remove Purchase.selected 2022-08-08 14:00:55 +02:00
rng.nim Change every dagger to codex (#102) 2022-05-19 13:56:03 -06:00
sales.nim [sales] Pass along request and slot index in onProve callback 2022-08-08 14:00:55 +02:00
storageproofs.nim Upload authenticators (#108) 2022-05-25 20:29:31 -06:00
stores.nim [node] add SQLiteStore backend 2022-08-08 02:12:43 -05:00
streams.nim adding AsyncStreamWrapper (#110) 2022-06-14 09:19:35 -06:00
utils.nim Download files without padding (#218) 2022-08-24 15:15:59 +03:00