status-im
/
nim-dagger
mirror of https://github.com/status-im/nim-dagger.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
285 Commits 167 Branches 9 Tags 214 MiB
Commit Graph

14 Commits

Author SHA1 Message Date
Michael Bradley, Jr 75666d01bf [node] add SQLiteStore backend 
Closes #138
 2022-08-08 02:12:43 -05:00
Dmitriy Ryajov 625facff4a
Better batching (#170) 
* add cleaner batching

* pass block instead of cid

* cleanup manifest fetching

* bug: reference correct cid version

* add batch tests

* make sure batch is always <= batchSize
 2022-07-29 14:04:12 -06:00
Dmitriy Ryajov 7ccde112f8
Contract store (#161) 
* split out manifest and block retrieval

* adding test for `fetchManifest`

* raise exceptions in onStore
 2022-07-28 11:44:59 -06:00
Bulat-Ziganshin 0bfe26440e
WIP: Harmonize BlockStore API (remaining functions) (#123) (#130) 
BlockStore API got new return types (rationale in https://github.com/status-im/nim-codex/issues/123#issuecomment-1163797753):
- getBlock: Future[?! (?Block)]
- putBlock/delBlock/listBlocks: Future[?!void]
- hasBlock: Future[?!bool]

Plus refactored readOnce(StoreStream) and check received data in its tests.

And replaced local use of AsyncHeapQueue with seq.sort.
 2022-07-28 03:39:17 +03:00
Mark Spanbroek afdb5be2d4 Add TODOs to indicate wanted architecture 
As discussed in team:
- Retrieving of blocks that are mentioned in the
  manifest should be moved into a BlockStore.
- Sales should be given an instance of BlockStore,
  and an instance of StorageProofs, instead of the
  onStore, onClear, and onProve callbacks
 2022-07-20 13:43:20 +02:00
Mark Spanbroek 571aa6f694 [sales] Consistent names for callbacks 2022-07-20 13:43:20 +02:00
Mark Spanbroek 592a3dce4d [sales] Add callback for clearing storage when no longer needed 2022-07-20 13:43:20 +02:00
Mark Spanbroek adcb91a5d5 [node] Store datasets locally when sales asks for it 2022-07-20 13:43:20 +02:00
Mark Spanbroek 7bc5280596 [node] rename PrefetchBatch -> Batch 
Reason: in a next commit we'll use the same
constant to fetch data in batches
 2022-07-20 13:43:20 +02:00
Mark Spanbroek d4025f4a5f [manifest] Extract logic for decoding a manifest given a cid 2022-07-20 13:43:20 +02:00
Mark Spanbroek 4a4588e2a0 [node] Set callbacks for Sales to retrieve and prove posession of data 2022-07-20 13:43:20 +02:00
Mark Spanbroek 94f6838d19 [node] Fix: ensure that retrieve works for small files 2022-07-20 13:43:20 +02:00
Dmitriy Ryajov 6ad7a6bb96
Batched blocks (#105) 
* remove sleep

* throttle prefetch

* break loop if conn closed

* logging

* prefetch blocks in batches

* add blocks batch constant
 2022-05-20 10:53:34 -06:00
Michael Bradley 209343087c
Change every dagger to codex (#102) 2022-05-19 13:56:03 -06:00