codex-storage
/
nim-codex
mirror of https://github.com/codex-storage/nim-codex.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
nim-codex/codex/stores.nim

17 lines
294 B
Nim
Raw Normal View History

Persist Dht providers (#257) * providers store integration
2022-09-30 02:16:59 +00:00
import ./stores/cachestore
import ./stores/blockstore
import ./stores/networkstore
Repo limits (#319) * initial implementation of repo store * allow isManifest on multicodec * rework with new blockstore * add raw codec * rework listBlocks * remove fsstore * reworking with repostore * bump datastore * fix listBlocks iterator * adding store's common tests * run common store tests * remove fsstore backend tests * bump datastore * add `listBlocks` tests * listBlocks filter based on block type * disabling tests in need of rewriting * allow passing block type * move BlockNotFoundError definition * fix tests * increase default advertise loop sleep to 10 mins * use `self` * add cache quota functionality * pass meta store and start repo * add `CacheQuotaNamespace` * pass meta store * bump datastore to latest master * don't use os `/` as key separator * Added quota limits support * tests for quota limits * add block expiration key * remove unnesesary space * use idleAsync in listBlocks * proper test name * re-add contrlC try/except * add storage quota and block ttl config options * clarify comments * change expires key format * check for block presence before storing * bump datastore * use dht with fixed datastore `has` * bump datastore to latest master * bump dht to latest master
2022-12-03 00:00:55 +00:00
import ./stores/repostore
style
2023-12-14 03:29:13 +00:00
import ./stores/maintenance
Blockstore maintenance (#347) * setting up * Implements timer utility * applies async and cancellation to timer loop * Sets up mocktimer and mockblockstore to set up first maintenance module test * wip: first test that calls blockChecker * wip: adding user type to timer callback * Chronicles doesn't like type-arguments? Disables logging in timer module for now * Implementing block check test for blockMaintainer module * Sets up additional tests for blockmaintainer * Removes generic from timer module. Implements numberOfBlocks per interval in blockMaintainer. * Implements blockMaintainer * Sets up tests for blockChecker * Some comments by Mark * Cleanup repostore tests * Setting up the use of std/times for block TTL tracking * repostore adds expiration timestamp * Defaults the repostore clock to the system clock * Applies updates to repostore interface. * Implements retrieving of block expiration information from repostore * Sets up tests for maintenance module behavior * Implements block maintenance module * Wires maintenance module into codex. Sets up integration tests for block expiration * Sets up test for missing behavior: removing timestamp metadata on block delete * Implements removing of expiration metadata in repoStore * Fixes integration tests for block expiration * Adds block expiration tests to integration test run * Handled some comments by Dmitriy * Review comment by Dmitriy: Removes seq[cid] from runBlockCheck * Review comment by Dmitriy: Moves key formatting methods to keyutils. * Review comment by Dmitriy: Encodes durations using chronos * Fixes conversion of TTL type in conf. * Review comments by Dmitriy * Adds unit tests for keyUtils. * Adds test coverage for exception in maintenance module
2023-03-08 15:04:54 +00:00
import ./stores/keyutils
Blockexchange uses merkle root and index to fetch blocks (#566) * Blockexchange uses merkle root and index to fetch blocks * Links the network store getTree to the local store. * Update codex/stores/repostore.nim Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com> Signed-off-by: Tomasz Bekas <tomasz.bekas@gmail.com> * Rework erasure.nim to include recent cleanup * Revert accidential changes to lib versions * Addressing review comments * Storing proofs instead of trees * Fix a comment * Fix broken tests * Fix for broken testerasure.nim * Addressing PR comments --------- Signed-off-by: Tomasz Bekas <tomasz.bekas@gmail.com> Co-authored-by: benbierens <thatbenbierens@gmail.com> Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2023-11-14 12:02:17 +00:00
import ./stores/treehelper
Rename bitswap (#25) * use PeerInfo in event handlers * use CidV1 and raw multicodec as default * add block stream abstraction * raises defect * adding dataset abstraction * move blockstream into own dir * reorg files and fix tests * rename dataset to blockset * wip * wip * adding basic test for treehash algo * run blockset tests along with with the rest * remove obsolete contents * fix chunker tests * rename bitswap and move to stores * rename bitwsap to blockexc and move to stores * moare project structure reorg
2021-08-30 19:25:20 +00:00
Rework merkle tree (#654) * rework merkle tree support * deps * rename merkletree -> codexmerkletree * treed and proof encoding/decoding * small change to invoke proof verification * rename merkletree to codexmerkletree * style * adding codex merkle and coders tests * fixup imports * remove new codecs for now * bump deps * adding trace statement * properly serde of manifest block codecs * use default hash codec * add more trace logging to aid debugging * misc * remove double import * revert un-needded change * proof size changed * bump poseidon2 * add from nodes test * shorte file names * remove upraises * wip poseidon tree * adjust file names * misc * shorten file names * fix bad `elements` iter * don't do asserts * add fromNodes and converters * root and getProof now return result * add poseidon2 tree tests * root now returns result * misc * had to make merkletree a ref, because nim blows up otherwise * root returns a result * root returns a result * import poseidon tests * bump * merkle poseidon2 digest * misc * add merkle digest tests * bump * don't use checksuite * Update tests/codex/merkletree/generictreetests.nim Co-authored-by: markspanbroek <mark@spanbroek.net> Signed-off-by: Dmitriy Ryajov <dryajov@gmail.com> * Update codex/merkletree/merkletree.nim Co-authored-by: markspanbroek <mark@spanbroek.net> Signed-off-by: Dmitriy Ryajov <dryajov@gmail.com> * Update codex/merkletree/merkletree.nim Co-authored-by: markspanbroek <mark@spanbroek.net> Signed-off-by: Dmitriy Ryajov <dryajov@gmail.com> * Update tests/codex/merkletree/generictreetests.nim Co-authored-by: markspanbroek <mark@spanbroek.net> Signed-off-by: Dmitriy Ryajov <dryajov@gmail.com> * missing return * make toBool private (it's still needed otherwise comparison won't work) * added `digestTree` that returns a tree and `digest` for root * test against both poseidon trees - codex and poseidon2 * shorten merkle tree names * don't compare trees - it's going to be too slow * move comparison to mekrle helper * remove merkle utils --------- Signed-off-by: Dmitriy Ryajov <dryajov@gmail.com> Co-authored-by: markspanbroek <mark@spanbroek.net>
2023-12-21 06:41:43 +00:00
export
cachestore,
blockstore,
networkstore,
repostore,
keyutils,
treehelper,
maintenance