status-im
/
nim-codex
mirror of https://github.com/status-im/nim-codex.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nim-codex/codex/stores/keyutils.nim

45 lines
1.4 KiB
Nim
Raw Normal View History

Blockstore maintenance (#347) * setting up * Implements timer utility * applies async and cancellation to timer loop * Sets up mocktimer and mockblockstore to set up first maintenance module test * wip: first test that calls blockChecker * wip: adding user type to timer callback * Chronicles doesn't like type-arguments? Disables logging in timer module for now * Implementing block check test for blockMaintainer module * Sets up additional tests for blockmaintainer * Removes generic from timer module. Implements numberOfBlocks per interval in blockMaintainer. * Implements blockMaintainer * Sets up tests for blockChecker * Some comments by Mark * Cleanup repostore tests * Setting up the use of std/times for block TTL tracking * repostore adds expiration timestamp * Defaults the repostore clock to the system clock * Applies updates to repostore interface. * Implements retrieving of block expiration information from repostore * Sets up tests for maintenance module behavior * Implements block maintenance module * Wires maintenance module into codex. Sets up integration tests for block expiration * Sets up test for missing behavior: removing timestamp metadata on block delete * Implements removing of expiration metadata in repoStore * Fixes integration tests for block expiration * Adds block expiration tests to integration test run * Handled some comments by Dmitriy * Review comment by Dmitriy: Removes seq[cid] from runBlockCheck * Review comment by Dmitriy: Moves key formatting methods to keyutils. * Review comment by Dmitriy: Encodes durations using chronos * Fixes conversion of TTL type in conf. * Review comments by Dmitriy * Adds unit tests for keyUtils. * Adds test coverage for exception in maintenance module
2023-03-08 15:04:54 +00:00
## Nim-Codex
## Copyright (c) 2022 Status Research & Development GmbH
## Licensed under either of
## * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
## * MIT license ([LICENSE-MIT](LICENSE-MIT))
## at your option.
## This file may not be copied, modified, or distributed except according to
## those terms.
import pkg/upraises
push: {.upraises: [].}
import pkg/questionable/results
import pkg/datastore
import pkg/libp2p
import ../namespaces
import ../manifest
const
CodexMetaKey* = Key.init(CodexMetaNamespace).tryGet
CodexRepoKey* = Key.init(CodexRepoNamespace).tryGet
CodexBlocksKey* = Key.init(CodexBlocksNamespace).tryGet
Add metrics (#478) * Adds metrics to block exchange. * Adds metrics to purchasing * Adds metrics to upload and download API * Adds metrics to the repo store * Fixes exception in repostore start. * Merge managed to mess up indentation.
2023-07-20 07:56:28 +00:00
CodexTotalBlocksKey* = Key.init(CodexBlockTotalNamespace).tryGet
Blockstore maintenance (#347) * setting up * Implements timer utility * applies async and cancellation to timer loop * Sets up mocktimer and mockblockstore to set up first maintenance module test * wip: first test that calls blockChecker * wip: adding user type to timer callback * Chronicles doesn't like type-arguments? Disables logging in timer module for now * Implementing block check test for blockMaintainer module * Sets up additional tests for blockmaintainer * Removes generic from timer module. Implements numberOfBlocks per interval in blockMaintainer. * Implements blockMaintainer * Sets up tests for blockChecker * Some comments by Mark * Cleanup repostore tests * Setting up the use of std/times for block TTL tracking * repostore adds expiration timestamp * Defaults the repostore clock to the system clock * Applies updates to repostore interface. * Implements retrieving of block expiration information from repostore * Sets up tests for maintenance module behavior * Implements block maintenance module * Wires maintenance module into codex. Sets up integration tests for block expiration * Sets up test for missing behavior: removing timestamp metadata on block delete * Implements removing of expiration metadata in repoStore * Fixes integration tests for block expiration * Adds block expiration tests to integration test run * Handled some comments by Dmitriy * Review comment by Dmitriy: Removes seq[cid] from runBlockCheck * Review comment by Dmitriy: Moves key formatting methods to keyutils. * Review comment by Dmitriy: Encodes durations using chronos * Fixes conversion of TTL type in conf. * Review comments by Dmitriy * Adds unit tests for keyUtils. * Adds test coverage for exception in maintenance module
2023-03-08 15:04:54 +00:00
CodexManifestKey* = Key.init(CodexManifestNamespace).tryGet
BlocksTtlKey* = Key.init(CodexBlocksTtlNamespace).tryGet
QuotaKey* = Key.init(CodexQuotaNamespace).tryGet
QuotaUsedKey* = (QuotaKey / "used").tryGet
QuotaReservedKey* = (QuotaKey / "reserved").tryGet
func makePrefixKey*(postFixLen: int, cid: Cid): ?!Key =
let
cidKey = ? Key.init(($cid)[^postFixLen..^1] & "/" & $cid)
if ? cid.isManifest:
success CodexManifestKey / cidKey
else:
success CodexBlocksKey / cidKey
proc createBlockExpirationMetadataKey*(cid: Cid): ?!Key =
BlocksTtlKey / $cid
proc createBlockExpirationMetadataQueryKey*(): ?!Key =
let queryString = ? (BlocksTtlKey / "*")
Key.init(queryString)