logos-co/logos-lips
1
0
Fork 0
mirror of https://github.com/logos-co/logos-lips.git synced 2026-08-31 10:11:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
deploy-develop
logos-lips/storage/raw/codex-store.html
T
status-im-auto 013adf3bd4 Update documentation
2026-07-28 02:38:22 +00:00

776 lines
56 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE HTML>
<html lang="en" class="ayu" dir="ltr">
<head>
<!-- Book generated using mdBook -->
<meta charset="UTF-8">
<title>CODEX-STORE - Logos LIP</title>
<!-- Custom HTML head -->
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="theme-color" content="#ffffff">
<link rel="icon" href="../../favicon.svg">
<link rel="shortcut icon" href="../../favicon.png">
<link rel="stylesheet" href="../../css/variables.css">
<link rel="stylesheet" href="../../css/general.css">
<link rel="stylesheet" href="../../css/chrome.css">
<link rel="stylesheet" href="../../css/print.css" media="print">
<!-- Fonts -->
<link rel="stylesheet" href="../../FontAwesome/css/font-awesome.css">
<link rel="stylesheet" href="../../fonts/fonts.css">
<!-- Highlight.js Stylesheets -->
<link rel="stylesheet" href="../../highlight.css">
<link rel="stylesheet" href="../../tomorrow-night.css">
<link rel="stylesheet" href="../../ayu-highlight.css">
<!-- Custom theme stylesheets -->
<link rel="stylesheet" href="../../theme/katex/katex.min.css">
<link rel="stylesheet" href="../../theme/fonts/fonts.css">
<link rel="stylesheet" href="../../theme/FontAwesome/css/font-awesome.css">
<link rel="stylesheet" href="../../custom.css">
</head>
<body class="sidebar-visible no-js">
<div id="body-container">
<!-- Provide site root to javascript -->
<script>
var path_to_root = "../../";
var default_theme = window.matchMedia("(prefers-color-scheme: dark)").matches ? "navy" : "ayu";
</script>
<!-- Work around some values being stored in localStorage wrapped in quotes -->
<script>
try {
var theme = localStorage.getItem('mdbook-theme');
var sidebar = localStorage.getItem('mdbook-sidebar');
if (theme.startsWith('"') && theme.endsWith('"')) {
localStorage.setItem('mdbook-theme', theme.slice(1, theme.length - 1));
}
if (sidebar.startsWith('"') && sidebar.endsWith('"')) {
localStorage.setItem('mdbook-sidebar', sidebar.slice(1, sidebar.length - 1));
}
} catch (e) { }
</script>
<!-- Set the theme before any content is loaded, prevents flash -->
<script>
var theme;
try { theme = localStorage.getItem('mdbook-theme'); } catch(e) { }
if (theme === null || theme === undefined) { theme = default_theme; }
var html = document.querySelector('html');
html.classList.remove('ayu')
html.classList.add(theme);
var body = document.querySelector('body');
body.classList.remove('no-js')
body.classList.add('js');
</script>
<input type="checkbox" id="sidebar-toggle-anchor" class="hidden">
<!-- Hide / unhide sidebar before it is displayed -->
<script>
var body = document.querySelector('body');
var sidebar = null;
var sidebar_toggle = document.getElementById("sidebar-toggle-anchor");
if (document.body.clientWidth >= 1080) {
try { sidebar = localStorage.getItem('mdbook-sidebar'); } catch(e) { }
sidebar = sidebar || 'visible';
} else {
sidebar = 'hidden';
}
sidebar_toggle.checked = sidebar === 'visible';
body.classList.remove('sidebar-visible');
body.classList.add("sidebar-" + sidebar);
</script>
<nav id="sidebar" class="sidebar" aria-label="Table of contents">
<div class="sidebar-scrollbox">
<ol class="chapter"><li class="chapter-item expanded affix "><a href="../../index.html">Introduction</a></li><li class="chapter-item expanded affix "><a href="../../about.html">About</a></li><li class="chapter-item expanded "><a href="../../messaging/index.html"><strong aria-hidden="true">1.</strong> Messaging</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../messaging/standards/core/index.html"><strong aria-hidden="true">1.1.</strong> Standards - Core</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../messaging/standards/core/10/waku2.html"><strong aria-hidden="true">1.1.1.</strong> 10/WAKU2</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/core/11/relay.html"><strong aria-hidden="true">1.1.2.</strong> 11/WAKU2-RELAY</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/core/12/filter.html"><strong aria-hidden="true">1.1.3.</strong> 12/WAKU2-FILTER</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../messaging/standards/core/12/previous-versions/00/filter.html"><strong aria-hidden="true">1.1.3.1.</strong> 00 (previous)</a></li></ol></li><li class="chapter-item expanded "><a href="../../messaging/standards/core/13/store.html"><strong aria-hidden="true">1.1.4.</strong> 13/WAKU2-STORE</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../messaging/standards/core/13/previous-versions/00/store.html"><strong aria-hidden="true">1.1.4.1.</strong> 00 (previous)</a></li></ol></li><li class="chapter-item expanded "><a href="../../messaging/standards/core/14/message.html"><strong aria-hidden="true">1.1.5.</strong> 14/WAKU2-MESSAGE</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/core/15/bridge.html"><strong aria-hidden="true">1.1.6.</strong> 15/WAKU-BRIDGE</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/core/17/rln-relay.html"><strong aria-hidden="true">1.1.7.</strong> 17/WAKU2-RLN-RELAY</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/core/19/lightpush.html"><strong aria-hidden="true">1.1.8.</strong> 19/WAKU2-LIGHTPUSH</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/core/31/enr.html"><strong aria-hidden="true">1.1.9.</strong> 31/WAKU2-ENR</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/core/33/discv5.html"><strong aria-hidden="true">1.1.10.</strong> 33/WAKU2-DISCV5</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/core/34/peer-exchange.html"><strong aria-hidden="true">1.1.11.</strong> 34/WAKU2-PEER-EXCHANGE</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/core/36/bindings-api.html"><strong aria-hidden="true">1.1.12.</strong> 36/WAKU2-BINDINGS-API</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/core/64/network.html"><strong aria-hidden="true">1.1.13.</strong> 64/WAKU2-NETWORK</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/core/66/metadata.html"><strong aria-hidden="true">1.1.14.</strong> 66/WAKU2-METADATA</a></li></ol></li><li class="chapter-item expanded "><a href="../../messaging/standards/application/index.html"><strong aria-hidden="true">1.2.</strong> Standards - Application</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../messaging/standards/application/20/toy-eth-pm.html"><strong aria-hidden="true">1.2.1.</strong> 20/TOY-ETH-PM</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/application/26/payload.html"><strong aria-hidden="true">1.2.2.</strong> 26/WAKU2-PAYLOAD</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/application/53/x3dh.html"><strong aria-hidden="true">1.2.3.</strong> 53/WAKU2-X3DH</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/application/54/x3dh-sessions.html"><strong aria-hidden="true">1.2.4.</strong> 54/WAKU2-X3DH-SESSIONS</a></li></ol></li><li class="chapter-item expanded "><a href="../../messaging/standards/legacy/index.html"><strong aria-hidden="true">1.3.</strong> Standards - Legacy</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../messaging/standards/legacy/6/waku1.html"><strong aria-hidden="true">1.3.1.</strong> 6/WAKU1</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/legacy/7/data.html"><strong aria-hidden="true">1.3.2.</strong> 7/WAKU-DATA</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/legacy/8/mail.html"><strong aria-hidden="true">1.3.3.</strong> 8/WAKU-MAIL</a></li><li class="chapter-item expanded "><a href="../../messaging/standards/legacy/9/rpc.html"><strong aria-hidden="true">1.3.4.</strong> 9/WAKU-RPC</a></li></ol></li><li class="chapter-item expanded "><a href="../../messaging/informational/index.html"><strong aria-hidden="true">1.4.</strong> Informational</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../messaging/informational/22/toy-chat.html"><strong aria-hidden="true">1.4.1.</strong> 22/TOY-CHAT</a></li><li class="chapter-item expanded "><a href="../../messaging/informational/23/topics.html"><strong aria-hidden="true">1.4.2.</strong> 23/WAKU2-TOPICS</a></li><li class="chapter-item expanded "><a href="../../messaging/informational/27/peers.html"><strong aria-hidden="true">1.4.3.</strong> 27/WAKU2-PEERS</a></li><li class="chapter-item expanded "><a href="../../messaging/informational/29/config.html"><strong aria-hidden="true">1.4.4.</strong> 29/WAKU2-CONFIG</a></li><li class="chapter-item expanded "><a href="../../messaging/informational/30/adaptive-nodes.html"><strong aria-hidden="true">1.4.5.</strong> 30/ADAPTIVE-NODES</a></li></ol></li><li class="chapter-item expanded "><a href="../../messaging/deprecated/index.html"><strong aria-hidden="true">1.5.</strong> Deprecated</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../messaging/deprecated/5/waku0.html"><strong aria-hidden="true">1.5.1.</strong> 5/WAKU0</a></li><li class="chapter-item expanded "><a href="../../messaging/deprecated/16/rpc.html"><strong aria-hidden="true">1.5.2.</strong> 16/WAKU2-RPC</a></li><li class="chapter-item expanded "><a href="../../messaging/deprecated/18/swap.html"><strong aria-hidden="true">1.5.3.</strong> 18/WAKU2-SWAP</a></li><li class="chapter-item expanded "><a href="../../messaging/deprecated/fault-tolerant-store.html"><strong aria-hidden="true">1.5.4.</strong> 21/WAKU2-FAULT-TOLERANT-STORE</a></li></ol></li></ol></li><li class="chapter-item expanded "><a href="../../blockchain/index.html"><strong aria-hidden="true">2.</strong> Blockchain</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../blockchain/raw/index.html"><strong aria-hidden="true">2.1.</strong> Raw</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../blockchain/raw/bedrock-anonymous-leaders-reward.html"><strong aria-hidden="true">2.1.1.</strong> BEDROCK-ANONYMOUS-LEADERS-REWARD</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/bedrock-architecture-overview.html"><strong aria-hidden="true">2.1.2.</strong> BEDROCK-ARCHITECTURE-OVERVIEW</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/bedrock-genesis-block.html"><strong aria-hidden="true">2.1.3.</strong> BEDROCK-GENESIS-BLOCK</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/bedrock-service-declaration-protocol.html"><strong aria-hidden="true">2.1.4.</strong> BEDROCK-SERVICE-DECLARATION-PROTOCOL</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/bedrock-service-reward-distribution.html"><strong aria-hidden="true">2.1.5.</strong> BEDROCK-SERVICE-REWARD-DISTRIBUTION</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/bedrock-v1.1-block-construction.html"><strong aria-hidden="true">2.1.6.</strong> BEDROCK-V1-1-BLOCK-CONSTRUCTION</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/bedrock-v1.1-mantle-specification.html"><strong aria-hidden="true">2.1.7.</strong> BEDROCK-V1.1-MANTLE-SPECIFICATION</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/fork-choice.html"><strong aria-hidden="true">2.1.8.</strong> CRYPTARCHIA-FORK-CHOICE</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/cryptarchia-proof-of-leadership.html"><strong aria-hidden="true">2.1.9.</strong> CRYPTARCHIA-PROOF-OF-LEADERSHIP</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/cryptarchia-total-stake-inference.html"><strong aria-hidden="true">2.1.10.</strong> CRYPTARCHIA-TOTAL-STAKE-INFERENCE</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/cryptarchia-v1-bootstr-sync.html"><strong aria-hidden="true">2.1.11.</strong> CRYPTARCHIA-V1-BOOTSTRAPPING-SYNCHRONIZATION</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/nomos-blend-protocol.html"><strong aria-hidden="true">2.1.12.</strong> NOMOS-BLEND-PROTOCOL</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/nomos-cryptarchia-v1-protocol.html"><strong aria-hidden="true">2.1.13.</strong> NOMOS-CRYPTARCHIA-V1-PROTOCOL</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/nomosda-network.html"><strong aria-hidden="true">2.1.14.</strong> NOMOS-DA-NETWORK</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/digital-signature.html"><strong aria-hidden="true">2.1.15.</strong> NOMOS-DIGITAL-SIGNATURE</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/nomos-key-types-and-generation.html"><strong aria-hidden="true">2.1.16.</strong> NOMOS-KEY-TYPES-GENERATION</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/nomos-message-encapsulation.html"><strong aria-hidden="true">2.1.17.</strong> NOMOS-MESSAGE-ENCAPSULATION</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/nomos-message-formatting.html"><strong aria-hidden="true">2.1.18.</strong> NOMOS-MESSAGE-FORMATTING</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/p2p-network.html"><strong aria-hidden="true">2.1.19.</strong> NOMOS-P2P-NETWORK</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/nomos-payload-formatting.html"><strong aria-hidden="true">2.1.20.</strong> NOMOS-PAYLOAD-FORMATTING</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/nomos-proof-of-quota.html"><strong aria-hidden="true">2.1.21.</strong> NOMOS-PROOF-OF-QUOTA</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/nomos-wallet-technical-standard.html"><strong aria-hidden="true">2.1.22.</strong> NOMOS-WALLET-TECHNICAL-STANDARD</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/da-cryptographic-protocol.html"><strong aria-hidden="true">2.1.23.</strong> NOMOSDA-CRYPTOGRAPHIC-PROTOCOL</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/da-rewarding.html"><strong aria-hidden="true">2.1.24.</strong> NOMOSDA-REWARDING</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/p2p-hardware-requirements.html"><strong aria-hidden="true">2.1.25.</strong> P2P-HARDWARE-REQUIREMENTS</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/p2p-nat-solution.html"><strong aria-hidden="true">2.1.26.</strong> P2P-NAT-SOLUTION</a></li><li class="chapter-item expanded "><a href="../../blockchain/raw/p2p-network-bootstrapping.html"><strong aria-hidden="true">2.1.27.</strong> P2P-NETWORK-BOOTSTRAPPING</a></li></ol></li><li class="chapter-item expanded "><a href="../../blockchain/deprecated/index.html"><strong aria-hidden="true">2.2.</strong> Deprecated</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../blockchain/deprecated/claro.html"><strong aria-hidden="true">2.2.1.</strong> CONSENSUS-CLARO</a></li></ol></li></ol></li><li class="chapter-item expanded "><a href="../../storage/index.html"><strong aria-hidden="true">3.</strong> Storage</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../storage/raw/index.html"><strong aria-hidden="true">3.1.</strong> Raw</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../storage/raw/codex-block-exchange.html"><strong aria-hidden="true">3.1.1.</strong> CODEX-BLOCK-EXCHANGE</a></li><li class="chapter-item expanded "><a href="../../storage/raw/community-history.html"><strong aria-hidden="true">3.1.2.</strong> CODEX-COMMUNITY-HISTORY</a></li><li class="chapter-item expanded "><a href="../../storage/raw/dht.html"><strong aria-hidden="true">3.1.3.</strong> CODEX-DHT</a></li><li class="chapter-item expanded "><a href="../../storage/raw/manifest.html"><strong aria-hidden="true">3.1.4.</strong> CODEX-MANIFEST</a></li><li class="chapter-item expanded "><a href="../../storage/raw/codex-store.html" class="active"><strong aria-hidden="true">3.1.5.</strong> CODEX-STORE</a></li><li class="chapter-item expanded "><a href="../../storage/raw/dataset-store.html"><strong aria-hidden="true">3.1.6.</strong> DATASET-STORE</a></li><li class="chapter-item expanded "><a href="../../storage/raw/datasets.html"><strong aria-hidden="true">3.1.7.</strong> DATASETS</a></li><li class="chapter-item expanded "><a href="../../storage/raw/merkle-tree.html"><strong aria-hidden="true">3.1.8.</strong> MERKLE-TREE</a></li></ol></li><li class="chapter-item expanded "><a href="../../storage/deprecated/index.html"><strong aria-hidden="true">3.2.</strong> Deprecated</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../storage/deprecated/erasure-coding.html"><strong aria-hidden="true">3.2.1.</strong> CODEX-ERASUE-CODING</a></li><li class="chapter-item expanded "><a href="../../storage/deprecated/codex-marketplace.html"><strong aria-hidden="true">3.2.2.</strong> CODEX-MARKETPLACE</a></li><li class="chapter-item expanded "><a href="../../storage/deprecated/codex-merkle-tree.html"><strong aria-hidden="true">3.2.3.</strong> CODEX-MERKLE-TREE</a></li><li class="chapter-item expanded "><a href="../../storage/deprecated/codex-prover.html"><strong aria-hidden="true">3.2.4.</strong> CODEX-PROVER</a></li><li class="chapter-item expanded "><a href="../../storage/deprecated/slot-buIlder.html"><strong aria-hidden="true">3.2.5.</strong> CODEX-SLOT-BUILDER</a></li></ol></li></ol></li><li class="chapter-item expanded "><a href="../../ift-ts/index.html"><strong aria-hidden="true">4.</strong> IFT-TS</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../ift-ts/raw/index.html"><strong aria-hidden="true">4.1.</strong> Raw</a></li><li><ol class="section"><li class="chapter-item expanded "><a href="../../ift-ts/raw/1/coss.html"><strong aria-hidden="true">4.1.1.</strong> 1/COSS</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/2/mvds.html"><strong aria-hidden="true">4.1.2.</strong> 2/MVDS</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/3/remote-log.html"><strong aria-hidden="true">4.1.3.</strong> 3/REMOTE-LOG</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/4/mvds-meta.html"><strong aria-hidden="true">4.1.4.</strong> 4/MVDS-META</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/25/libp2p-dns-discovery.html"><strong aria-hidden="true">4.1.5.</strong> 25/LIBP2P-DNS-DISCOVERY</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/32/rln-v1.html"><strong aria-hidden="true">4.1.6.</strong> 32/RLN-V1</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/decentralized-mls-offchain-consensus.html"><strong aria-hidden="true">4.1.7.</strong> DECENTRALIZED-MLS-OFFCHAIN-CONSENSUS</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/decentralized-messaging-ethereum.html"><strong aria-hidden="true">4.1.8.</strong> ETH-DCGKA</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/eth-mls-onchain.html"><strong aria-hidden="true">4.1.9.</strong> ETH-MLS-ONCHAIN</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/deleted/eth-secpm.html"><strong aria-hidden="true">4.1.10.</strong> ETH-SECPM</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/extended-kad-disco.html"><strong aria-hidden="true">4.1.11.</strong> EXTENDED-KADEMLIA-DISCOVERY</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/extensible-peer-records.html"><strong aria-hidden="true">4.1.12.</strong> EXTENSIBLE-PEER-RECORDS</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/gossipsub-tor-push.html"><strong aria-hidden="true">4.1.13.</strong> GOSSIPSUB-TOR-PUSH</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/consensus-hashgraphlike.html"><strong aria-hidden="true">4.1.14.</strong> HASHGRAPHLIKE CONSENSUS</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/logos-service-discovery.html"><strong aria-hidden="true">4.1.15.</strong> LOGOS-SERVICE-DISCOVERY</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/mix.html"><strong aria-hidden="true">4.1.16.</strong> MIX</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/multi-message_id-burn-rln.html"><strong aria-hidden="true">4.1.17.</strong> Multi-message_id Burn RLN</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/noise-x3dh-double-ratchet.html"><strong aria-hidden="true">4.1.18.</strong> NOISE-X3DH-DOUBLE-RATCHET</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/payment-streams.html"><strong aria-hidden="true">4.1.19.</strong> PAYMENT-STREAMS</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/mix-spam-protection-rln.html"><strong aria-hidden="true">4.1.20.</strong> RLN DoS Protection for Mixnet</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/rln-interep-spec.html"><strong aria-hidden="true">4.1.21.</strong> RLN-INTEREP-SPEC</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/rln-stealth-commitments.html"><strong aria-hidden="true">4.1.22.</strong> RLN-STEALTH-COMMITMENTS</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/rln-v2.html"><strong aria-hidden="true">4.1.23.</strong> RLN-V2</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/sds.html"><strong aria-hidden="true">4.1.24.</strong> SDS</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/status-rln-deployment.html"><strong aria-hidden="true">4.1.25.</strong> STATUS-RLN-DEPLOYMENT</a></li><li class="chapter-item expanded "><a href="../../ift-ts/raw/zerokit-api.html"><strong aria-hidden="true">4.1.26.</strong> Zerokit API</a></li></ol></li><li class="chapter-item expanded "><a href="../../ift-ts/template.html"><strong aria-hidden="true">4.2.</strong> TEMPLATE</a></li></ol></li></ol>
</div>
<div id="sidebar-resize-handle" class="sidebar-resize-handle">
<div class="sidebar-resize-indicator"></div>
</div>
</nav>
<!-- Track and set sidebar scroll position -->
<script>
var sidebarScrollbox = document.querySelector('#sidebar .sidebar-scrollbox');
sidebarScrollbox.addEventListener('click', function(e) {
if (e.target.tagName === 'A') {
sessionStorage.setItem('sidebar-scroll', sidebarScrollbox.scrollTop);
}
}, { passive: true });
var sidebarScrollTop = sessionStorage.getItem('sidebar-scroll');
sessionStorage.removeItem('sidebar-scroll');
if (sidebarScrollTop) {
// preserve sidebar scroll position when navigating via links within sidebar
sidebarScrollbox.scrollTop = sidebarScrollTop;
} else {
// scroll sidebar to current active section when navigating via "next/previous chapter" buttons
var activeSection = document.querySelector('#sidebar .active');
if (activeSection) {
activeSection.scrollIntoView({ block: 'center' });
}
}
</script>
<div id="page-wrapper" class="page-wrapper">
<div class="page">
<div id="menu-bar-hover-placeholder"></div>
<div id="menu-bar" class="menu-bar sticky">
<div class="left-buttons">
<label id="sidebar-toggle" class="icon-button" for="sidebar-toggle-anchor" title="Toggle Table of Contents" aria-label="Toggle Table of Contents" aria-controls="sidebar">
<i class="fa fa-bars"></i>
</label>
<button id="theme-toggle" class="icon-button" type="button" title="Change theme" aria-label="Change theme" aria-haspopup="true" aria-expanded="false" aria-controls="theme-list">
<i class="fa fa-paint-brush"></i>
</button>
<ul id="theme-list" class="theme-popup" aria-label="Themes" role="menu">
<li role="none"><button role="menuitem" class="theme" id="light">Light</button></li>
<li role="none"><button role="menuitem" class="theme" id="rust">Rust</button></li>
<li role="none"><button role="menuitem" class="theme" id="coal">Coal</button></li>
<li role="none"><button role="menuitem" class="theme" id="navy">Navy</button></li>
<li role="none"><button role="menuitem" class="theme" id="ayu">Ayu</button></li>
</ul>
<button id="search-toggle" class="icon-button" type="button" title="Search. (Shortkey: s)" aria-label="Toggle Searchbar" aria-expanded="false" aria-keyshortcuts="S" aria-controls="searchbar">
<i class="fa fa-search"></i>
</button>
</div>
<h1 class="menu-title">Logos LIP</h1>
<div class="right-buttons">
<a href="../../print.html" title="Print this book" aria-label="Print this book">
<i id="print-button" class="fa fa-print"></i>
</a>
<a href="https://github.com/logos-co/logos-lips" title="Git repository" aria-label="Git repository">
<i id="git-repository-button" class="fa fa-github"></i>
</a>
</div>
</div>
<div id="search-wrapper" class="hidden">
<form id="searchbar-outer" class="searchbar-outer">
<input type="search" id="searchbar" name="searchbar" placeholder="Search this book ..." aria-controls="searchresults-outer" aria-describedby="searchresults-header">
</form>
<div id="searchresults-outer" class="searchresults-outer hidden">
<div id="searchresults-header" class="searchresults-header"></div>
<ul id="searchresults">
</ul>
</div>
</div>
<!-- Apply ARIA attributes after the sidebar and the sidebar toggle button are added to the DOM -->
<script>
document.getElementById('sidebar-toggle').setAttribute('aria-expanded', sidebar === 'visible');
document.getElementById('sidebar').setAttribute('aria-hidden', sidebar !== 'visible');
Array.from(document.querySelectorAll('#sidebar a')).forEach(function(link) {
link.setAttribute('tabIndex', sidebar === 'visible' ? 0 : -1);
});
</script>
<div id="content" class="content">
<main>
<h1 id="codex-store"><a class="header" href="#codex-store">CODEX-STORE</a></h1>
<div class="table-wrapper"><table><thead><tr><th>Field</th><th>Value</th></tr></thead><tbody>
<tr><td>Name</td><td>Codex Store Module</td></tr>
<tr><td>Slug</td><td>80</td></tr>
<tr><td>Status</td><td>raw</td></tr>
<tr><td>Category</td><td>Standards Track</td></tr>
<tr><td>Editor</td><td>Codex Team</td></tr>
<tr><td>Contributors</td><td>Filip Dimitrijevic <a href="mailto:filip@status.im">filip@status.im</a></td></tr>
</tbody></table>
</div><!-- timeline:start -->
<h2 id="timeline"><a class="header" href="#timeline">Timeline</a></h2>
<ul>
<li><strong>2026-01-19</strong> — <a href="https://github.com/logos-co/logos-lips/blob/f24e567d0b1e10c178bfa0c133495fe83b969b76/docs/storage/raw/codex-store.md"><code>f24e567</code></a> — Chore/updates mdbook (#262)</li>
<li><strong>2026-01-16</strong> — <a href="https://github.com/logos-co/logos-lips/blob/89f2ea89fc1d69ab238b63c7e6fb9e4203fd8529/docs/storage/raw/codex-store.md"><code>89f2ea8</code></a> — Chore/mdbook updates (#258)</li>
</ul>
<!-- timeline:end -->
<h2 id="abstract"><a class="header" href="#abstract">Abstract</a></h2>
<p>This specification describes the Store Module,
the core storage abstraction in <a href="https://github.com/codex-storage/nim-codex">Codex</a>,
providing a unified interface for storing and retrieving content-addressed blocks
and associated metadata.</p>
<p>The Store Module decouples storage operations from underlying datastore semantics
by introducing the <code>BlockStore</code> interface,
which standardizes methods for storing and retrieving both ephemeral
and persistent blocks across different storage backends.
The module integrates a maintenance engine responsible for cleaning up
expired ephemeral data according to configured policies.</p>
<p>The Store Module is built on top of the generic
<a href="https://github.com/codex-storage/nim-datastore/blob/master/datastore/datastore.nim">DataStore (DS) interface</a>,
which is implemented by multiple backends such as SQLite, LevelDB,
and the filesystem.</p>
<h2 id="background--rationale--motivation"><a class="header" href="#background--rationale--motivation">Background / Rationale / Motivation</a></h2>
<p>The primary design goal is to decouple storage operations from the underlying
datastore semantics by introducing the <code>BlockStore</code> interface.
This interface standardizes methods for storing and retrieving both ephemeral
and persistent blocks,
ensuring a consistent API across different storage backends.</p>
<p>The DataStore provides a KV-store abstraction with <code>Get</code>, <code>Put</code>, <code>Delete</code>,
and <code>Query</code> operations, with backend-dependent guarantees.
At a minimum, row-level consistency and basic batching are expected.</p>
<p>The DataStore supports:</p>
<ul>
<li>Namespace mounting for isolating backend usage</li>
<li>Layering backends (e.g., caching in front of persistent stores)</li>
<li>Flexible stacking and composition of storage proxies</li>
</ul>
<p>The current implementation has several limitations:</p>
<ul>
<li>No dataset-level operations or advanced batching support</li>
<li>Lack of consistent locking and concurrency control,
which may lead to inconsistencies during crashes or long-running operations
on block groups (e.g., reference count updates, expiration updates)</li>
</ul>
<h2 id="theory--semantics"><a class="header" href="#theory--semantics">Theory / Semantics</a></h2>
<h3 id="blockstore-interface"><a class="header" href="#blockstore-interface">BlockStore Interface</a></h3>
<p>The <code>BlockStore</code> interface provides the following methods:</p>
<div class="table-wrapper"><table><thead><tr><th>Method</th><th>Description</th><th>Input</th><th>Output</th></tr></thead><tbody>
<tr><td><code>getBlock(cid: Cid)</code></td><td>Retrieve block by CID</td><td>CID</td><td><code>Future[?!Block]</code></td></tr>
<tr><td><code>getBlock(treeCid: Cid, index: Natural)</code></td><td>Retrieve block from a Merkle tree by leaf index</td><td>Tree CID, index</td><td><code>Future[?!Block]</code></td></tr>
<tr><td><code>getBlock(address: BlockAddress)</code></td><td>Retrieve block via unified address</td><td>BlockAddress</td><td><code>Future[?!Block]</code></td></tr>
<tr><td><code>getBlockAndProof(treeCid: Cid, index: Natural)</code></td><td>Retrieve block with Merkle proof</td><td>Tree CID, index</td><td><code>Future[?!(Block, CodexProof)]</code></td></tr>
<tr><td><code>getCid(treeCid: Cid, index: Natural)</code></td><td>Retrieve leaf CID from tree metadata</td><td>Tree CID, index</td><td><code>Future[?!Cid]</code></td></tr>
<tr><td><code>getCidAndProof(treeCid: Cid, index: Natural)</code></td><td>Retrieve leaf CID with inclusion proof</td><td>Tree CID, index</td><td><code>Future[?!(Cid, CodexProof)]</code></td></tr>
<tr><td><code>putBlock(blk: Block, ttl: Duration)</code></td><td>Store block with quota enforcement</td><td>Block, optional TTL</td><td><code>Future[?!void]</code></td></tr>
<tr><td><code>putCidAndProof(treeCid: Cid, index: Natural, blkCid: Cid, proof: CodexProof)</code></td><td>Store leaf metadata with ref counting</td><td>Tree CID, index, block CID, proof</td><td><code>Future[?!void]</code></td></tr>
<tr><td><code>hasBlock(...)</code></td><td>Check block existence (CID or tree leaf)</td><td>CID / Tree CID + index</td><td><code>Future[?!bool]</code></td></tr>
<tr><td><code>delBlock(...)</code></td><td>Delete block/tree leaf (with ref count checks)</td><td>CID / Tree CID + index</td><td><code>Future[?!void]</code></td></tr>
<tr><td><code>ensureExpiry(...)</code></td><td>Update expiry for block/tree leaf</td><td>CID / Tree CID + index, expiry timestamp</td><td><code>Future[?!void]</code></td></tr>
<tr><td><code>listBlocks(blockType: BlockType)</code></td><td>Iterate over stored blocks</td><td>Block type</td><td><code>Future[?!SafeAsyncIter[Cid]]</code></td></tr>
<tr><td><code>getBlockExpirations(maxNumber, offset)</code></td><td>Retrieve block expiry metadata</td><td>Pagination params</td><td><code>Future[?!SafeAsyncIter[BlockExpiration]]</code></td></tr>
<tr><td><code>blockRefCount(cid: Cid)</code></td><td>Get block reference count</td><td>CID</td><td><code>Future[?!Natural]</code></td></tr>
<tr><td><code>reserve(bytes: NBytes)</code></td><td>Reserve storage quota</td><td>Bytes</td><td><code>Future[?!void]</code></td></tr>
<tr><td><code>release(bytes: NBytes)</code></td><td>Release reserved quota</td><td>Bytes</td><td><code>Future[?!void]</code></td></tr>
<tr><td><code>start()</code></td><td>Initialize store</td><td>—</td><td><code>Future[void]</code></td></tr>
<tr><td><code>stop()</code></td><td>Gracefully shut down store</td><td>—</td><td><code>Future[void]</code></td></tr>
<tr><td><code>close()</code></td><td>Close underlying datastores</td><td>—</td><td><code>Future[void]</code></td></tr>
</tbody></table>
</div>
<h3 id="store-implementations"><a class="header" href="#store-implementations">Store Implementations</a></h3>
<p>The Store module provides three concrete implementations of the <code>BlockStore</code>
interface,
each optimized for a specific role in the Codex architecture:
RepoStore, NetworkStore, and CacheStore.</p>
<h4 id="repostore"><a class="header" href="#repostore">RepoStore</a></h4>
<p>The RepoStore is a persistent <code>BlockStore</code> implementation
that interfaces directly with low-level storage backends,
such as hard drives and databases.</p>
<p>It uses two distinct DataStore backends:</p>
<ul>
<li>FileSystem — for storing raw block data</li>
<li>LevelDB — for storing associated metadata</li>
</ul>
<p>This separation ensures optimal performance,
allowing block data operations to run efficiently
while metadata updates benefit from a fast key-value database.</p>
<p>Characteristics:</p>
<ul>
<li>Persistent storage via datastore backends</li>
<li>Quota management with precise usage tracking</li>
<li>TTL (time-to-live) support with automated expiration</li>
<li>Metadata storage for block size, reference count, and expiry</li>
<li>Transaction-like operations implemented through reference counting</li>
</ul>
<p>Configuration:</p>
<ul>
<li><code>quotaMaxBytes</code>: Maximum storage quota</li>
<li><code>blockTtl</code>: Default TTL for stored blocks</li>
<li><code>postFixLen</code>: CID key postfix length for sharding</li>
</ul>
<pre><code class="language-text">┌─────────────────────────────────────────────────────────────┐
│ RepoStore │
├─────────────────────────────────────────────────────────────┤
│ ┌─────────────┐ ┌──────────────────────────┐ │
│ │ repoDs │ │ metaDs │ │
│ │ (Datastore) │ │ (TypedDatastore) │ │
│ │ │ │ │ │
│ │ Block Data: │ │ Metadata: │ │
│ │ - Raw bytes │ │ - BlockMetadata │ │
│ │ - CID-keyed │ │ - LeafMetadata │ │
│ │ │ │ - QuotaUsage │ │
│ │ │ │ - Block counts │ │
│ └─────────────┘ └──────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
</code></pre>
<h4 id="networkstore"><a class="header" href="#networkstore">NetworkStore</a></h4>
<p>The NetworkStore is a composite <code>BlockStore</code> that combines local persistence
with network-based retrieval for distributed content access.</p>
<p>It follows a local-first strategy —
attempting to retrieve or store blocks locally first,
and falling back to network retrieval via the Block Exchange Engine
if the block is not available locally.</p>
<p>Characteristics:</p>
<ul>
<li>Integrates local storage with network retrieval</li>
<li>Works seamlessly with the block exchange engine for peer-to-peer access</li>
<li>Transparent block fetching from remote sources</li>
<li>Local caching of blocks retrieved from the network for future access</li>
</ul>
<pre><code class="language-text">┌────────────────────────────────────────────────────────────┐
│ NetworkStore │
├────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────┐ ┌──────────────────────┐ │
│ │ LocalStore - RS │ │ BlockExcEngine │ │
│ │ • Store blocks │ │ • Request blocks │ │
│ │ • Get blocks │ │ • Resolve blocks │ │
│ └─────────────────┘ └──────────────────────┘ │
│ │ │ │
│ └──────────────┬───────────────┘ │
│ │ │
│ ┌─────────────┐ │
│ │BS Interface │ │
│ │ │ │
│ │ • getBlock │ │
│ │ • putBlock │ │
│ │ • hasBlock │ │
│ │ • delBlock │ │
│ └─────────────┘ │
└────────────────────────────────────────────────────────────┘
</code></pre>
<h4 id="cachestore"><a class="header" href="#cachestore">CacheStore</a></h4>
<p>The CacheStore is an in-memory <code>BlockStore</code> implementation
designed for fast access to frequently used blocks.</p>
<p>This store maintains two separate LRU caches:</p>
<ol>
<li>Block Cache — <code>LruCache[Cid, Block]</code>
<ul>
<li>Stores actual block data indexed by CID</li>
<li>Acts as the primary cache for block content</li>
</ul>
</li>
<li>CID/Proof Cache — <code>LruCache[(Cid, Natural), (Cid, CodexProof)]</code>
<ul>
<li>Maps <code>(treeCid, index)</code> to <code>(blockCid, proof)</code></li>
<li>Supports direct access to block proofs keyed by <code>treeCid</code> and index</li>
</ul>
</li>
</ol>
<p>Characteristics:</p>
<ul>
<li>O(1) access times for cached data</li>
<li>LRU eviction policy for memory management</li>
<li>Configurable maximum cache size</li>
<li>No persistence — cache contents are lost on restart</li>
<li>No TTL — blocks remain in cache until evicted</li>
</ul>
<p>Configuration:</p>
<ul>
<li><code>cacheSize</code>: Maximum total cache size (bytes)</li>
<li><code>chunkSize</code>: Minimum block size unit</li>
</ul>
<h3 id="storage-layout"><a class="header" href="#storage-layout">Storage Layout</a></h3>
<div class="table-wrapper"><table><thead><tr><th>Key Pattern</th><th>Data Type</th><th>Description</th><th>Example</th></tr></thead><tbody>
<tr><td><code>repo/manifests/{XX}/{full-cid}</code></td><td>Raw bytes</td><td>Manifest block data</td><td><code>repo/manifests/Cd/bafy...Cd → [data]</code></td></tr>
<tr><td><code>repo/blocks/{XX}/{full-cid}</code></td><td>Raw bytes</td><td>Block data</td><td><code>repo/blocks/Ab/bafy...Ab → [data]</code></td></tr>
<tr><td><code>meta/ttl/{cid}</code></td><td>BlockMetadata</td><td>Expiry, size, refCount</td><td><code>meta/ttl/bafy... → {...}</code></td></tr>
<tr><td><code>meta/proof/{treeCid}/{index}</code></td><td>LeafMetadata</td><td>Merkle proof for leaf</td><td><code>meta/proof/bafy.../42 → {...}</code></td></tr>
<tr><td><code>meta/total</code></td><td>Natural</td><td>Total stored blocks</td><td><code>meta/total → 12039</code></td></tr>
<tr><td><code>meta/quota/used</code></td><td>NBytes</td><td>Used quota</td><td><code>meta/quota/used → 52428800</code></td></tr>
<tr><td><code>meta/quota/reserved</code></td><td>NBytes</td><td>Reserved quota</td><td><code>meta/quota/reserved → 104857600</code></td></tr>
</tbody></table>
</div>
<h3 id="workflows"><a class="header" href="#workflows">Workflows</a></h3>
<p>The following flow charts summarize how put, get, and delete operations
interact with the shared block storage, metadata store,
and quota management systems.</p>
<h4 id="putblock"><a class="header" href="#putblock">PutBlock</a></h4>
<p>The following flow chart shows how a block is stored
with metadata and quota management:</p>
<pre><code class="language-text">putBlock: blk, ttl
│
├─&gt; Calculate expiry = now + ttl
│
├─&gt; storeBlock: blk, expiry
│
├─&gt; Block empty?
│ ├─&gt; Yes: Return AlreadyInStore
│ └─&gt; No: Create metadata &amp; block keys
│
├─&gt; Block metadata exists?
│ ├─&gt; Yes: Size matches?
│ │ ├─&gt; Yes: Return AlreadyInStore
│ │ └─&gt; No: Return Error
│ └─&gt; No: Create new metadata
│
├─&gt; Store block data
│
├─&gt; Store successful?
│ ├─&gt; No: Return Error
│ └─&gt; Yes: Update quota usage
│
├─&gt; Quota update OK?
│ ├─&gt; No: Rollback: Delete block → Return Error
│ └─&gt; Yes: Update total blocks count
│
├─&gt; Trigger onBlockStored callback
│
└─&gt; Return Success
</code></pre>
<h4 id="getblock"><a class="header" href="#getblock">GetBlock</a></h4>
<p>The following flow chart explains how a block is retrieved by CID
or tree reference,
resolving metadata if necessary,
and returning the block or an error:</p>
<pre><code class="language-text">getBlock: cid/address
│
├─&gt; Input type?
│ ├─&gt; BlockAddress with leaf
│ │ └─&gt; getLeafMetadata: treeCid, index
│ │ ├─&gt; Leaf metadata found?
│ │ │ ├─&gt; No: Return BlockNotFoundError
│ │ │ └─&gt; Yes: Extract block CID from metadata
│ └─&gt; CID: Direct CID access
│
├─&gt; CID empty?
│ ├─&gt; Yes: Return empty block
│ └─&gt; No: Create prefix key
│
├─&gt; Query datastore: repoDs.get
│
├─&gt; Block found?
│ ├─&gt; No: Error type?
│ │ ├─&gt; DatastoreKeyNotFound: Return BlockNotFoundError
│ │ └─&gt; Other: Return Error
│ └─&gt; Yes: Create Block with verification
│
└─&gt; Return Block
</code></pre>
<h4 id="delblock"><a class="header" href="#delblock">DelBlock</a></h4>
<p>The following flow chart shows how a block is deleted
when it is unused or expired,
including metadata cleanup and quota/counter updates:</p>
<pre><code class="language-text">delBlock: cid
│
├─&gt; delBlockInternal: cid
│
├─&gt; CID empty?
│ ├─&gt; Yes: Return Deleted
│ └─&gt; No: tryDeleteBlock: cid, now
│
├─&gt; Metadata exists?
│ ├─&gt; No: Check if block exists in repo
│ │ ├─&gt; Block exists?
│ │ │ ├─&gt; Yes: Warn &amp; remove orphaned block
│ │ │ └─&gt; No: Return NotFound
│ │ └─&gt; Return NotFound
│ └─&gt; Yes: refCount = 0 OR expired?
│ ├─&gt; No: Return InUse
│ └─&gt; Yes: Delete block &amp; metadata → Return Deleted
│
├─&gt; Handle result
│
├─&gt; Result type?
│ ├─&gt; InUse: Return Error: Cannot delete dataset block
│ ├─&gt; NotFound: Return Success: Ignore
│ └─&gt; Deleted: Update total blocks count
│ └─&gt; Update quota usage
│ └─&gt; Return Success
│
└─&gt; Return Success
</code></pre>
<h3 id="data-models"><a class="header" href="#data-models">Data Models</a></h3>
<h4 id="stores"><a class="header" href="#stores">Stores</a></h4>
<pre><code class="language-nim">RepoStore* = ref object of BlockStore
postFixLen*: int
repoDs*: Datastore
metaDs*: TypedDatastore
clock*: Clock
quotaMaxBytes*: NBytes
quotaUsage*: QuotaUsage
totalBlocks*: Natural
blockTtl*: Duration
started*: bool
NetworkStore* = ref object of BlockStore
engine*: BlockExcEngine
localStore*: BlockStore
CacheStore* = ref object of BlockStore
currentSize*: NBytes
size*: NBytes
cache: LruCache[Cid, Block]
cidAndProofCache: LruCache[(Cid, Natural), (Cid, CodexProof)]
</code></pre>
<h4 id="metadata-types"><a class="header" href="#metadata-types">Metadata Types</a></h4>
<pre><code class="language-nim">BlockMetadata* {.serialize.} = object
expiry*: SecondsSince1970
size*: NBytes
refCount*: Natural
LeafMetadata* {.serialize.} = object
blkCid*: Cid
proof*: CodexProof
BlockExpiration* {.serialize.} = object
cid*: Cid
expiry*: SecondsSince1970
QuotaUsage* {.serialize.} = object
used*: NBytes
reserved*: NBytes
</code></pre>
<h3 id="functional-requirements"><a class="header" href="#functional-requirements">Functional Requirements</a></h3>
<h4 id="available-today"><a class="header" href="#available-today">Available Today</a></h4>
<ul>
<li>
<p>Atomic Block Operations</p>
<ul>
<li>Store, retrieve, and delete operations must be atomic.</li>
<li>Support retrieval via:
<ul>
<li>Direct CID</li>
<li>Tree-based addressing (<code>treeCid + index</code>)</li>
<li>Unified block address</li>
</ul>
</li>
</ul>
</li>
<li>
<p>Metadata Management</p>
<ul>
<li>Store protocol-level metadata (e.g., storage proofs, quota usage).</li>
<li>Store block-level metadata (e.g., reference counts, total block count).</li>
</ul>
</li>
<li>
<p>Multi-Datastore Support</p>
<ul>
<li>Pluggable datastore interface supporting various backends.</li>
<li>Typed datastore operations for metadata type safety.</li>
</ul>
</li>
<li>
<p>Lifecycle &amp; Maintenance</p>
<ul>
<li>BlockMaintainer service for removing expired data.</li>
<li>Configurable maintenance intervals (default: 10 min).</li>
<li>Batch processing (default: 1000 blocks/cycle).</li>
</ul>
</li>
</ul>
<h4 id="future-requirements"><a class="header" href="#future-requirements">Future Requirements</a></h4>
<ul>
<li>
<p>Transaction Rollback &amp; Error Recovery</p>
<ul>
<li>Rollback support for failed multi-step operations.</li>
<li>Consistent state restoration after failures.</li>
</ul>
</li>
<li>
<p>Dataset-Level Operations</p>
<ul>
<li>Handle Dataset level meta data.</li>
<li>Batch operations for dataset block groups.</li>
</ul>
</li>
<li>
<p>Concurrency Control</p>
<ul>
<li>Consistent locking and coordination mechanisms to prevent inconsistencies
during crashes or long-running operations.</li>
</ul>
</li>
<li>
<p>Lifecycle &amp; Maintenance</p>
<ul>
<li>Cooperative scheduling to avoid blocking.</li>
<li>State tracking for large datasets.</li>
</ul>
</li>
</ul>
<h3 id="non-functional-requirements"><a class="header" href="#non-functional-requirements">Non-Functional Requirements</a></h3>
<h4 id="currently-implemented"><a class="header" href="#currently-implemented">Currently Implemented</a></h4>
<ul>
<li>
<p>Security</p>
<ul>
<li>Verify block content integrity upon retrieval.</li>
<li>Enforce quotas to prevent disk exhaustion.</li>
<li>Safe orphaned data cleanup.</li>
</ul>
</li>
<li>
<p>Scalability</p>
<ul>
<li>Configurable storage quotas (default: 20 GiB).</li>
<li>Pagination for metadata queries.</li>
<li>Reference counting–based garbage collection.</li>
</ul>
</li>
<li>
<p>Reliability</p>
<ul>
<li>Metrics collection (<code>codex_repostore_*</code>).</li>
<li>Graceful shutdown with resource cleanup.</li>
</ul>
</li>
</ul>
<h4 id="planned-enhancements"><a class="header" href="#planned-enhancements">Planned Enhancements</a></h4>
<ul>
<li>
<p>Performance</p>
<ul>
<li>Batch metadata updates.</li>
<li>Efficient key lookups with configurable prefix lengths.</li>
<li>Support for both fast and slower storage tiers.</li>
<li>Streaming APIs optimized for extremely large datasets.</li>
</ul>
</li>
<li>
<p>Security</p>
<ul>
<li>Finer-grained quota enforcement across tenants/namespaces.</li>
</ul>
</li>
<li>
<p>Reliability</p>
<ul>
<li>Stronger rollback semantics for multi-node consistency.</li>
<li>Auto-recovery from inconsistent states.</li>
</ul>
</li>
</ul>
<h2 id="wire-format-specification--syntax"><a class="header" href="#wire-format-specification--syntax">Wire Format Specification / Syntax</a></h2>
<p>The Store Module does not define a wire format specification.
It provides an internal storage abstraction
for <a href="https://github.com/codex-storage/nim-codex">Codex</a>
and relies on underlying datastore implementations for serialization
and persistence.</p>
<h2 id="securityprivacy-considerations"><a class="header" href="#securityprivacy-considerations">Security/Privacy Considerations</a></h2>
<ul>
<li>
<p>Block Integrity: The Store Module verifies block content integrity
upon retrieval to ensure data has not been corrupted or tampered with.</p>
</li>
<li>
<p>Quota Enforcement: Storage quotas are enforced
to prevent disk exhaustion attacks.
The default quota is 20 GiB, but this is configurable.</p>
</li>
<li>
<p>Safe Data Cleanup: The maintenance engine safely removes expired
ephemeral data and orphaned blocks without compromising data integrity.</p>
</li>
<li>
<p>Reference Counting: Reference counting–based garbage collection ensures
that blocks are not deleted while they are still in use by other components.</p>
</li>
</ul>
<p>Future security enhancements include finer-grained quota enforcement
across tenants/namespaces and stronger rollback semantics
for multi-node consistency.</p>
<h2 id="rationale"><a class="header" href="#rationale">Rationale</a></h2>
<p>The Store Module design prioritizes:</p>
<ul>
<li>
<p>Decoupling: By introducing the <code>BlockStore</code> interface,
the Store Module decouples storage operations from underlying
datastore semantics,
allowing for flexible backend implementations.</p>
</li>
<li>
<p>Performance: The separation of block data (filesystem) and metadata (LevelDB)
in RepoStore ensures optimal performance for both types of operations.</p>
</li>
<li>
<p>Flexibility: The three store implementations
(RepoStore, NetworkStore, CacheStore) provide different trade-offs
between persistence, network access, and performance,
allowing Codex to optimize for different use cases.</p>
</li>
<li>
<p>Scalability: Reference counting, quota management, and pagination enable
the Store Module to scale to large datasets
while preventing resource exhaustion.</p>
</li>
</ul>
<p>The current limitations (lack of dataset-level operations, inconsistent locking)
are acknowledged and will be addressed in future versions.</p>
<h2 id="copyright"><a class="header" href="#copyright">Copyright</a></h2>
<p>Copyright and related rights waived via <a href="https://creativecommons.org/publicdomain/zero/1.0/">CC0</a>.</p>
<h2 id="references"><a class="header" href="#references">References</a></h2>
<h3 id="normative"><a class="header" href="#normative">normative</a></h3>
<ul>
<li><a href="https://github.com/codex-storage/nim-codex">Codex</a></li>
<li><a href="https://github.com/codex-storage/codex-docs-obsidian/blob/main/10%20Notes/Specs/Component%20Specification%20-%20Store.md">Component Specification - Store</a></li>
</ul>
<h3 id="informative"><a class="header" href="#informative">informative</a></h3>
<ul>
<li><a href="https://github.com/codex-storage/nim-datastore">nim-datastore</a></li>
<li><a href="https://github.com/codex-storage/nim-datastore/blob/master/datastore/datastore.nim">DataStore Interface</a></li>
<li><a href="https://github.com/status-im/nim-chronos">chronos</a> - Async runtime</li>
<li><a href="https://github.com/status-im/nim-libp2p">libp2p</a> - P2P networking and CID types</li>
<li><a href="https://github.com/codex-storage/questionable">questionable</a> - Error handling</li>
<li><a href="https://github.com/status-im/lrucache">lrucache</a> - LRU cache implementation</li>
</ul>
</main>
<nav class="nav-wrapper" aria-label="Page navigation">
<!-- Mobile navigation buttons -->
<a rel="prev" href="../../storage/raw/manifest.html" class="mobile-nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<i class="fa fa-angle-left"></i>
</a>
<a rel="next prefetch" href="../../storage/raw/dataset-store.html" class="mobile-nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
<div style="clear: both"></div>
</nav>
</div>
</div>
<nav class="nav-wide-wrapper" aria-label="Page navigation">
<a rel="prev" href="../../storage/raw/manifest.html" class="nav-chapters previous" title="Previous chapter" aria-label="Previous chapter" aria-keyshortcuts="Left">
<i class="fa fa-angle-left"></i>
</a>
<a rel="next prefetch" href="../../storage/raw/dataset-store.html" class="nav-chapters next" title="Next chapter" aria-label="Next chapter" aria-keyshortcuts="Right">
<i class="fa fa-angle-right"></i>
</a>
</nav>
</div>
<script>
window.playground_copyable = true;
</script>
<script src="../../elasticlunr.min.js"></script>
<script src="../../mark.min.js"></script>
<script src="../../searcher.js"></script>
<script src="../../clipboard.min.js"></script>
<script src="../../highlight.js"></script>
<script src="../../book.js"></script>
<!-- Custom JS scripts -->
<script src="../../theme/katex/katex.min.js"></script>
<script src="../../theme/katex/auto-render.min.js"></script>
<script src="../../scripts/katex-render.js"></script>
<script src="../../scripts/logos-lips.js"></script>
</div>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink