mirror of
https://github.com/logos-storage/logos-storage-docs-obsidian.git
synced 2026-07-27 17:03:11 +00:00
some renamings Codex => Logos Storage
This commit is contained in:
parent
0cbe7f4f86
commit
37c8ec286d
2
.obsidian/core-plugins.json
vendored
2
.obsidian/core-plugins.json
vendored
@ -26,7 +26,7 @@
|
||||
"audio-recorder": false,
|
||||
"workspaces": false,
|
||||
"file-recovery": true,
|
||||
"publish": false,
|
||||
"publish": true,
|
||||
"sync": true,
|
||||
"bases": true,
|
||||
"webviewer": false
|
||||
|
||||
@ -1,10 +0,0 @@
|
||||
|
||||
**Goal.** The goal of this page is to serve as an entry point to the use-case information that enters the Codex project. We flesh those out by project when possible. What has not yet been fleshed out goes under "Misc".
|
||||
|
||||
#### Project Use Cases
|
||||
|
||||
* [[Codex Use Cases - Status]]
|
||||
* [Codex Use Cases - Waku](https://github.com/waku-org/pm/pull/328)
|
||||
#### Misc
|
||||
|
||||
* [[Codex Use Cases - Václav's Thoughts]]
|
||||
@ -14,7 +14,7 @@ test "Should retrieve block expiration information":
|
||||
# ...
|
||||
```
|
||||
|
||||
Compiling this on Ubuntu (see also [[How to build Codex on Ubuntu 24 (VM)]]) fails with the following error:
|
||||
Compiling this on Ubuntu (see also [[How to build Storage on Ubuntu 24 (VM)]]) fails with the following error:
|
||||
|
||||
```bash
|
||||
stack trace: (most recent call last)
|
||||
|
||||
@ -15,4 +15,4 @@ The workflow is the following:
|
||||
10. Consider deleting the merged content and the "branch" folder from HackMD.
|
||||
_HackMD does not seem to have any convenient way to export comments, so if you need to preserve the conversation, I would consider creating a separate document and copy the conversation or its summary there. Keeping the document hanging in HackMD sooner or later will result in a mass. Let's build a good habit: **all documents in GitHub**._
|
||||
|
||||
Consider checking our [[Using Codex Obsidian Vault Video Tutorials]]. There is one about using HackMD, GitHub, and Obsidian together.
|
||||
Consider checking our [[Using Logos Storage Obsidian Vault Video Tutorials]]. There is one about using HackMD, GitHub, and Obsidian together.
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
We show how to use symmetric encryption in BearSSL to encrypt and decrypt the content.
|
||||
This is a very basic code demonstrating the use of the BearSSL symmetric encryption API. In [[Codex Encryption Design]] we show a more complete example. In the code below we also use [[BearSSL]] to generate a random number. See [[How to generate a random number using BearSSL]] for more details:
|
||||
This is a very basic code demonstrating the use of the BearSSL symmetric encryption API. In [[Storage Encryption Design]] we show a more complete example. In the code below we also use [[BearSSL]] to generate a random number. See [[How to generate a random number using BearSSL]] for more details:
|
||||
|
||||
```nim
|
||||
import std/sequtils
|
||||
|
||||
@ -1,5 +0,0 @@
|
||||
What was mentioned:
|
||||
|
||||
- [Handling of several active smart contracts](https://github.com/codex-storage/nim-codex/issues/1296)
|
||||
- [nim-groth16/pulls](https://github.com/codex-storage/nim-groth16/pulls)
|
||||
|
||||
@ -3,13 +3,13 @@ At a higher level, Block Exchange Module is responsible for sending and receivin
|
||||
When uploading the content to the network, the following steps are taken:
|
||||
|
||||
1. The content is chunked into fixed size blocks. The default block size is `64KiB`. The last block will be padded with `0`s.
|
||||
2. For each chunk, the corresponding `CID` is created, using `(codex-block, 0xCD02)` as a [[Multicodec]] and `(sha2-256, 0x12)` as [[Mutihash]].
|
||||
2. For each chunk, the corresponding `CID` is created, using `(storage-block, 0xCD02)` as a [[Multicodec]] and `(sha2-256, 0x12)` as [[Mutihash]].
|
||||
3. The resulting block, `(seq[byte], CID)` becomes the input to the Block Exchange Module `putBlock` operation.
|
||||
|
||||
When downloading the content from the network:
|
||||
|
||||
1. Given the Codex Manifest CID, the corresponding Codex Manifest is retrieved. Using manifest attributes - `datasetSize` and `blockSize`, the number of blocks - $blockCount$ is computed as $\lceil datasetSize/blockSize \rceil$.
|
||||
2. For each $blockIndex \in [0 .. blockCount]$, a $BlockAddress$ is defined as a tuple $(treeCid, blockIndex)$ where $treeCid$ is an attribute in the Codex Manifest. $BlockAddress$ is the input of the Block Exchange Module `requestBlock` operation.
|
||||
1. Given the Storage Manifest CID, the corresponding Storage Manifest is retrieved. Using manifest attributes - `datasetSize` and `blockSize`, the number of blocks - $blockCount$ is computed as $\lceil datasetSize/blockSize \rceil$.
|
||||
2. For each $blockIndex \in [0 .. blockCount]$, a $BlockAddress$ is defined as a tuple $(treeCid, blockIndex)$ where $treeCid$ is an attribute in the Storage Manifest. $BlockAddress$ is the input of the Block Exchange Module `requestBlock` operation.
|
||||
|
||||
The `putBlock` and `requestBlock` operations defined above, define the external interface of the Block Exchange Module. The Block Exchange Module directly interacts with other modules, mainly [[Discovery Module]] and the [[Repo Store]], used for the node discovery and local storage respectively.
|
||||
|
||||
@ -18,7 +18,7 @@ The `putBlock` and `requestBlock` operations defined above, define the external
|
||||
Before defining the operational model of the Block Exchange Module, which describes *how* the exchange module achieves its goals, let's the find the functional requirements that define *what* the exchange module is expected to achieve.
|
||||
|
||||
1. Given $BlockAddress = (treeCid, blockIndex)$, retrieve the corresponding block from the network. In the operational model, we will see that the exchange module will attempt $3000$ retires, waiting $500ms$ in each iteration for the block to be delivered. Thus, from the declarative perspective, here, we can claim that exchange module should deliver the block withing $3000 \times 500ms = 1500s = 25min$, or fail otherwise.
|
||||
2. When uploading content $C$, for each block $b \in C$, store $b$ in [[Repo Store]] and (1) announce the block *presence* to the peers signaling interest in $b$, (2) deliver $b$ to the peers that earlier requested $b$. Include Merkle Proof in the delivery if $b$ is not Codex Manifest block.
|
||||
2. When uploading content $C$, for each block $b \in C$, store $b$ in [[Repo Store]] and (1) announce the block *presence* to the peers signaling interest in $b$, (2) deliver $b$ to the peers that earlier requested $b$. Include Merkle Proof in the delivery if $b$ is not Storage Manifest block.
|
||||
|
||||
## Operational Model
|
||||
|
||||
|
||||
@ -1,10 +1,10 @@
|
||||
# Codex Block Exchange Specification
|
||||
# Block Exchange Specification
|
||||
|
||||
The Block Exchange (BE) is a core component of Codex and is responsible for peer-to-peer content distribution. It handles the sending and receiving of blocks across the network, enabling efficient data sharing between Codex nodes.
|
||||
The Block Exchange (BE) is a core component of Logos Storage and is responsible for peer-to-peer content distribution. It handles the sending and receiving of blocks across the network, enabling efficient data sharing between Storage nodes.
|
||||
|
||||
## Overview
|
||||
|
||||
The Codex Block Exchange defines both an internal service and a protocol through which Codex nodes can refer to, and provide data blocks to one another. Blocks are uniquely identifiable by means of an _address_, and represent fixed-length chunks of arbitrary data.
|
||||
The Storage Block Exchange defines both an internal service and a protocol through which Storage nodes can refer to, and provide data blocks to one another. Blocks are uniquely identifiable by means of an _address_, and represent fixed-length chunks of arbitrary data.
|
||||
|
||||
Whenever a peer $A$ wishes to obtain a block, it registers its unique address with the BE, and the BE will then be in charge of procuring it; i.e, of finding a peer that has block, if any, and then downloading it. The BE will also accept requests from peers connected to $A$ which might want blocks that $A$ has, and provide them.
|
||||
|
||||
@ -13,20 +13,20 @@ Whenever a peer $A$ wishes to obtain a block, it registers its unique address wi
|
||||
1. have $b$;
|
||||
2. are reasonably expected to obtain $b$ in the future.
|
||||
|
||||
In practical implementations, the BE will typically require the support of an underlying _discovery service_, e.g., the [Codex DHT](), to look up such peers, but this is beyond the scope of this document.
|
||||
In practical implementations, the BE will typically require the support of an underlying _discovery service_, e.g., the Storage DHT, to look up such peers, but this is beyond the scope of this document.
|
||||
|
||||
### Block Format
|
||||
|
||||
Blocks in Codex can be of two different types:
|
||||
Blocks in Storage can be of two different types:
|
||||
|
||||
* **standalone blocks** are self-contained pieces of data addressed by a content ID made from the SHA256 hash of the contents of the block;
|
||||
* **dataset blocks**, instead, are part of an ordered set (a dataset) and can be _additionally_ addressed by a `(datasetCID, index)` tuple which indexes the block within that dataset. `datasetCID`, here, represents the root of a Merkle tree computed over all the blocks in the dataset. In other words, a dataset block can be addressed both as a standalone block (by a CID computed over the contents of the block), or as an index within an ordered set identified by a Merkle root.
|
||||
|
||||
Formally, we can define a block as tuple consisting of raw data and its content identifier: `(data: seq[byte], cid: Cid)`, where standalone blocks are addressed by `cid`, and dataset blocks can be addressed either by `cid` or a `(datasetCID, index)` tuple.
|
||||
|
||||
**Creating blocks.** Blocks in Codex have default size of 64 KiB. Blocks within a dataset must be all of the same size. If a dataset does not contain enough data to fill its last block, it MUST be padded with zeroes.
|
||||
**Creating blocks.** Blocks in Storage have default size of 64 KiB. Blocks within a dataset must be all of the same size. If a dataset does not contain enough data to fill its last block, it MUST be padded with zeroes.
|
||||
|
||||
**Multicodec/Multihash.** The libp2p multicodec for a block CID is `codex-block` (0xCD02), while the multihash is `sha2-256` (0x12).
|
||||
**Multicodec/Multihash.** The libp2p multicodec for a block CID is `storage-block` (0xCD02), while the multihash is `sha2-256` (0x12).
|
||||
|
||||
### Service Interface
|
||||
|
||||
@ -56,7 +56,7 @@ In practice, the BE relies on other modules and services:
|
||||
The Block Exchange Protocol uses the following libp2p protocol identifier:
|
||||
|
||||
```
|
||||
/codex/blockexc/1.0.0
|
||||
/storage/blockexc/1.0.0
|
||||
```
|
||||
|
||||
## Connection Model
|
||||
@ -103,7 +103,7 @@ message Message {
|
||||
|
||||
### Block Addressing
|
||||
|
||||
Codex uses a block addressing scheme that supports both standalone content-addressed blocks and blocks within Merkle tree structures.
|
||||
Storage uses a block addressing scheme that supports both standalone content-addressed blocks and blocks within Merkle tree structures.
|
||||
|
||||
```protobuf
|
||||
message BlockAddress {
|
||||
@ -180,12 +180,12 @@ message BlockDelivery {
|
||||
- `cid`: Content identifier of the block
|
||||
- `data`: Raw block data (up to 100 MiB)
|
||||
- `address`: The address that was requested
|
||||
- `proof`: Merkle proof (CodexProof) verifying block correctness (required for dataset blocks)
|
||||
- `proof`: Merkle proof (StorageProof) verifying block correctness (required for dataset blocks)
|
||||
|
||||
**Merkle Proof Verification:**
|
||||
|
||||
When delivering dataset blocks (`address.leaf = true`):
|
||||
- The delivery must include a Merkle proof (CodexProof)
|
||||
- The delivery must include a Merkle proof (StorageProof)
|
||||
- The proof verifies that the block at the given index is correctly part of the Merkle tree identified by the tree CID
|
||||
- This applies to all datasets, irrespective of whether they have been erasure-coded or not
|
||||
- Recipients must verify the proof before accepting the block
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
---
|
||||
related-to:
|
||||
- "[[Codex Encryption Design]]"
|
||||
- "[[Codex Encryption Implementation]]"
|
||||
- "[[Storage Encryption Design]]"
|
||||
- "[[Storage Encryption Implementation]]"
|
||||
---
|
||||
We want some kind of encryption of user data, definitely on by default (maybe even enforced in all cases).
|
||||
|
||||
@ -1,9 +1,9 @@
|
||||
---
|
||||
related-to:
|
||||
- "[[Codex Encryption Basis]]"
|
||||
- "[[Storage Encryption Basis]]"
|
||||
- "[[How to get a pointer to a seq to pass it to a C library]]"
|
||||
---
|
||||
As summarized in [[Codex Encryption Basis#^b2e265|the proposal]], we:
|
||||
As summarized in [[Storage Encryption Basis#^b2e265|the proposal]], we:
|
||||
|
||||
- use a freshly generated random master key (at least 256 bits) per dataset (generated and kept on the user's machine)
|
||||
- derive a new encryption key and also an IV for each block from the master key and the block index
|
||||
0
10 Notes/Storage Encryption Implementation.md
Normal file
0
10 Notes/Storage Encryption Implementation.md
Normal file
10
10 Notes/Storage Use Cases.md
Normal file
10
10 Notes/Storage Use Cases.md
Normal file
@ -0,0 +1,10 @@
|
||||
|
||||
**Goal.** The goal of this page is to serve as an entry point to the use-case information that enters the Storage project. We flesh those out by project when possible. What has not yet been fleshed out goes under "Misc".
|
||||
|
||||
#### Project Use Cases
|
||||
|
||||
* [[Storage Use Cases - Status]]
|
||||
* [Codex Use Cases - Waku](https://github.com/waku-org/pm/pull/328)
|
||||
#### Misc
|
||||
|
||||
* [[Storage Use Cases - Václav's Thoughts]]
|
||||
@ -1,5 +1,5 @@
|
||||
|
||||
Undigested chat on the options we have for structuring a C Codex API on top of Chronos code.
|
||||
Undigested chat on the options we have for structuring a C Storage API on top of Chronos code.
|
||||
|
||||
### Giuliano (part-time) _—_ 9:15 AM
|
||||
|
||||
@ -1 +0,0 @@
|
||||
[[10 Notes/Meetings/Codex/Codex-mc2 - {{title}}|Client Team Meeting (mc2)- {{title}}]]
|
||||
1
90 Extras/91 Templates/Storage Team Meeting.md
Normal file
1
90 Extras/91 Templates/Storage Team Meeting.md
Normal file
@ -0,0 +1 @@
|
||||
[[10 Notes/Meetings/Storage/Storage-mc2 - {{title}}|Storage Team Meeting (mc2)- {{title}}]]
|
||||
@ -1,7 +1,7 @@
|
||||
```
|
||||
MIT License
|
||||
|
||||
Copyright (c) [2025] [Codex Team]
|
||||
Copyright (c) [2026] [Logos Storage Team]
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
|
||||
@ -14,6 +14,6 @@ Finally, open the cloned folder (a vault) in Obsidian.
|
||||
|
||||
Now you have a searchable Codex documentation vault under your full control. Everything you do in Obsidian remains private as long as you do not commit and push your changes to GitHub. Our `.gitignore` keeps some of the vault file ignored, meaning they will stay only visible to your Obsidian installation. For instance, the publishing settings and the workspace configuration (kept in the `.obsidian` folder) are excluded from GitHub syncing, as those are personal and change every time you open the vault. You can read more about what is ignored in [[Ignored Obsidian Vault files]].
|
||||
|
||||
> To get some background about Obsidian and *Knowledge Management Systems*, please check [[Short Introduction to Obsidian]]. We also have some video tutorials: see [[Using Codex Obsidian Vault Video Tutorials]].
|
||||
> To get some background about Obsidian and *Knowledge Management Systems*, please check [[Short Introduction to Obsidian]]. We also have some video tutorials: see [[Using Logos Storage Obsidian Vault Video Tutorials]].
|
||||
|
||||
Now, when you have everything up and running, learn a bit more about our documentation setup and how to take full advantage of it. See [[Codex Documentation Setup]] in - of course - Obsidian.
|
||||
Now, when you have everything up and running, learn a bit more about our documentation setup and how to take full advantage of it. See [[Logos Storage Documentation Setup]] in - of course - Obsidian.
|
||||
|
||||
8
index.md
8
index.md
@ -1,9 +1,7 @@
|
||||
---
|
||||
title: "Codex Storage Knowledge Base"
|
||||
title: Logos Storage Knowledge Base
|
||||
---
|
||||
|
||||
# Welcome to Codex Storage Knowledge Base
|
||||
|
||||
This is the knowledge base for the Codex project - a decentralized storage protocol. If you are looking for the official documentation, you should head to [https://docs.codex.storage](https://docs.codex.storage).
|
||||
|
||||
# Welcome to Logos Storage Knowledge Base
|
||||
|
||||
This is the knowledge base for the Logos Storage project - a decentralized file sharing client. If you are looking for the official documentation, you should head to [https://docs.logos.co/storage](https://docs.logos.co/storage).
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user