Add Download Flow (#29)

* Add mermaid support

* Add Download Flow

* Adjust outline to show all the headers for Run Codex

* Update doc links to the internal ones
This commit is contained in:
Slava 2024-10-01 13:46:00 +03:00
parent 27c009e07e
commit 291d65a865
No known key found for this signature in database
GPG Key ID: 351E7AA9BD0DFEB8
6 changed files with 1257 additions and 12 deletions

View File

@ -1,10 +1,10 @@
import { defineConfig } from 'vitepress'
import mdFootnote from "markdown-it-footnote"
// import { defineConfig } from 'vitepress'
import mdFootnote from 'markdown-it-footnote'
import { withMermaid } from 'vitepress-plugin-mermaid'
// const { BASE: base = '/' } = process.env;
// https://vitepress.dev/reference/site-config
export default defineConfig({
export default withMermaid({
lang: 'en-US',
title: 'Codex Docs',
description: 'Decentralised data storage platform',
@ -21,6 +21,10 @@ export default defineConfig({
},
// base: base,
mermaid:{
//mermaidConfig !theme here works for ligth mode since dark theme is forced in dark mode
},
// lite-youtube-embed
vue: {
template: {
@ -98,7 +102,8 @@ export default defineConfig({
{ text: 'Run Codex', link: '/learn/run' },
{ text: 'Using Codex', link: '/learn/using' },
{ text: 'Local Two Client Test', link: '/learn/local-two-client-test' },
{ text: 'Local Marketplace', link: '/learn/local-marketplace' }
{ text: 'Local Marketplace', link: '/learn/local-marketplace' },
{ text: 'Download Flow', link: '/learn/download-flow' }
]
},
{

67
learn/download-flow.md Normal file
View File

@ -0,0 +1,67 @@
# Download Flow
Sequence of interactions that result in dat blocks being transferred across the network.
## Local Store
When data is available in the local blockstore,
```mermaid
sequenceDiagram
actor Alice
participant API
Alice->>API: Download(CID)
API->>+Node/StoreStream: Retrieve(CID)
loop Get manifest block, then data blocks
Node/StoreStream->>NetworkStore: GetBlock(CID)
NetworkStore->>LocalStore: GetBlock(CID)
LocalStore->>NetworkStore: Block
NetworkStore->>Node/StoreStream: Block
end
Node/StoreStream->>Node/StoreStream: Handle erasure coding
Node/StoreStream->>-API: Data stream
API->>Alice: Stream download of block
```
## Network Store
When data is not found ih the local blockstore, the block-exchange engine is used to discover the location of the block within the network. Connection will be established to the node(s) that have the block, and exchange can take place.
```mermaid
sequenceDiagram
box
actor Alice
participant API
participant Node/StoreStream
participant NetworkStore
participant Discovery
participant Engine
end
box
participant OtherNode
end
Alice->>API: Download(CID)
API->>+Node/StoreStream: Retrieve(CID)
Node/StoreStream->>-API: Data stream
API->>Alice: Download stream begins
loop Get manifest block, then data blocks
Node/StoreStream->>NetworkStore: GetBlock(CID)
NetworkStore->>Engine: RequestBlock(CID)
opt CID not known
Engine->>Discovery: Discovery Block
Discovery->>Discovery: Locates peers who provide block
Discovery->>Engine: Peers
Engine->>Engine: Update peers admin
end
Engine->>Engine: Select optimal peer
Engine->>OtherNode: Send WantHave list
OtherNode->>Engine: Send BlockPresence
Engine->>Engine: Update peers admin
Engine->>Engine: Decide to buy block
Engine->>OtherNode: Send WantBlock list
OtherNode->>Engine: Send Block
Engine->>NetworkStore: Block
NetworkStore->>NetworkStore: Add to Local store
NetworkStore->>Node/StoreStream: Resolve Block
Node/StoreStream->>Node/StoreStream: Handle erasure coding
Node/StoreStream->>API: Push data to stream
end
API->>Alice: Download stream finishes
```

View File

@ -202,4 +202,4 @@ Want to stay up-date, or looking for further assistance? Try our [discord-server
Ready to explore Codex functionality? Please [Join Codex Testnet](/networks/testnet).
If you want to run Codex locally without joining the Testnet, consider trying the [two-client-test](https://github.com/codex-storage/nim-codex/blob/master/docs/TwoClientTest.md) or the [marketplace-test](https://github.com/codex-storage/nim-codex/blob/master/docs/Marketplace.md).
If you want to run Codex locally without joining the Testnet, consider trying the [Codex Two-Client Test](/learn/local-two-client-test) or the [Running a Local Codex Network with Marketplace Support](/learn/local-marketplace).

View File

@ -1,9 +1,8 @@
---
outline: [2, 4]
---
# Run Codex
[[TOC]]
<hr>
As for now, Codex is implemented only in [Nim](https://nim-lang.org) and can be found in [nim-codex](https://github.com/codex-storage/nim-codex) repository.
It is a command-line application which may be run in a different ways:

1174
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@ -3,7 +3,9 @@
"lite-youtube-embed": "^0.3.3",
"markdown-it-footnote": "^4.0.0",
"markdown-it-mathjax3": "^4.3.2",
"vitepress": "^1.3.2"
"mermaid": "^11.2.1",
"vitepress": "^1.3.2",
"vitepress-plugin-mermaid": "^2.0.17"
},
"scripts": {
"docs:dev": "vitepress dev",