logos-storage/logos-storage-docs
1
0
Fork 0
mirror of https://github.com/logos-storage/logos-storage-docs.git synced 2026-01-02 05:13:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

Testnet update (#90)

Browse Source 
* Updates learn and network docs

* Removes codex networks, removes "join the testnet" on main page

* updates es and ko run files to match english one

* Removes networks panel from main page
This commit is contained in:
ThatBen 2025-09-19 12:57:13 +02:00 committed by GitHub
parent 9312a13eeb
commit 4edfe2abb6
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
14 changed files with 238 additions and 1063 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
.vitepress/config.mts
View File

@ -119,14 +119,6 @@ export default withMermaid({
{ text: 'Troubleshoot', link: '/learn/troubleshoot' }
]
},
{
text: 'Codex networks',
collapsed: false,
items: [
{ text: 'Devnet', link: '/networks/devnet' },
{ text: 'Testnet', link: '/networks/testnet' }
]
},
{
text: 'Developers',
collapsed: false,
@ -310,13 +302,6 @@ export default withMermaid({
{ text: '문제 해결', link: '/ko/learn/troubleshoot' }
]
},
{
text: 'Codex networks',
collapsed: false,
items: [
{ text: '테스트넷', link: '/ko/networks/testnet' }
]
},
{
text: 'Developers',
collapsed: false,

1
README.md
View File

@ -74,7 +74,6 @@
Project structure
- `learn` - All information to learn about Codex
- `networks` - Codex networks related information
- `developers` - Codex development process and guides

2
es/learn/quick-start.md
View File

@ -145,6 +145,4 @@ Also, you can interact with Codex using [Codex API](/developers/api) and for a w
Want to stay up-date, or looking for further assistance? Try our [discord-server](https://discord.gg/codex-storage).
Ready to explore Codex functionality? Please [Join Codex Testnet](/networks/testnet).
If you want to run Codex locally without joining the Testnet, consider trying the [Codex Two-Client Test](/learn/local-two-client-test) or the [Running a Local Codex Network with Marketplace Support](/learn/local-marketplace).

431
es/learn/run.md
View File

@ -7,7 +7,7 @@ As for now, Codex is implemented only in [Nim](https://nim-lang.org) and can be
It is a command-line application which may be run in a different ways:
- [Using binary](#using-binary)
- [Run as a daemon in Linux](#run-as-a-daemon-in-linux) (not supported yet)
- [Run as a service in Linux](#run-as-a-service-in-linux)
- [Run as a service in Windows](#run-as-a-service-in-windows) (not supported yet)
- [Using Docker](#using-docker)
- [Using Docker Compose](#using-docker-compose)
@ -66,12 +66,13 @@ The following options are available:
-d, --data-dir The directory where codex will store configuration and data
[=/root/.cache/codex].
-i, --listen-addrs Multi Addresses to listen on [=/ip4/0.0.0.0/tcp/0].
-a, --nat IP Addresses to announce behind a NAT [=127.0.0.1].
-e, --disc-ip Discovery listen address [=0.0.0.0].
-a, --nat NAT traversal method for determining the public address.
Options: any, none, upnp, pmp, extip:<IP> [any]
-u, --disc-port Discovery (UDP) port [=8090].
--net-privkey Source of network (secp256k1) private key file path or name [=key].
-b, --bootstrap-node Specifies one or more bootstrap nodes to use when connecting to the network.
--max-peers The maximum number of peers to connect to [=160].
--num-threads Number of worker threads (\"0\" = use as many threads as there are CPU cores available).
--agent-string Node agent string which is used as identifier in network [=Codex].
--api-bindaddr The REST API bind address [=127.0.0.1].
-p, --api-port The REST Api port [=8080].
@ -94,13 +95,15 @@ codex persistence [OPTIONS]... command
The following options are available:
--eth-provider The URL of the JSON-RPC API of the Ethereum node [=ws://localhost:8545].
--eth-account The Ethereum account that is used for storage contracts.
--eth-private-key File containing Ethereum private key for storage contracts.
--marketplace-address Address of deployed Marketplace contract.
--validator Enables validator, requires an Ethereum node [=false].
--validator-max-slots Maximum number of slots that the validator monitors [=1000].
--reward-recipient Address to send payouts to (eg rewards and refunds).
--eth-provider The URL of the JSON-RPC API of the Ethereum node [=ws://localhost:8545].
--eth-account The Ethereum account that is used for storage contracts.
--eth-private-key File containing Ethereum private key for storage contracts.
--marketplace-address Address of deployed Marketplace contract.
--validator Enables validator, requires an Ethereum node [=false].
--validator-max-slots Maximum number of slots that the validator monitors [=1000].
--reward-recipient Address to send payouts to (eg rewards and refunds).
--request-cache-size Maximum number of StorageRequests kept in memory. Reduces fetching of StorageRequest data from the contract. [=128].
--max-priority-fee-per-gas Sets the default maximum priority fee per gas for Ethereum EIP-1559 transactions, in wei, when not provided by the network.
Available sub-commands:
@ -159,15 +162,10 @@ The Codex node can then read the configuration from this file using the `--confi
codex --config-file=/path/to/your/config.toml
```
Please check [Run as a service in Linux](#run-as-a-service-in-linux) for a full example of configuration file.
## Run
Basically, we can run Codex in three different modes:
- [Codex node](#codex-node) - useful for local testing/development and basic/files sharing.
- [Codex node with marketplace support](#codex-node-with-marketplace-support) - you can share files and buy the storage, this is the main mode and should be used by the end users.
- [Codex storage node](#codex-storage-node) - should be used by storage providers or if you would like to sell your local storage.
We also will touch in some words [Codex bootstrap node](#codex-bootstrap-node).
### Using binary
#### Codex node
@ -176,8 +174,6 @@ We can run Codex in a simple way like following:
```shell
codex
```
> [!WARNING]
> This command may not work properly when we use GitHub releases [^data-dir].
But, it will use a default `data-dir` value and we can pass a custom one:
```shell
@ -200,18 +196,19 @@ Also, to make your Codex node accessible for other network participants, it is r
codex \
--data-dir=datadir \
--bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P \
--nat=<your public IP>
--nat=any
```
> [!TIP]
> We can set public IP using curl and IP lookup service, like [ip.codex.storage](https://ip.codex.storage).
> In this case, you can override the IP by which your node annouces itself like this: `--nat=extip:1.2.3.4`
After that, node will announce itself using your public IP, default UDP ([discovery](https://docs.libp2p.io/concepts/discovery-routing/overview/)) and dynamic TCP port ([data transfer](https://docs.libp2p.io/concepts/transports/overview/)), which can be adjusted in the following way:
```shell
codex \
--data-dir=datadir \
--bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P \
--nat=`curl -s https://ip.codex.storage` \
--nat=any \
--disc-port=8090 \
--listen-addrs=/ip4/0.0.0.0/tcp/8070
```
@ -231,14 +228,14 @@ Basically, for P2P communication we should specify and configure two ports:
| 1 | UDP | [Discovery](https://docs.libp2p.io/concepts/discovery-routing/overview/) | `--disc-port` | `--disc-port=8090` |
| 2 | TCP | [Transport](https://docs.libp2p.io/concepts/transports/overview/) | `--listen-addrs` | `--listen-addrs=/ip4/0.0.0.0/tcp/8070` |
And, also it is required to setup port-forwarding on your Internet router, to make your node accessible for participants [^port-forwarding].
And, also it is required to setup [port-forwarding](#port-forwarding) on your Internet router, to make your node accessible for participants.
So, a fully working basic configuration will looks like following:
```shell
codex \
--data-dir=datadir \
--bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P \
--nat=`curl -s https://ip.codex.storage` \
--nat=any \
--disc-port=8090 \
--listen-addrs=/ip4/0.0.0.0/tcp/8070 \
--api-cors-origin="*"
@ -248,143 +245,19 @@ After node is up and running and port-forwarding configurations was done, we sho
You also can use [Codex App UI](https://app.codex.storage) for files upload/download.
And to be able to purchase a storage, we should run [Codex node with marketplace support](#codex-node-with-marketplace-support).
#### Codex node with marketplace support
[Marketplace](/learn/architecture.md#marketplace-architecture) support permits to purchase the storage in Codex network. Basically, we should add just a `persistence` sub-command and required [CLI options](#cli-options) to the [previous run](#codex-node).
> [!NOTE]
> Please ignore `--eth-account` CLI option, as it is obsolete [^eth-account].
1. For a daily use, we should consider to run a local blockchain node based on the [network](/networks/networks) you would like to join. That process is described in the [Join Codex Testnet](/networks/testnet) guide, but for a quick start we can use a public RPC endpoint.
2. Create a file with ethereum private key and set a proper permissions:
> [!CAUTION]
> Please use key generation service for demo purpose only.
```shell
response=$(curl -s https://key.codex.storage)
awk -F ': ' '/private/ {print $2}' <<<"${response}" > eth.key
awk -F ': ' '/address/ {print $2}' <<<"${response}" > eth.address
chmod 600 eth.key
```
Show your ethereum address:
```shell
cat eth.address
```
```
0x412665aFAb17768cd9aACE6E00537Cc6D5524Da9
```
3. Fill-up your ethereum address with ETH and Tokens based on the the [network](/networks/networks) you would like to join.
4. Specify bootstrap nodes and marketplace address based on the [network](/networks/networks) you would like to join.
5. Run the node:
```shell
codex \
--data-dir=datadir \
--bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P \
--nat=`curl -s https://ip.codex.storage` \
--disc-port=8090 \
--listen-addrs=/ip4/0.0.0.0/tcp/8070 \
--api-cors-origin="*" \
persistence \
--eth-provider=https://rpc.testnet.codex.storage \
--eth-private-key=eth.key \
--marketplace-address=0xAB03b6a58C5262f530D54146DA2a552B1C0F7648
```
> [!NOTE]
> Codex also has a marketplace contract address autodiscovery mechanism based on the chain id, that mapping is done in the [source code](https://github.com/codex-storage/nim-codex/blob/master/codex/contracts/deployment.nim). In that way we can skip `--marketplace-address` argument or use it to override a hardcoded value.
After node is up and running, and your address has founds, you should be able to [Purchase storage](/learn/using#purchase-storage) using [API](/developers/api).
You also can use [Codex App UI](https://app.codex.storage) for storage purchase.
#### Codex storage node
Codex [storage node](architecture#network-architecture) should be run by storage providers or in case you would like to sell your local storage.
For that, additionally to the [Codex node with marketplace support](#codex-node-with-marketplace-support) we should use `prover` sub-command and required [CLI options](#cli-options).
That sub-command will make Codex to listen for a proof requests on the blockchain and answer them. To compute an answer for the proof request, Codex will use stored data and circuit files generated by the code in the [codex-storage-proofs-circuits](https://github.com/codex-storage/codex-storage-proofs-circuits) repository.
Every [network](/networks/networks) uses its own generated set of the files which are stored in the [codex-contracts-eth](https://github.com/codex-storage/codex-contracts-eth/tree/master/verifier/networks) repository and also uploaded to the CDN. Hash of the files set is also known by the [marketplace smart contract](/learn/architecture#smart-contract).
To download circuit files and make them available to Codex app, we have a stand-alone utility - `cirdl`. It can be [compiled from the sources](/learn/build#circuit-download-tool) or downloaded from the [GitHub release page](https://github.com/codex-storage/nim-codex/releases).
1. Create ethereum key file
<details>
<summary>example</summary>
> [!CAUTION]
> Please use key generation service for demo purpose only.
```shell
response=$(curl -s https://key.codex.storage)
awk -F ': ' '/private/ {print $2}' <<<"${response}" > eth.key
awk -F ': ' '/address/ {print $2}' <<<"${response}" > eth.address
chmod 600 eth.key
```
Show your ethereum address:
```shell
cat eth.address
```
```
0x412665aFAb17768cd9aACE6E00537Cc6D5524Da9
```
</details>
2. To download circuit files, we should pass directory, RPC endpoint and marketplace address to the circuit downloader:
```shell
# Create circuit files folder
mkdir -p datadir/circuits
chmod 700 datadir/circuits
# Download circuit files
cirdl \
datadir/circuits \
https://rpc.testnet.codex.storage \
0xAB03b6a58C5262f530D54146DA2a552B1C0F7648
```
2. Start Codex storage node
```shell
codex \
--data-dir=datadir \
--bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P \
--nat=`curl -s https://ip.codex.storage` \
--disc-port=8090 \
--listen-addrs=/ip4/0.0.0.0/tcp/8070 \
persistence \
--eth-provider=https://rpc.testnet.codex.storage \
--eth-private-key=eth.key \
--marketplace-address=0xAB03b6a58C5262f530D54146DA2a552B1C0F7648 \
prover \
--circuit-dir=datadir/circuits
```
> [!NOTE]
> You would need to pass a bootstrap nodes, blockchain RPC endpoint and marketplace address based on the [network](/networks/networks) you would like to join.
After node is up and running, and your address has founds, you should be able to [sell the storage](/learn/using#create-storage-availability) using [API](/developers/api).
You also can use [Codex App UI](https://app.codex.storage) to sell the storage.
#### Codex bootstrap node
If you need an SPR to bootstrap your own node, go to [network](/networks/networks) and select which network you'd like to join.
Bootstrap nodes are used just to help peers with the initial nodes discovery and we need to run Codex with just some basic options:
```shell
codex \
--data-dir=datadir \
--nat=`curl -s https://ip.codex.storage` \
--nat=any \
--disc-port=8090
```
To get bootstrap node SPR we can use [API](https://api.codex.storage/#tag/Debug/operation/getDebugInfo) call:
To get bootstrap node SPR from your local node, we can use [API](https://api.codex.storage/#tag/Debug/operation/getDebugInfo) call:
```shell
curl -s localhost:8080/api/codex/v1/debug/info | jq -r '.spr'
```
@ -395,15 +268,129 @@ spr:CiUIAhIhApd79-AxPqwRDmu7Pk-berTDtoIoMz0ovKjo85Tz8CUdEgIDARo8CicAJQgCEiECl3v3
That SPR record then can be used then by other peers for initial nodes discovery.
We should keep in mind some important things about SPR record (see [ENR](https://eips.ethereum.org/EIPS/eip-778)):
- It uses node IP (`--nat`), discovery port (`--disc-port`) and private key (`--net-privkey`) for record creation
- It uses nodes public IP, discovery port (`--disc-port`) and private key (`--net-privkey`) for record creation
- Specified data is signed on each run and will be changed but still contain specified node data when decoded
- You can decode it by passing to the Codex node at run and with `--log-level=trace`
For bootstrap node, it is required to forward just discovery port on your Internet router.
### Run as a daemon in Linux
### Run as a service in Linux
This functionality is not supported yet :construction:
We can run Codex as a service via [systemd](https://systemd.io) using following steps
1. Create an user for Codex
```shell
sudo useradd \
--system \
--home-dir /opt/codex \
--shell /usr/sbin/nologin \
codex
```
In case you would like to run commands using a created user, you could do it like following `sudo -u codex ls -la /opt/codex`.
2. Install Codex [using a script](https://github.com/codex-storage/get-codex) or [build from sources](/learn/build)
```shell
# codex with cirdl
curl -s https://get.codex.storage/install.sh | INSTALL_DIR=/usr/local/bin CIRDL=true bash
```
3. Create directories
```shell
sudo mkdir -p /opt/codex/data
sudo mkdir -p /opt/codex/logs
```
4. Create a configuration file
```shell
sudo vi /opt/codex/codex.conf
```
```toml
data-dir = "/opt/codex/data"
listen-addrs = ["/ip4/0.0.0.0/tcp/8070"]
nat = "extip:<Public IP>"
disc-port = 8090
api-port = 8080
bootstrap-node = [
"spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P",
"spr:CiUIAhIhAyUvcPkKoGE7-gh84RmKIPHJPdsX5Ugm_IHVJgF-Mmu_EgIDARo8CicAJQgCEiEDJS9w-QqgYTv6CHzhGYog8ck92xflSCb8gdUmAX4ya78QoemesAYaCwoJBES39Q2RAnVOKkYwRAIgLi3rouyaZFS_Uilx8k99ySdQCP1tsmLR21tDb9p8LcgCIG30o5YnEooQ1n6tgm9fCT7s53k6XlxyeSkD_uIO9mb3",
"spr:CiUIAhIhA6_j28xa--PvvOUxH10wKEm9feXEKJIK3Z9JQ5xXgSD9EgIDARo8CicAJQgCEiEDr-PbzFr74--85TEfXTAoSb195cQokgrdn0lDnFeBIP0QzOGesAYaCwoJBK6Kf1-RAnVEKkcwRQIhAPUH5nQrqG4OW86JQWphdSdnPA98ErQ0hL9OZH9a4e5kAiBBZmUl9KnhSOiDgU3_hvjXrXZXoMxhGuZ92_rk30sNDA",
"spr:CiUIAhIhA7E4DEMer8nUOIUSaNPA4z6x0n9Xaknd28Cfw9S2-cCeEgIDARo8CicAJQgCEiEDsTgMQx6vydQ4hRJo08DjPrHSf1dqSd3bwJ_D1Lb5wJ4Qt_CesAYaCwoJBEDhWZORAnVYKkYwRAIgFNzhnftocLlVHJl1onuhbSUM7MysXPV6dawHAA0DZNsCIDRVu9gnPTH5UkcRXLtt7MLHCo4-DL-RCMyTcMxYBXL0",
"spr:CiUIAhIhAzZn3JmJab46BNjadVnLNQKbhnN3eYxwqpteKYY32SbOEgIDARo8CicAJQgCEiEDNmfcmYlpvjoE2Np1Wcs1ApuGc3d5jHCqm14phjfZJs4QrvWesAYaCwoJBKpA-TaRAnViKkcwRQIhANuMmZDD2c25xzTbKSirEpkZYoxbq-FU_lpI0K0e4mIVAiBfQX4yR47h1LCnHznXgDs6xx5DLO5q3lUcicqUeaqGeg",
"spr:CiUIAhIhAuN-P1D0HrJdwBmrRlZZzg6dqllRNNcQyMDUMuRtg3paEgIDARpJCicAJQgCEiEC434_UPQesl3AGatGVlnODp2qWVE01xDIwNQy5G2DeloQm_L2vQYaCwoJBI_0zSiRAnVsGgsKCQSP9M0okQJ1bCpHMEUCIQDgEVjUp1RJGb59eRPs7RPYMSGAI_fo1yv70iBtnTqefQIgVoXszc87EGFVO3aaqorEYZ21OGRko5ho_Pybdyqa6AI",
"spr:CiUIAhIhAsi_hgxFppWjHiKRwnYPX_qkB28dLtwK9c7apnlBanFuEgIDARpJCicAJQgCEiECyL-GDEWmlaMeIpHCdg9f-qQHbx0u3Ar1ztqmeUFqcW4Q2O32vQYaCwoJBNEmoCiRAnV2GgsKCQTRJqAokQJ1dipHMEUCIQDpC1isFfdRqNmZBfz9IGoEq7etlypB6N1-9Z5zhvmRMAIgIOsleOPr5Ra_Nk7BXmXGhe-YlLosH9jo83JtfWCy3-o"
]
storage-quota = "8gb"
block-ttl = "24h"
log-level = "info"
```
Make sure to use bootstrap nodes for the [network](/networks/networks) you would like to join, update `nat` variable with a node Public IP and adjust other settings by your needs.
5. Change folders ownership and permissions
```shell
sudo chown -R codex:codex /opt/codex
```
6. Create systemd unit file
```shell
sudo vi /lib/systemd/system/codex.service
```
```shell
[Unit]
Description=Codex service
Documentation=https://docs.codex.storage
After=local-fs.target network-online.target
[Service]
MemorySwapMax=0
TimeoutStartSec=infinity
Type=exec
User=codex
Group=codex
StateDirectory=codex
ExecStart=/usr/local/bin/codex --config-file="/opt/codex/codex.conf"
Restart=always
RestartSec=3
StandardOutput=append:/opt/codex/logs/codex.log
StandardError=append:/opt/codex/logs/codex.log
[Install]
WantedBy=multi-user.target
```
Check `man systemd`, `man systemd.service` and `man systemd.directives` for additional details.
7. Enable and start Codex service
```shell
sudo systemctl enable codex
sudo systemctl start codex
```
8. Check service status
```shell
sudo systemctl status codex
```
9. Enable logs rotation using logrotate
```shell
sudo vi /etc/logrotate.d/codex
```
```logrotate
/opt/codex/logs/*.log {
daily
missingok
rotate 5
copytruncate
nocreate
nomail
dateext
dateyesterday
}
```
1. Check the logs
```shell
tail -f /opt/codex/logs/codex.log
```
### Run as a service in Windows
@ -420,8 +407,10 @@ We also ship Codex in Docker containers, which can be run on `amd64` and `arm64`
- `ENV_PATH` - path to the file, in form `env=value` which will be sourced and available for Codex at run. That is useful for Kubernetes Pods configuration.
- `NAT_IP_AUTO` - when set to `true`, will set `CODEX_NAT` variable with container internal IP address. It also is useful for Kubernetes Pods configuration, when we perform automated tests.
- `NAT_PUBLIC_IP_AUTO` - used to set `CODEX_NAT` to public IP address using lookup services, like [ip.codex.storage](https://ip.codex.storage). Can be used for Docker/Kubernetes to set public IP in auto mode.
- `ETH_PRIVATE_KEY` - can be used to pass ethereum private key, which will be saved and passed as a value of the `CODEX_ETH_PRIVATE_KEY` variable. It should be considered as unsafe option and used for testing purposes only.
- When we set `prover` sub-command, entrypoint will run `cirdl` tool to download ceremony files, required by [Codex storage node](#codex-storage-node).
- `BOOTSTRAP_NODE_URL` - Codex node API URL in form of `http://bootstrap:8080`, to be used to get it's SPR as a bootstrap node. That is useful for Docker and Kubernetes configuration.
- `NETWORK` - is a helper variable to simply a specific network join. It helps to automate `BOOTSTRAP_NODE_FROM_URL` variable.
- `BOOTSTRAP_NODE_FROM_URL` - can be used to pass SPR nodes from an URL like [spr.codex.storage/testnet](https://spr.codex.storage/testnet).
#### Docker network
@ -440,36 +429,11 @@ In that case, we should consider the following solutions:
And we basically can use same options we [used for binary](#using-binary) and additionally it is required to mount volumes and map the ports.
[Codex storage node](#codex-storage-node)
1. Create ethereum key file
<details>
<summary>example</summary>
> [!CAUTION]
> Please use key generation service for demo purpose only.
```shell
response=$(curl -s https://key.codex.storage)
awk -F ': ' '/private/ {print $2}' <<<"${response}" > eth.key
awk -F ': ' '/address/ {print $2}' <<<"${response}" > eth.address
chmod 600 eth.key
```
Show your ethereum address:
```shell
cat eth.address
```
```
0x412665aFAb17768cd9aACE6E00537Cc6D5524Da9
```
</details>
2. Run Codex:
1. Run Codex:
```shell
docker run \
--rm \
-v $PWD/datadir:/datadir \
-v $PWD/eth.key:/opt/eth.key \
-p 8070:8070 \
-p 8080:8080 \
-p 8090:8090/udp \
@ -477,52 +441,22 @@ docker run \
codex \
--data-dir=/datadir \
--bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P \
--nat=`curl -s https://ip.codex.storage` \
--nat=any \
--disc-port=8090 \
--listen-addrs=/ip4/0.0.0.0/tcp/8070 \
--api-cors-origin="*" \
--api-bindaddr=0.0.0.0 \
--api-port=8080 \
persistence \
--eth-provider=https://rpc.testnet.codex.storage \
--eth-private-key=/opt/eth.key \
--marketplace-address=0xAB03b6a58C5262f530D54146DA2a552B1C0F7648 \
prover \
--circuit-dir=/datadir/circuits
--api-port=8080
```
> [!NOTE]
> You would need to pass a bootstrap nodes, blockchain RPC endpoint and marketplace address based on the [network](/networks/networks) you would like to join.
> You would need to pass a bootstrap nodes based on the [network](/networks/networks) you would like to join.
### Using Docker Compose
For Docker Compose, it is more suitable to use [environment variables](#environment-variables) for Codex configuration and we can reuse commands from example above, for Docker.
[Codex storage node](#codex-storage-node)
1. Create ethereum key file
<details>
<summary>example</summary>
> [!CAUTION]
> Please use key generation service for demo purpose only.
```shell
response=$(curl -s https://key.codex.storage)
awk -F ': ' '/private/ {print $2}' <<<"${response}" > eth.key
awk -F ': ' '/address/ {print $2}' <<<"${response}" > eth.address
chmod 600 eth.key
```
Show your ethereum address:
```shell
cat eth.address
```
```
0x412665aFAb17768cd9aACE6E00537Cc6D5524Da9
```
</details>
2. Create `docker-compose.yaml` file:
1. Create `docker-compose.yaml` file:
```yaml
services:
codex:
@ -530,15 +464,13 @@ For Docker Compose, it is more suitable to use [environment variables](#environm
container_name: codex
command:
- codex
- persistence
- prover
- --bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P
- --bootstrap-node=spr:CiUIAhIhAyUvcPkKoGE7-gh84RmKIPHJPdsX5Ugm_IHVJgF-Mmu_EgIDARo8CicAJQgCEiEDJS9w-QqgYTv6CHzhGYog8ck92xflSCb8gdUmAX4ya78QoemesAYaCwoJBES39Q2RAnVOKkYwRAIgLi3rouyaZFS_Uilx8k99ySdQCP1tsmLR21tDb9p8LcgCIG30o5YnEooQ1n6tgm9fCT7s53k6XlxyeSkD_uIO9mb3
- --bootstrap-node=spr:CiUIAhIhA6_j28xa--PvvOUxH10wKEm9feXEKJIK3Z9JQ5xXgSD9EgIDARo8CicAJQgCEiEDr-PbzFr74--85TEfXTAoSb195cQokgrdn0lDnFeBIP0QzOGesAYaCwoJBK6Kf1-RAnVEKkcwRQIhAPUH5nQrqG4OW86JQWphdSdnPA98ErQ0hL9OZH9a4e5kAiBBZmUl9KnhSOiDgU3_hvjXrXZXoMxhGuZ92_rk30sNDA
- --bootstrap-node=spr:CiUIAhIhA7E4DEMer8nUOIUSaNPA4z6x0n9Xaknd28Cfw9S2-cCeEgIDARo8CicAJQgCEiEDsTgMQx6vydQ4hRJo08DjPrHSf1dqSd3bwJ_D1Lb5wJ4Qt_CesAYaCwoJBEDhWZORAnVYKkYwRAIgFNzhnftocLlVHJl1onuhbSUM7MysXPV6dawHAA0DZNsCIDRVu9gnPTH5UkcRXLtt7MLHCo4-DL-RCMyTcMxYBXL0
- --bootstrap-node=spr:CiUIAhIhAzZn3JmJab46BNjadVnLNQKbhnN3eYxwqpteKYY32SbOEgIDARo8CicAJQgCEiEDNmfcmYlpvjoE2Np1Wcs1ApuGc3d5jHCqm14phjfZJs4QrvWesAYaCwoJBKpA-TaRAnViKkcwRQIhANuMmZDD2c25xzTbKSirEpkZYoxbq-FU_lpI0K0e4mIVAiBfQX4yR47h1LCnHznXgDs6xx5DLO5q3lUcicqUeaqGeg
- --bootstrap-node=spr:CiUIAhIhAgybmRwboqDdUJjeZrzh43sn5mp8jt6ENIb08tLn4x01EgIDARo8CicAJQgCEiECDJuZHBuioN1QmN5mvOHjeyfmanyO3oQ0hvTy0ufjHTUQh4ifsAYaCwoJBI_0zSiRAnVsKkcwRQIhAJCb_z0E3RsnQrEePdJzMSQrmn_ooHv6mbw1DOh5IbVNAiBbBJrWR8eBV6ftzMd6ofa5khNA2h88OBhMqHCIzSjCeA
- --bootstrap-node=spr:CiUIAhIhAntGLadpfuBCD9XXfiN_43-V3L5VWgFCXxg4a8uhDdnYEgIDARo8CicAJQgCEiECe0Ytp2l-4EIP1dd-I3_jf5XcvlVaAUJfGDhry6EN2dgQsIufsAYaCwoJBNEmoCiRAnV2KkYwRAIgXO3bzd5VF8jLZG8r7dcLJ_FnQBYp1BcxrOvovEa40acCIDhQ14eJRoPwJ6GKgqOkXdaFAsoszl-HIRzYcXKeb7D9
- --bootstrap-node=spr:CiUIAhIhAuN-P1D0HrJdwBmrRlZZzg6dqllRNNcQyMDUMuRtg3paEgIDARpJCicAJQgCEiEC434_UPQesl3AGatGVlnODp2qWVE01xDIwNQy5G2DeloQm_L2vQYaCwoJBI_0zSiRAnVsGgsKCQSP9M0okQJ1bCpHMEUCIQDgEVjUp1RJGb59eRPs7RPYMSGAI_fo1yv70iBtnTqefQIgVoXszc87EGFVO3aaqorEYZ21OGRko5ho_Pybdyqa6AI
- --bootstrap-node=spr:CiUIAhIhAsi_hgxFppWjHiKRwnYPX_qkB28dLtwK9c7apnlBanFuEgIDARpJCicAJQgCEiECyL-GDEWmlaMeIpHCdg9f-qQHbx0u3Ar1ztqmeUFqcW4Q2O32vQYaCwoJBNEmoCiRAnV2GgsKCQTRJqAokQJ1dipHMEUCIQDpC1isFfdRqNmZBfz9IGoEq7etlypB6N1-9Z5zhvmRMAIgIOsleOPr5Ra_Nk7BXmXGhe-YlLosH9jo83JtfWCy3-o
environment:
- CODEX_DATA_DIR=/datadir
- NAT_PUBLIC_IP_AUTO=https://ip.codex.storage
@ -547,17 +479,12 @@ For Docker Compose, it is more suitable to use [environment variables](#environm
- CODEX_API_CORS_ORIGIN="*"
- CODEX_API_PORT=8080
- CODEX_API_BINDADDR=0.0.0.0
- CODEX_ETH_PROVIDER=https://rpc.testnet.codex.storage
- CODEX_ETH_PRIVATE_KEY=/opt/eth.key
- CODEX_MARKETPLACE_ADDRESS=0xAB03b6a58C5262f530D54146DA2a552B1C0F7648
- CODEX_CIRCUIT_DIR=/datadir/circuits
ports:
- 8080:8080/tcp # API
- 8090:8090/udp # Discovery
- 8070:8070/tcp # Transport
volumes:
- ./datadir:/datadir
- ./eth.key:/opt/eth.key
logging:
driver: json-file
options:
@ -571,16 +498,66 @@ For Docker Compose, it is more suitable to use [environment variables](#environm
```
> [!NOTE]
> You would need to pass a bootstrap nodes, blockchain RPC endpoint and marketplace address based on the [network](/networks/networks) you would like to join.
> You would need to pass a bootstrap nodes based on the [network](/networks/networks) you would like to join.
### On Kubernetes
Helm chart code is available in [helm-charts](https://github.com/codex-storage/helm-charts) repository, but chart was not published yet.
## How-tos
### NAT Configuration
Use the `--nat` CLI flag to specify how your codex node should handle NAT traversal. Below are the available options:
**any**(default): This option will automatically try to detect your public IP by checking the routing table or using UPnP/PMP NAT traversal techniques. If successful, it will use the detected public IP and port for the announce address.
**upnp**: This option exclusively uses [UPnP](https://en.wikipedia.org/wiki/Universal_Plug_and_Play) to detect the public IP and create a port mapping entry, if your device supports UPnP.
**pmp**: This option uses only [NAT-PMP](https://en.wikipedia.org/wiki/NAT_Port_Mapping_Protocol) to detect the public IP and create a port mapping entry, if your device supports NAT-PMP.
**extIP:[Your Public IP]**:Use this option if you want to manually specify an external IP address and port for the announce address. When selecting this option, you'll need to configure **port forwarding** on your router to ensure that incoming traffic is directed to the correct internal IP and port.
### Port Forwarding
If you're running on a private network, you'll need to set up port forwarding to ensure seamless communication between the codex node and its peers. It's also recommended to configure appropriate firewall rules for TCP and UDP traffic.
While the specific steps required vary based on your router, they can be summarised as follows:
1. Find your public IP address by either visiting [ip-codex](https://ip.codex.storage/) or running `curl ip.codex.storage`
2. Identify your [private](#determine-your-private-ip) IP address
3. Access your router's settings by entering its IP address (typically [http://192.168.1.1](http://192.168.1.1/)) in your web browser
4. Sign in with administrator credentials and locate the port forwarding settings
5. Set up the discovery port forwarding rule with these settings:
- External Port: 8090
- Internal Port: 8090
- Protocol: UDP
- IP Address: Your device's private IP address
6. Set up the libp2p port forwarding rule with these settings:
- External Port: 8070
- Internal Port: 8070
- Protocol: TCP
- IP Address: Your device's private IP address
#### Determine your private IP
To determine your private IP address, run the appropriate command for your OS:
**Linux**:
```shell
ip addr show | grep "inet " | grep -v 127.0.0.1
```
**Windows**:
```shell
ipconfig | findstr /i "IPv4 Address"
```
**MacOs**:
```shell
ifconfig | grep "inet " | grep -v 127.0.0.1
```
## Known issues
[^multivalue-env-var]: Environment variables like `CODEX_BOOTSTRAP_NODE` and `CODEX_LISTEN_ADDRS` does not support multiple values. Please check [[Feature request] Support multiple SPR records via environment variable #525](https://github.com/codex-storage/nim-codex/issues/525), for more information.
[^sub-commands]: Sub-commands `persistence` and `persistence prover` can't be set via environment variables.
[^data-dir]: We should set data-dir explicitly when we use GitHub releases - [[BUG] Change codex default datadir from compile-time to run-time #923](https://github.com/codex-storage/nim-codex/issues/923)
[^port-forwarding]: [NAT traversal #753](https://github.com/codex-storage/nim-codex/issues/753) is not implemented yet and we would need to setup port-forwarding for discovery and transport protocols.
[^eth-account]: Please ignore `--eth-account` CLI option - [Drop support for --eth-account #727](https://github.com/codex-storage/nim-codex/issues/727).

7
index.md
View File

@ -13,19 +13,12 @@ hero:
- theme: alt
text: Quick start
link: /learn/quick-start
- theme: alt
text: Join Codex Testnet
link: /networks/testnet
features:
- title: Learn
details: Get more information about Codex
link: /learn/what-is-codex
icon: 📚
- title: Networks
details: Participate in Codex network as a Storage operator or Buyer
link: /networks/networks
icon: 🚦
- title: Developers
details: Build Codex
link: /developers/api

219
ko/learn/run.md
View File

@ -164,19 +164,8 @@ codex --config-file=/path/to/your/config.toml
Please check [Run as a service in Linux](#run-as-a-service-in-linux) for a full example of configuration file.
## Auto-discovery
Codex support marketplace contract auto-discovery based on the chain id, this mapping is done in the [source code](https://github.com/codex-storage/nim-codex/blob/master/codex/contracts/deployment.nim). In that way we can skip `--marketplace-address` argument and use it just to override a discovered value.
## Run
Basically, we can run Codex in three different modes:
- [Codex node](#codex-node) - useful for local testing/development and basic/files sharing.
- [Codex node with marketplace support](#codex-node-with-marketplace-support) - you can share files and buy the storage, this is the main mode and should be used by the end users.
- [Codex storage node](#codex-storage-node) - should be used by storage providers or if you would like to sell your local storage.
We also will touch in some words [Codex bootstrap node](#codex-bootstrap-node).
### Using binary
#### Codex node
@ -212,6 +201,7 @@ codex \
> [!TIP]
> We can set public IP using curl and IP lookup service, like [ip.codex.storage](https://ip.codex.storage).
> In this case, you can override the IP by which your node annouces itself like this: `--nat=extip:1.2.3.4`
After that, node will announce itself using your public IP, default UDP ([discovery](https://docs.libp2p.io/concepts/discovery-routing/overview/)) and dynamic TCP port ([data transfer](https://docs.libp2p.io/concepts/transports/overview/)), which can be adjusted in the following way:
```shell
@ -255,133 +245,10 @@ After node is up and running and port-forwarding configurations was done, we sho
You also can use [Codex App UI](https://app.codex.storage) for files upload/download.
And to be able to purchase a storage, we should run [Codex node with marketplace support](#codex-node-with-marketplace-support).
#### Codex node with marketplace support
[Marketplace](/learn/architecture.md#marketplace-architecture) support permits to purchase the storage in Codex network. Basically, we should add just a `persistence` sub-command and required [CLI options](#cli-options) to the [previous run](#codex-node).
> [!NOTE]
> Please ignore `--eth-account` CLI option, as it is obsolete [^eth-account].
1. For a daily use, we should consider to run a local blockchain node based on the [network](/networks/networks) you would like to join. That process is described in the [Join Codex Testnet](/networks/testnet) guide, but for a quick start we can use a public RPC endpoint.
2. Create a file with ethereum private key and set a proper permissions:
> [!CAUTION]
> Please use key generation service for demo purpose only.
```shell
response=$(curl -s https://key.codex.storage)
awk -F ': ' '/private/ {print $2}' <<<"${response}" > eth.key
awk -F ': ' '/address/ {print $2}' <<<"${response}" > eth.address
chmod 600 eth.key
```
Show your ethereum address:
```shell
cat eth.address
```
```
0x412665aFAb17768cd9aACE6E00537Cc6D5524Da9
```
3. Fill-up your ethereum address with ETH and Tokens based on the the [network](/networks/networks) you would like to join.
4. Specify bootstrap nodes based on the [network](/networks/networks) you would like to join.
5. Run the node:
```shell
codex \
--data-dir=datadir \
--bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P \
--nat=any \
--disc-port=8090 \
--listen-addrs=/ip4/0.0.0.0/tcp/8070 \
--api-cors-origin="*" \
persistence \
--eth-provider=https://rpc.testnet.codex.storage \
--eth-private-key=eth.key
```
> [!NOTE]
> We could skip `--marketplace-address` argument and rely on marketplace contract [Auto-discovery](#auto-discovery).
After node is up and running, and your address has founds, you should be able to [Purchase storage](/learn/using#purchase-storage) using [API](/developers/api).
You also can use [Codex App UI](https://app.codex.storage) for storage purchase.
#### Codex storage node
Codex [storage node](architecture#network-architecture) should be run by storage providers or in case you would like to sell your local storage.
For that, additionally to the [Codex node with marketplace support](#codex-node-with-marketplace-support) we should use `prover` sub-command and required [CLI options](#cli-options).
That sub-command will make Codex to listen for a proof requests on the blockchain and answer them. To compute an answer for the proof request, Codex will use stored data and circuit files generated by the code in the [codex-storage-proofs-circuits](https://github.com/codex-storage/codex-storage-proofs-circuits) repository.
Every [network](/networks/networks) uses its own generated set of the files which are stored in the [codex-contracts-eth](https://github.com/codex-storage/codex-contracts-eth/tree/master/verifier/networks) repository and also uploaded to the CDN. Hash of the files set is also known by the [marketplace smart contract](/learn/architecture#smart-contract).
To download circuit files and make them available to Codex app, we have a stand-alone utility - `cirdl`. It can be [compiled from the sources](/learn/build#circuit-download-tool) or downloaded from the [GitHub release page](https://github.com/codex-storage/nim-codex/releases).
1. Create ethereum key file
<details>
<summary>example</summary>
> [!CAUTION]
> Please use key generation service for demo purpose only.
```shell
response=$(curl -s https://key.codex.storage)
awk -F ': ' '/private/ {print $2}' <<<"${response}" > eth.key
awk -F ': ' '/address/ {print $2}' <<<"${response}" > eth.address
chmod 600 eth.key
```
Show your ethereum address:
```shell
cat eth.address
```
```
0x412665aFAb17768cd9aACE6E00537Cc6D5524Da9
```
</details>
2. To download circuit files, we should pass directory, RPC endpoint and optionally marketplace address to the circuit downloader:
```shell
# Create circuit files folder
mkdir -p datadir/circuits
chmod 700 datadir/circuits
# Download circuit files
cirdl \
datadir/circuits \
https://rpc.testnet.codex.storage
```
2. Start Codex storage node
```shell
codex \
--data-dir=datadir \
--bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P \
--nat=any \
--disc-port=8090 \
--listen-addrs=/ip4/0.0.0.0/tcp/8070 \
persistence \
--eth-provider=https://rpc.testnet.codex.storage \
--eth-private-key=eth.key \
prover \
--circuit-dir=datadir/circuits
```
> [!NOTE]
> You would need to pass a bootstrap nodes and blockchain RPC endpoint based on the [network](/networks/networks) you would like to join.
> [!NOTE]
> We could skip `--marketplace-address` argument and rely on marketplace contract [Auto-discovery](#auto-discovery).
After node is up and running, and your address has founds, you should be able to [sell the storage](/learn/using#create-storage-availability) using [API](/developers/api).
You also can use [Codex App UI](https://app.codex.storage) to sell the storage.
#### Codex bootstrap node
If you need an SPR to bootstrap your own node, go to [network](/networks/networks) and select which network you'd like to join.
Bootstrap nodes are used just to help peers with the initial nodes discovery and we need to run Codex with just some basic options:
```shell
codex \
@ -390,7 +257,7 @@ codex \
--disc-port=8090
```
To get bootstrap node SPR we can use [API](https://api.codex.storage/#tag/Debug/operation/getDebugInfo) call:
To get bootstrap node SPR from your local node, we can use [API](https://api.codex.storage/#tag/Debug/operation/getDebugInfo) call:
```shell
curl -s localhost:8080/api/codex/v1/debug/info | jq -r '.spr'
```
@ -540,12 +407,10 @@ We also ship Codex in Docker containers, which can be run on `amd64` and `arm64`
- `ENV_PATH` - path to the file, in form `env=value` which will be sourced and available for Codex at run. That is useful for Kubernetes Pods configuration.
- `NAT_IP_AUTO` - when set to `true`, will set `CODEX_NAT` variable with container internal IP address. It also is useful for Kubernetes Pods configuration, when we perform automated tests.
- `NAT_PUBLIC_IP_AUTO` - used to set `CODEX_NAT` to public IP address using lookup services, like [ip.codex.storage](https://ip.codex.storage). Can be used for Docker/Kubernetes to set public IP in auto mode.
- `ETH_PRIVATE_KEY` - can be used to pass ethereum private key, which will be saved and passed as a value of the `CODEX_ETH_PRIVATE_KEY` variable. It should be considered as unsafe option and used for testing purposes only.
- When we set `prover` sub-command, entrypoint will run `cirdl` tool to download ceremony files, required by [Codex storage node](#codex-storage-node).
- `BOOTSTRAP_NODE_URL` - Codex node API URL in form of `http://bootstrap:8080`, to be used to get it's SPR as a bootstrap node. That is useful for Docker and Kubernetes configuration.
- `NETWORK` - is a helper variable to simply a specific network join. It helps to automate `BOOTSTRAP_NODE_FROM_URL` variable.
- `BOOTSTRAP_NODE_FROM_URL` - can be used to pass SPR nodes from an URL like [spr.codex.storage/testnet](https://spr.codex.storage/testnet).
- `MARKETPLACE_ADDRESS_FROM_URL` - can be used to pass a Marketplace contract address from an URL like [marketplace.codex.storage/codex-testnet/latest](https://marketplace.codex.storage/codex-testnet/latest).
#### Docker network
@ -564,36 +429,11 @@ In that case, we should consider the following solutions:
And we basically can use same options we [used for binary](#using-binary) and additionally it is required to mount volumes and map the ports.
[Codex storage node](#codex-storage-node)
1. Create ethereum key file
<details>
<summary>example</summary>
> [!CAUTION]
> Please use key generation service for demo purpose only.
```shell
response=$(curl -s https://key.codex.storage)
awk -F ': ' '/private/ {print $2}' <<<"${response}" > eth.key
awk -F ': ' '/address/ {print $2}' <<<"${response}" > eth.address
chmod 600 eth.key
```
Show your ethereum address:
```shell
cat eth.address
```
```
0x412665aFAb17768cd9aACE6E00537Cc6D5524Da9
```
</details>
2. Run Codex:
1. Run Codex:
```shell
docker run \
--rm \
-v $PWD/datadir:/datadir \
-v $PWD/eth.key:/opt/eth.key \
-p 8070:8070 \
-p 8080:8080 \
-p 8090:8090/udp \
@ -606,49 +446,17 @@ docker run \
--listen-addrs=/ip4/0.0.0.0/tcp/8070 \
--api-cors-origin="*" \
--api-bindaddr=0.0.0.0 \
--api-port=8080 \
persistence \
--eth-provider=https://rpc.testnet.codex.storage \
--eth-private-key=/opt/eth.key \
prover \
--circuit-dir=/datadir/circuits
--api-port=8080
```
> [!NOTE]
> You would need to pass a bootstrap nodes and blockchain RPC endpoint based on the [network](/networks/networks) you would like to join.
> [!NOTE]
> We could skip `--marketplace-address` argument and rely on marketplace contract [Auto-discovery](#auto-discovery).
> You would need to pass a bootstrap nodes based on the [network](/networks/networks) you would like to join.
### Using Docker Compose
For Docker Compose, it is more suitable to use [environment variables](#environment-variables) for Codex configuration and we can reuse commands from example above, for Docker.
[Codex storage node](#codex-storage-node)
1. Create ethereum key file
<details>
<summary>example</summary>
> [!CAUTION]
> Please use key generation service for demo purpose only.
```shell
response=$(curl -s https://key.codex.storage)
awk -F ': ' '/private/ {print $2}' <<<"${response}" > eth.key
awk -F ': ' '/address/ {print $2}' <<<"${response}" > eth.address
chmod 600 eth.key
```
Show your ethereum address:
```shell
cat eth.address
```
```
0x412665aFAb17768cd9aACE6E00537Cc6D5524Da9
```
</details>
2. Create `docker-compose.yaml` file:
1. Create `docker-compose.yaml` file:
```yaml
services:
codex:
@ -656,8 +464,6 @@ For Docker Compose, it is more suitable to use [environment variables](#environm
container_name: codex
command:
- codex
- persistence
- prover
- --bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P
- --bootstrap-node=spr:CiUIAhIhAyUvcPkKoGE7-gh84RmKIPHJPdsX5Ugm_IHVJgF-Mmu_EgIDARo8CicAJQgCEiEDJS9w-QqgYTv6CHzhGYog8ck92xflSCb8gdUmAX4ya78QoemesAYaCwoJBES39Q2RAnVOKkYwRAIgLi3rouyaZFS_Uilx8k99ySdQCP1tsmLR21tDb9p8LcgCIG30o5YnEooQ1n6tgm9fCT7s53k6XlxyeSkD_uIO9mb3
- --bootstrap-node=spr:CiUIAhIhA6_j28xa--PvvOUxH10wKEm9feXEKJIK3Z9JQ5xXgSD9EgIDARo8CicAJQgCEiEDr-PbzFr74--85TEfXTAoSb195cQokgrdn0lDnFeBIP0QzOGesAYaCwoJBK6Kf1-RAnVEKkcwRQIhAPUH5nQrqG4OW86JQWphdSdnPA98ErQ0hL9OZH9a4e5kAiBBZmUl9KnhSOiDgU3_hvjXrXZXoMxhGuZ92_rk30sNDA
@ -673,16 +479,12 @@ For Docker Compose, it is more suitable to use [environment variables](#environm
- CODEX_API_CORS_ORIGIN="*"
- CODEX_API_PORT=8080
- CODEX_API_BINDADDR=0.0.0.0
- CODEX_ETH_PROVIDER=https://rpc.testnet.codex.storage
- CODEX_ETH_PRIVATE_KEY=/opt/eth.key
- CODEX_CIRCUIT_DIR=/datadir/circuits
ports:
- 8080:8080/tcp # API
- 8090:8090/udp # Discovery
- 8070:8070/tcp # Transport
volumes:
- ./datadir:/datadir
- ./eth.key:/opt/eth.key
logging:
driver: json-file
options:
@ -696,10 +498,7 @@ For Docker Compose, it is more suitable to use [environment variables](#environm
```
> [!NOTE]
> You would need to pass a bootstrap nodes and blockchain RPC endpoint based on the [network](/networks/networks) you would like to join.
> [!NOTE]
> We could skip `CODEX_MARKETPLACE_ADDRESS` variable and rely on marketplace contract [Auto-discovery](#auto-discovery).
> You would need to pass a bootstrap nodes based on the [network](/networks/networks) you would like to join.
### On Kubernetes

23
learn/local-two-client-test.md
View File

@ -8,18 +8,6 @@ Make sure you have [built the client](/learn/build) or obtained [compiled binary
## Steps
### 0. Setup blockchain node (optional)
You need to have installed NodeJS and npm in order to spinup a local blockchain node.
Go to directory `vendor/codex-contracts-eth` and run these two commands:
```
npm ci
npm start
```
This will launch a local Ganache blockchain.
### 1. Launch Node #1
Open a terminal and run:
@ -40,23 +28,12 @@ Open a terminal and run:
--listen-addrs="/ip4/127.0.0.1/tcp/8070"
```
Optionally, if you want to use the Marketplace blockchain functionality, you need to also include these flags: `--persistence --eth-account=<account>`, where `account` can be one following:
- `0x70997970C51812dc3A010C7d01b50e0d17dc79C8`
- `0x3C44CdDdB6a900fa2b585dd299e03d12FA4293BC`
- `0x90F79bf6EB2c4f870365E785982E1f101E93b906`
- `0x15d34AAf54267DB7D7c367839AAf71A00a2C6A65`
**For each node use a different account!**
| Argument | Description |
|----------------|-----------------------------------------------------------------------|
| `data-dir` | We specify a relative path where the node will store its data. |
| `listen-addrs` | Multiaddress where the node will accept connections from other nodes. |
| `api-port` | Port on localhost where the node will expose its API. |
| `disc-port` | Port the node will use for its discovery service. |
| `persistence` | Enables Marketplace functionality. Requires a blockchain connection. |
| `eth-account` | Defines which blockchain account the node should use. |
Codex uses sane defaults for most of its arguments. Here we specify some explicitly for the purpose of this walk-through.

14
learn/quick-start.md
View File

@ -74,6 +74,9 @@ Please follow the steps for your OS from the list:
We may [run Codex in different modes](/learn/run#run), and for a quick start we will run [Codex node](/learn/run#codex-node), to be able to share files in the network.
0. Obtain an SPR of the Codex network you want to join.
Go to [networks](/networks/networks)
1. Run Codex
**Linux/macOS**
@ -84,7 +87,7 @@ We may [run Codex in different modes](/learn/run#run), and for a quick start we
--listen-addrs=/ip4/0.0.0.0/tcp/8070 \
--nat=any \
--api-cors-origin="*" \
--bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P
--bootstrap-node=<SPR HERE>
```
**Windows**
@ -118,12 +121,9 @@ We may [run Codex in different modes](/learn/run#run), and for a quick start we
--listen-addrs=/ip4/0.0.0.0/tcp/8070 ^
--nat=any ^
--api-cors-origin="*" ^
--bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P
--bootstrap-node=<SPR HERE>
```
> [!TIP]
> In the example above we use [Codex Testnet](/networks/testnet#bootstrap-nodes) bootstrap nodes and thus we join Testnet. If you would like to join a different network, please use [appropriate value](/networks/networks).
2. Configure port-forwarding for the TCP/UDP ports on your Internet router
| Protocol | Service | Port |
| -------- | --------- | ------ |
@ -141,7 +141,3 @@ Also, you can interact with Codex using [Codex API](/developers/api) and for a w
## Stay in touch
Want to stay up-date, or looking for further assistance? Try our [discord-server](https://discord.gg/codex-storage).
Ready to explore Codex functionality? Please [Join Codex Testnet](/networks/testnet).
If you want to run Codex locally without joining the Testnet, consider trying the [Codex Two-Client Test](/learn/local-two-client-test) or the [Running a Local Codex Network with Marketplace Support](/learn/local-marketplace).

219
learn/run.md
View File

@ -164,19 +164,8 @@ codex --config-file=/path/to/your/config.toml
Please check [Run as a service in Linux](#run-as-a-service-in-linux) for a full example of configuration file.
## Auto-discovery
Codex support marketplace contract auto-discovery based on the chain id, this mapping is done in the [source code](https://github.com/codex-storage/nim-codex/blob/master/codex/contracts/deployment.nim). In that way we can skip `--marketplace-address` argument and use it just to override a discovered value.
## Run
Basically, we can run Codex in three different modes:
- [Codex node](#codex-node) - useful for local testing/development and basic/files sharing.
- [Codex node with marketplace support](#codex-node-with-marketplace-support) - you can share files and buy the storage, this is the main mode and should be used by the end users.
- [Codex storage node](#codex-storage-node) - should be used by storage providers or if you would like to sell your local storage.
We also will touch in some words [Codex bootstrap node](#codex-bootstrap-node).
### Using binary
#### Codex node
@ -212,6 +201,7 @@ codex \
> [!TIP]
> We can set public IP using curl and IP lookup service, like [ip.codex.storage](https://ip.codex.storage).
> In this case, you can override the IP by which your node annouces itself like this: `--nat=extip:1.2.3.4`
After that, node will announce itself using your public IP, default UDP ([discovery](https://docs.libp2p.io/concepts/discovery-routing/overview/)) and dynamic TCP port ([data transfer](https://docs.libp2p.io/concepts/transports/overview/)), which can be adjusted in the following way:
```shell
@ -255,133 +245,10 @@ After node is up and running and port-forwarding configurations was done, we sho
You also can use [Codex App UI](https://app.codex.storage) for files upload/download.
And to be able to purchase a storage, we should run [Codex node with marketplace support](#codex-node-with-marketplace-support).
#### Codex node with marketplace support
[Marketplace](/learn/architecture.md#marketplace-architecture) support permits to purchase the storage in Codex network. Basically, we should add just a `persistence` sub-command and required [CLI options](#cli-options) to the [previous run](#codex-node).
> [!NOTE]
> Please ignore `--eth-account` CLI option, as it is obsolete [^eth-account].
1. For a daily use, we should consider to run a local blockchain node based on the [network](/networks/networks) you would like to join. That process is described in the [Join Codex Testnet](/networks/testnet) guide, but for a quick start we can use a public RPC endpoint.
2. Create a file with ethereum private key and set a proper permissions:
> [!CAUTION]
> Please use key generation service for demo purpose only.
```shell
response=$(curl -s https://key.codex.storage)
awk -F ': ' '/private/ {print $2}' <<<"${response}" > eth.key
awk -F ': ' '/address/ {print $2}' <<<"${response}" > eth.address
chmod 600 eth.key
```
Show your ethereum address:
```shell
cat eth.address
```
```
0x412665aFAb17768cd9aACE6E00537Cc6D5524Da9
```
3. Fill-up your ethereum address with ETH and Tokens based on the the [network](/networks/networks) you would like to join.
4. Specify bootstrap nodes based on the [network](/networks/networks) you would like to join.
5. Run the node:
```shell
codex \
--data-dir=datadir \
--bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P \
--nat=any \
--disc-port=8090 \
--listen-addrs=/ip4/0.0.0.0/tcp/8070 \
--api-cors-origin="*" \
persistence \
--eth-provider=https://rpc.testnet.codex.storage \
--eth-private-key=eth.key
```
> [!NOTE]
> We could skip `--marketplace-address` argument and rely on marketplace contract [Auto-discovery](#auto-discovery).
After node is up and running, and your address has founds, you should be able to [Purchase storage](/learn/using#purchase-storage) using [API](/developers/api).
You also can use [Codex App UI](https://app.codex.storage) for storage purchase.
#### Codex storage node
Codex [storage node](architecture#network-architecture) should be run by storage providers or in case you would like to sell your local storage.
For that, additionally to the [Codex node with marketplace support](#codex-node-with-marketplace-support) we should use `prover` sub-command and required [CLI options](#cli-options).
That sub-command will make Codex to listen for a proof requests on the blockchain and answer them. To compute an answer for the proof request, Codex will use stored data and circuit files generated by the code in the [codex-storage-proofs-circuits](https://github.com/codex-storage/codex-storage-proofs-circuits) repository.
Every [network](/networks/networks) uses its own generated set of the files which are stored in the [codex-contracts-eth](https://github.com/codex-storage/codex-contracts-eth/tree/master/verifier/networks) repository and also uploaded to the CDN. Hash of the files set is also known by the [marketplace smart contract](/learn/architecture#smart-contract).
To download circuit files and make them available to Codex app, we have a stand-alone utility - `cirdl`. It can be [compiled from the sources](/learn/build#circuit-download-tool) or downloaded from the [GitHub release page](https://github.com/codex-storage/nim-codex/releases).
1. Create ethereum key file
<details>
<summary>example</summary>
> [!CAUTION]
> Please use key generation service for demo purpose only.
```shell
response=$(curl -s https://key.codex.storage)
awk -F ': ' '/private/ {print $2}' <<<"${response}" > eth.key
awk -F ': ' '/address/ {print $2}' <<<"${response}" > eth.address
chmod 600 eth.key
```
Show your ethereum address:
```shell
cat eth.address
```
```
0x412665aFAb17768cd9aACE6E00537Cc6D5524Da9
```
</details>
2. To download circuit files, we should pass directory, RPC endpoint and optionally marketplace address to the circuit downloader:
```shell
# Create circuit files folder
mkdir -p datadir/circuits
chmod 700 datadir/circuits
# Download circuit files
cirdl \
datadir/circuits \
https://rpc.testnet.codex.storage
```
2. Start Codex storage node
```shell
codex \
--data-dir=datadir \
--bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P \
--nat=any \
--disc-port=8090 \
--listen-addrs=/ip4/0.0.0.0/tcp/8070 \
persistence \
--eth-provider=https://rpc.testnet.codex.storage \
--eth-private-key=eth.key \
prover \
--circuit-dir=datadir/circuits
```
> [!NOTE]
> You would need to pass a bootstrap nodes and blockchain RPC endpoint based on the [network](/networks/networks) you would like to join.
> [!NOTE]
> We could skip `--marketplace-address` argument and rely on marketplace contract [Auto-discovery](#auto-discovery).
After node is up and running, and your address has founds, you should be able to [sell the storage](/learn/using#create-storage-availability) using [API](/developers/api).
You also can use [Codex App UI](https://app.codex.storage) to sell the storage.
#### Codex bootstrap node
If you need an SPR to bootstrap your own node, go to [network](/networks/networks) and select which network you'd like to join.
Bootstrap nodes are used just to help peers with the initial nodes discovery and we need to run Codex with just some basic options:
```shell
codex \
@ -390,7 +257,7 @@ codex \
--disc-port=8090
```
To get bootstrap node SPR we can use [API](https://api.codex.storage/#tag/Debug/operation/getDebugInfo) call:
To get bootstrap node SPR from your local node, we can use [API](https://api.codex.storage/#tag/Debug/operation/getDebugInfo) call:
```shell
curl -s localhost:8080/api/codex/v1/debug/info | jq -r '.spr'
```
@ -540,12 +407,10 @@ We also ship Codex in Docker containers, which can be run on `amd64` and `arm64`
- `ENV_PATH` - path to the file, in form `env=value` which will be sourced and available for Codex at run. That is useful for Kubernetes Pods configuration.
- `NAT_IP_AUTO` - when set to `true`, will set `CODEX_NAT` variable with container internal IP address. It also is useful for Kubernetes Pods configuration, when we perform automated tests.
- `NAT_PUBLIC_IP_AUTO` - used to set `CODEX_NAT` to public IP address using lookup services, like [ip.codex.storage](https://ip.codex.storage). Can be used for Docker/Kubernetes to set public IP in auto mode.
- `ETH_PRIVATE_KEY` - can be used to pass ethereum private key, which will be saved and passed as a value of the `CODEX_ETH_PRIVATE_KEY` variable. It should be considered as unsafe option and used for testing purposes only.
- When we set `prover` sub-command, entrypoint will run `cirdl` tool to download ceremony files, required by [Codex storage node](#codex-storage-node).
- `BOOTSTRAP_NODE_URL` - Codex node API URL in form of `http://bootstrap:8080`, to be used to get it's SPR as a bootstrap node. That is useful for Docker and Kubernetes configuration.
- `NETWORK` - is a helper variable to simply a specific network join. It helps to automate `BOOTSTRAP_NODE_FROM_URL` variable.
- `BOOTSTRAP_NODE_FROM_URL` - can be used to pass SPR nodes from an URL like [spr.codex.storage/testnet](https://spr.codex.storage/testnet).
- `MARKETPLACE_ADDRESS_FROM_URL` - can be used to pass a Marketplace contract address from an URL like [marketplace.codex.storage/codex-testnet/latest](https://marketplace.codex.storage/codex-testnet/latest).
#### Docker network
@ -564,36 +429,11 @@ In that case, we should consider the following solutions:
And we basically can use same options we [used for binary](#using-binary) and additionally it is required to mount volumes and map the ports.
[Codex storage node](#codex-storage-node)
1. Create ethereum key file
<details>
<summary>example</summary>
> [!CAUTION]
> Please use key generation service for demo purpose only.
```shell
response=$(curl -s https://key.codex.storage)
awk -F ': ' '/private/ {print $2}' <<<"${response}" > eth.key
awk -F ': ' '/address/ {print $2}' <<<"${response}" > eth.address
chmod 600 eth.key
```
Show your ethereum address:
```shell
cat eth.address
```
```
0x412665aFAb17768cd9aACE6E00537Cc6D5524Da9
```
</details>
2. Run Codex:
1. Run Codex:
```shell
docker run \
--rm \
-v $PWD/datadir:/datadir \
-v $PWD/eth.key:/opt/eth.key \
-p 8070:8070 \
-p 8080:8080 \
-p 8090:8090/udp \
@ -606,49 +446,17 @@ docker run \
--listen-addrs=/ip4/0.0.0.0/tcp/8070 \
--api-cors-origin="*" \
--api-bindaddr=0.0.0.0 \
--api-port=8080 \
persistence \
--eth-provider=https://rpc.testnet.codex.storage \
--eth-private-key=/opt/eth.key \
prover \
--circuit-dir=/datadir/circuits
--api-port=8080
```
> [!NOTE]
> You would need to pass a bootstrap nodes and blockchain RPC endpoint based on the [network](/networks/networks) you would like to join.
> [!NOTE]
> We could skip `--marketplace-address` argument and rely on marketplace contract [Auto-discovery](#auto-discovery).
> You would need to pass a bootstrap nodes based on the [network](/networks/networks) you would like to join.
### Using Docker Compose
For Docker Compose, it is more suitable to use [environment variables](#environment-variables) for Codex configuration and we can reuse commands from example above, for Docker.
[Codex storage node](#codex-storage-node)
1. Create ethereum key file
<details>
<summary>example</summary>
> [!CAUTION]
> Please use key generation service for demo purpose only.
```shell
response=$(curl -s https://key.codex.storage)
awk -F ': ' '/private/ {print $2}' <<<"${response}" > eth.key
awk -F ': ' '/address/ {print $2}' <<<"${response}" > eth.address
chmod 600 eth.key
```
Show your ethereum address:
```shell
cat eth.address
```
```
0x412665aFAb17768cd9aACE6E00537Cc6D5524Da9
```
</details>
2. Create `docker-compose.yaml` file:
1. Create `docker-compose.yaml` file:
```yaml
services:
codex:
@ -656,8 +464,6 @@ For Docker Compose, it is more suitable to use [environment variables](#environm
container_name: codex
command:
- codex
- persistence
- prover
- --bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P
- --bootstrap-node=spr:CiUIAhIhAyUvcPkKoGE7-gh84RmKIPHJPdsX5Ugm_IHVJgF-Mmu_EgIDARo8CicAJQgCEiEDJS9w-QqgYTv6CHzhGYog8ck92xflSCb8gdUmAX4ya78QoemesAYaCwoJBES39Q2RAnVOKkYwRAIgLi3rouyaZFS_Uilx8k99ySdQCP1tsmLR21tDb9p8LcgCIG30o5YnEooQ1n6tgm9fCT7s53k6XlxyeSkD_uIO9mb3
- --bootstrap-node=spr:CiUIAhIhA6_j28xa--PvvOUxH10wKEm9feXEKJIK3Z9JQ5xXgSD9EgIDARo8CicAJQgCEiEDr-PbzFr74--85TEfXTAoSb195cQokgrdn0lDnFeBIP0QzOGesAYaCwoJBK6Kf1-RAnVEKkcwRQIhAPUH5nQrqG4OW86JQWphdSdnPA98ErQ0hL9OZH9a4e5kAiBBZmUl9KnhSOiDgU3_hvjXrXZXoMxhGuZ92_rk30sNDA
@ -673,16 +479,12 @@ For Docker Compose, it is more suitable to use [environment variables](#environm
- CODEX_API_CORS_ORIGIN="*"
- CODEX_API_PORT=8080
- CODEX_API_BINDADDR=0.0.0.0
- CODEX_ETH_PROVIDER=https://rpc.testnet.codex.storage
- CODEX_ETH_PRIVATE_KEY=/opt/eth.key
- CODEX_CIRCUIT_DIR=/datadir/circuits
ports:
- 8080:8080/tcp # API
- 8090:8090/udp # Discovery
- 8070:8070/tcp # Transport
volumes:
- ./datadir:/datadir
- ./eth.key:/opt/eth.key
logging:
driver: json-file
options:
@ -696,10 +498,7 @@ For Docker Compose, it is more suitable to use [environment variables](#environm
```
> [!NOTE]
> You would need to pass a bootstrap nodes and blockchain RPC endpoint based on the [network](/networks/networks) you would like to join.
> [!NOTE]
> We could skip `CODEX_MARKETPLACE_ADDRESS` variable and rely on marketplace contract [Auto-discovery](#auto-discovery).
> You would need to pass a bootstrap nodes based on the [network](/networks/networks) you would like to join.
### On Kubernetes

2
learn/troubleshoot.md
View File

@ -3,7 +3,7 @@ outline: [2, 3]
---
# Troubleshoot
Having trouble getting your Codex node connected to the testnet? Here's a list of common Codex connection conundrums and steps to diagnose and solve them. If your troubles are not addressed here, check our open issues on Github or reach out via our Discord server.
Having trouble getting your Codex node connected? Here's a list of common Codex connection conundrums and steps to diagnose and solve them. If your troubles are not addressed here, check our open issues on Github or reach out via our Discord server.
## Some basics

149
learn/using.md
View File

@ -16,9 +16,6 @@ Command line interpreter on [Linux/macOS](#linux-macos) and [Windows](#windows)
2. [Upload a file](#upload-a-file)
3. [Download a file](#download-a-file)
4. [Local data](#local-data)
5. [Create storage availability](#create-storage-availability)
6. [Purchase storage](#purchase-storage)
7. [View purchase status](#view-purchase-status)
### Debug
An easy way to check that your node is up and running is:
@ -70,85 +67,6 @@ curl http://localhost:8080/api/codex/v1/data \
-w '\n'
```
### Create storage availability
> [!WARNING]
> This step requires that Codex was started with the [`prover`](/learn/run#codex-storage-node) option.
In order to start selling storage space to the network, you must configure your node with the following command. Once configured, the node will monitor on-chain requests-for-storage and will automatically enter into contracts that meet these specifications. In order to enter and maintain storage contracts, your node is required to submit zero-knowledge storage proofs. The calculation of these proofs will increase the CPU and RAM usage of Codex.
```shell
curl -X POST \
http://localhost:8080/api/codex/v1/sales/availability \
-H 'Content-Type: application/json' \
-w '\n' \
-d '{
"totalSize": "8000000",
"duration": "7200",
"minPricePerBytePerSecond": "1000",
"totalCollateral": "80000000"
}'
```
For descriptions of each parameter, please view the [spec](https://api.codex.storage/#tag/Marketplace/operation/offerStorage).
### Purchase storage
To purchase storage space from the network, first you must upload your data. Once you have the CID, use the following to create a request-for-storage.
Set your CID:
```shell
# paste your CID from the previous step here between the quotes
CID="..."
echo "CID: ${CID}"
```
Next you can run:
```shell
curl -X POST \
"http://localhost:8080/api/codex/v1/storage/request/${CID}" \
-w '\n' \
-d '{
"duration": "3600",
"pricePerBytePerSecond": "2000",
"proofProbability": "5",
"expiry": "1200",
"nodes": 5,
"tolerance": 2,
"collateralPerByte": "1"
}'
```
For descriptions of each parameter, please view the [spec](https://api.codex.storage/#tag/Marketplace/operation/createStorageRequest).
When successful, this request will return a Purchase-ID.
### View purchase status
Using a Purchase-ID, you can check the status of your request-for-storage contract:
```shell
# paste your PURCHASE_ID from the previous step here between the quotes
PURCHASE_ID="..."
```
Then:
```shell
curl "http://localhost:8080/api/codex/v1/storage/purchases/${PURCHASE_ID}" \
-w '\n'
```
This will display state and error information for your purchase.
| State | Description |
|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Pending | Request is waiting for chain confirmation. |
| Submitted | Request is on-chain. Hosts may now attempt to download the data. |
| Started | Hosts have downloaded the data and provided proof-of-storage. |
| Failed | The request was started, but (too many) hosts failed to provide proof-of-storage on time. While the data may still be available in the network, for the purpose of the purchase it is considered lost. |
| Finished | The request was started successfully and the duration has elapsed. |
| Expired | (Not enough) hosts have submitted proof-of-storage before the request's expiry elapsed. |
| Errored | An unfortunate state of affairs. The 'error' field should tell you more. |
## Windows
### Overview {#overview-windows}
@ -156,9 +74,6 @@ This will display state and error information for your purchase.
2. [Upload a file](#upload-a-file-windows)
3. [Download a file](#download-a-file-windows)
4. [Local data](#local-data-windows)
5. [Create storage availability](#create-storage-availability-windows)
6. [Purchase storage](#purchase-storage-windows)
7. [View purchase status](#view-purchase-status-windows)
### Debug {#debug-windows}
An easy way to check that your node is up and running is:
@ -207,69 +122,5 @@ You can view which datasets are currently being stored by your node:
curl http://localhost:8080/api/codex/v1/data
```
### Create storage availability {#create-storage-availability-windows}
> [!WARNING]
> This step requires that Codex was started with the [`prover`](/learn/run#codex-storage-node) option.
In order to start selling storage space to the network, you must configure your node with the following command. Once configured, the node will monitor on-chain requests-for-storage and will automatically enter into contracts that meet these specifications. In order to enter and maintain storage contracts, your node is required to submit zero-knowledge storage proofs. The calculation of these proofs will increase the CPU and RAM usage of Codex.
```batch
curl -X POST ^
http://localhost:8080/api/codex/v1/sales/availability ^
-H "Content-Type: application/json" ^
-d "{""totalSize"": ""8000000"", ""duration"": ""7200"", ""minPricePerBytePerSecond"": ""1000"", ""totalCollateral"": ""80000000""}"
```
For descriptions of each parameter, please view the [spec](https://api.codex.storage/#tag/Marketplace/operation/offerStorage).
### Purchase storage {#purchase-storage-windows}
To purchase storage space from the network, first you must upload your data. Once you have the CID, use the following to create a request-for-storage.
Set your CID:
```batch
:: paste your CID from the previous step here between the quotes
set CID="..."
echo CID: %CID%
```
Next you can run:
```batch
curl -X POST ^
"http://localhost:8080/api/codex/v1/storage/request/%CID%" ^
-H "Content-Type: application/json" ^
-d "{""duration"": ""3600"",""pricePerBytePerSecond"": ""2000"", ""proofProbability"": ""5"", ""expiry"": ""1200"", ""nodes"": 5, ""tolerance"": 2, ""**collateralPerByte**"": ""1""}"
```
For descriptions of each parameter, please view the [spec](https://api.codex.storage/#tag/Marketplace/operation/createStorageRequest).
When successful, this request will return a Purchase-ID.
### View purchase status {#view-purchase-status-windows}
Using a Purchase-ID, you can check the status of your request-for-storage contract:
```batch
:: paste your PURCHASE_ID from the previous step here between the quotes
set PURCHASE_ID="..."
```
Then:
```batch
curl "http://localhost:8080/api/codex/v1/storage/purchases/%PURCHASE_ID%"
```
This will display state and error information for your purchase.
| State | Description |
|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Pending | Request is waiting for chain confirmation. |
| Submitted | Request is on-chain. Hosts may now attempt to download the data. |
| Started | Hosts have downloaded the data and provided proof-of-storage. |
| Failed | The request was started, but (too many) hosts failed to provide proof-of-storage on time. While the data may still be available in the network, for the purpose of the purchase it is considered lost. |
| Finished | The request was started successfully and the duration has elapsed. |
| Expired | (Not enough) hosts have submitted proof-of-storage before the request's expiry elapsed. |
| Errored | An unfortunate state of affairs. The 'error' field should tell you more. |
## Known issues
1. We add a new line to the API calls to get more readable output, please check [[rest] Add line ending on responses #771](https://github.com/codex-storage/nim-codex/issues/771) for more details.

26
networks/devnet.md
View File

@ -3,28 +3,6 @@ outline: [2, 4]
---
# Codex Devnet
The Codex Devnet has been launched and is ready to be used for development :building_construction:
The Codex devnet is under construction :building_construction:
## Devnet data
### Bootstrap Nodes
For Devnet we are using separate bootstrap nodes and their SPR's could be found in [GitHub repository](https://github.com/codex-storage/codex-spr) and accessed via endpoint [`spr.codex.storage/devnet`](https://spr.codex.storage/devnet).
### Smart contracts
| Contract | Address |
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| Token | [`0xccbb4947f53f7ddf05f0E4A686E6927D17e13e66`](https://sepoliascan.status.network/address/0xccbb4947f53f7ddf05f0E4A686E6927D17e13e66) |
| Marketplace | [`marketplace.codex.storage/codex-devnet/latest`](https://marketplace.codex.storage/codex-devnet/latest) |
### Endpoints
Please check [Status Network Docs](https://docs.status.network) for additional details.
| # | Service | URL |
| - | --------------- | ------------------------------------------------------------------------------ |
| 1 | Public RPC | [public.sepolia.rpc.status.network](https://public.sepolia.rpc.status.network) |
| 2 | Block explorer | [sepoliascan.status.network](https://sepoliascan.status.network) |
| 3 | Faucet ETH | [faucet.status.network](https://faucet.status.network) |
| 4 | Faucet TST | [faucet-tst.devnet.codex.storage](https://faucet-tst.devnet.codex.storage) |
The Codex devnet is currently offline for a period of redesign and upgrades.

22
networks/networks.md
View File

@ -1,19 +1,7 @@
# Codex networks
At Codex, we are launching several networks which are used for different purposes
| Network | Status | Blockchain | Purpose |
| ------------------ | ------------------ | ----------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| [Devnet](devnet) | :white_check_mark: | [Status Network](https://status.network/) | Dedicated for development purposes and following latest `master` builds |
| [Testnet](testnet) | :white_check_mark: | [Geth PoA](https://geth.ethereum.org/docs/fundamentals/private-network) | A public network for testing purposes, following latest releases |
| Mainnet | :construction: | :construction: | A main public network |
The main difference between networks are following
- Network purpose
- Bootstrap nodes
- Number of storage nodes
- Available storage volume
- Blockchain network
- Marketplace contract version
- Circuit files set used for proof verification
The easiest way to start to use Codex is to [Join Testnet](testnet).
At Codex, we will be launching both a devnet and a testnet.
| Network | Status | Purpose |
| ------------------ | -------------- | ----------------------------------------------------------------------- |
| [Devnet](devnet) | :construction: | Dedicated for development purposes and following latest `master` builds |
| [Testnet](testnet) | :construction: | A public network for testing purposes, following latest releases |

171
networks/testnet.md
View File

@ -3,172 +3,7 @@ outline: [2, 4]
---
# Codex Testnet
The Codex Testnet has been launched and is ready to be used for testing.
The Codex testnet is under construction :building_construction:
Your participation in the Codex Testnet is subject to the [Codex Testnet Terms and Conditions](https://github.com/codex-storage/codex-testnet-starter/blob/master/Codex%20Testnet%20Terms%20and%20Conditions.pdf) and [Codex Testnet Privacy Policy](https://github.com/codex-storage/codex-testnet-starter/blob/master/Codex%20Testnet%20Privacy%20Policy.pdf).
**Guides.** We have basic guides covering how to set up a Storage Client which can be used to upload and persist files by buying storage in the Codex network. We recommend that you start with those.
Running a Storage Provider is more involved and is covered as a separate guide which demonstrates the storage sales side, as well as how to run Codex with its own local Ethereum execution client.
Guides are available either on Discord, as step-by-step, interactive guides, or here as simple instructions that you can follow:
- **Basic: running a storage client.** [[Discord](#sc-guide-discord) | [web](#sc-guide-web)]
- **Advanced: Running a storage provider.** [[web](#sp-guide-web)]
The guides were tested on the following operating systems:
- Linux: Ubuntu 24.04, Debian 12, Fedora 40
- macOS: 15
- Windows: 11, Server 2022
## Running a Storage Client (Discord Version) {#sc-guide-discord}
You can join [Codex Discord server](https://discord.gg/codex-storage) and jump into the [#:tv:|join-testnet](https://discord.com/channels/895609329053474826/1289923125928001702) channel.
It is mostly the same as a [Web guide](#sc-guide-web), but uses Discord capabilities so you can have an interactive, step-by-step guide, and you also can get a support in the [#:sos:|node-help](https://discord.com/channels/895609329053474826/1286205545837105224) channel.
## Running a Storage Client (Web Version) {#sc-guide-web}
**Prerequisites**
- Access to your Internet router so you can [configure port forwarding](#basic-common)
Steps for [Linux/macOS](#basic-linux-macos) and [Windows](#basic-windows) are slightly different, so please use ones for your OS.
<hr>
### Linux/macOS {#basic-linux-macos}
1. Install Codex binaries from GitHub releases:
```shell
curl -s https://get.codex.storage/install.sh | bash
```
2. Install dependencies when required:
```shell
# Debian-based Linux
sudo apt update && sudo apt install libgomp1
```
3. Generate an ethereum keypair:
```shell
curl -s https://get.codex.storage/generate.sh | bash
```
Your private key will be saved to `eth.key` and address to `eth.address` file.
4. Fill-up your address shown on the screen with the tokens:
- Use the web faucets to mint some [ETH](https://faucet-eth.testnet.codex.storage) and [TST](https://faucet-tst.testnet.codex.storage) tokens.
- We can also do this using Discord [# bot](https://discord.com/channels/895609329053474826/1230785221553819669) channel
- Use `/set ethaddress` command to enter your generated address
- Use `/mint` command to receive ETH and TST tokens
- Use `/balance` command to check if you have received test tokens successfully
5. Run Codex node:
```shell
curl -s https://get.codex.storage/run.sh | bash
```
6. Configure [port forwarding](#basic-common) and we are ready go to.
### Windows {#basic-windows}
1. Download the master tarball from the Codex testnet starter repository, and untar its contents:
> [!WARNING]
> Windows antivirus software and built-in firewalls may cause steps to fail. We will cover some possible errors here, but always consider checking your setup if requests fail - in particular, if temporarily disabling your antivirus fixes it, then it is likely to be the culprit.
```batch
curl -LO https://github.com/codex-storage/codex-testnet-starter/archive/master.tar.gz
```
If you see an error like:
```batch
curl: (35) schannel: next InitializeSecurityContext failed: CRYPT_E_NO_REVOCATION_CHECK (0x80092012) - The revocation function was unable to check revocation for the certificate.
```
You may need to add the `--ssl-no-revoke` option to your curl call, e.g.:
```batch
curl -LO --ssl-no-revoke https://github.com/codex-storage/codex-testnet-starter/archive/master.tar.gz
```
1. Extract the contents of the tar file, and then delete it:
```batch
tar xzvf master.tar.gz
del master.tar.gz
```
2. Navigate to the scripts folder:
```batch
cd codex-testnet-starter-master\scripts\windows
```
3. Download Codex binaries from GitHub releases:
```batch
download-online.bat
```
4. Generate an ethereum keypair:
```batch
generate.bat
```
Your private key will be saved to `eth.key` and address to `eth.address` file.
5. Fill-up your address shown on the screen with the tokens:
- Use the web faucets to mint some [ETH](https://faucet-eth.testnet.codex.storage) and [TST](https://faucet-tst.testnet.codex.storage) tokens.
- We can also do this using Discord [# bot](https://discord.com/channels/895609329053474826/1230785221553819669) channel
- Use `/set ethaddress` command to enter your generated address
- Use `/mint` command to receive ETH and TST tokens
- Use `/balance` command to check if you have received test tokens successfully
6. Run Codex node:
```batch
run-client.bat
```
7. Configure [port forwarding](#basic-common) and we are ready go to.
### All OS {#basic-common}
Configure [port forwarding](https://en.wikipedia.org/wiki/Port_forwarding) on your Internet router
| # | Protocol | Port | Description |
| - | -------- | ------ | ----------------- |
| 1 | `UDP` | `8090` | `Codex Discovery` |
| 2 | `TCP` | `8070` | `Codex Transport` |
After your node is up and running, you can use the [Codex API](/developers/api) to be able to interact with your Codex node, please check our [API walk-through](/learn/using) for more details.
You also can use [Codex App UI](https://app.codex.storage) to interact with your local Codex node.
Need help? Reach out to us in [#:sos:|node-help](https://discord.com/channels/895609329053474826/1286205545837105224) channel or check [troubleshooting guide](/learn/troubleshoot.md).
## Running a Storage Provider (Web Version) {#sp-guide-web}
Work in progress :construction:
## Testnet Data
### Bootstrap Nodes
Testnet bootstrap nodes records could be found in [GitHub repository](https://github.com/codex-storage/codex-spr) and accessed via endpoints
- Codex - [`spr.codex.storage/testnet`](https://spr.codex.storage/testnet)
- Geth - [`spr.codex.storage/testnet/geth`](https://spr.codex.storage/testnet/geth)
### Smart contracts
| Contract | Address |
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| Token | [`0x34a22f3911De437307c6f4485931779670f78764`](https://explorer.testnet.codex.storage/address/0x34a22f3911De437307c6f4485931779670f78764) |
| Verifier | [`0x1f60B2329775545AaeF743dbC3571e699405263e`](https://explorer.testnet.codex.storage/address/0x1f60B2329775545AaeF743dbC3571e699405263e) |
| Marketplace | [`marketplace.codex.storage/codex-testnet/latest`](https://marketplace.codex.storage/codex-testnet/latest) |
### Endpoints
| # | Service | URL |
| - | --------------- | ---------------------------------------------------------------------------- |
| 1 | Geth Public RPC | [rpc.testnet.codex.storage](https://rpc.testnet.codex.storage) |
| 2 | Block explorer | [explorer.testnet.codex.storage](https://explorer.testnet.codex.storage) |
| 3 | Faucet ETH | [faucet-eth.testnet.codex.storage](https://faucet-eth.testnet.codex.storage) |
| 4 | Faucet TST | [faucet-tst.testnet.codex.storage](https://faucet-tst.testnet.codex.storage) |
| 5 | Status page | [status.testnet.codex.storage](https://status.testnet.codex.storage) |
The Codex testnet is currently offline for a period of redesign and upgrades.
See [Pausing Testnet for Our Next Phase of Development](https://blog.codex.storage/codex-august-updates-2/)