From 39d772176baef136972ebe95309a424670ffac01 Mon Sep 17 00:00:00 2001 From: Slava <20563034+veaceslavdoina@users.noreply.github.com> Date: Sun, 29 Sep 2024 10:28:42 +0300 Subject: [PATCH] Spelling and refactoring (#22) * Spelling and use GitHub style for Alerts * Spelling and refactoring --- learn/quick-start.md | 13 ++++--- learn/run.md | 85 +++++++++++++++++++++++--------------------- 2 files changed, 50 insertions(+), 48 deletions(-) diff --git a/learn/quick-start.md b/learn/quick-start.md index 328a5e1..0cc8610 100644 --- a/learn/quick-start.md +++ b/learn/quick-start.md @@ -8,7 +8,7 @@ To run Codex through this guide we would need to perform the following steps: ## Get Codex binary -For quick start we will use precompiled binaries from [GitHub release page](https://github.com/codex-storage/nim-codex/releases). If you prefer to compile from the sources, please check [Build Codex](/learn/build). +For quick a start we will use precompiled binaries from [GitHub release page](https://github.com/codex-storage/nim-codex/releases). If you prefer to compile from the sources, please check [Build Codex](/learn/build). Please follow the steps for your OS from the list: - [Linux/macOS](#linux-macos) @@ -83,9 +83,8 @@ Please follow the steps for your OS from the list: ### Windows 1. Download binary and checksum for your platform/architecture - ::: warning - For Windows, only amd64 architecture is supported now. - ::: + > [!WARNING] + > For Windows, only amd64 architecture is supported now. ```batch set version=v0.1.4 set platform=windows @@ -137,6 +136,7 @@ Please follow the steps for your OS from the list: Nim Compiler Version 1.6.14 [Windows: amd64] Compiled at 2024-06-27 Copyright (c) 2006-2023 by Andreas Rumpf + git hash: 38640664088251bbc88917b4bacfd86ec53014b8 active boot switches: -d:release ``` @@ -179,9 +179,8 @@ We may [run Codex in different modes](/learn/run#run), and for a quick start we --bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P ``` - ::: tip - In the example above we use [Codex Testnet](/networks/testnet#bootstrap-nodes) bootstrap nodes and thus we join Testnet. If you would like to join a different network, please use [appropriate value](/networks/networks). - ::: + > [!TIP] + > In the example above we use [Codex Testnet](/networks/testnet#bootstrap-nodes) bootstrap nodes and thus we join Testnet. If you would like to join a different network, please use [appropriate value](/networks/networks). 2. Configure port-forwarding for the TCP/UDP ports on your Internet router | Protocol | Service | Port | diff --git a/learn/run.md b/learn/run.md index 58cc118..cc4ca80 100644 --- a/learn/run.md +++ b/learn/run.md @@ -12,7 +12,6 @@ It is a command-line application which may be run in a different ways: - [Run as a service in Windows](#run-as-a-service-in-windows) (not supported yet) - [Using Docker](#using-docker) - [Using Docker Compose](#using-docker-compose) - - [Using Ansible](#using-ansible) - [On Kubernetes](#on-kubernetes) During the run, it is required to pass [configuration](#configuration) option to the application, which can be done in a different ways. @@ -31,7 +30,7 @@ The order of priority is the same as above: #### Units -For some configuration options, we can pass values in common units like: +For some configuration options, we can pass values in common units like following: ```shell --cache-size=1m/1M/1mb/1MB --storage-quota=2m/2M/2mb/2MB @@ -138,7 +137,7 @@ and then transform it in the following way: For example, to configure `--log-level`, use `CODEX_LOG_LEVEL` as the environment variable name. > [!WARNING] -> Some options can't be configured via environment variables for now. Please read [Known issues](#known-issues) for more details. +> Some options can't be configured via environment variables for now [^multivalue-env-var] [^sub-commands]. ### Configuration file @@ -179,7 +178,7 @@ We can run Codex in a simple way like following: codex ``` > [!WARNING] -> This command may not work properly when we use GitHub releases, please read [Known issues](#known-issues) for more details. +> This command may not work properly when we use GitHub releases [^data-dir]. But, it will use a default `data-dir` value and we can pass a custom one: ```shell @@ -208,12 +207,13 @@ codex \ > [!TIP] > We can set public IP using curl and IP lookup service, like [ip.codex.storage](https://ip.codex.storage). -After that, node will announce itself using your public IP and dynamic TCP port for [libp2p transport](https://docs.libp2p.io/concepts/transports/overview/) (data transfer), which can be adjusted in the following way: +After that, node will announce itself using your public IP, default UDP ([discovery](https://docs.libp2p.io/concepts/discovery-routing/overview/)) and dynamic TCP port ([data transfer](https://docs.libp2p.io/concepts/transports/overview/)), which can be adjusted in the following way: ```shell codex \ --data-dir=datadir \ --bootstrap-node=spr:CiUIAhIhAiJvIcA_ZwPZ9ugVKDbmqwhJZaig5zKyLiuaicRcCGqLEgIDARo8CicAJQgCEiECIm8hwD9nA9n26BUoNuarCEllqKDnMrIuK5qJxFwIaosQ3d6esAYaCwoJBJ_f8zKRAnU6KkYwRAIgM0MvWNJL296kJ9gWvfatfmVvT-A7O2s8Mxp8l9c8EW0CIC-h-H-jBVSgFjg3Eny2u33qF7BDnWFzo7fGfZ7_qc9P \ --nat=`curl -s https://ip.codex.storage` \ + --disc-port=8090 \ --listen-addrs=/ip4/0.0.0.0/tcp/8070 ``` @@ -226,13 +226,13 @@ curl -s localhost:8080/api/codex/v1/debug/info | jq -r '.announceAddresses' "/ip4//tcp/8070" ] ``` -Basically, for P2P communication we should specify and adjust two ports: +Basically, for P2P communication we should specify and configure two ports: | # | Protocol function | CLI option | Example | | - | ------------------------------------------------------------------------ | ---------------- | -------------------------------------- | | 1 | [Discovery](https://docs.libp2p.io/concepts/discovery-routing/overview/) | `--disc-port` | `--disc-port=8090` | | 2 | [Transport](https://docs.libp2p.io/concepts/transports/overview/) | `--listen-addrs` | `--listen-addrs=/ip4/0.0.0.0/tcp/8070` | -And, also it is required to setup port-forwarding on your Internet device, to make your node accessible for participants. Please read [Known issues](#known-issues) for more details. +And, also it is required to setup port-forwarding on your Internet router, to make your node accessible for participants [^port-forwarding]. So, a fully working basic configuration will looks like following: ```shell @@ -245,36 +245,38 @@ codex \ --api-cors-origin="*" ``` -After node is up and running and port-forwarding configurations was done, we should be able to [Upload a file](/learn/using#upload-a-file)/[Download a file](/learn/using#download-a-file) in the network. +After node is up and running and port-forwarding configurations was done, we should be able to [Upload a file](/learn/using#upload-a-file)/[Download a file](/learn/using#download-a-file) in the network using [API](/developers/api). -You also can use [Codex Marketplace UI](https://marketplace.codex.storage) for files upload/download. +We also can use [Codex Marketplace UI](https://marketplace.codex.storage) for files upload/download. -And to be able to purchase storage, we should run [Codex node with marketplace support](#codex-node-with-marketplace-support). +And to be able to purchase a storage, we should run [Codex node with marketplace support](#codex-node-with-marketplace-support). #### Codex node with marketplace support -[Marketplace](/learn/architecture.md#marketplace-architecture) support permits to purchase the storage in Codex network. Basically, we should add just a `persistence` sub-command and required CLI options to the [previous run](#codex-node). +[Marketplace](/learn/architecture.md#marketplace-architecture) support permits to purchase the storage in Codex network. Basically, we should add just a `persistence` sub-command and required [CLI options](#cli-options) to the [previous run](#codex-node). > [!NOTE] -> Please ignore `--eth-account` CLI option, as it is obsolete. Please read [Known issues](#known-issues) for more details. +> Please ignore `--eth-account` CLI option, as it is obsolete [^eth-account]. -1. For a daily use, we should consider to run a local blockchain node based on the [network](/networks/networks) you would like to join. That process is described in the [Join Testnet](/networks/testnet) guide, but for quick start we can use a public RPC endpoint. +1. For a daily use, we should consider to run a local blockchain node based on the [network](/networks/networks) you would like to join. That process is described in the [Join Codex Testnet](/networks/testnet) guide, but for a quick start we can use a public RPC endpoint. + +2. Create a file with ethereum private key and set a proper permissions: + > [!CAUTION] + > Please use key generation service for demo purpose only. -2. Create a file with ethereum private key and set a proper permissions. ```shell key=$(curl -s https://key.codex.storage) echo $key | awk -F ':' '/address/ {print $2}' >eth.address echo $key | awk -F ':' '/private/ {print $2}' >eth.key chmod 600 eth.key ``` - Show your ethereum address + Show your ethereum address: ```shell cat eth.address - + ``` + ``` 0x412665aFAb17768cd9aACE6E00537Cc6D5524Da9 ``` - > [!CAUTION] - > Please use mentioned key generation service for demo purpose only. 3. Fill-up your ethereum address with ETH and Tokens based on the the [network](/networks/networks) you would like to join. @@ -289,32 +291,36 @@ And to be able to purchase storage, we should run [Codex node with marketplace s --nat=`curl -s https://ip.codex.storage` \ --disc-port=8090 \ --listen-addrs=/ip4/0.0.0.0/tcp/8070 \ + --api-cors-origin="*" \ --eth-provider=https://rpc.testnet.codex.storage \ --eth-private-key=eth.key \ - --marketplace-address=0xCDef8d6884557be4F68dC265b6bB2E3e52a6C9d6 \ - --api-cors-origin="*" + --marketplace-address=0xCDef8d6884557be4F68dC265b6bB2E3e52a6C9d6 ``` -After node is up and running, and your address has founds, you should be able to [Purchase storage](/learn/using#purchase-storage). +After node is up and running, and your address has founds, you should be able to [Purchase storage](/learn/using#purchase-storage) using [API](/developers/api). + +We also can use [Codex Marketplace UI](https://marketplace.codex.storage) for storage purchase. #### Codex storage node Codex [storage node](architecture#network-architecture) should be run by storage providers or in case you would like to sell your local storage. -For that, additionally to the [Codex node with marketplace support](#codex-node-with-marketplace-support) we should use `prover` sub-command and required CLI options. +For that, additionally to the [Codex node with marketplace support](#codex-node-with-marketplace-support) we should use `prover` sub-command and required [CLI options](#cli-options). -That sub-command will make Codex to listen for a proof request on the blockchain and answer them. To compute an answer for the proof request, Codex will use stored data and circuit files generated by the code in the [codex-storage-proofs-circuits](https://github.com/codex-storage/codex-storage-proofs-circuits) repository. +That sub-command will make Codex to listen for a proof requests on the blockchain and answer them. To compute an answer for the proof request, Codex will use stored data and circuit files generated by the code in the [codex-storage-proofs-circuits](https://github.com/codex-storage/codex-storage-proofs-circuits) repository. Every [network](/networks/networks) uses its own generated set of the files which are stored in the [codex-contracts-eth](https://github.com/codex-storage/codex-contracts-eth/tree/master/verifier/networks) repository and also uploaded to the CDN. Hash of the files set is also known by the [marketplace smart contract](/learn/architecture#smart-contract). To download circuit files and make them available to Codex app, we have a stand-alone utility - `cirdl`. It can be compiled from the sources (`make cirdl`) or downloaded from the [GitHub release page](https://github.com/codex-storage/nim-codex/releases). -You would need to pass a bootstrap nodes, blockchain RPC endpoint and marketplace address based on the [network](/networks/networks) you would like to join. +> [!TIP] +> You would need to pass a bootstrap nodes, blockchain RPC endpoint and marketplace address based on the [network](/networks/networks) you would like to join. -1. To download circuit files, we should pass directory, RPC endpoint and marketplace address +1. To download circuit files, we should pass directory, RPC endpoint and marketplace address to the circuit downloader: ```shell # Create circuit files folder mkdir -p datadir/circuit + chmod 700 datadir/circuit # Download circuit files cirdl \ @@ -339,12 +345,13 @@ You would need to pass a bootstrap nodes, blockchain RPC endpoint and marketplac --circuit-dir=datadir/circuits ``` -After node is up and running, and your address has founds, you should be able to [Create storage availability](/learn/using#create-storage-availability). +After node is up and running, and your address has founds, you should be able to [sell the storage](/learn/using#create-storage-availability) using [API](/developers/api). +We also can use [Codex Marketplace UI](https://marketplace.codex.storage) to sell the storage. #### Codex bootstrap node -Bootstrap nodes are used just to help peers with initial nodes discovery and we need to run Codex with some basic options: +Bootstrap nodes are used just to help peers with the initial nodes discovery and we need to run Codex with just some basic options: ```shell codex \ --data-dir=datadir \ @@ -360,12 +367,12 @@ curl -s localhost:8080/api/codex/v1/debug/info | jq -r '.spr' spr:CiUIAhIhApd79-AxPqwRDmu7Pk-berTDtoIoMz0ovKjo85Tz8CUdEgIDARo8CicAJQgCEiECl3v34DE-rBEOa7s-T5t6tMO2gigzPSi8qOjzlPPwJR0Qjv_WtwYaCwoJBFxzjbKRAh-aKkYwRAIgCiTq5jBTaJJb6lUxN-0uNCj8lkV9AGY682D21kIAMiICIE1yxrjbDdiSCiARnS7I2zqJpXC2hOvjB4JoL9SAAk67 ``` -That SPR record then can be used by other peer for initial nodes discovery. +That SPR record then can be used then by other peers for initial nodes discovery. We should keep in mind some important things about SPR record (see [ENR](https://eips.ethereum.org/EIPS/eip-778)) - It uses node IP (`--nat`), discovery port (`--disc-port`) and private key (`--net-privkey`) for record creation - Specified data is signed on each run and will be changed but still contain specified node data when decoded -- You can decode it by passing to the Codex node at run with `--log-level=trace` +- You can decode it by passing to the Codex node at run and with `--log-level=trace` For bootstrap node, it is required to forward just discovery port on your Internet router. @@ -388,13 +395,13 @@ We also ship Codex in Docker containers, which can be run on `amd64` and `arm64` - `NAT_IP_AUTO` - when set to `true`, will set `CODEX_NAT` variable with container internal IP address. It also is useful for Kubernetes Pods configuration, when we perform automated tests. - `NAT_PUBLIC_IP_AUTO` - used to set `CODEX_NAT` to public IP address using lookup services, like [ip.codex.storage](https://ip.codex.storage). Can be used for Docker/Kubernetes, and binary as well, to set public IP in auto mode. - `PRIV_KEY` - can be used to pass ethereum private key, which will be saved and passed as a value of the `CODEX_ETH_PRIVATE_KEY` variable. It should be considered as unsafe option and used for testing purposes only. -- When we set `prover` sub-command, entrypoint will run `cirdl` tool to download ceremony files, required by prover. +- When we set `prover` sub-command, entrypoint will run `cirdl` tool to download ceremony files, required by [Codex storage node](#codex-storage-node). #### Docker network When we are running Codex using Docker with default [bridge network](https://docs.docker.com/engine/network/drivers/bridge/), it will create a double NAT - One on the Docker side - - Second on your Internet connection + - Second on your Internet router If your Internet router does not support [Full Cone NAT](https://learningnetwork.cisco.com/s/question/0D56e0000CWxJ9sCQF/lets-explain-in-details-full-cone-nat-restricted-cone-nat-and-symmetric-nat-terminologies-vs-cisco-nat-terminologies), you might have an issue and peer discovery and data transport will not work or might work unexpected. @@ -425,10 +432,10 @@ docker run \ --nat=`curl -s https://ip.codex.storage` \ --disc-port=8090 \ --listen-addrs=/ip4/0.0.0.0/tcp/8070 \ + --api-cors-origin="*" \ --eth-provider=https://rpc.testnet.codex.storage \ --eth-private-key=/opt/eth.key \ --marketplace-address=0xCDef8d6884557be4F68dC265b6bB2E3e52a6C9d6 \ - --api-cors-origin="*" \ --api-bindaddr=0.0.0.0 \ --api-port=8080 ``` @@ -437,10 +444,6 @@ docker run \ To be added :construction: -### Using Ansible - -Planning :construction: - ### On Kubernetes To be added :construction: @@ -450,8 +453,8 @@ Helm chart code is available in [helm-charts](https://github.com/codex-storage/h ## Known issues -1. Environment variables like `CODEX_BOOTSTRAP_NODE` and `CODEX_LISTEN_ADDRS` does not support multiple values. Please check [[Feature request] Support multiple SPR records via environment variable #525](https://github.com/codex-storage/nim-codex/issues/525), for more information. -2. Sub-commands configuration like `persistence` and `persistence prover` can't be done via environment variables for now. -3. [NAT traversal #753](https://github.com/codex-storage/nim-codex/issues/753) is not implemented yet and we would need to setup port-forwarding for discovery and transport protocols. -4. Please ignore `--eth-account` CLI option - [Drop support for --eth-account #727](https://github.com/codex-storage/nim-codex/issues/727). -5. We should set data-dir explicitly when we use GitHub releases - [[BUG] Change codex default datadir from compile-time to run-time #923](https://github.com/codex-storage/nim-codex/issues/923). +[^multivalue-env-var]: Environment variables like `CODEX_BOOTSTRAP_NODE` and `CODEX_LISTEN_ADDRS` does not support multiple values. Please check [[Feature request] Support multiple SPR records via environment variable #525](https://github.com/codex-storage/nim-codex/issues/525), for more information. +[^sub-commands]: Sub-commands configuration like `persistence` and `persistence prover` can't be done via environment variables for now. +[^data-dir]: We should set data-dir explicitly when we use GitHub releases - [[BUG] Change codex default datadir from compile-time to run-time #923](https://github.com/codex-storage/nim-codex/issues/923) +[^port-forwarding]: [NAT traversal #753](https://github.com/codex-storage/nim-codex/issues/753) is not implemented yet and we would need to setup port-forwarding for discovery and transport protocols. +[^eth-account]: Please ignore `--eth-account` CLI option - [Drop support for --eth-account #727](https://github.com/codex-storage/nim-codex/issues/727).