Docs/update setup run and test steps (#342)
* Adds instructions how to integrate MSYS2 terminal into VSCode * Adds instructions for the installation of the ethereum test node required for the integration tests. * work-in-progress updating steps for two-client scenario. * Moves two-client test to separate MD file * Rename * Process review comments * Indent mistake * Attempt to sign commit
This commit is contained in:
Ben Bierens
GitHub
parent
3b4a078052
commit
9f73c86477
63
BUILDING.md
63
BUILDING.md
|
|
@ -76,6 +76,32 @@ $ pacman -S base-devel git unzip mingw-w64-ucrt-x86_64-toolchain mingw-w64-ucrt-
|
||||||
<!-- add instructions re: getting setup with MSYS2 in a Windows container -->
|
<!-- add instructions re: getting setup with MSYS2 in a Windows container -->
|
||||||
<!-- https://github.com/StefanScherer/windows-docker-machine -->
|
<!-- https://github.com/StefanScherer/windows-docker-machine -->
|
||||||
|
|
||||||
|
#### Optional: VSCode Terminal integration
|
||||||
|
|
||||||
|
You can link the MSYS2-UCRT64 terminal into VSCode by modifying the configuration file as shown below.
|
||||||
|
File: `C:/Users/<username>/AppData/Roaming/Code/User/settings.json`
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
...
|
||||||
|
"terminal.integrated.profiles.windows": {
|
||||||
|
...
|
||||||
|
"MSYS2-UCRT64": {
|
||||||
|
"path": "C:\\msys64\\usr\\bin\\bash.exe",
|
||||||
|
"args": [
|
||||||
|
"--login",
|
||||||
|
"-i"
|
||||||
|
],
|
||||||
|
"env": {
|
||||||
|
"MSYSTEM": "UCRT64",
|
||||||
|
"CHERE_INVOKING": "1",
|
||||||
|
"MSYS2_PATH_TYPE": "inherit"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
### Other
|
### Other
|
||||||
|
|
||||||
It is possible that nim-codex can be built and run on other platforms supported by the [Nim](https://nim-lang.org/) language: BSD family, older versions of Windows, etc. There has not been sufficient experimentation with nim-codex on such platforms, so instructions are not provided. Community contributions to these docs and our build system are welcome!
|
It is possible that nim-codex can be built and run on other platforms supported by the [Nim](https://nim-lang.org/) language: BSD family, older versions of Windows, etc. There has not been sufficient experimentation with nim-codex on such platforms, so instructions are not provided. Community contributions to these docs and our build system are welcome!
|
||||||
|
|
@ -121,23 +147,32 @@ $ make test
|
||||||
|
|
||||||
### testAll
|
### testAll
|
||||||
|
|
||||||
|
#### Prerequisites
|
||||||
|
|
||||||
|
To run the integration tests, an Ethereum test node is required. Follow these instructions to set it up.
|
||||||
|
|
||||||
|
##### Windows (do this before 'All platforms')
|
||||||
|
1. Download and install Visual Studio 2017 or newer. (Not VSCode!) In the Workloads overview, enable `Desktop development with C++`. ( https://visualstudio.microsoft.com )
|
||||||
|
|
||||||
|
##### All platforms
|
||||||
|
1. Install NodeJS (tested with v18.14.0), consider using NVM as a version manager. [Node Version Manager (`nvm`)](https://github.com/nvm-sh/nvm#readme)
|
||||||
|
1. Open a terminal
|
||||||
|
1. Go to the vendor/codex-contracts-eth folder: `cd /<git-root>/vendor/codex-contracts-eth/`
|
||||||
|
1. `npm install` -> Should complete with the number of packages added and an overview of known vulnerabilities.
|
||||||
|
1. `npm test` -> Should output test results. May take a minute.
|
||||||
|
|
||||||
|
Before the integration tests are started, you must start the Ethereum test node manually.
|
||||||
|
1. Open a terminal
|
||||||
|
1. Go to the vendor/codex-contracts-eth folder: `cd /<git-root>/vendor/codex-contracts-eth/`
|
||||||
|
1. `npm start` -> This should launch Hardhat, and output a number of keys and a warning message.
|
||||||
|
|
||||||
|
#### Run
|
||||||
|
|
||||||
The `testAll` target runs the same tests as `make test` and also runs tests for nim-codex's Ethereum contracts, as well a basic suite of integration tests.
|
The `testAll` target runs the same tests as `make test` and also runs tests for nim-codex's Ethereum contracts, as well a basic suite of integration tests.
|
||||||
|
|
||||||
To run `make testAll`, Node.js needs to be installed. [Node Version Manager (`nvm`)](https://github.com/nvm-sh/nvm#readme) is a flexible means to do that and it works on Linux, macOS, and Windows + MSYS2.
|
To run `make testAll`.
|
||||||
|
|
||||||
With `nvm` installed, launch a separate terminal and download the latest LTS version of Node.js
|
Use a new terminal to run:
|
||||||
```text
|
|
||||||
$ nvm install --lts
|
|
||||||
```
|
|
||||||
|
|
||||||
In that same terminal run
|
|
||||||
```text
|
|
||||||
$ cd repos/nim-codex/vendor/codex-contracts-eth && npm install && npm start
|
|
||||||
```
|
|
||||||
|
|
||||||
Those commands install and launch a [Hardhat](https://hardhat.org/) environment with nim-codex's Ethereum contracts.
|
|
||||||
|
|
||||||
In the other terminal run
|
|
||||||
```text
|
```text
|
||||||
$ make testAll
|
$ make testAll
|
||||||
```
|
```
|
||||||
|
|
|
||||||
32
README.md
32
README.md
|
|
@ -68,37 +68,7 @@ codex initNode
|
||||||
|
|
||||||
### Example: running two Codex clients
|
### Example: running two Codex clients
|
||||||
|
|
||||||
```bash
|
To get acquainted with Codex, consider running the manual two-client test described [HERE](docs/TWOCLIENTTEST.md).
|
||||||
build/codex --data-dir="$(pwd)/Codex1" -i=127.0.0.1
|
|
||||||
```
|
|
||||||
|
|
||||||
This will start codex with a data directory pointing to `Codex1` under the current execution directory and announce itself on the DHT under `127.0.0.1`.
|
|
||||||
|
|
||||||
To run a second client that automatically discovers nodes on the network, we need to get the Signed Peer Record (SPR) of first client, Client1. We can do this by querying the `/debug/info` endpoint of the node's REST API.
|
|
||||||
|
|
||||||
`curl http://127.0.0.1:8080/api/codex/v1/debug/info`
|
|
||||||
|
|
||||||
This should output information about Client1, including its PeerID, TCP/UDP addresses, data directory, and SPR:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"id": "16Uiu2HAm92LGXYTuhtLaZzkFnsCx6FFJsNmswK6o9oPXFbSKHQEa",
|
|
||||||
"addrs": [
|
|
||||||
"/ip4/0.0.0.0/udp/8090",
|
|
||||||
"/ip4/0.0.0.0/tcp/49336"
|
|
||||||
],
|
|
||||||
"repo": "/repos/status-im/nim-codex/Codex1",
|
|
||||||
"spr": "spr:CiUIAhIhAmqg5fVU2yxPStLdUOWgwrkWZMHW2MHf6i6l8IjA4tssEgIDARpICicAJQgCEiECaqDl9VTbLE9K0t1Q5aDCuRZkwdbYwd_qLqXwiMDi2ywQ5v2VlAYaCwoJBH8AAAGRAh-aGgoKCAR_AAABBts3KkcwRQIhAPOKl38CviplVbMVnA_9q3N1K_nk5oGuNp7DWeOqiJzzAiATQ2acPyQvPxLU9YS-TiVo4RUXndRcwMFMX2Yjhw8k3A"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Now, let's start a second client, Client2. Because we're already using the default ports TCP (:8080) and UDP (:8090) for the first client, we have to specify new ports to avoid a collision. Additionally, we can specify the SPR from Client1 as the bootstrap node for discovery purposes, allowing Client2 to determine where content is located in the network.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
build/codex --data-dir="$(pwd)/Codex2" -i=127.0.0.1 --api-port=8081 --udp-port=8091 --bootstrap-node=spr:CiUIAhIhAmqg5fVU2yxPStLdUOWgwrkWZMHW2MHf6i6l8IjA4tssEgIDARpICicAJQgCEiECaqDl9VTbLE9K0t1Q5aDCuRZkwdbYwd_qLqXwiMDi2ywQ5v2VlAYaCwoJBH8AAAGRAh-aGgoKCAR_AAABBts3KkcwRQIhAPOKl38CviplVbMVnA_9q3N1K_nk5oGuNp7DWeOqiJzzAiATQ2acPyQvPxLU9YS-TiVo4RUXndRcwMFMX2Yjhw8k3A
|
|
||||||
```
|
|
||||||
|
|
||||||
There are now two clients running. We could upload a file to Client1 and download that file (given its CID) using Client2, by using the clients' REST API.
|
|
||||||
|
|
||||||
## Interacting with the client
|
## Interacting with the client
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,112 @@
|
||||||
|
# Codex Two-Client Test
|
||||||
|
|
||||||
|
The two-client test is a manual test you can perform to check your setup and familiarize yourself with the Codex API. These steps will guide you through running and connecting two nodes, in order to upload a file to one and then download that file from the other. For the purpose of this test we will be running Codex disconnected from any Ethereum nodes, so no currency is required. Additionally, the contracts/sales/marketplace APIs will be unavailable for this reason.
|
||||||
|
|
||||||
|
## Prerequisite
|
||||||
|
|
||||||
|
Make sure you have built the client, and can run it as explained in the [README](../README.md).
|
||||||
|
|
||||||
|
## Steps
|
||||||
|
|
||||||
|
### 1. Launch Node #1
|
||||||
|
|
||||||
|
Open a terminal and run:
|
||||||
|
- Mac/Unx: `"build/codex" --data-dir="$(pwd)\Data1" --listen-addrs=/ip4/127.0.0.1/tcp/8070 --api-port=8080 --disc-port=8090`
|
||||||
|
- Windows: `"build/codex.exe" --data-dir="Data1" --listen-addrs=/ip4/127.0.0.1/tcp/8070 --api-port=8080 --disc-port=8090`
|
||||||
|
|
||||||
|
(Hint: If your terminal interprets the '/' in the listen-address as a reference to your root path, try running the command from a shell-script!)
|
||||||
|
|
||||||
|
| Argument | Description |
|
||||||
|
| -------------- | --------------------------------------------------------------------- |
|
||||||
|
| `data-dir` | We specify a relative path where the node will store its data. |
|
||||||
|
| `listen-addrs` | Multiaddress where the node will accept connections from other nodes. |
|
||||||
|
| `api-port` | Port on localhost where the node will expose its API. |
|
||||||
|
| `disc-port` | Port the node will use for its discovery service. |
|
||||||
|
|
||||||
|
Codex uses sane defaults for most of its arguments. Here we specify some explicitly for the purpose of this walk-through.
|
||||||
|
|
||||||
|
### 2. Sign of life
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl --request GET \
|
||||||
|
--url http://127.0.0.1:8080/api/codex/v1/debug/info
|
||||||
|
```
|
||||||
|
|
||||||
|
This GET request will return the node's debug information. The response JSON should look like:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "16Uiu2HAmGzJQEvNRYVVJNxHSb1Gd5xzxTK8XRZuMJzuoDaz7fADb",
|
||||||
|
"addrs": [
|
||||||
|
"/ip4/127.0.0.1/tcp/8070"
|
||||||
|
],
|
||||||
|
"repo": "Data1",
|
||||||
|
"spr": "spr:CiUIAhIhA0BhMXo12O4h8DSdfnvU6MWUQx3kd-xw_2sCZrWOWChOEgIDARo8CicAJQgCEiEDQGExejXY7iHwNJ1-e9ToxZRDHeR37HD_awJmtY5YKE4Q7aqInwYaCwoJBH8AAAGRAh-aKkYwRAIgSHGvrb4mxQbOTU5wdcJJYz3fErkVx4v09nqHE4n9d4ECIGWyfF58pmfUKeC7MWCtIhBDCgNJkjHz2JkKfJoYgqHW"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
| Field | Description |
|
||||||
|
| ------- | ---------------------------------------------------------------------------------------- |
|
||||||
|
| `id` | Id of the node. Also referred to as 'peerId'. |
|
||||||
|
| `addrs` | Multiaddresses currently open to accept connections from other nodes. |
|
||||||
|
| `repo` | Path of this node's data folder. |
|
||||||
|
| `spr` | Signed Peer Record, encoded information about this node and its location in the network. |
|
||||||
|
|
||||||
|
### 3. Launch Node #2
|
||||||
|
|
||||||
|
Replace `<SPR HERE>` in the next command with the string value for `spr`, returned by the first node's `debug/info` response.
|
||||||
|
|
||||||
|
Open a new terminal and run:
|
||||||
|
- Mac/Unx: `"build/codex" --data-dir="$(pwd)\Data2" --listen-addrs=/ip4/127.0.0.1/tcp/8071 --api-port=8081 --disc-port=8091 --bootstrap-node=<SPR HERE>`
|
||||||
|
- Windows: `"build/codex.exe" --data-dir="Data2" --listen-addrs=/ip4/127.0.0.1/tcp/8071 --api-port=8081 --disc-port=8091 --bootstrap-node=<SPR HERE>`
|
||||||
|
|
||||||
|
Notice we're using a new data-dir, and we've increased each port number by one. This is needed so that the new node won't try to open ports already in use by the first node.
|
||||||
|
|
||||||
|
We're now also including the `bootstrap-node` argument. This allows us to link the new node to another one, bootstrapping our own little peer-to-peer network. (SPR strings always start with "spr:".)
|
||||||
|
|
||||||
|
### 4. Connect The Two
|
||||||
|
|
||||||
|
Use the command we've used in step 2 to retrieve the debug information from node 2:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl --request GET \
|
||||||
|
--url http://127.0.0.1:8081/api/codex/v1/debug/info
|
||||||
|
```
|
||||||
|
|
||||||
|
In the JSON response copy the value for `id`, and use it to replace `<PEER ID HERE>` in the following command:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl --request GET \
|
||||||
|
--url http://127.0.0.1:8080/api/codex/v1/connect/<PEER ID HERE>?addrs=/ip4/127.0.0.1/tcp/8071
|
||||||
|
```
|
||||||
|
|
||||||
|
Notice that we are sending the peerId and the multiaddress of node 2 to the `/connect` endpoint of node 1. This provides node 1 all the information it needs to communicate with node 2. The response to this request should be `Successfully connected to peer`.
|
||||||
|
|
||||||
|
### 5. Upload The File
|
||||||
|
|
||||||
|
We're now ready to upload a file to the network. In this example we'll use node 1 for uploading and node 2 for downloading. But the reverse also works.
|
||||||
|
|
||||||
|
Replace `<FILE PATH>` with the path to the file you want to upload in the following command:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -H "content-type: application/octet-stream" -H "Expect: 100-continue" -T "<FILE PATH>" 127.0.0.1:8080/api/codex/v1/upload -X POST
|
||||||
|
```
|
||||||
|
|
||||||
|
(Hint: if curl is reluctant to show you the response, add `-o <FILENAME>` to write the result to a file.)
|
||||||
|
|
||||||
|
Depending on the file size this may take a moment. Codex is processing the file by cutting it into blocks and generating erasure-recovery data. When the process is finished, the request will return the content-identifier (CID) of the uploaded file. It should look something like `zdj7WVxH8HHHenKtid8Vkgv5Z5eSUbCxxr8xguTUBMCBD8F2S`.
|
||||||
|
|
||||||
|
### 6. Download The File
|
||||||
|
|
||||||
|
Replace `<CID>` with the identifier returned in the previous step. Replace `<OUTPUT FILE>` with the filename where you want to store the downloaded file.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl 127.0.0.1:8081/api/codex/v1/download/zdj7Wfm18wewSWL9SPqddhJuu5ii1TJD39rtt3JbVYdKcqM1K --output <OUTPUT FILE>
|
||||||
|
```
|
||||||
|
|
||||||
|
Notice we are connecting to the second node in order to download the file. The CID we provide contains the information needed to locate the file within the network.
|
||||||
|
|
||||||
|
### 7. Verify The Results
|
||||||
|
|
||||||
|
If your file is downloaded and identical to the file you uploaded, then this manual test has passed. Rejoice! If on the other hand that didn't happen or you were unable to complete any of these steps, please leave us a message detailing your troubles.
|
||||||
|
|
||||||
Loading…
Reference in New Issue