status-im
/
libp2p-test-plans
mirror of https://github.com/status-im/libp2p-test-plans.git
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
libp2p-test-plans/perf
History
Max Inden ad43602e63
fix(perf): increase root volume size (#249) 
Multiple perf runs have failed due to "no space left on device". See e.g. https://github.com/libp2p/test-plans/actions/runs/5727165977/job/15527780097.

The perf terraform launch template previously did not specify a root volume
size. Thus terraform used the default:

```
df -h

Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        4.0M     0  4.0M   0% /dev
tmpfs            63G     0   63G   0% /dev/shm
tmpfs            25G  540K   25G   1% /run
/dev/nvme0n1p1  8.0G  1.9G  6.1G  24% /
tmpfs            63G     0   63G   0% /tmp
tmpfs            13G     0   13G   0% /run/user/1000
```

With this commit the root volume is increased to 100GB:

```
df -h

Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        4.0M     0  4.0M   0% /dev
tmpfs            63G     0   63G   0% /dev/shm
tmpfs            25G  564K   25G   1% /run
/dev/nvme0n1p1  100G  2.6G   98G   3% /
tmpfs            63G  8.0K   63G   1% /tmp
tmpfs            13G     0   13G   0% /run/user/0
tmpfs            13G     0   13G   0% /run/user/1000
```
 		2023-08-04 18:50:32 +02:00
..
impl fix(perf): replace rust-libp2p-quinn/ with rust-libp2p/master (#246) 2023-08-01 16:40:02 +02:00
runner fix(perf): increase root volume size (#249) 2023-08-04 18:50:32 +02:00
terraform fix(perf): increase root volume size (#249) 2023-08-04 18:50:32 +02:00
Makefile feat(perf): add (automation, provision, build, run) tooling (#184) 2023-06-22 13:54:09 +02:00
README.md docs(perf): document how to add a new version of existing impl (#243) 2023-07-26 18:41:49 +02:00

README.md

libp2p performance benchmarking

This project includes the following components:

  • terraform/: a Terraform scripts to provision infrastructure
  • impl/: implementations of the libp2p perf protocol running on top of e.g. go-libp2p, rust-libp2p or Go's std-library https stack
  • runner/: a set of scripts building and running the above implementations on the above infrastructure, reporting the results in benchmark-results.json

Benchmark results can be visualized with https://observablehq.com/@libp2p-workspace/performance-dashboard.

Running via GitHub Action

  1. Create a pull request with your changes on https://github.com/libp2p/test-plans/.
  2. Trigger GitHub Action for branch on https://github.com/libp2p/test-plans/actions/workflows/perf.yml (see Run workflow button).
  3. Wait for action run to finish and to push a commit to your branch.
  4. Visualize results on https://observablehq.com/@libp2p-workspace/performance-dashboard.

Running via local machine

Provision infrastructure

  1. Save your public SSH key as the file ./terraform/modules/short_lived/files/perf.pub; or generate a new key pair with make ssh-keygen and add it to your SSH agent with make ssh-add.
  2. cd terraform/configs/local
  3. terraform init
  4. terraform apply
  5. CLIENT_IP=$(terraform output -raw client_ip)
  6. SERVER_IP=$(terraform output -raw server_ip)

Build and run implementations

WARNING: Running the perf tests might take a while.

  1. cd runner
  2. npm ci
  3. npm run start -- --client-public-ip $CLIENT_IP --server-public-ip $SERVER_IP

Deprovision infrastructure

  1. cd terraform/configs/local
  2. terraform destroy

Adding a new implementation or a new version

  1. Add implementation to impl/.
    • For a new implementation, create a folder impl/<your-implementation-name>/
    • For a new version of an existing implementation, create a folder impl/<your-implementation-name>/<your-implementation-version>.
    • In that folder include a Makefile that builds an executable and stores it next to the Makefile under the name perf.
    • Requirements for the executable:
      • Running as a libp2p-perf server
        • Command line flags
          • --run-server
      • Running as a libp2p-perf client
        • Input via command line
          • --server-ip-address
          • --transport (see runner/versions.ts for possible variants)
          • --upload-bytes number of bytes to upload per stream.
          • --download-bytes number of bytes to download per stream.
        • Output
          • Logging MUST go to stderr.
          • Measurement output is printed to stdout as JSON in the form of: 
            {"latency": 0.246442851}
            Note that the measurement includes the time to (1) establish the connection, (2) upload the bytes and (3) download the bytes.
  2. For a new implementation, in impl/Makefile include your implementation in the all target.
  3. For a new version, reference version in runner/src/versions.ts.