Commit Graph

10 Commits

Author SHA1 Message Date
Max Inden fd1889fedb
feat(perf): use m5.xlarge (#284)
Instead of `m5n.8xlarge`, use `m5.xlarge`.

Rational:
- Closer to real-world libp2p deployments (e.g. < 25 Gibps)
- Cheaper (~2 USD vs ~0.2 USD)

m5n.8xlarge:
- 32 vCPUs
- 128.0 GiB of memory
- 25 Gibps of bandwidth - 2019

m5.xlarge
- 4 vCPUs
- 16.0 GiB of memory
- up to 10 Gibps of bandwidth
2023-09-01 19:55:45 +02:00
Max Inden 826444d867
fix(perf): install nvm for ec2-user instead of root (#267)
On root the nvm install script can not find a shell config.

```
=> Profile not found. Tried ~/.bashrc, ~/.bash_profile, ~/.zprofile, ~/.zshrc, and ~/.profile.
```
2023-08-21 10:22:39 +02:00
Max Inden f57408de37
chore(perf): install NodeJS LTS on VMs (#266)
Needed for js-libp2p perf implementation.

https://github.com/libp2p/test-plans/pull/244/
2023-08-18 14:23:06 +02:00
Max Inden e5b7e0e16e
fix(perf): use `$Default` launch template (#257)
Our perf terraform setup differentiates in long lived and short lived resources.
On CI, our long lived resources are spun up once and our short lived resources
are spun up on each CI run. From time to time we have to adjust the long lived
resources. End result is a new launch template that needs to be referenced in
the short lived resources by version.

Next to our CI, the perf terraform setup can as well be used on personal AWS
accounts. Their long lived launch template version likely doesn't match the
configured launch template version of the short lived aws_instance.

Instead of specifying a specific version, instruct terraform to use the default,
thus supporting both our CI and personal AWS account use-case.
2023-08-17 15:02:23 +02:00
Max Inden 8e2c837599
fix(perf): increase maximum udp send buffer size (#254)
Previously only the maximum udp **receive** buffer size was increased to
`2500000` bytes. With this commit both the send and the receive buffer size is increase.
2023-08-14 16:08:41 +02:00
Max Inden ad43602e63
fix(perf): increase root volume size (#249)
Multiple perf runs have failed due to "no space left on device". See e.g. https://github.com/libp2p/test-plans/actions/runs/5727165977/job/15527780097.

The perf terraform launch template previously did not specify a root volume
size. Thus terraform used the default:

```
df -h

Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        4.0M     0  4.0M   0% /dev
tmpfs            63G     0   63G   0% /dev/shm
tmpfs            25G  540K   25G   1% /run
/dev/nvme0n1p1  8.0G  1.9G  6.1G  24% /
tmpfs            63G     0   63G   0% /tmp
tmpfs            13G     0   13G   0% /run/user/1000
```

With this commit the root volume is increased to 100GB:

```
df -h

Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        4.0M     0  4.0M   0% /dev
tmpfs            63G     0   63G   0% /dev/shm
tmpfs            25G  564K   25G   1% /run
/dev/nvme0n1p1  100G  2.6G   98G   3% /
tmpfs            63G  8.0K   63G   1% /tmp
tmpfs            13G     0   13G   0% /run/user/0
tmpfs            13G     0   13G   0% /run/user/1000
```
2023-08-04 18:50:32 +02:00
Max Inden cfd1049e8a
fix(perf): run iperf on TCP and set TCP windows to bdp (#241)
This commit makes iperf use TCP instead of UDP. Using UDP allows the sender to
send as much data as they can, not as much data as the receiver can handle.
Using TCP measures the actual bandwidth of the connection.

This commit sets the TCP send and receive window to the bandwidth-delay-product.
In order to exhaust the whole bandwidth between the two nodes, TCP needs to be
able to buffer all bytes on the wire in the case of packet loss.
2023-08-04 10:23:47 +02:00
Max Inden acea8b5e0c
fix(perf): increase MAX_AGE_MINUTES to 120 (#215)
- When experimenting with increasing iterations, instances are taken down
prematurely. See e.g. https://github.com/libp2p/test-plans/actions/runs/5418669631/jobs/9850992882

- When benchmarking from a local developer machine, experimenting for longer is useful.

- The AWS Lambda is only a safety net machanism. In the rare case where we
miss to clean up, running the machines for <1h or 2h doesn't matter cost wise.
2023-07-05 07:06:39 +02:00
Piotr Galar 1b187a379c
chore: move perf cleanup test code to a test directory (#216) 2023-07-05 05:46:58 +02:00
Max Inden 53ff8b5db1
feat(perf): add (automation, provision, build, run) tooling (#184)
This project includes the following components:

- `terraform/`: a Terraform scripts to provision infrastructure
- `impl/`: implementations of the [libp2p perf
  protocol](https://github.com/libp2p/specs/blob/master/perf/perf.md) running on
  top of e.g. go-libp2p, rust-libp2p or Go's std-library https stack
- `runner/`: a set of scripts building and running the above implementations on
  the above infrastructure, reporting the results in `benchmark-results.json`

Benchmark results can be visualized with
https://observablehq.com/@mxinden-workspace/libp2p-performance-dashboard.

Co-authored-by: Marco Munizaga <git@marcopolo.io>
Co-authored-by: Marten Seemann <martenseemann@gmail.com>
Co-authored-by: Piotr Galar <piotr.galar@gmail.com>
2023-06-22 13:54:09 +02:00