waku-org/nwaku
1
0
Fork 0
mirror of https://github.com/waku-org/nwaku.git synced 2025-02-22 11:58:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update to apply PR comment suggestions

Browse Source
This commit is contained in:
stubbsta 2025-02-10 07:34:47 +02:00
parent fb3b409bfc
commit adcdbee439
No known key found for this signature in database
1 changed files with 33 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

47
docs/benchmarks/test-results-summary.md
View File

@ -8,20 +8,19 @@ This page summarises key performance metrics for nwaku and provides links to det
> ## TL;DR > ## TL;DR
> >
> - libp2p bandwidth usage fluctuates between 5 and 15 KB/s for topologies of up to 1000 nodes, with average bandwidth usage at **10 KB/s**. > - Average Waku bandwidth usage: ~**10 KB/s** (minus discv5 Discovery) for 1KB message size and message injection rate of 1msg/s.
> - The average bandwidth usage remains roughly the same at **9 KB/s** for a larger topology of 2000 nodes. Confirmed for topologies of up to 2000 Relay nodes.
This is expected for Relay networks and the slight fluctuation could be due to simulation artifacts or chance differences in routing or connectivity between test runs. > - Average time for a message to propagate to 100% of nodes: **0.4s** for topologies of up to 2000 Relay nodes.
> - The average time for a message to propagate to 100% of nodes in topologies of up to 2000 Relay nodes is **0.4s**. > - Average per-node bandwidth usage of the discv5 protocol: **8 KB/s** for incoming traffic and **7.4 KB/s** for outgoing traffic,
> - The average per-node bandwidth usage of the discv5 protocol is **8 KB/s** for incoming traffic and **7.4 KB/s** for outgoing traffic. in a network with 100 continuously online nodes.
This is for a network with 100 continuously online nodes, sending 1KB messages at 1s intervals. > - Relevancy to Status App: **TODO**
> - Relevancy to Status App: TODO
## Insights ## Insights
### Relay Bandwidth Usage: nwaku v0.34.0 ### Relay Bandwidth Usage: nwaku v0.34.0
Average `libp2p` per-node bandwidth usage for various message injection rates into a Relay network of constant size `1000`. The average per-node `libp2p` bandwidth usage in a 1000-node Relay network with 1KB messages at varying injection rates.
The messages are all 1KB in size.
| Message Injection Rate | Average libp2p incoming bandwidth (KB/s) | Average libp2p outgoing bandwidth (KB/s) | | Message Injection Rate | Average libp2p incoming bandwidth (KB/s) | Average libp2p outgoing bandwidth (KB/s) |
|------------------------|------------------------------------------|------------------------------------------| |------------------------|------------------------------------------|------------------------------------------|
@ -29,13 +28,21 @@ The messages are all 1KB in size.
| 1 msg/10s | ~1.8 | ~1.9 | | 1 msg/10s | ~1.8 | ~1.9 |
### Message Propagation Latency: nwaku v0.34.0-rc1 ### Message Propagation Latency: nwaku v0.34.0-rc1
The results for the average time for messages to reach all nodes in different network configurations are shown below. The message propagation latency is measured as the total time for a message to reach all nodes.
For each simulation 600 messages of 1KB were sent at a message injection rate of 1msg/s. We compare the latency in different network configurations for the following simulation parameters:
- Total messages published: 600
- Message size: 1KB
- Message injection rate: 1msg/s
The different network configurations tested are:
- Relay Config: 1000 nodes with relay enabled
- Mixed Config: 210 nodes, consisting of bootstrap nodes, filter clients and servers, lightpush clients and servers, store nodes
- Non-persistent Relay Config: 500 persistent relay nodes, 10 store nodes and 100 non-persistent relay nodes
Click on a specific config to see the detailed test report. Click on a specific config to see the detailed test report.
| Config | Average Message Propagation Latency (s) | Max Message Propagation Latency (s)|
| Config | Avg Message Propagation Latency (s) | Max Message Propagation Latency (s)| |------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------|------------------------------------|
|------------------------------------------------------------------------------------------------------------------------------|-------------------------------------|------------------------------------|
| [Relay](https://www.notion.so/Waku-regression-testing-v0-34-1618f96fb65c803bb7bad6ecd6bafff9) (1000 nodes) | 0.05 | 1.6 | | [Relay](https://www.notion.so/Waku-regression-testing-v0-34-1618f96fb65c803bb7bad6ecd6bafff9) (1000 nodes) | 0.05 | 1.6 |
| [Mixed](https://www.notion.so/Mixed-environment-analysis-1688f96fb65c809eb235c59b97d6e15b) (210 nodes) | 0.0125 | 0.007 | | [Mixed](https://www.notion.so/Mixed-environment-analysis-1688f96fb65c809eb235c59b97d6e15b) (210 nodes) | 0.0125 | 0.007 |
| [Non-persistent Relay](https://www.notion.so/High-Churn-Relay-Store-Reliability-16c8f96fb65c8008bacaf5e86881160c) (510 nodes)| 0.0125 | 0.25 | | [Non-persistent Relay](https://www.notion.so/High-Churn-Relay-Store-Reliability-16c8f96fb65c8008bacaf5e86881160c) (510 nodes)| 0.0125 | 0.25 |
@ -52,24 +59,28 @@ The measurements are based on a stable network where all nodes have already conn
## Testing ## Testing
### DST ### DST
The VAC DST team performs regression testing on all new **nwaku** releases, comparing performance with previous versions. They simulate large Waku networks with a variety of network and protocol configurations that are representative of real-world usage. The VAC DST team performs regression testing on all new **nwaku** releases, comparing performance with previous versions.
They simulate large Waku networks with a variety of network and protocol configurations that are representative of real-world usage.
**Test Reports**: [DST Reports](https://www.notion.so/DST-Reports-1228f96fb65c80729cd1d98a7496fe6f) **Test Reports**: [DST Reports](https://www.notion.so/DST-Reports-1228f96fb65c80729cd1d98a7496fe6f)
### QA ### QA
The VAC QA team performs interoperability tests for **nwaku** and **go-waku** using the latest main branch builds. These tests run daily and verify protocol functionality by targeting specific features of each protocol. The VAC QA team performs interoperability tests for **nwaku** and **go-waku** using the latest main branch builds.
These tests run daily and verify protocol functionality by targeting specific features of each protocol.
**Test Reports**: [QA Reports](https://discord.com/channels/1110799176264056863/1196933819614363678) **Test Reports**: [QA Reports](https://discord.com/channels/1110799176264056863/1196933819614363678)
### nwaku ### nwaku
The **nwaku** team follows a structured release procedure for all release candidates. This involves deploying RCs to `status.staging` fleet for validation and performing sanity checks. The **nwaku** team follows a structured release procedure for all release candidates.
This involves deploying RCs to `status.staging` fleet for validation and performing sanity checks.
**Release Process**: [nwaku Release Procedure](https://github.com/waku-org/nwaku/blob/master/.github/ISSUE_TEMPLATE/prepare_release.md) **Release Process**: [nwaku Release Procedure](https://github.com/waku-org/nwaku/blob/master/.github/ISSUE_TEMPLATE/prepare_release.md)
### Research ### Research
The Waku Research team conducts a variety of benchmarking, performance testing, proof-of-concept validations and debugging efforts. They also maintaining a Waku simulator designed for small-scale, single-purpose, on-demand testing. The Waku Research team conducts a variety of benchmarking, performance testing, proof-of-concept validations and debugging efforts.
They also maintain a Waku simulator designed for small-scale, single-purpose, on-demand testing.
**Test Reports**: [Waku Research Reports](https://www.notion.so/Miscellaneous-2c02516248db4a28ba8cb2797a40d1bb) **Test Reports**: [Waku Research Reports](https://www.notion.so/Miscellaneous-2c02516248db4a28ba8cb2797a40d1bb)