mirror of
https://github.com/logos-messaging/docs.waku.org.git
synced 2026-01-02 12:53:12 +00:00
chore: misc. content and style updates on research
This commit is contained in:
parent
1a988a7ba6
commit
fedb551f4c
@ -1,8 +1,10 @@
|
||||
---
|
||||
title: Postgres adoption
|
||||
title: PostgreSQL
|
||||
description: Document that describes why Nim-Waku started to use Postgres and shows some benchmark and comparison results.
|
||||
---
|
||||
|
||||
## Introduction
|
||||
|
||||
The *Nim Waku Node*, *nwaku*, has the capability of archiving messages until a certain limit (e.g. 30 days) so that other nodes can synchronize their message history throughout the *Store* protocol.
|
||||
|
||||
The *nwaku* originally used *SQLite* to archive messages but this has an impact on the node. *Nwaku* is single-threaded and therefore, any *SQLite* operation impacts the performance of other protocols, like *Relay.*
|
||||
@ -11,7 +13,7 @@ Therefore, the *Postgres* adoption is needed to enhance that.
|
||||
|
||||
[https://github.com/waku-org/nwaku/issues/1888](https://github.com/waku-org/nwaku/issues/1888)
|
||||
|
||||
# How to connect the *nwaku* to *Postgres*
|
||||
## How to connect the *nwaku* to *Postgres*
|
||||
|
||||
Simply pass the next parameter to *nwaku*
|
||||
|
||||
@ -26,21 +28,21 @@ Notice that this only makes sense if the _nwaku_ has the _Store_ protocol mounte
|
||||
|
||||
(start the _nwaku_ node with `--help` parameter for more _Store_ options)
|
||||
|
||||
# Examples of *nwaku* using *Postgres*
|
||||
## Examples of *nwaku* using *Postgres*
|
||||
|
||||
[https://github.com/waku-org/nwaku-compose](https://github.com/waku-org/nwaku-compose)
|
||||
|
||||
[https://github.com/waku-org/test-waku-query](https://github.com/waku-org/test-waku-query)
|
||||
|
||||
# Stress tests
|
||||
## Stress tests
|
||||
|
||||
The following repository was created as a tool to stress and compare performance between *nwaku*+*Postgres* and *nwaku*+*SQLite*:
|
||||
|
||||
[https://github.com/waku-org/test-waku-query](https://github.com/waku-org/test-waku-query)
|
||||
|
||||
## Insert test results
|
||||
### Insert test results
|
||||
|
||||
### Maximum insert throughput
|
||||
#### Maximum insert throughput
|
||||
|
||||
**Scenario**
|
||||
|
||||
@ -64,7 +66,7 @@ The reason why few messages were lost is because the message rate was higher tha
|
||||
|
||||
As a conclusion, the bottleneck is within the *Relay* protocol itself and not the underlying databases. Or, in other words, both *SQLite* and *Postgres* can support the maximum insert rate a Waku node will operate within normal conditions.
|
||||
|
||||
## Query test results (jmeter)
|
||||
### Query test results (jmeter)
|
||||
|
||||
In this case, we are comparing *Store* performance by means of Rest service.
|
||||
|
||||
@ -88,7 +90,7 @@ With this, the *node_b* brings a higher throughput than the *node_a* and that in
|
||||
|
||||

|
||||
|
||||
## Query test results (only Store protocol)
|
||||
### Query test results (only Store protocol)
|
||||
|
||||
In this test suite, only the Store protocol is being analyzed, i.e. without REST. For that, a go-waku node is used, which acts as *Store* client. On the other hand, we have another go-waku app that publishes random *Relay* messages periodically. Therefore, this can be considered a more realistic approach.
|
||||
|
||||
@ -105,7 +107,7 @@ That topology is defined in [this](https://github.com/waku-org/test-waku-query/b
|
||||
|
||||
Notice that the two `nwaku` nodes run the very same version, which is compiled locally.
|
||||
|
||||
### Comparing archive SQLite & Postgres performance in [nwaku-b6dd6899](https://github.com/waku-org/nwaku/tree/b6dd6899030ee628813dfd60ad1ad024345e7b41)
|
||||
#### Comparing archive SQLite & Postgres performance in [nwaku-b6dd6899](https://github.com/waku-org/nwaku/tree/b6dd6899030ee628813dfd60ad1ad024345e7b41)
|
||||
|
||||
The next results were obtained by running the docker-compose-manual-binaries.yml from [test-waku-queryc078075](https://github.com/waku-org/test-waku-query/tree/c07807597faa781ae6c8c32eefdf48ecac03a7ba) in the sandbox machine (metal-01.he-eu-hel1.wakudev.misc.statusim.net.)
|
||||
|
||||
@ -149,7 +151,7 @@ In this case, the performance is similar regarding the timings. The store rate i
|
||||
|
||||

|
||||
|
||||
### Comparing archive SQLite & Postgres performance in [nwaku-b452ed8](https://github.com/waku-org/nwaku/tree/b452ed865466a33b7f5b87fa937a8471b28e466e)
|
||||
#### Comparing archive SQLite & Postgres performance in [nwaku-b452ed8](https://github.com/waku-org/nwaku/tree/b452ed865466a33b7f5b87fa937a8471b28e466e)
|
||||
|
||||
This nwaku commit is after a few **Postgres** optimizations were applied.
|
||||
|
||||
@ -187,7 +189,7 @@ It cannot be appreciated but the average *****Store***** time was 11ms.
|
||||
|
||||

|
||||
|
||||
### Conclusions
|
||||
#### Conclusions
|
||||
|
||||
After comparing both systems, *SQLite* performs much better than *Postgres* However, a benefit of using *Postgres* is that it performs asynchronous operations, and therefore doesn’t consume CPU time that would be better invested in *Relay* for example.
|
||||
|
||||
@ -199,7 +201,7 @@ Notice that we usually have a rate below 1100 req/minute in _status.prod_ fleet
|
||||
|
||||
-----------------------------
|
||||
|
||||
## Multiple nodes & one single database
|
||||
### Multiple nodes & one single database
|
||||
|
||||
This study aims to look for possible issues when having only one single database while several Waku nodes insert or retrieve data from it.
|
||||
The following diagram shows the scenery used for such analysis.
|
||||
@ -213,7 +215,7 @@ ERR 2023-11-27 13:18:07.575+00:00 failed to insert message top
|
||||
|
||||
The `db-postgres-hammer` is aimed to stress the database from the `select` point of view. It performs `N` concurrent `select` queries with a certain rate.
|
||||
|
||||
### Results
|
||||
#### Results
|
||||
|
||||
The following results were obtained by using the sandbox machine (metal-01.he-eu-hel1.wakudev.misc) and running nim-waku nodes from https://github.com/waku-org/nwaku/tree/b452ed865466a33b7f5b87fa937a8471b28e466e and using the `test-waku-query` project from https://github.com/waku-org/test-waku-query/tree/fef29cea182cc744c7940abc6c96d38a68739356
|
||||
|
||||
|
||||
@ -32,3 +32,7 @@ html[data-theme="dark"] .header-github-link:before {
|
||||
.hidden {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.theme-doc-toc-desktop {
|
||||
display: none;
|
||||
}
|
||||
Loading…
x
Reference in New Issue
Block a user