waku-org
/
nwaku
mirror of https://github.com/waku-org/nwaku.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

docs: extend waku store config tutorial of the operator docs

This commit is contained in:
Lorenzo Delgado 2022-11-10 15:37:25 +01:00 committed by GitHub
parent 47fe697b47
commit 4e5318bfbb
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 94 additions and 37 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

56
docs/operators/how-to/configure-store-v0.12.0.md Normal file
View File

@ -0,0 +1,56 @@
# Configure store protocol (versions prior to v0.13.0)
Store protocol is enabled by default on a nwaku node.
This is controlled by the `--store` CLI option.
```sh
# Disable store protocol on startup
./build/wakunode2 --store:false
```
Note that this only mounts the `store` protocol,
meaning your node will indicate to other peers that it supports `store`.
It does not yet allow your node to either retrieve historical messages as a client
or store and serve historical messages itself.
## Configuring a store client
Ensure that `store` is enabled (this is `true` by default) and provide at least one store service node address with the `--storenode` CLI option.
See the following example, using the peer at `/dns4/node-01.ac-cn-hongkong-c.wakuv2.test.statusim.net/tcp/30303/p2p/16Uiu2HAkvWiyFsgRhuJEb9JfjYxEkoHLgnUQmr1N5mKWnYjxYRVm` as store service node.
```sh
./build/wakunode2 \
--store:true \
--storenode:/dns4/node-01.ac-cn-hongkong-c.wakuv2.test.statusim.net/tcp/30303/p2p/16Uiu2HAkvWiyFsgRhuJEb9JfjYxEkoHLgnUQmr1N5mKWnYjxYRVm
```
Your node can now send queries to retrieve historical messages
from the configured store service node.
One way to trigger such queries is asking your node for historical messages using the [Waku v2 JSON RPC API](https://rfc.vac.dev/spec/16/).
## Configuring a store service node
To store historical messages on your node which can be served to store clients the `--persist-messages` CLI option must be enabled.
By default a node would store up to the latest `50 000` messages.
This is configurable using the `--store-capacity` option.
A node that has a `--db-path` set will backup historical messages to a local database at the DB path
and persist these messages even after a restart.
```sh
./build/wakunode2 \
--store:true \
--persist-messages:true \
--db-path:/mnt/nwaku/data/db1/ \
--store-capacity:150000
```
### How much resources should I allocate?
Currently store service nodes use an in-memory key-value store as primary storage with the disk-based database only used for backups.
Most Waku messages average a size of 1KB - 2KB,
implying a minimum memory requirement of at least ~250MB
for a medium capacity store of 100k messages.
Note, however, that the allowable maximum size for Waku messages is up to 1MB.
We are working on a disk-only and hybrid store to lower the memory requirement.
It will soon also be possible to configure store capacity on maximum store size or number of days' history to keep.

75
docs/operators/how-to/configure-store.md
View File

@ -1,56 +1,57 @@
# Configure store protocol
Store protocol is enabled by default on a nwaku node.
This is controlled by the `--store` CLI option.
> :information_source: This instructions apply to nwaku version v0.13.0+. For versions prior to v0.13.0, check [this page](./configure-store-v0.12.0.md).
```sh
# Disable store protocol on startup
./build/wakunode2 --store:false
The waku store protocol is enabled by default the nwaku node.
This is controlled by the `--store` option. To disable waku store protocol on startup, specify explicitly the `--store` option set to `false`:
```shell
wakunode2 --store=false
```
Note that this only mounts the `store` protocol,
meaning your node will indicate to other peers that it supports `store`.
It does not yet allow your node to either retrieve historical messages as a client
or store and serve historical messages itself.
This option controls the mounting of the Waku Store protocol, meaning that your node will indicate to other peers that it supports the Waku store protocol.
## Configuring a store client
## Configuring the node as a waku store client
Ensure that `store` is enabled (this is `true` by default) and provide at least one store service node address with the `--storenode` CLI option.
Provide at least one store service node address with the `--storenode` option. This option is independent of the `--store` option i.e., one node can act as a waku store client without mounting the Waku Store protocol.
See the following example, using the peer at `/dns4/node-01.ac-cn-hongkong-c.wakuv2.test.statusim.net/tcp/30303/p2p/16Uiu2HAkvWiyFsgRhuJEb9JfjYxEkoHLgnUQmr1N5mKWnYjxYRVm` as store service node.
For example, to use the peer at `/dns4/node-01.ac-cn-hongkong-c.wakuv2.test.statusim.net/tcp/30303/p2p/16Uiu2HAkvWiyFsgRhuJEb9JfjYxEkoHLgnUQmr1N5mKWnYjxYRVm` as the waku store service node:
```sh
./build/wakunode2 \
--store:true \
--storenode:/dns4/node-01.ac-cn-hongkong-c.wakuv2.test.statusim.net/tcp/30303/p2p/16Uiu2HAkvWiyFsgRhuJEb9JfjYxEkoHLgnUQmr1N5mKWnYjxYRVm
```shell
wakunode2 \
--storenode=/dns4/node-01.ac-cn-hongkong-c.wakuv2.test.statusim.net/tcp/30303/p2p/16Uiu2HAkvWiyFsgRhuJEb9JfjYxEkoHLgnUQmr1N5mKWnYjxYRVm
```
Your node can now send queries to retrieve historical messages
from the configured store service node.
One way to trigger such queries is asking your node for historical messages using the [Waku v2 JSON RPC API](https://rfc.vac.dev/spec/16/).
from the configured store service node. One way to trigger such queries is asking your node for historical messages using the [Waku v2 JSON RPC API](https://rfc.vac.dev/spec/16/).
## Configuring a store service node
## Configuring the node as a store service node
To store historical messages on your node which can be served to store clients the `--persist-messages` CLI option must be enabled.
By default a node would store up to the latest `50 000` messages.
This is configurable using the `--store-capacity` option.
A node that has a `--db-path` set will backup historical messages to a local database at the DB path
and persist these messages even after a restart.
If the waku store node is enabled (the `--store` option is set to `true`) the node will store historical messages and will be able to serve those messages to the waku store clients.
```sh
./build/wakunode2 \
--store:true \
--persist-messages:true \
--db-path:/mnt/nwaku/data/db1/ \
--store-capacity:150000
There is a set of configuration options to customize the waku store protocol's message store. These are the most relevant:
* `--store-message-retention-policy`: This option controls the retention policy i.e., how long certain messages will be persisted. Two different retention policies are supported:
+ The time retention policy,`time:<duration-in-seconds>` (e.g., `time:14400`)
+ The capacity retention policy,`capacity:<messages-count>` (e.g, `capacity:25000`)
+ To disable the retention policy, explicitly, set this option to to `""`, an empty string.
* `--store-message-db-url`: The message store database url option controls the message storage engine. This option follows the [_SQLAlchemy_ database URL format](https://docs.sqlalchemy.org/en/14/core/engines.html#database-urls).
+ SQLite engine: The only database engine supported by the nwaku node. The database URL has this shape: `sqlite://<database-file-path>`. If the `<database-file-path>` is not an absolute path (preceded by a `/` character), the file will be created in the current working directory. The SQLite engine also supports to select a non-persistent in-memory database by setting the `<database-file-path>` to `:memory:`.
+ In the case you don't want to use a persistent message store; set the `--store-message-db-url` to an empty string, `""`. This will instruct the node to use the fallback in-memory message store.
By default the node message store will be configured with a time retention policy set to `14400` seconds (4 hours). Additionaly, by default, the node message store will use the SQLite database engine to store historical messages in order to persist these between restarts.
> :warning: Note the 3 slashes, `///`, after the SQLite database URL schema. The third slash indicates that it is an absolute path: `/mnt/nwaku/data/db1/store.sqlite3`
```shell
wakunode2 \
--store=true \
--store-message-db-url=sqlite:///mnt/nwaku/data/db1/store.sqlite3 \
--store-message-retention-policy=capacity:150000
```
### How much resources should I allocate?
Currently store service nodes use an in-memory key-value store as primary storage with the disk-based database only used for backups.
Most Waku messages average a size of 1KB - 2KB,
implying a minimum memory requirement of at least ~250MB
for a medium capacity store of 100k messages.
Note, however, that the allowable maximum size for Waku messages is up to 1MB.
We are working on a disk-only and hybrid store to lower the memory requirement.
It will soon also be possible to configure store capacity on maximum store size or number of days' history to keep.
Currently store service nodes use, by default, a message store backed by an in-disk SQLite database. Most Waku messages average a size of 1KB - 2KB, implying a minimum memory requirement of at least ~250MB
for a typical store capacity of 100k messages. Note, however, that the allowable maximum size for Waku messages is up to 1MB.