storenode-messages-counter/README.md

# Storenode Message Counter

used to compare the number of Status messages across Status nodes to understand the potential discrepancies and odd behaviour of messages being inserted in the past in a given pubsub topic.

# Querying the data

The following tables are available in https://superset.bi.status.im/superset/sqllab/ `telemetry` database:

### `missingMessages`
This table contains the list of missing messages, and in which storenode the messages was identified to be missing. The `msgStatus` column indicates the possible scenarios in which a message can be considered missing:
- `does_not_exist`. The message is not stored in the storenode's db
- `unknown`. It was not possible to retrieve information about this message. The store node was likely not available during the time the query to verify the message status was done.

### `storeNodeUnavailable`
This table records timestamps on which a specific storenode was not available. This information should be used along with data from https://kibana.infra.status.im/ and https://grafana.infra.status.im/ to determine the reason why the storenode was not available.

# Dashboard
A grafana dashboard is available in https://grafana.infra.status.im/d/hmX6pJ_Iz/storenode-message-counter?orgId=1

# Browsing the logs

You can browse the logs in kibana at https://kibana.infra.status.im/goto/7342ed70-6162-11ef-bfa2-75c6d129a2cb

Assuming the link is not available use the following parameters:
- `logsource`: `node-01.he-eu-hel1.tmetry.misc`
- `program`: `docker/telemetry-counter-status-prod`, `docker/telemetry-counter-status-staging`

# Development

You need to setup a postgres db as such:
1) Create an user with a password
2) Create a db
3) Execute `make build`


Then you can run the program with
```
./build/storemsgcounter --storenode=some_multiaddress --storenode=some_multiaddress  --pubsub-topic=some_pubsubtopic --cluster-id=16 --db-url=postgres://user:password@127.0.0.1:5432/telemetry
```

A dockerfile is also available for ease of setup
```
docker build -t wakuorg/storenode-messages:latest .

docker run wakuorg/storenode-messages:latest --help
```