status-im/infra-status
2
0
Fork 0
mirror of https://github.com/status-im/infra-status.git synced 2025-02-10 14:26:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
125 Commits 5 Branches 0 Tags 820 KiB
Commit Graph

69 Commits

Author SHA1 Message Date
Jakub Sokołowski
f869ae1199
node-db: fix shared memory calculation 
We were casting `0.3` to `int` which results in `0`.

Resolves:
https://github.com/status-im/infra-waku/issues/28

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-10-30 11:42:14 +01:00
markoburcul
de6c49db9d boot: update sonda and nim waku networking 
Define docker default network for nim-waku on boot nodes and use that
network when deploying sonda. Remove existing nftables rule for opening
nim waku rest api port since it is not needed anymore.

Signed-off-by: markoburcul <marko@status.im>
 2024-10-30 10:01:47 +01:00
markoburcul
d319a1e405 boot-node: add sonda to the host 
Referenced issue: https://github.com/status-im/infra-hq/issues/135

Signed-off-by: markoburcul <marko@status.im>
 2024-10-29 13:21:11 +01:00
Jakub Sokołowski
d2e1b8f880
all: replace old naming of hq.metrics fleet 
https://github.com/status-im/infra-hq/issues/158

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-10-18 12:49:25 +02:00
Jakub Sokołowski
2675759faa
all: drop bootstrap__firewall_nftables=true 
No longer necessary, iptables support was dropped.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-10-17 12:47:22 +02:00
stubbsta
d1bae047ed remove store sync protocol becasue network is crashing 2024-10-14 13:57:47 +05:30
stubbsta
f464107b39
Add store-sync protocol for staging store nodes 2024-10-10 07:43:43 +02:00
stubbsta
64b2d5927c
Add ssh access to store node for tanya 2024-10-03 09:42:19 +02:00
Anton Iakimov
f8c8dac98d
boot,store,store-db: switch to nftables 
https://github.com/status-im/infra-misc/issues/301
 2024-09-12 16:40:48 +02:00
Jakub Sokołowski
901a62f455
store,boot: bump max connections from 300 to 500 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-09-05 18:17:16 +02:00
Alexis Pentori
ecf29207bd
store-db: increase container share memory 
Increase share memory to allow vacuuming

Signed-off-by: Alexis Pentori <alexis@status.im>
 2024-09-05 14:40:32 +02:00
Jakub Sokołowski
5649191b4f
all: add Harbor Docker registry credentials 
Otherwise hosts create a lot of `/v2/` calls that fail with 401.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-09-03 09:31:31 +02:00
Ivan Folgueira Bande
d8d4d5d890
store: debug log level for all stages 
Having the log level in trace severely damaged the node's performance
and disk usage. It wasn't a great idea.

Signed-off-by: Ivan Folgueira Bande <ivansete@status.im>
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-08-29 09:17:46 +02:00
Alexis Pentori
6a730e7d4d
logrotate: update frequency 
Signed-off-by: Alexis Pentori <alexis@status.im>
 2024-08-28 09:47:36 +02:00
Ivan Folgueira Bande
96ec175dd6 store staging: set nim_waku_log_level to trace 2024-08-26 15:32:51 +02:00
Ivan Folgueira Bande
46146c87b0
store-db: update db settings to new hardware 16GB RAM and 8 CPUs 2024-08-16 17:30:02 +02:00
Ivan Folgueira Bande
87d2fc1605
store-db: disable autovacuum by default 
The bigger table, messages, is a partitioned one where only INSERTS
happens. Furthermore, there are no dead tuples there because we
directly drop old partitions

We may need to manually perform vacuum on other tables
 2024-08-16 17:29:37 +02:00
Jakub Sokołowski
f32d99fb06
store-db: increase consul alert tresholds 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-08-11 15:24:26 +02:00
Jakub Sokołowski
491b6d37b6
boot: increase consul check interval, warning threshold 
To match settings for `store` nodes.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-08-11 15:00:14 +02:00
Alexis Pentori
af9162fc2e
store-db: applying postgres system setting based on the stage 
Signed-off-by: Alexis Pentori <alexis@status.im>
 2024-08-08 12:23:43 +02:00
Ivan Folgueira Bande
f85fc71b50
store-db add more appropriate db settings for current db hw 
Signed-off-by: Ivan Folgueira Bande <ivansete@status.im>
 2024-08-07 13:27:29 +02:00
stubbsta
e6b39e4b8f
store-db: add SSH access for tanya@status.im 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-08-02 09:12:59 +02:00
Hanno Cornelius
91117c823c
store-db: add SSH access for hanno@status.im 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-26 09:08:33 +02:00
Jakub Sokołowski
eb8045326c
boot,store: fix ENRTree DNS entry for status.prod 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-25 09:14:40 +02:00
Jakub Sokołowski
7df38c149d
rename the shards.test fleet to status.prod 
https://github.com/status-im/infra-shards/issues/33

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-24 12:13:50 +02:00
Jakub Sokołowski
55b31f42f5
all: do not send trace level logs to logstash 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-23 12:12:54 +02:00
Jakub Sokołowski
5591327ea3
store: lower staging retention to 75 GB to avoid alerts 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-17 09:26:42 +02:00
Jakub Sokołowski
770dad967e
store,boot: fix name of Docker tag name 
We are in the middle of renaming fleets and this will make it more
robust.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-16 16:55:32 +02:00
Jakub Sokołowski
749c281209
store-db: fix variable name for Postgres Alter System 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-16 13:42:51 +02:00
Jakub Sokołowski
f554fe7185
store-db: set autovacuum_work_mem to 10% of memory 
We have seen host crashes caused by PostgreSQL using up all memory by
trying to run `autovacuum` workers.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-16 13:42:50 +02:00
Jakub Sokołowski
97544ad634
store: set retention policy using size instead of time 
Using time causes the DB to be filled quickly.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-09 11:16:38 +02:00
Jakub Sokołowski
040b9d4949
rename shards fleet to status fleet 
While also retaining the old domain names.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-03 22:00:29 +02:00
Jakub Sokołowski
b1da421448
boot: uncomment setting for boot node key 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-03 01:19:08 +02:00
Ivan Folgueira Bande
062cb6d51a
set max_locks_per_transaction to 2160 
We are using partitions in our postgres DBs. And we have one
partition per hour (24 partitions per day.)

The default max_locks_per_transaction value (64) can cause
"our of memory" and block issues in the DB because we use to
have more than 64 partitions.

With 2160 we aim to avoid that issue for 90 days (2160 == 90*24.)
if we consider a time retention policy of 90 days. Nevertheless,
we usually have time retention policy of 30 days in our Status fleets,
but we are just adding some extra margin.
 2024-06-26 13:15:24 +02:00
Alexis Pentori
7178fc4d83
store: set logrotate frequency to hourly 
Signed-off-by: Alexis Pentori <alexis@status.im>
 2024-06-10 10:31:15 +02:00
Ivan Folgueira Bande
dc9b6f5a81
boot: set max msg size to 1024KiB to fit store nodes 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-06-03 10:21:08 +02:00
Jakub Sokołowski
e1b4be4a24
store: un-command nim_waku_node_key variable 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-05-23 18:10:15 +02:00
Jakub Sokołowski
aa3e653a53
store: lower sensitivity of consul healthchecks 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-05-12 10:30:30 +02:00
Anton Iakimov
f39afef54d
boot: logrotate hourly due to lots of DBG logs 2024-04-24 16:01:04 +02:00
Jakub Sokołowski
883893f547
deploy new shards.staging fleet 
https://github.com/status-im/infra-shards/issues/29

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-03-18 20:48:58 +01:00
Jakub Sokołowski
3c60a6dcde
boot,store: go back to using proper deploy branches 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-03-15 14:27:38 +01:00
Jakub Sokołowski
74be1115c6
boot,store: use both new and old domain names 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-03-15 14:27:37 +01:00
Jakub Sokołowski
717b37aa0c
node: expose config.toml using Nginx server 
This can then be linked from the new https://fleets.waku.org/.

https://github.com/status-im/infra-misc/issues/229

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-03-07 12:44:09 +01:00
Jakub Sokołowski
bde743c656
boot,store: add /waku/2/rs/16/1 topic 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-03-04 22:09:38 +01:00
Jakub Sokołowski
1e025a18ff
boot,store: temporarily lock image at v0.24.0 
Attempt to upgrade to 0.25.0 caused major connectivity issues.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-02-29 20:11:22 +01:00
Jakub Sokołowski
6004610d63
boot,store: add cluster ID required for 0.25.0 
16 is the value "reserved" for status fleets with static sharding
https://rfc.vac.dev/spec/51/#static-sharding

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-02-29 12:34:10 +01:00
Jakub Sokołowski
1813cf46ca
store: set max-msg-size to 1024KiB 
https://github.com/waku-org/nwaku/issues/2305

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-02-01 13:41:18 +01:00
Anton Iakimov
abe3642480
nim-waku: add --ip-colocation-limit flag 
https://github.com/status-im/infra-shards/issues/27
 2024-01-24 15:25:34 +01:00
Jakub Sokołowski
8a94903ad0
store-db: disable database backups 
We don't need them currently.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-09 14:53:23 +01:00
Jakub Sokołowski
32662b3bd8
store-db: add Write Ahead Log size limits 
Related to:
https://github.com/status-im/infra-role-postgres-ha/issues/3

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-09 14:05:26 +01:00