status-im
/
infra-nimbus
mirror of https://github.com/status-im/infra-nimbus.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
841 Commits 10 Branches 0 Tags 31 MiB
Commit Graph

841 Commits

Author SHA1 Message Date
Jakub Sokołowski 766092f537
nimbus.prater: fix validator monitor details condition 
It was causing very high cardinality due to lots of validators.
Crashed our Hetzner metrics server.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-22 12:07:16 +02:00
Jakub Sokołowski 7d9ebb1744
nimbus.prater: double beacon node open files limit 
Causes issues with metrics responses due to inability to open more files.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-22 09:31:22 +02:00
Jakub Sokołowski 276efe33cf
requirements: bump oauth-proxy and elasticsearch-lb 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-21 17:14:26 +02:00
Jakub Sokołowski 5f151fb828
all: grant admin rights to cheatfate user 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-21 16:50:17 +02:00
Jakub Sokołowski 6f0ff4ad19
nimbus.prater: run only two nodes on MacOS host 
Not enough disk space for 3 nodes.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-20 19:47:20 +02:00
Jakub Sokołowski 666cde43bb
requirements: bump oauth to 7.4.0 and geth to 1.12.0 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-16 13:05:57 +02:00
Jakub Sokołowski 1aa507c359
prater: move validators to new linux-01 and linux-02 
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-16 12:57:24 +02:00
Jakub Sokołowski 43b238c87e
nimbus.all: use new --validator-monitor-details flag 
Replaced `--validator-monitor-totals` in:
https://github.com/status-im/nimbus-eth2/pull/4468

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-07 19:04:37 +02:00
Etan Kissling b33a35c5e1
nimbus: set --rest-allow-origin='*' for public_api 
To allow browser based clients to access our API servers.

Depends on:

- https://github.com/status-im/infra-role-beacon-node-linux/pull/14
- https://github.com/status-im/infra-role-beacon-node-linux/pull/15
- https://github.com/status-im/infra-role-beacon-node-macos/pull/7
- https://github.com/status-im/infra-role-beacon-node-macos/pull/8
- https://github.com/status-im/infra-role-beacon-node-windows/pull/6
- https://github.com/status-im/infra-role-beacon-node-windows/pull/7

Signed-off-by: Etan Kissling <etan@status.im>
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-05 13:45:55 +02:00
Jakub Sokołowski 0441d524a9
drop linux-01.he-eu-hel1.nimbus.sepolia host 
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-05 12:08:43 +02:00
Jakub Sokołowski e08586be04
nimbus.sepolia: move validators to new Innova host 
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-05 12:08:01 +02:00
Jakub Sokołowski 10e0c7d74a
requirements: fix nimbus-eth1 folder creation 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-02 17:31:52 +02:00
Jakub Sokołowski 96a96de630
nimbus-sepolia-trial: add new Innova host 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-02 14:26:46 +02:00
Jakub Sokołowski 6ae7dc00f0
nimbus.prater: add Innova host config 
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-02 10:35:18 +02:00
Jakub Sokołowski da40f535ba
nimbus.sepolia: add Innova host config 
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-02 10:34:20 +02:00
Jakub Sokołowski 6618658396
add linux-01 and linux-02 for nimbus.prater 
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-02 10:28:57 +02:00
Jakub Sokołowski d3bb14a070
add linux-01.ih-eu-mda1.nimbus.sepolia host 
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-02 10:27:20 +02:00
Jakub Sokołowski 15a3e6968f
mainnet: add comment on Innova server hardware 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-30 20:44:27 +02:00
Jakub Sokołowski 948b5885fe
nimbus.prater: move disabling discovery to Hetzner group 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-29 21:20:03 +02:00
Jakub Sokołowski e9a05d73ab
nimbus.prater: add DNS discovery for Erigon 
Necessary because we disable normal discovery.

Source:
https://github.com/ethereum/discv4-dns-lists

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-29 21:03:53 +02:00
Jakub Sokołowski a8741b7019
nimbus.prater: , disable discovery for Erigon 
It appears peer discovery can be detected by hetzner as an attempt at
network scanning:
https://github.com/ledgerwatch/erigon/issues/7234

Which can lead to the server being shut down.
For this reason I'm lowering max peers and disabling discovery.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-29 20:06:58 +02:00
Jakub Sokołowski 3f3d763990
nimbus.prater: double nofile limit to 32768 
Which needs to be more than 4 x 7899 validators.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-29 18:36:09 +02:00
Jakub Sokołowski e5324beaaa
nimbus-geth-goerli: remove one of three Geth nodes 
Not nough storage space to run 3 nodes on the hosts.

Also made AWS, Windows and MacOS nodes discover available
Geth nodes using Consul catalog.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-24 18:34:03 +02:00
Jakub Sokołowski 001baf0367
requirements: bump beacon node roles, use prune 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-23 19:20:24 +02:00
Jakub Sokołowski 597b85a0c8
nimbus.sepolia: open EL WebSocket to wakuv2.test 
https://github.com/waku-org/nwaku/issues/1624
https://github.com/status-im/infra-nim-waku/pull/69

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-17 18:15:47 +02:00
Jakub Sokołowski 2e14ecc79e
nimbus.sepolia: enable SMART metrics service 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-17 18:11:27 +02:00
Jakub Sokołowski c15e1ea358
nimbus.mainnet: 10k max peers for stable-02@linux-03 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-16 12:38:24 +02:00
Jakub Sokołowski a270f4044c
all: get Consul certificates from BitWarden 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-16 12:14:38 +02:00
Jakub Sokołowski 0cd6de1ea6
nimbus.mainnet: use ncli_db from stable node 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-16 12:14:27 +02:00
Jakub Sokołowski 5e790e8585
nimbus-era-files: fix trailing newline in launchd 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-16 12:14:09 +02:00
Jakub Sokołowski d04a98ea11
remove metal-01.he-eu-hel1.nimbus.mainnet 
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-16 11:05:30 +02:00
Jakub Sokołowski 05f98abd53
nimbus.prater: use more CL endpoints on MacOS 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-16 11:05:23 +02:00
Jakub Sokołowski 4dbb3c165c
nimbus.mainnet: fix config of ERA files hosting 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-16 10:53:21 +02:00
Jakub Sokołowski 116adbaaf9
geth: bump data volume size to 2.5 TB 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-15 20:00:16 +02:00
Jakub Sokołowski a145963a2a
ih-eu-mda1: disable Consul check for SMART metrics 
The service fails on InnovaHosting servers due to HBA controller:
https://github.com/prometheus-community/smartctl_exporter/issues/26

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-15 19:51:41 +02:00
Jakub Sokołowski 9102e1954a
nimbus.sepolia: fix exposed REST API endpoint 
Missing redirect config.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-15 16:19:19 +02:00
Jakub Sokołowski 9a3f1f8b4c
nimbus.mainnet: fix filenames for host_vars 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-15 13:24:52 +02:00
Jakub Sokołowski 0256a21dd0
nimbus.mainnet: add two more hosts from Innova 
The `linux-06` is a new host, while `linux-07` is re-purposed `linux-01`
from the prater fleet. This is done because the Innova company is out of
Xeon E5-2690 CPUs, and we want consistent performance across the fleet.

This way all seven hosts in `nimbus.mainnet` will use the E5-2690 CPU.

https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-09 15:07:22 +02:00
Jakub Sokołowski 406ac58f16
nimbus.mainnet: use --no-el for even number nodes 
Attempted fix issues with syncing where latency on the Geth endpoint is
too high for beacon nodes to finish syncing properly.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-05 20:58:53 +02:00
Jakub Sokołowski 01cf22c9f8
get-geth-api-urls: rename variables for clarity 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-05-05 19:21:49 +02:00
Jakub Sokołowski 5635b39e50
mainnet: decommission 6 Hetzner hosts 
Being replaced by InnovaHosting servers
https://github.com/status-im/infra-nimbus/issues/132

I'm leaving old `metal-07` as `metal-01` since we don't have a server
with `libp2p` nodes on mainnet yet.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-04-28 12:22:46 +02:00
Jakub Sokołowski 87517a2823
mainnet.tf: add comments with hostnames 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-04-28 11:28:27 +02:00
Jakub Sokołowski cd0841eba2
ih-eu-mda1: fix destintion DC for sending logs 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-04-25 18:06:00 +02:00
Jakub Sokołowski e43ede93a5
nimbus.mainnet: fix exec layer URL logic 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-04-20 22:28:26 +02:00
Jakub Sokołowski 940b2577da
nimbus.mainnet: add node layout for new Innova hosts 
Once they are synced and working we'll remove the Hetzner ones.

https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-04-18 18:14:53 +02:00
Jakub Sokołowski 3c0348cb81
add linux-05.ih-eu-mda1.nimbus.mainnet host 
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-04-18 18:13:00 +02:00
Jakub Sokołowski 1c177eb944
add linux-04.ih-eu-mda1.nimbus.mainnet host 
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-04-18 17:58:48 +02:00
Jakub Sokołowski 8ddbd86139
nimbus.mainnet: revert removing of stable-02 node 
This reverts commit ab266d132d.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-04-17 20:06:57 +02:00
Jakub Sokołowski 53ccd34927
nimbus.prater: bootstrap new Innova servers 
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-04-17 17:11:27 +02:00
Jakub Sokołowski 1f5ad05893
nimbus.fluffy: 24h to make healthcheck critical 
This is not a critical service and there's a LOT of sevices that cause a
lot of alerts when developers are stopping and starting them.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-04-12 19:28:37 +02:00