Commit Graph

1045 Commits

Author SHA1 Message Date
Jakub Sokołowski 5b85f5c26d
add three InnovaHosting servers to nimbus.geth fleet
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-23 21:15:19 +02:00
Jakub Sokołowski de28d46133
nimbus.mainnet: enable yamux for 2 libp2p nodes
https://github.com/status-im/infra-nimbus/issues/148

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-23 15:14:42 +02:00
Jakub Sokołowski 052a8485aa
nimubs.prater: add 4 new hosts from InnovaHosting
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-22 20:44:31 +02:00
Jakub Sokołowski e91d72114e
requirements: fix format of valitaror monitor flags
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-22 12:16:08 +02:00
Jakub Sokołowski 766092f537
nimbus.prater: fix validator monitor details condition
It was causing very high cardinality due to lots of validators.
Crashed our Hetzner metrics server.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-22 12:07:16 +02:00
Jakub Sokołowski 7d9ebb1744
nimbus.prater: double beacon node open files limit
Causes issues with metrics responses due to inability to open more files.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-22 09:31:22 +02:00
Jakub Sokołowski 276efe33cf
requirements: bump oauth-proxy and elasticsearch-lb
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-21 17:14:26 +02:00
Jakub Sokołowski 5f151fb828
all: grant admin rights to cheatfate user
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-21 16:50:17 +02:00
Jakub Sokołowski 6f0ff4ad19
nimbus.prater: run only two nodes on MacOS host
Not enough disk space for 3 nodes.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-20 19:47:20 +02:00
Jakub Sokołowski 666cde43bb
requirements: bump oauth to 7.4.0 and geth to 1.12.0
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-16 13:05:57 +02:00
Jakub Sokołowski 1aa507c359
prater: move validators to new linux-01 and linux-02
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-16 12:57:24 +02:00
Jakub Sokołowski 43b238c87e
nimbus.all: use new --validator-monitor-details flag
Replaced `--validator-monitor-totals` in:
https://github.com/status-im/nimbus-eth2/pull/4468

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-07 19:04:37 +02:00
Etan Kissling b33a35c5e1
nimbus: set --rest-allow-origin='*' for public_api
To allow browser based clients to access our API servers.

Depends on:

- https://github.com/status-im/infra-role-beacon-node-linux/pull/14
- https://github.com/status-im/infra-role-beacon-node-linux/pull/15
- https://github.com/status-im/infra-role-beacon-node-macos/pull/7
- https://github.com/status-im/infra-role-beacon-node-macos/pull/8
- https://github.com/status-im/infra-role-beacon-node-windows/pull/6
- https://github.com/status-im/infra-role-beacon-node-windows/pull/7

Signed-off-by: Etan Kissling <etan@status.im>
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-05 13:45:55 +02:00
Jakub Sokołowski 0441d524a9
drop linux-01.he-eu-hel1.nimbus.sepolia host
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-05 12:08:43 +02:00
Jakub Sokołowski e08586be04
nimbus.sepolia: move validators to new Innova host
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-05 12:08:01 +02:00
Jakub Sokołowski 10e0c7d74a
requirements: fix nimbus-eth1 folder creation
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-02 17:31:52 +02:00
Jakub Sokołowski 96a96de630
nimbus-sepolia-trial: add new Innova host
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-02 14:26:46 +02:00
Jakub Sokołowski 6ae7dc00f0
nimbus.prater: add Innova host config
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-02 10:35:18 +02:00
Jakub Sokołowski da40f535ba
nimbus.sepolia: add Innova host config
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-02 10:34:20 +02:00
Jakub Sokołowski 6618658396
add linux-01 and linux-02 for nimbus.prater
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-02 10:28:57 +02:00
Jakub Sokołowski d3bb14a070
add linux-01.ih-eu-mda1.nimbus.sepolia host
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-02 10:27:20 +02:00
Jakub Sokołowski 15a3e6968f
mainnet: add comment on Innova server hardware
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-30 20:44:27 +02:00
Jakub Sokołowski 948b5885fe
nimbus.prater: move disabling discovery to Hetzner group
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-29 21:20:03 +02:00
Jakub Sokołowski e9a05d73ab
nimbus.prater: add DNS discovery for Erigon
Necessary because we disable normal discovery.

Source:
https://github.com/ethereum/discv4-dns-lists

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-29 21:03:53 +02:00
Jakub Sokołowski a8741b7019
nimbus.prater: , disable discovery for Erigon
It appears peer discovery can be detected by hetzner as an attempt at
network scanning:
https://github.com/ledgerwatch/erigon/issues/7234

Which can lead to the server being shut down.
For this reason I'm lowering max peers and disabling discovery.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-29 20:06:58 +02:00
Jakub Sokołowski 3f3d763990
nimbus.prater: double nofile limit to 32768
Which needs to be more than 4 x 7899 validators.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-29 18:36:09 +02:00
Jakub Sokołowski e5324beaaa
nimbus-geth-goerli: remove one of three Geth nodes
Not nough storage space to run 3 nodes on the hosts.

Also made AWS, Windows and MacOS nodes discover available
Geth nodes using Consul catalog.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-24 18:34:03 +02:00
Jakub Sokołowski 001baf0367
requirements: bump beacon node roles, use prune
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-23 19:20:24 +02:00
Jakub Sokołowski 597b85a0c8
nimbus.sepolia: open EL WebSocket to wakuv2.test
https://github.com/waku-org/nwaku/issues/1624
https://github.com/status-im/infra-nim-waku/pull/69

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-17 18:15:47 +02:00
Jakub Sokołowski 2e14ecc79e
nimbus.sepolia: enable SMART metrics service
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-17 18:11:27 +02:00
Jakub Sokołowski c15e1ea358
nimbus.mainnet: 10k max peers for stable-02@linux-03
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 12:38:24 +02:00
Jakub Sokołowski a270f4044c
all: get Consul certificates from BitWarden
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 12:14:38 +02:00
Jakub Sokołowski 0cd6de1ea6
nimbus.mainnet: use ncli_db from stable node
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 12:14:27 +02:00
Jakub Sokołowski 5e790e8585
nimbus-era-files: fix trailing newline in launchd
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 12:14:09 +02:00
Jakub Sokołowski d04a98ea11
remove metal-01.he-eu-hel1.nimbus.mainnet
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 11:05:30 +02:00
Jakub Sokołowski 05f98abd53
nimbus.prater: use more CL endpoints on MacOS
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 11:05:23 +02:00
Jakub Sokołowski 4dbb3c165c
nimbus.mainnet: fix config of ERA files hosting
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 10:53:21 +02:00
Jakub Sokołowski 116adbaaf9
geth: bump data volume size to 2.5 TB
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-15 20:00:16 +02:00
Jakub Sokołowski a145963a2a
ih-eu-mda1: disable Consul check for SMART metrics
The service fails on InnovaHosting servers due to HBA controller:
https://github.com/prometheus-community/smartctl_exporter/issues/26

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-15 19:51:41 +02:00
Jakub Sokołowski 9102e1954a
nimbus.sepolia: fix exposed REST API endpoint
Missing redirect config.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-15 16:19:19 +02:00
Jakub Sokołowski 9a3f1f8b4c
nimbus.mainnet: fix filenames for host_vars
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-15 13:24:52 +02:00
Jakub Sokołowski 0256a21dd0
nimbus.mainnet: add two more hosts from Innova
The `linux-06` is a new host, while `linux-07` is re-purposed `linux-01`
from the prater fleet. This is done because the Innova company is out of
Xeon E5-2690 CPUs, and we want consistent performance across the fleet.

This way all seven hosts in `nimbus.mainnet` will use the E5-2690 CPU.

https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-09 15:07:22 +02:00
Jakub Sokołowski 406ac58f16
nimbus.mainnet: use --no-el for even number nodes
Attempted fix issues with syncing where latency on the Geth endpoint is
too high for beacon nodes to finish syncing properly.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-05 20:58:53 +02:00
Jakub Sokołowski 01cf22c9f8
get-geth-api-urls: rename variables for clarity
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-05 19:21:49 +02:00
Jakub Sokołowski 5635b39e50
mainnet: decommission 6 Hetzner hosts
Being replaced by InnovaHosting servers
https://github.com/status-im/infra-nimbus/issues/132

I'm leaving old `metal-07` as `metal-01` since we don't have a server
with `libp2p` nodes on mainnet yet.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-28 12:22:46 +02:00
Jakub Sokołowski 87517a2823
mainnet.tf: add comments with hostnames
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-28 11:28:27 +02:00
Jakub Sokołowski cd0841eba2
ih-eu-mda1: fix destintion DC for sending logs
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-25 18:06:00 +02:00
Jakub Sokołowski e43ede93a5
nimbus.mainnet: fix exec layer URL logic
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-20 22:28:26 +02:00
Jakub Sokołowski 940b2577da
nimbus.mainnet: add node layout for new Innova hosts
Once they are synced and working we'll remove the Hetzner ones.

https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-18 18:14:53 +02:00
Jakub Sokołowski 3c0348cb81
add linux-05.ih-eu-mda1.nimbus.mainnet host
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-18 18:13:00 +02:00