Commit Graph

898 Commits

Author SHA1 Message Date
Jakub Sokołowski 23343875b8
ansible/add_ssh_key: plabook for adding SSH keys
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-16 09:10:27 +02:00
Jakub Sokołowski dae14e2a59
holesky.tf: add and bootstrap two more hosts
https://github.com/status-im/infra-nimbus/issues/152

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-16 09:09:45 +02:00
Jakub Sokołowski f200a1b4c5
nimbus.holesky: fleet config and validator layout
https://github.com/status-im/infra-nimbus/issues/152

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-15 16:25:38 +02:00
Jakub Sokołowski 08eed94351
holesky.tf: add first 15 hosts for nimbus.holesky
https://github.com/status-im/infra-nimbus/issues/152

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-15 16:24:33 +02:00
Alexis Pentori 73184446d6
nimbus.parter: removing ephemeral debug option
Signed-off-by: Alexis Pentori <alexis@status.im>
2023-09-15 14:37:39 +02:00
Alexis Pentori e8834f4a9e
nimbus.sepolia: removing ephemeral debug option
Signed-off-by: Alexis Pentori <alexis@status.im>
2023-09-15 14:37:39 +02:00
Jakub Sokołowski 2aeea3edc5
requirements: update beacon nodes to support holesky
https://github.com/status-im/infra-nimbus/issues/152

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-15 08:20:19 +02:00
Jakub Sokołowski 82a5403837
requirements: upgrade geth, erigon, nethermind
Upgrades necessary for Holesky testnet support.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-12 16:33:07 +02:00
Daniil Sobol b9373b7889
all: grant SSH access to daniil@status.im
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-11 09:41:05 +02:00
Jakub Sokołowski 73c4b72972
ansible: drop tags from include_role tasks
This does not actually work. And what we need is to include a `tags` key
in each of the elements provided via `with_items`. To do this in a way
that's not disgusting we would probably need a custom lookup plugin.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-08 11:59:50 +02:00
Jakub Sokołowski f1ec33383a
requirements: macos role increase of soft max files
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-06 10:53:47 +02:00
Jakub Sokołowski 0d323c70fc
drop macos-01 file from host_vars, add missing exec url
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-06 10:53:46 +02:00
Jakub Sokołowski 160276121b
drop macos-01.ms-eu-dublin.nimbus.prater host
Replaced by `macm1-01.ih-eu-mda1.nimbus.prater`.
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-06 10:19:22 +02:00
Jakub Sokołowski a658d312a8
nimbus.prater: add stable node to macm1-01 host
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-06 10:19:21 +02:00
Jakub Sokołowski 227206c82d
nimbus.prater: move validators to macm1-01
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-06 10:18:44 +02:00
Jakub Sokołowski ae61880a5d
requirements: bump rocketpool role
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-06 10:07:13 +02:00
Jakub Sokołowski 9df1b95f43
nimbus.prater: fix trusted node URL for MacOS hosts
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-23 16:38:24 +02:00
Jakub Sokołowski 0cdb2d6e02
macos-01.nimbus.prater: disable resync timer
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-21 19:47:12 +02:00
Jakub Sokołowski a823709dfe
add macm1-01.ih-eu-mda1.nimbus.prater host
Replacement for `macos-01.ms-eu-dublin.nimbus.prater`.

https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-18 21:46:25 +02:00
Jakub Sokołowski 2a99b6ab43
nimbus.prater: add Nethermind node on linux-04
https://github.com/status-im/infra-eth2/issues/11

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-17 13:26:08 +02:00
Jakub Sokołowski c662e92d51
nimbus.prater: configure Nethermind metrics endpoint
https://github.com/status-im/infra-eth2/issues/11

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-17 13:05:28 +02:00
Jakub Sokołowski 4f729075bf
requirements: upgrade Geth from 1.12.0 to 1.12.2
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-16 13:50:37 +02:00
Jakub Sokołowski 8b6a22110a
nimbus.prater: open ports for Nethermind EL node
https://github.com/status-im/infra-eth2/issues/11

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-11 14:21:19 +02:00
Jakub Sokołowski 1e24f891fe
nimbus.prater: add DNS discovery entry for nethermind
https://github.com/status-im/infra-eth2/issues/11

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-11 14:20:34 +02:00
Jakub Sokołowski 600c6b02df
nimbus.prater: add chronos node on linux-06 host
For Eugene for tracking regressions in chronos library.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-08 15:54:11 +02:00
Jakub Sokołowski 8d5d8a3935
refactor handling of long libp2p branch name
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-08 15:22:03 +02:00
Jakub Sokołowski 5344827479
nimbus.prater: deploy Nethermind node on linux-02
Part of work to use Nethermind for eth2.prod fleet:
https://github.com/status-im/infra-eth2/issues/11

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-07 23:05:33 +02:00
Jakub Sokołowski 45d5261400
nimbus.prater: bump limits for consul alerts on linux-06
That host is overloaded and causes too many alerts.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-07 14:54:36 +02:00
Jakub Sokołowski 5586db729d
nimbus.prater: reduce max_headers_size to 128 KB
Probably the reason for elevated memory usage when using Validator
Client with large number of validators attached.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-07 14:54:07 +02:00
Jakub Sokołowski 5e3e8cbda4
requirements: bump rocketpool role
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-07 13:26:12 +02:00
Jakub Sokołowski c17be7a34b
ansible/bootstrap: use become:true for MacOS hosts
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-02 21:08:02 +02:00
Jakub Sokołowski 086cf0c208
requirements: apply fixes for beacon node on windows
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-20 17:21:21 +02:00
Jakub Sokołowski e35da2accf
nimbus.prater: put era files on the data volume
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-20 15:57:18 +02:00
Jakub Sokołowski 579dcce780
nimbus.prater: use Windows data volume for beacon nodes
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-20 11:17:04 +02:00
Jakub Sokołowski df5ebd3a88
nimbus.prater: disable defender and add exec layer URLs
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-20 11:12:56 +02:00
Jakub Sokołowski 2c9bbe832b
nimbus.sepolia: open Geth Websocket ports for Vac
Requested by p1ge0nh8er for vacdev.misc host.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-16 10:57:40 +02:00
Jakub Sokołowski f304db1cc0
nimbus.prater: bump Eirgon memory limit to 15%
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-16 10:18:40 +02:00
Jakub Sokołowski 3c97dd23fc
ansible/bootstrap.yml: fix become setting and hosts
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-12 16:51:57 +02:00
Jakub Sokołowski 3b4a872b00
nimbus.mainnet: add 16 GB SWAP file, no SWAP partition
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-12 09:45:51 +02:00
Jakub Sokołowski 41025265e2
nimbus.prater: add 16 GB SWAP file, no SWAP partition
There were OOM killer logs on `linux-06` due to Erigon.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-12 09:38:19 +02:00
Jakub Sokołowski 7dff81fb2f
add windows-01.ih-eu-mda1.nimbus.prater host
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-07 12:17:33 +02:00
Jakub Sokołowski 2f7bf02480
ansible/bootstrap.yml: separate windows and macos
Avoids wasting time on running "skipped" things.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-07 12:16:11 +02:00
Jakub Sokołowski 358f7b3890
remove cancelled hosts from ansible inventory
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-07 12:03:21 +02:00
Jakub Sokołowski 3773305151
prater.tf: drop remaining Hetzner hosts
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-06 09:32:04 +02:00
Jakub Sokołowski 4480d292be
nimbus.sepolia: debug flag for old attestation stability
As requested by Dustin.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-06 08:32:55 +02:00
Jakub Sokołowski cec778f4f1
nimbus.prater: debug flag for old attestation stability
As requested by Dustin.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-06 08:32:54 +02:00
Jakub Sokołowski 52e518d3c3
nimbus.sepolia: drop nim_commit=version-1-6 flag
It no longer has any effect.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-06 08:32:53 +02:00
Jakub Sokołowski 9455a6a5ae
nimbus.prater: drop nim_commit=version-1-6 flag
It no longer has any effect.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-06 08:29:09 +02:00
Jakub Sokołowski 44d77c54f4
nimbus.prater: move validators from Hetzner hosts
The InnovaHosting servers are almost fully synced, time to move.
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-06 08:29:08 +02:00
Jakub Sokołowski be6aba83b8
requirements: bump most roles, including fluffy
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-29 18:49:34 +02:00
kdeme e0140a1497
nimbus.fluffy: drop bootstrap nodes, use network flag
The `testnet0` network used with `--network` flag already includes our
own Fluffy nodes from this fleet, so no need to define them separately.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-29 18:23:53 +02:00
Jakub Sokołowski aa80ab67df
nimbus.fluffy: add two new hosts from InnovaHosting
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-26 22:50:47 +02:00
Jakub Sokołowski f24fa170ee
add metal-01.ih-eu-mda1.nimbus.eth1 host
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-26 21:59:13 +02:00
Jakub Sokołowski f1c54e1faf
nimbus.prater: use new geth hosts for exec layer
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-26 21:59:04 +02:00
Jakub Sokołowski ee7675543a
nimbus-geth-goerli: fix paths for extra data volume
Otherwise `/docker` is part of root and there's not enough space.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-26 12:54:54 +02:00
Jakub Sokołowski 5b85f5c26d
add three InnovaHosting servers to nimbus.geth fleet
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-23 21:15:19 +02:00
Jakub Sokołowski de28d46133
nimbus.mainnet: enable yamux for 2 libp2p nodes
https://github.com/status-im/infra-nimbus/issues/148

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-23 15:14:42 +02:00
Jakub Sokołowski 052a8485aa
nimubs.prater: add 4 new hosts from InnovaHosting
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-22 20:44:31 +02:00
Jakub Sokołowski e91d72114e
requirements: fix format of valitaror monitor flags
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-22 12:16:08 +02:00
Jakub Sokołowski 766092f537
nimbus.prater: fix validator monitor details condition
It was causing very high cardinality due to lots of validators.
Crashed our Hetzner metrics server.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-22 12:07:16 +02:00
Jakub Sokołowski 7d9ebb1744
nimbus.prater: double beacon node open files limit
Causes issues with metrics responses due to inability to open more files.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-22 09:31:22 +02:00
Jakub Sokołowski 276efe33cf
requirements: bump oauth-proxy and elasticsearch-lb
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-21 17:14:26 +02:00
Jakub Sokołowski 5f151fb828
all: grant admin rights to cheatfate user
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-21 16:50:17 +02:00
Jakub Sokołowski 6f0ff4ad19
nimbus.prater: run only two nodes on MacOS host
Not enough disk space for 3 nodes.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-20 19:47:20 +02:00
Jakub Sokołowski 666cde43bb
requirements: bump oauth to 7.4.0 and geth to 1.12.0
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-16 13:05:57 +02:00
Jakub Sokołowski 1aa507c359
prater: move validators to new linux-01 and linux-02
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-16 12:57:24 +02:00
Jakub Sokołowski 43b238c87e
nimbus.all: use new --validator-monitor-details flag
Replaced `--validator-monitor-totals` in:
https://github.com/status-im/nimbus-eth2/pull/4468

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-07 19:04:37 +02:00
Etan Kissling b33a35c5e1
nimbus: set --rest-allow-origin='*' for public_api
To allow browser based clients to access our API servers.

Depends on:

- https://github.com/status-im/infra-role-beacon-node-linux/pull/14
- https://github.com/status-im/infra-role-beacon-node-linux/pull/15
- https://github.com/status-im/infra-role-beacon-node-macos/pull/7
- https://github.com/status-im/infra-role-beacon-node-macos/pull/8
- https://github.com/status-im/infra-role-beacon-node-windows/pull/6
- https://github.com/status-im/infra-role-beacon-node-windows/pull/7

Signed-off-by: Etan Kissling <etan@status.im>
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-05 13:45:55 +02:00
Jakub Sokołowski 0441d524a9
drop linux-01.he-eu-hel1.nimbus.sepolia host
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-05 12:08:43 +02:00
Jakub Sokołowski e08586be04
nimbus.sepolia: move validators to new Innova host
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-05 12:08:01 +02:00
Jakub Sokołowski 10e0c7d74a
requirements: fix nimbus-eth1 folder creation
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-02 17:31:52 +02:00
Jakub Sokołowski 96a96de630
nimbus-sepolia-trial: add new Innova host
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-02 14:26:46 +02:00
Jakub Sokołowski 6ae7dc00f0
nimbus.prater: add Innova host config
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-02 10:35:18 +02:00
Jakub Sokołowski da40f535ba
nimbus.sepolia: add Innova host config
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-02 10:34:20 +02:00
Jakub Sokołowski 6618658396
add linux-01 and linux-02 for nimbus.prater
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-02 10:28:57 +02:00
Jakub Sokołowski d3bb14a070
add linux-01.ih-eu-mda1.nimbus.sepolia host
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-02 10:27:20 +02:00
Jakub Sokołowski 948b5885fe
nimbus.prater: move disabling discovery to Hetzner group
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-29 21:20:03 +02:00
Jakub Sokołowski e9a05d73ab
nimbus.prater: add DNS discovery for Erigon
Necessary because we disable normal discovery.

Source:
https://github.com/ethereum/discv4-dns-lists

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-29 21:03:53 +02:00
Jakub Sokołowski a8741b7019
nimbus.prater: , disable discovery for Erigon
It appears peer discovery can be detected by hetzner as an attempt at
network scanning:
https://github.com/ledgerwatch/erigon/issues/7234

Which can lead to the server being shut down.
For this reason I'm lowering max peers and disabling discovery.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-29 20:06:58 +02:00
Jakub Sokołowski 3f3d763990
nimbus.prater: double nofile limit to 32768
Which needs to be more than 4 x 7899 validators.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-29 18:36:09 +02:00
Jakub Sokołowski e5324beaaa
nimbus-geth-goerli: remove one of three Geth nodes
Not nough storage space to run 3 nodes on the hosts.

Also made AWS, Windows and MacOS nodes discover available
Geth nodes using Consul catalog.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-24 18:34:03 +02:00
Jakub Sokołowski 001baf0367
requirements: bump beacon node roles, use prune
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-23 19:20:24 +02:00
Jakub Sokołowski 597b85a0c8
nimbus.sepolia: open EL WebSocket to wakuv2.test
https://github.com/waku-org/nwaku/issues/1624
https://github.com/status-im/infra-nim-waku/pull/69

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-17 18:15:47 +02:00
Jakub Sokołowski 2e14ecc79e
nimbus.sepolia: enable SMART metrics service
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-17 18:11:27 +02:00
Jakub Sokołowski c15e1ea358
nimbus.mainnet: 10k max peers for stable-02@linux-03
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 12:38:24 +02:00
Jakub Sokołowski a270f4044c
all: get Consul certificates from BitWarden
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 12:14:38 +02:00
Jakub Sokołowski 0cd6de1ea6
nimbus.mainnet: use ncli_db from stable node
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 12:14:27 +02:00
Jakub Sokołowski 5e790e8585
nimbus-era-files: fix trailing newline in launchd
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 12:14:09 +02:00
Jakub Sokołowski d04a98ea11
remove metal-01.he-eu-hel1.nimbus.mainnet
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 11:05:30 +02:00
Jakub Sokołowski 05f98abd53
nimbus.prater: use more CL endpoints on MacOS
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 11:05:23 +02:00
Jakub Sokołowski 4dbb3c165c
nimbus.mainnet: fix config of ERA files hosting
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-16 10:53:21 +02:00
Jakub Sokołowski a145963a2a
ih-eu-mda1: disable Consul check for SMART metrics
The service fails on InnovaHosting servers due to HBA controller:
https://github.com/prometheus-community/smartctl_exporter/issues/26

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-15 19:51:41 +02:00
Jakub Sokołowski 9102e1954a
nimbus.sepolia: fix exposed REST API endpoint
Missing redirect config.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-15 16:19:19 +02:00
Jakub Sokołowski 9a3f1f8b4c
nimbus.mainnet: fix filenames for host_vars
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-15 13:24:52 +02:00
Jakub Sokołowski 0256a21dd0
nimbus.mainnet: add two more hosts from Innova
The `linux-06` is a new host, while `linux-07` is re-purposed `linux-01`
from the prater fleet. This is done because the Innova company is out of
Xeon E5-2690 CPUs, and we want consistent performance across the fleet.

This way all seven hosts in `nimbus.mainnet` will use the E5-2690 CPU.

https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-09 15:07:22 +02:00
Jakub Sokołowski 406ac58f16
nimbus.mainnet: use --no-el for even number nodes
Attempted fix issues with syncing where latency on the Geth endpoint is
too high for beacon nodes to finish syncing properly.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-05 20:58:53 +02:00
Jakub Sokołowski 01cf22c9f8
get-geth-api-urls: rename variables for clarity
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-05 19:21:49 +02:00
Jakub Sokołowski 5635b39e50
mainnet: decommission 6 Hetzner hosts
Being replaced by InnovaHosting servers
https://github.com/status-im/infra-nimbus/issues/132

I'm leaving old `metal-07` as `metal-01` since we don't have a server
with `libp2p` nodes on mainnet yet.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-28 12:22:46 +02:00
Jakub Sokołowski cd0841eba2
ih-eu-mda1: fix destintion DC for sending logs
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-25 18:06:00 +02:00
Jakub Sokołowski e43ede93a5
nimbus.mainnet: fix exec layer URL logic
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-20 22:28:26 +02:00
Jakub Sokołowski 940b2577da
nimbus.mainnet: add node layout for new Innova hosts
Once they are synced and working we'll remove the Hetzner ones.

https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-18 18:14:53 +02:00
Jakub Sokołowski 3c0348cb81
add linux-05.ih-eu-mda1.nimbus.mainnet host
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-18 18:13:00 +02:00
Jakub Sokołowski 1c177eb944
add linux-04.ih-eu-mda1.nimbus.mainnet host
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-18 17:58:48 +02:00
Jakub Sokołowski 8ddbd86139
nimbus.mainnet: revert removing of stable-02 node
This reverts commit ab266d132d.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-17 20:06:57 +02:00
Jakub Sokołowski 53ccd34927
nimbus.prater: bootstrap new Innova servers
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-17 17:11:27 +02:00
Jakub Sokołowski 1f5ad05893
nimbus.fluffy: 24h to make healthcheck critical
This is not a critical service and there's a LOT of sevices that cause a
lot of alerts when developers are stopping and starting them.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-12 19:28:37 +02:00
Jakub Sokołowski 9ec35f149b
nimbus.fluff: deploy smart-metrics exporter
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-12 13:24:29 +02:00
Jakub Sokołowski 39ceeaf5ec
requirements: bump rocketpool, geth, erigon
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-07 18:16:50 +02:00
Jakub Sokołowski ab266d132d
nimbus.mainnet: remove stable-02 node to save space
We can always re-add it when we deploy to our new hosts.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-07 12:00:20 +02:00
Jakub Sokołowski 171acedac7
nimbus.prater: un-comment roles
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-06 16:39:31 +02:00
Jakub Sokołowski debe4bd960
nimbus.mainnet: fix data volume mount on metal-05
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-06 16:38:19 +02:00
Jakub Sokołowski d41b3c89f8
dash.nimbus: increase logs retention to 30 days
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-05 11:04:09 +02:00
Jakub Sokołowski d6de0faf1a
nimbus.prater: open Erigon metrics port on VPN
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-23 15:14:38 +01:00
Jakub Sokołowski 0956c88d9a
nimbus.prater: enable metrics for Erigon nodes
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-22 13:47:15 +01:00
Jakub Sokołowski 9aa83f8d47
nimbus.prater: use Erigon EL node if available
Beginning of work on testing multi-EL node setup:
https://github.com/status-im/infra-nimbus/issues/144

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-22 12:10:34 +01:00
Jakub Sokołowski 0943ae2f76
nimbus-era-files: fix script template for verify timer
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-22 10:58:39 +01:00
Jakub Sokołowski cdf362f2a9
nimbus.prater: provide private key for mining
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-21 13:07:23 +01:00
Jakub Sokołowski f92bcb2802
nimbus.prater: fix erigon metrics port conflict
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-21 13:07:13 +01:00
Jakub Sokołowski 4a5b7d7107
nimbus.prater: deploy two Erigon nodes
Part of work to support multiple EL nodes driven by one CL node:
https://github.com/status-im/infra-nimbus/issues/144

Disabled Geth websocket as unused and conflicting with Erigon.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-20 22:27:50 +01:00
Jakub Sokołowski 5926a3da4d
nimbus.prater: add extra data volumes for CL nodes
https://github.com/status-im/infra-nimbus/issues/144

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-20 21:30:38 +01:00
Jakub Sokołowski e677decb38
requirements: bump ElasticSearch to 7.17.9
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-20 21:30:23 +01:00
Jakub Sokołowski 4ea58aeda7
deploy SMART metrics to all metal hosts
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-18 18:02:58 +01:00
Jakub Sokołowski e1e21119da
use new exec layer variables for beacon nodes
Role changes:
- https://github.com/status-im/infra-role-beacon-node-linux/commit/99a445d1
- https://github.com/status-im/infra-role-beacon-node-macos/commit/eecf9fea
- https://github.com/status-im/infra-role-beacon-node-windows/commit/dba21b36

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-16 00:59:48 +01:00
Jakub Sokołowski 03ef7c54bb
nimbus.sepolia: use nimbus-eth1 Engine API WS port
Related to:
https://github.com/status-im/nimbus-eth1/issues/1486

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-06 11:29:51 +01:00
Jakub Sokołowski c854752503
ansible/resize.yml: fix unmounting /mnt volumes
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-02 14:37:27 +01:00
Jakub Sokołowski 5783656224
nimbus.mainnet: make AWS nodes have archive history
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-02 12:22:51 +01:00
Jakub Sokołowski 768ecef371
nimbus.prater: use AWS hosts to sync first node
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-02 12:22:45 +01:00
Jakub Sokołowski 090c4661a9
nimbus.mainnet: use AWS hosts to sync first node
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-02 12:22:44 +01:00
Jakub Sokołowski 10f3132bcd
logs.nimbus: use G1 garbage collector
https://github.com/status-im/infra-nimbus/issues/138

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-01 18:44:02 +01:00
Jakub Sokołowski 4e20fd2c14
nimbus.prater: use a new set of MEV boost URLs
The one from `flashbots.net` seems to have connectivity issues:
```
REST request was interrupted
```

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-24 13:16:06 +01:00
Jakub Sokołowski 15d96ca3be
nimbus.mainnet/prater: use more threads than 1
This setting is quite old and it's time to use more than one thread.
The `libp2p` branch is the exception that will keep using 1 thread.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-24 11:37:17 +01:00
Jakub Sokołowski 3e3cff6840
nimbus.mainnet/prater: enable pruned history
For most nodes this is fine, except for nodes with publicly exposed API,
which we should keep as synced from genesis and in archive mode.

Every other node is fine being prunned. I'm keeping AWS nodes not pruned
as fallback since increasing their storage is quick and easy.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-21 22:52:41 +01:00
Jakub Sokołowski 6e05ef6eec
nimbus.prater: add linux-01.cs-eu-nld1.nimbus.prater
Part of research into alternative hosting for Nimbus:
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-21 21:25:42 +01:00
Jakub Sokołowski e3cbc05ab9
requirements: bump roles, drop versions for bootstrap
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-21 12:55:22 +01:00
Jakub Sokołowski d6da1ce336
nimbus.mainnet: drop db_purge and db_sync vars
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-21 12:38:17 +01:00
Jakub Sokołowski 297f347934
drop linux-01.cs-eu-nld1.nimbus.prater host
Decided against using it due to maintenance costs:
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-21 12:37:15 +01:00
Jakub Sokołowski 77278c2b65
logs.nimbus: lower ES JVM heap size to 40 GB
Never ending issues with `Data too big` errors:
https://github.com/status-im/infra-nimbus/issues/138

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-21 00:11:55 +01:00
Jakub Sokołowski fc236d0132
remove SSH access from mamy, miran, s1fr0
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-17 16:35:09 +01:00
Jakub Sokołowski c05fad0e01
add linux-01.cs-eu-nld1.nimbus.prater host
Test for replacement of Hetzner hosts:
https://github.com/status-im/infra-nimbus/issues/132

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-01-30 21:34:21 +01:00
Jakub Sokołowski 61d586fa8c
logs.nimbus: set static max heap size to 48 GB
Trying to fix the `Data too large` circuit breaker errors:
https://github.com/status-im/infra-nimbus/issues/138

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-01-28 14:17:04 +01:00
Jakub Sokołowski ee1ed5203b
logs/dash.nimbus: drop manual JVM heap size setting
Lets try how well the automatic sizing works:
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/important-settings.html#heap-size-settings

https://github.com/status-im/infra-nimbus/issues/138

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-01-19 15:00:14 +01:00
Jakub Sokołowski e73125a215
replace logs.nimbus hosts with new AX101 servers
Constant memory issues due to volume of logs:
https://github.com/status-im/infra-nimbus/issues/138

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-01-19 14:08:34 +01:00
Jakub Sokołowski 9f0ab35e5d
requirements: bump elasticsearch and kiban to 7.17.8
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-01-19 14:08:33 +01:00
Nikola K 2c3a01e1fe
add crypt1d as new nimbus devops
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-01-13 10:46:17 +01:00
Jakub Sokołowski 5930389580
nimbus.prater: bump AWS host data volumes to 300 GB
Also add a `resize.yml` playbook for easier resizing.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-01-12 12:31:57 +01:00
Jakub Sokołowski 3089a52ff0
requirements: bump bootstrap roles, upgrade rocketpool
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-01-12 12:31:36 +01:00
Jakub Sokołowski c18ead4714
nimbus.prater: add resync timer configuration
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-01-04 17:24:02 +01:00
Jakub Sokołowski cd982c44f3
drop unused nimbus-db-purge role in favor of resync
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-01-03 15:00:39 +01:00
Jakub Sokołowski d0bb5fa009
nimbus.ropsten: decomission, network shutdown
https://blog.ethereum.org/2022/11/30/ropsten-shutdown-announcement

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-12-09 08:51:13 +01:00
Jakub Sokołowski ffa5032735
requirements: bump macos bootstrap, fix consul meta
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-12-09 08:47:23 +01:00