status-im
/
infra-nimbus
mirror of https://github.com/status-im/infra-nimbus.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,072 Commits 10 Branches 0 Tags 31 MiB
Commit Graph

1072 Commits

Author SHA1 Message Date
Jakub Sokołowski c48ac78797
nimbus.prater: use stable instead of unstable branch 
Avoid an RPC bug found in `unstable` for Dencun upgrade:
https://github.com/status-im/nimbus-eth2/issues/5753

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-16 09:27:44 +03:00
Jakub Sokołowski 1253d61233
nimbus.prater: move windows-01 validators to linux-01 
Validators necessary for the Dencun upgrade due to low participation.

Dropped and decomissioned Hetzner windows-01 as no longer necessary.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-12 16:35:26 +01:00
Jakub Sokołowski afc0e93e36
nimbus.mainnet: discover only EL nodes from nimbus.geth 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-11 14:51:43 +01:00
Jakub Sokołowski eed8a7145d
requirements: upgrade geth, erigon, and nethermind 
Upgrades include support for Cancun upgrade for the Goerli network.
https://blog.ethereum.org/2024/01/10/goerli-dencun-announcement#execution-layer-goerli-releases

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-11 09:50:15 +01:00
Jakub Sokołowski 13fa6ea608
get-geth-api-urls: allow specifying multiple fleets 
More useful than specifying multiple servicer names.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-09 11:10:44 +01:00
Jakub Sokołowski 9fbf61806a
nimbus.mainnet: use all geth nodes in the fleet 
To allow individual geth nodes to be re-synced gradually.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-09 09:40:14 +01:00
Jakub Sokołowski 2c766a7991
dash.nimbus: reduce indices kept to 21 from 30 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-08 23:15:07 +01:00
Jakub Sokołowski 781b9e7524
nimbus.mainnet: switch exec URLs on linux-03 
Necessary to allow the geth node to resync using new schema in order to
save disk space.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-08 22:09:07 +01:00
Alexis Pentori c7f77f5e32
nimbus.prater: switching MEV boots url. 
Removing invalid url. List of valid URL available at https://ethstaker.cc/mev-relay-list#mev-relay-list-for-goerli-testnet

Signed-off-by: Alexis Pentori <alexis@status.im>
 2024-01-03 16:28:08 +01:00
Jakub Sokołowski a015bed507
requirements: bump systemd-timer role 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-03 16:27:56 +01:00
Jakub Sokołowski 8db25c70b8
nimbus.sepolia: enable MEV builder for first node 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-02 14:30:11 +01:00
Jakub Sokołowski 74a567757b
requirements: Upgrade Geth to 1.13.8, Neth to 1.24.0 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-02 13:23:28 +01:00
Jakub Sokołowski 7a239812ec
nimbus.prater: remove Erigon node to save disk space 
It was using 1.1 TB on `linux-05`, which is unreasonable.
And the network is deprecated anyway.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-02 13:22:54 +01:00
Anton Iakimov c2a939087d nimbus.mainnet: increase stable-small root volume size 
Nimbus logs take up to 10G on hosts.
And current disk space is 15-25%
 2023-12-22 16:26:55 +01:00
Jakub Sokołowski 08db65a52e
nimbus.sepolia: remove obsolete vacdev.misc ipset 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-12-08 17:44:55 +01:00
Jakub Sokołowski 5d266bcd4d
requirements: upgrade erigon and nethermind 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-12-08 13:55:27 +01:00
Jakub Sokołowski 9d99b031ac
nimbus.prater: reduce Consul alerts sensitivity 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-12-06 11:13:54 +01:00
Jakub Sokołowski 9be87e32ce
nimbus.mainnet: reduce Consul alerts sensitivity 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-12-06 11:13:26 +01:00
Jakub Sokołowski e38ceca4f7
readme: add link to holesky beaconchain dashboard 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-12-01 11:55:33 +01:00
Jakub Sokołowski 39e12fdc6f
nimbus.holesky: add public beacon node API endpoints 
https://github.com/status-im/infra-nimbus/issues/157

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-12-01 11:50:23 +01:00
Jakub Sokołowski b4dc2f9c44
readme: fix hostnames in public API table 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-12-01 11:34:33 +01:00
Jakub Sokołowski 58cbfee30f
nimbus.prater: disable resyncing on all hosts 
It just causes unnecessary alerts for an obsolete network.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-27 11:45:16 +01:00
Jakub Sokołowski 0814b65965
sepolia: add missing layout file to vars_files 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-21 16:32:29 +01:00
Jakub Sokołowski f0fc719f0f
sepolia: add missing layout file to vars_files 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-21 16:31:45 +01:00
Jakub Sokołowski e186c73301
requirements: bump Geth to 1.13.5, Erigon to 2.54.0 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-21 15:51:06 +01:00
Jakub Sokołowski 24020d0962
all: reduce MTR report cycle from 10 to 1 
We have received a complaint from InnovaHosting about them being hit by
about 150 ICMP `ttl1` packets/s on their routers, causing excess CPU usage.
https://client.innovahosting.net/viewticket.php?tid=532874&c=8gALx9vm

By using `tcpdump` I have identified that `mtr` by default pings the
target 10 times, which means that the default value of `-c`/`--report-cycles`
is 10, although this is not documented in the manual.

We can see this when calling `mtr github.com` and watching with `tcpdump`:
```
 > sudo tcpdump -v -i eno1 icmp and src 185.181.230.78 and dst github.com | grep 'ttl 1,'
tcpdump: listening on eno1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
19:54:53.981243 IP (tos 0x0, ttl 1, id 37119, offset 0, flags [none], proto ICMP (1), length 64)
...(8 packets omitted)...
19:55:03.025460 IP (tos 0x0, ttl 1, id 38226, offset 0, flags [none], proto ICMP (1), length 64)
```
We don't need to run the test 10 times to get a result for our metric.

Related to:
https://github.com/status-im/infra-role-bootstrap-linux/commit/ea22bdfe

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-20 21:07:17 +01:00
Jakub Sokołowski 6b800a5342
nimbus.fluffy: re-enable Consul healthchecks 
It appears the RPC issues was resolved in:
https://github.com/status-im/nimbus-eth1/issues/1880

Most probably caused by DB size.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-20 09:57:46 +01:00
Jakub Sokołowski 2cac3081a0
layouts: add script and generate TSVs of validators 
Helps developers identify which host holds which validator.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-17 15:01:57 +01:00
Jakub Sokołowski c29b23c6dc
nimbus.sepolia: open ports for waku.test fleet 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-14 11:56:16 +01:00
Jakub Sokołowski 40776004de
nimbus.holesky: rename hosts to reflect EL usage 
This way it will be easier to find what you need, and also run Ansible
more selectively against these 3 distinct groups of hosts.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-11 16:12:11 +01:00
kdeme ce37186651
all: update SSH key for kim 
This one is from a YubiKey.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-07 18:48:05 +01:00
Jakub Sokołowski 10dd722e29
all: grant admin rights to kim 
Necessary to run 'perf'.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-07 16:19:24 +01:00
Jakub Sokołowski c1be589960
all: add debug tools like gdb and perf 
Also allow use of 'perf' without root.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-07 12:48:05 +01:00
Jakub Sokołowski 4df34ac3c1
nimbus.sepolia: enable payload builder for 4th node 
Also drop unnecessary Nim build flags.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-07 11:50:26 +01:00
Jakub Sokołowski 36f78a5970
nimbus.fluffy: disable Consul healthchecks 
They are too flaky to be useful, see:
https://github.com/status-im/nimbus-eth1/issues/1880

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-02 12:51:01 +01:00
Jakub Sokołowski d2feb628c4
nimbus.fluffy: raise Consul alert threshold limits 
This host has constantly issue with nodes and nobody cares.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-02 10:51:57 +01:00
Jakub Sokołowski 7272d55105
nimbus.prater: drop chronos and erigon from linux-06 
The host was overloaded and ran out of disk space on `/docker` volume.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-11-02 10:25:40 +01:00
Jakub Sokołowski a6dc16830d
all: grant SSH access to ujscale, mumar@status 
Necessary to look at full Nimbus Prater logs.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-10-31 14:04:26 +01:00
Jakub Sokołowski 3c6df282d1
requirements: bump Geth and Nethermind versions 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-10-31 14:02:31 +01:00
Jakub Sokołowski bd9d7cc752
all: prevent SPAM Nimbus logs from reaching Logstash 
Depends on:
https://github.com/status-im/infra-role-bootstrap-linux/commit/20609731
https://github.com/status-im/infra-role-bootstrap-linux/commit/98816e2a

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-10-24 16:13:01 +02:00
Jakub Sokołowski a86a65c4bc
nimbus.prater: disable log aggregation for the fleet 
Zahary agreed that we need to start phasing out use of Prater.
This also helps us avoid paying extra for 10 Gbps link for aggr host.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-10-23 13:02:39 +02:00
Jakub Sokołowski a63ca92319
requirements: upgrade Rocketpool, Geth, Erigon 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-10-23 12:48:38 +02:00
Jakub Sokołowski d2270feece
logs.nimbus: increase total_fields.limit to 1500 
This is a special case since these logs are all custom JSON, so
increasing this is fine for now. I can't control what they put in logs.
https://discuss.elastic.co/t/approaches-to-deal-with-limit-of-total-fields-1000-in-index-has-been-exceeded/241039

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-10-12 18:56:21 +02:00
Jakub Sokołowski 63de71f759
all: remove SSH access for tanguy 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-10-12 18:00:59 +02:00
Jakub Sokołowski 6855fc016b
ih-eu-mda1: drop data center override 
We now have Consul, logs, and metrics hosts in `ih-eu-mda1`.
https://github.com/status-im/infra-hq/issues/105

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-10-12 17:26:08 +02:00
Jakub Sokołowski 2c93e44a77
linux-06.nimbus.prater: bump consul thresholds 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-10-10 14:30:33 +02:00
Jakub Sokołowski 136d341214
requirements: bump systemd-timer to fix dir mode 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-10-10 12:32:46 +02:00
Jakub Sokołowski 874771e109
nimbus.prater: fix port clash between Erigon and Geth 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-10-09 11:15:32 +02:00
Jakub Sokołowski 702fa0b485
requirements: bump Erigon to 2.50.2 version 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-10-05 10:50:57 +02:00
Jakub Sokołowski 293404a290
requirements: bump nimbus-eth1 role with healcheck fix 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-10-03 15:23:44 +02:00