Commit Graph

523 Commits

Author SHA1 Message Date
Jakub Sokołowski 9153bb02f1
logclean-job: fix bug in script that deletes newer first
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-27 15:37:49 +01:00
Jakub Sokołowski b1ad100a50
logclean-job: increase timer timeout to 5 minutes
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-27 15:26:53 +01:00
Jakub Sokołowski d31094d85c
logclean-job: reduce kept ES indices to 20
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-27 15:24:34 +01:00
Jakub Sokołowski 86c637c224
geth: increase mainnet data volume to 1.5 TB
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-26 12:26:05 +01:00
Jakub Sokołowski ebb5910a4b
Makefile: upgrade Ansible plugin to 2.5.0, check MD5 sum
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-23 13:24:47 +01:00
Jakub Sokołowski aee3b5eff5
beacon-node-builds: allow for repo updates for select builds
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-22 21:37:38 +01:00
Jakub Sokołowski f931c3cde1
upgrade eth2 image to 1.0.8
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-22 17:55:20 +01:00
Jakub Sokołowski dd70e0652d
beacon-node-builds: add commit to labels
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-22 16:05:56 +01:00
Jakub Sokołowski a269c3e490
distribute-validators: add --delete for rsyncing secrets
Also drop some debug messages, comment out ignore_errors.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-22 15:49:05 +01:00
Jakub Sokołowski 73bde9a51a
add any_errors_fatal:true to avoid duplicate validators
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-22 15:48:51 +01:00
Zahary Karadjov 4668266eb6
Specify the double-checking slashing DB mode on all large servers
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-22 15:16:47 +01:00
Zahary Karadjov f887391f04 More varied validator assignments 2021-02-22 15:14:16 +01:00
Jakub Sokołowski 8dfe06a9bd
change testing branch build to 6am
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-22 15:09:28 +01:00
Jakub Sokołowski 86ce49aae4
upgrade Terraform to 0.14.4
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-17 11:40:17 +01:00
Jakub Sokołowski a40c0ab49c
geth-goerli: increase cache size from default 1GB to 2GB
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-12 23:06:04 +01:00
Jakub Sokołowski bb1e3ff3ba
bump size of Goerli Geth node to t3a.large
It keeps hitting memory limits and failing to sync.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-12 22:58:42 +01:00
Jakub Sokołowski a5a950bb91
set become=false in ansible.cfg, set it to true in playbooks
See: https://github.com/status-im/nimbus-private/pull/9

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-09 20:27:13 +01:00
Jakub Sokołowski 0e82fa7c03
beacon-node-builds: don't update git branch when deploying
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-09 18:35:32 +01:00
Jakub Sokołowski 4cdd5f01ce
increase Goerli container memory limit to 80%
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-09 18:32:11 +01:00
Jakub Sokołowski d8b643fd74
add SSH users to systemd-journal group
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-08 12:36:14 +01:00
Jakub Sokołowski f96fe517e7
beacon-node-builds: allow staff group membets to start builds
I was thinking of defining the jobs as nimbus user systemd units, but
that would not work unless nimbus user logged in at least once.

And Systemd services have no functionality for allowing non-root users
to start/stop/restart system services, so sudo is the simplest way.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-08 12:36:02 +01:00
Jakub Sokołowski e5037a6751
geth: bump instance data volume to 1200 GB
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-06 20:38:33 +01:00
Jakub Sokołowski 8e0966fb0e
beacon-node-builds: don't fetch if we're on a detached head
Causes failures when a tag is checked out, nothing to fetch.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-04 23:48:26 +01:00
zah 33ba3b1e8b Upgrade the production fleet to 1.0.7 2021-02-04 21:41:22 +01:00
Jakub Sokołowski b7226818d3
disable kibana-logtrail, it's not been updated to 7.10.2
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-01 16:05:50 +01:00
Jakub Sokołowski dbac764185
upgrade ElasticSearch to 7.10.2
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-01 15:57:10 +01:00
Jakub Sokołowski 921b1c3ea7
ansible.cfg: enable ansible ssh multiplexing
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-01 13:21:44 +01:00
Jakub Sokołowski 157d215230
specify beacon_node_netkey for nimbus-mainnet-small bootnodes
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-29 13:26:55 +01:00
Jakub Sokołowski 87a0503c22
increase volume size for Mainnet Geth to 1TB
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-27 16:47:25 +01:00
Ștefan Talpalaru 37b4e558b2 fix the Makefile and a shell script
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-26 15:31:26 +01:00
Jakub Sokołowski 26a31c5d63
use 8000+ UIDs for extra SSH users to not clash with core
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-26 12:11:33 +01:00
Jakub Sokołowski d12ccc626b
ansible.cfg: shell instead of command as default module
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-25 15:10:25 +01:00
Jakub Sokołowski 8f76c6810a
log-store: fix mounting of data volumes
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-25 15:08:34 +01:00
Jakub Sokołowski 62f44b8a05
drop old fleet from group and host vars and pyrmont playbook
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-22 15:59:47 +01:00
Jakub Sokołowski 49bd1fa267
remove old pyrmont fleet, move new one back to pyrmont.tf
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-22 15:56:26 +01:00
Jakub Sokołowski 8aa381ca2e
update names in inventory for nimbus.mainnet hosts
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-22 15:46:04 +01:00
Jakub Sokołowski 84be761116
fix hostname in nimbus-pyrmont-unstable-libp2p validators layout
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-22 15:19:58 +01:00
Jakub Sokołowski 8920b57c34 rename mainnet hosts to match stable branch
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-22 14:27:39 +01:00
Jakub Sokołowski 37c1c62a5d add validators distributio for the new fleet
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-22 14:27:39 +01:00
Jakub Sokołowski 8ad0027936
log-store: add 2GB swap files to hosts
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-20 20:35:33 +01:00
Jakub Sokołowski 1218ab1772
lower volume size for logs to 200 GB for now
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-20 20:35:16 +01:00
Jakub Sokołowski 9a9619138b
add dashboard link to README
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-20 19:27:04 +01:00
Jakub Sokołowski 6e138997e2
re-enable sending logs to logstash
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-20 19:27:04 +01:00
Jakub Sokołowski 45a0315bb0
deploy elasticsearch and a kibana dashboard
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-20 16:00:29 +01:00
Jakub Sokołowski ef8e77a1ac
re-add the ELK stack hosts
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-20 15:59:11 +01:00
Jakub Sokołowski 8d939e1cd9
bump data volume IOPS to 2500
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-19 16:29:53 +01:00
Jakub Sokołowski 2b4c2320ce
move pyrmont variables to pyrmont.tf
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-19 10:33:27 +01:00
Jakub Sokołowski 67ebb93be3
fix wrong build nodes listed in playbook
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-18 19:53:52 +01:00
Jakub Sokołowski 0bc722ca21
dist-validators: don't run any tasks if no layout is defined
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-18 19:47:51 +01:00
Jakub Sokołowski 89dfe7ca5c
add new fleet hosts to inventory
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-18 19:35:03 +01:00