Commit Graph

27 Commits

Author SHA1 Message Date
tersec 1051c3c9c5
use upstream/version-2-0 of Nim in CI () 2024-05-09 02:22:28 +02:00
Jakub Sokołowski 7acf383da2
ci: pin Nim for PR builds to v2.0.4 temporarily ()
Mitigation for:
https://github.com/nim-lang/Nim/issues/23568

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-05-06 22:33:53 +00:00
Jakub Sokołowski 4a1a2c8196
add packages to Nix flake config, move to repo root
This way we can actually build and run a node using just:
```sh
nix run 'github:status-im/nimbus-eth2?submodules=1'
```
The `?submodules=1` part should eventually not be necessary.
For more details see:
https://github.com/NixOS/nix/issues/4423

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-05-06 16:57:23 +02:00
Jakub Sokołowski 824bb17115
ci: use clang 18 from Homebrew on aarch64
Not sure if `LDFLAGS` and `CPPFLAGS` are absolutely necessary but Brew docs recommend it.

Depends on:
https://github.com/status-im/infra-ci/commit/67fafcb5
https://github.com/status-im/status-jenkins-lib/pull/90

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-04-25 14:39:19 +02:00
Jakub Sokołowski 80edebfd20
fix Nim v2 builds by checking for correct job name ()
Since job folder structure has been changed we don't need the dash.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-04-18 17:08:44 +02:00
tersec c2a2b76f93
allow Jenkins to specify Nim version to use () 2024-04-03 10:47:32 +00:00
Etan Kissling e677d98119
fix check docs stage in jenkins ()
Ignore color information when checking consistency of options.md with
the built binary.
2024-03-28 21:37:21 +01:00
Jakub Sokołowski 0fa363e022
ci: move help docs check script to its own file ()
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-03-28 10:01:15 +01:00
tersec 06e2ca5286
don't explicitly filter removed file parsedBeaconState.json from holesky () 2023-11-09 17:45:07 +03:00
Etan Kissling eb7c8b7db2
do not download Holesky genesis on `git clone` ()
Holesky genesis.ssz file may be unavailable due to quota limits on
`eth-clients/holesky`; do not download it by default during checkout.
Nimbus will download it on first startup from a mirror instead.
2023-11-06 12:10:36 +01:00
Etan Kissling 63440766ef
exclude 'public-keys/all.txt' file from checkout ()
The 'vendor/holesky/public-keys/all.txt' file is not used by Nimbus,
but sometimes fails to download due to payment issues in `@eth-clients`.
Skipping the download of that file should reduce that risk.

```
Downloading public-keys/all.txt (240 MB)
Error downloading object: public-keys/all.txt (ba977b5): Smudge error: Error downloading public-keys/all.txt (ba977b5b1da180914c934623fce187860e1b54cff939e6208533b2cb5f589e07): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
```

* bump nbs: Patch libnatpmp.a rule to support llvm-mingw in targets.mk

* bump nbs: Allow passing config to submodule initialization in `update-common`
2023-09-18 20:24:13 +02:00
Etan Kissling 8993d576b8
refer to full path in lint hint ()
Update hint message for correcting outdated options.md with full path
and full command.
2023-08-30 11:55:29 +00:00
Etan Kissling 6daa7542a7
add hint when `options.md` is outdated ()
Make it clearer what went wrong when lint fails due to 'options.md'
being out of date.
2023-08-29 12:16:15 +00:00
Etan Kissling bb672a7e90
bump Jenkins build timeout ()
With the gnosis binary, building all targets sometimes exceeds the 40m.
Bump timeout to 50m.

```
Cancelling nested steps due to timeout
Sending interrupt signal to process
gcc: fatal error: Terminated signal terminated program lto1
compilation terminated.
make[2]: *** [/tmp/cctcWxAN.mk:134: /tmp/cciCtCWJ.ltrans44.ltrans.o] Error 1
make[2]: *** Waiting for unfinished jobs....
gcc: fatal error: Terminated signal terminated program lto1
compilation terminated.
make[2]: *** [/tmp/cctcWxAN.mk:155: /tmp/cciCtCWJ.ltrans51.ltrans.o] Error 1
lto-wrapper: fatal error: make returned 2 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[1]: *** [nimcache/release/nimbus_beacon_node_gnosis/nimbus_beacon_node_gnosis.makefile:3148: build] Error 1
make: *** [Makefile:699: gnosis-build] Error 2
```
2023-08-07 19:42:28 +00:00
Jakub Sokołowski 54e1a3655e
ci: bump Jenkins maxConcurrentTotal to 9
We have added more powerful hosts with more slots(3 instead of 2)
which should be able to accomodate more Nimbus CI jobs.

https://github.com/status-im/infra-ci/issues/85

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-07 11:50:40 +02:00
zah 8833acbe23
Add support for using custom remote signers in local sim ()
* Add support for using custom remote signers in local sim

Other changes:

* Enable the Nimbus remote signer in the minimal simulation
* Move all log files into the `logs` folder of the simulation
* Create PID files for all processes and use them during the clean-up
  phase instead of the previous more fragile methods for killing the
  remaining processes.
2023-05-25 15:05:38 +00:00
Etan Kissling 18de7a517a
avoid polluting Jenkins CI with geth binaries ()
When `scripts/geth_binaries.sh` was updated, as a side effect of dumping
downloaded Geth tarballs / zips into the repo root directory, artifacts
for Jenkins jobs started preserving those downloads with every job,
repeatedly leading to full disks. Exclude those downloads from Jenkins.
2023-05-14 23:32:45 +02:00
Etan Kissling 5ada931c5d
add CI check for options documentation ()
* add CI check for options documentation

Ensure that documented options reflect available beacon node options.

* escape `\`

* use bash

* override `COLUMNS` for extracting help text
2023-04-10 21:12:39 +00:00
Jakub Sokołowski 60394ba114
ci: add VERBOSITY parameter to Jenkinsfile ()
It will default to `V=0` since it's the first choice in the list.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-27 17:00:27 +01:00
Zahary Karadjov ef619d618c
[Jenkins] Produce artifacts for the REST API test 2023-02-20 13:47:40 +02:00
Etan Kissling c46dc3d7e8
check that tests are up-to-date in CI ()
Enforce the `AllTests-mainnet.md` files and others to be up-to-date.
2022-11-30 02:01:04 +01:00
Etan Kissling 0e32ed0cc0
cancel non-started finalization testnets on cancel ()
When cancelling the `minimal` CI finalization testnet on Jenkins, the
internal Jenkins cancellation exception is being caught by `catchError`
and the `mainnet` CI finalization testnet will still run. Replacing the
logic with `try` / `finally` instead, and moving the `archiveArtifacts`
step to the post-build (to run that even on cancellation / failure).
2022-11-18 16:08:56 +00:00
Etan Kissling 23007eac3b
finalize 🐼 transition in local testnet CI ()
When running local testnets as part of Jenkins CI, add one more epoch
to finalize the merge transition block.
2022-09-19 11:10:47 +00:00
tersec e367f612e8
increase Jenkins timeouts by 10m () 2022-09-07 09:51:08 +00:00
Jakub Sokołowski 5b66cd141e
ci: specify explicit throttle limits for Jenkins ()
Otherwise they appear to not have any effect:
https://issues.jenkins.io/browse/JENKINS-49173
https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/68

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-07-25 16:24:02 +02:00
Etan Kissling fd4cf35c20
fix concurrent Jenkins stages ()
The ports for the concurrently executing REST and Minimal testnet clash,
leading to some CI failures since  introduced further concurrency.
Adjusting the ports to be distinct across various tests should fix this.
2022-07-23 14:28:10 +00:00
Jakub Sokołowski c33989e490
ci: refactor Jenkinsfile to be a pipeline ()
Changes:
- Name local testnet output folders same as the `make` taget.
- Move both `Jenkinsfile`s to `ci` folder to avoid cluttering repo root.
- Separate builds by platform so logs from macos and linux hosts don't get mixed.
- Detect platform and architecture from Jenkins Job path to use one Jenkinsfile.
- Divide shell commands into as many stages as possible to make debugging easier.
- Generalize running testnets via a `launchLocalTestnet()` Groovy method.
- Handle uploading of results of running testnets stage-by-stage basis.
- Use `catchError()` to upload test results while marking job as failed.
- Abort previously started PR build jobs usin `disableConcurrentBuilds()`.
- Throttle jobs using the new `throttleJobProperty()` function.

Builds:
- https://ci.status.im/job/nimbus/job/nimbus-eth2/job/platforms/job/linux/job/x86_64/
- https://ci.status.im/job/nimbus/job/nimbus-eth2/job/platforms/job/macos/job/x86_64/
- https://ci.status.im/job/nimbus/job/nimbus-eth2/job/platforms/job/macos/job/aarch64/

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-07-22 01:34:31 +03:00