Commit Graph

30 Commits

Author SHA1 Message Date
Jakub Sokołowski da48c6f3f4
ci: bump archiving timeout, list artifacts (#6539)
We suspect in some cases the archive is bigger than normally.
I'm adding a listing of the file sizes to catch that.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-09-11 22:50:20 +00:00
Jakub Sokołowski 8c621b9ae6
ci: bump status-jenkins-lib version, don't use a branch (#6446)
* ci: bump status-jenkins-lib version, don't use a branch

Also renamed `Jenkinsfile.nix` to `nix.Jenkinsfile` because the filename makes no sense.

But left a symlink temporarily to not break things.

Signed-off-by: Jakub Sokołowski <jakub@status.im>

* nix: fix hash for Nim compiler checksums repo

Signed-off-by: Jakub Sokołowski <jakub@status.im>

* nix: do not create variable name conflicts

Signed-off-by: Jakub Sokołowski <jakub@status.im>

---------

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-07-25 11:07:02 +02:00
Miran 597f47317f
use Nim 2.0.6 (#6366) 2024-06-18 03:54:15 +02:00
tersec 1051c3c9c5
use upstream/version-2-0 of Nim in CI (#6277) 2024-05-09 02:22:28 +02:00
Jakub Sokołowski 7acf383da2
ci: pin Nim for PR builds to v2.0.4 temporarily (#6267)
Mitigation for:
https://github.com/nim-lang/Nim/issues/23568

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-05-06 22:33:53 +00:00
Jakub Sokołowski 4a1a2c8196
add packages to Nix flake config, move to repo root
This way we can actually build and run a node using just:
```sh
nix run 'github:status-im/nimbus-eth2?submodules=1'
```
The `?submodules=1` part should eventually not be necessary.
For more details see:
https://github.com/NixOS/nix/issues/4423

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-05-06 16:57:23 +02:00
Jakub Sokołowski 824bb17115
ci: use clang 18 from Homebrew on aarch64
Not sure if `LDFLAGS` and `CPPFLAGS` are absolutely necessary but Brew docs recommend it.

Depends on:
https://github.com/status-im/infra-ci/commit/67fafcb5
https://github.com/status-im/status-jenkins-lib/pull/90

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-04-25 14:39:19 +02:00
Jakub Sokołowski 80edebfd20
fix Nim v2 builds by checking for correct job name (#6218)
Since job folder structure has been changed we don't need the dash.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-04-18 17:08:44 +02:00
tersec c2a2b76f93
allow Jenkins to specify Nim version to use (#6162) 2024-04-03 10:47:32 +00:00
Etan Kissling e677d98119
fix check docs stage in jenkins (#6156)
Ignore color information when checking consistency of options.md with
the built binary.
2024-03-28 21:37:21 +01:00
Jakub Sokołowski 0fa363e022
ci: move help docs check script to its own file (#6140)
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-03-28 10:01:15 +01:00
tersec 06e2ca5286
don't explicitly filter removed file parsedBeaconState.json from holesky (#5584) 2023-11-09 17:45:07 +03:00
Etan Kissling eb7c8b7db2
do not download Holesky genesis on `git clone` (#5573)
Holesky genesis.ssz file may be unavailable due to quota limits on
`eth-clients/holesky`; do not download it by default during checkout.
Nimbus will download it on first startup from a mirror instead.
2023-11-06 12:10:36 +01:00
Etan Kissling 63440766ef
exclude 'public-keys/all.txt' file from checkout (#5439)
The 'vendor/holesky/public-keys/all.txt' file is not used by Nimbus,
but sometimes fails to download due to payment issues in `@eth-clients`.
Skipping the download of that file should reduce that risk.

```
Downloading public-keys/all.txt (240 MB)
Error downloading object: public-keys/all.txt (ba977b5): Smudge error: Error downloading public-keys/all.txt (ba977b5b1da180914c934623fce187860e1b54cff939e6208533b2cb5f589e07): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
```

* bump nbs: Patch libnatpmp.a rule to support llvm-mingw in targets.mk

* bump nbs: Allow passing config to submodule initialization in `update-common`
2023-09-18 20:24:13 +02:00
Etan Kissling 8993d576b8
refer to full path in lint hint (#5372)
Update hint message for correcting outdated options.md with full path
and full command.
2023-08-30 11:55:29 +00:00
Etan Kissling 6daa7542a7
add hint when `options.md` is outdated (#5367)
Make it clearer what went wrong when lint fails due to 'options.md'
being out of date.
2023-08-29 12:16:15 +00:00
Etan Kissling bb672a7e90
bump Jenkins build timeout (#5270)
With the gnosis binary, building all targets sometimes exceeds the 40m.
Bump timeout to 50m.

```
Cancelling nested steps due to timeout
Sending interrupt signal to process
gcc: fatal error: Terminated signal terminated program lto1
compilation terminated.
make[2]: *** [/tmp/cctcWxAN.mk:134: /tmp/cciCtCWJ.ltrans44.ltrans.o] Error 1
make[2]: *** Waiting for unfinished jobs....
gcc: fatal error: Terminated signal terminated program lto1
compilation terminated.
make[2]: *** [/tmp/cctcWxAN.mk:155: /tmp/cciCtCWJ.ltrans51.ltrans.o] Error 1
lto-wrapper: fatal error: make returned 2 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[1]: *** [nimcache/release/nimbus_beacon_node_gnosis/nimbus_beacon_node_gnosis.makefile:3148: build] Error 1
make: *** [Makefile:699: gnosis-build] Error 2
```
2023-08-07 19:42:28 +00:00
Jakub Sokołowski 54e1a3655e
ci: bump Jenkins maxConcurrentTotal to 9
We have added more powerful hosts with more slots(3 instead of 2)
which should be able to accomodate more Nimbus CI jobs.

https://github.com/status-im/infra-ci/issues/85

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-07-07 11:50:40 +02:00
zah 8833acbe23
Add support for using custom remote signers in local sim (#4989)
* Add support for using custom remote signers in local sim

Other changes:

* Enable the Nimbus remote signer in the minimal simulation
* Move all log files into the `logs` folder of the simulation
* Create PID files for all processes and use them during the clean-up
  phase instead of the previous more fragile methods for killing the
  remaining processes.
2023-05-25 15:05:38 +00:00
Etan Kissling 18de7a517a
avoid polluting Jenkins CI with geth binaries (#4952)
When `scripts/geth_binaries.sh` was updated, as a side effect of dumping
downloaded Geth tarballs / zips into the repo root directory, artifacts
for Jenkins jobs started preserving those downloads with every job,
repeatedly leading to full disks. Exclude those downloads from Jenkins.
2023-05-14 23:32:45 +02:00
Etan Kissling 5ada931c5d
add CI check for options documentation (#4804)
* add CI check for options documentation

Ensure that documented options reflect available beacon node options.

* escape `\`

* use bash

* override `COLUMNS` for extracting help text
2023-04-10 21:12:39 +00:00
Jakub Sokołowski 60394ba114
ci: add VERBOSITY parameter to Jenkinsfile (#4676)
It will default to `V=0` since it's the first choice in the list.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-27 17:00:27 +01:00
Zahary Karadjov ef619d618c
[Jenkins] Produce artifacts for the REST API test 2023-02-20 13:47:40 +02:00
Etan Kissling c46dc3d7e8
check that tests are up-to-date in CI (#4373)
Enforce the `AllTests-mainnet.md` files and others to be up-to-date.
2022-11-30 02:01:04 +01:00
Etan Kissling 0e32ed0cc0
cancel non-started finalization testnets on cancel (#4334)
When cancelling the `minimal` CI finalization testnet on Jenkins, the
internal Jenkins cancellation exception is being caught by `catchError`
and the `mainnet` CI finalization testnet will still run. Replacing the
logic with `try` / `finally` instead, and moving the `archiveArtifacts`
step to the post-build (to run that even on cancellation / failure).
2022-11-18 16:08:56 +00:00
Etan Kissling 23007eac3b
finalize 🐼 transition in local testnet CI (#4142)
When running local testnets as part of Jenkins CI, add one more epoch
to finalize the merge transition block.
2022-09-19 11:10:47 +00:00
tersec e367f612e8
increase Jenkins timeouts by 10m (#4086) 2022-09-07 09:51:08 +00:00
Jakub Sokołowski 5b66cd141e
ci: specify explicit throttle limits for Jenkins (#3913)
Otherwise they appear to not have any effect:
https://issues.jenkins.io/browse/JENKINS-49173
https://github.com/jenkinsci/throttle-concurrent-builds-plugin/pull/68

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-07-25 16:24:02 +02:00
Etan Kissling fd4cf35c20
fix concurrent Jenkins stages (#3904)
The ports for the concurrently executing REST and Minimal testnet clash,
leading to some CI failures since #3827 introduced further concurrency.
Adjusting the ports to be distinct across various tests should fix this.
2022-07-23 14:28:10 +00:00
Jakub Sokołowski c33989e490
ci: refactor Jenkinsfile to be a pipeline (#3827)
Changes:
- Name local testnet output folders same as the `make` taget.
- Move both `Jenkinsfile`s to `ci` folder to avoid cluttering repo root.
- Separate builds by platform so logs from macos and linux hosts don't get mixed.
- Detect platform and architecture from Jenkins Job path to use one Jenkinsfile.
- Divide shell commands into as many stages as possible to make debugging easier.
- Generalize running testnets via a `launchLocalTestnet()` Groovy method.
- Handle uploading of results of running testnets stage-by-stage basis.
- Use `catchError()` to upload test results while marking job as failed.
- Abort previously started PR build jobs usin `disableConcurrentBuilds()`.
- Throttle jobs using the new `throttleJobProperty()` function.

Builds:
- https://ci.status.im/job/nimbus/job/nimbus-eth2/job/platforms/job/linux/job/x86_64/
- https://ci.status.im/job/nimbus/job/nimbus-eth2/job/platforms/job/macos/job/x86_64/
- https://ci.status.im/job/nimbus/job/nimbus-eth2/job/platforms/job/macos/job/aarch64/

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-07-22 01:34:31 +03:00