Commit Graph

65 Commits

Author SHA1 Message Date
Igor Sirotin ea290d97ef
feat_: Integrate Codecov (#5796)
* feat_: added `codecov-cli` nix derivation

* feat_: added simple codecov configuration

* feat_: add UNIT_TEST_REPORT_CODECOV ci flag

* feat_: upload logs and reports to codecov
2024-09-05 10:11:16 +01:00
Igor Sirotin 7623f68679
fix_: nightly test runs (#5791)
* fix_: run nightly protocol tests as separate processes

* fix_: calculate timeout

* fix_: TEST_WITH_COVERAGE_REPORTS_DIR

* fix_: proper filter protocol package

* chore_: log run_unit_tests stages

* fix_: coverage reports merging

* chore_: more logs

* chore_: fix typo

* chore_: increase test timeouts

* fix_: properly filter packages

* feat_: UNIT_TEST_DRY_RUN flag for tests run

* fix_: UNIT_TEST_PACKAGES_FILTERED calculation

* fix_: remove the force-run waku test first

* fix_: delete unused file
2024-09-03 12:50:09 +01:00
Igor Sirotin 85fba77b7d
chore_: bring back jenkins commit check (#5777) 2024-08-27 23:40:17 +01:00
Igor Sirotin 8491e76a34
fix_: separate commit message check (#5773)
* chore_: add github action conventional commits

* chore_: remove commit check from tests run

* fix_: continue checking commits when breaking change found

* fix_: don't run check on pr edit
2024-08-27 22:42:06 +01:00
Igor Sirotin cc722359b5
chore_: run tests concurrently (#5731)
* chore_: fake flaky tests

* chore_: run tests in parallel (wip)

* chore_: test-with-coverage-old

* chore_: lint-fix fake flaky tests

* chore_: cleanup

* chore_:

* chore_: lint-fix fake flaky tests

* chore_: enable running some tests sequentially

* chore_: enable back all tests

* chore_: no sequential packages

* chore_: cleanup

* chore_: remove fake flaky packages

* chore_: cleanup

* chore_: attempt UNIT_TEST_COUNT as test count

* chore_: spawn multiple processes

* fix_: don't spawn multiple processes

* chore_: revert redirect_stdout changes

* fix_: update Jenkinsfile

* fix_: use env variables

* fix_: typo
2024-08-26 10:03:26 +01:00
Patryk Osmaczko 8914a79284 chore(ci)_: adjust nightly tests build retention to 14 runs
Increased build and artifact retention, as more context is usually
needed when analyzing flakiness thorugh nightly tests.
2024-08-21 15:32:55 +02:00
Jakub Sokołowski bc984e0cb1
chore(ci)_: fix formatting, bump library version
Also drop unused `_assets/ci/validate-vendor.sh` script.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-08-20 09:40:22 +02:00
richΛrd bb6d5f602b
chore_: fix `TestBasicWakuV2` (#5718) 2024-08-15 14:43:37 -04:00
Igor Sirotin dd443b030d
fix(commit_check)_: use proper base branch (#5700) 2024-08-13 10:58:48 +01:00
Igor Sirotin 325429b218
chore_: add test-coverage.html report (#5490) 2024-07-10 13:37:50 +01:00
Jakub Sokołowski 769ea7799b
feat(ci)_: add ansiColor(xterm) for nice colors
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-07-01 09:47:53 +02:00
Jakub Sokołowski b6d2b9eb53
fix(ci)_: fix Nix version label and bump library
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-06-26 17:55:26 +02:00
Igor Sirotin d2f4cae18f
fix: merge coverage reports from each test rerun (#5360)
* fix_: fix test coverage when rerunning tests

* fix_: make lint-fix

* chore_: change test coverage report artifact name

* chore_: added codeclimate.json to artifacts

* chore_: bring in gocovmerge util

---------

Co-authored-by: Siddarth Kumar <siddarthkay@gmail.com>
2024-06-19 11:40:52 +01:00
Patryk Osmaczko e9882e88d3 chore(ci)_: archive logs in case of test failure 2024-06-17 17:17:54 +02:00
Igor Sirotin 12dfe285d7
chore_: archive test coverage result as artifact (#5280) 2024-06-03 17:31:27 +01:00
Igor Sirotin 0dde5a3b17
chore_: enable nightly codeclimate reports (#5127)
* chore_: enable nightly codeclimate reports
* fix_: report codeclimate before exiting
2024-05-08 21:57:50 +01:00
Jakub Sokołowski a97f1bb681
chore(ci)_: simplify Jenkinsfile for tests, fix params (#5136)
Because most of those functions just obscure what's happening.
Also formatting.

Also, it appears Jenkins sets `params` on first run, but not `env`
variables from those `params`.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2024-05-08 21:04:15 +02:00
Igor Sirotin 068ae3b67c
chore_: setup ci codeclimate coverage reports (#5101)
* chore_: nix derivation with cc-test-reporter
2024-05-08 09:25:01 +01:00
Patryk Osmaczko 853afccfae revert_: "ci: temporarily disable migration check"
This reverts commit d4c0500200e2decd85668a02dbf6c12d92678322.
2024-04-10 16:40:57 +02:00
Patryk Osmaczko 448758e64a ci_: temporarily disable migration check
Temporary, to cherry-pick migration from release branch.
2024-04-10 14:27:14 +02:00
Andrea Maria Piana 8c7a6aa54d chore_: Add tag-version command 2024-04-10 11:43:13 +01:00
Alexis Pentori f9df853e4a
nix: Updating Nix Shell and Makefile to use nix-shell
Signed-off-by: Alexis Pentori <alexis@status.im>
2024-04-09 12:49:06 +03:00
Patryk Osmaczko c98acf76f9 fix: prevent migration check on nightly tests
mitigates: #4993
2024-03-28 14:41:18 +01:00
Patryk Osmaczko 30fe620ff0 chore: extend tests timeout
Due to an increased volume of tests, numerous test jobs have been timing
out. The timeout has been extended by 10 minutes.
2024-03-26 17:47:11 +01:00
Patryk Osmaczko a068b64d5e chore: run nightly tests one by one
fixes: #4585
fixes: #4586
2024-02-28 22:00:26 +01:00
Anton Iakimov c3240b296c
ci: cleanup more build artifacts
Should be cleaning artifacts for more than 1+1 build for each branch.
2024-02-28 15:47:23 +01:00
Patryk Osmaczko fdd979096c chore: add capability to rerun failed tests
- Use `gotestsum` to generate junit test reports and re-run failed tests.
- Archive re-run reports in `${package_dir}/report_rerun_fails.txt`.
2024-02-27 11:00:29 +01:00
Anton Iakimov d2379aa129
ci: publish tests-nightly log/xml for debug (#4384) 2023-11-30 12:09:16 +01:00
Jakub Sokołowski 568371e836
chore(ci): update Nix Jenkins label to 2.14 version
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-11-29 15:15:51 +01:00
Anton Iakimov 1f076b9ff3
ci: set tests-nightly timeout to 5 hours 2023-11-27 12:42:09 +01:00
Anton Iakimov 2d251e9a08
ci: increase nightly runs and add more info to test stats (#4378) 2023-11-24 17:42:44 +01:00
Anton Iakimov c62957f621
feat: publish test_stats report (#4349)
Let's publish test_stats as a Jenkins report instead of printing to the
build log.
2023-11-22 11:37:05 +01:00
Anton Iakimov d1a9ad599c
feat: detect flaky tests
This reverts commit f12803ec95.
2023-11-21 17:03:37 +01:00
Anton Iakimov f12803ec95
Revert "feat: detect flaky tests"
There are signs that test runs became unstable.
Might be related to this commit.

This reverts commit 9f168eacea.
2023-11-21 16:34:43 +01:00
Anton Iakimov 9f168eacea
feat: detect flaky tests 2023-11-20 15:08:51 +01:00
Jakub Sokołowski 494c6707ba
ci: parametrize DB port for unit tests
Otherwise we can't run tests in parallel on the same host. Also the
container name has to be different depending on executor.

Resolves errors like:
```
docker: Error response from daemon: Conflict.
    The container name "/status-go-test-db" is already in use by container "123...".
    You have to remove (or rename) that container to be able to reuse that name.
```
Resolves: https://github.com/status-im/status-go/issues/4040

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-18 19:54:21 +02:00
Andrea Maria Piana 9a1443d716 Test generate handlers 2023-08-28 14:17:33 +01:00
Andrea Maria Piana 8dd1b66d69 Always use protobufs by reference & generate handlers 2023-08-22 12:08:54 +01:00
Jakub Sokołowski c0c74010e7
ci: set GOCACHE to ws tmp to avoid cache poisoning
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-18 18:01:19 +02:00
Jakub Sokołowski 78b61ffe2c
ci: drop wrongly calling make install-lint
Linting tools should already be installed by Nix, and we should use
them.

Also switching most shells to pure mode to see what happens.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-08-18 18:01:18 +02:00
Anton Iakimov 3a41a2550a
ci: generate junit reports for unit tests 2023-08-17 15:29:18 +02:00
Andrea Maria Piana 5f55a7bcc8 fetch develop before checking 2023-06-28 19:58:21 +01:00
Andrea Maria Piana 75c9374420 Add migration check script 2023-06-23 12:01:30 +01:00
Jakub Sokołowski c1509637a5
ci: bump tests timeout to 30 minutes
The test running time can be a bit unpredictable due to server load,
which can result in tests that finish in 5 minutes, or over 20.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-02 12:38:43 +02:00
Jakub Sokołowski 589cc965e3
ci: add timeouts to all Jenkinsfiles
This is especially important for tests.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-29 12:40:08 +02:00
Jakub Sokołowski 22bc76c39f
ci: use Nix shell for Linux library builds
Just noticed that was the only one not using it.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-06 01:12:14 +02:00
Jakub Sokołowski 34f7354e6c
ci: upgrade Jenkins library to fix getting version
Fixed in:
https://github.com/status-im/status-jenkins-lib/pull/56

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-22 13:29:28 +01:00
Jakub Sokołowski 78c677742e
ci: delete Postgres DB container after tests run
Otherwise we end up with leftover containers hanging around:
```
admin@linux-03.he-eu-hel1.ci.devel:~ % d ps -a
CONTAINER ID   NAMES                IMAGE                 CREATED          STATUS
6c683f3083bf   boring_yonath        postgres:9.6-alpine   20 minutes ago   Created
b3d462925a91   dazzling_albattani   postgres:9.6-alpine   7 days ago       Created
```

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-21 13:57:10 +01:00
Jakub Sokołowski 0048b70466
ci: use WORKSPACE_TMP for TMPDIR to not abuse /run
Because a lot of tests use `MkdirTemp` and do not clean up.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-01-03 16:54:22 +01:00
Jakub Sokołowski 9dd80e7f1e
ci: use Nix shell to provide Android SDK for builds
Installing the SDK via Ansible is prone to error and not exactly
reproduceable. This way we can also track the exact tooling version as
used in Status Mobile app:
https://github.com/status-im/status-mobile/blob/develop/nix/pkgs.nix

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-12-05 15:48:27 +01:00