From 2f3f56898f3251254523703cc3a3dec905eedbee Mon Sep 17 00:00:00 2001 From: Darshan <35736874+darshankabariya@users.noreply.github.com> Date: Wed, 18 Feb 2026 11:51:15 +0530 Subject: [PATCH] fix: update release process (#3727) --- .../ISSUE_TEMPLATE/prepare_beta_release.md | 27 ++++++---- .../ISSUE_TEMPLATE/prepare_full_release.md | 53 +++++++++++-------- docs/contributors/release-process.md | 33 +++++++----- 3 files changed, 67 insertions(+), 46 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/prepare_beta_release.md b/.github/ISSUE_TEMPLATE/prepare_beta_release.md index 2e4226e67..3c4e76854 100644 --- a/.github/ISSUE_TEMPLATE/prepare_beta_release.md +++ b/.github/ISSUE_TEMPLATE/prepare_beta_release.md @@ -21,15 +21,21 @@ All items below are to be completed by the owner of the given release. - [ ] Assign release candidate tag to the release branch HEAD (e.g. `v0.X.0-beta-rc.0`, `v0.X.0-beta-rc.1`, ... `v0.X.0-beta-rc.N`). - [ ] Generate and edit release notes in CHANGELOG.md. -- [ ] **Waku test and fleets validation** - - [ ] Ensure all the unit tests (specifically logos-delivery-js tests) are green against the release candidate. - - [ ] Deploy the release candidate to `waku.test` only through [deploy-waku-test job](https://ci.infra.status.im/job/nim-waku/job/deploy-waku-test/) and wait for it to finish (Jenkins access required; ask the infra team if you don't have it). - - After completion, disable [deployment job](https://ci.infra.status.im/job/nim-waku/) so that its version is not updated on every merge to master. - - Verify the deployed version at https://fleets.waku.org/. - - Confirm the container image exists on [Harbor](https://harbor.status.im/harbor/projects/9/repositories/logos-delivery/artifacts-tab). - - [ ] Analyze Kibana logs from the previous month (since the last release was deployed) for possible crashes or errors in `waku.test`. - - Most relevant logs are `(fleet: "waku.test" AND message: "SIGSEGV")`. - - [ ] Enable again the `waku.test` fleet to resume auto-deployment of the latest `master` commit. +- [ ] **Validation of release candidate** + - [ ] **Automated testing** + - [ ] Ensure all the unit tests (specifically logos-messaging-js tests) are green against the release candidate. + - [ ] **Waku fleet testing** + - [ ] Deploy the release candidate to `waku.test` through [deploy-waku-test job](https://ci.infra.status.im/job/nim-waku/job/deploy-waku-test/) and wait for it to finish (Jenkins access required; ask the infra team if you don't have it). + - After completion, disable fleet so that daily CI does not override your release candidate. + - Verify at https://fleets.waku.org/ that the fleet is locked to the release candidate image. + - Confirm the container image exists on [Harbor](https://harbor.status.im/harbor/projects/9/repositories/nwaku/artifacts-tab). + - [ ] Search [Kibana logs](https://kibana.infra.status.im/app/discover) from the previous month (since the last release was deployed) for possible crashes or errors in `waku.test`. + - Set time range to "Last 30 days" (or since last release). + - Most relevant search query: `(fleet: "waku.test" AND message: "SIGSEGV")`, `(fleet: "waku.test" AND message: "exception")`, `(fleet: "waku.test" AND message: "error")`. + - Document any crashes or errors found. + - [ ] If `waku.test` validation is successful, deploy to `waku.sandbox` using the [deploy-waku-sandbox job](https://ci.infra.status.im/job/nim-waku/job/deploy-waku-sandbox/). + - [ ] Search [Kibana logs](https://kibana.infra.status.im/app/discover) for `waku.sandbox`: `(fleet: "waku.sandbox" AND message: "SIGSEGV")`, `(fleet: "waku.sandbox" AND message: "exception")`, `(fleet: "waku.sandbox" AND message: "error")`. most probably if there are no crashes or errors in `waku.test`, there will be no crashes or errors in `waku.sandbox`. + - [ ] Enable the `waku.test` fleet again to resume auto-deployment of the latest `master` commit. - [ ] **Proceed with release** @@ -53,4 +59,5 @@ All items below are to be completed by the owner of the given release. - [Infra-nim-waku](https://github.com/status-im/infra-nim-waku) - [Jenkins](https://ci.infra.status.im/job/nim-waku/) - [Fleets](https://fleets.waku.org/) -- [Harbor](https://harbor.status.im/harbor/projects/9/repositories/logos-delivery/artifacts-tab) +- [Harbor](https://harbor.status.im/harbor/projects/9/repositories/nwaku/artifacts-tab) +- [Kibana](https://kibana.infra.status.im/app/) diff --git a/.github/ISSUE_TEMPLATE/prepare_full_release.md b/.github/ISSUE_TEMPLATE/prepare_full_release.md index d919f9ed0..4df808bd4 100644 --- a/.github/ISSUE_TEMPLATE/prepare_full_release.md +++ b/.github/ISSUE_TEMPLATE/prepare_full_release.md @@ -24,33 +24,39 @@ All items below are to be completed by the owner of the given release. - [ ] **Validation of release candidate** - [ ] **Automated testing** - - [ ] Ensure all the unit tests (specifically logos-delivery-js tests) are green against the release candidate. - - [ ] Ask Vac-QA and Vac-DST to perform the available tests against the release candidate. - - [ ] Vac-DST (an additional report is needed; see [this](https://www.notion.so/DST-Reports-1228f96fb65c80729cd1d98a7496fe6f)) + - [ ] Ensure all the unit tests (specifically logos-messaging-js tests) are green against the release candidate. - [ ] **Waku fleet testing** - - [ ] Deploy the release candidate to `waku.test` and `waku.sandbox` fleets. - - Start the [deployment job](https://ci.infra.status.im/job/nim-waku/) for both fleets and wait for it to finish (Jenkins access required; ask the infra team if you don't have it). - - After completion, disable [deployment job](https://ci.infra.status.im/job/nim-waku/) so that its version is not updated on every merge to `master`. - - Verify the deployed version at https://fleets.waku.org/. + - [ ] Deploy the release candidate to `waku.test` fleet. + - Start the [deployment job](https://ci.infra.status.im/job/nim-waku/) and wait for it to finish (Jenkins access required; ask the infra team if you don't have it). + - After completion, disable fleet so that daily CI does not override your release candidate. + - Verify at https://fleets.waku.org/ that the fleet is locked to the release candidate image. - Confirm the container image exists on [Harbor](https://harbor.status.im/harbor/projects/9/repositories/nwaku/artifacts-tab). - - [ ] Search _Kibana_ logs from the previous month (since the last release was deployed) for possible crashes or errors in `waku.test` and `waku.sandbox`. - - Most relevant logs are `(fleet: "waku.test" AND message: "SIGSEGV")` OR `(fleet: "waku.sandbox" AND message: "SIGSEGV")`. - - [ ] Enable again the `waku.test` fleet to resume auto-deployment of the latest `master` commit. + - [ ] Search [Kibana logs](https://kibana.infra.status.im/app/discover) from the previous month (since the last release was deployed) for possible crashes or errors in `waku.test`. + - Set time range to "Last 30 days" (or since last release). + - Most relevant search query: `(fleet: "waku.test" AND message: "SIGSEGV")`, `(fleet: "waku.test" AND message: "exception")`, `(fleet: "waku.test" AND message: "error")`. + - Document any crashes or errors found. + - [ ] If `waku.test` validation is successful, deploy to `waku.sandbox` using the same [deployment job](https://ci.infra.status.im/job/nim-waku/). + - [ ] Search [Kibana logs](https://kibana.infra.status.im/app/discover) for `waku.sandbox`: `(fleet: "waku.sandbox" AND message: "SIGSEGV")`, `(fleet: "waku.sandbox" AND message: "exception")`, `(fleet: "waku.sandbox" AND message: "error")`. most probably if there are no crashes or errors in `waku.test`, there will be no crashes or errors in `waku.sandbox`. + - [ ] Enable the `waku.test` fleet again to resume auto-deployment of the latest `master` commit. -- [ ] **Status fleet testing** - - [ ] Deploy release candidate to `status.staging` - - [ ] Perform [sanity check](https://www.notion.so/How-to-test-Nwaku-on-Status-12c6e4b9bf06420ca868bd199129b425) and log results as comments in this issue. - - [ ] Connect 2 instances to `status.staging` fleet, one in relay mode, the other one in light client. - - 1:1 Chats with each other - - Send and receive messages in a community - - Close one instance, send messages with second instance, reopen first instance and confirm messages sent while offline are retrieved from store - - [ ] Perform checks based on _end user impact_ - - [ ] Inform other (Waku and Status) CCs to point their instances to `status.staging` for a few days. Ping Status colleagues on their Discord server or in the [Status community](https://status.app/c/G3kAAMSQtb05kog3aGbr3kiaxN4tF5xy4BAGEkkLwILk2z3GcoYlm5hSJXGn7J3laft-tnTwDWmYJ18dP_3bgX96dqr_8E3qKAvxDf3NrrCMUBp4R9EYkQez9XSM4486mXoC3mIln2zc-TNdvjdfL9eHVZ-mGgs=#zQ3shZeEJqTC1xhGUjxuS4rtHSrhJ8vUYp64v6qWkLpvdy9L9) (this is not a blocking point.) - - [ ] Ask Status-QA to perform sanity checks (as described above) and checks based on _end user impact_; specify the version being tested - - [ ] Ask Status-QA or infra to run the automated Status e2e tests against `status.staging` - - [ ] Get other CCs' sign-off: they should comment on this PR, e.g., "Used the app for a week, no problem." If problems are reported, resolve them and create a new RC. - - [ ] **Get Status-QA sign-off**, ensuring that the `status.test` update will not disturb ongoing activities. + - [ ] **QA and DST testing** + - [ ] Ask Vac-QA and Vac-DST to run their available tests against the release candidate; share all release candidates with both teams. + - [ ] Vac-DST: An additional report is needed ([see this example](https://www.notion.so/DST-Reports-1228f96fb65c80729cd1d98a7496fe6f)). Inform DST team about what are the expectations for this rc. For example, if we expect higher or lower bandwidth consumption. + + - [ ] **Status fleet testing** + - [ ] Deploy release candidate to `status.staging` + - [ ] Perform [sanity check](https://www.notion.so/How-to-test-Nwaku-on-Status-12c6e4b9bf06420ca868bd199129b425) and log results as comments in this issue. + - [ ] Connect 2 instances to `status.staging` fleet, one in relay mode, the other one in light client. + - 1:1 Chats with each other + - Send and receive messages in a community + - Close one instance, send messages with second instance, reopen first instance and confirm messages sent while offline are retrieved from store + - [ ] Perform checks based on _end user impact_ + - [ ] Inform other (Waku and Status) CCs to point their instances to `status.staging` for a few days. Ping Status colleagues on their Discord server or in the [Status community](https://status.app/c/G3kAAMSQtb05kog3aGbr3kiaxN4tF5xy4BAGEkkLwILk2z3GcoYlm5hSJXGn7J3laft-tnTwDWmYJ18dP_3bgX96dqr_8E3qKAvxDf3NrrCMUBp4R9EYkQez9XSM4486mXoC3mIln2zc-TNdvjdfL9eHVZ-mGgs=#zQ3shZeEJqTC1xhGUjxuS4rtHSrhJ8vUYp64v6qWkLpvdy9L9) (this is not a blocking point.) + - [ ] Ask Status-QA to perform sanity checks (as described above) and checks based on _end user impact_; specify the version being tested + - [ ] Ask Status-QA or infra to run the automated Status e2e tests against `status.staging` + - [ ] Get other CCs' sign-off: they should comment on this PR, e.g., "Used the app for a week, no problem." If problems are reported, resolve them and create a new RC. + - [ ] **Get Status-QA sign-off**, ensuring that the `status.test` update will not disturb ongoing activities. - [ ] **Proceed with release** @@ -74,3 +80,4 @@ All items below are to be completed by the owner of the given release. - [Jenkins](https://ci.infra.status.im/job/nim-waku/) - [Fleets](https://fleets.waku.org/) - [Harbor](https://harbor.status.im/harbor/projects/9/repositories/nwaku/artifacts-tab) +- [Kibana](https://kibana.infra.status.im/app/) diff --git a/docs/contributors/release-process.md b/docs/contributors/release-process.md index bde63aa6f..8aa9282cd 100644 --- a/docs/contributors/release-process.md +++ b/docs/contributors/release-process.md @@ -20,7 +20,7 @@ For more context, see https://trunkbaseddevelopment.com/branch-for-release/ - **Full release**: follow the entire [Release process](#release-process--step-by-step). -- **Beta release**: skip just `6a` and `6c` steps from [Release process](#release-process--step-by-step). +- **Beta release**: skip just `6c` and `6d` steps from [Release process](#release-process--step-by-step). - Choose the appropriate release process based on the release type: - [Full Release](../../.github/ISSUE_TEMPLATE/prepare_full_release.md) @@ -70,20 +70,26 @@ For more context, see https://trunkbaseddevelopment.com/branch-for-release/ 6a. **Automated testing** - Ensure all the unit tests (specifically js-waku tests) are green against the release candidate. - - Ask Vac-QA and Vac-DST to run their available tests against the release candidate; share all release candidates with both teams. - - > We need an additional report like [this](https://www.notion.so/DST-Reports-1228f96fb65c80729cd1d98a7496fe6f) specifically from the DST team. 6b. **Waku fleet testing** - - Start job on `waku.sandbox` and `waku.test` [Deployment job](https://ci.infra.status.im/job/nim-waku/), wait for completion of the job. If it fails, then debug it. - - After completion, disable [deployment job](https://ci.infra.status.im/job/nim-waku/) so that its version is not updated on every merge to `master`. - - Verify at https://fleets.waku.org/ that the fleet is locked to the release candidate version. + - Start job on `waku.test` [Deployment job](https://ci.infra.status.im/job/nim-waku/), wait for completion of the job. If it fails, then debug it. + - After completion, disable fleet so that daily ci not override your release candidate. + - Verify at https://fleets.waku.org/ that the fleet is locked to the release candidate image. - Check if the image is created at [Harbor](https://harbor.status.im/harbor/projects/9/repositories/nwaku/artifacts-tab). - - Search _Kibana_ logs from the previous month (since the last release was deployed) for possible crashes or errors in `waku.test` and `waku.sandbox`. - - Most relevant logs are `(fleet: "waku.test" AND message: "SIGSEGV")` OR `(fleet: "waku.sandbox" AND message: "SIGSEGV")`. + - Search [Kibana logs](https://kibana.infra.status.im/app/discover) from the previous month (since the last release was deployed) for possible crashes or errors in `waku.test`. + - Set time range to "Last 30 days" (or since last release). + - Most relevant search query: `(fleet: "waku.test" AND message: "SIGSEGV")`, `(fleet: "waku.test" AND message: "exception")`, `(fleet: "waku.test" AND message: "error")`. + - Document any crashes or errors found. + - If `waku.test` validation is successful, deploy to `waku.sandbox` using the same [Deployment job](https://ci.infra.status.im/job/nim-waku/). + - Search [Kibana logs](https://kibana.infra.status.im/app/discover) for `waku.sandbox`: `(fleet: "waku.sandbox" AND message: "SIGSEGV")`, `(fleet: "waku.sandbox" AND message: "exception")`, `(fleet: "waku.sandbox" AND message: "error")`. most probably if there are no crashes or errors in `waku.test`, there will be no crashes or errors in `waku.sandbox`. - Enable the `waku.test` fleet again to resume auto-deployment of the latest `master` commit. - 6c. **Status fleet testing** + 6c. **QA and DST testing** + - Ask Vac-QA and Vac-DST to run their available tests against the release candidate; share all release candidates with both teams. + + > We need an additional report like [this](https://www.notion.so/DST-Reports-1228f96fb65c80729cd1d98a7496fe6f) specifically from the DST team. Inform DST team about what are the expectations for this rc. For example, if we expect higher or lower bandwidth consumption. + + 6d. **Status fleet testing** - Deploy release candidate to `status.staging` - Perform [sanity check](https://www.notion.so/How-to-test-Nwaku-on-Status-12c6e4b9bf06420ca868bd199129b425) and log results as comments in this issue. - Connect 2 instances to `status.staging` fleet, one in relay mode, the other one in light client. @@ -120,10 +126,10 @@ We also need to merge the release branch back into master as a final step. 2. Deploy the release image to [Dockerhub](https://hub.docker.com/r/wakuorg/nwaku) by triggering [the manual Jenkins deployment job](https://ci.infra.status.im/job/nim-waku/job/docker-manual/). > Ensure the following build parameters are set: > - `MAKE_TARGET`: `wakunode2` - > - `IMAGE_TAG`: the release tag (e.g. `v0.36.0`) + > - `IMAGE_TAG`: the release tag (e.g. `v0.38.0`) > - `IMAGE_NAME`: `wakuorg/nwaku` > - `NIMFLAGS`: `--colors:off -d:disableMarchNative -d:chronicles_colors:none -d:postgres` - > - `GIT_REF` the release tag (e.g. `v0.36.0`) + > - `GIT_REF` the release tag (e.g. `v0.38.0`) ### Performing a patch release @@ -154,4 +160,5 @@ We also need to merge the release branch back into master as a final step. - [Infra-nim-waku](https://github.com/status-im/infra-nim-waku) - [Jenkins](https://ci.infra.status.im/job/nim-waku/) - [Fleets](https://fleets.waku.org/) -- [Harbor](https://harbor.status.im/harbor/projects/9/repositories/nwaku/artifacts-tab) \ No newline at end of file +- [Harbor](https://harbor.status.im/harbor/projects/9/repositories/nwaku/artifacts-tab) +- [Kibana](https://kibana.infra.status.im/app/) \ No newline at end of file