status-im/status-react
2
0
Fork 0
mirror of https://github.com/status-im/status-react.git synced 2025-02-19 22:28:40 +00:00
Code Issues Projects Releases Wiki Activity

e2e: updating PR pipeline (#19478)

Browse Source
This commit is contained in:
Tetiana Churikova 2024-04-03 18:00:04 +02:00 committed by GitHub
parent f114908c4d
commit 354cda3be5
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 48 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

60
doc/pipeline_process.md
View File

@ -2,7 +2,7 @@
[Pipeline for QA](https://github.com/status-im/status-mobile/projects/7) is a project board for developers and testers used to track the status of a pull request, get reviews and manual testing, and run autotests. [Pipeline for QA](https://github.com/status-im/status-mobile/projects/7) is a project board for developers and testers used to track the status of a pull request, get reviews and manual testing, and run autotests.
The generally accepted recommendations for its use are described below: The generally accepted requirements for its use are described below:
## Opening a PR ## Opening a PR
- Once a PR is created, it moves to the ```REVIEW``` column where a review will be requested automatically. - Once a PR is created, it moves to the ```REVIEW``` column where a review will be requested automatically.
@ -13,35 +13,70 @@ The generally accepted recommendations for its use are described below:
### What if the work is still in progress? ### What if the work is still in progress?
- If PR work is not finished yet, please mark it as a draft or add [WIP] to the title and keep it in the `CONTRIBUTOR` column until it's ready to be reviewed/tested. - If PR work is not finished yet, please mark it as a draft or add [WIP] to the title and keep it in the `CONTRIBUTOR` column until it's ready to be reviewed/tested.
- Alternatively, you can open a `draft`.
### When is a PR considered ready to move forward?
### When is a PR considered to be Ready for testing by QA team? 

 Ready for testing, a PR should meet the following criteria:
Ready for testing PR should meet the following criteria:
1. Reviewed and has at least 1 approval 1. Reviewed and has at least 1 approval.
2. Rebased to `develop` branch (both `status-mobile` and `status-go` if needed, depending on what part has changes) 2. Rebased to the `develop` branch (both `status-mobile` and `status-go` if needed, depending on what part has changes).
3. All possible conflicts have been resolved - **NOTE:** QAs can use the `Update with rebase` button from GitHub if:
4. Has the label: `request-manual-qa` - The PR has no `status-go` changes.
- The PR has no conflict with the develop branch.
3. All possible conflicts have been resolved.
4. Has the label: `request-manual-qa` or `skip-manual-qa`.
5. PRs **MUST** identify what area is affected and should have a description.
**NOTE:** Make sure that QAs are OK with that
### Adding `skip-manual-qa`
- Please ask another team member before adding the `skip-manual-qa` label (PR/Status community/DMs) so that there's a second opinion.
- The PR MUST have a proper reasoning why manual QA is skipped.
- The PR MUST include the steps of testing that has been done by the developer prior to moving it forward.
**From the perspective of a developer it means that once work on PR is finished:** **From the perspective of a developer it means that once work on PR is finished:**
1. It should be rebased to the latest `develop`. If there are conflicts - they should be resolved if possible. 1. It should be rebased to the latest `develop`. If there are conflicts - they should be resolved if possible.
2. If the PR was in the `Contributor` column - it should be moved to `Review` column. 2. If the PR was in the `Contributor` column - it should be moved to `Review` column.
3. Wait for the review. 3. Wait for the review.
4. Make sure that after review and before requesting manual QA your PR is rebased to current develop. 4. Make sure that after review and before requesting manual QA your PR is rebased to current develop.
5. Once the PR has been approved by reviewer(s) - label `request-manual-qa` should be applied to the PR 5. The PR **MUST** be moved to the E2E column when it is ready for testing (**mandatory for all PRs**).
6. Move PR to the E2E column when it is ready for testing (**mandatory for all PRs**). That will also trigger e2e tests run. QAs are monitoring PRs from E2E column and take it into test. That will also trigger e2e tests run. QAs are monitoring PRs from E2E column and take it into test.
After that - PR will be taken into manual testing by the QA team. 6. After that - PR will be taken into manual testing by the QA team.
### E2E tests and analyzing the results
This step cannot be skipped. So, at least one comment from the `status-im-auto` bot with results is a prerequisite for moving forward.
Information on how to analyze tests can be found [here](https://github.com/status-im/status-mobile/blob/develop/doc/how-to-launch-e2e.md).
Tests might be flaky, as they depend on infrastructure - SauceLabs and Waku.
If there are `Failed tests` and you are not sure about the reason, you can always ping the mobile QAs for help (preferably in PRs by `@status-im/mobile-qa`).
**For now, it is the only reliable source of regression in the app because, due to time constraints, it is not possible to test thoroughly each PR.
Please, respect this rule.**
## Testing PR ## Testing PR
### Manual testing ### Manual testing
- If you think PR needs and is ready for manual testing, please add the ```request-manual-qa``` label.
#### Prerequisites for manual testing
1. PR has a description of the change and details the area of the application affected by the change.
2. PR has automation results from the previous E2E test run step.
3. PR includes a demo, test steps, and any possible regression(s)
4. If appropriate, PR details what elements of a feature are out of scope.
**If one of the prerequisites is missing, the PR will be moved to the contributor folder with a corresponding comment.**
- If you think a PR needs and is ready for manual testing, please add the `request-manual-qa` label.
- QA engineer picks up one of PRs with the ```request-manual-qa``` label, drags the item to the ```IN TESTING``` column and assigns it to themselves. - QA engineer picks up one of PRs with the ```request-manual-qa``` label, drags the item to the ```IN TESTING``` column and assigns it to themselves.
- During testing, QA will add comments describing the issues found, and also review automation tests results. - During testing, QA will add comments describing the issues found, and also review automation tests results.
Usually found issues are numbered as "Issue 1, Issue 2", etc. Usually found issues are numbered as "Issue 1, Issue 2", etc.
When the first round of testing is completed and all issues for this stage are found, the QA can add the ```Tested - Issues``` label and drag the card to the ```CONTRIBUTOR``` column. These two actions are optional. When the first round of testing is completed and all issues for this stage are found, the QA can add the ```Tested - Issues``` label and drag the card to the ```CONTRIBUTOR``` column. These two actions are optional.
**IMPORTANT NOTE:** when the issues are fixed, developer **MUST** notify the QA that it is ready to be re-tested again by mention them in the PR.
- When manual testing of the PR is fully completed and all the issues are fixed, the QA adds the ```Tested - OK``` label and drags the card to the ```Design review``` column (the cases when design review is necessary are described below). - When manual testing of the PR is fully completed and all the issues are fixed, the QA adds the ```Tested - OK``` label and drags the card to the ```Design review``` column (the cases when design review is necessary are described below).
- If design review is not required, the QA drags the PR to the ```MERGE``` column. After that the developer merges PR into develop. - If design review is not required, the QA drags the PR to the ```MERGE``` column. After that the developer merges PR into develop.
- If design review has been done, the designer (```@Francesca-G```) drags the PR to the ```MERGE``` column. - If design review has been done, the designer (```@Francesca-G```) drags the PR to the ```MERGE``` column.
@ -68,7 +103,7 @@ There are three possible scenarios when the design review is completed:
--- ---
#### Why my PR is in `Contributor` column? #### Why my PR is in `Contributor` column?
PR can be moved to this column by the ```status-github-bot``` or by QA engineer with label `Tested-issues`. PR can be moved to this column by the ```status-github-bot``` or by QA engineer with label `Tested-issues` or if one of the requirements for manual QA was not met.
In the first case most often this happens due to conflicting files in PR. In the first case most often this happens due to conflicting files in PR.
In the second case - after fixing of all found issues, the developer should ping the QA in the PR comments for retesting. In the second case - after fixing of all found issues, the developer should ping the QA in the PR comments for retesting.
@ -86,7 +121,6 @@ In the second case - after fixing of all found issues, the developer should ping
6. In case of manual testing - the label ```Tested - OK``` from QA 6. In case of manual testing - the label ```Tested - OK``` from QA
7. In case of design review - the approval from the designer 7. In case of design review - the approval from the designer
You can merge your PR into develop - some useful clues you can find [here](https://notes.status.im/setup-e2e#3-Merging-PR) You can merge your PR into develop - some useful clues you can find [here](https://notes.status.im/setup-e2e#3-Merging-PR)
HAPPY DEVELOPMENT! :tada: HAPPY DEVELOPMENT! :tada: