e2e: updating PR pipeline (#19478)
This commit is contained in:
Tetiana Churikova
GitHub
parent
f114908c4d
commit
354cda3be5
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
[Pipeline for QA](https://github.com/status-im/status-mobile/projects/7) is a project board for developers and testers used to track the status of a pull request, get reviews and manual testing, and run autotests.
|
||||
|
||||
The generally accepted recommendations for its use are described below:
|
||||
The generally accepted requirements for its use are described below:
|
||||
|
||||
## Opening a PR
|
||||
- Once a PR is created, it moves to the ```REVIEW``` column where a review will be requested automatically.
|
||||
|
|
@ -13,35 +13,70 @@ The generally accepted recommendations for its use are described below:
|
|||
### What if the work is still in progress?
|
||||
|
||||
- If PR work is not finished yet, please mark it as a draft or add [WIP] to the title and keep it in the `CONTRIBUTOR` column until it's ready to be reviewed/tested.
|
||||
- Alternatively, you can open a `draft`.
|
||||
|
||||
### When is a PR considered ready to move forward?
|
||||
|
||||
### When is a PR considered to be Ready for testing by QA team?
|
||||
Ready for testing PR should meet the following criteria:
|
||||
Ready for testing, a PR should meet the following criteria:
|
||||
|
||||
1. Reviewed and has at least 1 approval
|
||||
2. Rebased to `develop` branch (both `status-mobile` and `status-go` if needed, depending on what part has changes)
|
||||
3. All possible conflicts have been resolved
|
||||
4. Has the label: `request-manual-qa`
|
||||
1. Reviewed and has at least 1 approval.
|
||||
2. Rebased to the `develop` branch (both `status-mobile` and `status-go` if needed, depending on what part has changes).
|
||||
- **NOTE:** QAs can use the `Update with rebase` button from GitHub if:
|
||||
- The PR has no `status-go` changes.
|
||||
- The PR has no conflict with the develop branch.
|
||||
3. All possible conflicts have been resolved.
|
||||
4. Has the label: `request-manual-qa` or `skip-manual-qa`.
|
||||
5. PRs **MUST** identify what area is affected and should have a description.
|
||||
|
||||
**NOTE:** Make sure that QAs are OK with that
|
||||
|
||||
### Adding `skip-manual-qa`
|
||||
|
||||
- Please ask another team member before adding the `skip-manual-qa` label (PR/Status community/DMs) so that there's a second opinion.
|
||||
- The PR MUST have a proper reasoning why manual QA is skipped.
|
||||
- The PR MUST include the steps of testing that has been done by the developer prior to moving it forward.
|
||||
**From the perspective of a developer it means that once work on PR is finished:**
|
||||
|
||||
1. It should be rebased to the latest `develop`. If there are conflicts - they should be resolved if possible.
|
||||
2. If the PR was in the `Contributor` column - it should be moved to `Review` column.
|
||||
3. Wait for the review.
|
||||
4. Make sure that after review and before requesting manual QA your PR is rebased to current develop.
|
||||
5. Once the PR has been approved by reviewer(s) - label `request-manual-qa` should be applied to the PR
|
||||
6. Move PR to the E2E column when it is ready for testing (**mandatory for all PRs**). That will also trigger e2e tests run. QAs are monitoring PRs from E2E column and take it into test.
|
||||
5. The PR **MUST** be moved to the E2E column when it is ready for testing (**mandatory for all PRs**).
|
||||
That will also trigger e2e tests run. QAs are monitoring PRs from E2E column and take it into test.
|
||||
|
||||
After that - PR will be taken into manual testing by the QA team.
|
||||
6. After that - PR will be taken into manual testing by the QA team.
|
||||
|
||||
### E2E tests and analyzing the results
|
||||
|
||||
This step cannot be skipped. So, at least one comment from the `status-im-auto` bot with results is a prerequisite for moving forward.
|
||||
Information on how to analyze tests can be found [here](https://github.com/status-im/status-mobile/blob/develop/doc/how-to-launch-e2e.md).
|
||||
Tests might be flaky, as they depend on infrastructure - SauceLabs and Waku.
|
||||
|
||||
If there are `Failed tests` and you are not sure about the reason, you can always ping the mobile QAs for help (preferably in PRs by `@status-im/mobile-qa`).
|
||||
|
||||
**For now, it is the only reliable source of regression in the app because, due to time constraints, it is not possible to test thoroughly each PR.
|
||||
Please, respect this rule.**
|
||||
|
||||
## Testing PR
|
||||
|
||||
### Manual testing
|
||||
- If you think PR needs and is ready for manual testing, please add the ```request-manual-qa``` label.
|
||||
|
||||
#### Prerequisites for manual testing
|
||||
1. PR has a description of the change and details the area of the application affected by the change.
|
||||
2. PR has automation results from the previous E2E test run step.
|
||||
3. PR includes a demo, test steps, and any possible regression(s)
|
||||
4. If appropriate, PR details what elements of a feature are out of scope.
|
||||
|
||||
**If one of the prerequisites is missing, the PR will be moved to the contributor folder with a corresponding comment.**
|
||||
|
||||
- If you think a PR needs and is ready for manual testing, please add the `request-manual-qa` label.
|
||||
- QA engineer picks up one of PRs with the ```request-manual-qa``` label, drags the item to the ```IN TESTING``` column and assigns it to themselves.
|
||||
- During testing, QA will add comments describing the issues found, and also review automation tests results.
|
||||
Usually found issues are numbered as "Issue 1, Issue 2", etc.
|
||||
When the first round of testing is completed and all issues for this stage are found, the QA can add the ```Tested - Issues``` label and drag the card to the ```CONTRIBUTOR``` column. These two actions are optional.
|
||||
When the first round of testing is completed and all issues for this stage are found, the QA can add the ```Tested - Issues``` label and drag the card to the ```CONTRIBUTOR``` column. These two actions are optional.
|
||||
|
||||
**IMPORTANT NOTE:** when the issues are fixed, developer **MUST** notify the QA that it is ready to be re-tested again by mention them in the PR.
|
||||
|
||||
- When manual testing of the PR is fully completed and all the issues are fixed, the QA adds the ```Tested - OK``` label and drags the card to the ```Design review``` column (the cases when design review is necessary are described below).
|
||||
- If design review is not required, the QA drags the PR to the ```MERGE``` column. After that the developer merges PR into develop.
|
||||
- If design review has been done, the designer (```@Francesca-G```) drags the PR to the ```MERGE``` column.
|
||||
|
|
@ -68,7 +103,7 @@ There are three possible scenarios when the design review is completed:
|
|||
---
|
||||
|
||||
#### Why my PR is in `Contributor` column?
|
||||
PR can be moved to this column by the ```status-github-bot``` or by QA engineer with label `Tested-issues`.
|
||||
PR can be moved to this column by the ```status-github-bot``` or by QA engineer with label `Tested-issues` or if one of the requirements for manual QA was not met.
|
||||
In the first case most often this happens due to conflicting files in PR.
|
||||
In the second case - after fixing of all found issues, the developer should ping the QA in the PR comments for retesting.
|
||||
|
||||
|
|
@ -86,7 +121,6 @@ In the second case - after fixing of all found issues, the developer should ping
|
|||
6. In case of manual testing - the label ```Tested - OK``` from QA
|
||||
7. In case of design review - the approval from the designer
|
||||
|
||||
|
||||
You can merge your PR into develop - some useful clues you can find [here](https://notes.status.im/setup-e2e#3-Merging-PR)
|
||||
|
||||
HAPPY DEVELOPMENT! :tada:
|
||||
|
|
|
|||
Loading…
Reference in New Issue