status-im
/
status-go
mirror of https://github.com/status-im/status-go.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

Added test policy

This commit is contained in:
Samuel Hawksby-Robinson 2024-03-11 16:31:53 +00:00
parent c85bf76f21
commit 004de053c2
2 changed files with 51 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -13,6 +13,10 @@
- [How to run a Waku node](./_examples/README.md#run-waku-node)
- [How to configure status-go](/config)
## Policies
- [Test policies](./_docs/policies/tests.md)
# License
[Mozilla Public License 2.0](https://github.com/status-im/status-go/blob/develop/LICENSE.md)

47
_docs/policies/tests.md Normal file
View File

@ -0,0 +1,47 @@
# `status-go` Test Policy
- [Creating Tests](#creating-tests)
- [Flaky Tests](#flaky-tests)
## Creating Tests
- All new functionality MUST be introduced with tests that:
- Prove that the functionality performs as described
- Can be falsified
- Are resistant to fuzzing
- All new tests MUST BE validated via 1000 local tests. Ensuring that the test runs and passes consistently every time gives confidence that the test is not flaky.
- add the `-count` flag to the test command eg: `-count 1000`
- OR wrap your test in a for loop
```go
func TestTheThing(t *testing.T) {
for i := 1; i < 1000; i++ {
fmt.Println("Test Run", i)
// your test body goes in here
}
}
```
## Flaky Tests
- All flaky tests / failing tests must be resolved
- Do not introduce flaky tests to the codebase
- Steps to resolving flaky test
- Ensure that the test is definitely flaky and that it wasn’t you that introduced a bug that made the test flaky.
- Is a new test you’ve written flaky?
- Ok, you need to fix that before merge is acceptable.
- Has an old test become flaky?
- Ok, did you touch this test or the functionality it tests? If yes, you need to fix that before merge is acceptable. Sorry.
- If an old test fails and seems flaky either locally or in CI, you must check the `status-go` GitHub repo issues for the test name(s) failing.
- If the test appears in the list of flaky test issues
- If the issue is open
- Add a comment to the issue
- Detail that you have experienced the test being flaky and in what context (local vs CI, link to the PR or branch).
- If the issue is closed
- Reopen the issue OR create a new issue referencing the previous issue
- Either is fine, use your best judgement in this case.
- Detail that you have experienced the test being flaky and in what context (local vs CI, link to the PR or branch).
- I just added this to see if you read the details of this PR `status-go` label for use `E:Flaky Test` may want to be not an epic, but I don't know
- If the test does not appear in the list of flaky test issues
- create a new issue
- The issue title should include the flaky test name
- The issue should use the `E:Flaky Test` label, see here for all flaky tests https://github.com/status-im/status-go/labels/E%3AFlaky%20Test
- Detail that you have experienced the test being flaky and in what context (local vs CI, link to the PR or branch).