updates docs about testing

This commit is contained in:
Marcin Czenko 2025-10-27 13:57:38 +01:00
parent 7424963666
commit 5bee36df64
No known key found for this signature in database
GPG Key ID: A0449219BDBA98AE
3 changed files with 324 additions and 0 deletions

View File

@ -1,3 +1,8 @@
---
related-to:
- "[[Running tests with gotestsum]]"
- "[[go testify assertions]]"
---
Let's start with what we find in the [status-go build instructions](https://github.com/status-im/status-go/blob/develop/_docs/how-to-build.md) as lots of things just work.
We have two options: (1) use the [nix](https://nixos.org/) develop shell or (2) just use standard shell you have in your shell. Theoretically, nix should give you better isolation and repeatability, yet, it is quite opinionated, has learning curve, and adds quite a bit complexity. For now thus, I decided to a more conservative shell environment, where I feel more comfortable.
@ -86,6 +91,54 @@ After that make sure that `$HOME/.local/share/go/bin` is in your path, and you s
protoc --version
libprotoc 32.1
```
#### go-zerokit-rln-x86_64 vendoring problem
If you try to run the tests for the first time, you may face the following error:
```bash
gotestsum --packages="./protocol/communities" -f standard-verbose --rerun-fails -- -v -run "TestCodexClientTestSuite" -count 1
FAIL github.com/status-im/status-go/protocol/communities [setup failed]
=== Failed
=== FAIL: protocol/communities (0.00s)
FAIL github.com/status-im/status-go/protocol/communities [setup failed]
=== Errors
vendor/github.com/waku-org/go-zerokit-rln/rln/link/x86_64.go:8:8: cannot find module providing package github.com/waku-org/go-zerokit-rln-x86_64/rln: import lookup disabled by -mod=vendor
(Go version in go.mod is at least 1.14 and vendor directory exists.)
DONE 0 tests, 1 failure, 1 error in 0.000s
ERROR rerun aborted because previous run had errors
```
The problem can be outlined as follows:
1. **The package *IS* declared in dependencies**:
- `go.mod` has: `github.com/waku-org/go-zerokit-rln-x86_64 v0.0.0-20230916171518-2a77c3734dd1 // indirect`
- `modules.txt` lists it as *vendored*
2. **BUT it's excluded from git**:
- `.gitignore` has: `vendor/github.com/waku-org/go-zerokit-rln-x86_64/`
- as the result. the actual vendor directory is **missing** from your file system
3. **Why it's excluded**: these seems to platform-specific native libraries (RLN - Rate Limiting Nullifier) with binary/compiled components that are large and platform-specific. The project excludes them from version control.
4. **The build tag restriction**: The file x86_64.go has build tags:
`//go:build (linux || windows) && amd64 && !android`
So it only compiles on x86_64 Linux/Windows, but when it tries to compile, it needs the vendored package.
#### The Solution
We need to vendor the missing dependencies:
```bash
# This will download the missing vendored dependencies
go mod vendor
```
This will populate the `vendor/github.com/waku-org/go-zerokit-rln-x86_64/` directory with the necessary files, even though they're gitignored.
### Building backend and the libs
Just to check if everything is setup correctly, let's build `status-backend` (which is a wrapper over status-go that provides web API - handy for testing), and then status-go static and shared libraries:

View File

@ -0,0 +1,158 @@
A summary of running different test scenarios with `gotestsum` based on an example
## gotestsum Command Patterns
### 1. **Only Selected Test**
```bash
# Run a specific test function
gotestsum --packages="./communities" -f testname --rerun-fails -- -run "TestCodexArchiveDownloader_BasicSingleArchive$" -count 1
# Run specific testify test
gotestsum --packages="./communities" -f testname --rerun-fails -- -run "TestCodexArchiveDownloader_BasicSingleArchive_Testify$" -count 1
# Run multiple archives testify test
gotestsum --packages="./communities" -f testname --rerun-fails -- -run "TestCodexArchiveDownloader_MultipleArchives_Testify$" -count 1
```
### 2. **Only Selected Test Suite**
```bash
# Run the entire testify suite (all methods in the suite)
gotestsum --packages="./communities" -f testname --rerun-fails -- -run "TestCodexArchiveDownloaderSuite" -count 1
# Run a specific test method within the suite
gotestsum --packages="./communities" -f testname --rerun-fails -- -run "TestCodexArchiveDownloaderSuite/TestBasicSingleArchive" -count 1
```
### 3. **All Tests for Given Package**
```bash
# Run all tests in communities package
gotestsum --packages="./communities" -f testname --rerun-fails -- -count 1
# Alternative syntax (same result)
gotestsum --packages="./communities" -f testname --rerun-fails
# Run all tests with verbose output
gotestsum --packages="./communities" -f testname --rerun-fails -- -v -count 1
```
### 4. **Integration Tests**
```bash
# Run only integration tests (using build tags)
gotestsum --packages="./communities" -f testname --rerun-fails -- -tags=integration -run "Integration" -count 1
# Run integration tests with timeout (since they may take longer)
gotestsum --packages="./communities" -f testname --rerun-fails -- -tags=integration -run "Integration" -timeout=60s -count 1
# Run integration tests with specific environment variables
CODEX_HOST=localhost CODEX_API_PORT=8080 gotestsum --packages="./communities" -f testname --rerun-fails -- -tags=integration -run "Integration" -count 1
```
## Advanced gotestsum Patterns
### **Filter by Pattern (Multiple Tests)**
```bash
# Run all archive downloader tests (both standard and testify)
gotestsum --packages="./communities" -f testname --rerun-fails -- -run "ArchiveDownloader" -count 1
# Run only testify tests
gotestsum --packages="./communities" -f testname --rerun-fails -- -run "Testify" -count 1
# Run all CodexClient tests
gotestsum --packages="./communities" -f testname --rerun-fails -- -run "CodexClient" -count 1
```
### **Output Formats**
```bash
# Different output formats
gotestsum --packages="./communities" -f dots # Dots progress
gotestsum --packages="./communities" -f pkgname # Show package names
gotestsum --packages="./communities" -f testname # Show test names (recommended)
gotestsum --packages="./communities" -f standard-quiet # Minimal output
```
### **Multiple Packages**
```bash
# Run tests across multiple packages
gotestsum --packages="./communities,./cmd/upload,./cmd/download" -f testname --rerun-fails -- -count 1
# Run all packages recursively
gotestsum --packages="./..." -f testname --rerun-fails -- -count 1
```
### **Race Detection and Coverage**
```bash
# Run with race detection
gotestsum --packages="./communities" -f testname --rerun-fails -- -race -count 1
# Run with coverage
gotestsum --packages="./communities" -f testname --rerun-fails -- -cover -count 1
# Run with both race detection and coverage
gotestsum --packages="./communities" -f testname --rerun-fails -- -race -cover -count 1
```
## Key gotestsum Advantages
1. **Better Output Formatting**: Clean, colored output with test names
2. **Automatic Retry**: `--rerun-fails` reruns failed tests automatically
3. **JUnit XML Output**: `--junitfile=results.xml` for CI/CD integration
4. **JSON Output**: `--jsonfile=results.json` for parsing
5. **Watch Mode**: `--watch` to rerun tests on file changes
6. **Parallel Execution**: Better handling of parallel test output
## Complete Examples for Your Project
```bash
# Quick test of archive downloader functionality
gotestsum --packages="./communities" -f testname --rerun-fails -- -run "ArchiveDownloader" -count 1
# Full test suite with coverage
gotestsum --packages="./communities" -f testname --rerun-fails -- -cover -count 1
# Integration tests (when you have a Codex node running)
gotestsum --packages="./communities" -f testname --rerun-fails -- -tags=integration -timeout=60s -count 1
# Development workflow with watch mode
gotestsum --packages="./communities" -f testname --watch -- -count 1
```
The key difference from `go test` is that `gotestsum` provides much better visual feedback, automatic retry capabilities, and better CI/CD integration options while using the same underlying Go test infrastructure.
## Logs
In your tests you can include custom logs.
`go test -v` prints them without an issue, but for `gotestsum` to do the same you have to use `standard-verbose` format option:
```bash
gotestsum --packages="./communities" -f standard-verbose --rerun-fails -- -run "TestCodexArchiveDownloaderSuite" -v -count 1
=== RUN TestCodexArchiveDownloaderSuite
=== RUN TestCodexArchiveDownloaderSuite/TestBasicSingleArchive
codex_archive_downloader_testify_test.go:112: ✅ Basic single archive download test passed (testify version)
codex_archive_downloader_testify_test.go:113: - All mock expectations satisfied
codex_archive_downloader_testify_test.go:114: - Callback invoked: true
=== RUN TestCodexArchiveDownloaderSuite/TestMultipleArchives
codex_archive_downloader_testify_test.go:190: ✅ Multiple archives test passed (suite version)
codex_archive_downloader_testify_test.go:191: - Completed 3 out of 3 archives
--- PASS: TestCodexArchiveDownloaderSuite (0.20s)
--- PASS: TestCodexArchiveDownloaderSuite/TestBasicSingleArchive (0.10s)
--- PASS: TestCodexArchiveDownloaderSuite/TestMultipleArchives (0.10s)
PASS
ok go-codex-client/communities 0.205s
DONE 3 tests in 0.205s
```
Compare this output with `-f testname`:
```bash
gotestsum --packages="./communities" -f testname --rerun-fails -- -run "TestCodexArchiveDownloaderSuite" -count 1
PASS communities.TestCodexArchiveDownloaderSuite/TestBasicSingleArchive (0.10s)
PASS communities.TestCodexArchiveDownloaderSuite/TestMultipleArchives (0.10s)
PASS communities.TestCodexArchiveDownloaderSuite (0.20s)
PASS communities
DONE 3 tests in 0.205s
```
> Notice that the test suite itself is also counted as a test - this is one we see `DONE 3 tests` instead of `DONE 2 tests`.

View File

@ -0,0 +1,113 @@
Some summary about testify assertions.
## `assert.Equal()` Function Signature
```go
func Equal(t TestingT, expected, actual interface{}, msgAndArgs ...interface{}) bool
```
### Parameters Breakdown:
1. **`t TestingT`** - The testing interface (first parameter)
2. **`expected interface{}`** - What you expect the value to be
3. **`actual interface{}`** - The actual value you're testing
4. **`msgAndArgs ...interface{}`** - Optional custom message and formatting arguments
## What is `suite.T()`?
When you're using testify's **test suite pattern**, `suite.T()` returns the underlying `*testing.T` instance associated with the current test method.
```go
type CodexArchiveDownloaderTestifySuite struct {
suite.Suite // Embeds testify's Suite type
// ... other fields
}
func (suite *CodexArchiveDownloaderTestifySuite) TestBasicSingleArchive() {
// suite.T() returns the *testing.T for this specific test method
assert.Equal(suite.T(), 1, downloader.GetTotalArchivesCount())
}
```
## Comparison: Suite vs Function-Based
### In Test Suite (what you're seeing):
```go
assert.Equal(suite.T(), expected, actual, "optional message")
```
### In Regular Function-Based Test:
```go
func TestSomething(t *testing.T) {
assert.Equal(t, expected, actual, "optional message")
}
```
## Why `suite.T()` is Needed
The testify suite embeds a `*testing.T`, but assertions need direct access to it for:
- Reporting failures
- Marking tests as failed
- Logging output
- Integration with Go's test runner
## Other Common Assert Functions
```go
// Basic equality
assert.Equal(t, expected, actual)
assert.NotEqual(t, expected, actual)
// Boolean checks
assert.True(t, condition)
assert.False(t, condition)
// Nil checks
assert.Nil(t, value)
assert.NotNil(t, value)
// Collection checks
assert.Len(t, collection, expectedLength)
assert.Contains(t, collection, element)
assert.Empty(t, collection)
// Error checks
assert.NoError(t, err)
assert.Error(t, err)
// Type checks
assert.IsType(t, expectedType, actual)
// All with optional custom messages
assert.Equal(t, 42, result, "The calculation should return 42")
assert.True(t, isValid, "Validation should pass for input: %s", input)
```
## `require` vs `assert`
Both take the same parameters, but behave differently on failure:
```go
// assert continues test execution on failure
assert.Equal(t, 1, count)
assert.True(t, isReady) // This will still run even if above fails
// require stops test execution immediately on failure
require.Equal(t, 1, count)
require.True(t, isReady) // This won't run if above fails
```
## In Your Code Context
```go
// Line 90 you're looking at:
assert.Equal(suite.T(), 1, downloader.GetTotalArchivesCount(), "Total archives count should be 1")
```
This means:
- **`suite.T()`** - The testing context from the suite
- **`1`** - Expected value
- **`downloader.GetTotalArchivesCount()`** - Actual value being tested
- **`"Total archives count should be 1"`** - Custom error message if assertion fails
The assertion will fail if `GetTotalArchivesCount()` returns anything other than `1`, and it will display your custom message along with the expected vs actual values.