This commit brings numerous improvements to integration tests. The next step
will be to apply the same improvements to contract tests.
Fixes https://github.com/status-im/status-mobile/issues/18676
Improvements:
- Setting up the application and logged account per test is now done with an
async test fixture, which is a very idiomatic way to solve this problem. No
need anymore to write macros to wrap day8.re-frame.test/wait-for. The macros
in test-helpers.integration will be removed once we apply the same
improvements to contract tests.
- Integration test timeouts can be controlled per test, with a configurable,
global default (60s).
- Now the integration test suite will fail-fast by default, i.e. a test failure
short-circuits the entire suite immediately. This option can be overridden on
a test-by-test basis. This improvement is very useful when investigating
failures because the error will be shown on the spot, with no need to search
backwards across lots of logs.
- Noisy messages from re-frame can be silenced with a test fixture. We can
silence even more in the future if we remove the hardcoded printf call from
C++ on every signal and control it with Clojure. We can disable most logs as
well with the more direct (status-im.common.log/setup "ERROR") at the top of
tests.integration-test.core. We can make verbosity even more convenient to
control, but I think this should be designed outside this PR.
- Removed dependency on lib day8.re-frame/test for integration tests (see
detailed explanation below).
- Each call to (our) wait-for can customize the timeout to process re-frame
event IDs passed to it.
- Syntax is now flat, instead of being nested on every call to wait-for. You
can now compose other async operations anywhere in a test.
Notes:
- promesa.core/do is essential in the integration test suite, as it makes sync &
async operations play nice with each other without the developer having to
promisify anything manually.
- There are lots of logs related to async storage ("unexpected token u in JSON at
position..."). This isn't fixed yet.
Are we not going to use day8.re-frame.test?
We don't need this library in integration tests and we won't need it in contract
tests. Whether it will be useful after we remove it from integration and
contract tests is yet to be seen (probably not).
A few reasons:
- The async model of promises is well understood and battle tested. The one
devised in the lib is poorly understood and rigid.
- There's basically no way to correctly integrate other async operations in the
test, unless they can be fully controlled via re-frame events. For instance,
how would you control timeouts? How would you retry in a test? How would
forcefully delay an operation by a few seconds? These things are useful (to me
at least) when developing integration/contract tests.
- Every call to day8.re-frame.test/wait-for forces you to nest code one more
level. Code readability suffers from that choice.
- Have you ever looked up the implementation of wait-for? It's quite convoluted.
One could say the source code is not that important, but many times I had to
look it up because I couldn't understand the async model they built with their
macro approach. The de facto primitive in JS for asynchronicity is promises,
and we fully leverage it in this PR.
- The lib has an interesting macro run-test-sync, but we have no usage for it. I
used it in status-mobile for a while. At one point, all event unit tests for
the Activity Center used it (e.g. commit
08fb0de7b0), but I replaced them with the
simpler pure function style.
5d1e1f8005