Upgrade re-frame to latest, from v1.3.0 (released on
2022-08-27) to latest v1.4.3 (released on 2024-01-25).
Important changes:
- [Added] re-frame.alpha namespace, for testing proposed features (see flows
(https://github.com/day8/re-frame/discussions/795) and polymorphic
subscriptions https://github.com/day8/re-frame/issues/680#issuecomment-1676487563).
- [Added] dispatch-sync now emits a :sync trace to indicate when it has
finished.
- Re-frame upgraded its dependency on Reagent to latest v1.2.0.
- There are two breaking changes in v1.4.0, but they don't affect us because we
don't use interceptors path and unwrap.
Reverts the preloading mechanism from PR
https://github.com/status-im/status-mobile/pull/19927, which introduced a
problem I couldn't find any satisfactory solution. The namespaces user and
dev.user were being preloaded by shadow-cljs, but the problem is that preloading
will fail if those namespaces don't exist in your file system.
I couldn't find any solution to conditionally preload namespaces via shadow-cljs
and ClojureScript unfortunately doesn't support conditional requires like
Clojure.
At least we got to keep the code to not lint them and the whole idea to start to
commit dev-only code inside src/dev/.
Preload the user namespace (src/user.cljs and src/dev/user.cljs) for the mobile
target and for dev-only purposes. The files are git-ignored.
Just a reminder that you'll be responsible for making sure your user namespace
is correct. If it's broken in any way (e.g. calling non-existent code) the app
will crash at initialization (dev-only environment obviously).
Why? When the app initializes, it loads namespaces that were required at least
once. If you create a user namespace, it won't be automatically required for
you. And if you, like some Clojure devs, like to use the user namespace as
your safe heaven for experimentation and dev-only utilities, you'll need to
remember to evaluate the namespace at least once.
This is tedious and many times I forgot to do so and the app crashed because the
compiler didn't know where the symbols were coming from.
This PR upgrades clj-kondo (our Clojure linter) from v2023-09-07 to
v2024-03-13, so ~6 months of development updates.
You can check out the changes starting at
https://github.com/clj-kondo/clj-kondo/blob/master/CHANGELOG.md#20231020.
Nothing terribly useful to us this time, but as usual, clj-kondo can catch
more problems and more reliably than before.
This commit adds support for the Sepolia test network and Rarible collectible/collection provider.
---------
Signed-off-by: Mohamed Javid <19339952+smohamedjavid@users.noreply.github.com>
- make test still exists, so if you have been using it, as well as make
test-watch, they should all work exactly the same.
- [Changed] As part of the check stage, Jenkins will run Lint and Unit Tests
in parallel. Integration Tests run later because running them at the same
time as Unit Tests caused errors.
- [Added] "make unit-test" and "make unit-test-watch" run unit tests only.
Watching all unit tests is faster now because we ignore integration tests and
we only compile shadow-cljs :mock target once. We are at approximately 10-15s
to re-run all unit tests after saving a watched file, depending on your
hardware. If you change mocks.js_dependencies.cljs, you must re-run the make
target.
- [Added] "make integration-test" and "make integration-test-watch" run
integration tests only. These are much slower than the unit tests.
Fixes https://github.com/status-im/status-mobile/issues/18112
Fix all component tests after the latest RN upgrade.
Fixes https://github.com/status-im/status-mobile/issues/18157
Closes https://github.com/status-im/status-mobile/pull/18235
Dependency changes
- Upgraded Jest: from 26.6.3 to latest 29.7.0.
- Upgraded @testing-library/jest-native: from 5.3.0 to latest 5.4.3
- Upgraded @testing-library/react-native: from 11.5.4 to 12.4.2
- Removed explicit dependency on jest-circus, this is now the default test
runner.
- Removed explicit dependency on jest-environment-node. This is handled by the
package manager.
- Added jest-silent-reporter at version 0.5.0.
### Why component tests were failing?
Many tests were failing because we were using RN Testing Library (RNTL) in an
unreliable fashion. With the recent library upgrades, the unreliability was
excerbated. Other times, the tests were incorrectly arranging data.
### with-redefs does not work with async code
Generally speaking, with-redefs should not be used with async code, assume the
worst. The scope of the macro will cease to exist by the time the async code
runs. In many tests we were using with-redefs, then calling render, but for some
components that use use-effect, JS timers, animations, etc it's unreliable and
were the reason for failures.
It's easy to reproduce too:
```clojure
(defn foo []
:foo)
(foo)
;; => :foo
(with-redefs [foo (constantly :bar)]
(foo))
;; => :bar
(js/setTimeout
(fn []
(tap> [:calling-foo (foo)]))
100)
;; Taps [:calling-foo :foo]
;; As you would expect, when running without with-redefs, it prints :foo.
;; So far so good, but whatch what happens with async code:
(with-redefs [foo (constantly :bar)]
(js/setTimeout
(fn []
(tap> [:calling-foo (foo)]))
100))
;; Taps [:calling-foo :foo]
;; ====> PROBLEM: Taps :foo, not :bar as one might expect
```
### Not waiting on wait-for
When test-helpers.component/wait-for is used, subsequent assertions/etc should
be done after the promise returned by wait-for is resolved. But remember to not
perform side-effects inside the wait-for callback (check out the docs
https://callstack.github.io/react-native-testing-library/docs/api#waitfor).
Most, if not all of our usages of wait-for were not waiting.
#### Improvement 1 - Silence Jest on demand
If you need to re-run component tests frequently, you may want to reduce the
output verbosity. By passing JEST_USE_SILENT_REPORTER=true to make
component-test or make component-test-watch you will see a lot less noise and be
able to focus on what really matters to you.
#### Improvement 2 - Selectively focus/disable tests
Because of our need to first compile CLJS to JS before running tests via Jest,
we couldn't easily skip or focus on specific tests. From this commit onwards, we
should never again have to change the list of requires in files core_spec.cljs.
Commenting out required namespaces gives a bad DX because it causes constant
rebasing issues.
#### Improvement 3 - Translations now work as in prod code (but only English)
Translations performed by *-by-translation-text can be done now without any
workaround under the hood. The query functions are now linted just like
i18n/label, which means static translation keywords must be qualified with :t/,
which is good for consistency.
Problem: failed equality checks as in "(is (= expected actual))" will give a
single, long line of output that for anything but the simplest data structures
is unreadable by humans, and the output doesn't give a useful diff.
Solution: use library https://github.com/nubank/matcher-combinators and its test
directive "match?" which will pinpoint where two data structures differ. Then,
instead of "(is (= ...", use "(is (match? expected actual)". It works
beautifully.
The library offers other nice matchers, but the majority of the time match? is
sufficient.
Can we use another test runner like Kaocha? kaocha-cljs2
(https://github.com/lambdaisland/kaocha-cljs2) would be able to print better
test errors out of the box, among other features, but I have no clue if it would
work well or at all in our stack (in theory yes, but it's a larger piece of
work).
This commit is the foundational step to start using malli
(https://github.com/metosin/malli) in this project.
Take in consideration we will only be able to realize malli's full power in
future iterations.
For those without context: the mobile team watched a presentation about malli
and went through a light RFC to put everyone on the same page, among other
discussions here and there in PRs.
To keep things relatively short:
1. Unit, integration and component tests will short-circuit (fail) when
inputs/outputs don't conform to their respective function schemas (CI should
fail too).
2. Failed schema checks will not block the app from initializing, nor throw an
exception that would trigger the LogBox. Exceptions are only thrown in the
scope of automated tests.
3. There's zero performance impact in production code because we only
instrument. Instrumentation is removed from the compiled code due to the
usage of "^boolean js.goog/DEBUG".
4. We shouldn't expect any meaningful slowdown during development.
**What are we instrumenting in this PR?**
Per our team's agreement, we're only instrumenting the bare minimum to showcase 2 examples.
- Instrument a utility function utils.money/format-amount using the macro
approach.
- Instrument a quo component quo.components.counter.step.view/view using the
functional approach.
Both approaches are useful, the functional approach is powerful and allow us to
instrument anonymous functions, like the ones we pass to subscriptions or event
handlers, or the higher-order function quo.theme/with-theme. The macro approach
is perfect for functions already defined with defn.
**I evaluated the schema or function in the REPL but nothing changes**
- If you evaluate the source function, you need to evaluate schema/=> or
schema/instrument as well.
- Remember to *var quote* when using schema/instrument.
- You must call "(status-im2.setup.schema/setup!)" after any var is
re-instrumented. It's advisable to add a keybinding in your editor to send
this expression automatically to the CLJS REPL, or add the call at the end of
the namespace you are working on (similar to how some devs add "(run-tests)"
at the end of test namespaces).
**Where should schemas be defined?**
For the moment, we should focus on instrumenting quo components, so define each
function schema in the same namespace as the component's public "view" var.
To be specific:
- A schema used only to instrument a single function and not used elsewhere,
like a quo component schema, wouldn't benefit from being defined in a separate
namespace because that would force the developer to constantly open two files
instead of one to check function signatures.
- A common schema reused across the repo, like ":schema.common/theme" should be
registered in the global registry "schema.registry" so that consumers can just
refer to it by keyword, as if it was a built-in malli schema.
- A common schema describing status-go entities like message, notification,
community, etc can be stored either in the respective
"src/status_im2/contexts/*" or registered globally, or even somewhere else.
This is yet to be defined, but since I chose not to include schemas for them,
we can postpone this guideline.
Upgrades and cleans up all production Clojure dependencies and 1 dev-only
dependency (com.taoensso/tufte).
- Remove warning "WARNING: update-keys already refers to:
#'clojure.core/update-keys in namespace: io.aviso.exception"
- Remove hickory and mvxcvi/alphabase dependencies they are not used.
- Upgrade com.taoensso/tufte from 2.1.0 to 2.6.3
- Upgrade transit-cljs from 0.8.248 to 0.8.280
- Upgrade cljs-bean from 1.3.0 to 1.9.0
- Remove workaround for com.taoensso/timbre in shadow-cljs.edn
- Upgrade com.taoensso/timbre from 4.10 (Status fork) to 6.3.1
This commit adds a custom linter to verify i18n/label is called with a qualified
keyword, like :t/foo. More sophisticated linters are possible too.
We also set the stage for other developers to consider more lint automation
instead of manually reviewing conventions in PRs.
If you want to understand how to write custom linters, check out
https://github.com/clj-kondo/clj-kondo/blob/master/doc/hooks.md. You can fire
the Clojure JVM REPL in status-mobile and play with the clj-kondo hook too, it
works beautifully.
Why do we care? By making sure all translation keywords are qualified with "t",
it is trivial to grep or replace them because they're unique in the repo, and
can't be confused with other words if you search by ":t/<something>".
Note: It's a best practice to commit clj-kondo configuration from external
libraries in the .clj-kondo directory. The directory .clj-kondo/babashka is
auto-generated, that's why it was added.
Change the make "lint" target default behavior to NOT show clj-kondo warnings.
In the CI, I kept the same behavior, i.e. show all warnings and errors
simultaneously.
Motivation:
When devs run make lint, most of the time, they don't want to see a long list of
warnings. Their focus is on the errors. Additionally, the majority of devs in
the mobile team see clj-kondo warnings in their editors of choice already.
We (some of us) believe the editor feedback/warnings are sufficiently noisy.
Add the following somewhere in your config files if you want to see
warnings.
export CLJ_LINTER_PRINT_WARNINGS=true
Disable :fn-deprecated warnings from the Closure compiler for the mobile
shadow-cljs target during development. The compiler warnings are no longer
necessary, because we're managing deprecation via clj-kondo.
Without the compiler setting's change, every code reload will generate warnings
for each and every deprecated call in the repo, which can quickly grow to the
hundreds or more, thus making the terminal output horrendous to understand.
Also set clj-kondo's "fail-level" to "error" (the default is "warning").
This commit upgrades Shadow CLJS from 2.11.16 (released on Feb/21) to latest
2.25.0 (Jul/23), so ~1.5 years worth of upgrades. By upgrading shadow we
can finally use the latest major Clojure version 1.11.x.
Why upgrade shadow?
- Shadow CLJS controls the ClojureScript version we can use. In order to use the
latest major Clojure version we must upgrade Shadow CLJS.
- Shadow CLJS releases new versions very frequently, and if you take a look at
its changelog https://github.com/thheller/shadow-cljs/blob/master/CHANGELOG.md, you'll see
it had tons and tons of bug fixes over the years. I hope some of them help
improve the DX for certain contributors who recently reported issues with
it.
- Clojure 1.11 brings new features, bug fixes and even performance improvements
(although I think the performance mostly impacts Clojure on the JVM). See the
changelog https://github.com/clojure/clojure/blob/master/changes.md#changes-to-clojure-in-version-1110
Things that can be beneficial to us, or are interesting nonetheless:
- New :as-alias to be used in require, which is like :as but does not require
the namespace to load. This means namespaced keywords using :as-alias can't
cause circular dependency errors. This feature would very useful if we used
namespaced keywords, but we don't, so...
https://github.com/clojure/clojure/blob/master/changes.md#22-as-alias-in-require
- New macros run-test and run-test-var to run single test with fixtures and
report.
- New iteration function, useful for processing paginated data.
https://www.abhinavomprakash.com/posts/clojure-iteration/
- New update-keys function: applies a function to every key in a map.
- New update-vals function: applies a function to every value in a map.
Examples for update-vals and update-keys. They should perform better than the
common reduce-kv approach since they use a transient data structure.
(let [m {:a 1 :b 2}]
(update-vals m inc)) ; => {:a 2, :b 3}
(let [m {:a 1 :b 2}]
(update-keys m name)) ; => {"a" 1, "b" 2}
Why change namespaces within __tests__ directories?
Any namespace with the word --tests-- throws an error, like the one below. I
didn't bother investigating why, so I changed the guidelines to reflect the new
convention. It's probably related to the double dashes in the name.
Namespace quo2.components.dividers.--tests--.divider-label-component-spec has a
segment starting with an invalid JavaScript identifier at line 1
This commit upgrades re-frame to v1.3.0 (latest stable release), released ~9
months ago, in 2022-08-27. This is a solid upgrade, with no breaking changes as
far as I tested status-mobile. It's a great testament of re-frame's stability
and commitment to backwards compatibility, as are many Clojure libs.
Fixes https://github.com/status-im/status-mobile/issues/15963
The big, and truly relevant addition is the introduction of the :fx built-in
effect that was added ~3 years ago in Aug/2020 in v1.1.0.
Relevant changelog:
- Global interceptors are now supported (added in v1.0.0).
- reg-event-fx will just warn (not generate an error) if the effect map returned
contains an unknown effect key.
- re-frame will now warn us when we are calling subscribe outside of a reactive
context.
- "re-frame now guarantees that a :db effect, if present, will be actioned
before any other sibling effects. re-frame continues to provide NO guarantees
about the order in which other effects will be actioned."
(https://day8.github.io/re-frame/releases/2020/#110-2020-08-24)
- There's syntactic sugar for trivial reg-sub declarations (added in v1.3.0).
See the documentation for reg-sub for more details
https://day8.github.io/re-frame/api-re-frame.core/#reg-sub
- "The built-in effect :dispatch-later can now take a single map value.
Supplying a sequence of maps is now deprecated in favor of using multiple
:dispatch-later effects within the new :fx effect."
https://day8.github.io/re-frame/releases/2020/#111-2020-08-26
For some unknown to me reason we are using a different Yarn call to
Shadow-cljs to generate the JSBundle for iOS builds, while the one
created by the Android derivation shoudl be exactly the same.
I'm changing the target to just be `make jsbundle` while keeping aliases
referencing old naming, and moving things around in `nix` folder to
reflect the fact that the derivation is no longer Android-specific.
Also, crucially, I've changed the `import` in `index.js` to use the
`./result/index.js` path, since that's what Nix creates. I'm not sure if
this clashes with any developer workflow that takes place locally, so
I'd appreciate some testing from developers.
Depends on: https://github.com/status-im/status-jenkins-lib/pull/67
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This commit makes the test-helpers.component namespace loadable in the REPL,
plus other changes that allow for a reasonably enjoyable RDD (REPL-Driven
Development) workflow.
Why? I want to be able to get instant feedback when I render a component with
the RN Testing Library (RNTL), and only once I'm satisfied with my findings is
when I proceed to write/update the tests. This nearly instant feedback loop is
only feasible using the ClojureScript REPL, and I'd rather not endure long
recompilation cycles.
Note that by REPL I mean connecting to the CLJS REPL of the Shadow-CLJS :mobile
target.
Essentially, this is what this commit does:
- [x] Allow the test-helpers.component namespace to be evaluated in the REPL.
This is now possible because I changed all functions that assumed js/jest
existed with a guard clause using the CLJS macro exists?. Without the
guard clauses, evaluating the namespace explodes due to stuff like
js/jest.useFakeTimers that fail in compile time (it's a syntax sugar
macro).
- [x] Change the family of functions to get the translation by text to either
translate using i18n/label or translate with the dummy prefix tx:,
depending if the code is running inside the Jest runtime or not.
- [x] Wrap remaining RNTL query functions, except for the find-* ones, since
they don't work at all outside the Jest runtime.
- [x] All wrapped functions support the original arguments supported by RNTL.
Arguments are always converted with clj->js.
- [x] All wrapped functions can optionally take a node (ReactTestInstance) as
their first argument, otherwise the global screen object will be used.
This is very important! See the explanation on section Doesn't RNTL
recommend using the screen object?
- [x] Update Shadow-CLJS preloads, so that (in development) you can fire off the
REPL and always be ready to call component test helpers. This is critical!
What else would be possible? Just an idea, but now that we can easily render
components using the same machinery provided by RNTL in the tests, we can
roughly implement Storybook's Play function
https://storybook.js.org/docs/react/writing-stories/play-function
Lesson learned: In the REPL, you may need to call
(re-frame.core/clear-subscription-cache!), otherwise you will experience
subscriptions returning the same value if their arguments are the same. For
example, I faced this while playing with the namespace
status-im2.contexts.communities.menus.community-options.component-spec. There
are better ways to solve this particular problem in the context of tests if we
use the tooling provided by day8.re-frame.test.
Doesn't RNTL recommend using the screen object? Indeed, it is recommended to use
the screen object instead of destructuring the results of RNTL render. It's just
easier and less error prone, but this only works reliably within the Jest
runtime, since it automatically cleans up rendered state after each test. When
using the REPL this is no longer the case, and I faced some errors, like Unable
to find node on an unmounted component, where RNTL would refuse to re-render
components, even if I explicitly unmounted them or called cleanup.
The only reliable solution I found was to store the result of render (a node)
and pass it to every subsequent call. This is not a workaround, it's officially
supported, but it's a tad less convenient. You can also not pass the node
reference and it should work most of the time.
Practical examples
Workflow suggestion: write your local experiments in the same namespace as the
component spec and within the comment macro. This way, you can have the Jest
watcher running and a REPL connected to :mobile, and they won't step on each
other. For the test watcher, I usually change quo2-core-spec or
status-im2.core-spec to only require what I'm interested, otherwise Jest
consumes way too many resources.
```clojure
;; Namespace quo2.components.colors.color-picker.component-spec
(h/test "color picker color changed"
(let [selected (reagent/atom nil)]
(h/render [color-picker/view {:on-change #(reset! selected %)}])
(h/fire-event :press (get (h/get-all-by-label-text :color-picker-item) 0))
(-> (h/expect @selected)
(.toStrictEqual :blue))))
(comment
(def selected (atom nil))
(def c (h/render [color-picker/view {:on-change #(reset! selected %)}]))
(h/fire-event :press (get (h/get-all-by-label-text c :color-picker-item) 0))
;; Options are passed down converted to JS types.
(h/debug c {:message "Rendering header"})
@selected ; => :blue
)
```
```clojure
;; Namespace quo2.components.tags.--tests--.status-tags-component-spec
(h/test "renders status tag with pending type"
(render-status-tag {:status {:type :pending}
:label "Pending"
:size :small})
(-> (h/expect (h/get-all-by-label-text :status-tag-pending))
(.toBeTruthy))
(-> (h/expect (h/get-by-text "Pending"))
(.toBeTruthy)))
(comment
(def c (render-status-tag {:status {:type :pending}
:label "Pending"
:size :small}))
(h/get-all-by-label-text c :status-tag-pending))
```
```clojure
;; Namespace status-im2.contexts.communities.menus.community-options.component-spec
(h/test "joined and muted community"
(setup-subs {:communities/my-pending-request-to-join nil
:communities/community {:joined true
:muted true
:token-gated? true}})
(h/render [options/community-options-bottom-sheet {:id "test"}])
(-> (h/expect (h/get-by-translation-text :unmute-community))
(.toBeTruthy)))
(comment
(setup-subs {:communities/my-pending-request-to-join nil
:communities/community {:joined true
:muted true
:token-gated? true}})
(def c (h/render [options/community-options-bottom-sheet {:id "test"}]))
(some? (h/get-by-translation-text c :invite-people-from-contacts)) ; => true
)
```
* nix: upgrade zprint from 1.2.4 to 1.2.5
To address issue described in:
https://github.com/kkinnear/zprint/issues/273
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* chore: use zprint :multi-lhs-hang
* refactor: re-format clojure using zprint 1.2.5
---------
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Co-authored-by: yqrashawn <namy.19@gmail.com>
On M1 calling `shadow-cljs` fails with:
```
Execution error (UnsatisfiedLinkError) at java.lang.ClassLoader$NativeLibrary/load (ClassLoader.java:-2).
/private/var/folders/__/x311ykg17rqgq2wyl4kn1pdr0001yh/T/jna8753030888504535661.tmp:
dlopen(/private/var/folders/__/x311ykg17rqgq2wyl4kn1pdr0001yh/T/jna8753030888504535661.tmp, 0x0001):
tried: '/private/var/folders/__/x311ykg17rqgq2wyl4kn1pdr0001yh/T/jna8753030888504535661.tmp'
(fat file, but missing compatible architecture (have (unknown,i386,x86_64), need (arm64e)))
```
This is due to an outdeted dependency on JNA 3.2.2, which is pulled in
by `hawk` package which up until release `2.11.16` was a `shadow-clj`
dependency which was removed because it was:
>Only used to be used on macOS since it was slightly faster than the default
>JVM implementation. However in Big Sur it seems to cause issues and break
>completely or just be a lot slower.
https://github.com/thheller/shadow-cljs/commit/f3b89b5a
Dropped the explicit dependency on `org.clojure/core.async` to avoid:
```
WARNING: The org.clojure/core.async dependency in shadow-cljs.edn was ignored.
Default version is used and override is not allowed to ensure compatibility.
```
Resolves: https://github.com/status-im/status-mobile/issues/14196
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Icaro Motta
Brian Sztamfater
Lungu Cristian
Jamie Caprani
Mohamed Javid
Ajay Sivan
flexsurfer
andrey
yqrashawn
Parvesh Monu
Jakub Sokołowski
Omar Basem
Alexander
Roman Volosovskyi