Problem: failed equality checks as in "(is (= expected actual))" will give a
single, long line of output that for anything but the simplest data structures
is unreadable by humans, and the output doesn't give a useful diff.
Solution: use library https://github.com/nubank/matcher-combinators and its test
directive "match?" which will pinpoint where two data structures differ. Then,
instead of "(is (= ...", use "(is (match? expected actual)". It works
beautifully.
The library offers other nice matchers, but the majority of the time match? is
sufficient.
Can we use another test runner like Kaocha? kaocha-cljs2
(https://github.com/lambdaisland/kaocha-cljs2) would be able to print better
test errors out of the box, among other features, but I have no clue if it would
work well or at all in our stack (in theory yes, but it's a larger piece of
work).
This commit is the foundational step to start using malli
(https://github.com/metosin/malli) in this project.
Take in consideration we will only be able to realize malli's full power in
future iterations.
For those without context: the mobile team watched a presentation about malli
and went through a light RFC to put everyone on the same page, among other
discussions here and there in PRs.
To keep things relatively short:
1. Unit, integration and component tests will short-circuit (fail) when
inputs/outputs don't conform to their respective function schemas (CI should
fail too).
2. Failed schema checks will not block the app from initializing, nor throw an
exception that would trigger the LogBox. Exceptions are only thrown in the
scope of automated tests.
3. There's zero performance impact in production code because we only
instrument. Instrumentation is removed from the compiled code due to the
usage of "^boolean js.goog/DEBUG".
4. We shouldn't expect any meaningful slowdown during development.
**What are we instrumenting in this PR?**
Per our team's agreement, we're only instrumenting the bare minimum to showcase 2 examples.
- Instrument a utility function utils.money/format-amount using the macro
approach.
- Instrument a quo component quo.components.counter.step.view/view using the
functional approach.
Both approaches are useful, the functional approach is powerful and allow us to
instrument anonymous functions, like the ones we pass to subscriptions or event
handlers, or the higher-order function quo.theme/with-theme. The macro approach
is perfect for functions already defined with defn.
**I evaluated the schema or function in the REPL but nothing changes**
- If you evaluate the source function, you need to evaluate schema/=> or
schema/instrument as well.
- Remember to *var quote* when using schema/instrument.
- You must call "(status-im2.setup.schema/setup!)" after any var is
re-instrumented. It's advisable to add a keybinding in your editor to send
this expression automatically to the CLJS REPL, or add the call at the end of
the namespace you are working on (similar to how some devs add "(run-tests)"
at the end of test namespaces).
**Where should schemas be defined?**
For the moment, we should focus on instrumenting quo components, so define each
function schema in the same namespace as the component's public "view" var.
To be specific:
- A schema used only to instrument a single function and not used elsewhere,
like a quo component schema, wouldn't benefit from being defined in a separate
namespace because that would force the developer to constantly open two files
instead of one to check function signatures.
- A common schema reused across the repo, like ":schema.common/theme" should be
registered in the global registry "schema.registry" so that consumers can just
refer to it by keyword, as if it was a built-in malli schema.
- A common schema describing status-go entities like message, notification,
community, etc can be stored either in the respective
"src/status_im2/contexts/*" or registered globally, or even somewhere else.
This is yet to be defined, but since I chose not to include schemas for them,
we can postpone this guideline.
Upgrades and cleans up all production Clojure dependencies and 1 dev-only
dependency (com.taoensso/tufte).
- Remove warning "WARNING: update-keys already refers to:
#'clojure.core/update-keys in namespace: io.aviso.exception"
- Remove hickory and mvxcvi/alphabase dependencies they are not used.
- Upgrade com.taoensso/tufte from 2.1.0 to 2.6.3
- Upgrade transit-cljs from 0.8.248 to 0.8.280
- Upgrade cljs-bean from 1.3.0 to 1.9.0
- Remove workaround for com.taoensso/timbre in shadow-cljs.edn
- Upgrade com.taoensso/timbre from 4.10 (Status fork) to 6.3.1
Since the `default` shell doesn't have Ruby, the `nix-update-gems`
target would incorrectly use the system Ruby instead of the one from Nix.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Clojure dependencies require only JARs to work. Downloading POMs is both
a waste of time, space, and bandwidth. In addition POMs create edge
cases that we would have to handle, an would rather avoid.
For example, the `guice` package which shows up in the classpath
includes a JAR named `guice-4.2.2-no_aop.jar`. The issue with that is
that there is no corresponding POM in the directory:
https://repo1.maven.org/maven2/com/google/inject/guice/4.2.2/
Either we have to make a special case for such packages, or we can just
skip POMs entirely and avoid the mess.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This commit adds a custom linter to verify i18n/label is called with a qualified
keyword, like :t/foo. More sophisticated linters are possible too.
We also set the stage for other developers to consider more lint automation
instead of manually reviewing conventions in PRs.
If you want to understand how to write custom linters, check out
https://github.com/clj-kondo/clj-kondo/blob/master/doc/hooks.md. You can fire
the Clojure JVM REPL in status-mobile and play with the clj-kondo hook too, it
works beautifully.
Why do we care? By making sure all translation keywords are qualified with "t",
it is trivial to grep or replace them because they're unique in the repo, and
can't be confused with other words if you search by ":t/<something>".
Note: It's a best practice to commit clj-kondo configuration from external
libraries in the .clj-kondo directory. The directory .clj-kondo/babashka is
auto-generated, that's why it was added.
Fixes partially #15595
In order to build less targets, when not needed we introduce this
mapping logic.
If only specific ABI is required - status-go will have the same
arhitecuture.
This commit implements the "gradient cover" component which is needed for wallet screen development, and upgrades the "react-native-linear-gradient" library to "v2.8.0".
Signed-off-by: Mohamed Javid <19339952+smohamedjavid@users.noreply.github.com>
This commit upgrades Shadow CLJS from 2.11.16 (released on Feb/21) to latest
2.25.0 (Jul/23), so ~1.5 years worth of upgrades. By upgrading shadow we
can finally use the latest major Clojure version 1.11.x.
Why upgrade shadow?
- Shadow CLJS controls the ClojureScript version we can use. In order to use the
latest major Clojure version we must upgrade Shadow CLJS.
- Shadow CLJS releases new versions very frequently, and if you take a look at
its changelog https://github.com/thheller/shadow-cljs/blob/master/CHANGELOG.md, you'll see
it had tons and tons of bug fixes over the years. I hope some of them help
improve the DX for certain contributors who recently reported issues with
it.
- Clojure 1.11 brings new features, bug fixes and even performance improvements
(although I think the performance mostly impacts Clojure on the JVM). See the
changelog https://github.com/clojure/clojure/blob/master/changes.md#changes-to-clojure-in-version-1110
Things that can be beneficial to us, or are interesting nonetheless:
- New :as-alias to be used in require, which is like :as but does not require
the namespace to load. This means namespaced keywords using :as-alias can't
cause circular dependency errors. This feature would very useful if we used
namespaced keywords, but we don't, so...
https://github.com/clojure/clojure/blob/master/changes.md#22-as-alias-in-require
- New macros run-test and run-test-var to run single test with fixtures and
report.
- New iteration function, useful for processing paginated data.
https://www.abhinavomprakash.com/posts/clojure-iteration/
- New update-keys function: applies a function to every key in a map.
- New update-vals function: applies a function to every value in a map.
Examples for update-vals and update-keys. They should perform better than the
common reduce-kv approach since they use a transient data structure.
(let [m {:a 1 :b 2}]
(update-vals m inc)) ; => {:a 2, :b 3}
(let [m {:a 1 :b 2}]
(update-keys m name)) ; => {"a" 1, "b" 2}
Why change namespaces within __tests__ directories?
Any namespace with the word --tests-- throws an error, like the one below. I
didn't bother investigating why, so I changed the guidelines to reflect the new
convention. It's probably related to the double dashes in the name.
Namespace quo2.components.dividers.--tests--.divider-label-component-spec has a
segment starting with an invalid JavaScript identifier at line 1
watchman was upgraded significantly during the last #14944 (4.9.0 (Aug 16, 2017) to 2023.01.30.00 - 6 years between):
status-im/nixpkgs@4e9c02b
Probably causing developers to have "too many files open" issue #16341
This PR is an attempt to fix the issue by downgrading the watchman
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise if in Nix context we can accidentally use GNU stat which
results in a different output due to different command line flags:
```
[nix-shell:~/status-mobile]$ stat -Lf "%Sg" "/nix/store"
stat: cannot read file system information for '%Sg': No such file or directory
File: "/nix/store"
ID: 10000110000001a Namelen: ? Type: apfs
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 242837545 Free: 199242283 Available: 199242283
Inodes: Total: 7971454780 Free: 7969691320
```
And it should be just owner group name or ID. Which in turn results in:
```
Unknown Nix installtion type!
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Android SDK in nixpkg doesn't support `aarch64-darwin`, for details see
https://github.com/status-im/status-mobile/issues/12794
The workaround was to run terminal application with Rosetta or make with `aarch` tool.
This PR overrides system architecture to always be `x86_64` for Android shell and build targets ran by make.
CI is not affected as it runs `nix` directly.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
On macOS `git-lfs` checks fail sometimes and they also take a lot of time to execute.
```log
FAIL github.com/git-lfs/git-lfs/v3/lfshttp 0.820s
FAIL
error: builder for '/nix/store/abd6spw876qvcmlpcwsljsnbpbnvx1fg-git-lfs-3.3.0.drv' failed with exit code 1;
last 10 log lines:
> Error Trace: certs_test.go:210
> Error: Expected value not to be nil.
> Test: TestCertFromSSLCAPathEnv
> certs_test.go:210:
> Error Trace: certs_test.go:210
> Error: Expected value not to be nil.
> Test: TestCertFromSSLCAPathEnv
> FAIL
> FAIL github.com/git-lfs/git-lfs/v3/lfshttp 0.820s
> FAIL
For full logs, run 'nix-store -l /nix/store/abd6spw876qvcmlpcwsljsnbpbnvx1fg-git-lfs-3.3.0.drv'.
```
Failures initially observed by me and @smohamedjavid and then today again reported by @yqrashawn
This commit adds a `nix` overlay that skips those checks
This is a workaround for a well known issue of Nix failing to download
bigger packages from Nix cache due to errors like this:
```
HTTP error 200 (curl error: Transferred a partial file)
```
or
```
HTTP error 200 (curl error: Stream error in the HTTP/2 framing layer)
```
From docs: https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-fallback
>If set to true, Nix will fall back to building from source if a binary
>substitute fails. This is equivalent to the --fallback flag. The default is false.
Realted: https://github.com/status-im/infra-ci/issues/17
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Fixes following error:
```
ld: framework not found IOKit
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [status_nodejs_addon.target.mk:175: Release/status_nodejs_addon.node] Error 1
```
The `make test` target started to fail on macOS after #14944.
This PR adds 2 Apple SDK frameworks to default shell: `IOKit` and `CoreServices`
The other 2 mentioned in `binding.gyp` are not added, but build is ok.
Resolves: https://github.com/status-im/status-mobile/issues/16356
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise Nix produces warnings like this:
```
warning: ignoring untrusted substituter 'https://nix-cache.status.im/', you are not a trusted user.
```
Since adding users to `trusted-users` essentially gives them `root`:
>Adding a user to trusted-users is essentially equivalent to giving that user root access to the system.
> — https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-trusted-users
A more kosher approach might be to just add the cache config itself globally.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Should resolve errors like this:
```
error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
We use `builtins.fetchTarball` to avoid having to first import a
`nixpkgs` from existing channels, which breaks if there are no channels.
Otherwise new Nix 2.14 version can result in:
```
error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)
```
Which happens due to no default `nixpkgs` channel existing:
```
> nix-channel --list
```
Resolves: https://github.com/status-im/status-mobile/issues/16342
Signed-off-by: Jakub Sokołowski <jakub@status.im>
According to this line from the docs:
>The system-wide configuration file sysconfdir/nix/nix.conf (i.e. /etc/nix/nix.conf),
>or $NIX_CONF_DIR/nix.conf if NIX_CONF_DIR is set. Values loaded in this file are not
>forwarded to the Nix daemon. The client assumes that the daemon has already loaded them.
https://nixos.org/manual/nix/stable/command-ref/conf-file.html#description
Our usage of `NIX_CONF_DIR` has been wrong for a while now.
The correct way of applying this config is using `NIX_USER_CONF_FILES`.
In addition the `extra-substituters` no longer exists in the docs.
Use of `trusted-substituters` is necessary according to:
>At least one of the following conditions must be met for Nix to use a substituter:
>
>- the substituter is in the trusted-substituters list
>- the user calling Nix is in the trusted-users list
https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-substituters
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Notable upgrades:
* Bash `5.1` to `5.2`
* Git `2.37.3` to `2.40.1`
* Curl `7.85.0` to `8.0.1`
* OpenSSL `3.0.5` to `3.0.8`
* Go `1.18.6` to `1.18.9`
* NodeJS `18.9.1` to `18.16.0`
* Java `1.8.0_322` to `11.0.11`
* Ruby `3.1.2` to `3.1.4`
* Python `2.7.18` to `3.10.11`
* Clojure `1.11.1.1165` to `1.11.1.1273`
* Clj-kondo `v2022.10.05` to `v2023.04.14`
* Zprint `1.2.5` to `1.2.6`
* Bundler `2.3.22` to `2.4.13`
* Gradle `6.9.2` to `6.9.4`
* Android Platform Tools `33.0.2` to `33.0.3`
* Android SDK Tools to Android SDK Command-Line Tools
Removals:
* Zprint since the version in `nixpkgs` was newer than in overlay.
* Xcode wrapper definition was removed since my fixes were merged:
- https://github.com/NixOS/nixpkgs/pull/204278
- https://github.com/NixOS/nixpkgs/pull/228696
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This commit upgrades re-frame to v1.3.0 (latest stable release), released ~9
months ago, in 2022-08-27. This is a solid upgrade, with no breaking changes as
far as I tested status-mobile. It's a great testament of re-frame's stability
and commitment to backwards compatibility, as are many Clojure libs.
Fixes https://github.com/status-im/status-mobile/issues/15963
The big, and truly relevant addition is the introduction of the :fx built-in
effect that was added ~3 years ago in Aug/2020 in v1.1.0.
Relevant changelog:
- Global interceptors are now supported (added in v1.0.0).
- reg-event-fx will just warn (not generate an error) if the effect map returned
contains an unknown effect key.
- re-frame will now warn us when we are calling subscribe outside of a reactive
context.
- "re-frame now guarantees that a :db effect, if present, will be actioned
before any other sibling effects. re-frame continues to provide NO guarantees
about the order in which other effects will be actioned."
(https://day8.github.io/re-frame/releases/2020/#110-2020-08-24)
- There's syntactic sugar for trivial reg-sub declarations (added in v1.3.0).
See the documentation for reg-sub for more details
https://day8.github.io/re-frame/api-re-frame.core/#reg-sub
- "The built-in effect :dispatch-later can now take a single map value.
Supplying a sequence of maps is now deprecated in favor of using multiple
:dispatch-later effects within the new :fx effect."
https://day8.github.io/re-frame/releases/2020/#111-2020-08-26
For some unknown to me reason we are using a different Yarn call to
Shadow-cljs to generate the JSBundle for iOS builds, while the one
created by the Android derivation shoudl be exactly the same.
I'm changing the target to just be `make jsbundle` while keeping aliases
referencing old naming, and moving things around in `nix` folder to
reflect the fact that the derivation is no longer Android-specific.
Also, crucially, I've changed the `import` in `index.js` to use the
`./result/index.js` path, since that's what Nix creates. I'm not sure if
this clashes with any developer workflow that takes place locally, so
I'd appreciate some testing from developers.
Depends on: https://github.com/status-im/status-jenkins-lib/pull/67
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This way we will use the same format as Nix logs and errors when a hash
changes, which avoids confusion.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise we can get a failure like this:
```
Regenerating Nix files...
Found 40 sub-projects...
Found 609 direct dependencies...
Found 889 dependency URLs...
! Failed to fetch: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-bom/9.5/asm-bom-9.5.jar
```
When using `make nix-update-gradle`.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise we see weird errors like this:
```
/nix/store/g0jijpgcb4q54zbvz5p8yvxcnb6lshnk-stdenv-darwin/setup: line 1391: /nix/store/sja8sqq4y5s9ijkb97i3pi2jrhsy40cz-xcode-wrapper-14.3/bin/xcodebuild: Operation not permitted
We require xcodebuild version: 14.3
error: builder for '/nix/store/d1dji0ywl851wgj9vv58ibpm32gq3wsm-xcode-wrapper-14.3.drv' failed with exit code 1;
last 2 log lines:
> /nix/store/g0jijpgcb4q54zbvz5p8yvxcnb6lshnk-stdenv-darwin/setup: line 1391: /nix/store/sja8sqq4y5s9ijkb97i3pi2jrhsy40cz-xcode-wrapper-14.3/bin/xcodebuild: Operation not permitted
> We require xcodebuild version: 14.3
```
Related to: https://github.com/NixOS/nixpkgs/pull/228696
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Also adds an assertion that gives a better error when trying to use an
`ios` Nix shell on an unsupported operating system.
Resolves:
https://github.com/status-im/status-mobile/issues/15695
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Icaro Motta
Dmitri Akatov
yqrashawn
Ibrahem Khalil
Jakub Sokołowski
Vitaliy Vlasov
Anton Iakimov
Andrea Maria Piana
Mohamed Javid
Siddarth Kumar
Jamie Caprani
flexsurfer
Ajay Sivan