feat: command-line option can be used to specify app's data directory

In the repo:
```
$ bin/nim_status_client --help
```
In the packaged app (macOS example):
```
$ cd /Applications/Status.app/Contents/MacOS
$ ./nim_status_client --help
```
Output:
```
Usage:

nim_status_client [OPTIONS]...

The following options are available:

 -d, --dataDir      Status Desktop data directory.
```

**Using the option**

```
$ cd ~/status-ci-builds/master/Status.app/Contents/MacOS
$ ./nim_status_client --dataDir:"${HOME}/status-dirs/master"
```
In another terminal:
```
$ cd ~/status-ci-builds/PR-4242/Status.app/Contents/MacOS
$ ./nim_status_client --dataDir:"${HOME}/status-dirs/PR-4242"
```

The path supplied can be relative or absolute, and can be specified with
`--dataDir:[path]`, `--dataDir=[path]`, `-d:[path]`, or `-d=[path]`.

Either `:` or `=` must be used, i.e. this *will not* work: `--dataDir [path]`
or `-d [path]`.

The name of the option follows Nim's partial case-insensitivity rules, so
`--dataDir`, `--datadir`, and `--data_dir` are all equivalent. See
[Identifier equality][ieq] in the Nim Manual.

It is possible to run the same build in multiple terminals by supplying
different `--dataDir`, i.e. this works:
```
$ cd /Applications/Status.app/Contents/MacOS
$ ./nim_status_client --dataDir="${HOME}/temp/some1"
```
In another terminal:
```
$ cd /Applications/Status.app/Contents/MacOS
$ ./nim_status_client --dataDir="${HOME}/temp/some2"
```

**Windows**

It is recommended to use a Git Bash or MSYS2 terminal when invoking
`bin/nim_status_client.exe` (development build) or `bin/Status.exe` (production
build) on the command-line. The reason is that if the exe is invoked in a
session of `cmd.exe` it will return to the prompt immediately; the app will run
but there will be no output in the terminal. In any case, the `--dataDir`
option will take effect whether the exe is invoked in `cmd.exe` or a
recommended terminal.

For development builds, when invoking `bin/nim_status_client.exe` directly
instead of via `make run`, because e.g. you wish to use the `--dataDir` option,
it is required to first setup the `PATH` environment variable correctly. See
the `run-windows` target in this repo's Makefile for more information.

**Linux**

The `--dataDir` option may be passed to command-line invocation of a
production (AppImage) build in the same way as passing it to a development
build:

```
$ Status.AppImage --dataDir:/path/to/wherever
```

For development builds, when invoking `bin/nim_status_client` directly instead
of via `make run`, because e.g. you wish to use the `--dataDir` option, it is
required to setup the `LD_LIBRARY_PATH` environment variable correctly. See the
`run-linux` target in this repo's Makefile for more information.

---

BREAKING CHANGE: The `qt` subdir of the app's data directory is now a sibling
of the status-go directory rather than a subdir of the status-go directory:

```
Status (app data directory)
├── data (status-go directory)
├── qt
└── tmp
```

Because app settings are stored in the `qt` directory that means that existing
installations will lose their customized settings.

At app startup, it would be possible to detect `Status/data/qt` and if
`Status/qt` doesn't exist yet then copy `Status/data/qt` to
`Status/qt`. However, there was some concern that behavior could lead to
problems later on if we forget the workaround is in place. So for now that
settings preservation strategy has not been implemented, but it might be before
this commit is merged pending full team awareness/consensus.

---

Command-line option support is provided by
[nim-confutils](https://github.com/status-im/nim-confutils).

The environment variable `NIM_STATUS_CLIENT_DEV` has been removed in favor of
passing a "define" option to the Nim compiler: `-d:development` for development
builds (e.g. `make V=1`) and `-d:production` for packaged builds (e.g. `make
V=1 pkg`). Passing the correct option is handled automatically by the Makefile.

A make variable named `RELEASE` has been introduced, which defaults to
`false`. Presently the `RELEASE` variable should not be set on the command-line
nor in CI as more work needs to be done to toggle the proper compiler flags. In
the case of Status Desktop, "release vs. debug" is a concern orthogonal to
"production vs. development". At present, production builds and development
builds are all debug builds, but that will likely change in the future: we can
have non-release CI production builds and local development builds be debug
builds, while release builds in CI would be production builds with
`RELEASE=true` (the compiled executable will be fully optimized).

Prior to the changes in this PR, symmetry is somewhat lacking between
development and production (packaged) builds with respect to the concept of the
"data directory". In development builds the root of the repo effectively serves
as the `Status` directory used by production builds, e.g. on macOS
`~/Library/Application Support/Status`. Also, there's a bit of confusion as to
whether "data directory" refers to a directory for the desktop app's overall
data (including status-go data) or to the specific directory used by status-go.

This PR attempts to provide symmetry and reduce confusion:
* The term "data directory" means the directory used by the desktop app to
store multiple kinds of data and is not a reference to the subdirectory used by
status-go.
* For development builds the "data directory" defaults to `./Status/` relative
to the root of the repo.
* For production builds the "data directory" default is the same as before,
e.g. on macOS it's ` ~/Library/Application Support/Status/`.

The directory used by status-go is `Status/data/`. To be clear, that should be
referred to as the "status-go directory" and not the app's "data directory". It
would nice if we could rename it from `Status/data/` to `Status/status-go/`. We
can do that, I already checked that it works correctly; however, for existing
installations it would require that at app launch we check for the presence of
`Status/data/` and rename it to `Status/status-go`. While simple enough to do,
I was concerned that there might be edge cases where the directory rename could
cause a problem (e.g. if another copy of the app is running) so chose for now
to stick with the status-go directory being `Status/data/`.

---

**NOTES**

More work needs to be done to ensure that all data written by the app is
contained in the default or cli-specified data directory. Currently, both
development and production (packaged) builds are writing to common directories
outside of the data directory, e.g. located within `~/Library/` on
macOS. Changing that behavior seems like it will mainly involve changing
defaults related to Qt components such as the web engine. See:
https://github.com/status-im/status-desktop/issues/1141.

In general, additional refactoring could be done in the future. For
example, implementing `StatusDesktopConfig` in
`src/status/libstatus/accounts/constants.nim` (as done in this PR) works fine
for now, but better code organization is desirable.

---

Closes #2268

[ieq]: https://nim-lang.org/docs/manual.html#lexical-analysis-identifier-equality

.gitignore

@@ -1,3 +1,4 @@
+/Status
 /data
 noBackup/
 .idea

.gitmodules

@@ -97,3 +97,6 @@
 [submodule "vendor/edn.nim"]
 	path = vendor/edn.nim
 	url = https://github.com/status-im/edn.nim.git
+[submodule "vendor/nim-confutils"]
+	path = vendor/nim-confutils
+	url = https://github.com/status-im/nim-confutils.git

Makefile

@@ -151,12 +151,20 @@ else
  NIM_EXTRA_PARAMS := --passL:"-lsetupapi -lhid"
 endif
 
-# TODO: control debug/release builds with a Make var
+RELEASE ?= false
-# We need `-d:debug` to get Nim's default stack traces.
+ifeq ($(RELEASE),false)
-NIM_PARAMS += --outdir:./bin -d:debug
+ # We need `-d:debug` to get Nim's default stack traces
-# Enable debugging symbols in DOtherSide, in case we need GDB backtraces from it.
+ NIM_PARAMS += -d:debug
-CFLAGS += -g
+ # Enable debugging symbols in DOtherSide, in case we need GDB backtraces
-CXXFLAGS += -g
+ CFLAGS += -g
+ CXXFLAGS += -g
+else
+ # Additional optimization flags for release builds are not included at present;
+ # adding them will involve refactoring config.nims in the root of this repo
+ NIM_PARAMS += -d:release
+endif
+
+NIM_PARAMS += --outdir:./bin
 
 $(DOTHERSIDE): | deps
 	echo -e $(BUILD_MSG) "DOtherSide"
@@ -215,6 +223,9 @@ DEFAULT_TOKEN := 220a1abb4b6943a093c35d0ce4fb0732
 INFURA_TOKEN ?= $(DEFAULT_TOKEN)
 NIM_PARAMS += -d:INFURA_TOKEN:"$(INFURA_TOKEN)"
 
+RESOURCES_LAYOUT := -d:development
+
+nim_status_client: NIM_PARAMS += $(RESOURCES_LAYOUT)
 nim_status_client: | $(DOTHERSIDE) $(STATUSGO) $(QRCODEGEN) $(FLEETS) rcc deps
 	echo -e $(BUILD_MSG) "$@" && \
 		$(ENV_SCRIPT) nim c $(NIM_PARAMS) --passL:"-L$(STATUSGO_LIBDIR)" --passL:"-lstatus" $(NIM_EXTRA_PARAMS) --passL:"$(QRCODEGEN)" --passL:"-lm" src/nim_status_client.nim && \
@@ -238,6 +249,7 @@ $(APPIMAGE_TOOL):
 
 STATUS_CLIENT_APPIMAGE ?= pkg/NimStatusClient-x86_64.AppImage
 
+$(STATUS_CLIENT_APPIMAGE): override RESOURCES_LAYOUT := -d:production
 $(STATUS_CLIENT_APPIMAGE): nim_status_client $(APPIMAGE_TOOL) nim-status.desktop
 	rm -rf pkg/*.AppImage
 	rm -rf tmp/linux/dist
@@ -279,6 +291,7 @@ MACOS_INNER_BUNDLE := $(MACOS_OUTER_BUNDLE)/Contents/Frameworks/QtWebEngineCore.
 
 STATUS_CLIENT_DMG ?= pkg/Status.dmg
 
+$(STATUS_CLIENT_DMG): override RESOURCES_LAYOUT := -d:production
 $(STATUS_CLIENT_DMG): nim_status_client $(DMG_TOOL)
 	rm -rf tmp/macos pkg/*.dmg
 	mkdir -p $(MACOS_OUTER_BUNDLE)/Contents/MacOS
@@ -349,6 +362,7 @@ endif
 
 STATUS_CLIENT_ZIP ?= pkg/Status.zip
 
+$(STATUS_CLIENT_ZIP): override RESOURCES_LAYOUT := -d:production
 $(STATUS_CLIENT_ZIP): nim_status_client nim_windows_launcher $(NIM_WINDOWS_PREBUILT_DLLS)
 	rm -rf pkg/*.zip
 	rm -rf tmp/windows/dist
@@ -426,30 +440,24 @@ $(ICON_TOOL):
 	echo -e "\e[92mInstalling:\e[39m fileicon"
 	npm i
 
-NIM_STATUS_CLIENT_DEV ?= t
+# Currently not in use: https://github.com/status-im/status-desktop/pull/1858
-STATUS_PORT ?= 30306
+# STATUS_PORT ?= 30306
 
 set-status-macos-dev-icon: $(ICON_TOOL)
 	npx fileicon set bin/nim_status_client status-dev.icns
 
 run-linux:
 	echo -e "\e[92mRunning:\e[39m bin/nim_status_client"
-	NIM_STATUS_CLIENT_DEV="$(NIM_STATUS_CLIENT_DEV)" \
 	LD_LIBRARY_PATH="$(QT5_LIBDIR)":"$(STATUSGO_LIBDIR)" \
-	STATUS_PORT="$(STATUS_PORT)" \
 	./bin/nim_status_client
 
 run-macos: set-status-macos-dev-icon
 	echo -e "\e[92mRunning:\e[39m bin/nim_status_client"
-	NIM_STATUS_CLIENT_DEV="$(NIM_STATUS_CLIENT_DEV)" \
-	STATUS_PORT="$(STATUS_PORT)" \
 	./bin/nim_status_client
 
 run-windows: $(NIM_WINDOWS_PREBUILT_DLLS)
 	echo -e "\e[92mRunning:\e[39m bin/nim_status_client.exe"
-	NIM_STATUS_CLIENT_DEV="$(NIM_STATUS_CLIENT_DEV)" \
 	PATH="$(shell pwd)"/"$(shell dirname "$(DOTHERSIDE)")":"$(STATUSGO_LIBDIR)":"$(shell pwd)"/"$(shell dirname "$(NIM_WINDOWS_PREBUILT_DLLS)")":"$(PATH)" \
-	STATUS_PORT="$(STATUS_PORT)" \
 	./bin/nim_status_client.exe
 
 endif # "variables.mk" was not included

src/app/utilsView/view.nim

@@ -30,9 +30,6 @@ QtObject:
     result.status = status
     result.setup
 
-  proc getDataDir*(self: UtilsView): string {.slot.} =
-    result = accountConstants.DATADIR
-
   proc getOs*(self: UtilsView): string {.slot.} =
     if defined(windows):
       return "windows"

src/nim_status_client.nim

@@ -26,7 +26,7 @@ logScope:
 
 proc mainProc() =
   let fleets =
-    if defined(windows) and getEnv("NIM_STATUS_CLIENT_DEV").string == "":
+    if defined(windows) and defined(production):
       "/../resources/fleets.json"
     else:
       "/../fleets.json"
@@ -46,7 +46,7 @@ proc mainProc() =
 
   let app = newQApplication("Status Desktop")
   let resources =
-    if defined(windows) and getEnv("NIM_STATUS_CLIENT_DEV").string == "":
+    if defined(windows) and defined(production):
       "/../resources/resources.rcc"
     else:
       "/../resources.rcc"
@@ -55,9 +55,9 @@ proc mainProc() =
   let statusAppIcon =
     if defined(macosx):
       "" # not used in macOS
-    elif defined(windows) and getEnv("NIM_STATUS_CLIENT_DEV").string == "":
+    elif defined(windows) and defined(production):
       "/../resources/status.svg"
-    elif getEnv("NIM_STATUS_CLIENT_DEV").string != "":
+    elif defined(development):
       "/../status-dev.svg"
     else:
       "/../status.svg"
@@ -65,7 +65,7 @@ proc mainProc() =
     app.icon(app.applicationDirPath & statusAppIcon)
 
   var i18nPath = ""
-  if (getEnv("NIM_STATUS_CLIENT_DEV").string != ""):
+  if defined(development):
     i18nPath = joinPath(getAppDir(), "../ui/i18n")
   elif (defined(windows)):
     i18nPath = joinPath(getAppDir(), "../resources/i18n")

src/nim_windows_launcher.nim

@@ -1,9 +1,9 @@
-from os import getCurrentDir, joinPath
+from os import getAppDir, joinPath
 from winlean import Handle, shellExecuteW
 
 const NULL: Handle = 0
-let cwd = getCurrentDir()
+let launcherDir = getAppDir()
-let workDir_str = joinPath(cwd, "bin")
+let workDir_str = joinPath(launcherDir, "bin")
 let exePath_str = joinPath(workDir_str, "Status.exe")
 let open_str = "open"
 let params_str = ""

src/status/libstatus/accounts.nim

@@ -59,15 +59,9 @@ proc generateAlias*(publicKey: string): string =
 proc generateIdenticon*(publicKey: string): string =
   result = $status_go.identicon(publicKey)
 
-proc ensureDir(dirname: string) =
-  if not existsDir(dirname):
-    # removeDir(dirname)
-    createDir(dirname)
-
 proc initNode*() =
-  ensureDir(DATADIR)
+  createDir(STATUSGODIR)
-  ensureDir(KEYSTOREDIR)
+  createDir(KEYSTOREDIR)
-
   discard $status_go.initKeystore(KEYSTOREDIR)
 
 proc parseIdentityImage*(images: JsonNode): IdentityImage =
@@ -81,7 +75,7 @@ proc parseIdentityImage*(images: JsonNode): IdentityImage =
         result.large = image["uri"].getStr
 
 proc openAccounts*(): seq[NodeAccount] =
-  let strNodeAccounts = status_go.openAccounts(DATADIR).parseJson
+  let strNodeAccounts = status_go.openAccounts(STATUSGODIR).parseJson
   # FIXME fix serialization
   result = @[]
   if (strNodeAccounts.kind != JNull):

src/status/libstatus/accounts/constants.nim

@@ -1,3 +1,4 @@
+import confutils
 import json
 import os
 
@@ -170,15 +171,15 @@ var NODE_CONFIG* = %* {
 
 const DEFAULT_NETWORK_NAME* = "mainnet_rpc"
 
-let sep = if defined(windows): "\\" else: "/"
+const sep = when defined(windows): "\\" else: "/"
 
-let homeDir = getHomeDir()
+proc defaultDataDir(): string =
-
+  let homeDir = getHomeDir()
-let parentDir =
+  let parentDir =
-  if getEnv("NIM_STATUS_CLIENT_DEV").string != "":
+    if defined(development):
-    "."
+      parentDir(getAppDir())
     elif homeDir == "":
-    "."
+      getCurrentDir()
     elif defined(macosx):
       joinPath(homeDir, "Library", "Application Support")
     elif defined(windows):
@@ -193,13 +194,22 @@ let parentDir =
         joinPath(homeDir, ".config")
       else:
         targetDir
+  absolutePath(joinPath(parentDir, "Status"))
 
-let clientDir =
+type StatusDesktopConfig = object
-  if parentDir != ".":
+    dataDir* {.
-    joinPath(parentDir, "Status")
+      defaultValue: defaultDataDir()
-  else:
+      desc: "Status Desktop data directory"
-    parentDir
+      abbr: "d" .}: string
 
-let DATADIR* = joinPath(clientDir, "data") & sep
+let desktopConfig = StatusDesktopConfig.load()
-let KEYSTOREDIR* = joinPath(clientDir, "data", "keystore") & sep
+
-let TMPDIR* = joinPath(clientDir, "tmp") & sep
+let
+  baseDir = absolutePath(expandTilde(desktopConfig.dataDir))
+  DATADIR* = baseDir & sep
+  STATUSGODIR* = joinPath(baseDir, "data") & sep
+  KEYSTOREDIR* = joinPath(baseDir, "data", "keystore") & sep
+  TMPDIR* = joinPath(baseDir, "tmp") & sep
+
+createDir(DATADIR)
+createDir(TMPDIR)

src/status/tasks/marathon/mailserver/model.nim

@@ -62,7 +62,7 @@ proc newMailserverModel*(vptr: ByteAddress): MailserverModel =
 proc init*(self: MailserverModel) =
   trace "MailserverModel::init()"
   let fleets =
-    if defined(windows) and getEnv("NIM_STATUS_CLIENT_DEV").string == "":
+    if defined(windows) and defined(production):
       "/../resources/fleets.json"
     else:
       "/../fleets.json"

vendor/nim-confutils

@@ -0,0 +1 @@
+Subproject commit f091a70a5bf95ec772c8b4d9978e81b8ae89af0c