status-desktop/.gitignore

30 lines
360 B
Plaintext
Raw Normal View History

/data
2020-05-06 17:40:00 +00:00
noBackup/
2020-05-08 16:08:59 +00:00
.idea
2020-05-11 19:54:02 +00:00
*.pro.user
*.pro.autosave
*.qml.autosave
feat: introduce Task Manager threadpool The `TaskManager` threadpool is a memory-safe replacement for the `spawnAndSend` operations that are currently causing memory issues in status-desktop. From a fundamental memory management point of view, `libstatus/settings`, `libstatus/contracts`, and `libstatus/tokens` (custom tokens) have all been converted to `{.threadvar.}`s and `Atomic[bool]`s to maintain the cache and `dirty` flag across threads, respectively, eliminating the need for thread locks and incorrect `{.gcsafe.}` compiler overrides. The successful [recyclable threadpool experiment from `nim-task-runner`](https://github.com/status-im/nim-task-runner/blob/test/use-cases/test/use_cases/test_sync.nim) using `AsyncChannel[ThreadSafeString]`s was brought over to `status-desktop` and implemented in somewhat of a hardcoded manner, as we knew this would save some time instead of trying to create a fully fleshed out `nim-task-runner` API and build a miraculous macro that may or may not be able to generate the needed API. The threadpool is started by the `TaskManager` and both the `TaskManager` and the `TaskManager`'s threadpool are started as early as possible in the application lifecycle (in `nim_status_client.nim`). The `TaskManager` creates a thread to run the threadpool. During its initialization, the threadpool then spools up all the threads it will manage and puts them in an idle thread sequence. This is to prevent expensive thread creation and teardown happening during the app's lifetime as it is quite expensive and blocks the main thread. When tasks comes in to the pool, the task is sent to an idle thread, or put in a queue if all threads are busy. The idle thread is moved to the busy thread sequence. When a task is completed, the thread is taken out of the busy threads sequence and moved back in to the sequence of idle threads, effectively recycling it. The first `spawnAndSend` we were able to change over to the new threadpool was `estimate`, which estimates the gas of a sticker purchase transaction. From the consumer point of view, the existing `spawnAndSend` to achieve this looks like: ```nim proc estimate*(self: StickersView, packId: int, address: string, price: string, uuid: string) {.slot.} = let status_stickers = self.status.stickers spawnAndSend(self, "setGasEstimate") do: var success: bool var estimate = status_stickers.estimateGas(packId, address, price, success) if not success: estimate = 325000 let result: tuple[estimate: int, uuid: string] = (estimate, uuid) Json.encode(result) ``` And the new syntax looks like this: ```nim proc estimate*(self: StickersView, packId: int, address: string, price: string, uuid: string) {.slot.} = self.status.taskManager.threadPool.stickers.stickerPackPurchaseGasEstimate(cast[pointer](self.vptr), "setGasEstimate", packId, address, price, uuid) ``` The logic inside the `spawnAndSend` body was moved to [src/status/tasks/stickers.nim](https://github.com/status-im/status-desktop/compare/experiment/tasks-3?expand=1#diff-09e57eef00b0cee5c4abdb9039f948d8372e7003e09e934a9b4c7e9167d47658). This is just the first migration of `spawnAndSend`, however moving the majority of the remaining `spawnAndSend`s will likely just be an exercise in copy/pasta. There will be one or two that may require a bit more thinking, depending how they rely on data from the model. Once the `spawnAndSend`s have been converted to the threadpool, we can start implementing the [long-running process from the task runner use case experiments](https://github.com/status-im/nim-task-runner/blob/test/use-cases/test/use_cases/test_long_running.nim). And finally, we can then implement the [async tasks](https://github.com/status-im/nim-task-runner/blob/test/use-cases/test/use_cases/test_async.nim) if needed. @michaelsbradleyjr and I spent many hours digging in to the depths of nim's memory management in an attempt to understand it. We have created [a presentation with our task runner experiment findings](https://docs.google.com/presentation/d/1ItCxAfsVTcIoH_E4bgvmHljhbU-tC3T6K2A6ahwAedk/edit?usp=sharing), and @michaelsbradleyjr has spent time [answering questions off the back of that presentation.](https://gist.github.com/michaelsbradleyjr/1eaa9937b3fbb4ffff3fb814f0dd82a9). We have created a fork of `edn.nim` at `status-im/edn.nim` and we need the PR to be merged and the commit hash updated before we can merge this PR in to `status-desktop`.
2021-03-17 06:25:41 +00:00
.update.timestamp
.vscode
2020-05-14 20:19:01 +00:00
bin/
chore: refactor Linux and macOS build/packaging steps Replaces PR #105. Implement a `pkg-macos` target that ultimately results in `Status.dmg` being written to `pkg/`. Due to [limitations][limits] of the OpenSSL `.dylib`s in `/usr/lib/` on macOS, `libssl.a` and `libcrypto.a` are statically linked into `bin/nim_status_client` from a [Homebrew][brew] "bottle" that is compatible with macOS 10.13 (the oldest macOS version compatible with Qt v5.14). `pkg-macos` creates an `.app` bundle layout in `tmp/macos/dist` based partly on information in a very helpful [StackOverflow answer][so-answer]. Note the part of the answer (toward the end) that explains a problem with the working directory and how to fix it. That's the reason for the `nim_status_client.sh` script introduced in this commit (it gets copied into the bundle). It's also the reason for having `Info.plist` copied into the bundle before `macdeployqt` is run (see below) and then overwriting it with `Info.runner.plist` before creating the `.dmg` file. The app icons file `status-icon.icns` was taken from `deployment/macos/` in the [status-react][sr] repo. The [`macdeployqt`][macdeployqt] tool is used to copy the needed portions of Qt into the bundle; it automatically updates `rpath`, etc. so the `nim_status_client` executable in the bundle can locate the libs within the bundle. `macdeployqt` is run twice, for the "outer" and "inner" `.app` bundles, because of an apparent bug in `macdeployqt` that results in QtWebEngine related resources not being processed correctly on the first pass. This results in some bloat in the final bundle but it seems unavoidable at present. The [create-dmg][cdmg] tool is used to package the bundle into a `.dmg` file. There are two reasons for this: 1. It produces a nice looking icon for the `.dmg` that overlays the Status logo on an external disk icon. 2. `Info.plist` needs to be overwritten after running `macdeployqt` (see explanation above) but before creating the `.dmg` file. If we passed the `-dmg` cli option to `macdeployqt` to have it generate the `.dmg` file then it wouldn't be possible to overwrite `Info.plist`. So there is a cosmetic reason and a practical reason for using another tool. Probably the biggest downside is that `create-dmg` is implemented in Node.js so it needs to be installed with `npm`; that's the reason this commit introduces `package.json`, etc. Note that zero code from `node_modules` ends up in the `.app` bundle or the `.dmg` file. Code signing of the macOS `.app` bundle and `.dmg` is attempted if the environment variable `MACOS_CODESIGN_IDENT` is defined. In that case, the environment variable `MACOS_KEYCHAIN_OPT` may optionally be defined with the path to a preferred keychain database file. Refactor a number of sections in the Makefile for consistency's sake, e.g. the `appimage` target becomes `pkg-linux` and ultimately results in `NimStatusClient-x86_64.AppImage` being written to `pkg/`. Make a number of changes to bring the Linux packaging steps up-to-date and use the `-qmlimport` cli option of `linuxdeployqt` to simplify resolution of Qt plugins. Note that `make pkg` will correctly resolve to `make pkg-linux` or `make pkg-macos` depending on the OS in use. Consistently use lower-case "c" in the name of *components* directories and imports. [limits]: https://developer.apple.com/forums/thread/124782 [brew]: https://brew.sh/ [so-answer]: https://stackoverflow.com/a/3251285 [sr]: https://github.com/status-im/status-react/tree/develop/deployment/macos [macdeployqt]: https://doc.qt.io/qt-5/macos-deployment.html [cdmg]: https://github.com/sindresorhus/create-dmg
2020-06-04 20:56:44 +00:00
/bottles/
pkg/
2020-05-14 20:19:01 +00:00
vendor/.nimble
2020-05-15 13:00:49 +00:00
*.AppImage
tmp
2020-05-19 18:10:46 +00:00
nimcache
.DS_Store
chore: refactor Linux and macOS build/packaging steps Replaces PR #105. Implement a `pkg-macos` target that ultimately results in `Status.dmg` being written to `pkg/`. Due to [limitations][limits] of the OpenSSL `.dylib`s in `/usr/lib/` on macOS, `libssl.a` and `libcrypto.a` are statically linked into `bin/nim_status_client` from a [Homebrew][brew] "bottle" that is compatible with macOS 10.13 (the oldest macOS version compatible with Qt v5.14). `pkg-macos` creates an `.app` bundle layout in `tmp/macos/dist` based partly on information in a very helpful [StackOverflow answer][so-answer]. Note the part of the answer (toward the end) that explains a problem with the working directory and how to fix it. That's the reason for the `nim_status_client.sh` script introduced in this commit (it gets copied into the bundle). It's also the reason for having `Info.plist` copied into the bundle before `macdeployqt` is run (see below) and then overwriting it with `Info.runner.plist` before creating the `.dmg` file. The app icons file `status-icon.icns` was taken from `deployment/macos/` in the [status-react][sr] repo. The [`macdeployqt`][macdeployqt] tool is used to copy the needed portions of Qt into the bundle; it automatically updates `rpath`, etc. so the `nim_status_client` executable in the bundle can locate the libs within the bundle. `macdeployqt` is run twice, for the "outer" and "inner" `.app` bundles, because of an apparent bug in `macdeployqt` that results in QtWebEngine related resources not being processed correctly on the first pass. This results in some bloat in the final bundle but it seems unavoidable at present. The [create-dmg][cdmg] tool is used to package the bundle into a `.dmg` file. There are two reasons for this: 1. It produces a nice looking icon for the `.dmg` that overlays the Status logo on an external disk icon. 2. `Info.plist` needs to be overwritten after running `macdeployqt` (see explanation above) but before creating the `.dmg` file. If we passed the `-dmg` cli option to `macdeployqt` to have it generate the `.dmg` file then it wouldn't be possible to overwrite `Info.plist`. So there is a cosmetic reason and a practical reason for using another tool. Probably the biggest downside is that `create-dmg` is implemented in Node.js so it needs to be installed with `npm`; that's the reason this commit introduces `package.json`, etc. Note that zero code from `node_modules` ends up in the `.app` bundle or the `.dmg` file. Code signing of the macOS `.app` bundle and `.dmg` is attempted if the environment variable `MACOS_CODESIGN_IDENT` is defined. In that case, the environment variable `MACOS_KEYCHAIN_OPT` may optionally be defined with the path to a preferred keychain database file. Refactor a number of sections in the Makefile for consistency's sake, e.g. the `appimage` target becomes `pkg-linux` and ultimately results in `NimStatusClient-x86_64.AppImage` being written to `pkg/`. Make a number of changes to bring the Linux packaging steps up-to-date and use the `-qmlimport` cli option of `linuxdeployqt` to simplify resolution of Qt plugins. Note that `make pkg` will correctly resolve to `make pkg-linux` or `make pkg-macos` depending on the OS in use. Consistently use lower-case "c" in the name of *components* directories and imports. [limits]: https://developer.apple.com/forums/thread/124782 [brew]: https://brew.sh/ [so-answer]: https://stackoverflow.com/a/3251285 [sr]: https://github.com/status-im/status-react/tree/develop/deployment/macos [macdeployqt]: https://doc.qt.io/qt-5/macos-deployment.html [cdmg]: https://github.com/sindresorhus/create-dmg
2020-06-04 20:56:44 +00:00
nim-status-client*.tgz
node_modules
npm-debug.log*
npm-shrinkwrap.json
/package/
yarn-debug.log*
yarn-error.log*
yarn.lock
TODO
resources.rcc
resources.qrc
status-react-translations/
/.update.timestamp