There are three separate instances of process log APIs: embark logs, blockchain logs (when in standalone mode), and child process logs (storage, communication, blockchain, etc). Each one was repeating the implementation of creating a process log API endpoint. This commit centralises the API declaration by using the class `ProcessLogsApi`.
`ProcessLogsApi` is started for all three components mentioned above: blockchain (in standalone) in the `BlockchainListener` module, embark in the `EmbarkListener` module, and for all child processes in the `ProcessLauncher`.
These listeners have two functions:
1. Create the process logs API endpoints for `get` and `ws`, and
2. Ensure that all logs are logged through the `LogHandler`, which normalises the output of the log and ensures each log has a timestamp and id (used in the cockpit for log ordering).
Also, this commit moved the pipeline in to a module, so that the `embark` object could be passed to the `ProcessLogsApi` (to be used for registering API endpoints).
When running `embark blockchain` followed by `embark run` previously, logs generated in the standalone `embark blockchain` process were black boxed and not accessible to the main Embark process.
This is fixed by creating a client IPC connection in the `embark blockchain` process that connects to the IPC server connection running in `embark run`. The connection is made by way of polling `ipc.connect` and continues polling even after a connection is made in case `embark run` is killed and restarted without restarting `embark blockchain`.
`LogHandler` was introduced to extrapolate functionality used in `ProcessLauncher` that needed to also be used in the standalone blockchain process. It also caps the number of logs that are stored in memory per process by a constant value defined in `constants.json`.
A `blockchain_listener` was module was created (and run inside of `embark run`) that listens for logs emitted by the `embark blockchain` client IPC and runs them through the `LogHandler`. Additionally, this module registers the API endpoints needed to handle requests for blockchain process logs in the cockpit (which were 404’ing before).
# Conflicts:
# lib/modules/blockchain_process/blockchain.js
display last line on tx
fix debugger call
listen to source event only after jumping to the end
keep track of last tx; add minimal debug feature; fix ast issue
initial debugger apis & ui integration
prevent crash when step is out of bounds; send all all available data in websocket
add debugger commands
fix line number tracking in editor; toggle breakpoints
replace timeouts with callbacks
add debugger manager & refactor
refactor debugger api
refactor cmd line debugger
reduce debugger decoupling
reduce debugger decoupling
fix debug buttons
trigger source update so api triggers ws event to update source location
move locals and contracts vars to a json view
improve debugger icons
simplify debugger data
update debug package
add command handler to get a contract given a tx; update debugger so it can get a contract by its tx instead of tracking latest txs only
update debugger package
*Console.js*
- Moved `DEFAULT_PROCESS` const to outside of the `Console` class (but inside the module).
- Removed `(` and `)` from `.filter` in `getProcessLogs()`.
- Updated comments
*logger.js*
- Moved `dateFormat` and `logRegex` to constants outside of `Logger` class
- Moved the `parseLogFile` method inside of the `Logger` class (ES6 style)
- Added a log limit to the `parseLogFile` method
- Added the log path to the constants file and used inside of `Logger`
*cmd_controller.js*
- Defaulted `this.context` to `[constants.context.any]` in the constructor.
- Changed `’embark’` to split modules`coreProcess` and `loggerApi`.
*engine.js*
- Changed `’embark’` to split modules`coreProcess` and `loggerApi`.
The embark log file is being overwritten each time embark is run. There is a separate log file for each context, so that running, for example, `embark run` then `embark console` doesn’t get the `run` log overwritten with the `console` log.
- Created an “embark” module so that an “embark” process could be registered in the correct way. This service is only used on `embark run` (can be extended to other commands if needed).
- extracted “embark” to a const `DEFAULT_PROCESS` param in the `Console` component.
- extracted commands result rendering to it’s own function to keep the `renderTabs` function from getting cluttered
- Added sorting of logs by timestamp
- Added milliseconds to the log file data (which helps in sorting log messages).
A logfile is now generated by default, in the format `.embark/embark-log__YYYY-MM-DD_HH-mm-ss.log`.
When the home tab is loaded, the process logs are fetched for all the processes. The list of processes returned now includes `embark`, and when `/embark-api/process-logs/embark` is fetched, the logFile is parsed and an array of log messages are returned.
Cargo in fact, bundles up subsequent tasks in to an array, so any tasks that are not immediately run get bundled in to another run later. This helps when lots of changes have been made in a short period of time.
Due to a process being spawned for every pipeline run, concurrency should remain around 3 to keep number of child processes from running away and allowing the CPU to stay on top.
For file changes that do not require a webpack run, ie HTML, the assets will still be copied to the output directory, but webpack will not run (as it’s too slow).
Addons
- New chain initialization and genesis management
- Option to choose client to use
- Option to "ping forever" for Geth
- Creation and unlock of accounts at client's start
- Utility to fund accounts with ethers
- Miner settings inside the ethereum client
- Workaround to CORS problem: origin is now http://embark
- Several double callback's checks
Updates
- Boilerplate, templates, configuration files and demo stuff
- Messages and i18n strings
- Tests
Fixes
- Geth client now uses miner.gastarget instead of the deprecated targetGasLimit
- Workaround for shh_version with Parity
Reworks of other PRs into the new code
- Included delayed proxy
- Send ready only when the proxy is started
- Start HTTP and WS proxies individually
- Async setupProxy
- Fixed datadir for GethMiner
Prior to this commit `$ embark build --contracts` spinned up a blockchain node
which is not necessary as `--contracts` can be seen as a "compile only" option.
This commit ensures we don't start any web3 services with `--contracts` is used.
As part of a bigger refactoring to make Embark's build pipeline pluggable,
this commit moves the watcher into its own plugin module so it can be
consumed via Embark's event bus.
It also introduces new command handlers for all watcher related APIs respectively:
- watcher:start
- watcher:stop
- watcher:restart
This is the first step of refactoring Embark's pipeline abstraction into
dedicated plugin modules that take advantage of Embark's event system.
With this commit we're moving `Pipeline` into `lib/modules/pipeline` and
introduce a new command handler `pipeline:build`. Embark's engine now
requests builds via this command handler.
Notice that `Watch` still lives in `lib/pipeline` as this is a step-by-step
refactoring to reduce chances of introducing regressions.
Iuri Matias
emizzle
Jonathan Rainville
Pascal Precht
Richard Ramos
Anthony Laibe
Subramanian Venkatesan
Giuseppe Bertone
Michael Bradley, Jr