Default behavior of logging to file is no longer needed now that Embark log history can be properly served using the `ProcessLogsApi` and `LogHandler` classes.
# Conflicts:
# lib/core/logger.js
# lib/modules/blockchain_process/blockchain.js
There are three separate instances of process log APIs: embark logs, blockchain logs (when in standalone mode), and child process logs (storage, communication, blockchain, etc). Each one was repeating the implementation of creating a process log API endpoint. This commit centralises the API declaration by using the class `ProcessLogsApi`.
`ProcessLogsApi` is started for all three components mentioned above: blockchain (in standalone) in the `BlockchainListener` module, embark in the `EmbarkListener` module, and for all child processes in the `ProcessLauncher`.
These listeners have two functions:
1. Create the process logs API endpoints for `get` and `ws`, and
2. Ensure that all logs are logged through the `LogHandler`, which normalises the output of the log and ensures each log has a timestamp and id (used in the cockpit for log ordering).
Also, this commit moved the pipeline in to a module, so that the `embark` object could be passed to the `ProcessLogsApi` (to be used for registering API endpoints).
When running `embark blockchain` followed by `embark run` previously, logs generated in the standalone `embark blockchain` process were black boxed and not accessible to the main Embark process.
This is fixed by creating a client IPC connection in the `embark blockchain` process that connects to the IPC server connection running in `embark run`. The connection is made by way of polling `ipc.connect` and continues polling even after a connection is made in case `embark run` is killed and restarted without restarting `embark blockchain`.
`LogHandler` was introduced to extrapolate functionality used in `ProcessLauncher` that needed to also be used in the standalone blockchain process. It also caps the number of logs that are stored in memory per process by a constant value defined in `constants.json`.
A `blockchain_listener` was module was created (and run inside of `embark run`) that listens for logs emitted by the `embark blockchain` client IPC and runs them through the `LogHandler`. Additionally, this module registers the API endpoints needed to handle requests for blockchain process logs in the cockpit (which were 404’ing before).
# Conflicts:
# lib/modules/blockchain_process/blockchain.js
display last line on tx
fix debugger call
listen to source event only after jumping to the end
keep track of last tx; add minimal debug feature; fix ast issue
initial debugger apis & ui integration
prevent crash when step is out of bounds; send all all available data in websocket
add debugger commands
fix line number tracking in editor; toggle breakpoints
replace timeouts with callbacks
add debugger manager & refactor
refactor debugger api
refactor cmd line debugger
reduce debugger decoupling
reduce debugger decoupling
fix debug buttons
trigger source update so api triggers ws event to update source location
move locals and contracts vars to a json view
improve debugger icons
simplify debugger data
update debug package
add command handler to get a contract given a tx; update debugger so it can get a contract by its tx instead of tracking latest txs only
update debugger package
So far, `ProcessManager` was able to only register a `process:launch` handler.
There was no way to tell `ProcessManager` how to stop processes. This hasn't
been a problem so far as most of the service processes can be started without the usage
of the `ProcessManager`, but turns out to be necessary if we want Embark UI to be able
to pick up running services.
A good example is the webserver process, which until now bypasses the `ProcessManager`
all together. The webserver sets up two event handlers to start and stop it respectively:
```
this.events.setCommandHandler('start-webserver', () => this.server.start());
this.events.setCommandHandler('stop-webserver', () => this.server.stop());
```
In the future, this should happen through the `ProcessManager` instead, so the webserver
process can be picked up by Embark UI, like this:
```
this.request('process:register', 'webserver', () => {
this.server.start();
});
// and then
this.request('process:launch', 'webserver', () => {
// server started
});
```
Notice that the given callback to registering a process is actually the function that
gets called to launch the process.
Having that in mind, and considering that we also need a way to stop the process through
`ProcessManager, so we don't introduce a regression, we need a way to register a stop
call back as well.
The new API introduced in this commit looks like this:
```
this.request('process:register', 'webserver', {
launchFn: (callback) => { this.server.start(callback) },
stopFn: (callback) => this.server.stop(callback) }
});
// and then
this.request('process:launch', 'webserver', (err, message, port) => {
// server started
});
this.request('process:stop', 'webserver', err => {
// server stopped
});
```
Notice that `process:register` works exactly the same way as before as well.
Another thing to notice is that all parameters emitted by the underlying process
are propagated to the outside caller, which is why `err`, `message` and `port` are
available inside the launch callback.
After the code was rebased, there were some additional changes for getting websockets logs that needed to be catered for.
When there is a call to get all logs for a process, the state entity is updated with a new array item containing all the logs (this is then reduced and selected for rendering). In the case of a websocket log that simply returns only one log item, the latest full log for the process is found in the state entities, and it’s logs are appending to with the data from the websocket.
Additionally, log limits were updated to be passed in as a parameter to the API calls from the frontend. Parameter validation (for `limit`) was also added in this commit.
*Console.js*
- Moved `DEFAULT_PROCESS` const to outside of the `Console` class (but inside the module).
- Removed `(` and `)` from `.filter` in `getProcessLogs()`.
- Updated comments
*logger.js*
- Moved `dateFormat` and `logRegex` to constants outside of `Logger` class
- Moved the `parseLogFile` method inside of the `Logger` class (ES6 style)
- Added a log limit to the `parseLogFile` method
- Added the log path to the constants file and used inside of `Logger`
*cmd_controller.js*
- Defaulted `this.context` to `[constants.context.any]` in the constructor.
- Changed `’embark’` to split modules`coreProcess` and `loggerApi`.
*engine.js*
- Changed `’embark’` to split modules`coreProcess` and `loggerApi`.
The embark log file is being overwritten each time embark is run. There is a separate log file for each context, so that running, for example, `embark run` then `embark console` doesn’t get the `run` log overwritten with the `console` log.
- Created an “embark” module so that an “embark” process could be registered in the correct way. This service is only used on `embark run` (can be extended to other commands if needed).
- extracted “embark” to a const `DEFAULT_PROCESS` param in the `Console` component.
- extracted commands result rendering to it’s own function to keep the `renderTabs` function from getting cluttered
- Added sorting of logs by timestamp
- Added milliseconds to the log file data (which helps in sorting log messages).
A logfile is now generated by default, in the format `.embark/embark-log__YYYY-MM-DD_HH-mm-ss.log`.
When the home tab is loaded, the process logs are fetched for all the processes. The list of processes returned now includes `embark`, and when `/embark-api/process-logs/embark` is fetched, the logFile is parsed and an array of log messages are returned.
- Add no cache via helmet
- Fix linting (no-return-else)
- Rebase Fix: Use option.name for process log to avoid endpoint being called
blockchainProcess.js
- Rebase Fix: use option when compiling solidity
Jonathan Rainville
andrea zuccarini
André Medeiros
Iuri Matias
emizzle
Pascal Precht
Anthony Laibe
Richard Ramos