Update `embark-pipeline` dependencies.
Add unit tests for `embark-pipeline`.
Update Embark testing framework to include assertion testsing for API calls and event actions.
feat(@embark/utils): add method to verify if a plugin is installed & configured
feature(@embark/utils): add method to verify if a plugin is installed & configured
feature: warn about packages that will be independent plugins and are not configured
chore: update templates to specify plugins
refactor: add to plugin api params so that blockchain plugins no longer need to be passed options
address changes in code review
remove unneded space
Update packages/core/utils/src/index.ts
Co-Authored-By: Jonathan Rainville <rainville.jonathan@gmail.com>
Update packages/core/utils/src/index.ts
Co-Authored-By: Michael Bradley <michaelsbradleyjr@gmail.com>
fix linting issue
add missing import
update dependency
fix plugins object
add missing whitespace
Turns out that 17cec1b787 has never worked as intended.
Custom provided `abiDefinition` values have been simply ignored. Embark always used the
`abiDefnition` that resulted from the Smart Contract compilation.
Adds back the watch on contract events and writes them to a file
with the same method as contract logs from transaction-logger, so
I extracted those methods to utils/file so that both could use the
same functions.
This was an issue experienced with ENS contracts not being deployed
because the configs used `strategy: explicit` and ENS configs are
not in configs.
To fix, I changed the way we check for deploy. If `deploy` is set as
`true` it should always deploy even if not in configs. It just means
they were added from a plugin (like ENS)
Also, I needed to set to `false` the `deploy` property of contracts
compiled that were not in config, because otherwise, they tried to
deploy, which goes against the strategy. This is because those
contracts get initiated as `deploy: true`.
Before, we added the gas for all receipts that came in because they
had a `gasUsed`, instead of adding the gas for receipts that came
with a transaction
For all instances where a `Contract` instance is serialized using `JSON.stringify`, the `logger` property was being stringified and written to logs and contract artifact files.
Add Serializer class that allows ignoring of class properties during serialization when using `JSON.stringify`.
NOTE: The `Serializer` relies on TypeScript’s decorators which are still listed as experimental (requiring the necessary compiler flag) despite being around for several years. Decorators are a stage 2 proposal for JavaScript.
Remove `bignumber.js` workaround (in the root, from PR #2152) because it's no
longer needed (verified locally).
Remove the `"skipLibCheck"` workaround (in `packages/plugins/solidity-tests`,
from PR #2152) because it's no longer needed (verified locally).
Refactor a typing in `packages/plugins/geth`. What's happening is that in web3
v1.2.4 `sendTransaction` has a return type of `PromiEvent<TransactionReceipt>`
but in v1.2.6 it has a return type of `PromiEvent<TransactionReceipt |
TransactionRevertInstructionError>`.
Compare:
* [v1.2.4/packages/web3-eth/types/index.d.ts#L291-L294](https://github.com/ethereum/web3.js/blob/v1.2.4/packages/web3-eth/types/index.d.ts#L291-L294)
* [v1.2.6/packages/web3-eth/types/index.d.ts#L295-L298](https://github.com/ethereum/web3.js/blob/v1.2.6/packages/web3-eth/types/index.d.ts#L295-L298)
The problem is that the `TransactionRevertInstructionError` type doesn't have a
`transactionHash` property. Since at present the code in
`packages/plugins/geth/src/devtxs.ts` only deals with the success case re:
`sendTransaction`, import the `TransactionReceipt` type from `web3-eth` and
cast the resolved return value's type using TypeScript's `as` operator.
This was caused by the fact that we add the ENS contract to the
manager when before they deploy, but the dependency resolution was
done while building the contracts, so even before.
So the solution was to add a "before build" action so that the ENS
module could add its contracts to the manager if needed.
generateAll was async, but it called the write functions with a sync
loop, so at the end of the function, the files were not written yet.
This is a problem in `embark build` because the process ends after
genrateAll is done, so no artifacts were written
## User reported error
i recently updated to embark 5.0 im having issues connecting to a local node each time i connect to it i get the following output from the embark console
```
Error during proxy setup: Port should be >= 0 and < 65536. Received 754510.. Use '--loglevel debug' for more detailed information.
```
This is what i have under the blockchain.js file
```
localDev: {
endpoint: "http://127.0.0.1:7545",
accounts: [{
nodeAccounts: true,
}]
}
```
### Problem
The port to start the proxy on is incremented by a constant value (using the `+` operator), however the port comes from the config and in the case where it is a string, the `+` operator acts as a string concatentation.
### Fix
Ensure the port from the config is always parsed to a number before attempting to add the constant proxy port offset.
Remove the `<12.0.0` restriction re: Node.js version in the `"engines"`
settings for all the packages in the monorepo that had that restriction.
Add missing `"engines"` settings in `packages/plugins/snark/package.json`.
Adjust the Azure Pipelines config to include builds for Node.js v12.x and
v13.x.
Bump `solc` to `0.4.26` in `dapps/tests/app` and `dapps/tests/contracts`. It
was discovered that older versions suffered a fatal `Maximum call stack size
exceeded` error when run on Windows with Node.js v12.x or newer. Display a
warning re: the bad combo (solc version + Windows + Node version) if it's
detected at runtime.
Adjust the root `yarn.lock` so that the `sha3` transitive dependency resolves
to a newer version that is compatible with Node v13.x.
Fixes the use of Infura to connect to the ENS contracts. When
connecting directly to Infura, it would throw with `rejected due to
project ID settings`, because it doesn't accept the VM as the domain
Instead, when passing from the proxy, it works. So I changed the
default when no dappConnection to ['$EMBARK']. I also added a
message when the error happens to help users fix it themselves
When in the testnet, we don,t register because we already have the
addresses, which is fine, but we also didn't populate the ensConfig
object which contains the important information about the addresses
and ABI.
There was a lot of lint problems in a couple of files so I cleaned
that up
This commit introduces a new `global.getEvmVersion()` that can be used to
conditionally run tests, such as when tests rely on RPC APIs that are only
available in specific evm nodes.
When running tests that expect the EVM to fail, Embark's proxy keeps logging
the VM errors to stdout making it look like tests weren't successful, while they
are actually passing.
Inside a test-runner it's probably expected not to see any error logs when the
tests in question expect errors.
This commit changes the proxy to use `debug()` logs instead of `error()`, making
it configurable through log verbosity how much errors are seen.
The problem was that putting the artifacts in src caused them to be watched
and thus creating an infinite loop, because a change in src triggers the build of the artifacts
This was fixed by ignoring the artifacts by chokidar, wherever they are
If using `embark-snark` in the dapp as a plugin, and the dapp didn’t have any contracts, the `embark-snark` plugin would not simply exit early and essentially do nothing.
Fix `embark-snark` plugin so that it will run its routine regardless of whether or not the dapp has contracts or not.
Bump `circom` and `snarkjs` to their latest patch version.
Refactor typings as necessary. In order for `bignumber.js` in the root
`node_modules` to not conflict with `@types/bignumber.js`, specify
`"bignumber.js": "5.0.0"` in `devDependencies` and `"embark-ui/bignumber.js"`
in `nohoist` of the root `package.json`.
In `packages/plugins/rpc-manager` switch from callback to promise usage with
respect to `web3.eth.accounts.signTransaction` because of a [bug][bug]
discovered in web3 v1.2.4.
In `packages/plugins/solidity-tests` specialize the tsc compiler options with
`"skipLibCheck": true` because of a problematic web3-related typing in the
`.d.ts` files of the remix-tests package.
Bump ganache-cli from 6.4.3 to 6.7.0 (latest) because 6.4.3 doesn't support
`eth_chainId` but web3 1.2.4 makes use of the `eth_chainId` RPC method (EIP
695).
BREAKING CHANGE: bump embark's minimum supported version of parity from
`>=2.0.0` to `>=2.2.1`. This is necessary since web3 1.2.4 makes use of the
`eth_chainId` RPC method (EIP 695) and that parity version is the earliest one
to implement it.
[bug]: https://github.com/ethereum/web3.js/issues/3283
This fixes a bug that redeployment of contracts has not been properly
reflected in the cockpit contract list.
The reason for that was that it's listening for a websocket API that
only updates when the `contractsState` event has been emitted.
The commit ensures `contractsState` is emitted during dedicated deployment
events.
This PR replaces #2057.
Implement a collective typecheck action that can be invoked in the root of the
monorepo with `yarn typecheck` or in watch-mode with `yarn watch:typecheck`.
Include the watch-mode typecheck action as part of `yarn start` (a.k.a
`yarn watch`).
To activate collective typecheck for a package in the monorepo, its
`package.json` file should specify:
```
{
"embark-collective": {
"typecheck": true
}
}
```
*-or-*
```
{
"embark-collective": {
"typecheck": {...}
}
}
```
Where `{...}` above is a `tsconfig.json` fragment that will be merged into the
config generated for the package according the same rules that `tsc` applies
when merging [configs][config].
When collective typecheck begins, it generates a `tsconfig.json` for the root
of the monorepo and for each package that is activated for the action. If the
generated JSON is different than what's on disk for the respective root/package
config, or if the config is not present on disk, then it will be
written. Changes to generated `tsconfig.json` files should be committed; such
changes will arise when there are structural changes to the monorepo, e.g. a
package is added, removed, moved and/or the directory containing it is
renamed. Since the configs are only generated at the beginning of collective
typecheck, when structural changes are made in the monorepo `yarn typecheck`
(or `yarn start` or `yarn watch:typecheck`) should be restarted.
Nearly all of the packages in the monorepo (i.e. all those for which it makes
sense) have been activated for collective typecheck. Even those packages that
don't contain `.ts` sources are activated because `tsc` can make better sense
of the code base as a whole owing to the project references included in the
generated `tsconfig.json` files. Also, owing to the fully cross-referenced
`tsconfig.json` files, it's possible for `tsc` to type check the whole code
base without babel (`yarn build` or `yarn watch:build`) having been run
beforehand.
**NOTE** that a *"cold typecheck"* of the whole monorepo is resource intensive:
on this author's 2019 MacBook Pro it takes around three minutes, the fans spin
up, and `tsc` uses nearly 0.5 GB of RAM. However, once a full typecheck has
completed, the next full typecheck will complete in a few seconds or less; and
when running in watch-mode there is likewise a *big* speedup once a full
typecheck has completed, whether that full check happened before it's running
in watch-mode or when watch-mode itself resulted in a full check before
switching automatically to incremental check, as well a corresponding *big*
reduction in resource consumption. A full check will be needed any time
`yarn typecheck` (or `yarn start` or `yarn watch:typecheck`) is run in a fresh
clone plus `yarn install`, or after doing `yarn reboot[:full]` or `yarn reset`.
The combination of options in each generated package-level `tsconfig.json` and
the root `tsconfig.base.json` result in `tsc` writing `.d.ts` files (TypeScript
declaration files) into the `dist/` directory of each package. That
output is intended to live side-by-side with babel's output, and therefore the
`"rootDir"` option in each generated config is set to `"./src"`.
In projects activated for collective typecheck, `.js` may be converted to `.ts`
and/or `.ts` sources may be added without any additional changes needed in
package-level `package.json`.
---
Reorganize types in `packages/core/typings` (a.k.a `@types/embark`) into
`packages/core/core` (`embark-core`), refactor other packages' imports
accordingly, and delete `packages/core/typings` from the monorepo. This results
in some similarly named but incompatible types exported from `embark-core`
(e.g. `Events` and `EmbarkEvents`, the latter being the one from
`packages/core/typings`); future refactoring should consolidate those types. To
avoid circular dependency relationships it's also necessary to split out
`Engine` from `embark-core` into its own package (`embark-engine`) and to
introduce a bit of duplication, e.g. the `Maybe` type that's now defined in
both `embark-i18n` and `embark-core`.
In the process of the types reorg, move many dependencies spec'd in various
`package.json` to the `package.json` of the package/s that actually depend on
them, e.g. many are moved from `packages/embark/package.json` to
`packages/core/engine/package.json`. Related to those moves, fix some Node.js
`require`-logic related to bug-prone dependency resolution.
Fix all type errors that appeared as a result of activating collective
typecheck across the whole monorepo.
Reactivate `tslint` in `packages/core/core` and fix the remaining linter errors.
Tidy up and add a few items in the root `package.json` scripts.
Bump lerna from `3.16.4` to `3.19.0`.
Bumpt typescript from `3.6.3` to `3.7.2`.
Bumpt tslint from `5.16.0` to `5.20.1`.
Make various changes related to packages' `import`/`require`ing packages that
weren't spec'd in their respective `package.json`. More refactoring is needed
in this regard, but changes were made as the problems were observed in the
process of authoring this PR.
[config]: https://www.typescriptlang.org/docs/handbook/tsconfig-json.html
It's a relevant `package.json` if the package has a test suite. If `test` isn't
specified then the tests won't run in CI (they have't been running).
Include a fix for one failing test in `packages/plugins/ens` and two failing
test in `packages/stack/deployment`. In all three cases, the mock embark
object's config needed a `blockchainConfig` section with `{ enabled: true }`.
Tests were exiting early and Mocha was reporting an exit code of 0. This allowed CI to complete as if the tests were successful.
Change `compiler:contracts` event request to `compiler:contracts:compile` and update documentation. Because the `compiler:contracts` event didn’t exist, this test was silently failing.
Update the `TestEvents` mock object to allow passing of parameters to an event that is not a callback.
Add unit tests to test for contracts loaded via node_modules.
1. This PR contains a change where any contracts loaded from node_modules will be imported in to `.embark/contracts/node_modules`. Previously, these contracts were loaded in to `.embark/node_modules` (with the `/contracts`).
Re-enable debugger for cockpit and console.
Add missing request `blockchain:getTransaction` required to start a debugging session.
Fix error in console when stopping a debug session.
Users can turn of blockchain support if they want to using the blockchain.js
configuration. In practice however, this has never properly worked as several
places in Embark's codebase weren't actually honoring that configuration value.
This commit introduces the necessary changes so that disabling blockchain support will:
No longer generate blockchain related EmbarkJS artifacts
No longer try to deploy Smart Contracts but still compile them
Improve the reliability of the expiration unit test in the test dapp by explicitly setting the `block.timestamp` and comparing an expiration value against that.
This improves on the current implementation that relies on time passed which varies depending on the speed of unit tests run (CPU speed, logs shown, etc).
When `eth_unsubscribe` is received in the proxy, ensure this request is forwarded through on the correct socket (the same socket that was used for the corresponding `eth_subscribe`).
Move subscription handling for `eth_subscribe` and `eth_unsubscribe` to RpcModifiers (in `rpc-manager` package).
For each `eth_subscribe` request, a new `RequestManager` is created. Since the endpoint property on the proxy class was updated to be a provider, the same provider was being assigned to each new `RequestManager` and thus creating multiple event handlers for each subscription created. To circumvent this, we are now creating a new provider for each `RequestManager`.
Co-authored-by: Pascal Precht <pascal.precht@googlemail.com>
This PR adds support for EIP-712 (https://github.com/ethereum/EIPs/blob/master/EIPS/eip-712.md), by allowing the signing of typed data transactions using the `eth_signTypeData_v3` or `eth_signTypedData` request.
Add a module called `embark-rpc-manager` to handle all RPC transaction modifications.
The internal events management module extracted to `embark-core`. This allows other modules (ie plugins) to use a private instance of Embark’s event API.
Remove transaction modifications in `embark-accounts-manager` and leave only the functionality in this module that handle account management (ie funding of accounts). Other functionality was moved to `embark-rpc-manager`.
- Transactions should now reflect when a new node account is added via `personal_newAccount` or via the node itself (and once `eth_accounts` is requested).
- In the proxy, errors are handled for all cases and are now JSON-RPC spec-compliant.
- Always register `eth_signTypedData` RPC response and display error message if account cannot be found.
NOTE: Updated yarn.lock due to conflict after rebase.
Avoid race condition with rpc modifications
Refactor proxy to make request handling asynchronous, and to allow circumvention of forwarding requests to the node in cases where it is not needed.
Move loading of plugins to after core and stack components (in cmd_controller).
We should load custom plugins LAST (ie after all core and stack components, and after all Embark plugins). To do this, we disable automatic loading of plugins in the config, and allow them to be loading explicitly in cmd_controller.
Create a system that allows registration of custom plugins. Each plugin is generated in EmbarkJS by instantiating the registered class. For example, we can register a Plasma plugin and it will be generated in to EmbarkJS.Plasma with an instantiation of `embarkjs-plasma`. The DApp itself can then do what it wants with `EmbarkJS.Plasma` (ie call `EmbarkJS.Plasma.init()`.
NOTE: loading of custom plugins needs to be applied to all other commands in `cmd_controller`. This PR only loads custom plugins for the `run` command.
NOTE: This PR is based on branch fix/ws-eth-subscribe (#1850). I will rebase this PR when (and if) #1850 is merged.
The blockchain module should not contain any ethereum-specific code, and currently it contains a check to see if the blockchain has already been started.
This PR moves this check to blockchain client (ie geth or parity). This check function is registered along with the started callback.
style(@embark/stack/blockchain): add missing semicolon
Fix the proxy’s handling of WebSocket connections when subscribing to contract events and node data using the `eth_subscribe` RPC request.
Previously, the client connection that the subscription data was sent to was often in a closed state. It was determined that this connection was the wrong connection to forward the data in the first place. The connection was in fact generally the connection created for the Ethereum service check which was then (correctly and subsequently) closed after it had finished its operation.
The flow of a proxy request handling a WebSocket “eth_subscribe” RPC request is now as follows:
1. A WebSocket RPC request `”eth_subscribe”` is sent from a client to the proxy.
2. Proxy forwards the request to the node by way of a new instance of `RequestManager`.
3. When the node receives an event matching the subscription, it sends the event data back to same socket connection it received the request on (ie the specific instance of `RequestManager`).
4. The `RequestManager` fires the `”data”` event containing the subscription data, and this event is picked up in the proxy.
5. The proxy then forwards the subscription data on to the originating WS client connection.
All other requests (ie non-WS or WS RPC requests that are not `eth_subscribe`) will be serviced to/from the node using a single `RequestManager` instance.
Co-authored-by: Pascal Precht <pascal.precht@gmail.com>
* Revert "fix(@embark/core): set loglevel back to info"
This reverts commit a03ffd56e5.
* Revert "fix(@embark/proxy): Fix contract event subscriptions"
This reverts commit 173d53de2f.
Fix the proxy’s handling of WebSocket connections when subscribing to contract events and node data using the `eth_subscribe` RPC request.
Previously, the client connection that the subscription data was sent to was often in a closed state. It was determined that this connection was the wrong connection to forward the data in the first place. The connection was in fact generally the connection created for the Ethereum service check which was then (correctly and subsequently) closed after it had finished its operation.
The flow of a proxy request handling a WebSocket “eth_subscribe” RPC request is now as follows:
1. A WebSocket RPC request `”eth_subscribe”` is sent from a client to the proxy.
2. Proxy forwards the request to the node by way of a new instance of `RequestManager`.
3. When the node receives an event matching the subscription, it sends the event data back to same socket connection it received the request on (ie the specific instance of `RequestManager`).
4. The `RequestManager` fires the `”data”` event containing the subscription data, and this event is picked up in the proxy.
5. The proxy then forwards the subscription data on to the originating WS client connection.
All other requests (ie non-WS or WS RPC requests that are not `eth_subscribe`) will be serviced to/from the node using a single `RequestManager` instance.
Co-authored-by: Pascal Precht <pascal.precht@gmail.com>
When making use of the `useBuiltIns: 'usage'` option for @babel/preset-env
(which is the case for all transpiled packages in Embark's monorepo) a package
needs to have both @babel/runtime-corejs3 and core-js@3 specified as
dependencies.
* build(deps): move @types/os-locale from packages/embark to packages/core/i18n
* build(deps) move @types/globule from packages/embark to packages/plugins/coverage
* build(deps): refactor stack/{api,proxy,webserver} deps relative to packages/embark
* build(deps): remove unneeded @types/async dep from packages/stack/test-runner
* build(deps): remove unneeded deps from packages/embark
Introduce some light refactoring related to embark-testing facilities, and also
a configurable stdout option so the output of the reporter implemented in this
package isn't confusingly mixed into unit test reporting for this package.
BREAKING CHANGE:
node: >=10.17.0 <12.0.0
npm: >=6.11.3
yarn: >=1.19.1
node v10.17.0 is the latest in the 10.x series and is still in the Active LTS
lifecycle. Embark is still not compatible with node's 12.x and 13.x
series (because of some dependencies), otherwise it would probably make sense
to bump our minimum supported node version all the way to the most recent 12.x
release.
npm v6.11.3 is the version that's bundled with node v10.17.0.
yarn v1.19.1 is the most recent version as of the time node v10.17.0 was
released.
* refactor(@embark/dapps/tests/app): use function syntax
These changes don't fix the race conditions related to the test dapp's tests
but are a step in the right direction.
* refactor(@embark/dapps/tests/contracts): adjustments to get tests passing
Further refactoring is needed re: potentially duplicated or overlapping logic
in `packages/plugins/solidity-tests` and
`packages/core/utils/src/solidity/remapImports.ts`, as well in disabled test
dapp tests
* test(dapps/tests/app): temporarily disable intermittently failing tests
They are failing because of a race condition; once that race condition has been
fixed these tests should be reenabled.
* fix(@embark/solidity-tests): fix importing the library for the tests
* build(@embark/stack/blockchain-client): remove unneeded typescript related scripts and deps
In addition to being unneeded their presence is causing build errors.
* build(embark): remove unneeded typescript related scripts and deps
In addition to being unneeded their presence is causing build errors.
* build(@embark/plugins/ethereum-blockchain-client): remove unneeded typescript related scripts, deps
In addition to being unneeded their presence is causing build errors.
* build(@embark/plugins/ganache): remove unneeded typescript related scripts and deps
In addition to being unneeded their presence is causing build errors.
* build(@embark/plugins/geth): remove unneeded typescript related scripts and deps
In addition to being unneeded their presence is causing build errors.
* fix(@embark/plugins/transaction-logger): require 'web3' not 'Web3'
* fix(@embark/utils/solo): spawn npm(.cmd) instead of npx(.cmd)
* test(@embark/plugins/basic-pipeline): add test stub
* test(@embark/stack/blockchain): add test stub
Fix a lot of bugs and reenable a couple of modules
Some tests were kept disabled, mostly the ENS and EmbarkJS tests
Those need to add back a fairly significant feature to work
Add back missing solidity contracts
In d6bf5c24b9 we've ensured that certain modules of
embark only executed if their functionality is actually enabled.
This broke one of our tests in the communication module.
This commit fixes the test by explicitly enabling the module's functionality.
* fix: fix tests hanging because the console is not started
* fix(@embark/proxy): send back errors correctly to the client
Code originally by @emizzle and fixed by me
* feat(@embark/test-runner): add assert.reverts to test reverts
* fix: make test app actually run its test and not hang
* fix(@embark/proxy): fix listening to contract event in the proxy
* feat(@embark/test-runner): add assertion for events being triggered
* docs(@embark/site): add docs for the new assert functions
* feat(@embark/test-runner): add increaseTime util function to globals
* docs(@embark/site): add docs for increaseTime
emizzle
Iuri Matias
Pascal Precht
EmbarkBot
Jonathan Rainville
Michael Bradley, Jr
dependabot-preview[bot]