Commit Graph

6027 Commits

Author SHA1 Message Date
emizzle 0d72ebeda8 fix(@embark/storage): Fix storage not connecting error
In addition to `EmbarkJS.Storage` not being available in the console (ie `await EmbarkJS.Storage.isAvailable() alway returned false), the error `Could not connect to a storage provider using any of the dappConnections in the storage config` would always appear in the console. The storage operations in the dapps were working OK.

The fix to this was three-fold:
1) Wait for the ipfs process to be started before attempting to run the `EmbarkJS.Storage.registerProvider/setProvider` in the console.
2) Wait for `EmbarkJS.Storage.registerProvider` to be called before `EmbarkJS.Storage.setProvider`. This was actually handled in the previous PR.
3) Remove any async operations from the `setProviders` method in the storage module. This was causing `callback is not defined` errors which were being swallowed and masqueraded as an unsuccessful attempt to connect to a `dappConnection` or `upload` config.
2019-02-22 10:38:50 +11:00
emizzle d378ccf150 feat(@embark/core): Auto generate EmbarkJS events
For every provider registered and set with EmbarkJS, auto generate events to inform other modules of the state of the provider.

Any provider that belongs to EmbarkJS (currently Blockchain, Names, Messages, and Storage) will have both a `runcode:<provider name>:providerRegistered` and a `runcode:<provider name>:providerSet` event created automatically when the `CodeRunner` is instantiated.

Once the `registerProvider` code is run through the VM, ie `EmbarkJS.Blockchain.registerProvider(…)`, the corresponding event will be fired, ie `runcode:blockchain:providerRegistered`.

Likewise, once the `setProvider` code is run through the VM, ie `EmbarkJS.Blockchain.setProvider(…)`, the corresponding event will be fired, ie `runcode:blockchain:providerSet`.

Additional updates/fixes with this PR:
* Move `CodeRunner` to TypeScript
* Fix console errors with ENS and Storage due to a recent PR that waits for `code-generator:ready`. The solution here was to ensure that `code-generator:ready` is requested, so that premature events can be handled.
2019-02-22 10:38:50 +11:00
emizzle 58ea3d9c30 fix(@embark/demo): Fix demo
Fix demo not running, by fixing the following issues:

This was due to the removal of `Web3` from the VM initialisation in a recent PR. The reasoning behind the removal was so that the modules that need Web3 could require it as needed, however, it is needed in 3 modules (ENS, Whisper, and web3Connector), all of which would define `const Web = …` in the code generated EmbarkJS, thus causing issues in itself. This needs to be rethought prior to removing it from the VM sandbox.

Because Web3 is back to being required in the global sandbox of the VM, it does not need to be registered in the `web3Connector` module, hence the removal of the `runcode:register`.

Module not found: Error: Can't resolve 'embarkArtifacts/contracts/SimpleStorage.js' in '/Users/emizzle/temp/embark_demo/app/components’`
This was fixed by replacing
```
import SimpleStorage from 'Embark/contracts/SimpleStorage';
```
with
```
import SimpleStorage from '../../embarkArtifacts/contracts/SimpleStorage';
```

revert changes changing __Web3 to Web3
2019-02-22 10:38:50 +11:00
Jonathan Rainville 5e9955e828 fix(coverage): fix coverage regex on Windows 2019-02-21 13:38:42 -05:00
Jonathan Rainville 84d4f3a252 fix(ens): fix ens transactions getting stuck
Only happened during tests connected to a WS node somehow
2019-02-21 13:38:42 -05:00
Jonathan Rainville e64ed36092 fix(test): fix tests getting stuck when embark is run next to it 2019-02-21 13:38:42 -05:00
Jonathan Rainville d33425fb4c fixes for the test_dapp to work 2019-02-21 12:00:10 -05:00
Jonathan Rainville e049b459c8 chore(templates): add web3connector plugin to templates 2019-02-21 12:00:10 -05:00
Andre Medeiros 1334900eae fix: log command in console 2019-02-21 11:59:59 -05:00
Jonathan Rainville ebcc3c4a8d feat: enable embark to be run with an external pipeline 2019-02-20 15:06:12 -05:00
Andre Medeiros c1d08c667a fix: specify full path for password files (geth) 2019-02-20 14:16:35 +01:00
Anthony Laibe e6eea1df06 fix(vyper): file path 2019-02-20 14:15:41 +01:00
Andre Medeiros 271a219c8b fix: template dependency errors don't exit 2019-02-20 14:15:03 +01:00
emizzle fc823bb7eb
fix(@embark/console): Fix console not working with VM2/monorepo
The console was not working correctly with the latest VM2/monorepo updates. This PR addresses namely fixes this problem, but also adds a few more notable changes:

* SIGNIFICANT improvement in loading time for `embark console` with an already running `embark run`. This is due to removing unneeded services starting, and instead forwarding user input to the main `embark run` process.
* All user input commands are now forwarded to the `embark run` process via IPC insteaad of evaluating the command in the `embark console` process.
* Removed IPC console history as it's no longer needed due to the above. Side effects:
  ** The signature of the `runcode:eval` and `runcode:register` events was changed to remove the `toRecord` parameter.
  ** Old `runcode:eval` signature: `events.request("runcode:eval", "code to be evaluated", (err, result) => {}, isNotUserInput, tolerateError)`
  ** New `runcode:eval` signature: `events.request("runcode:eval", "code to be evaluated", (err, result) => {}, tolerateError)`
  ** Old `runcode:register` signature: `events.request("runcode:register", "varName", variableValue, toRecord, (err, result) => {})`
  ** New `runcode:register` signature: `events.request("runcode:register", "varName", variableValue, (err, result) => {})`

* Removed unneeded `forceRegister` flag.
* Removed the `VM.getWeb3Config` method as it's no longer being used (EmbarkJS contracts are pulled out from the VM instead).
* Updated `web3Connector` `blockchain:connector:ready` to allow for event requests or event emissions.
* In the tests, removed the initial `initWeb3Provider` in the `init` as it was being called twice.
* In the tests, removed the `web3Connector` check message as the tests are now using the Console, and the console does this check. This was causing duplicate messages to be displayed in the output.
* Fix `web3 is not defined` browser error
2019-02-20 14:12:30 +01:00
emizzle 84ca98f962 feat(@embark/core): Add events oracle
Add the ability to request an event before it’s command handler has been set. Also, add the ability to emit an event before a listener has been set.

The most useful use case for this is to allow modules/plugins to load asynchronously and without need to know their load order beforehand. This let’s us request events in advance of having the event command handler set by the module/plugin.

BEFORE:
```
this.events.request(“event:name”, () => {
  // never fired
});
this.events.setCommandHandler(“event:name”, cb);
```

AFTER:
```
this.events.request(“event:name”, () => {
  // YAY it fires!
});
this.events.setCommandHandler(“event:name”, cb);
```
2019-02-20 13:05:08 +01:00
emizzle e943d03ce0 fix(@embark/cockpit): Fix contract method autosuggest
Prior to this PR, typing `SimpleStorage.methods.` in to the cockpit console would not show autosuggestions for the contract methods.

Contract names are case sensitive in the VM (VM2). Update the `Suggestions.getSuggestions` method to be case-sensitive when evaluating `Object.keys(<contract class name>.methods)`.
2019-02-15 13:22:57 -05:00
emizzle 876eee5354 fix(@embark/embarkjs): Fix potential race condition
`EmbarkJS.Blockchain.connectConsole` has a potential race condition with both `cb` and `doneCb` firing at the end of the method.

This PR is an attempt to fix that by first awaiting the `cb`, then finally calling `doneCb`, as suggested by @michaelsbradleyjr in https://github.com/embark-framework/embark/pull/1319#discussion_r256850820
2019-02-15 12:43:36 -05:00
Jonathan Rainville 9ccc453379 fix: use our global Web3 and not Metamask's 2019-02-14 12:17:16 -05:00
emizzle 129a35484f @fix(embark/core): Fix VM2 + Remap Imports + Monorepo + Tests
This PR introduces a large number of changes (for that, I am very sorry). Breaking this up in to smaller PR's was causing tests to fail, and the likelihood of them getting merged diminished. There a couple of PRs this PR closes, and as such, I have kep the original commits to preserve the history. The first two (of three) commits are the close PRs, and the last is the glue bringin it all together.

The main goal of this PR is to fix the fragility of the tests with EmbarkJS, however in doing so, a number of recent features have been updated:
Remapping of imports still had a few edge cases that needed to be ironed out, as well as have unit tests created for them. More details of the changes an be seen in the closed PR (below).
The main issue with VM2 was running the code generated EmbarkJS code inside the VM, and getting EmbarkJS contracts out of the VM and available to the tests. This fixed issues where ENS may not have been available to the tests. Notable additions include adding `EmbarkJS.Blockchain.connectTests` to the tests lifecycle, and ensuring `EmbarkJS` is only every required in the console module and not used or passed around elsewhere.
As mentioned above, the main issue with the tests in the context of a monorepo and with embark running as module inside the dapp’s `node_modules`, there were issues getting the correct contract state available inside of the tests. For some reason, this particular case was causing the tests to fail, with ENS never being available (assuming this to be an issue with `EmbarkJS.Blockchain.connect()` never being called). The main fix for this came with passing `web3` as an option in to `EmbarkJS.Blockchain.setProvider()`.
---
1. https://github.com/embark-framework/embark/pull/1286
2. https://github.com/embark-framework/embark/pull/1275
Go to bottom for details
---
There are few known issues with this PR. Instead of trying to fix all of them with this PR, I was hoping to get these issues tackled in separate PRs.
1. `deployIf` directive on contracts defined in the config are not working, however the tests are passing. The issue is that if `ContractA` has a `deployIf` set to `!!ContractB.options.address`, it means that it will not deploy if `ContractB` is not deployed. However, it appears that `ContractA` is attempted to be deployed prior to `ContractB` being deployed, and therefore `ContractA` fails to deploy. Instead, because `ContractA` depends on `ContractB`, `ContractB` should be deployed before `ContractA`.
2. `embark test --node embark` does not seem to be functioning for reasons unknown.
3. Remix tests: Currently there is support for adding contract tests that get process by `remix-tests`, however, there is an error that I believe is not due to embark, but due to the Assert library. For example, if we add a `test/remix_test.sol` to the `test_app` with the following content:
```
pragma solidity ^0.4.24;

import "remix_tests.sol";
import "../app/contracts/simple_storage.sol";

contract SimpleStorageTest {
  SimpleStorage simpleStorage;

  function beforeAll() public {
    simpleStorage = new SimpleStorage(100);
  }

  function initialValueShouldBeCorrect() public {
    return Assert.equal(
      100,
      simpleStorage.storedData,
      "stored data is not what I expected"
    );
  }
}
```
After compilation, we would get the error:
```
remix_test.sol:14:12: TypeError: Member "equal" not found or not visible after argument-dependent lookup in type(library Assert)
    return Assert.equal(
           ^—————^
```
---
This branch is based off of ()`refactor/embarkjs-in-monorepo`)[https://github.com/embark-framework/embark/tree/refactor/embarkjs-in-monorepo], which does not have passing tests due to the `EmbarkJS` ENS issue mentioned above. However, you should (hopefully) see the tests passing in this branch, meaning if both branches are merged, the tests should be passing.
Related PRs: https://github.com/embark-framework/embark-solc/pull/24

---

Changes include:
1. Add unit tests for recursively remapping imports
2. Handle plugin contracts correctly
3. Allow `prepareForCompilation` to be called from `File`, allowing for external compilers, like `embark-solc` to call this function before compilation.
4. Add flattened remappings to `File` that gets prepared for compilation (ie has it's imports remapped)
5. Return remapped contract content when file type is http. Previously this was broken, as always freshly downloaded (original) content was returned.
6. Handle additional cases for `custom` and http file types.

This PR was tested with:
- `embark` unit tests
- `embark` test_app
- `embark` test_app with `embark-solc` plugin
- `embark` test_app with `embark-flattener` plugin
- `Giveth/lpp-campaign`

Related change to get `embark-solc` up-to-date` with these changes: https://github.com/embark-framework/embark-solc/pull/24

When embark was running as module inside the dapp’s `node_modules`, the tests were failing due to several issues:
1. `web3` was not being set in the global namespace of vm2. `EmbarkJS.Blockchain.setProvider` was therefore failing because it relies on `global.web3` to be set. I guess somehow this works when the test app was running in a child tree of the executing program. maybe this is a security feature of vm2, but i’m not sure.
2. `embarkjs` provider code being injected to the vm twice. This really was the initial point of failure, as this piece of code is requiring embarkjs, so i’m assuming, but again not sure, that maybe it was getting a second instance of `EmbarkJS` which did not have it’s providers set up correctly (hence the error with `ENS provider not set`).

Fixes for those issues in this PR:
1. To circumvent the web3 issue, we are now setting `global.web3` for tests only (the global web3 accessible in the tests), and `web3` is now explicitly passed in to `EmbarkJS.Blockchain.setProvider`
2. To fix the `embarkjs` code being called twice, we are not re-injecting this code to the VM during test initialisations
2019-02-14 12:16:55 -05:00
emizzle 2f354c9323 fix(@embark/solc): Import remapping update
Changes include:
- Add unit tests for recursively remapping imports
- Handle plugin contracts
- Allow `prepareForCompilation` to be called from `File`. Allows external compilers, like `embark-solc` to call this.
- Add flattened remappings to file getting prepared for compilation
- Return remapped contract content when file type is http
- Add check allowing files to be remapped/prepared multiple times
2019-02-14 12:16:55 -05:00
emizzle eb4da28a61 fix(@embark/core): Fix tests for shim/monorepo
When embark was running as module inside the dapp’s `node_modules`, the tests were failing due to several issues:
1. `web3` was not being set in the global namespace of vm2. `EmbarkJS.Blockchain.setProvider` was therefore failing because it relies on `global.web3` to be set. I guess somehow this works when the test app was running in a child tree of the executing program. maybe this is a security feature of vm2, but i’m not sure.
2. `embarkjs` provider code being injected to the vm twice. This really was the initial point of failure, as this piece of code is requiring embarkjs, so i’m assuming, but again not sure, that maybe it was getting a second instance of `EmbarkJS` which did not have it’s providers set up correctly (hence the error with `ENS provider not set`).

Fixes for those issues in this PR:
1. To circumvent the web3 issue, we are now setting `global.web3` for tests only (the global web3 accessible in the tests), and `web3` is now explicitly passed in to `EmbarkJS.Blockchain.setProvider`
2. To fix the `embarkjs` code being called twice, we are not re-injecting this code to the VM during test initialisations
2019-02-14 12:16:55 -05:00
Michael Bradley, Jr 756a916428 build: specify a high network timeout in root .yarnc
Yarn can be sensitive to timeouts when downloading large-ish packages that
aren't in its cache, e.g. `@reactivex/rxjs`. Specify a high-timeout in the root
`.yarnrc` to workaround `yarn install` hanging indefinitely and reporting
network connection errors, which seems to be experienced more frequently on
Windows than on Linux or macOS. Note: this is probably buggy behavior on yarn's
part.
2019-02-14 01:26:22 -06:00
Michael Bradley, Jr 57445d0819 build(embarkjs): clean script should invoke 'npx rimraf'
Packages' `"clean"` scripts should run regardless of whether `node_modules` is
in place at the root and/or package levels, but the `embarkjs` package was
missing a needed `npx` in front of `rimraf` in its script.

Also, bump its devDeps version of rimraf to be in line with the rest of
`packages/*` and `test_dapps/*`.
2019-02-14 01:26:10 -06:00
Michael Bradley, Jr 23ae78a6d6 refactor: be consistent with callbacks and promises
If a function receives a callback argument then it should not return a promise
if the caller's callback will be invoked. Both invoking a callback and
returning a promise can lead to at best confusion (in code review and at
runtime) and at worst non-deterministic behavior, such as race
conditions. Also, a caller supplying a callback may not handle a returned
promise, leading to unhandled rejection errors.

Refactor all readily identified functions where a callback argument can be
supplied but the function returns a promise regardless. Make use of
`callbackify` and `promisify` where it made sense to do so during the
refactoring. Some callsites of the revised functions may have been accidentally
overlooked and still need to be updated. Some functions that take callback
arguments may execute them synchronously, at odds with control flow of a
returned promise (if a callback wasn't supplied). Such cases should be
identified and fixed so that asynchronous behavior is fully consistent whether
the caller supplies a callback or receives a promise.

Make sure promises that pass control flow to a callback ignore rejections,
since those should be handled by the callback.

Don't return promise instances unnecessarily from async functions (since they
always return promises) and change some functions that return promises to async
functions (where it's simple to do so).

Whisper was using an ad hoc promise-like `messageEvents` object. However, that
object behaved more like an observable, since promises either resolve or
reject, and only do so one time. `messageEvents` was also intertwined with
callbacks. Replace `messageEvents` with RxJS Observable. `listenTo` now returns
Observable instances and callers can subscribe to them.

`Blockchain.connect` of embarkjs could suffer from a race condition where tasks
associated with `execWhenReady` might be ongoing when `connect`'s returned
promise resolves/rejects (or a caller supplied callback fires). Attempt to
ensure that returned-promise / supplied-callback control flow proceeds only
after `execWhenReady` tasks have finished. The control flow involved
is... rather involved, and it could use some further review and refactoring.

Bump webpack and the hard-source-plugin for webpack.

[util]: https://www.npmjs.com/package/util
2019-02-13 14:27:42 -05:00
Andre Medeiros 8fd6a6dc3d fix: estimate gas automatically when not specified 2019-02-13 13:44:03 -05:00
Pascal Precht fdd51cfebb fix(@embark/solidity): ensure SolidityProcess receives proper Logger instance
Prior to this commit Embark had a bug where, when running `$ embark run` inside
an Embark project that was freshly cloned or reset, it'd get stuck at loading
the Solidity compiler.

However, this only seemed to happen when Embark was used with its dashboard. When
running

```
$ embark run --nodashboard
```

instead, Embark loaded Solidity successfully and moved on with compilation.

This bug pretty much boiled down to `SolidityProcess` not receiving a valid `Logger`
dependency when instantiated via `SolcW`. The reason this was happening
is that we were sending the needed dependencies via `ProcessLauncher.send()`, which
serializes/deserializes its sent data.

```
solidityProcessLauncher.send({action: 'init', options: { logger: this.logger }});
```

And inside `SolidityProcess`

```
process.on('message', (msg) => {
  if (msg.action === "init") {
    solcProcess = new SolcProcess(msg.options);
  }
  ...
}
```

`SolcProcess` passes its `logger` instance down to `longRunningProcessTimer` which
uses methods on `Logger`.

However, since the data had been serialized, all prototype methods on the data is
gone, resulting in an error, which made Embark stop.

**Why was this only a problem when using the dashboard?**

Turns out we've only relied on `Logger.info()` in case the dashboard is used.
2019-02-13 12:18:26 -05:00
Anthony Laibe 193abd4780 feat(ui): color console item info as success 2019-02-13 12:12:28 -05:00
emizzle 07ede9017e fix(@embark/core): fix tests as paths have changed again
URLs for test_app files have changed on the remote again, this PR updates the tests to reflect that change.
2019-02-12 18:25:13 -05:00
Jonathan Rainville 401f7bc52f fix hoisting problem 2019-02-12 16:59:54 -05:00
Jonathan Rainville 8ee32c19dc feat(console): wait for connector onReady to have warning 2019-02-12 16:59:54 -05:00
Jonathan Rainville d7da38f6c4 remove path events and use function directly
also use await
2019-02-12 16:59:54 -05:00
Jonathan Rainville b49acee143 change dependency names 2019-02-12 16:59:54 -05:00
Jonathan Rainville b89e049f25 change package version to beta.0 2019-02-12 16:59:54 -05:00
Jonathan Rainville 69ac239ca0 fix ci tests 2019-02-12 16:59:54 -05:00
Jonathan Rainville f6beeff9c7 fix errors in code_generator 2019-02-12 16:59:54 -05:00
Jonathan Rainville 8ebba404e2 fix(test/blockchain): fix race condition of installing web3 and test 2019-02-12 16:59:54 -05:00
Jonathan Rainville 976f9944a8 feat(web3connector): add web3 connector plugin to connect to web3 2019-02-12 16:59:54 -05:00
Jonathan Rainville 3b0a2b9b3e feat(embark/generation): remove generation of web3instance.js 2019-02-12 16:59:54 -05:00
Jonathan Rainville bd9fc6683a feat(embarkjs/blockchain): remove dependency on web3instance.js 2019-02-12 16:59:54 -05:00
Jonathan Rainville fdd8ad5321 fix(embarkjs/blockchain): pass error only when there is an error 2019-02-12 16:59:54 -05:00
Michael Bradley, Jr 991d958436 build: merge test_dapps with the root monorepo
[PR 1318][PR1318] introduces a monorepo member that's used as a DApp
dependency, but the present arrangement whereby `test_dapps/` is an embedded
monorepo makes it difficult to develop and test such packages in tandem with
changes to `test_dapps/packages/*`.

Reorganize `test_dapps/*` as members of the root monorepo and make various
adjustments accordingly. This makes it easy to develop test dapps and any
packages they depend on simultaneously, but we lose the CI/QA arrangement where
test dapps are run with an embark installed from fresh tarballs. That
arrangement, which is quite worthwhile as a means to detect problems arising
from transitive dependencies as soon as possible, will be re-introduced in
another PR, possibly involving an auxiliary repository such as
embark-framework/dapp-bin.

Since the `package.json` `"test"` scripts of `test_dapps/*` are now kicked off
as part of `yarn test` in the root, and since port allocation isn't fully
dynamic, it's important to run `yarn test` with `lerna run`'s `--concurrency=1`
option. For the same reasons, the root `ci` and `qa` scripts are similarly
restricted. In the future, this setup can be refactored and improved, along
with use of `lerna run`'s `--since` option to increase testing efficiency in
CI.

[PR1318]: https://github.com/embark-framework/embark/pull/1318
2019-02-12 16:59:54 -05:00
Anthony Laibe cd326303ae feat(ui): keep state in frame 2019-02-12 16:42:16 -05:00
Anthony Laibe bc24598175 feat: create async wrapper 2019-02-12 08:24:20 +00:00
Iuri Matias c1d84fe8d8 refactor(@embark) add fs to plugin api; change (most) modules to use it instead of coupling them with require
refactor(@embark) add fs to plugin api; change (most) modules to use it instead of coupling them with require

refactor(@embark) add fs to plugin api; change (most) modules to use it instead of coupling them with require

refactor(@embark) add fs to plugin api; change (most) modules to use it instead of coupling them with require

refactor(@embark) add fs to plugin api; change (most) modules to use it instead of coupling them with require

refactor(@embark) add fs to plugin api; change (most) modules to use it instead of coupling them with require

refactor(@embark) add fs to plugin api; change (most) modules to use it instead of coupling them with require
2019-02-08 19:12:57 -05:00
Jonathan Rainville f0605bed2b refactor(generation/contracts): generate contracts to artifact dir 2019-02-08 17:13:48 -05:00
Jonathan Rainville c91e145981 refactor(generation/embarkjs): generate embarkjs in artifacts dir 2019-02-08 17:13:48 -05:00
Pascal Precht 40b9ac3c70 fix(@embark/core): fix memory leak when contract files are loaded
Prior to this commits `this.contractsFiles` kept growing at run time
for every file change that was detected by Embark.

The reason for that was a bug in a condition that determines whether
a subset of contract files needed to be added to memory or not.

In addition to that we kept pushing externally loaded contract files into
memory, even if they already existed in memory.
2019-02-08 17:11:54 -05:00
Michael Bradley, Jr 2e763f3720 feature(embark): add coveralls integration 2019-02-08 17:07:05 -05:00
André Medeiros edd1e0f968
fix: add watch script on top level (#1320) 2019-02-08 13:22:59 -05:00
Michael Bradley, Jr 9f0a370c34 refactor: add embarkjs to the monorepo
Don't import git history of embark-framework/EmbarkJS, simply copy over the
sources. Modify `package.json`, etc. re: being situated in the monorepo.

Make use of the root babel config but extend with
`packages/embarkjs/.babelrc.js`.

Build `test/` scripts into `build-test/` and git-ignore `build-test/`.

Revise `Blockchain.connect()` so that if the caller supplies a callback then a
promise is not returned.

Revise tests to test `Blockchain.connect()` usage with and without a callback.
2019-02-08 11:58:02 -06:00