Commit Graph

71 Commits

Author SHA1 Message Date
Iuri Matias fc4faa8ba9 fix: alleviate races re: embarkjs by introducing Plugin#addGeneratedCode and related refactors 2019-06-27 14:10:12 -05:00
Michael Bradley, Jr 2ce9ca6bb0 fix(@embark/coverage): function types and single statement ifs 2019-06-27 08:40:24 -05:00
Michael Bradley, Jr 3c9ed4183d refactor(@embark/watcher): downgrade from chokidar 3.x to latest 2.x
chokidar 3.x has fsevents `^2.0.6` [as a dependency][dependency], which
introduced node v12.x compatibility, a sought-after goal of our PR
1638. However, fsevents `2.0.6` node compatibility range is quite
[complicated][complicated]. After fsevents `2.0.6` was published its
maintainers published `1.2.9`, which backports NodeJS v12.x compatibility minus
some performance increases, while offering [much more
straightforward][straightforward] node compatibility. fsevents `1.2.9` [is
compatible][compatible] with the latest v2.x release of
chokidar (`2.1.6`). Since Embark advertises runtime compatibility with node
`>=8.11.3`, downgrade chokidar to `2.1.6`.

[dependency]: https://github.com/paulmillr/chokidar/blob/3.0.0/package.json#L25
[complicated]: https://github.com/fsevents/fsevents/blob/v2.0.6/package.json#L10
[straightforward]: https://github.com/fsevents/fsevents/blob/v1.2.9/package.json#L14
[compatible]: https://github.com/paulmillr/chokidar/blob/2.1.6/package.json#L49
2019-05-29 22:09:23 -05:00
Michael Bradley, Jr d116549c32 refactor(@embark/watcher): upgrade chokidar to 3.0.0
Upgrade chokidar to a version that's compatible with NodeJS v12.x.

Unfortunately, embark has other transitive dependencies that are not compatible
with v12.x, but upgrading chokidar is still a good step.
2019-05-28 09:36:06 +02:00
snyk-bot 9029bfe947 fix: packages/embark/package.json to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-HANDLEBARS-174183
- https://snyk.io/vuln/SNYK-JS-TAR-174125
- https://snyk.io/vuln/npm:mem:20180117
2019-05-24 10:38:22 -05:00
Michael Bradley, Jr cae583137c refactor: bump ganache-cli to latest stable version for compat with newer solc
Fixes #1461
2019-05-14 10:12:24 -05:00
Andre Medeiros 85cf9ac44b chore: upgrade ethereumjs-wallet
This pull-request upgrades `ethereumjs-wallet`, which has upgraded the underlying dependency on scrypt.js to 0.3.0, making scrypt an optional dependency and offering a pure JS version as a fallback.

The reasoning behind this is that scrypt is problematic to install in some systems, particularly those that don't have node-gyp setup and we have seen some weird issues when installing with elevated privileges (i.e. `sudo npm install -g scrypt`)
2019-05-12 18:20:34 -04:00
Michael Bradley, Jr efb63af71e hotfix: bump yarn.lock
PR #1607 didn't include changes to `yarn.lock` after a dependency was added to
`dapps/templates/demo`.
2019-05-10 18:04:36 -05:00
Andre Medeiros 661ce74877 chore: move console to module 2019-05-03 09:24:09 -05:00
Michael Bradley, Jr 2044ed5e9b build: bump tslint in all relevant pkgs from v5.11.0 to v5.16.0
Related to bumping the typescript package to its latest version
2019-05-03 08:53:43 -05:00
Michael Bradley, Jr 142130e533 build: bump typescript in all relevant pkgs from v3.3.1 to v3.4.5
The more recent version of TypeScript works around a [problem][chalk-problem]
encountered with the chalk package's type definitions.

[chalk-problem]: https://github.com/chalk/chalk/pull/296#issuecomment-418789492
2019-05-02 20:24:02 -05:00
Jonathan Rainville bcd006a1cf refactor(@embark/utils): move copyToCliboard to embark-utils 2019-04-30 12:06:54 -04:00
Jonathan Rainville 7495095d49 refactor(@embark/ens): move ens module to embark-ens package 2019-04-30 16:31:07 +02:00
Andre Medeiros 740196684c fix: update solidity-parser-antlr to allow the use of the payable keyword in contracts 2019-04-22 11:44:32 -04:00
Michael Bradley, Jr a39c2c82d7 refactor(@embark/api): in dev use cockpit redirect instead of proxy
Embark API server's development proxy from port 55555 to 3000 was attempting to
inappropriately forward an `/embark-api/` endpoint for the blockchain process
logs to Create React App's development server. Why it was only happening for
the one endpoint is not known but probably has to do with timing around
registration of the API server's express routes.

The problem can be fixed with a one-line `filter:` function in the options for
`express-http-proxy`. However, it was realized that to fix an unrelated
problem, whereby the proxy doesn't forward websockets to CRA such that hot
reload doesn't work when accessing `embark-ui` in development on port 55555, a
switch to `http-proxy-middleware` would be required. That was quickly
attempted (easy switch) but there are outstanding [difficulties][bug] with
`webpack-dev-server` and `node-http-proxy` that cause CRA to crash.

Switch strategies and refactor the API module to serve a page on port 55555 (in
development only) that alerts the developer `embark-ui` should be accessed on
port 3000. The page redirects (client-side) after 10 seconds, with URL query
params and/or hash preserved. A future version could instead do client-side
polling of port 3000 with `fetch` and then redirect only once it's
available. The reason for not redirecting immediately is that the intermediate
page makes it more obvious what needs to be done, e.g. CRA dev server may need
to be started with `yarn start`.

[bug]: https://github.com/webpack/webpack-dev-server/issues/1642
2019-04-16 14:25:35 -04:00
Pascal Precht 47ba899c85 chore(@embark/embark): update handlebars dependency
Fixes: #1339
2019-04-09 10:45:16 +02:00
emizzle f27cde9261 feat(@cockpit/editor): Make tabs draggable
Make tabs draggable so they can be arranged how the user would like.

The dragging functionality locks the tabs to the parent container.

Support for multiple rows of tabs.

Styling updates for selected tabs.
2019-04-04 12:46:47 -04:00
Pascal Precht 4c3ec26fff uiux(@cockpit/explorer): render timestamp of transactions 2019-04-04 12:45:00 -04:00
Iuri Matias a46ea71dbd refactor(@embark/embark-whisper): move whisper module to its own package
refactor(@embark/embark-whisper): move whisper module to its own package

add eslint to dependencies

register whisper module
2019-03-28 14:33:10 -04:00
Michael Bradley, Jr 9829e925b4 refactor(@embark/api): catch-all route, fallback pages for Cockpit
Implement a `/*` server-side catch-all route for Cockpit that loads Cockpit's
`index.html`.

This change is necessary with intent to disable offline-first behavior in
production builds of Cockpit. Cockpit's service worker effectively translates
server-side route unavailability into client-side behaviors of
connected-react-router. When the service worker is unregistered the same will
be accomplished via the server-side catch-all route.

Implement fallback pages for when embark is in the monorepo but Cockpit's
Create React App development server isn't yet started or isn't yet responsive.

Implement a fallback page for when the static build of Cockpit is missing. When
embark is in the monorepo, give instructions for building Cockpit. Otherwise,
report that the distribution is broken.

Deprecate the environment variable `EMBARK_DEVELOPMENT` in favor of
`EMBARK_UI_STATIC`. Unless the latter is truthy at runtime, when embark is in
the monorepo the CRA dev server of Cockpit will be accessible at
`localhost:55555` via proxied requests to `localhost:3000`. The deprecation is
not a breaking change as `EMBARK_DEVELOPMENT` / `EMBARK_UI_STATIC` are not
relevant to normal users, but only to developers working on embark itself.

Bump `express-http-proxy` to the latest version.
2019-03-26 16:00:13 -04:00
Michael Bradley, Jr db40f03d68 fix(embark): specify "rxjs" as a dependency vs. "@reactivex/rxjs"
`"rxjs"` is [preferable][rxjs-pkg-name] in a CommonJS context,
e.g. node. Existing embark code is `require`-ing from the unscoped package
name, which worked in the monorepo because some other dependency specifies
`"rxjs"` and yarn hoists it to the root of the workspace. In a production
install of embark, though, `require('rxjs')` was failing.

[rxjs-pkg-name]: https://github.com/ReactiveX/rxjs/issues/2577
2019-03-17 20:07:12 -04:00
Andre Medeiros aba4e0e059 fix: track and debug last transaction correctly 2019-03-14 14:15:19 -04:00
Jonathan Rainville 359c28f5b5 fix(cockpit:editor): fix arrow not turning 2019-03-08 09:26:13 -05:00
André Medeiros 913b4e1817
fix: upgrade packages with vulnerabilities (#1388) 2019-03-05 14:29:36 -05:00
Michael Bradley, Jr de0f02d00a build: make DApp templates member packages of the monorepo
Previously, templates were in a subdirectory of `packages/embark`. Reorganize
them so that they are member packages of the monorepo. This allows them to
cleanly depend on other members of the monorepo,
e.g. `embarkjs-connector-web3`.

It is desirable for the templates, in the context of the monorepo, to specify
embark as a dependency, to take advantage of `npx embark test` (and it's a
"forward looking" setup re: how we plan to evolve embark). However, if embark
were to specify the template packages as dependencies a circular relationship
would be introduced, which is [unsupported by Lerna][circular]. Therefore,
revise the template generator so that all templates are resolved / fetched at
runtime, i.e. `boilerplate`, `demo`, and `simple` are no longer
"built-ins" *per se*. This change won't be apparent to embark's users, but it
does mean that the template generator won't work (in a production install of
embark) if it can't connect to the npm registry, i.e. when the user runs
`embark demo` or `embark new [--simple]`. When embark is inside the monorepo,
templates are resolved and copied from the yarn workspace rather than being
fetched from the registry, which is convenient for development. Also, any
template dependencies that are members of the monorepo are linked into the
copied template's `node_modules` rather than being installed from the registry,
again for convenience. During template generation, remove scripts and
dependencies that pertain only to membership in the monorepo; for now, that
involves removing embark as a dependency since we're not quite ready for that
arrangement to be the default, i.e. outside of the monorepo.

Refactor the root scripts so that more of them can consistently be used with
Lerna's filter options, e.g. `--scope` and `--ignore`. "Combo" scripts that
don't support filtering generally have a `:full` postfix.

Flip `clean` and `reset` scripts at the root and in the member packages for
consistency re: Lerna's notion of `clean` and embark's notion of `reset`. Have
each package run its `reset` script when its `clean` script is invoked (and
that's all for now), relying on `lerna clean` to delete packages'
`node_modules` in view of how Lerna's topological sorting works.

Lift the implementation of `embark reset` into a private package in
`packages/embark-reset` and make it a bundled dependency of embark. Packages in
`dapps/*` depend on `embark-reset` directly and make use of it with `npx
embark-reset` (but only in monorepo context). This removes a "wart" where
reboots could show errors when embark's sources aren't already built in
`packages/embark/dist`. Users will not notice any difference since `embark
reset` works as before, transparently making use of the `embark-reset`
package. The only downside to having it be a bundled dependency of embark is
that bundled deps have all of their `node_modules` included in the tarball
built with `npm pack` (that's why having the templates as bundled dependencies
of embark isn't a viable approach). However, `embark-reset` only has one
dependency, `rimraf`, which is a tiny module, so the cost seems acceptable.

As part of the reorganization, move `test_dapps` into `dapps/tests` and
`packages/embark/templates` into `dapps/templates`. Keep the directory names
short but revise the package names to facilitate simple filtering with
`embark-dapp-*`. Consolidate `.yarnrc` and `.gitignore` and clean up some
redundant ignore listings.

Scripts run with `--scope embark-dapp-*` use `--concurrency=1` to avoid
conflicts that could arise over network ports. The `ci:full` and `qa:full`
scripts use `--concurrency=1` in all scopes, for two reasons: resource
limitations on Travis and AppVeyor result in slower runs with concurrency >1,
and if something fails in those contexts it's easier to see what went wrong
when Lerna's output isn't interleaved from a bunch of scripts in `packages/*`.

Bump the Lerna version.

[circular]: https://github.com/lerna/lerna/issues/1198#issuecomment-442278902
2019-03-05 10:20:57 -06:00
Michael Bradley, Jr 57445d0819 build(embarkjs): clean script should invoke 'npx rimraf'
Packages' `"clean"` scripts should run regardless of whether `node_modules` is
in place at the root and/or package levels, but the `embarkjs` package was
missing a needed `npx` in front of `rimraf` in its script.

Also, bump its devDeps version of rimraf to be in line with the rest of
`packages/*` and `test_dapps/*`.
2019-02-14 01:26:10 -06:00
Michael Bradley, Jr 23ae78a6d6 refactor: be consistent with callbacks and promises
If a function receives a callback argument then it should not return a promise
if the caller's callback will be invoked. Both invoking a callback and
returning a promise can lead to at best confusion (in code review and at
runtime) and at worst non-deterministic behavior, such as race
conditions. Also, a caller supplying a callback may not handle a returned
promise, leading to unhandled rejection errors.

Refactor all readily identified functions where a callback argument can be
supplied but the function returns a promise regardless. Make use of
`callbackify` and `promisify` where it made sense to do so during the
refactoring. Some callsites of the revised functions may have been accidentally
overlooked and still need to be updated. Some functions that take callback
arguments may execute them synchronously, at odds with control flow of a
returned promise (if a callback wasn't supplied). Such cases should be
identified and fixed so that asynchronous behavior is fully consistent whether
the caller supplies a callback or receives a promise.

Make sure promises that pass control flow to a callback ignore rejections,
since those should be handled by the callback.

Don't return promise instances unnecessarily from async functions (since they
always return promises) and change some functions that return promises to async
functions (where it's simple to do so).

Whisper was using an ad hoc promise-like `messageEvents` object. However, that
object behaved more like an observable, since promises either resolve or
reject, and only do so one time. `messageEvents` was also intertwined with
callbacks. Replace `messageEvents` with RxJS Observable. `listenTo` now returns
Observable instances and callers can subscribe to them.

`Blockchain.connect` of embarkjs could suffer from a race condition where tasks
associated with `execWhenReady` might be ongoing when `connect`'s returned
promise resolves/rejects (or a caller supplied callback fires). Attempt to
ensure that returned-promise / supplied-callback control flow proceeds only
after `execWhenReady` tasks have finished. The control flow involved
is... rather involved, and it could use some further review and refactoring.

Bump webpack and the hard-source-plugin for webpack.

[util]: https://www.npmjs.com/package/util
2019-02-13 14:27:42 -05:00
Michael Bradley, Jr 991d958436 build: merge test_dapps with the root monorepo
[PR 1318][PR1318] introduces a monorepo member that's used as a DApp
dependency, but the present arrangement whereby `test_dapps/` is an embedded
monorepo makes it difficult to develop and test such packages in tandem with
changes to `test_dapps/packages/*`.

Reorganize `test_dapps/*` as members of the root monorepo and make various
adjustments accordingly. This makes it easy to develop test dapps and any
packages they depend on simultaneously, but we lose the CI/QA arrangement where
test dapps are run with an embark installed from fresh tarballs. That
arrangement, which is quite worthwhile as a means to detect problems arising
from transitive dependencies as soon as possible, will be re-introduced in
another PR, possibly involving an auxiliary repository such as
embark-framework/dapp-bin.

Since the `package.json` `"test"` scripts of `test_dapps/*` are now kicked off
as part of `yarn test` in the root, and since port allocation isn't fully
dynamic, it's important to run `yarn test` with `lerna run`'s `--concurrency=1`
option. For the same reasons, the root `ci` and `qa` scripts are similarly
restricted. In the future, this setup can be refactored and improved, along
with use of `lerna run`'s `--since` option to increase testing efficiency in
CI.

[PR1318]: https://github.com/embark-framework/embark/pull/1318
2019-02-12 16:59:54 -05:00
Michael Bradley, Jr 2e763f3720 feature(embark): add coveralls integration 2019-02-08 17:07:05 -05:00
Michael Bradley, Jr 9f0a370c34 refactor: add embarkjs to the monorepo
Don't import git history of embark-framework/EmbarkJS, simply copy over the
sources. Modify `package.json`, etc. re: being situated in the monorepo.

Make use of the root babel config but extend with
`packages/embarkjs/.babelrc.js`.

Build `test/` scripts into `build-test/` and git-ignore `build-test/`.

Revise `Blockchain.connect()` so that if the caller supplies a callback then a
promise is not returned.

Revise tests to test `Blockchain.connect()` usage with and without a callback.
2019-02-08 11:58:02 -06:00
Michael Bradley, Jr 41fb31ae73 fix: use a yarn.lock that wasn't completely regenerated
PR 1304 introduced a regenerated `yarn.lock` at the root. Somehow, that's
responsible for the error described in [a comment on PR 1307][error].

Restore the `yarn.lock` from 3f61e314d9, then
catch it up with `yarn reboot:full` in the root. `yarn typecheck` then runs
without error in `packages/embark`.

[error]: https://github.com/embark-framework/embark/pull/1307#issuecomment-461184291
2019-02-06 17:17:01 -06:00
Iuri Matias 8f393f7298 refactor(@embark/embark-vyper) move vyper to its own module 2019-02-06 14:32:54 -05:00
Iuri Matias 8e7c8e940a refactor(@embark/embark-compiler) move compiler to its own module
refactor(@embark/embark-compiler) move compiler to its own module

 This is a combination of 2 commits.

refactor(@embark/embark-compiler) move compiler to its own module

refactor(@embark/embark-compiler) move compiler to its own module

Update packages/embark-compiler/package.json

Co-Authored-By: iurimatias <iuri.matias@gmail.com>

Update packages/embark-compiler/package.json

Co-Authored-By: iurimatias <iuri.matias@gmail.com>

Update packages/embark-compiler/package.json

Co-Authored-By: iurimatias <iuri.matias@gmail.com>

refactor(@embark/embark-compiler) move compiler to its own module

refactor(@embark/embark-compiler) move compiler to its own module

refactor(@embark/embark-compiler) move compiler to its own module

refactor(@embark/embark-compiler) move compiler to its own module
2019-02-06 13:24:45 -05:00
Michael Bradley, Jr 3f61e314d9 build: implement a common babel config
Setup a `babel.config.js` in the root of the monorepo to be used by
`packages/*`. It won't be used by some packages, e.g. `packages/embark-ui`, but
most of them should use it instead of rolling their own.

Allow for package-level modifications by specifying `babelrcRoots` in the root
`babel.config.js`

Use the babel `--root-mode upward` option in `packages/embark`'s `build`
script. Other packages intending to use the common config should do likewise.

Use a `.babelrc.js` in `packages/embark` to supply the package-specific
`ignore` settings.

Make packages used by the common config devDeps of the root.

Extract babel-related devDeps from `packages/embark`, but don't extract the
non-dev deps since those are used by embark's pipeline in a production
install. Normally, it should only be necessary to have `@babel/cli` and
`@babel/core` in devDeps, and possibly `@babel/runtime-corejs2` in deps, plus
any package-specific babel-related dev/deps. Once we deprecate the pipeline, we
can finish the extraction.

Use `ncu -f '/babel/' -u` to bump the versions of all babel-related deps in the
root and in `packages/embark`. We get better space/time savings from the yarn
workspace when versions match.
2019-02-06 11:34:47 -05:00
Iuri Matias fbc8ab0fea refactor(@embark/embark-typings) move typings to its own module
fix typings

WIP: refactor

WIP: just needed an annotation 🎉 thanks @emizzle!

WIP: don't pack a tarball since it's a private package

WIP: no need to list as a devDep
2019-02-06 11:07:44 -05:00
Michael Bradley, Jr 7a93e4b6a7 build: implement a monorepo with Lerna
TL;DR
=====

`yarn install` in a fresh clone of the repo.

`yarn reboot` when switching branches.

When pulling in these changes, there may be untracked files at the root in
all/some of:

```
.embark/
.nyc_output/
coverage/
dist/
embark-ui/
test_apps/
```

They can be safely deleted since those paths are no longer in use at the root.

Many of the scripts in the top-level `package.json` support Lerna's [filter
options]. For example:

`yarn build --scope embark` build only `packages/embark`.

`yarn build --ignore embark-ui` build everything except `packages/embark-ui`.

Scoping scripts will be more useful when there are more packages in the
monorepo and, for example, `yarn start` doesn't need to be invoked for all of
them while working on just a few of them simultaneously, e.g `embark` and
`embarkjs`.

It's also possible to `cd` into a particular package and run its scripts
directly:

```
cd packages/embark && yarn watch
```

Hot Topics & Questions
======================

What should be done about the [README][embark-readme] for `packages/embark`?
Should the top-level README be duplicated in that package?

Lerna is setup to use [Fixed/Locked mode][fixed-locked], and accordingly
`packages/embark-ui` is set to `4.0.0-beta.0`. The same will be true when
adding embarkjs, swarm-api, etc. to the monorepo. Is this acceptable or do we
want to use [Independent mode][independent]?

Scripts
=======

If a package doesn't have a matching script, `lerna run` skips it
automatically. For example, `packages/embark-ui` doesn't have a `typecheck`
script.

`yarn build`
------------

Runs babel, webpack, etc. according to a package's `build` script.

`yarn build:no-ui` is a shortcut for `yarn build --ignore embark-ui`.

`yarn ci`
---------

Runs a series of scripts relevant in a CI context according to a package's `ci`
script. For `packages/embark` that's `lint typecheck build test package`.

Also runs the `ci` script of the embedded `test_dapps` monorepo.

`yarn clean`
------------

Runs rimraf, etc. according to a package's `clean` script.

`yarn globalize`
----------------

Makes the development embark available on the global PATH, either via
symlink (Linux, macOS) or a shim script (Windows).

`yarn lint`
-----------

Runs eslint, etc. according to a package's `lint` script.

`yarn package`
--------------

Invokes `npm pack` according to a package's `package` script.

`yarn qa`
---------

Very similar to `ci`, runs a series of scripts according to a package's `qa`
script. The big difference between `ci` and `qa` is that at the top-level `qa`
first kicks off `reboot:full`.

There is a `preqa` script ([invoked automatically][npm-scripts]), which is a
bit of a wart. It makes sure that `embark reset` can be run successfully in
`packages/embark/templates/*` when the `reboot` script invokes the `reset`
script.

The `qa` script is invoked by `yarn release` before the latter proceeds to
invoke `lerna publish`.

`yarn reboot`
-------------

Invokes the `reset` script and then does `yarn install`.

The `reboot:full` variant invokes `reset:full` and then does `yarn install`.

`yarn release`
--------------

Works in concert with [lerna publish], which will prompt to verify the version
before proceeding. Use `n` to cancel instead of `ctrl-c` as `lerna publish` has
been seen to occasionally misbehave when not exited cleanly (e.g. creating a
tag when it shouldn't have).

```
yarn release [bump] [--options]
```

* `[bump]` see [`publish` positionals][pub-pos] and [`version`
  positionals][ver-pos]; an exact version can also be specified.
* `--preid` prerelease identifier, e.g. `beta`; when doing a prerelease bump
  will default to whatever identifier is currently in use.
* `--dist-tag` registry distribution tag, defaults to `latest`.
* `--message` commit message format, defaults to `chore(release): %v`.
* `--sign` indicates that the git commit and tag should be signed; not signed
  by default.
* `--release-branch` default is `master`; must match the current branch.
* `--git-remote` default is `origin`.
* `--registry` default is `https://registry.npmjs.org/` per the top-level
  [`lerna.json`][lerna-json].

To release `4.0.0-beta.1` as `embark@next` (assuming version is currently at
`4.0.0-beta.0`) could do:

```
yarn release prerelease --dist-tag next
```

For *test releases* (there is no longer a `--dry-run` option) [verdaccio] and a
filesystem git remote can be used.

Condensend instructions:

```
mkdir -p ~/temp/clones && cd ~/temp/clones
git clone git@github.com:embark-framework/embark.git
cd ~/repos/embark
git remote add FAKEembark ~/temp/clones/embark
```
in another terminal:
```
npm i -g verdaccio && verdaccio
```
in the first terminal:
```
yarn release --git-remote FAKEembark --registry http://localhost:4873/
```

`yarn reset`
------------

Invokes cleaning and resetting steps according to a package's `reset`
script. The big difference between `clean` and `reset` is that `reset` is
intended to delete a package's `node_modules`.

The `reset:full` variant deletes the monorepo's top-level `node_modules` at the
end. That shouldn't be necessary too often, e.g. in day-to-day work when
switching branches, which is why there is `reboot` / `reset` vs. `reboot:full`
/ `reset:full`.

Errors may be seen related to invocation of `embark reset` if embark is not
built, but `reset` will still complete successfully.

`yarn start`
------------

Runs babel, webpack, tsc, etc. (in parallel, in watch mode) according to a
package's `start` script.

`yarn test`
-----------

Run mocha, etc. according to a package's `test` script.

The `test:full` variant runs a series of scripts: `lint typecheck test
test_dapps`.

`yarn test_dapps`
-----------------

Runs the `test` script of the embedded `test_dapps` monorepo.

The `test_dapps:ci` and `test_dapps:qa` variants run the `ci` and `qa` scripts
of the embedded `test_dapps` monorepo, respectively.

`yarn typecheck`
----------------

Runs tsc, etc. according to a package's `typecheck` script.

Notes
=====

`npx` is used in some of the top-level and package scripts to ensure the
scripts can run even if `node_modules` is missing.

[`"nohoist"`][nohoist] specifies a couple of embark packages because
[`restrictPath`][restrictpath] is interfering with access to modules that are
located in a higher-up `node_modules`.

All dependencies in `packages/embark-ui` have been made `devDependencies` since
its production build is self-contained.

`packages/embark`'s existing CHANGELOG's formatting has been slightly adjusted
to match the formatting that Lerna will use going forward (entries in the log
haven't been modified).

Lerna will generate a CHANGELOG at the top-level and in each package. Since
we're transitioning to a monorepo, things may look a little wonky with respect
to old entries in `packages/embark/CHANGELOG.md` and going forward we need to
consider how scoping our commits corresponds to member-packages of the
monorepo.

In `packages/embark`, `test` invokes `scripts/test`, which starts a child
process wherein `process.env.DAPP_PATH` is a temporary path that has all of
`packages/embark/dist/test` copied into it, so that paths to test
helpers/fixtures don't need to be prefixed with `dist/test/` and so that a
`.embark` directory doesn't get written into `packages/embark`.

The `"engines"` specified in top-level and packages' `package.json` reflect a
node and npm pair that match (a source of confusion in the past). The pair was
chosen according to the first post v5 npm that's bundled with node. A
`"runtime"` key/object has been introduced in `packages/embark/package.json`
which is used as the basis for specifying the minimum version of node that can
be used to run embark, and that's what is checked by `bin/embark`.

Some changes have been introduced, e.g. in `lib/core/config` and
`lib/utils/solidity/remapImports` so that it's *not* implicitly assumed that
`process.env.DAPP_PATH` / `fs.dappPath()` are the same as
`process.cwd()`. There are probably several++ places where that assumption is
still in effect, and we should work to identify and correct them.

`embark reset` now deletes `embarkArtifacts/` within a dapp root, and
`embarkArtifacts/` is git-ignored.

`lib/core/env` adds all `node_modules` relative to `process.env.EMBARK_PATH` to
`NODE_PATH` so that embark's modules can be resolved as expected whether
embark's `node_modules` have been deduped or are installed in npm's flat
"global style".

`checkDependencies` has been inlined (see `lib/utils/checkDependencies`) and
slightly modified to support dependencies that have been hoisted into a
higher-up `node_modules`, e.g. as part of a yarn workspace. eslint has been
disabled for that script to avoid more involved changes to it.

`test_apps` is not in `packages/embark`; rather, there is `test_dapps` at the
top-level of the monorepo. `test_dapps` is an embedded monorepo, and its `ci` /
`qa` scripts `npm install` embark from freshly built tarballs of the packages
in the outer monorepo and then use that installation to run `embark test` in
the dapps. This should allow us to rapidly detect breakage related to
auto-bumps in transitive dependencies.

[filter options]: https://github.com/lerna/lerna/tree/master/core/filter-options
[embark-readme]: https://github.com/embark-framework/embark/blob/build/lerna/packages/embark/README.md
[fixed-locked]: https://github.com/lerna/lerna#fixedlocked-mode-default
[independent]: https://github.com/lerna/lerna#independent-mode
[npm-scripts]: https://docs.npmjs.com/misc/scripts
[lerna publish]: https://github.com/lerna/lerna/tree/master/commands/publish
[pub-pos]: https://github.com/lerna/lerna/tree/master/commands/publish#positionals
[ver-pos]: https://github.com/lerna/lerna/tree/master/commands/version#positionals
[lerna-json]: https://github.com/embark-framework/embark/blob/build/lerna/lerna.json#L11
[verdaccio]: https://www.npmjs.com/package/verdaccio
[nohoist]: https://github.com/embark-framework/embark/blob/build/lerna/package.json#L52-L55
[restrictpath]: https://github.com/embark-framework/embark/blob/build/lerna/packages/embark/src/lib/core/fs.js#L9
2019-02-04 14:28:49 -06:00
Anthony Laibe 29db66be23 feat: import resolver 2019-01-25 16:37:09 -05:00
emizzle 2613e6d683 feat(@embark/core): Recursively import contracts
Add support to recursively import contracts. If we have three contracts

1. A imports B
2. B imports C

Then prior to this PR, contract A would import contract B, and a remapping would be added to the contract so the compiler would know how to find contract B. However, contract B imports contracts C, and because the `parseFileForImport` method was not recursive, the remappings were not able to go one level deeper to remap the path to contract C, and thus the compiler would not know how to locate contract C, and would complain with the error `File outside of allowed directories.`

With the introduction of this PR, the `parseFileForImport` method is now recursive, and so any contract imported is also checked for it's own imports that can be remapped. Specifically, this use case is applicable when there is a dependency containing contracts that imports one of it's own dependency's contracts, ie:

```
pragma solididty ^0.5.0;

import "dependency-1/contract-1.sol";
```
where the dependencies look like:
```
|- node_modules
|--- dependency-1
|----- contract-1.sol <--- contains import "dependency-2/contract-2.sol"
|--- dependency-2
|----- contract-2.sol
```

Add unit tests that verify recursive imports work.
Add embark depdendency that installs a contract used in the recursive unit tests.
2019-01-22 14:46:38 +11:00
emizzle 9a9eb45836 feat(@embark/core): Run all code in VM2
All code to be run in the console is run through a completely sandboxed VM2 instance, instead of the default Node VM.

VM2 will only allow whitelisted packages in a `require` statement. The whitelisted packages needed to run EmbarkJS scripts are:
```
[
  "@babel/runtime-corejs2/helpers/interopRequireDefault",
  "@babel/runtime-corejs2/core-js/json/stringify",
  "@babel/runtime-corejs2/core-js/promise",
  "@babel/runtime-corejs2/core-js/object/assign",
  "eth-ens-namehash"
]
```

This can be circumvented in an Embark context (ie Plugin) if needed, for example in a Plugin constructor:
```
Embark.events.emit('runcode:register', 'require', require('lodash'), false);

Embark.events.request("runcode:eval", "_.head(['a', 'b', 'c', 'd']);", (err, result) => {
  if(err) return console.log('========> error: ' + err);
  console.log('========> ' + result);
});
```
Will emit `========> a`.

NOTE: Attempts to use this method to override `require` and `eval` should be handled by Embark and not allowed.

NOTE: VM2 seems to allow `eval`, however it is in a completely sandboxed environment, so I'm unsure that we need to be too concerned with this. Thoughts?

Refactor tests to use standalone instance of the newly created VM class, so that code is not evaluated through the console. This was done based on the new unit test case where accounts are redefined in a subsequent unit test, which was not originally working with the initial VM2 PR.

Refactor `codeRunner`, put all code-affecting logic in the `VM` class.

Changed `runCode` to `VM` and converted to TypeScript

Add unit tests for `VM`.
2019-01-16 10:16:54 +11:00
Anthony Laibe 2505fa5284 feat: add development mode to cockpit 2019-01-09 17:34:13 -05:00
Anthony Laibe d67863cff6 feat: add API server 2019-01-09 14:11:03 +00:00
Anthony Laibe 8a6d075cd1 feat: add coverage events
Instead of looking at the transaction, instrument the
source to publish coverage event
2018-12-20 16:21:31 -05:00
Michael Bradley, Jr aba551e84f refactor(proxy): proxy blockchain accounts so that they are available in the Dapp 2018-12-18 14:07:46 -06:00
Michael Bradley, Jr acf62668ab refactor(@embark/blockchain_process): remove http-proxy-middleware
The problems described in embark PR #1166 can be resolved by implementing the
blockchain proxy with `http-proxy` directly instead of using `express` together
with `http-proxy-middleware`. The ultimate cause of the buggy behavior (the
"stuck sockets" problems described in #1166) is unknown.

The need to swallow some errors as described in embark PR #1181 is also
eliminated by dropping `http-proxy-middleware` and `express`.
2018-12-18 14:07:46 -06:00
Michael Bradley, Jr e738efe15a refactor(@embark/blockchain_process): swallow errors, revise streams
For reasons unknown, `ECONNRESET` errors on websocket connections to embark's
blockchain proxy are not automatically handled on Windows as they are on macOS
and Linux (or those errors aren't happening on those platforms, it's difficult
to determine). Explicitly swallow such errors so the blockchain process doesn't
crash. Prior to this PR, the crash-behavior can be reproduced on Windows by
running `embark blockchain` and `embark run` in separate terminals and quitting
`embark run` while `embark blockchain` is still running.

Consistently use the `simples` package's `WsParser` to process websocket
traffic instead of using `WsParser` for requests and the `ws` package's
`Websocket.Receiver` for responses.

Consistently use `pump` to connect parser streams instead of using `pump` in
some places and `chain` in others. Drop use of `cloneable` (and the package
dependency) since it was used previously in hopes it would fix the errors, but
it's unnecessary and didn't fix them.
2018-12-13 16:28:08 +01:00
Jonathan Rainville 33f2b02ee7 chore(web3): update web3 beta to version 37 2018-12-12 14:44:31 -05:00
Andre Medeiros c98dbc24de feature: introduce code coverage 2018-12-12 10:42:00 -05:00
Iuri Matias 8cbbcfe89e chore(@embark): re-add linting dependencies to devDependencies 2018-12-07 21:47:18 -05:00
Iuri Matias 74985f4174 chore(@embark): update remix-debug dependency 2018-12-07 21:47:18 -05:00
Iuri Matias d733906c9b chore(@embark): remove no longer needed dependency compression-webpack-plugin 2018-12-07 21:47:18 -05:00