embark

Commit Graph

Author	SHA1	Message	Date
Michael Bradley, Jr	3693ebd90d	fix: ensure that packages properly specify their dependencies Many packages in the monorepo did not specify all of their dependencies; they were effectively relying on resolution in the monorepo's root `node_modules`. In a production release of `embark` and `embark[js]-` packages this can lead to broken packages. To fix the problem currently and to help prevent it from happening again, make use of the `eslint-plugin-import` package's `import/no-extraneous-dependencies` and `import/no-unresolved` rules. In the root `tslint.json` set `"no-implicit-dependencies": true`, wich is the tslint equivalent of `import/no-extraneous-dependencies`; there is no tslint equivalent for `import/no-unresolved`, but we will eventually replace tslint with an eslint configuration that checks both `.js` and `.ts` files. For `import/no-unresolved` to work in our monorepo setup, in most packages add an `index.js` that has: ```js module.exports = require('./dist'); // or './dist/lib' in some cases ``` And point `"main"` in `package.json` to `"./index.js"`. Despite what's indicated in npm's documentation for `package.json`, it's also necessary to add `"index.js"` to the `"files"` array. Make sure that all `.js` files that can and should be linted are in fact linted. For example, files in `packages/embark/src/cmd/` weren't being linted and many test suites weren't being linted. Bump all relevant packages to `eslint@6.8.0`. Fix all linter errors that arose after these changes. Implement a `check-yarn-lock` script that's run as part of `"ci:full"` and `"qa:full"`, and can manually be invoked via `yarn cylock` in the root of the monorepo. The script exits with error if any specifiers are found in `yarn.lock` for `embark[js][-]` and/or `@embarklabs/*` (with a few exceptions, cf. `scripts/check-yarn-lock.js`).	2020-02-25 14:52:10 -06:00
Michael Bradley, Jr	ee56f37713	build: implement collective typecheck This PR replaces #2057. Implement a collective typecheck action that can be invoked in the root of the monorepo with `yarn typecheck` or in watch-mode with `yarn watch:typecheck`. Include the watch-mode typecheck action as part of `yarn start` (a.k.a `yarn watch`). To activate collective typecheck for a package in the monorepo, its `package.json` file should specify: ``` { "embark-collective": { "typecheck": true } } ``` -or- ``` { "embark-collective": { "typecheck": {...} } } ``` Where `{...}` above is a `tsconfig.json` fragment that will be merged into the config generated for the package according the same rules that `tsc` applies when merging [configs][config]. When collective typecheck begins, it generates a `tsconfig.json` for the root of the monorepo and for each package that is activated for the action. If the generated JSON is different than what's on disk for the respective root/package config, or if the config is not present on disk, then it will be written. Changes to generated `tsconfig.json` files should be committed; such changes will arise when there are structural changes to the monorepo, e.g. a package is added, removed, moved and/or the directory containing it is renamed. Since the configs are only generated at the beginning of collective typecheck, when structural changes are made in the monorepo `yarn typecheck` (or `yarn start` or `yarn watch:typecheck`) should be restarted. Nearly all of the packages in the monorepo (i.e. all those for which it makes sense) have been activated for collective typecheck. Even those packages that don't contain `.ts` sources are activated because `tsc` can make better sense of the code base as a whole owing to the project references included in the generated `tsconfig.json` files. Also, owing to the fully cross-referenced `tsconfig.json` files, it's possible for `tsc` to type check the whole code base without babel (`yarn build` or `yarn watch:build`) having been run beforehand. NOTE that a "cold typecheck" of the whole monorepo is resource intensive: on this author's 2019 MacBook Pro it takes around three minutes, the fans spin up, and `tsc` uses nearly 0.5 GB of RAM. However, once a full typecheck has completed, the next full typecheck will complete in a few seconds or less; and when running in watch-mode there is likewise a big speedup once a full typecheck has completed, whether that full check happened before it's running in watch-mode or when watch-mode itself resulted in a full check before switching automatically to incremental check, as well a corresponding big reduction in resource consumption. A full check will be needed any time `yarn typecheck` (or `yarn start` or `yarn watch:typecheck`) is run in a fresh clone plus `yarn install`, or after doing `yarn reboot[:full]` or `yarn reset`. The combination of options in each generated package-level `tsconfig.json` and the root `tsconfig.base.json` result in `tsc` writing `.d.ts` files (TypeScript declaration files) into the `dist/` directory of each package. That output is intended to live side-by-side with babel's output, and therefore the `"rootDir"` option in each generated config is set to `"./src"`. In projects activated for collective typecheck, `.js` may be converted to `.ts` and/or `.ts` sources may be added without any additional changes needed in package-level `package.json`. --- Reorganize types in `packages/core/typings` (a.k.a `@types/embark`) into `packages/core/core` (`embark-core`), refactor other packages' imports accordingly, and delete `packages/core/typings` from the monorepo. This results in some similarly named but incompatible types exported from `embark-core` (e.g. `Events` and `EmbarkEvents`, the latter being the one from `packages/core/typings`); future refactoring should consolidate those types. To avoid circular dependency relationships it's also necessary to split out `Engine` from `embark-core` into its own package (`embark-engine`) and to introduce a bit of duplication, e.g. the `Maybe` type that's now defined in both `embark-i18n` and `embark-core`. In the process of the types reorg, move many dependencies spec'd in various `package.json` to the `package.json` of the package/s that actually depend on them, e.g. many are moved from `packages/embark/package.json` to `packages/core/engine/package.json`. Related to those moves, fix some Node.js `require`-logic related to bug-prone dependency resolution. Fix all type errors that appeared as a result of activating collective typecheck across the whole monorepo. Reactivate `tslint` in `packages/core/core` and fix the remaining linter errors. Tidy up and add a few items in the root `package.json` scripts. Bump lerna from `3.16.4` to `3.19.0`. Bumpt typescript from `3.6.3` to `3.7.2`. Bumpt tslint from `5.16.0` to `5.20.1`. Make various changes related to packages' `import`/`require`ing packages that weren't spec'd in their respective `package.json`. More refactoring is needed in this regard, but changes were made as the problems were observed in the process of authoring this PR. [config]: https://www.typescriptlang.org/docs/handbook/tsconfig-json.html	2019-12-13 13:59:47 -05:00
Eric Mastro	e2efbbd846	refactor(@embark/parity): move module into own package (#1951 ) Contains bug fixes to get parity to work as a blockchain client. NOTE: Please merge https://github.com/embark-framework/embark/pull/1950 first before merging this PR, as this PR contains removal of dependencies from `packages/embark` that are needed for geth. So if this is merged first, and the geth PR (https://github.com/embark-framework/embark/pull/1950) is not merged, then geth will not have some of the dependencies it needs.	2019-10-16 09:35:26 -04:00

Author

SHA1

Message

Date

Michael Bradley, Jr

3693ebd90d

fix: ensure that packages properly specify their dependencies

Many packages in the monorepo did not specify all of their dependencies; they
were effectively relying on resolution in the monorepo's root
`node_modules`. In a production release of `embark` and `embark[js]-*` packages
this can lead to broken packages.

To fix the problem currently and to help prevent it from happening again, make
use of the `eslint-plugin-import` package's `import/no-extraneous-dependencies`
and `import/no-unresolved` rules. In the root `tslint.json` set
`"no-implicit-dependencies": true`, wich is the tslint equivalent of
`import/no-extraneous-dependencies`; there is no tslint equivalent for
`import/no-unresolved`, but we will eventually replace tslint with an eslint
configuration that checks both `.js` and `.ts` files.

For `import/no-unresolved` to work in our monorepo setup, in most packages add
an `index.js` that has:

```js
module.exports = require('./dist'); // or './dist/lib' in some cases
```

And point `"main"` in `package.json` to `"./index.js"`. Despite what's
indicated in npm's documentation for `package.json`, it's also necessary to add
`"index.js"` to the `"files"` array.

Make sure that all `.js` files that can and should be linted are in fact
linted. For example, files in `packages/embark/src/cmd/` weren't being linted
and many test suites weren't being linted.

Bump all relevant packages to `eslint@6.8.0`.

Fix all linter errors that arose after these changes.

Implement a `check-yarn-lock` script that's run as part of `"ci:full"` and
`"qa:full"`, and can manually be invoked via `yarn cylock` in the root of the
monorepo. The script exits with error if any specifiers are found in
`yarn.lock` for `embark[js][-*]` and/or `@embarklabs/*` (with a few exceptions,
cf. `scripts/check-yarn-lock.js`).

2020-02-25 14:52:10 -06:00

Michael Bradley, Jr

ee56f37713

build: implement collective typecheck

This PR replaces #2057.

Implement a collective typecheck action that can be invoked in the root of the
monorepo with `yarn typecheck` or in watch-mode with `yarn watch:typecheck`.
Include the watch-mode typecheck action as part of `yarn start` (a.k.a
`yarn watch`).

To activate collective typecheck for a package in the monorepo, its
`package.json` file should specify:

```
{
  "embark-collective": {
    "typecheck": true
  }
}
```
*-or-*
```
{
  "embark-collective": {
    "typecheck": {...}
  }
}
```

Where `{...}` above is a `tsconfig.json` fragment that will be merged into the
config generated for the package according the same rules that `tsc` applies
when merging [configs][config].

When collective typecheck begins, it generates a `tsconfig.json` for the root
of the monorepo and for each package that is activated for the action. If the
generated JSON is different than what's on disk for the respective root/package
config, or if the config is not present on disk, then it will be
written. Changes to generated `tsconfig.json` files should be committed; such
changes will arise when there are structural changes to the monorepo, e.g. a
package is added, removed, moved and/or the directory containing it is
renamed. Since the configs are only generated at the beginning of collective
typecheck, when structural changes are made in the monorepo `yarn typecheck`
(or `yarn start` or `yarn watch:typecheck`) should be restarted.

Nearly all of the packages in the monorepo (i.e. all those for which it makes
sense) have been activated for collective typecheck. Even those packages that
don't contain `.ts` sources are activated because `tsc` can make better sense
of the code base as a whole owing to the project references included in the
generated `tsconfig.json` files. Also, owing to the fully cross-referenced
`tsconfig.json` files, it's possible for `tsc` to type check the whole code
base without babel (`yarn build` or `yarn watch:build`) having been run
beforehand.

**NOTE** that a *"cold typecheck"* of the whole monorepo is resource intensive:
on this author's 2019 MacBook Pro it takes around three minutes, the fans spin
up, and `tsc` uses nearly 0.5 GB of RAM. However, once a full typecheck has
completed, the next full typecheck will complete in a few seconds or less; and
when running in watch-mode there is likewise a *big* speedup once a full
typecheck has completed, whether that full check happened before it's running
in watch-mode or when watch-mode itself resulted in a full check before
switching automatically to incremental check, as well a corresponding *big*
reduction in resource consumption. A full check will be needed any time
`yarn typecheck` (or `yarn start` or `yarn watch:typecheck`) is run in a fresh
clone plus `yarn install`, or after doing `yarn reboot[:full]` or `yarn reset`.

The combination of options in each generated package-level `tsconfig.json` and
the root `tsconfig.base.json` result in `tsc` writing `.d.ts` files (TypeScript
declaration files) into the `dist/` directory of each package. That
output is intended to live side-by-side with babel's output, and therefore the
`"rootDir"` option in each generated config is set to `"./src"`.

In projects activated for collective typecheck, `.js` may be converted to `.ts`
and/or `.ts` sources may be added without any additional changes needed in
package-level `package.json`.

---

Reorganize types in `packages/core/typings` (a.k.a `@types/embark`) into
`packages/core/core` (`embark-core`), refactor other packages' imports
accordingly, and delete `packages/core/typings` from the monorepo. This results
in some similarly named but incompatible types exported from `embark-core`
(e.g. `Events` and `EmbarkEvents`, the latter being the one from
`packages/core/typings`); future refactoring should consolidate those types. To
avoid circular dependency relationships it's also necessary to split out
`Engine` from `embark-core` into its own package (`embark-engine`) and to
introduce a bit of duplication, e.g. the `Maybe` type that's now defined in
both `embark-i18n` and `embark-core`.

In the process of the types reorg, move many dependencies spec'd in various
`package.json` to the `package.json` of the package/s that actually depend on
them, e.g. many are moved from `packages/embark/package.json` to
`packages/core/engine/package.json`. Related to those moves, fix some Node.js
`require`-logic related to bug-prone dependency resolution.

Fix all type errors that appeared as a result of activating collective
typecheck across the whole monorepo.

Reactivate `tslint` in `packages/core/core` and fix the remaining linter errors.

Tidy up and add a few items in the root `package.json` scripts.

Bump lerna from `3.16.4` to `3.19.0`.

Bumpt typescript from `3.6.3` to `3.7.2`.

Bumpt tslint from `5.16.0` to `5.20.1`.

Make various changes related to packages' `import`/`require`ing packages that
weren't spec'd in their respective `package.json`. More refactoring is needed
in this regard, but changes were made as the problems were observed in the
process of authoring this PR.

[config]: https://www.typescriptlang.org/docs/handbook/tsconfig-json.html

2019-12-13 13:59:47 -05:00

Eric Mastro

e2efbbd846

refactor(@embark/parity): move module into own package (#1951 )

Contains bug fixes to get parity to work as a blockchain client.

**NOTE:** Please merge https://github.com/embark-framework/embark/pull/1950 first before merging this PR, as this PR contains removal of dependencies from `packages/embark` that are needed for geth. So if this is merged first, and the geth PR (https://github.com/embark-framework/embark/pull/1950) is not merged, then geth will not have some of the dependencies it needs.

2019-10-16 09:35:26 -04:00

3 Commits