embark/scripts/stable-release.js

335 lines
10 KiB
JavaScript
Raw Normal View History

ci: implement a nightlies GitHub Actions workflow Implement a GitHub Actions workflow in `.github/workflows/nightlies.yml` named *Nightlies*, which is scheduled to run once daily at 00:00 UTC. At present the workflow includes one job named *release*, which is responsible for publishing prerelease GitHub releases and NPM packages. Each prerelease created (per package) will have a `nightly` [semver identifier][preid], and each successive nightly release will be paired with the `nightly` [dist-tag][dist-tag] on the NPM registry (per package). During the release job, actions taken in this GitHub repository (commits, tags, releases) and on the NPM registry (package publication) will be performed using credentials associated with the following accounts: * https://github.com/embarkbot * https://www.npmjs.com/~embarkbot For that purpose, corresponding [secrets][secrets] (link requires admin access) were created in this repository consisting of API tokens generated for the @embarkbot GitHub and NPM accounts. Logins for the @embarkbot accounts themselves are protected by 2FA. Implement `scripts/nightly-release.js` (`npm run release:nightly`), which is responsible for running `lerna publish` in the GitHub Actions workflow. Also implement `scripts/stable-release.js` (`npm run release:stable`), which is intended to be run locally by someone on the Embark Team. Both scripts borrow heavily from the existing `scripts/release.js`, and the process of authoring and experimenting with them influenced refactors to the latter. Use a `--force-publish` major-release strategy to prevent major-version drift between packages in the monorepo. How it works: when the stable-release script is run (`npm run release:stable`), if the current prerelease version involves a major version increase relative to the most recent stable release then **all** packages are bumped to the new major stable version. Otherwise, only the packages currently in prerelease are graduated to the new minor/patch stable version. In either case, the `nightly` dist-tag of each package published is updated to resolve to the new stable version. The reason for adopting this strategy *(a decision which can be revisited and changed any time in the future)* is based on a concern that downstream users would have a confusing developer UX if across `embark-*` packages there are differing major versions. To understand how the major-version drift would happen, consider the following hypothetical scenario where `--force-publish` *isn't* used in stable releases and `nightly` dist-tags aren't updated to resolve to the latest stable version: assume the current stable version is `6.5.4`. A breaking change lands for `embark-core`. The next nightly release bumps `embark-core` and about 40 other packages to `7.0.0-nightly.0`. However, `embark-utils` (and others) isn't bumped because it doesn't depend on `embark-core`. Later, without any intervening changes to `embark-utils`, the prerelease is graduated so that `embark-core`, etc. bump to `7.0.0`. So then some `embark-*` packages are at major version `7` while others are still at `6`. *Note* that this is the case even though this monorepo uses Lerna's *"fixed"* versioning mode. Inside the monorepo, `lerna` makes sure that everything is okay, i.e. with respect to automatically updating dependents' version specifiers for their dependencies that are within the monorepo. But for downstream users things are a bit more complex. If someone wanted to use `embark-utils` on its own and specified `^7.0.0` as the version range (after observing that `embark` itself is in a `7.x` series) it won't work because `embark-utils` is still in `6.x`. In the general case, users may have to manually cross-check major versions of various `embark-*` packages that they specify in their projects' `package.json` files. There are tools like [npm-check-updates][ncu] that can make the task easier, but there is still likely to be some confusion, especially given the large and growing number of packages in this monorepo. Another area of confusion would exist around the `nightly` dist-tag. In the scenario above, `embark-core@nightly` (and/or `@nightly` of its dependents, e.g. `embark`) would resolve to `7.0.0-nightly.0` but `embark-utils@nightly` would resolve to some `6.5.4-nightly.N` (:bomb:), i.e. a prerelease version that predates the current stable `6.5.4` release of `embark-utils` (and *might* not include all changes that landed in `embark-utils` prior to that stable release). By bumping all packages each time there is a major stable release, and by having the `nightly` dist-tag always point to a package's most recent release (whether stable or prerelease), the problems described above are avoided. To see the `--force-publish` major-release strategy in action take a look at the [commit history][history] for the [nightly-release-workflow-tester][mb-nrwt] repo together with the *Versions* tab of the NPM pages for the [foo][foo], [bar][bar], [baz][baz], and [quux][quux] packages. Ignore the version history for `<= 2.0.1` because those pre/releases were made with a different strategy than the current one. Refactor the existing `scripts/release.js` to make it more flexible generally and with respect to options that can be forwarded to `lerna`. In particular, it's now possible to run `lerna version` instead of `lerna publish` (the default behavior) by using the `--version-only` cli option; when combining that usage with `--skip-qa` and `--no-push` it's possible to conveniently and quickly experiment with the [`bump` positional][bump] and additional options such as `--force-publish`, `--conventional-prerelease`, and `--conventional-graduate`, i.e. to better understand how `lerna` will update package versions. That ability should make it much simpler to figure out the best course of action to take locally (manually) when a nightly release completely or partially failed (which could happen for a number of reasons), as well for other scenarios such as making a minor/patch release in a previous line of major releases, or when making two/more successive stable releases without a nightly release having happened in the meantime. An important change to `scripts/release.js` is that by default it makes use of the `--create-release github` option for `lerna version|publish`. For that to work, an environment variable named `GH_TOKEN` must be defined with a properly [scoped][scopes] GitHub [personal access token][pa-token] (`public_repo` scope is sufficient for creating releases). The same is true for `scripts/stable-release.js`. Delete the `.github/PULL_REQUEST_TEMPLATE` directory and the templates it contained. Unlike for GitHub issue creation, there is no prompt-page for picking from a repo's PR templates; to use a PR template a `template=[name]` [query parameter][template-query] must be appended to the URL of the PR creation page. So the PR templates ended up unused by the Embark Team and external contributors because it's not convenient to use them. Restore the default PR template we had in place some time ago (with some small revisions) since it seems like a helpful starting point, especially for external contributors. Consistently use all-lowercase filenames for ISSUE/PR templates. [preid]: https://semver.org/#spec-item-9 [dist-tag]: https://docs.npmjs.com/cli/dist-tag [secrets]: https://github.com/embarklabs/embark/settings/secrets [ncu]: https://www.npmjs.com/package/npm-check-updates [history]: https://github.com/michaelsbradleyjr/nightly-release-workflow-tester/commits/master [mb-nrwt]: https://github.com/michaelsbradleyjr/nightly-release-workflow-tester/ [foo]: https://www.npmjs.com/package/nightly-release-workflow-tester-foo?activeTab=versions [bar]: https://www.npmjs.com/package/nightly-release-workflow-tester-bar?activeTab=versions [baz]: https://www.npmjs.com/package/nightly-release-workflow-tester-baz?activeTab=versions [quux]: https://www.npmjs.com/package/nightly-release-workflow-tester-quux?activeTab=versions [bump]: https://github.com/lerna/lerna/tree/master/commands/version#semver-bump [scopes]: https://developer.github.com/apps/building-oauth-apps/understanding-scopes-for-oauth-apps/ [pa-token]: https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line [template-query]: https://help.github.com/en/github/building-a-strong-community/creating-a-pull-request-template-for-your-repository
2020-01-07 17:09:38 +00:00
const chalk = require('chalk');
const {execSync} = require('child_process');
const minimist = require('minimist');
const path = require('path');
const {readJsonSync} = require('fs-extra');
const semver = require('semver');
const args = minimist(process.argv.slice(2));
const DEFAULT_SKIP_QA = false;
const skipQa = args['skip-qa'] || DEFAULT_SKIP_QA;
const branch = `master`;
const commitMsg = `chore(release): %v`;
const distTag = `latest`;
const remote = `origin`;
const cyan = (str) => chalk.cyan(str);
const execSyncInherit = (cmd) => execSync(cmd, {stdio: 'inherit'});
const log = (mark, str, which = 'log') => console[which](
mark, str.filter(s => !!s).join(` `)
);
const logError = (...str) => log(chalk.red(``), str, 'error');
const logInfo = (...str) => log(chalk.blue(``), str);
const logSuccess = (...str) => log(chalk.green(``), str);
const logWarning = (...str) => log(chalk.yellow('‼︎'), str);
const failMsg = `${chalk.red(`STABLE RELEASE FAILED!`)} Stopping right here.`;
const runCommand = (cmd, inherit = true, display) => {
logInfo(`Running command ${cyan(display || cmd)}.`);
let out;
if (inherit) {
execSyncInherit(cmd);
} else {
out = execSync(cmd);
}
return out;
};
fix: ensure that packages properly specify their dependencies Many packages in the monorepo did not specify all of their dependencies; they were effectively relying on resolution in the monorepo's root `node_modules`. In a production release of `embark` and `embark[js]-*` packages this can lead to broken packages. To fix the problem currently and to help prevent it from happening again, make use of the `eslint-plugin-import` package's `import/no-extraneous-dependencies` and `import/no-unresolved` rules. In the root `tslint.json` set `"no-implicit-dependencies": true`, wich is the tslint equivalent of `import/no-extraneous-dependencies`; there is no tslint equivalent for `import/no-unresolved`, but we will eventually replace tslint with an eslint configuration that checks both `.js` and `.ts` files. For `import/no-unresolved` to work in our monorepo setup, in most packages add an `index.js` that has: ```js module.exports = require('./dist'); // or './dist/lib' in some cases ``` And point `"main"` in `package.json` to `"./index.js"`. Despite what's indicated in npm's documentation for `package.json`, it's also necessary to add `"index.js"` to the `"files"` array. Make sure that all `.js` files that can and should be linted are in fact linted. For example, files in `packages/embark/src/cmd/` weren't being linted and many test suites weren't being linted. Bump all relevant packages to `eslint@6.8.0`. Fix all linter errors that arose after these changes. Implement a `check-yarn-lock` script that's run as part of `"ci:full"` and `"qa:full"`, and can manually be invoked via `yarn cylock` in the root of the monorepo. The script exits with error if any specifiers are found in `yarn.lock` for `embark[js][-*]` and/or `@embarklabs/*` (with a few exceptions, cf. `scripts/check-yarn-lock.js`).
2020-02-20 23:47:01 +00:00
// eslint-disable-next-line complexity
ci: implement a nightlies GitHub Actions workflow Implement a GitHub Actions workflow in `.github/workflows/nightlies.yml` named *Nightlies*, which is scheduled to run once daily at 00:00 UTC. At present the workflow includes one job named *release*, which is responsible for publishing prerelease GitHub releases and NPM packages. Each prerelease created (per package) will have a `nightly` [semver identifier][preid], and each successive nightly release will be paired with the `nightly` [dist-tag][dist-tag] on the NPM registry (per package). During the release job, actions taken in this GitHub repository (commits, tags, releases) and on the NPM registry (package publication) will be performed using credentials associated with the following accounts: * https://github.com/embarkbot * https://www.npmjs.com/~embarkbot For that purpose, corresponding [secrets][secrets] (link requires admin access) were created in this repository consisting of API tokens generated for the @embarkbot GitHub and NPM accounts. Logins for the @embarkbot accounts themselves are protected by 2FA. Implement `scripts/nightly-release.js` (`npm run release:nightly`), which is responsible for running `lerna publish` in the GitHub Actions workflow. Also implement `scripts/stable-release.js` (`npm run release:stable`), which is intended to be run locally by someone on the Embark Team. Both scripts borrow heavily from the existing `scripts/release.js`, and the process of authoring and experimenting with them influenced refactors to the latter. Use a `--force-publish` major-release strategy to prevent major-version drift between packages in the monorepo. How it works: when the stable-release script is run (`npm run release:stable`), if the current prerelease version involves a major version increase relative to the most recent stable release then **all** packages are bumped to the new major stable version. Otherwise, only the packages currently in prerelease are graduated to the new minor/patch stable version. In either case, the `nightly` dist-tag of each package published is updated to resolve to the new stable version. The reason for adopting this strategy *(a decision which can be revisited and changed any time in the future)* is based on a concern that downstream users would have a confusing developer UX if across `embark-*` packages there are differing major versions. To understand how the major-version drift would happen, consider the following hypothetical scenario where `--force-publish` *isn't* used in stable releases and `nightly` dist-tags aren't updated to resolve to the latest stable version: assume the current stable version is `6.5.4`. A breaking change lands for `embark-core`. The next nightly release bumps `embark-core` and about 40 other packages to `7.0.0-nightly.0`. However, `embark-utils` (and others) isn't bumped because it doesn't depend on `embark-core`. Later, without any intervening changes to `embark-utils`, the prerelease is graduated so that `embark-core`, etc. bump to `7.0.0`. So then some `embark-*` packages are at major version `7` while others are still at `6`. *Note* that this is the case even though this monorepo uses Lerna's *"fixed"* versioning mode. Inside the monorepo, `lerna` makes sure that everything is okay, i.e. with respect to automatically updating dependents' version specifiers for their dependencies that are within the monorepo. But for downstream users things are a bit more complex. If someone wanted to use `embark-utils` on its own and specified `^7.0.0` as the version range (after observing that `embark` itself is in a `7.x` series) it won't work because `embark-utils` is still in `6.x`. In the general case, users may have to manually cross-check major versions of various `embark-*` packages that they specify in their projects' `package.json` files. There are tools like [npm-check-updates][ncu] that can make the task easier, but there is still likely to be some confusion, especially given the large and growing number of packages in this monorepo. Another area of confusion would exist around the `nightly` dist-tag. In the scenario above, `embark-core@nightly` (and/or `@nightly` of its dependents, e.g. `embark`) would resolve to `7.0.0-nightly.0` but `embark-utils@nightly` would resolve to some `6.5.4-nightly.N` (:bomb:), i.e. a prerelease version that predates the current stable `6.5.4` release of `embark-utils` (and *might* not include all changes that landed in `embark-utils` prior to that stable release). By bumping all packages each time there is a major stable release, and by having the `nightly` dist-tag always point to a package's most recent release (whether stable or prerelease), the problems described above are avoided. To see the `--force-publish` major-release strategy in action take a look at the [commit history][history] for the [nightly-release-workflow-tester][mb-nrwt] repo together with the *Versions* tab of the NPM pages for the [foo][foo], [bar][bar], [baz][baz], and [quux][quux] packages. Ignore the version history for `<= 2.0.1` because those pre/releases were made with a different strategy than the current one. Refactor the existing `scripts/release.js` to make it more flexible generally and with respect to options that can be forwarded to `lerna`. In particular, it's now possible to run `lerna version` instead of `lerna publish` (the default behavior) by using the `--version-only` cli option; when combining that usage with `--skip-qa` and `--no-push` it's possible to conveniently and quickly experiment with the [`bump` positional][bump] and additional options such as `--force-publish`, `--conventional-prerelease`, and `--conventional-graduate`, i.e. to better understand how `lerna` will update package versions. That ability should make it much simpler to figure out the best course of action to take locally (manually) when a nightly release completely or partially failed (which could happen for a number of reasons), as well for other scenarios such as making a minor/patch release in a previous line of major releases, or when making two/more successive stable releases without a nightly release having happened in the meantime. An important change to `scripts/release.js` is that by default it makes use of the `--create-release github` option for `lerna version|publish`. For that to work, an environment variable named `GH_TOKEN` must be defined with a properly [scoped][scopes] GitHub [personal access token][pa-token] (`public_repo` scope is sufficient for creating releases). The same is true for `scripts/stable-release.js`. Delete the `.github/PULL_REQUEST_TEMPLATE` directory and the templates it contained. Unlike for GitHub issue creation, there is no prompt-page for picking from a repo's PR templates; to use a PR template a `template=[name]` [query parameter][template-query] must be appended to the URL of the PR creation page. So the PR templates ended up unused by the Embark Team and external contributors because it's not convenient to use them. Restore the default PR template we had in place some time ago (with some small revisions) since it seems like a helpful starting point, especially for external contributors. Consistently use all-lowercase filenames for ISSUE/PR templates. [preid]: https://semver.org/#spec-item-9 [dist-tag]: https://docs.npmjs.com/cli/dist-tag [secrets]: https://github.com/embarklabs/embark/settings/secrets [ncu]: https://www.npmjs.com/package/npm-check-updates [history]: https://github.com/michaelsbradleyjr/nightly-release-workflow-tester/commits/master [mb-nrwt]: https://github.com/michaelsbradleyjr/nightly-release-workflow-tester/ [foo]: https://www.npmjs.com/package/nightly-release-workflow-tester-foo?activeTab=versions [bar]: https://www.npmjs.com/package/nightly-release-workflow-tester-bar?activeTab=versions [baz]: https://www.npmjs.com/package/nightly-release-workflow-tester-baz?activeTab=versions [quux]: https://www.npmjs.com/package/nightly-release-workflow-tester-quux?activeTab=versions [bump]: https://github.com/lerna/lerna/tree/master/commands/version#semver-bump [scopes]: https://developer.github.com/apps/building-oauth-apps/understanding-scopes-for-oauth-apps/ [pa-token]: https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line [template-query]: https://help.github.com/en/github/building-a-strong-community/creating-a-pull-request-template-for-your-repository
2020-01-07 17:09:38 +00:00
(async () => {
if (!process.env.GH_TOKEN) {
logError(
`Environment variable ${cyan('GH_TOKEN')} was not defined or falsy. It`,
`must be defined with a properly scoped GitHub personal access token.`
);
logError(failMsg);
process.exit(1);
}
const lernaJsonPath = path.join(__dirname, '../lerna.json');
logInfo(`Reading ${cyan(lernaJsonPath)}...`);
let currentVersion, lernaJson, registry;
try {
lernaJson = readJsonSync(lernaJsonPath);
currentVersion = lernaJson.version;
if (!currentVersion) throw new Error('missing version in lerna.json');
registry = lernaJson.command.publish.registry;
if (!registry) throw new Error('missing registry in lerna.json');
} catch (e) {
console.error(e.stack || e);
logError(
`Could not read values from ${cyan(lernaJsonPath)}. Please check the`,
`error above.`
);
logError(failMsg);
process.exit(1);
}
try {
if (!semver(currentVersion).prerelease.length) {
logError(
`Current version in ${cyan('lerna.json')} is not a prerelease. This`,
`script is intended only for graduating a prerelease to a stable`,
`release.`
);
logError(failMsg);
process.exit(1);
}
logSuccess(
`Current version in ${cyan('lerna.json')} is ${cyan(currentVersion)}.`
);
} catch (e) {
logError(`A problem occured. Please check the error above.`);
logError(failMsg);
process.exit(1);
}
logInfo(`Determining the latest stable version...`);
let latestStableVersion;
try {
const stableReleaseTags = runCommand(`git tag --list`, false)
.toString()
.trim()
.split('\n')
.filter(tag => tag.startsWith('v') && !tag.includes('-'))
.map(tag => tag.slice(1));
latestStableVersion = semver.rsort(stableReleaseTags)[0];
if (!latestStableVersion) {
logError(`Unable to determine the latest stable version.`);
logError(failMsg);
process.exit(1);
}
logSuccess(`Latest stable version is ${cyan(latestStableVersion)}.`);
} catch (e) {
logError(`A problem occured. Please check the error above.`);
logError(failMsg);
process.exit(1);
}
let forcePublish;
try {
const currentMajor = semver(currentVersion).major;
const stableMajor = semver(latestStableVersion).major;
if (currentMajor > stableMajor) {
forcePublish = true;
} else if (currentMajor < stableMajor) {
logError(
`Major version of the current version in ${cyan('lerna.json')} is less`,
`than the major version of the latest stable version. wat.`
);
logError(failMsg);
process.exit(1);
}
} catch (e) {
logError(`A problem occured. Please check the error above.`);
logError(failMsg);
process.exit(1);
}
logInfo(`Determining the current branch...`);
let currentBranch;
try {
currentBranch = runCommand(`git rev-parse --abbrev-ref HEAD`, false)
.toString()
.trim();
} catch (e) {
logError(`Could not determine the branch. Please check the error above.`);
logError(failMsg);
process.exit(1);
}
if (currentBranch === branch) {
logSuccess(`Current branch and release branch are the same.`);
} else {
logError(
`Current branch ${cyan(currentBranch)} is not the same as release branch`,
`${cyan(branch)}.`
);
logError(failMsg);
process.exit(1);
}
logInfo(
`Fetching commits from ${cyan(remote)} to compare local and remote`,
`branches...`
);
try {
runCommand(`git fetch ${remote}`, false);
} catch (e) {
logError(`Could not fetch latest commits. Please check the error above.`);
logError(failMsg);
process.exit(1);
}
let localRef, remoteRef;
try {
localRef = runCommand(`git rev-parse ${branch}`, false).toString().trim();
remoteRef = runCommand(`git rev-parse ${remote}/${branch}`, false)
.toString()
.trim();
} catch (e) {
logError(`A problem occured. Please check the error above.`);
logError(failMsg);
process.exit(1);
}
if (localRef === remoteRef) {
logSuccess(`Local branch is in sync with remote branch.`);
} else {
logError(
`Local branch ${cyan(branch)} is not in sync with`,
`${cyan(`${remote}/${branch}`)}.`
);
logError(failMsg);
process.exit(1);
}
if (skipQa) {
logWarning(
`Skipping the QA suite. You already built the packages, right?`
);
} else {
logInfo(
`It's time to run the QA suite, this will take awhile...`
);
try {
runCommand(`npm run qa:full`);
logSuccess(`All steps succeeded in the QA suite.`);
} catch (e) {
logError(`A step failed in the QA suite. Please check the error above.`);
logError(failMsg);
process.exit(1);
}
}
// The `--all` option is not supplied below because `lerna changed` is being
// used to calculate which packages should be tagged via:
// `npm dist-tag add [pkg]@[newStableVersion] nightly`
// Private packages are never tagged because dist-tags only pertain to
// packages published to the NPM registry. By tagging every newly released
// stable version of public packages as `nightly` (in addition to tagging as
// `latest`) we avoid version drift in nightly releases
ci: implement a nightlies GitHub Actions workflow Implement a GitHub Actions workflow in `.github/workflows/nightlies.yml` named *Nightlies*, which is scheduled to run once daily at 00:00 UTC. At present the workflow includes one job named *release*, which is responsible for publishing prerelease GitHub releases and NPM packages. Each prerelease created (per package) will have a `nightly` [semver identifier][preid], and each successive nightly release will be paired with the `nightly` [dist-tag][dist-tag] on the NPM registry (per package). During the release job, actions taken in this GitHub repository (commits, tags, releases) and on the NPM registry (package publication) will be performed using credentials associated with the following accounts: * https://github.com/embarkbot * https://www.npmjs.com/~embarkbot For that purpose, corresponding [secrets][secrets] (link requires admin access) were created in this repository consisting of API tokens generated for the @embarkbot GitHub and NPM accounts. Logins for the @embarkbot accounts themselves are protected by 2FA. Implement `scripts/nightly-release.js` (`npm run release:nightly`), which is responsible for running `lerna publish` in the GitHub Actions workflow. Also implement `scripts/stable-release.js` (`npm run release:stable`), which is intended to be run locally by someone on the Embark Team. Both scripts borrow heavily from the existing `scripts/release.js`, and the process of authoring and experimenting with them influenced refactors to the latter. Use a `--force-publish` major-release strategy to prevent major-version drift between packages in the monorepo. How it works: when the stable-release script is run (`npm run release:stable`), if the current prerelease version involves a major version increase relative to the most recent stable release then **all** packages are bumped to the new major stable version. Otherwise, only the packages currently in prerelease are graduated to the new minor/patch stable version. In either case, the `nightly` dist-tag of each package published is updated to resolve to the new stable version. The reason for adopting this strategy *(a decision which can be revisited and changed any time in the future)* is based on a concern that downstream users would have a confusing developer UX if across `embark-*` packages there are differing major versions. To understand how the major-version drift would happen, consider the following hypothetical scenario where `--force-publish` *isn't* used in stable releases and `nightly` dist-tags aren't updated to resolve to the latest stable version: assume the current stable version is `6.5.4`. A breaking change lands for `embark-core`. The next nightly release bumps `embark-core` and about 40 other packages to `7.0.0-nightly.0`. However, `embark-utils` (and others) isn't bumped because it doesn't depend on `embark-core`. Later, without any intervening changes to `embark-utils`, the prerelease is graduated so that `embark-core`, etc. bump to `7.0.0`. So then some `embark-*` packages are at major version `7` while others are still at `6`. *Note* that this is the case even though this monorepo uses Lerna's *"fixed"* versioning mode. Inside the monorepo, `lerna` makes sure that everything is okay, i.e. with respect to automatically updating dependents' version specifiers for their dependencies that are within the monorepo. But for downstream users things are a bit more complex. If someone wanted to use `embark-utils` on its own and specified `^7.0.0` as the version range (after observing that `embark` itself is in a `7.x` series) it won't work because `embark-utils` is still in `6.x`. In the general case, users may have to manually cross-check major versions of various `embark-*` packages that they specify in their projects' `package.json` files. There are tools like [npm-check-updates][ncu] that can make the task easier, but there is still likely to be some confusion, especially given the large and growing number of packages in this monorepo. Another area of confusion would exist around the `nightly` dist-tag. In the scenario above, `embark-core@nightly` (and/or `@nightly` of its dependents, e.g. `embark`) would resolve to `7.0.0-nightly.0` but `embark-utils@nightly` would resolve to some `6.5.4-nightly.N` (:bomb:), i.e. a prerelease version that predates the current stable `6.5.4` release of `embark-utils` (and *might* not include all changes that landed in `embark-utils` prior to that stable release). By bumping all packages each time there is a major stable release, and by having the `nightly` dist-tag always point to a package's most recent release (whether stable or prerelease), the problems described above are avoided. To see the `--force-publish` major-release strategy in action take a look at the [commit history][history] for the [nightly-release-workflow-tester][mb-nrwt] repo together with the *Versions* tab of the NPM pages for the [foo][foo], [bar][bar], [baz][baz], and [quux][quux] packages. Ignore the version history for `<= 2.0.1` because those pre/releases were made with a different strategy than the current one. Refactor the existing `scripts/release.js` to make it more flexible generally and with respect to options that can be forwarded to `lerna`. In particular, it's now possible to run `lerna version` instead of `lerna publish` (the default behavior) by using the `--version-only` cli option; when combining that usage with `--skip-qa` and `--no-push` it's possible to conveniently and quickly experiment with the [`bump` positional][bump] and additional options such as `--force-publish`, `--conventional-prerelease`, and `--conventional-graduate`, i.e. to better understand how `lerna` will update package versions. That ability should make it much simpler to figure out the best course of action to take locally (manually) when a nightly release completely or partially failed (which could happen for a number of reasons), as well for other scenarios such as making a minor/patch release in a previous line of major releases, or when making two/more successive stable releases without a nightly release having happened in the meantime. An important change to `scripts/release.js` is that by default it makes use of the `--create-release github` option for `lerna version|publish`. For that to work, an environment variable named `GH_TOKEN` must be defined with a properly [scoped][scopes] GitHub [personal access token][pa-token] (`public_repo` scope is sufficient for creating releases). The same is true for `scripts/stable-release.js`. Delete the `.github/PULL_REQUEST_TEMPLATE` directory and the templates it contained. Unlike for GitHub issue creation, there is no prompt-page for picking from a repo's PR templates; to use a PR template a `template=[name]` [query parameter][template-query] must be appended to the URL of the PR creation page. So the PR templates ended up unused by the Embark Team and external contributors because it's not convenient to use them. Restore the default PR template we had in place some time ago (with some small revisions) since it seems like a helpful starting point, especially for external contributors. Consistently use all-lowercase filenames for ISSUE/PR templates. [preid]: https://semver.org/#spec-item-9 [dist-tag]: https://docs.npmjs.com/cli/dist-tag [secrets]: https://github.com/embarklabs/embark/settings/secrets [ncu]: https://www.npmjs.com/package/npm-check-updates [history]: https://github.com/michaelsbradleyjr/nightly-release-workflow-tester/commits/master [mb-nrwt]: https://github.com/michaelsbradleyjr/nightly-release-workflow-tester/ [foo]: https://www.npmjs.com/package/nightly-release-workflow-tester-foo?activeTab=versions [bar]: https://www.npmjs.com/package/nightly-release-workflow-tester-bar?activeTab=versions [baz]: https://www.npmjs.com/package/nightly-release-workflow-tester-baz?activeTab=versions [quux]: https://www.npmjs.com/package/nightly-release-workflow-tester-quux?activeTab=versions [bump]: https://github.com/lerna/lerna/tree/master/commands/version#semver-bump [scopes]: https://developer.github.com/apps/building-oauth-apps/understanding-scopes-for-oauth-apps/ [pa-token]: https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line [template-query]: https://help.github.com/en/github/building-a-strong-community/creating-a-pull-request-template-for-your-repository
2020-01-07 17:09:38 +00:00
const lernaChanged = [
`lerna`,
`changed`,
(!forcePublish && `--conventional-graduate`) || ``,
(forcePublish && `--force-publish`) || ``,
`--json`
ci: implement a nightlies GitHub Actions workflow Implement a GitHub Actions workflow in `.github/workflows/nightlies.yml` named *Nightlies*, which is scheduled to run once daily at 00:00 UTC. At present the workflow includes one job named *release*, which is responsible for publishing prerelease GitHub releases and NPM packages. Each prerelease created (per package) will have a `nightly` [semver identifier][preid], and each successive nightly release will be paired with the `nightly` [dist-tag][dist-tag] on the NPM registry (per package). During the release job, actions taken in this GitHub repository (commits, tags, releases) and on the NPM registry (package publication) will be performed using credentials associated with the following accounts: * https://github.com/embarkbot * https://www.npmjs.com/~embarkbot For that purpose, corresponding [secrets][secrets] (link requires admin access) were created in this repository consisting of API tokens generated for the @embarkbot GitHub and NPM accounts. Logins for the @embarkbot accounts themselves are protected by 2FA. Implement `scripts/nightly-release.js` (`npm run release:nightly`), which is responsible for running `lerna publish` in the GitHub Actions workflow. Also implement `scripts/stable-release.js` (`npm run release:stable`), which is intended to be run locally by someone on the Embark Team. Both scripts borrow heavily from the existing `scripts/release.js`, and the process of authoring and experimenting with them influenced refactors to the latter. Use a `--force-publish` major-release strategy to prevent major-version drift between packages in the monorepo. How it works: when the stable-release script is run (`npm run release:stable`), if the current prerelease version involves a major version increase relative to the most recent stable release then **all** packages are bumped to the new major stable version. Otherwise, only the packages currently in prerelease are graduated to the new minor/patch stable version. In either case, the `nightly` dist-tag of each package published is updated to resolve to the new stable version. The reason for adopting this strategy *(a decision which can be revisited and changed any time in the future)* is based on a concern that downstream users would have a confusing developer UX if across `embark-*` packages there are differing major versions. To understand how the major-version drift would happen, consider the following hypothetical scenario where `--force-publish` *isn't* used in stable releases and `nightly` dist-tags aren't updated to resolve to the latest stable version: assume the current stable version is `6.5.4`. A breaking change lands for `embark-core`. The next nightly release bumps `embark-core` and about 40 other packages to `7.0.0-nightly.0`. However, `embark-utils` (and others) isn't bumped because it doesn't depend on `embark-core`. Later, without any intervening changes to `embark-utils`, the prerelease is graduated so that `embark-core`, etc. bump to `7.0.0`. So then some `embark-*` packages are at major version `7` while others are still at `6`. *Note* that this is the case even though this monorepo uses Lerna's *"fixed"* versioning mode. Inside the monorepo, `lerna` makes sure that everything is okay, i.e. with respect to automatically updating dependents' version specifiers for their dependencies that are within the monorepo. But for downstream users things are a bit more complex. If someone wanted to use `embark-utils` on its own and specified `^7.0.0` as the version range (after observing that `embark` itself is in a `7.x` series) it won't work because `embark-utils` is still in `6.x`. In the general case, users may have to manually cross-check major versions of various `embark-*` packages that they specify in their projects' `package.json` files. There are tools like [npm-check-updates][ncu] that can make the task easier, but there is still likely to be some confusion, especially given the large and growing number of packages in this monorepo. Another area of confusion would exist around the `nightly` dist-tag. In the scenario above, `embark-core@nightly` (and/or `@nightly` of its dependents, e.g. `embark`) would resolve to `7.0.0-nightly.0` but `embark-utils@nightly` would resolve to some `6.5.4-nightly.N` (:bomb:), i.e. a prerelease version that predates the current stable `6.5.4` release of `embark-utils` (and *might* not include all changes that landed in `embark-utils` prior to that stable release). By bumping all packages each time there is a major stable release, and by having the `nightly` dist-tag always point to a package's most recent release (whether stable or prerelease), the problems described above are avoided. To see the `--force-publish` major-release strategy in action take a look at the [commit history][history] for the [nightly-release-workflow-tester][mb-nrwt] repo together with the *Versions* tab of the NPM pages for the [foo][foo], [bar][bar], [baz][baz], and [quux][quux] packages. Ignore the version history for `<= 2.0.1` because those pre/releases were made with a different strategy than the current one. Refactor the existing `scripts/release.js` to make it more flexible generally and with respect to options that can be forwarded to `lerna`. In particular, it's now possible to run `lerna version` instead of `lerna publish` (the default behavior) by using the `--version-only` cli option; when combining that usage with `--skip-qa` and `--no-push` it's possible to conveniently and quickly experiment with the [`bump` positional][bump] and additional options such as `--force-publish`, `--conventional-prerelease`, and `--conventional-graduate`, i.e. to better understand how `lerna` will update package versions. That ability should make it much simpler to figure out the best course of action to take locally (manually) when a nightly release completely or partially failed (which could happen for a number of reasons), as well for other scenarios such as making a minor/patch release in a previous line of major releases, or when making two/more successive stable releases without a nightly release having happened in the meantime. An important change to `scripts/release.js` is that by default it makes use of the `--create-release github` option for `lerna version|publish`. For that to work, an environment variable named `GH_TOKEN` must be defined with a properly [scoped][scopes] GitHub [personal access token][pa-token] (`public_repo` scope is sufficient for creating releases). The same is true for `scripts/stable-release.js`. Delete the `.github/PULL_REQUEST_TEMPLATE` directory and the templates it contained. Unlike for GitHub issue creation, there is no prompt-page for picking from a repo's PR templates; to use a PR template a `template=[name]` [query parameter][template-query] must be appended to the URL of the PR creation page. So the PR templates ended up unused by the Embark Team and external contributors because it's not convenient to use them. Restore the default PR template we had in place some time ago (with some small revisions) since it seems like a helpful starting point, especially for external contributors. Consistently use all-lowercase filenames for ISSUE/PR templates. [preid]: https://semver.org/#spec-item-9 [dist-tag]: https://docs.npmjs.com/cli/dist-tag [secrets]: https://github.com/embarklabs/embark/settings/secrets [ncu]: https://www.npmjs.com/package/npm-check-updates [history]: https://github.com/michaelsbradleyjr/nightly-release-workflow-tester/commits/master [mb-nrwt]: https://github.com/michaelsbradleyjr/nightly-release-workflow-tester/ [foo]: https://www.npmjs.com/package/nightly-release-workflow-tester-foo?activeTab=versions [bar]: https://www.npmjs.com/package/nightly-release-workflow-tester-bar?activeTab=versions [baz]: https://www.npmjs.com/package/nightly-release-workflow-tester-baz?activeTab=versions [quux]: https://www.npmjs.com/package/nightly-release-workflow-tester-quux?activeTab=versions [bump]: https://github.com/lerna/lerna/tree/master/commands/version#semver-bump [scopes]: https://developer.github.com/apps/building-oauth-apps/understanding-scopes-for-oauth-apps/ [pa-token]: https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line [template-query]: https://help.github.com/en/github/building-a-strong-community/creating-a-pull-request-template-for-your-repository
2020-01-07 17:09:38 +00:00
].filter(str => !!str).join(` `);
let pkgsToTag;
try {
pkgsToTag = JSON.parse(
runCommand(`${lernaChanged} 2>/dev/null || true`, false, lernaChanged)
.toString()
.trim() || '[]'
);
} catch (e) {
logError(`A problem occured. Please check the error above.`);
logError(failMsg);
process.exit(1);
}
const lernaPublish = [
`lerna`,
`publish`,
`--conventional-commits`,
(!forcePublish && `--conventional-graduate`) || ``,
`--create-release github`,
`--dist-tag ${distTag}`,
(forcePublish && `--force-publish`) || ``,
`--git-remote ${remote}`,
`--message "${commitMsg}"`,
`--registry ${registry}`
].filter(str => !!str).join(` `);
try {
runCommand(lernaPublish);
if (localRef ===
runCommand(`git rev-parse ${branch}`, false).toString().trim()) {
logWarning(
chalk.yellow(`STABLE RELEASE STOPPED!`),
`No commit or tag was created. No packages were published. This is`,
`most likely due to no qualifying changes having been made to the`,
`${cyan(branch)} branch since the previous release. Please check the`,
`output above.`
);
process.exit(0);
}
} catch (e) {
logError(`A problem occured. Please check the error above.`);
const lernaDocsUrl = 'https://github.com/lerna/lerna/tree/master/commands/publish#bump-from-package';
logError(
failMsg,
`Make sure to clean up commits, tags, and releases as necessary. Check`,
`the package registry to verify if any packages were published. If so`,
`they may need to be flagged as deprecated since the`,
`${cyan('lerna publish')} command exited with error. In some cases it`,
`may be possible to salvage an imcomplete release by using the`,
`${cyan('from-package')} keyword with the ${cyan('lerna publish')}`,
`command. See: ${lernaDocsUrl}`
);
process.exit(1);
}
if (pkgsToTag.length) {
logInfo(`Reading ${cyan(lernaJsonPath)}...`);
let updatedCurrentVersion;
try {
lernaJson = readJsonSync(lernaJsonPath);
updatedCurrentVersion = lernaJson.version;
if (!updatedCurrentVersion) throw new Error('missing version in lerna.json');
logSuccess(`Updated current version is ${updatedCurrentVersion}`);
} catch (e) {
console.error(e.stack || e);
logError(
`Could not read values from ${cyan(lernaJsonPath)}. Please check the`,
`error above.`
);
logError(failMsg);
process.exit(1);
}
logInfo(
`Updating ${cyan('nightly')} dist-tags to point to the new stable`,
`version...`
);
logInfo('Packages to tag:', pkgsToTag.map(({name}) => cyan(name)).join(', '));
const _pkgsToTag = pkgsToTag.slice();
try {
for (const {name} of pkgsToTag) {
runCommand(`npm dist-tag add ${name}@${updatedCurrentVersion} nightly`);
_pkgsToTag.shift();
}
} catch (e) {
logError(`A problem occured. Please check the error above.`);
const packages = _pkgsToTag.map(({name}) => cyan(name)).join(', ');
logError(
`NPM dist-tag ${cyan('nightly')} was not updated for the following`,
`packages: ${packages}. Make sure to complete the updates manually.`
);
logError(failMsg);
process.exit(1);
}
}
logSuccess(`${chalk.green(`STABLE RELEASE SUCCEEDED!`)} Woohoo! Done.`);
})().then(() => {
process.exit(0);
}).catch(e => {
console.error(e.stack || e);
logError(`A problem occured. Please check the error above.`);
logError(failMsg);
process.exit(1);
});