Summary:
We’ve hitherto only run `yarn test` on each commit, to reduce latency.
This commit introduces an advisory (non-blocking) `yarn test --full`
run. Our GitHub branch protection rule is configured to only require
that the `test` task pass before blessing the PR, which is why the
Docker tag preview job doesn’t also block merging. In the case that a
commit is approved quickly and needs to be merged immediately, this
doesn’t get in your way. In all other cases, this can help prevent
breakages.
Test Plan:
Watch the CI run for this commit. Note that all jobs are running, but
only the `test` job is marked as required; see [screenshot][1].
[1]: https://user-images.githubusercontent.com/4317806/68623255-edce3900-0488-11ea-948f-a0cab5174a35.png
wchargin-branch: ci-advisory-full
Ever since I upgraded all of the dependencies, we've been having
regular CI failures, which seem to share a common root cause of memory
exhaustion. Here are some examples: [1], [2].
[1]: https://circleci.com/gh/sourcecred/sourcecred/1246
[2]: https://circleci.com/gh/sourcecred/sourcecred/1239
After some experimentation, I've found that we can solve the
issue by ensuring that jest runs on its own in CI, so that it doesn't
contend with other tests for memories. Also, I reduce its max workers to
2, which matches the number of CPUs in the CircleCI containers.
Unfortunately, this does increase our build time. The postcommit (non
full) test now takes 45-60s (up from 30-50s), and the full test is also
a little slower. However, building in about one minute is still
acceptably fast, and having regular flakey test failures is not
acceptable, so this is still a win.
If we want to improve on this in the future, we should look into the git
shells getting spawned in `config/env.js`. I noticed that they were
often involved in the out-of-memory failures.
Also, I modified `.circleci/config.yml` so that any branch matching the
regular expression `/ci-.*/` will trigger a full build. That makes it
easier to test against CI failures.
Test plan: I ran about ~10 full builds with this change, and more with
similar variations, and they all passed. Verify that the full builds
that are run for this commit also all pass! Also, verify that running
yarn test locally has unchanged behavior, and running
`yarn test --ci` locally lets jest run to completion before running
any other test.
This modifies config.yml so that it will run full tests on a branch
called "ci-test". This will make it easier for us to test attemtps to
fix full build issues, without needing to iterate against master
directly.
This commit changes CI to test against node 12 and 10 instead of node 8.
I test against node 12 by default (it will be LTS soon, and it has a
number of nice improvements compared to 10). We test node10 on the
nightly and post-merge, that way we will still discover quickly if we
have a problem with node 10, but it won't slow down CI for merges.
I'm just dropping explicit support for node 8 entirely, since node 8 is
end-of-life soon (Dec 19).
Test plan: I've locally verified that `yarn test --full` passes for both
node 10 and node 12.
Summary:
We currently run quicktest on pull requests, quicktest again on commits
_after_ they are merged into `master`, and fulltest nightly. This commit
retains quicktest on pull requests, but switches to fulltest once a
commit is merged into master. We still run fulltest nightly as well, to
detect any changes in remote APIs that we hit.
Test Plan:
Push this commit. Check that Circle CI runs its workflow normally (as
quicktest), verifying that the config parses correctly. Then, merge this
commit, and verify that the full tests have run.
wchargin-branch: ci-test-full-on-master
Summary:
CircleCI seems fast.
This config file copied from the one suggested by CircleCI, with the
following modifications: Node version changed from 7.10 to 8.12.0;
commented-out MongoDB dependency removed; capitalization error in header
comment fixed.
Test Plan:
Per the CircleCI instructions, merge this PR and then turn on the
project.
wchargin-branch: circleci-config
William Chargin
Vanessasaurus
Dandelion Mané
