2018-09-28 23:58:54 +00:00
|
|
|
# JavaScript Node CircleCI 2.0 configuration file
|
|
|
|
|
#
|
|
|
|
|
# Check https://circleci.com/docs/2.0/language-javascript/ for more details
|
2018-10-02 17:57:33 +00:00
|
|
|
# See https://circleci.com/docs/2.0/config-intro/#section=configuration for spec
|
2018-09-28 23:58:54 +00:00
|
|
|
#
|
2018-10-02 17:57:33 +00:00
|
|
|
version: 2.1
|
|
|
|
|
|
2019-08-28 21:17:50 +00:00
|
|
|
orbs:
|
|
|
|
|
# https://circleci.com/orbs/registry/orb/circleci/docker
|
|
|
|
|
docker: circleci/docker@0.5.13
|
|
|
|
|
|
2018-10-02 17:57:33 +00:00
|
|
|
executors:
|
2019-07-06 14:21:10 +00:00
|
|
|
node10:
|
2018-09-28 23:58:54 +00:00
|
|
|
docker:
|
2019-07-06 14:21:10 +00:00
|
|
|
- image: circleci/node:10
|
|
|
|
|
working_directory: ~/repo
|
|
|
|
|
node12:
|
|
|
|
|
docker:
|
|
|
|
|
- image: circleci/node:12
|
2018-09-28 23:58:54 +00:00
|
|
|
working_directory: ~/repo
|
|
|
|
|
|
2018-10-02 17:57:33 +00:00
|
|
|
commands:
|
|
|
|
|
set_up_node_modules:
|
|
|
|
|
description: "load node_modules, updating and storing back into cache if necessary"
|
2018-09-28 23:58:54 +00:00
|
|
|
steps:
|
|
|
|
|
- restore_cache:
|
|
|
|
|
keys:
|
2018-10-02 17:57:33 +00:00
|
|
|
- v1-dependencies-{{ checksum "package.json" }}
|
|
|
|
|
# fall back to using the latest cache if no exact match is found
|
|
|
|
|
- v1-dependencies-
|
2018-09-28 23:58:54 +00:00
|
|
|
- run: yarn install
|
|
|
|
|
- save_cache:
|
|
|
|
|
paths:
|
|
|
|
|
- node_modules
|
|
|
|
|
key: v1-dependencies-{{ checksum "package.json" }}
|
2019-09-06 16:16:28 +00:00
|
|
|
pull_cache:
|
|
|
|
|
description: Pulls docker images usable for our cache
|
|
|
|
|
steps:
|
|
|
|
|
- run: docker pull sourcecred/sourcecred:dev
|
|
|
|
|
- run: docker pull node:12
|
2018-09-28 23:58:54 +00:00
|
|
|
|
2018-10-02 17:57:33 +00:00
|
|
|
jobs:
|
|
|
|
|
test:
|
2019-07-06 14:21:10 +00:00
|
|
|
executor: node12
|
2018-10-02 17:57:33 +00:00
|
|
|
steps:
|
|
|
|
|
- checkout
|
|
|
|
|
- set_up_node_modules
|
Fix flakey CI memory issues (#1230)
Ever since I upgraded all of the dependencies, we've been having
regular CI failures, which seem to share a common root cause of memory
exhaustion. Here are some examples: [1], [2].
[1]: https://circleci.com/gh/sourcecred/sourcecred/1246
[2]: https://circleci.com/gh/sourcecred/sourcecred/1239
After some experimentation, I've found that we can solve the
issue by ensuring that jest runs on its own in CI, so that it doesn't
contend with other tests for memories. Also, I reduce its max workers to
2, which matches the number of CPUs in the CircleCI containers.
Unfortunately, this does increase our build time. The postcommit (non
full) test now takes 45-60s (up from 30-50s), and the full test is also
a little slower. However, building in about one minute is still
acceptably fast, and having regular flakey test failures is not
acceptable, so this is still a win.
If we want to improve on this in the future, we should look into the git
shells getting spawned in `config/env.js`. I noticed that they were
often involved in the out-of-memory failures.
Also, I modified `.circleci/config.yml` so that any branch matching the
regular expression `/ci-.*/` will trigger a full build. That makes it
easier to test against CI failures.
Test plan: I ran about ~10 full builds with this change, and more with
similar variations, and they all passed. Verify that the full builds
that are run for this commit also all pass! Also, verify that running
yarn test locally has unchanged behavior, and running
`yarn test --ci` locally lets jest run to completion before running
any other test.
2019-07-16 00:51:14 +00:00
|
|
|
- run: yarn test --ci
|
2018-10-02 17:57:33 +00:00
|
|
|
test_full:
|
2019-07-06 14:21:10 +00:00
|
|
|
executor: node12
|
|
|
|
|
steps:
|
|
|
|
|
- checkout
|
|
|
|
|
- set_up_node_modules
|
Fix flakey CI memory issues (#1230)
Ever since I upgraded all of the dependencies, we've been having
regular CI failures, which seem to share a common root cause of memory
exhaustion. Here are some examples: [1], [2].
[1]: https://circleci.com/gh/sourcecred/sourcecred/1246
[2]: https://circleci.com/gh/sourcecred/sourcecred/1239
After some experimentation, I've found that we can solve the
issue by ensuring that jest runs on its own in CI, so that it doesn't
contend with other tests for memories. Also, I reduce its max workers to
2, which matches the number of CPUs in the CircleCI containers.
Unfortunately, this does increase our build time. The postcommit (non
full) test now takes 45-60s (up from 30-50s), and the full test is also
a little slower. However, building in about one minute is still
acceptably fast, and having regular flakey test failures is not
acceptable, so this is still a win.
If we want to improve on this in the future, we should look into the git
shells getting spawned in `config/env.js`. I noticed that they were
often involved in the out-of-memory failures.
Also, I modified `.circleci/config.yml` so that any branch matching the
regular expression `/ci-.*/` will trigger a full build. That makes it
easier to test against CI failures.
Test plan: I ran about ~10 full builds with this change, and more with
similar variations, and they all passed. Verify that the full builds
that are run for this commit also all pass! Also, verify that running
yarn test locally has unchanged behavior, and running
`yarn test --ci` locally lets jest run to completion before running
any other test.
2019-07-16 00:51:14 +00:00
|
|
|
- run: yarn test --full --ci
|
2019-07-06 14:21:10 +00:00
|
|
|
test_full_10:
|
|
|
|
|
executor: node10
|
2018-10-02 17:57:33 +00:00
|
|
|
steps:
|
|
|
|
|
- checkout
|
|
|
|
|
- set_up_node_modules
|
Fix flakey CI memory issues (#1230)
Ever since I upgraded all of the dependencies, we've been having
regular CI failures, which seem to share a common root cause of memory
exhaustion. Here are some examples: [1], [2].
[1]: https://circleci.com/gh/sourcecred/sourcecred/1246
[2]: https://circleci.com/gh/sourcecred/sourcecred/1239
After some experimentation, I've found that we can solve the
issue by ensuring that jest runs on its own in CI, so that it doesn't
contend with other tests for memories. Also, I reduce its max workers to
2, which matches the number of CPUs in the CircleCI containers.
Unfortunately, this does increase our build time. The postcommit (non
full) test now takes 45-60s (up from 30-50s), and the full test is also
a little slower. However, building in about one minute is still
acceptably fast, and having regular flakey test failures is not
acceptable, so this is still a win.
If we want to improve on this in the future, we should look into the git
shells getting spawned in `config/env.js`. I noticed that they were
often involved in the out-of-memory failures.
Also, I modified `.circleci/config.yml` so that any branch matching the
regular expression `/ci-.*/` will trigger a full build. That makes it
easier to test against CI failures.
Test plan: I ran about ~10 full builds with this change, and more with
similar variations, and they all passed. Verify that the full builds
that are run for this commit also all pass! Also, verify that running
yarn test locally has unchanged behavior, and running
`yarn test --ci` locally lets jest run to completion before running
any other test.
2019-07-16 00:51:14 +00:00
|
|
|
- run: yarn test --full --ci
|
2018-10-02 17:57:33 +00:00
|
|
|
|
|
|
|
|
workflows:
|
2019-08-28 21:17:50 +00:00
|
|
|
version: 2.0
|
2018-10-02 17:57:33 +00:00
|
|
|
commit:
|
|
|
|
|
jobs:
|
|
|
|
|
- test
|
2019-08-28 21:17:50 +00:00
|
|
|
- docker/publish:
|
|
|
|
|
deploy: false
|
|
|
|
|
image: sourcecred/sourcecred
|
|
|
|
|
tag: latest
|
|
|
|
|
requires:
|
|
|
|
|
- test
|
|
|
|
|
filters:
|
|
|
|
|
branches:
|
|
|
|
|
ignore:
|
|
|
|
|
- master
|
2019-09-06 16:16:28 +00:00
|
|
|
extra_build_args: --cache-from=node:12,sourcecred/sourcecred:dev
|
|
|
|
|
before_build:
|
|
|
|
|
- pull_cache
|
2019-08-28 21:17:50 +00:00
|
|
|
after_build:
|
|
|
|
|
- run:
|
|
|
|
|
name: Preview Docker Tag for Build
|
|
|
|
|
command: |
|
|
|
|
|
DOCKER_TAG=$(docker run sourcecred/sourcecred --version | cut -d' ' -f2)
|
|
|
|
|
echo "Version that would be used for Docker tag is ${DOCKER_TAG}"
|
|
|
|
|
|
|
|
|
|
- docker/publish:
|
|
|
|
|
image: sourcecred/sourcecred
|
|
|
|
|
tag: latest
|
|
|
|
|
requires:
|
|
|
|
|
- test
|
|
|
|
|
filters:
|
|
|
|
|
branches:
|
|
|
|
|
ignore: /.*/
|
|
|
|
|
tags:
|
|
|
|
|
only: /^v.*/
|
2019-09-06 16:16:28 +00:00
|
|
|
extra_build_args: --cache-from=node:12,sourcecred/sourcecred:dev
|
|
|
|
|
before_build:
|
|
|
|
|
- pull_cache
|
2019-08-28 21:17:50 +00:00
|
|
|
after_build:
|
|
|
|
|
- run:
|
|
|
|
|
name: Publish Docker Tag with Sourcecred Version
|
|
|
|
|
command: |
|
|
|
|
|
DOCKER_TAG=$(docker run sourcecred/sourcecred --version | cut -d' ' -f2)
|
|
|
|
|
echo "Version for Docker tag is ${DOCKER_TAG}"
|
|
|
|
|
docker tag sourcecred/sourcecred:latest sourcecred/sourcecred:${DOCKER_TAG}
|
|
|
|
|
docker push sourcecred/sourcecred
|
|
|
|
|
|
2018-11-01 17:57:53 +00:00
|
|
|
- test_full:
|
|
|
|
|
filters:
|
|
|
|
|
branches:
|
|
|
|
|
only:
|
|
|
|
|
- master
|
Fix flakey CI memory issues (#1230)
Ever since I upgraded all of the dependencies, we've been having
regular CI failures, which seem to share a common root cause of memory
exhaustion. Here are some examples: [1], [2].
[1]: https://circleci.com/gh/sourcecred/sourcecred/1246
[2]: https://circleci.com/gh/sourcecred/sourcecred/1239
After some experimentation, I've found that we can solve the
issue by ensuring that jest runs on its own in CI, so that it doesn't
contend with other tests for memories. Also, I reduce its max workers to
2, which matches the number of CPUs in the CircleCI containers.
Unfortunately, this does increase our build time. The postcommit (non
full) test now takes 45-60s (up from 30-50s), and the full test is also
a little slower. However, building in about one minute is still
acceptably fast, and having regular flakey test failures is not
acceptable, so this is still a win.
If we want to improve on this in the future, we should look into the git
shells getting spawned in `config/env.js`. I noticed that they were
often involved in the out-of-memory failures.
Also, I modified `.circleci/config.yml` so that any branch matching the
regular expression `/ci-.*/` will trigger a full build. That makes it
easier to test against CI failures.
Test plan: I ran about ~10 full builds with this change, and more with
similar variations, and they all passed. Verify that the full builds
that are run for this commit also all pass! Also, verify that running
yarn test locally has unchanged behavior, and running
`yarn test --ci` locally lets jest run to completion before running
any other test.
2019-07-16 00:51:14 +00:00
|
|
|
- /ci-.*/
|
2019-08-28 21:17:50 +00:00
|
|
|
|
2019-07-06 14:21:10 +00:00
|
|
|
- test_full_10:
|
|
|
|
|
filters:
|
|
|
|
|
branches:
|
|
|
|
|
only:
|
|
|
|
|
- master
|
Fix flakey CI memory issues (#1230)
Ever since I upgraded all of the dependencies, we've been having
regular CI failures, which seem to share a common root cause of memory
exhaustion. Here are some examples: [1], [2].
[1]: https://circleci.com/gh/sourcecred/sourcecred/1246
[2]: https://circleci.com/gh/sourcecred/sourcecred/1239
After some experimentation, I've found that we can solve the
issue by ensuring that jest runs on its own in CI, so that it doesn't
contend with other tests for memories. Also, I reduce its max workers to
2, which matches the number of CPUs in the CircleCI containers.
Unfortunately, this does increase our build time. The postcommit (non
full) test now takes 45-60s (up from 30-50s), and the full test is also
a little slower. However, building in about one minute is still
acceptably fast, and having regular flakey test failures is not
acceptable, so this is still a win.
If we want to improve on this in the future, we should look into the git
shells getting spawned in `config/env.js`. I noticed that they were
often involved in the out-of-memory failures.
Also, I modified `.circleci/config.yml` so that any branch matching the
regular expression `/ci-.*/` will trigger a full build. That makes it
easier to test against CI failures.
Test plan: I ran about ~10 full builds with this change, and more with
similar variations, and they all passed. Verify that the full builds
that are run for this commit also all pass! Also, verify that running
yarn test locally has unchanged behavior, and running
`yarn test --ci` locally lets jest run to completion before running
any other test.
2019-07-16 00:51:14 +00:00
|
|
|
- /ci-.*/
|
2019-08-28 21:17:50 +00:00
|
|
|
|
|
|
|
|
- docker/publish:
|
|
|
|
|
image: sourcecred/sourcecred
|
2019-09-06 16:16:28 +00:00
|
|
|
extra_build_args: --cache-from=node:12,sourcecred/sourcecred:dev
|
|
|
|
|
before_build:
|
|
|
|
|
- pull_cache
|
2019-08-28 21:17:50 +00:00
|
|
|
requires:
|
|
|
|
|
- test
|
|
|
|
|
- test_full
|
|
|
|
|
- test_full_10
|
|
|
|
|
tag: dev
|
|
|
|
|
filters:
|
|
|
|
|
branches:
|
|
|
|
|
only: master
|
|
|
|
|
|
|
|
|
|
|
2018-10-02 17:57:33 +00:00
|
|
|
nightly:
|
|
|
|
|
triggers:
|
|
|
|
|
- schedule:
|
|
|
|
|
cron: "0 22 * * *" # 22:00 UTC
|
|
|
|
|
filters:
|
|
|
|
|
branches:
|
|
|
|
|
only:
|
|
|
|
|
- master
|
|
|
|
|
jobs:
|
|
|
|
|
- test_full
|
2019-07-06 14:21:10 +00:00
|
|
|
- test_full_10
|
