Summary:
This fixes a bug introduced in #317, which only occurred in the cron job
variant of the CI script (`yarn travis --full`): the two scripts run in
the cron job depend on `yarn backend` having previously written to the
`bin/` directory, but this is precisely what we wanted to prevent. To
fix this, we simply add an additional target for `yarn backend` during
the cron job. This is a little bit wasteful in that we compile the
backend applications twice, but it’s not a big deal because (a) it only
runs in cron jobs, so it won’t slow down normal builds, and (b) it only
takes about 5 seconds, anyway.
Test Plan:
Export a `GITHUB_TOKEN` and run `yarn travis --full`, which fails before
this change and passes after it.
wchargin-branch: cron-ci-overwrite-bin
Summary:
Previously, our CI script would run `yarn backend`, which has the
side-effect of erasing the `bin/` directory. By itself, this is not
great, but not awful. However, this frequently triggers a race condition
in Prettier, causing the `check-pretty` step of the build to fail. (More
details: https://github.com/prettier/prettier/issues/4468.)
This patch changes the CI script to build the backend scripts into a
temporary directory.
Test Plan:
Before applying this patch: `yarn backend` and then `yarn travis`. If
this consistently causes a Travis failure due to `check-pretty`, then
your machine can reproduce the race condition that we‛re trying to
eliminate. (Otherwise, you can try creating a bunch more Git history…
I’m not really sure what to say. It is a race condition, after all.)
Then, apply this patch, and repeat the above steps; note that the error
no longer occurs, and that the build output is to a temporary directory.
wchargin-branch: ci-preserve-bin
As of this commit, adding the comment `//$ExpectFlowError` in flow-typed
code asserts that the next line must cause a flow error. If it does, no
error or warning is generated. If it does not, then this produces a flow
warning, which is visible to developers running `yarn flow` and
additionally causes travis to fail.
Test plan:
- As committed, `yarn travis` passes.
- I added `//$ExpectFlowError` above some line of flow-checked code which does
not currently throw an error. Afterwards, `yarn travis` failed (and a
helpful message was displayed in console on running `yarn flow`)
- I added the following bad code into one of our files:
```javascript
//$ExpectFlowError
const foo: string = 3;
```
As expected, `yarn flow` and `yarn travis` both passed.
Test Plan:
Run `yarn flow stop; yarn travis | cat` and note the absence of the
really long line that has ~2500 bytes of “Server is initializing”.
wchargin-branch: quiet-flow-server
Summary:
Placing `STOPSHIP` or `stopship` (or any case variant) in any file
tracked by Git will now cause a `yarn travis` failure. If you need to
use this string, you can concatenate it as `"stop" + "ship"` or
equivalent.
Test Plan:
In `travis.js`, change `"check-stop" + "ships"` to `"check-stopships"`,
and note that this causes the build to fail with a nice message. Note
that this also causes `check-stopships.sh` to fail even when invoked
from an unrelated directory, like `src`.
wchargin-branch: check-stopships
This script ensures that either //@flow or //@no-flow is present in
every js file. Every existing js file that would fail this check has
been given //@no-flow, we should work to remove all of these in the
future.
Test plan:
I verified that `yarn travis` fails before fixing the other js files,
and passes afterwards.
Summary:
To be honest, I have no idea what exactly this does or why it’s
necessary, but if we don’t do this then it is not possible to `import`
the exported member from a Webpack-bundled script. I’ve seen this
pattern before; one day I’ll actually figure out what it does. :-)
Test Plan:
Note that `yarn travis` (success) and `yarn travis --full` (failure; no
GitHub token) both have the expected behaviors.
wchargin-branch: execdependencygraph-export
Summary:
We’d like to use the same abstraction for creating multiple cred graphs
and then combining them together. This will enable us to do that.
Test Plan:
Run `yarn travis` to test the success case, and `yarn travis --full`
(without setting a `GITHUB_TOKEN`) to test the failure case.
wchargin-branch: execdepgraph
Summary:
This could catch failures in build configuration or with Webpack. It’s
unlikely to catch any logic errors, because no production code is run.
In any case, it’s fast enough; it finishes at about the same time as
`ci-test` and `check-pretty`.
Test Plan:
From the repository root, run `rm -r bin; yarn travis`, and note that
the `bin/` directory is regenerated.
wchargin-branch: ci-backend
Summary:
This prevents the boilerplate output of the form
```
> sourcecred-explorer@0.1.0 check-pretty /home/wchargin/git/sourcecred
> prettier --list-different '**/*.js'
```
(superfluous linebreaks included). In the case that a script fails, it
also omits the giant “this is most likely not a problem with npm” block.
The downside to this is that it suppresses any errors in npm-run-script
itself. For instance, `npm run wat` produces “missing script: wat”,
while `npm run --silent wat` just silently exits with 1. This does not
silence the actual scripts themselves, so things like lint errors or
test failures will still appear.
Test Plan:
Run `yarn travis` before and after this commit, and note that the
resulting build log is prettier after.
wchargin-branch: ci-silent
Summary:
This CI script accomplishes two tasks:
1. It speeds up our build by parallelizing where possible.
2. It opens the possibility for running Travis cron jobs.
Currently, this script by default does the same amount of work as our
current CI script. However, I’d like to move `yarn backend` into the
list of basic actions: a backend build failure should fail CI.
Note: this script is written to be executable directly by Node, so we
can’t use Flow types with the standard syntax. Instead, we use the
comment syntax: https://flow.org/en/docs/types/comments/
Test Plan:
The following should pass with useful output:
- `npm run travis`
- `GITHUB_TOKEN="your_github_token" npm run travis -- --full`
The following should fail with useful output:
- `npm run travis -- --full` (fail)
To test different failure modes, it can be helpful to add
```js
{id: "doomed", cmd: ["false"], deps: []},
{id: "orphan", cmd: ["whoami"], deps: ["who", "are", "you"]},
```
to the list of `basicTasks` in `travis.js`.
To test performance:
```shell
$ time node ./config/travis.js >/dev/null 2>/dev/null
real 0m8.306s
user 0m20.336s
sys 0m1.364s
$ time bash -c \
> 'npm run check-pretty && npm run lint && npm run flow && CI=1 npm run test' \
> >/dev/null 2>/dev/null
real 0m12.427s
user 0m13.752s
sys 0m0.804s
```
A 50% savings is not bad at all—and the raw time saved should only
improve from here on, as the individual steps start taking more time.
wchargin-branch: custom-ci
William Chargin
Dandelion Mané
Dandelion Mané