sourcecred/sharness
William Chargin d2727c01ba
Fix insidious quoting bug in build test script (#772)
Summary:
This patch fixes a particularly sneaky bug. Our test script contains a
literal backtick inside single quotes. This is generally not a problem,
because backticks inside single quotes do nothing. But the contents of
the single quotes are interpreted as Bash by our test runner, and at
that time the single quotes are expanded to a command substitution.
Therefore, `grep` is invoked as if writing

    grep -e "warning: running $(yarn backend)"

at the CLI. This will actually invoke `yarn backend`!

The magnificent aspect of this bug is that it both makes the test script
slower by about ten seconds _and_ completely and silently defeats the
assertion in which it’s contained. The output of `yarn backend` contains
several blank lines. Therefore, one of the literal patterns to `grep`
contains a blank line. This causes `grep` to match _every_ line in the
error file, regardless of whether it is one of the intended messages.

This patch is the 666th PR to SourceCred. In my opinion, it deserves
this dubious honor.

Test Plan:
Note that `yarn test --full` works, but fails if one of the expected
error message patterns is deleted or munged.

Confirm the behavior by prepending `echo backend >>/tmp/log &&` to the
`yarn backend` script in `package.json`, noting that the resulting log
file contains four lines before this patch and two lines after it.
(Don’t forget to delete/clear the log file before invocations.)

Confirm the behavior of `grep` by writing:

```shell
$ printf 'things went wrong!\n' >err
$ printf 'wat\n\nwot\n' >patterns
$ grep -vF -e "okay" -e "warn: `cat patterns`" err; echo $?
1
$ printf 'wat\nwot\n' >patterns  # no empty line
$ grep -vF -e "okay" -e "warn: `cat patterns`" err; echo $?
things went wrong!
0
```

wchargin-branch: fix-build-test-quoting
2018-09-05 10:53:05 -07:00
..
.gitignore Add a `.gitignore` for Sharness (#673) 2018-08-15 16:31:00 -07:00
Makefile Add `sharness` for shell-based testing (#597) 2018-08-06 12:56:25 -07:00
SHARNESS_LICENSE Add `sharness` for shell-based testing (#597) 2018-08-06 12:56:25 -07:00
aggregate-results.sh Add `sharness` for shell-based testing (#597) 2018-08-06 12:56:25 -07:00
dummy.t Add `sharness` for shell-based testing (#597) 2018-08-06 12:56:25 -07:00
sharness.sh Add `sharness` for shell-based testing (#597) 2018-08-06 12:56:25 -07:00
test_build_static_site.t Fix insidious quoting bug in build test script (#772) 2018-09-05 10:53:05 -07:00