sourcecred/sharness/aggregate-results.sh
William Chargin baa0cbff1b
Add sharness for shell-based testing (#597)
Summary:
We will shortly want to perform testing of shell scripts; it makes the
most sense to do so via the shell. We could roll our own testing
framework, but it makes more sense to use an existing one. By choosing
Sharness, we’re in good company: `go-ipfs` and `go-multihash` use it as
well, and it’s derived from Git’s testing library. I like it a lot.

For now, we need a dummy test file; our test runner will fail if there
are no tests to run. As soon as we have a real test, we can remove this.

This commit was generated by following the “per-project installation”
instructions at https://github.com/chriscool/sharness, and by
additionally including that repository’s `COPYING` file as
`SHARNESS_LICENSE`, with a header prepended. I considered instead adding
Sharness as a submodule, which is supported and has clear advantages
(e.g., you can update the thing), but opted to avoid the complexity of
submodules for now.

Test Plan:
Create the following tests in the `sharness` directory:

```shell
$ cat sharness/good.t
#!/bin/sh
test_description='demo of passing tests'
. ./sharness.sh
test_expect_success "look at me go" true
test_expect_success EXPENSIVE "this may take a while" 'sleep 2'
test_done
# vim: ft=sh
$ cat sharness/bad.t
#!/bin/sh
test_description='demo of failing tests'
. ./sharness.sh
test_expect_success "I don't feel so good" false
test_done
# vim: ft=sh
```

Note that `yarn sharness` and `yarn test` fail appropriately. Note that
`yarn sharness-full` fails appropriately after taking two extra seconds,
and `yarn test --full` runs the latter. Each failure message should
print the name of the failing test case, not just the suite name, and
should indicate that the passing tests passed.

Then, remove `sharness/bad.t`, and note that the above commands all
pass, with the `--full` variants still taking longer.

Finally, remove `sharness/good.t`, and note that the above commands all
pass (and all pass quickly).

wchargin-branch: add-sharness
2018-08-06 12:56:25 -07:00

58 lines
1.4 KiB
Bash
Executable File

#!/bin/sh
#
# Copyright (c) 2008-2012 Git project
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see http://www.gnu.org/licenses/ .
failed_tests=
fixed=0
success=0
failed=0
broken=0
total=0
while read file; do
while read type value; do
case $type in
'')
continue ;;
fixed)
fixed=$(($fixed + $value)) ;;
success)
success=$(($success + $value)) ;;
failed)
failed=$(($failed + $value))
if test $value != 0; then
test_name=$(expr "$file" : 'test-results/\(.*\)\.[0-9]*\.counts')
failed_tests="$failed_tests $test_name"
fi
;;
broken)
broken=$(($broken + $value)) ;;
total)
total=$(($total + $value)) ;;
esac
done <"$file"
done
if test -n "$failed_tests"; then
printf "\nfailed test(s):$failed_tests\n\n"
fi
printf "%-8s%d\n" fixed $fixed
printf "%-8s%d\n" success $success
printf "%-8s%d\n" failed $failed
printf "%-8s%d\n" broken $broken
printf "%-8s%d\n" total $total