sourcecred/sharness/load_test_instance.t
Dandelion Mané 86fd5638bb
Add snapshot testing for instances (#1897)
This adds a simple sharness snapshot test for the new instance system,
in the model of the old [test_load_example_github.t][old].

I've setup a very simple test instance with the GitHub and Discourse
plugins, and we verify that the output generated by running `load`,
`graph`, and `score` in succession is stable. (Cache is not
persisted.) This is a nice sanity check to verify that nothing ever gets
totally broken; we'll still want to add unit testing for more specific
features and edge cases.

Test plan:
`yarn test --full` passes.
If you `rm -rf sharness/test-instance/output`, then `yarn test --full`
fails.
If you then run `./scripts/update_snapshots.sh`, then the output
directory will be restored; afterwards `yarn test --full` passes again.

To verify that the snapshots are valid, you can test them with the
frontend:
`yarn start --instance sharness/__snapshots__/test-instance`

If you are actually debugging this script, rather than using
`yarn test --full` you'll want to invoke
`(cd sharness; ./load_test_instance.t -l -v)`

[old]: https://github.com/sourcecred/sourcecred/blob/v0.6.0/sharness/test_load_example_github.t
2020-06-27 20:41:49 -07:00

66 lines
1.9 KiB
Bash
Executable File

#!/bin/sh
# Disable these lint rules globally:
# 2034 = unused variable (used by sharness)
# 2016 = parameter expansion in single quotes
# 1004 = backslash-newline in single quotes
# shellcheck disable=SC2034,SC2016,SC1004
:
# If this test is failing, it probably means you need to update snapshots.
# You can do so by setting your SOURCECRED_GITHUB_TOKEN (see README) and then
# running scripts/update_snapshots.sh.
test_description='test snapshot integrity for sourcecred instance loading'
export GIT_CONFIG_NOSYSTEM=1
export GIT_ATTR_NOSYSTEM=1
# shellcheck disable=SC1091
. ./sharness.sh
test_expect_success "environment and Node linking setup" '
toplevel="$(git -C "$(dirname "$0")" rev-parse --show-toplevel)" &&
snapshot_directory="${toplevel}/sharness/__snapshots__/test-instance/" &&
if [ -z "${SOURCECRED_BIN}" ]; then
printf >&2 "warn: missing environment variable SOURCECRED_BIN\n" &&
printf >&2 "warn: using repository bin directory as fallback\n" &&
export SOURCECRED_BIN="${toplevel}/bin"
fi &&
export NODE_PATH="${toplevel}/node_modules${NODE_PATH:+:${NODE_PATH}}" &&
test_set_prereq SETUP
'
if [ -n "${SOURCECRED_GITHUB_TOKEN:-}" ]; then
test_set_prereq HAVE_GITHUB_TOKEN
fi
test_expect_success EXPENSIVE,SETUP,HAVE_GITHUB_TOKEN \
"should load the example instance" '
cp -r "${snapshot_directory}" . &&
cd test-instance &&
node "${SOURCECRED_BIN}/sourcecred.js" load &&
node "${SOURCECRED_BIN}/sourcecred.js" graph &&
node "${SOURCECRED_BIN}/sourcecred.js" score &&
rm -rf cache &&
test_set_prereq LOADED
'
if [ -n "${UPDATE_SNAPSHOT}" ]; then
test_set_prereq UPDATE_SNAPSHOT
fi
test_expect_success LOADED,UPDATE_SNAPSHOT \
"should update the snapshot" '
rm -rf "$snapshot_directory" &&
cp -r . "$snapshot_directory"
'
test_expect_success LOADED "should be identical to the snapshot" '
diff -qr . "$snapshot_directory"
'
test_done
# vim: ft=sh