status-im
/
sourcecred
mirror of https://github.com/status-im/sourcecred.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
sourcecred/sharness/test_build_static_site.t

263 lines
9.0 KiB
Perl
Raw Normal View History

Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
#!/bin/sh
# Disable these lint rules globally:
# 2034 = unused variable (used by sharness)
# 2016 = parameter expansion in single quotes
# 1004 = backslash-newline in single quotes
# shellcheck disable=SC2034,SC2016,SC1004
:
test_description='tests for scripts/build_static_site.sh'
export GIT_CONFIG_NOSYSTEM=1
export GIT_ATTR_NOSYSTEM=1
# shellcheck disable=SC1091
. ./sharness.sh
run() (
set -eu
toplevel="$(git -C "$(dirname "$0")" rev-parse --show-toplevel)"
"${toplevel}"/scripts/build_static_site.sh "$@"
)
#
# Start by checking a bunch of easy cases related to the argument
# parser, mostly about rejecting various ill-formed invocations.
test_expect_success "should print a help message" '
run --help >msg 2>err &&
test_must_be_empty err &&
test_path_is_file msg &&
grep -qF "usage: build_static_site.sh" msg
'
test_expect_success "should fail with no target" '
test_must_fail run 2>err &&
grep -qF -- "target directory not specified" err
'
test_expect_success "should fail with missing target value" '
test_must_fail run --target 2>err &&
grep -qF -- "missing value for --target" err
'
test_expect_success "should fail with multiple targets" '
mkdir one two &&
test_must_fail run --target one --target two 2>err &&
grep -qF -- "--target specified multiple times" err
'
test_expect_success "should fail with a file as target" '
printf "important\nstuff" >important_data &&
test_must_fail run --target important_data 2>err &&
build_static_site.sh: create target if nonexistent (#695) Summary: The current version of the build script has the safe but annoying property that the target directory must be an existing, empty directory. It seems reasonable and convenient to allow the build script to create the directory with `mkdir -p`. It still fails if the directory is not empty or is a file. Test Plan: Unit tests updated; run `yarn sharness-full`. wchargin-branch: build-mkdir-p
2018-08-21 22:35:13 +00:00
grep -qF -- "target is not a directory" err &&
printf "important\nstuff" | test_cmp - important_data
'
test_expect_success "should fail with a target under a file" '
printf "important\nstuff" >important_data &&
test_must_fail run --target important_data/something 2>err &&
grep -q -- "cannot create directory.*Not a directory" err &&
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
printf "important\nstuff" | test_cmp - important_data
'
test_expect_success "should fail with a nonempty directory as target" '
mkdir important_dir &&
printf "redacted\n" >important_dir/.wallet.dat &&
test_must_fail run --target important_dir 2>err &&
build_static_site.sh: create target if nonexistent (#695) Summary: The current version of the build script has the safe but annoying property that the target directory must be an existing, empty directory. It seems reasonable and convenient to allow the build script to create the directory with `mkdir -p`. It still fails if the directory is not empty or is a file. Test Plan: Unit tests updated; run `yarn sharness-full`. wchargin-branch: build-mkdir-p
2018-08-21 22:35:13 +00:00
grep -qF -- "target directory is nonempty: important_dir" err &&
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
printf "redacted\n" | test_cmp - important_dir/.wallet.dat
'
mkdir putative_output
test_expect_success "should fail with missing repo value" '
test_must_fail run --target putative_output --repo 2>err &&
grep -qF -- "missing value for --repo" err &&
printf "redacted\n" | test_cmp - important_dir/.wallet.dat
'
test_expect_success "should fail with missing cname value" '
test_must_fail run --target putative_output --cname 2>err &&
grep -qF -- "missing value for --cname" err &&
printf "redacted\n" | test_cmp - important_dir/.wallet.dat
'
test_expect_success "should fail with empty cname" '
test_must_fail run --target putative_output --cname "" 2>err &&
grep -qF -- "empty value for --cname" err &&
printf "redacted\n" | test_cmp - important_dir/.wallet.dat
'
test_expect_success "should fail with multiple cname values" '
test_must_fail run --target putative_output \
--cname a.com --cname b.com 2>err &&
grep -qF -- "--cname specified multiple times" err &&
printf "redacted\n" | test_cmp - important_dir/.wallet.dat
'
#
# Now, actually generate output in two cases: one with repositories, and
# one with no repositories. We can only do this if we have a token.
if [ -n "${SOURCECRED_GITHUB_TOKEN:-}" ]; then
test_set_prereq HAVE_GITHUB_TOKEN
fi
# run_build PREREQ_NAME DESCRIPTION [FLAGS...]
# Build the site with the given FLAGS, and create a prereq PREREQ_NAME
# to be used in any tests that depend on this build. The build will
# itself have the EXPENSIVE prereq.
run_build() {
prereq_name="$1"; shift
description="$1"; shift
build_static_site.sh: create target if nonexistent (#695) Summary: The current version of the build script has the safe but annoying property that the target directory must be an existing, empty directory. It seems reasonable and convenient to allow the build script to create the directory with `mkdir -p`. It still fails if the directory is not empty or is a file. Test Plan: Unit tests updated; run `yarn sharness-full`. wchargin-branch: build-mkdir-p
2018-08-21 22:35:13 +00:00
output_dir="build_output/output_${prereq_name}"
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
api_dir="${output_dir}/api/v1/data"
data_dir="${api_dir}/data"
Add ":" as a shell-safe character (#753) Test Plan: Running `./test_build_static_site.t --long -v` no longer detects the feedback URL as unsafe. (Prior to this commit, it emitted a message to this effect.) The build is still broken on Linux for other reasons, but works on macOS or any other system where `sh` resolves to Bash. As a regression test, the “potentially unsafe argument” warning has been made to actually fail the test case. To verify this, remove `:` from the list of `unusual_chars`, run the test, and note that it fails outright. wchargin-branch: shell-safe-colon
2018-09-03 06:22:53 +00:00
unsafe_arg=
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
for arg in "${output_dir}" "$@"; do
Add ":" as a shell-safe character (#753) Test Plan: Running `./test_build_static_site.t --long -v` no longer detects the feedback URL as unsafe. (Prior to this commit, it emitted a message to this effect.) The build is still broken on Linux for other reasons, but works on macOS or any other system where `sh` resolves to Bash. As a regression test, the “potentially unsafe argument” warning has been made to actually fail the test case. To verify this, remove `:` from the list of `unusual_chars`, run the test, and note that it fails outright. wchargin-branch: shell-safe-colon
2018-09-03 06:22:53 +00:00
unusual_chars="$(printf '%s' "$arg" | sed -e 's#[A-Za-z0-9:/_.-]##g')"
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
if [ -n "${unusual_chars}" ]; then
Add ":" as a shell-safe character (#753) Test Plan: Running `./test_build_static_site.t --long -v` no longer detects the feedback URL as unsafe. (Prior to this commit, it emitted a message to this effect.) The build is still broken on Linux for other reasons, but works on macOS or any other system where `sh` resolves to Bash. As a regression test, the “potentially unsafe argument” warning has been made to actually fail the test case. To verify this, remove `:` from the list of `unusual_chars`, run the test, and note that it fails outright. wchargin-branch: shell-safe-colon
2018-09-03 06:22:53 +00:00
unsafe_arg="${arg}"
break
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
fi
done
Add ":" as a shell-safe character (#753) Test Plan: Running `./test_build_static_site.t --long -v` no longer detects the feedback URL as unsafe. (Prior to this commit, it emitted a message to this effect.) The build is still broken on Linux for other reasons, but works on macOS or any other system where `sh` resolves to Bash. As a regression test, the “potentially unsafe argument” warning has been made to actually fail the test case. To verify this, remove `:` from the list of `unusual_chars`, run the test, and note that it fails outright. wchargin-branch: shell-safe-colon
2018-09-03 06:22:53 +00:00
flags="--target $output_dir $*" # only used if ! [ -n "${unsafe_arg}" ]
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
test_expect_success EXPENSIVE,HAVE_GITHUB_TOKEN \
"${prereq_name}: ${description}" '
Add ":" as a shell-safe character (#753) Test Plan: Running `./test_build_static_site.t --long -v` no longer detects the feedback URL as unsafe. (Prior to this commit, it emitted a message to this effect.) The build is still broken on Linux for other reasons, but works on macOS or any other system where `sh` resolves to Bash. As a regression test, the “potentially unsafe argument” warning has been made to actually fail the test case. To verify this, remove `:` from the list of `unusual_chars`, run the test, and note that it fails outright. wchargin-branch: shell-safe-colon
2018-09-03 06:22:53 +00:00
if [ -n "${unsafe_arg}" ]; then
printf >&2 "fatal: potentially unsafe argument: %s\n" "${arg}" &&
false
fi &&
webpack: expose repo registry at build time (#981) Summary: We want to remove the repository selector dropdown on the cred explorer homepage and instead render a separate web page for each project. To do this, we need to know which pages to render statically. We choose to ingest this information from the state of the repository registry at build time. This commit adds an environment variable `REPO_REGISTRY` whose contents are the stringified version of the repository registry, or `null` if SourceCred has been built for the backend. This variable is defined with Webpack’s `DefinePlugin`, so any code bundled by Webpack can refer to it via `process.env.REPO_REGISTRY` both on the server and in the browser. Paired with @wchargin. Test Plan: Sharness tests modified; running `yarn test --full` suffices.
2018-11-01 19:38:18 +00:00
run '"${flags}"' >out 2>err &&
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
test_must_fail grep -vF \
Fix expected error message in build test (#771) Summary: This change should have happened in #768. However, I didn’t catch it then because `yarn test --full` passes even before this commit, despite the expected error being clearly wrong! It turns out that a very sneaky bug conspires with this one to result in the test passing no matter what kinds of warnings `yarn backend` may output. This bug is fixed in #772. Test Plan: Observe that the error message is now correct by comparing against the source in `config/RemoveBuildDirectoryPlugin.js`. Then, apply #772 and note that `yarn test --full` still passes, but does not pass when #772 is applied and this change is reverted. wchargin-branch: fix-expected-error-message
2018-09-05 17:41:53 +00:00
-e "Removing contents of build directory: " \
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
-e "info: loading repository" \
err &&
test_path_is_dir "${output_dir}" &&
test_path_is_dir "${api_dir}" &&
test_set_prereq "${prereq_name}"
'
Expose a cache directory to plugins at load time (#616) Summary: The `node ./bin/sourcecred.js load` command invokes plugin code by providing an output directory into which the plugin may store data. As of this patch, it also provides a cache directory that the plugin may use to store data that will not be available at runtime. For instance, the Git plugin might choose to clone the repository herein, or the GitHub plugin may choose to store partial GraphQL query results to deal with interruptions. The contract is that the cache directory may be removed at any time and that the plugin should continue to operate normally. Test Plan: The build script has been updated and tested. Reverting the change to the build script causes the newly added test to fail. (Each plugin has a cache directory, though the cache directories are empty for now.) wchargin-branch: create-plugin-cache
2018-08-07 20:10:15 +00:00
test_expect_success "${prereq_name}" \
"${prereq_name}: should have no cache" '
test_must_fail test_path_is_dir "${api_dir}/cache"
'
Add a configurable feedback URL to prototype (#715) Summary: We can now set, at build time, a URL to be displayed at the top of the prototype, encouraging users to provide feedback. If the URL is not provided, it defaults to the appropriate topic on the SourceCred Discourse instance. The result looks like this: ![Screenshot of the feedback URL in the prototype][screenshot] [screenshot]: https://user-images.githubusercontent.com/4317806/44814824-a238b380-ab92-11e8-88c8-dfbae27ca496.png Test Plan: Unit tests added to `yarn sharness-full` and `yarn unit`. You can run `yarn start` to see the message with the default URL, or `SOURCECRED_FEEDBACK_URL=http://example.com/ yarn start` to specify a custom URL. wchargin-branch: feedback-url
2018-08-29 22:06:12 +00:00
test_expect_success "${prereq_name}" \
"${prereq_name}: should have a bundle" '
Make js_bundle_path test POSIX-compliant (#754) Summary: In #715, I used Bash arrays for convenience. Our tests should run under POSIX `sh` (as on Travis and standard GNU/Linux). This patch reimplements the check using only POSIX features. Fixes #752. Test Plan: As is, `yarn test --full` passes on GNU/Linux and macOS(+GNU coreutils). Change the glob from `main.*.js` to `*.js` and note that running the test emits an error: ``` fatal: multiple main bundles found: build_output/output_NO_REPOS/static/js/main.6307f660.js build_output/output_NO_REPOS/static/js/ssr.e92af807.js ``` Change the glob from `main.*.js` to `nope.*.js` and note that running the test emits an error: ``` fatal: no main bundle found ``` Revert the glob to normal and note that all tests run and pass. (To run tests, `./test_build_static_site.t --chain-lint --long -v` from the `sharness/` directory.) wchargin-branch: posix-bundle-check
2018-09-03 17:51:16 +00:00
js_bundle_path= &&
js_bundle_path_glob="${output_dir}"/static/js/main.*.js &&
for main_js in ${js_bundle_path_glob}; do
if ! [ -e "${main_js}" ]; then
printf >&2 "fatal: no main bundle found\n" &&
return 1
elif [ -n "${js_bundle_path}" ]; then
printf >&2 "fatal: multiple main bundles found:\n" &&
printf >&2 " %s\n" ${js_bundle_path_glob} &&
return 1
else
js_bundle_path="${main_js}"
fi
done
Add a configurable feedback URL to prototype (#715) Summary: We can now set, at build time, a URL to be displayed at the top of the prototype, encouraging users to provide feedback. If the URL is not provided, it defaults to the appropriate topic on the SourceCred Discourse instance. The result looks like this: ![Screenshot of the feedback URL in the prototype][screenshot] [screenshot]: https://user-images.githubusercontent.com/4317806/44814824-a238b380-ab92-11e8-88c8-dfbae27ca496.png Test Plan: Unit tests added to `yarn sharness-full` and `yarn unit`. You can run `yarn start` to see the message with the default URL, or `SOURCECRED_FEEDBACK_URL=http://example.com/ yarn start` to specify a custom URL. wchargin-branch: feedback-url
2018-08-29 22:06:12 +00:00
'
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
}
# test_pages PREREQ_NAME
# Test that the PREREQ_NAME build output includes a valid home page, a
# valid prototype page, and a valid Discord invite page (which should be
# a redirect).
test_pages() {
prereq="$1"
Add a favicon (#637) Summary: In addition to the obvious benefit of having a favicon, this gets rid of a 404 Not Found error on our home page, tremendously boosting our hacker cred. Test Plan: The favicon is displayed in both `yarn start` and the static site (as a result of the build script). The added build test fails before this change. wchargin-branch: add-favicon
2018-08-10 20:15:49 +00:00
test_expect_success "${prereq}" "${prereq}: should have a favicon" '
test_path_is_file "${output_dir}/favicon.png" &&
file -b --mime-type "${output_dir}/favicon.png" >./favicon_filetype &&
printf "image/png\n" | test_cmp - ./favicon_filetype &&
rm ./favicon_filetype
'
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
test_expect_success "${prereq}" \
"${prereq}: should have a home page and a prototype" '
test_path_is_file "${output_dir}/index.html" &&
grep -qF "<script src=" "${output_dir}/index.html" &&
test_path_is_file "${output_dir}/prototype/index.html" &&
grep -qF "<script src=" "${output_dir}/prototype/index.html"
'
test_expect_success "${prereq}" \
"${prereq}: should have a discord-invite with redirect" '
file="${output_dir}/discord-invite/index.html" &&
test_path_is_file "${file}" &&
test_must_fail grep -qF "<script src=" "${file}" &&
url="https://discord.gg/tsBTgc9" &&
needle="<meta http-equiv=\"refresh\" content=\"0;url=$url\" />" &&
grep -qxF "${needle}" "${file}"
'
}
run_build TWO_REPOS \
"should build the site with two repositories and a CNAME" \
test: invoke `yarn backend` only once (#784) Summary: Lots of tests need the output of `yarn backend`. Before this commit, they tended to create it themselves. This was slow and wasteful, and also could in principle have race conditions (though in practice usually tended not to). This commit updates tests to respect a `SOURCECRED_BIN` environment variable indicating the path to an existing directory of backend applications. Closes #765. Test Plan: Running `yarn test --full` passes. Prepending `echo run >>/tmp/log &&` to the `backend` script in `package.json` and running `yarn test --full` results in a log file containing only one line, indicating that the script really is run only once. wchargin-branch: deduplicate-backend
2018-09-05 19:47:54 +00:00
--no-backend \
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
--cname sourcecred.example.com \
--repo sourcecred/example-git \
--repo sourcecred/example-github \
;
test_pages TWO_REPOS
test_expect_success TWO_REPOS \
"TWO_REPOS: should have a registry with two repositories" '
registry_file="${api_dir}/repositoryRegistry.json" &&
test_path_is_file "${registry_file}" &&
grep -oF "\"name\":" "${registry_file}" | wc -l >actual_count &&
printf "2\n" | test_cmp - actual_count
'
webpack: expose repo registry at build time (#981) Summary: We want to remove the repository selector dropdown on the cred explorer homepage and instead render a separate web page for each project. To do this, we need to know which pages to render statically. We choose to ingest this information from the state of the repository registry at build time. This commit adds an environment variable `REPO_REGISTRY` whose contents are the stringified version of the repository registry, or `null` if SourceCred has been built for the backend. This variable is defined with Webpack’s `DefinePlugin`, so any code bundled by Webpack can refer to it via `process.env.REPO_REGISTRY` both on the server and in the browser. Paired with @wchargin. Test Plan: Sharness tests modified; running `yarn test --full` suffices.
2018-11-01 19:38:18 +00:00
test_expect_success TWO_REPOS \
"TWO_REPOS: should have a repo registry loaded into env" '
grep -F "REPO_REGISTRY" out &&
core: allow repo ID registry to store metadata (#1003) Summary: Our registry was defined to simply be a list of IDs. This is insufficiently flexible; we want to be able to annotate these IDs with, e.g., last-updated times (#989). This commit wraps the entries in a simple object, updating clients appropriately. Test Plan: - Run `node ./bin/sourcecred.js load sourcecred/example-github` with a repository registry in the old format, and note that it errors appropriately. - Run `yarn build` with a repository registry in the old format, and note that it errors (“Compat mismatch”). - Delete the old registry and re-run the `load` command. Note that it runs successfully and outputs a registry. Run `yarn build`; note that this works. - Load data for two repositories. Run `yarn start`. Note that the list of prototypes still works, and that you can navigate to and render attributions for individual project pages. - Verify that `yarn test --full` passes. wchargin-branch: repo-id-registry-metadata
2018-11-10 01:28:39 +00:00
grep -xF "REPO_REGISTRY: [{\"repoId\":{\"name\":\"example-git\",\"owner\":\"sourcecred\"}},{\"repoId\":{\"name\":\"example-github\",\"owner\":\"sourcecred\"}}]" out
webpack: expose repo registry at build time (#981) Summary: We want to remove the repository selector dropdown on the cred explorer homepage and instead render a separate web page for each project. To do this, we need to know which pages to render statically. We choose to ingest this information from the state of the repository registry at build time. This commit adds an environment variable `REPO_REGISTRY` whose contents are the stringified version of the repository registry, or `null` if SourceCred has been built for the backend. This variable is defined with Webpack’s `DefinePlugin`, so any code bundled by Webpack can refer to it via `process.env.REPO_REGISTRY` both on the server and in the browser. Paired with @wchargin. Test Plan: Sharness tests modified; running `yarn test --full` suffices.
2018-11-01 19:38:18 +00:00
'
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
test_expect_success TWO_REPOS \
"TWO_REPOS: should have data for the two repositories" '
for repo in sourcecred/example-git sourcecred/example-github; do
Store GitHub data gzipped at rest (#751) Summary: We store the relational view in `view.json.gz` instead of `view.json`, taking advantage of the isomorphic `pako` library for gzip encoding and decoding. Sample space savings (note that post bodies are included; i.e., #747 has not been applied): SAVE OLD (B) NEW (B) REPO 89.7% 25326 2617 sourcecred/example-github 82.9% 3257576 555948 sourcecred/sourcecred 85.2% 11287621 1665884 ipfs/js-ipfs 88.0% 20953425 2520358 gitcoinco/web 84.4% 38196825 5951459 ipfs/go-ipfs 84.9% 205770642 31101452 tensorflow/tensorflow <details> <summary>Script to generate space savings output</summary> ```shell savings() { printf '% 7s % 11s % 11s %s\n' 'SAVE' 'OLD (B)' 'NEW (B)' 'REPO' for repo; do file="${SOURCECRED_DIRECTORY}/data/${repo}/github/view.json.gz" if ! [ -f "${file}" ]; then printf >&2 'warn: no such file %s\n' "${file}" continue fi script="$(sed -e 's/^ *//' <<EOF repo = '${repo}' pre_size = $(<"${file}" gzip -dc | wc -c) post_size = $(<"${file}" wc -c) percentage = '%0.1f%%' % (100 * (1 - post_size / pre_size)) p = '% 7s % 11d % 11d %s' % (percentage, pre_size, post_size, repo) print(p) EOF )" python3 -c "${script}" done } ``` </details> Closes #750. Test Plan: Comparing the raw old version with the decompressed new version shows that they are identical: ``` $ <~/tmp/sourcecred/data/sourcecred/example-github/github/view.json \ > shasum -a 256 - 63853b9d3f918274aafacf5198787e18185a61b9c95faf640a1e61f5d11fa19f - $ <~/tmp/sourcecred/data/sourcecred/example-github/github/view.json.gz \ > gzip -dc | shasum -a 256 63853b9d3f918274aafacf5198787e18185a61b9c95faf640a1e61f5d11fa19f - ``` Additionally, `yarn test --full` passes, and `yarn start` still loads data and runs PageRank properly. wchargin-branch: gzip-relational-view
2018-09-01 17:42:30 +00:00
for file in github/view.json.gz; do
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
test -s "${data_dir}/${repo}/${file}" || return
done
done
'
test_expect_success TWO_REPOS "TWO_REPOS: should have a correct CNAME record" '
test_path_is_file "${output_dir}/CNAME" &&
printf "sourcecred.example.com" | test_cmp - "${output_dir}/CNAME"
'
test_pages NO_REPOS
test_expect_success NO_REPOS \
"NO_REPOS: should not have a repository registry" '
registry_file="${api_dir}/repositoryRegistry.json" &&
test_must_fail test -e "${registry_file}"
'
webpack: expose repo registry at build time (#981) Summary: We want to remove the repository selector dropdown on the cred explorer homepage and instead render a separate web page for each project. To do this, we need to know which pages to render statically. We choose to ingest this information from the state of the repository registry at build time. This commit adds an environment variable `REPO_REGISTRY` whose contents are the stringified version of the repository registry, or `null` if SourceCred has been built for the backend. This variable is defined with Webpack’s `DefinePlugin`, so any code bundled by Webpack can refer to it via `process.env.REPO_REGISTRY` both on the server and in the browser. Paired with @wchargin. Test Plan: Sharness tests modified; running `yarn test --full` suffices.
2018-11-01 19:38:18 +00:00
test_expect_success NO_REPOS \
"NO_REPOS: should have empty repo registry loaded into env" '
grep -F "REPO_REGISTRY" out &&
grep -xF "REPO_REGISTRY: []" out
'
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
test_expect_success NO_REPOS \
"NO_REPOS: should not have repository data" '
for repo in sourcecred/example-git sourcecred/example-github; do
Store GitHub data gzipped at rest (#751) Summary: We store the relational view in `view.json.gz` instead of `view.json`, taking advantage of the isomorphic `pako` library for gzip encoding and decoding. Sample space savings (note that post bodies are included; i.e., #747 has not been applied): SAVE OLD (B) NEW (B) REPO 89.7% 25326 2617 sourcecred/example-github 82.9% 3257576 555948 sourcecred/sourcecred 85.2% 11287621 1665884 ipfs/js-ipfs 88.0% 20953425 2520358 gitcoinco/web 84.4% 38196825 5951459 ipfs/go-ipfs 84.9% 205770642 31101452 tensorflow/tensorflow <details> <summary>Script to generate space savings output</summary> ```shell savings() { printf '% 7s % 11s % 11s %s\n' 'SAVE' 'OLD (B)' 'NEW (B)' 'REPO' for repo; do file="${SOURCECRED_DIRECTORY}/data/${repo}/github/view.json.gz" if ! [ -f "${file}" ]; then printf >&2 'warn: no such file %s\n' "${file}" continue fi script="$(sed -e 's/^ *//' <<EOF repo = '${repo}' pre_size = $(<"${file}" gzip -dc | wc -c) post_size = $(<"${file}" wc -c) percentage = '%0.1f%%' % (100 * (1 - post_size / pre_size)) p = '% 7s % 11d % 11d %s' % (percentage, pre_size, post_size, repo) print(p) EOF )" python3 -c "${script}" done } ``` </details> Closes #750. Test Plan: Comparing the raw old version with the decompressed new version shows that they are identical: ``` $ <~/tmp/sourcecred/data/sourcecred/example-github/github/view.json \ > shasum -a 256 - 63853b9d3f918274aafacf5198787e18185a61b9c95faf640a1e61f5d11fa19f - $ <~/tmp/sourcecred/data/sourcecred/example-github/github/view.json.gz \ > gzip -dc | shasum -a 256 63853b9d3f918274aafacf5198787e18185a61b9c95faf640a1e61f5d11fa19f - ``` Additionally, `yarn test --full` passes, and `yarn start` still loads data and runs PageRank properly. wchargin-branch: gzip-relational-view
2018-09-01 17:42:30 +00:00
for file in git/graph.json github/view.json.gz; do
Create script to build static site (#592) Summary: Currently, we create the static site and deploy it all at once in `scripts/deploy.sh`. This commit creates a new script that only builds the static site. This has the advantage that it is easier/less scary to change that script (because it can be tested without worrying about deploying to a local test target), and that we can write automated tests for it. Test Plan: Run `yarn sharness`; note that it completes very quickly. Then, in a shell with your GitHub token exported, run `yarn sharness-full`. Expect all tests to pass. For a sanity check, you can run: ```shell outdir="$(mktemp -d --suffix .sourcecred-site)" ./scripts/build_static_site.sh --target "${outdir}" \ --cname sourcecred.io \ --repo sourcecred/example-git \ --repo sourcecred/example-github \ ; (cd "${outdir}" && python -m SimpleHTTPServer) ``` and ensure that <http://localhost:8000/> is as expected. One test case that is not covered is the following: _if_ the actual app somehow tries to emit a `CNAME` file at root, _and_ our script’s logic to catch this is broken, then we will not catch this failure. I’ve tested the logic manually by adding `>"${cname_file}"` after definition of that variable, but I don’t see a good way to test it automatically, without adding flags like `--but-actually-emit-cname-too` to the build. The compound probability of this happening is sufficiently low that this doesn’t bother me. wchargin-branch: build-static-site-script
2018-08-06 20:05:40 +00:00
test_must_fail test -f "${data_dir}/${repo}/${file}" || return
done
done
'
test_expect_success NO_REPOS "NO_REPOS: should have no CNAME record" '
test_must_fail test -e "${output_dir}/CNAME"
'
test_done
# vim: ft=sh