Commit Graph

52 Commits

Author SHA1 Message Date
Dandelion Mané c58315fe4d
Hackily add support for mixed GitHub/Discourse projects (#1378)
For phase one of the CredSperiment, I need a SourceCred instance which combines GitHub and Discourse servers. I'll also need to be able to give it very specific configuration to collapse certain user identities together.

Shortly after launching the CredSperiment, I plan to come back and totally re-write SourceCred's command line interface and site building system, in a way that will throw away most of the existing codebase.

As such, I found it expedient to add rather hacky and untested support for loading combined GitHub/Discourse instances, so I can land the promised features. This PR does so by:

- adding sourcecred gen-project for constructing project.json files
- adding sourcecred load --project for loading a project.json file
- ensuring that load provides the right plugins based on the project that's in scope
- updating build_static_site so that it can use the new --project flag

Test plan:
I have done some end-to-end testing, but the overall commit stack lacks automated testing. This is a deliberate tradeoff: I'm planning to re-write this section of the codebase, and the testing ergonomics are not great, so I'd rather accept some technical debt, especially since I plan to pay it off soon.

See the pull request on GitHub for the individual constituent commits.
2019-09-12 17:35:21 +02:00
William Chargin fa6697719a
update_snapshots: fix Discourse key check (#1360)
Summary:
This was doing exactly the wrong thing, attempting to update snapshots
whenever the Discourse API token was _not_ present.

Test Plan:
Running `env -u DISCOURSE_TEST_API_KEY ./scripts/update_snapshots.sh`
now successfully updates non-Discourse snapshots, rather than emitting
an error, “Please set the DISCOURSE_TEST_API_KEY environment variable.”.

wchargin-branch: update-snapshots-discourse
2019-09-07 16:27:29 -07:00
Vanessasaurus 08408a9706 Adding Docker container with instructions for running sourcecred (#1288)
Adding docker container recipe and instructions in README for running sourcecred

Signed-off-by: Vanessa Sochat <vsochat@stanford.edu>

Test plan: @decentralion verified that the commands work on a fresh setup prior to merging.
2019-08-23 13:23:35 +02:00
Dandelion Mané fd95be68a9
Add class for fetching data from Discourse (#1265)
The `DiscourseFetcher` class abstracts over fetching from the Discourse
API, and post-processing and filtering the result into a form that's
convenient for us.

Testing is a bit tricky because the Discourse API keys are sensitive
(they are admin keys) and so I'm reluctant to commit them, even for our
test instance. As a workaround, I've added a shell script which
downloads some data from the SourceCred test instance, and saves it with
a filename which is an encoding of the actual endpoint. Then, in
testing, we can use a mocked fetch which actually hits the snapshots
directory, and thus validate the processing logic on "real" data from
the server. We also test that the fetch headers are set correctly, and
that we handle non-200 error codes appropriately.

Test plan: In addition to the included tests, I have an end-to-end test
which actually uses this fetcher to fully populate the mirror and then
generate a valid SourceCred graph.

This builds on API investigations
[here](https://github.com/sourcecred/sourcecred/issues/865#issuecomment-478026449),
and is general progress towards #865. Thanks to @erlend-sh, without whom
we wouldn't have a test instance.
2019-08-15 13:22:06 +02:00
Dandelion Mané fae015496c
Include full projects on sourcecred.io (#1270)
This updates the deploys script so that we now load full projects for
@libp2p, @ipld, @sourcecred, and @filecoin-project.

I'd like to support @ipfs, but we need to tackle #1256 first.
2019-08-07 20:07:31 +02:00
Dandelion Mané c15e97b4d4 change the world: track projects not repos
This commit swaps usage over to the new implementation of `cli/load`
(the one that wraps `api/load`) and makes changes throughout the project
to accomodate that we now track instances by Project rather than by
RepoId.

Test plan: Unit tests updated; run `yarn test --full`. Also, for safety:
actually load a project (by whole org, why not) and verify that the
frontend still works.
2019-07-23 01:01:09 +01:00
Dandelion Mané 4ae502bfd3 remove old-style git loading, and its testing
I'm re-organizing SC data to be oriented on the graph, rather than on
plugin-specific data structures. So there is no longer a need for the
git loading logic which orients around saving a repository.json file
that's been potentially merged across repos, or indeed the logic for
merging repositories at all. So I'm removing `git/loadGitData`,
`git/mergeRepository`, and relatives.

Test plan: `yarn test --full` passes.
2019-07-23 01:01:09 +01:00
Robin van Boven 7509a78f65 Add --weights as load option (#1224)
Includes a change to `cli/load` and `build_static_site.sh` to accept a `--weights WEIGHTS_FILE` argument.
This allows overriding the default weights at build-time using a `weights.json` that has the same format as previously generated in the frontend.

Test plan:
Adds an additional test-case as well for propagating the optional parameter.
The file I/O of loading and parsing a weights.json file was tested manually. As analysis/weights' fromJSON() is tested elsewhere as is passing weight parameters.
2019-07-15 15:25:28 +01:00
Dandelion Mané 897c0f6eb9
Fix #1226 (#1228)
This resolves an issue where build_static_site.sh fails if the
$SOURCECRED_DIRECTORY is not set.

I tried to add a test for this, but was unsuccessful; see discussion in
[a575638].

Test plan: I manually verified that running build_static_site.sh with an
unset SOURCECRED_DIRECTORY fails before this commit, and passes
afterwards.

[a575638]: a575638fa4
2019-07-14 18:31:02 +01:00
Dandelion Mané 88f736d180
add `sourcecred/scores` (#1223)
The scores are lightly processed from their internal representation.
Example usage:

```
$ yarn backend;
$ node bin/sourcecred.js load sourcecred/sourcecred
$ node bin/sourcecred.js scores sourcecred/sourcecred > scores.json
```

The data structure is as follows:

```js
export type NodeOutput = {|
  +id: string,
  +totalCred: number,
  +intervalCred: $ReadOnlyArray<number>,
|};

export type ScoreOutput = Compatible<{|
  +users: $ReadOnlyArray<NodeOutput>,
  +intervals: $ReadOnlyArray<Interval>,
|}>;
```

Test plan: I added sharness tests at `sharness/test_cli_scores.t`.
In the past, we've used javascript tests for CLI commands. However,
those are pretty time-consuming to write, and are less robust than
simply running the command from bash. Check the snapshot for a sense of
what the new data format looks like. Also, the snapshot updater now
updates this snapshot too.

Relevant for #1047.
Thanks to @Beanow for feedback on the output format and design.
Thanks to @wchargin for help in code review.
2019-07-14 17:05:13 +01:00
Dandelion Mané ff026817d0 give build_static_site.sh access to cache
This modifies `scripts/build_static_site.sh` so that it uses the cache
available in `$SOURCECRED_DIRECTORY/cache`. This makes the script less
hermetic, but also enormously faster for regular usage.

We use a symbolic link for efficiency, and so that the user's main cache
is updated by the data loaded by build_static_site.

Test plan: I've manually verified that running build_static_site.sh is
now fast, when building repos which are already locally present in
cache. Existing tests pass. The user's cache is not removed.
2019-07-11 15:18:02 +01:00
Dandelion Mané 1f0fe3a796 Add ipfs/go-ipfs to website
I've been using it a lot as a test for a larger repo. May as well
canonicalize it.
2019-07-11 06:37:56 +01:00
William Chargin 99b2b12a14 tools: make `update_snapshots.sh` reentrant
Summary:
General cleanup to `update_snapshots.sh`, primarily such that it is free
of race conditions and does not clobber your build output directory. The
mechanism is the same as used in `yarn test`, via `SOURCECRED_BIN`.

Test Plan:
Remove your build directories (`rm -r ./bin ./build`), then run this
script (`./scripts/update_snapshots.sh`) and check that each subprocess
is properly invoked and all tests pass. Check that the temporary build
directory is cleaned up.

wchargin-branch: update-snapshots-cleanup
2019-07-04 13:45:10 +03:00
Dandelion Mané f5a46f8b31
update_snapshots.sh updates generated flow types (#1176)
The generated GitHub GraphQL flow types are a kind of snapshot, and it
can be hard to remember/discover how to update them when making changes
to schema. Therefore, I've included them in the snapshot update script.

Test plan: I used it to update the types, and it worked as intended.
2019-06-04 02:48:51 +03:00
Dandelion Mané 14eee06799
Add a universal snapshot updater (#1159)
As SourceCred has evolved, we've grown more and more snapshot tests that
are not included in Jest. The GitHub plugin has two ad-hoc snapshot
tests, the Git plugin has one, and the sharness test suites have one.

This makes it difficult to keep track of where to update snapshots when
core changes are made. To fix this, I've added a script,
`scripts/update_snapshots.sh`, which updates snapshot tests across the
project.

Test plan: I removed existing snapshots across the codebase, ran the
snapshot tester, and they correctly regenerated.
2019-05-28 18:59:50 +03:00
Dandelion Mané e465919281
Add sourcecred/{research,pm} to sourcecred.io (#1125)
Test plan: Carefully read the diff
2019-04-12 10:14:38 +02:00
Dandelion Mané eac0a3ebee
Add helpful message when missing gnu-coreutils (#1034)
As described in #1033: Currently, developers in environments without gnu
coreutils (notably: macOS) get an extremely confusing error message when
they run `yarn test`. This commit ensures that such users instead get a
helpful message with a link to the fix.

Follows @wchargin's proposed approach for fixing #1033, so that Mac
developers get clear guidance on how to get their development
environment working properly.

Expected behavior is that `yarn test` passes when GNU coreutils are
present, and fails with the following message when they are not:

Fixes #1033

```
 FAIL  check-gnu-coreutils
Exit code: 1
Contents of stdout:
    /home/dandelion/git/sc/sourcecred
Contents of stderr:
    Error: Your environment does not provide GNU coreutils
    You're likely developing on macOS.
    Please see the following link for a fix:
    https://github.com/sourcecred/sourcecred/issues/698#issuecomment-417202213
```

Test plan:

Verify that on a machine with gnu coreutils present, the command passes
and does not print irrelevant output.

Verify that on a machine without gnu coreutils available, the command
fails, and prints exactly the error message expected, without any extra
output related to the test's implementation.

Paired-with: @anthrocypher
2019-01-05 12:20:07 -08:00
Dandelion Mané 6b8cb66013
Remove cred feedback url configurability (#991)
We added a configurable cred feedback url on the theory that we would
create a separate discourse post to collect feedback for each project in
particular.

We've now realized that no one is using this, so it's just vestigial
complexity now. So I'm removing the logic for configuring the feedback
url on a per-project basis.

Instead, we will always link to a Google form for collecting feedback.

Test plan: `yarn test --full` passes, and I manually checked the links.
2018-11-01 17:43:37 -07:00
Dandelion Mané a9db2b0919
webpack: expose repo registry at build time (#981)
Summary:
We want to remove the repository selector dropdown on the cred explorer
homepage and instead render a separate web page for each project. To do
this, we need to know which pages to render statically. We choose to
ingest this information from the state of the repository registry at
build time.

This commit adds an environment variable `REPO_REGISTRY` whose contents
are the stringified version of the repository registry, or `null` if
SourceCred has been built for the backend. This variable is defined with
Webpack’s `DefinePlugin`, so any code bundled by Webpack can refer to it
via `process.env.REPO_REGISTRY` both on the server and in the browser.

Paired with @wchargin.

Test Plan:
Sharness tests modified; running `yarn test --full` suffices.
2018-11-01 12:38:18 -07:00
William Chargin 3dda4ab35c
test: invoke `yarn backend` only once (#784)
Summary:
Lots of tests need the output of `yarn backend`. Before this commit,
they tended to create it themselves. This was slow and wasteful, and
also could in principle have race conditions (though in practice usually
tended not to).

This commit updates tests to respect a `SOURCECRED_BIN` environment
variable indicating the path to an existing directory of backend
applications.

Closes #765.

Test Plan:
Running `yarn test --full` passes.

Prepending `echo run >>/tmp/log &&` to the `backend` script in
`package.json` and running `yarn test --full` results in a log file
containing only one line, indicating that the script really is run only
once.

wchargin-branch: deduplicate-backend
2018-09-05 12:47:54 -07:00
William Chargin c48597651e
backend: invoke Webpack directly in package script (#782)
Summary:
This commit removes the `config/backend.js` script and replaces it with
a direct invocation of Webpack. This enables us to use command-line
arguments to Webpack, like `--output-path`.

Test Plan:
Note that `rm -rf bin; yarn backend` still works, and that the resulting
applications work (`node bin/sourcecred.js load`). Note that `yarn test`
and `yarn test --full` still work.

wchargin-branch: backend-webpack-direct
2018-09-05 12:28:27 -07:00
William Chargin 4e8c89abfe
backend: export a normal Webpack config object (#779)
Summary:
Previously, our `webpack.config.backend.js` file actually exported a
function that could be used to make a Webpack configuration object.
(This is not to be confused with the late `makeWebpackConfig.js`, which
actually exported a configuration object!)

In addition to being confusing nomenclature, this was a sneaky trap for
CLI users. Invoking `webpack --config config/webpack.config.backend.js`
would actually work, but do the wrong thing: Webpack _allows_ your
configuration object to be a function, but with different semantics. In
particular, the result was that Webpack would emit the build output into
your current directory instead of into `bin/`.

This commit fixes that by making `webpack.config.backend.js` export the
Webpack configuration object for the backend JavaScript applications.
The logic to change the path is now handled by the caller, by
overwriting `config.output.path`; this is exactly [the same approach
that the Webpack CLI takes when given an `--output-path`][1], so it’s
okay with me.

[1]: 368e2640e6/bin/convert-argv.js (L406-L409)

Test Plan:
Run `yarn backend` and `yarn backend --dry-run`. Note that each runs
with appropriate output (both emitted files and console logs).

wchargin-branch: backend-webpack-config-object
2018-09-05 11:58:20 -07:00
William Chargin 846363465d
backend: remove file size measurement logic (#778)
Summary:
This logic isn’t currently useful to us. If we want this functionality,
we should consider using a Webpack plugin like `size-limit`. In the
meantime, this removes functionality from `scripts/backend.js`,
continuing on the path to #765.

Recommend reviewing with `-w`.

Test Plan:
Run `yarn backend` and `yarn backend --dry-run`, noting that each works.

wchargin-branch: backend-remove-file-sizes
2018-09-05 11:41:00 -07:00
William Chargin c8175b9e3c
backend: remove superfluous file existence check (#774)
Summary:
Webpack will fail (quickly) if any required entry points do not exist.
The `scripts/backend.js` script superfluously checks this, too. This
patch removes that check.

Test Plan:
In `config/paths.js`, change `src/cli/main.js` to `src/cli/wat.js`.
Then, `yarn backend`, and note that Webpack fails quickly, with an error
“entry module not found”. Note that Webpack fails equally quickly if you
change the path of the last entry point rather than the first one.

wchargin-branch: backend-remove-exists-check
2018-09-05 11:26:22 -07:00
William Chargin 8c0bbbc732
backend: move build dir removal into Webpack (#776)
Summary:
Both the backend and the web builds want to empty the build directory
before starting. This commit makes them use the same codepath, reducing
the amount of work that `scripts/backend.js` does so that we can more
easily remove it (#765).

Test Plan:

```shell
$ touch ./bin/wat
$ yarn backend >/dev/null 2>/dev/null
$ file ./bin/wat
./bin/wat: cannot open `./bin/wat' (No such file or directory)
```

wchargin-branch: backend-empty-backend-build-directory
2018-09-05 11:25:52 -07:00
William Chargin 08f6602389
build_static_site.sh: add --feedback-url help text (#722)
Summary:
This improves the documentation for the change made in #715.

Test Plan:
Note that `./scripts/build_static_site.sh  --help` and `yarn sharness`
pass.

wchargin-branch: feedback-url-help-text
2018-08-29 16:51:28 -07:00
William Chargin d4202b2304
Add a configurable feedback URL to prototype (#715)
Summary:
We can now set, at build time, a URL to be displayed at the top of the
prototype, encouraging users to provide feedback. If the URL is not
provided, it defaults to the appropriate topic on the SourceCred
Discourse instance.

The result looks like this:

![Screenshot of the feedback URL in the prototype][screenshot]

[screenshot]: https://user-images.githubusercontent.com/4317806/44814824-a238b380-ab92-11e8-88c8-dfbae27ca496.png

Test Plan:
Unit tests added to `yarn sharness-full` and `yarn unit`.

You can run `yarn start` to see the message with the default URL, or
`SOURCECRED_FEEDBACK_URL=http://example.com/ yarn start` to specify a
custom URL.

wchargin-branch: feedback-url
2018-08-29 15:06:12 -07:00
William Chargin 66cf3b3aba
ensure-flow.sh: simplify, removing dep on GNU grep (#602)
Summary:
By using Git’s magic pathspecs instead of post-processing stream
operations, we reduce the pipeline to a single operation. Git implements
its own version of `grep`, so this should be platform-independent.
Previously, we had needed the `-z` argument to `grep(1)`, which is a GNU
extension.

Fixes #594.

Test Plan:
Ensure that the script passes with no output. Then,

```shell
mkdir -p src/flow-typed/
touch src/foo.js src/flow-typed.js src/flow-typed/foo.js
git add src/foo.js src/flow-typed.js src/flow-typed/foo.js
cd scripts
./ensure-flow.js
```

and ensure that script exits with code 1, empty stdout, and stderr with:

```
../src/flow-typed.js
../src/flow-typed/foo.js
../src/foo.js
```

This verifies that the pathspec is properly excluding the root directory
`flow-typed`, but not files that just happen to have `flow-typed` in
their paths.

wchargin-branch: simplify-ensure-flow
2018-08-22 11:44:20 -07:00
William Chargin d839fcae95
build_static_site.sh: create target if nonexistent (#695)
Summary:
The current version of the build script has the safe but annoying
property that the target directory must be an existing, empty directory.
It seems reasonable and convenient to allow the build script to create
the directory with `mkdir -p`. It still fails if the directory is not
empty or is a file.

Test Plan:
Unit tests updated; run `yarn sharness-full`.

wchargin-branch: build-mkdir-p
2018-08-21 15:35:13 -07:00
William Chargin c02a81ff43
Expose a cache directory to plugins at load time (#616)
Summary:
The `node ./bin/sourcecred.js load` command invokes plugin code by
providing an output directory into which the plugin may store data.
As of this patch, it also provides a cache directory that the plugin may
use to store data that will not be available at runtime. For instance,
the Git plugin might choose to clone the repository herein, or the
GitHub plugin may choose to store partial GraphQL query results to deal
with interruptions. The contract is that the cache directory may be
removed at any time and that the plugin should continue to operate
normally.

Test Plan:
The build script has been updated and tested. Reverting the change to
the build script causes the newly added test to fail. (Each plugin has a
cache directory, though the cache directories are empty for now.)

wchargin-branch: create-plugin-cache
2018-08-07 13:10:15 -07:00
William Chargin 383f8d406e
deploy.sh: depend on build_static_site.sh (#611)
Summary:
This is a minimal patch to make the deploy script depend on the
separate, well-tested build script introduced in #592. The patch
suggests a number of improvements that can be made to the deploy script
itself, but I’m deferring those as they’re not on the critical path.
These are tracked at #610.

Test Plan:
Apply the below patch. Then run `./scripts/deploy.sh -n`, and verify
that the served output is correct, including the CNAME file. Verify that
when the `DEPLOY_CNAME_URL` default value is removed, the resulting
output does _not_ have a CNAME file.

Patch, for ease of building:

```diff
diff --git a/scripts/deploy.sh b/scripts/deploy.sh
index cddcece..8925e5b 100755
--- a/scripts/deploy.sh
+++ b/scripts/deploy.sh
@@ -23,7 +23,7 @@ main() {
     preview_dir=
     trap cleanup EXIT

-    ensure_clean_working_tree
+    #ensure_clean_working_tree
     build_and_deploy
 }

@@ -76,8 +76,7 @@ build_and_deploy() {
     "${sourcecred_repo}/scripts/build_static_site.sh" \
         --target "${static_site}" \
         ${DEPLOY_CNAME_URL:+--cname "${DEPLOY_CNAME_URL}"} \
-        --repo ipfs/js-ipfs \
-        --repo sourcecred/sourcecred \
+        --repo sourcecred/example-github \
         ;

     sourcecred_site="$(mktemp -d --suffix ".sourcecred-site")"
```

wchargin-branch: deploy-via-build
2018-08-06 19:00:35 -07:00
William Chargin d7cb4c65fa
Create script to build static site (#592)
Summary:
Currently, we create the static site and deploy it all at once in
`scripts/deploy.sh`. This commit creates a new script that only builds
the static site. This has the advantage that it is easier/less scary to
change that script (because it can be tested without worrying about
deploying to a local test target), and that we can write automated tests
for it.

Test Plan:
Run `yarn sharness`; note that it completes very quickly. Then, in a
shell with your GitHub token exported, run `yarn sharness-full`. Expect
all tests to pass.

For a sanity check, you can run:

```shell
outdir="$(mktemp -d --suffix .sourcecred-site)"
./scripts/build_static_site.sh --target "${outdir}" \
    --cname sourcecred.io \
    --repo sourcecred/example-git \
    --repo sourcecred/example-github \
    ;
(cd "${outdir}" && python -m SimpleHTTPServer)
```

and ensure that <http://localhost:8000/> is as expected.

One test case that is not covered is the following: _if_ the actual app
somehow tries to emit a `CNAME` file at root, _and_ our script’s logic
to catch this is broken, then we will not catch this failure. I’ve
tested the logic manually by adding `>"${cname_file}"` after definition
of that variable, but I don’t see a good way to test it automatically,
without adding flags like `--but-actually-emit-cname-too` to the build.
The compound probability of this happening is sufficiently low that this
doesn’t bother me.

wchargin-branch: build-static-site-script
2018-08-06 13:05:40 -07:00
William Chargin dbf5caa08f
Permit STOPSHIPs in Markdown files (#573)
Summary:
This way, we can document what STOPSHIPs are without having to tip-toe
around the phrase.

Test Plan:
Add `STOPSHIP` to `README.md`, and note that `yarn test` fails before
this change but passes after it.

wchargin-branch: stopship-markdown-okay
2018-07-31 10:58:29 -07:00
William Chargin 3b5ad594bd
package.json: reorganize test commands (#571)
Summary:
Running `yarn test` (equiv. `npm test` or `npm run test`) now runs all
checks. It takes the place of the former `yarn travis`. This is more in
line with the expectation of a top-level `test` command: if it passes,
your code is good.

The `unit` command now runs Jest once, not in watch mode. It takes the
place of the former `ci-test`. To run tests in watch mode, run any of
the following:

  - `yarn unit --watch`, or
  - `npm run unit -- --watch`, or
  - `npm unit -- --watch`.

This behavior is more consistent with the standard behavior of commands
like `make test`. It is also empirically what @wchargin and
@decentralion want most of the time.

Test Plan:
Verify that each of the scripts `test`, `unit`, and `coverage` passes.
Verify that each of the aforementioned `--watch` invocations works.
Verify that `.travis.yml` has the correct `script:` command.

wchargin-branch: reorganize-test-command
2018-07-31 10:53:10 -07:00
William Chargin c1cb29b1e6
new-webpack: remove old scripts and configs (#570)
Summary:
These are no longer used.

Test Plan:
Both `yarn start` and `yarn build` still work.

wchargin-branch: webpack-remove-old
2018-07-30 18:14:37 -07:00
Dandelion Mané 3266eb31fa
CLI takes repo strings as `owner/name` (#559)
Test plan:
```
$ yarn backend
$ node bin/sourcecred.js load sourcecred/sourcecred
```
2018-07-27 23:44:41 -07:00
Dandelion Mané dd09e28d6e
Make deploy script wait for valid response (#557)
Currently, any non-yes keystroke causes the deply script to abort.
This is frustrating, as it may take the user several minutes of waiting
to get to the prompt, only to have the program inadvertently terminate.

As of this change, the deploys script patiently re-prompts until it gets
a valid input.

Paired with @wchargin.

Test plan: Ran locally and tested the yes, no, and invalid response
cases. Also ran the script through shellcheck.
2018-07-27 21:30:20 -07:00
Dandelion Mané 81ccda773a
Deploy script adds necessary CNAME file (#558)
The CNAME file is needed so that our custom domain will continue working
after deploys.

Test plan:
- Verified that the generated build now includes the cname file.
- Verified that if a CNAME file is already present, the script will
fail.

Paired with @wchargin
2018-07-27 21:22:39 -07:00
Dandelion Mané e845e8cbbc
Modify sourcecred.github.io's example repos (#533)
Showing our example-github and example-git repos on sourcecred.github.io
is not particularly interesting. Let's show ipfs/js-ipfs instead!

Since sourcecred/sourcecred is the last repo to load, as of #531 it will
be the default option.

Test plan: Dry run of deploy script
2018-07-26 21:57:43 -07:00
Dandelion Mané a0c07bf0d7
Import the sourcecred.github.io deploy script (#532)
This copies the deploy script from sourcecred/sourcecred.github.io into
the sourcecred/sourcecred repository. It's been modified to reflect the
fact that it is no longer run from within the sourcecred.github.io
repository.

We’re moving this script into the main SourceCred repository so that it
can be versioned in the same tree as everything else, and in particular
so that the hash-of-built-commit depends on the build script itself.

The script was originally created by @wchargin in
https://github.com/sourcecred/sourcecred.github.io/pull/1 at hash
e8bc5b669b31df34b439b194cb62af4fd9a789d8, subsequently modified in
https://github.com/sourcecred/sourcecred.github.io/pull/2 and
https://github.com/sourcecred/sourcecred.github.io/pull/3 and
https://github.com/sourcecred/sourcecred.github.io/pull/4

Paired with @wchargin

Test plan: Ran locally (on dry-run) and inspected output.
2018-07-26 21:05:18 -07:00
William Chargin 2be413b77c
Unify a command-line entry point module (#344)
Summary:
For now, this contains the logic to register an `unhandledRejection`
error. I’ve removed all instances of those handlers, and `require`d this
module at every top-level entry point. (The individual CLI commands had
the handler before, but didn’t need it; conversely, the top-level CLI
entry point did not have the handler, but should have.)

Test Plan:
To test that the CLI commands still error on unhandled rejections, apply
the following patch:

```diff
diff --git a/src/v1/cli/commands/combine.js b/src/v1/cli/commands/combine.js
index b60f91e..d55b965 100644
--- a/src/v1/cli/commands/combine.js
+++ b/src/v1/cli/commands/combine.js
@@ -24,6 +24,7 @@ export default class CombineCommand extends Command {
     "  where each GRAPH is a JSON file generated by plugin-graph";

   async run() {
+    Promise.reject("wat");
     const {argv} = this.parse(CombineCommand);
     combine(argv);
   }
```

Then run `yarn backend` and `node bin/sourcecred.js`, and note that the
rejection handler is triggered.

wchargin-branch: unify-entry
2018-06-05 11:11:48 -07:00
William Chargin f8242c8cab
Don’t erase the `bin/` folder in CI (#317)
Summary:
Previously, our CI script would run `yarn backend`, which has the
side-effect of erasing the `bin/` directory. By itself, this is not
great, but not awful. However, this frequently triggers a race condition
in Prettier, causing the `check-pretty` step of the build to fail. (More
details: https://github.com/prettier/prettier/issues/4468.)

This patch changes the CI script to build the backend scripts into a
temporary directory.

Test Plan:
Before applying this patch: `yarn backend` and then `yarn travis`. If
this consistently causes a Travis failure due to `check-pretty`, then
your machine can reproduce the race condition that we‛re trying to
eliminate. (Otherwise, you can try creating a bunch more Git history…
I’m not really sure what to say. It is a race condition, after all.)
Then, apply this patch, and repeat the above steps; note that the error
no longer occurs, and that the build output is to a temporary directory.

wchargin-branch: ci-preserve-bin
2018-05-29 15:40:42 -07:00
William Chargin 6663c4f8ad
Fix `ensure-flow.sh` running under Node (#314)
Summary:
The use of `tee /dev/stderr` failed when running as a child process
under Node for some reason. (I haven’t been able to figure out why—it
works fine when run as a standalone script or when run as a child
process under Python.) This is also technically Linux-specific, so I’ve
changed it to use a process substitution. After looking around for a
bit, there doesn’t seem to be a way to do this in a way that is
portable, uses only POSIX shell features, and doesn’t create temporary
files all at the same time, so the script is now run under `bash`.

Test Plan:
Run `yarn travis` and note that the `ensure-flow.sh` output no longer
contains the line `tee: /dev/stderr: No such device or address`.

wchargin-branch: no-tee-devstderr
2018-05-29 12:20:53 -07:00
William Chargin f0fcf02791
Check for STOPSHIPs in CI (#301)
Summary:
Placing `STOPSHIP` or `stopship` (or any case variant) in any file
tracked by Git will now cause a `yarn travis` failure. If you need to
use this string, you can concatenate it as `"stop" + "ship"` or
equivalent.

Test Plan:
In `travis.js`, change `"check-stop" + "ships"` to `"check-stopships"`,
and note that this causes the build to fail with a nice message. Note
that this also causes `check-stopships.sh` to fail even when invoked
from an unrelated directory, like `src`.

wchargin-branch: check-stopships
2018-05-25 19:27:31 -07:00
William Chargin 47bec6cc10
Make `ensure-flow.sh` more precise and accurate (#259)
Summary:
This fixes two problems in the previous version:
  - A new JS file not checked into git, but with a `@flow` directive,
    would cause `ensure-flow` to fail, because one list of files was
    from `git grep` and the other was from `find`.
  - Only the hard-coded directories `src config scripts` were searched.

Now, we search all JS files checked into Git, except for some hard-coded
exceptions, namely `flow-typed`.

Test Plan:
  1. Add `foo.js`, not checked into Git. Note that `ensure-flow` passes.
  2. Add `@flow` to `foo.js`, and note that `ensure-flow` still passes.
  3. Remove `@flow` from `.eslintrc.js`, and note that `ensure-flow`
     fails and nicely prints the filename. (Note: this file is at the
     repository root.)
  4. Create a file `echo stuff >$'naughty\nfilename.js'`, and note that
     `ensure-flow` has the correct behavior in both positive and
     negative cases.

wchargin-branch: ensure-flow-improvements
2018-05-10 12:38:39 -07:00
William Chargin 9ea1f981aa
Proxy Webpack dev server through to an API server (#245)
Summary:
This way, our frontend can talk to a backend that can read from the
filesystem (among other things).

Paired with @decentralion.

Test Plan:
```
$ yarn backend
$ SOURCECRED_DIRECTORY=/tmp/srccrd yarn start
$ # verify that the browser looks good
$ mkdir /tmp/srccrd
$ echo hello >/tmp/srccrd/world
$ curl localhost:3000/api/v1/data/world
hello
$ curl localhost:4000/api/v1/data/world
hello
```

wchargin-branch: webpack-proxy
2018-05-08 16:09:37 -07:00
William Chargin 2e8653a3ee
Turn on flow for scripts/start.js (#242)
Summary:
  - The value of `process.stdout.isTTY` is either `true` or `undefined`.
    Flow (reasonably) dislikes this, so we add an explicit check.
  - More `package.json` burnination.

Test Plan:
Note that `require("./package.json").proxy === undefined` in the Node
console, and that `yarn start` works.

wchargin-branch: flow--scripts-start
2018-05-08 14:51:55 -07:00
Dandelion Mané 824df7e916
Turn on flow for scripts/{backend,build,test}.js (#241)
- scripts/backend.js: We incorrectly set an environment variable to
a boolean, when in fact it must be a string. Fixed it to set a string
value "true", and updated usage in config/babel.js
- scripts/test.js: No changes
- scripts/build.js: Removed a call to printHostingInstructions, so that
we don't need to require the package.json.

Test plan:
`yarn travis --full` passes, and the SourceCred cli still works.
2018-05-08 14:35:56 -07:00
Dandelion Mané 93e2798f37
Ensure that flow is used in all js files (#232)
This script ensures that either //@flow or //@no-flow is present in
every js file. Every existing js file that would fail this check has
been given //@no-flow, we should work to remove all of these in the
future.

Test plan:
I verified that `yarn travis` fails before fixing the other js files,
and passes afterwards.
2018-05-07 20:02:19 -07:00
William Chargin e9dbdeca96
Target latest Node for backend applications (#213)
Summary:
Consequently, Babel won’t transform classes to their roughly equivalent
ES5 counterparts, etc.

Test Plan:
Create `src/classy.js` with `class X {}; console.log(X);`. Then, add a
build target for `classy: resolveApp("src/classy.js"),` in `paths.js`.
Use `yarn backend` and inspect the contents of `bin/classy.js`; in
particular, look at the definition of `X` (whatever the argument to
`console.log` is). Before this commit, the result will be a big
complicated mess. After this commit, it will be `class X {}`.

Note also that `yarn travis --full` passes, indicating that the two
manual tests, which call out to the utilities in `bin/`, still work.

wchargin-branch: target-node
2018-05-04 19:22:39 -07:00