Commit Graph

91 Commits

Author SHA1 Message Date
Dandelion Mané 9b105ee4ce Add `core/project` and `core/project_io`
This creates a new `Project` type which will replace `RepoId` as the
index type for saving and loading data.

The basic data type is added to `project.js`. Rather than having a
`RepoIdRegistry`, I intend to infer the registry at build time by
scanning for available projects saved in the sourcecred directory. I've
added the `project_io` module for this task. It has methods for setting
up a project subdirectory, and loading the `Project` info from that
subdirectory.

To ensure that projects ids can be encoded even if they have symbols
like `/` and `@`, we base64 encode them.

To ensure that project ids can be retrieved at build time, the
`getProjectIds` method is factored out into its own plain ECMAScript
module. For all non-build time needs, it is re-exported from
`project_io`.

Test plan: Unit tests added; run `yarn test`.
2019-07-21 12:24:10 +01:00
greenkeeper[bot] 34117642cf chore(package): update flow-bin to version 0.103.0 2019-07-19 01:17:01 +01:00
greenkeeper[bot] 3dfc209f76 chore(package): update eslint-plugin-flowtype to version 3.12.0 2019-07-18 18:50:54 +01:00
Dandelion Mané d88c72f612
Update babel dependencies (#1236)
* chore(package): update @babel/core to version 7.5.5

* chore(package): update @babel/plugin-proposal-class-properties to version 7.5.5

* chore(package): update @babel/preset-env to version 7.5.5

Test plan: CI passes.
2019-07-18 14:04:05 +01:00
greenkeeper[bot] 8d8804c246 Update dependencies to enable Greenkeeper 🌴 (#1231)
* chore(package): update dependencies

* revert tmp upgrade

I'm having test failures when `tmp` is upgraded; they seem to repro only
when many tests are running at once. Since we have no issues with the
older version of tmp, let's just keep an old tmp and inform Greenkeeper
not to touch it.

Test plan: `yarn test`
2019-07-18 14:02:52 +01:00
Dandelion Mané 493d7332c6 tweak: jest loads .mjs last
This resolves an issue that caused jest tests to fail when depending on
a module that ships .mjs files (encountered via a transitive dep of
react-markdown).

See https://github.com/facebook/create-react-app/pull/4085 for context.

Test plan: I have a future commit which tests a file that depends on
react-markdown. The tests fail to run before this commit, and they run
without issue afterwards.
2019-07-14 20:11:30 +01:00
Dandelion Mané 8e0bbcf597 Change version to 0.3.0 2019-07-11 21:53:11 +01:00
Dandelion Mané 5dc7f440ce Initial Timeline Explorer
This commit adds a TimelineExplorer for visualizing timeline cred data.
The centerpiece is the TimelineCredChart, a d3-based line chart showing
how the top users' cred evolved over time. It has features like tooltips,
reasonable ticks on the x axis, a legend, and filtering out line
segments that stay on the x axis.

An inspection test is included, which you can check out here:
http://localhost:8080/test/TimelineCredView/

Also, you can run it for any loaded repository at:
http://localhost:8080/timeline/$repoOwner/$repoName

This commit also includes new dependencies:
- recharts (for the charts)
- react-markdown (for rendering the Markdown descriptions)
- remove-markdown (so the legend will be clean text)
- d3-time-format for date axis generation
- d3-scale and d3-scale-chromatic for color scales

Test plan: The frontend code is mostly untested, in keeping with my
observation that the costs of testing the old explorer were really high,
and the tests brought little benefit. However, I have manually tested it
thoroughly. Also, there is an inspection test for the TimelineCredView
(see above).
2019-07-11 06:33:41 +01:00
Dandelion Mané a0b754bb43 upgrade to webpack 4
Now that babel is upgraded, upgrading webpack was pretty
straightforward.

- We take advantage of the new `mode` config option, and no longer need
to manually set up Uglify plugin
- Uglifyjs is back, I checked the prod build output: it's very ugly
- I updated the RemoveBuildDirectoryPlugin per instructions, and
verified it still works.
- I verified that of `yarn backend`, `yarn build`, and `yarn start` all
still work as expected.
2019-07-11 05:52:54 +01:00
Dandelion Mané f77ff3ecd0 Upgrade babel to 7
I moved `config/babel.js` to `.babelrc.js` because it seemed like babel
7 really wanted that. I also blew away our (complicated, copied from
create-react-app) config and replaced it with a much, much simpler one.

Test plan: `yarn test` passes, `yarn start` still serves a working
server, and `scripts/build_static_site.sh` still produces a working
site.

Possibly we lost some nice features re: React debugging; if so I'll add
them back as I miss them.
2019-07-11 05:52:54 +01:00
Dandelion Mané 2335c5d844 add `analysis/timeline/interval`
This commit adds an `interval` module which defines intervals (time
ranges), and methods for slicing up a graph into its consistuent time
intervals. This is pre-requisite work for #862.

I've added a dep on d3-array.

Test plan: Unit tests added; run `yarn test`
2019-07-11 01:30:27 +01:00
Dandelion Mané b7eae8b3eb
Switch to flow v0.100.0 to try to fix CI issues (#1203)
Since upgrading to flow 0.102.0, we've been having CI issues where flow
fails as "out of retries". In my testing, downgrading flow seems to
resolve this, although it's hard to be certain as the issue strikes
sporadically.

Test plan: This commit definitely works locally; hopefully it will
consistently pass in CI as well.
2019-07-06 14:39:32 +01:00
Dandelion Mané 0ade04b66c
try reverting tmp upgrade (#1202)
After merging #1201, we started seeing build failures ([1], [2]) on
CircleCI. I can't reproduce them locally, but they seem related to the
`tmp` module, so let's try reverting the upgrade.

Test plan: `yarn test` passes locally. Let's see how it does on CI.

[1]: https://circleci.com/gh/sourcecred/sourcecred/1091
[2]: https://circleci.com/gh/sourcecred/sourcecred/1092
2019-07-06 00:22:04 +01:00
greenkeeper[bot] 4758cea2f8 Update dependencies to enable Greenkeeper 🌴 (#1201)
This commit enables Greenkeeper, along with an initial upgrade push for our dependencies.

I've reverted a number of upgrades, and also added them to the
greenkeeper ignore list. They mostly relate to babel/webpack stuff,
which I'm reluctant to dive into now (there have been major upgrades for
both babel and webpack) but we should address eventually. There are also
a few oddballs like whatwg-fetch and history.

Test plan: `yarn test --full` passes.
2019-07-05 20:10:02 +01:00
Dandelion Mané 125e8b0486 Tweak package.json
- Remove obsolete eslint react app config
- Pin webpack major version

Test plan: yarn && yarn test
2019-07-05 19:34:35 +01:00
Dandelion Mané 7a493b596d Update eslint and eslint configuration
This commit updates eslint from v4 to v6. In doing so, I've moved off of
the create-react-app base eslint config. We were on an old version (v2)
and it doesn't make sense to update to v4, as in v4 create-react-app
uses typescript. Also, it didn't make sense to stay on
create-react-app's v2 config, because then it had unmet peer dependency
constraints on old versions of eslint.

Instead, I've moved us to use the default rules for eslint,
eslint-plugin-react, and eslint-plugin-flowtype.

I also made some changes to the codebase to satisfy the new lint rules
that came with this change.

Test plan: `yarn test` passes.
2019-07-05 18:39:00 +01:00
Dandelion Mané eadcca8999 Upgrade flow to to 0.102.0
This necessitated a number of type fixes:
- Upgraded the express flow-typed file to latest
- Added manual flow error suppression to where the express flow-typed
file is still using a deprecated utility type
- Removed type polymorphism support on map.merge (see context here[1]).
We weren't using the polymorphism anywhere so I figured it was simplest
to just remove it.
- Improve typing around jest mocks throughout the codebase.

Test plan: `yarn test --full` passes.

[1]: https://github.com/flow-typed/flow-typed/issues/2991
2019-07-05 17:21:56 +01:00
Dandelion Mané 230756ffec Upgrade jest from v23 to v24
Just some general housekeeping. `yarn test --full` passes without issue.
2019-07-04 18:57:50 +01:00
Dandelion Mané 6a13248b09 Upgrade prettier
This commit updates our prettier version from `1.13` to `1.18`. Looks
like software does get better over time! I like all of the changes.

Test plan: `yarn test` passes. I've manually inspected the diffs.
2019-07-04 20:33:42 +03:00
Dandelion Mané 29c9229c28 Update better-sqlite3 to v5
When we took a dep on better-sqlite3 in #836, we used a fork, because
better-sqlite3 did not yet support private in-memory databases via the
`:memory:` filepath. As of better-sqlite3 v5, this has been added to
mainline, so we no longer need the fork.

The v4->v5 transition involves some breaking changes. The only ones that
affected us were two field renames, from `lastUpdateROWID` to
`lastUpdateRowid`, and `returnsData` to `reader`.

Test plan:
After updating the field accesses, `yarn test --full` passes. For added
safety, I also blew away cache, loaded a nontrivial repository, and
verified that the full cred workflow still works.

cc @wchargin
2019-07-04 20:31:32 +03:00
Dandelion Mané fb89559e44
Add a FileUploader component, with inspection test (#1149)
* Add FileUploader with inspection test

TODO: get it working

* Add a FileUploader component, with inspection test

This adds a FileUploader component, which allows the user to upload JSON
files. Rather than using automated testing, it has an inspection test.
The inspection test may be run by navigating to:
http://localhost:8080/test/FileUploader/

This commit also adds some basic utility functions for defining
inspection tests to `routeData.js`. We should improve support for
inspection tests in the future; see [#1148].

[#1148]: https://github.com/sourcecred/sourcecred/issues/1148

Test plan: Ran the included inspection test.
2019-05-20 17:12:57 +03:00
Brian Litwin 0f038305a2 Add CLI command to clear sourcecred data directory (#1111)
Resolves #1067

Adds the CLI commands:
`sourcecred clear --all` -- removes the $SOURCECRED_DIRECTORY
`sourcecred clear --cache` -- removes the cache directory
`sourcecred clear --help` -- provides usage info
`sourcecred clear` -- prompts the user to be more specific

Test plan:
The unit tests ensure that the command is properly wired into the
 sourcecred CLI, including help text integration. However, just to be
safe, we can start by verifying that calling `sourcecred` without
arguments lists the `clear` command as a valid option, and that
calling `sourcecred help clear` prints help information. (Note: it's
necessary to run `yarn backend` before testing these changes)

The unit tests also ensure that the command removes the proper
directories, so there isn't really a need to manually test it,
although the reviewer may choose to do so to be safe.

Although out of scope for unit tests on this function, we can also do
integration tests, to make sure that running the clear command doesn't
leave the sourcecred directory in an invalid state from the perspective of the `load` command.

```js
$ yarn backend;
$ node bin/sourcecred.js load sourcecred/example-github;
$ node bin/sourcecred.js clear --cache;
$ node bin/sourcecred.js load sourcecred/example-github;
$ node bin/sourcecred.js clear --all;
$ node bin/sourcecred.js load sourcecred/example-github;
```
The expected behavior of the above command block is that the load command never fails or throws an error.

@decentralion and I discussed the scenario where `rimraf` errors.
We decided that testing this scenario wasn't necessary, because
`rimraf` doesn't error if a directory doesn't exist, and
rimraf's maintainer suggests [monkey-patching the fs module]
to get rimraf to error in testing scenarios.

Thanks @decentralion for reviewing and pair-programming this with me.

[monkey-patching the fs module]: https://github.com/isaacs/rimraf/issues/31#issuecomment-29534796
2019-05-13 12:59:58 +03:00
Dandelion Mané bed476517c
Port skeleton of Odyssey frontend (#1132)
This commit integrates an bare skeleton of the odyssey frontend that we
implemented in the [odyssey-hackathon] repository. You can see the
working frontend that we are trying to port over at
[sourcecred.io/odyssey-hackathon/][scio].

The prototype in the other repository has some tooling choices which are
incompatible/redundant with decisions in our codebase (sass vs
aphrodite), and requires some tools not yet present here
(svg-react-loader). This commit includes the build and integration work
needed to port the prototype frontend into mainline SourceCred. The
frontend scaffold isn't yet integrated with any "real" Odyssey data.

One potential issue: right now, every page that is rendered from the
SourceCred homepage is contained within a [homepage/Page], meaning that
it has full SourceCred website styling, along with the SourceCred
website header. The [application][scio] also has a header. Currently, I
work around this by having the Odyssey UI cover up the base header (via
absolute positioning), which works but is hacky. We can consider more
principled solutions:

- Finding a way to specify routes which aren't contained by
[homepage/Page]; maybe by adding a new top-level route
[here][route-alternative].
- Unify the headers for the Odyssey viewer and the page as a whole
(sounds like inappropriate entanglement?)
- Have a website header and also an application header (sounds ugly?)

[homepage/Page]: ee1d2fb996/src/homepage/Page.js
[route-alternative]: ee1d2fb996/src/homepage/createRoutes.js (L17)

Test plan: Run `yarn start`, and then navigate to
`localhost:8080/odyssey/`. observe that a working website is displayed,
and that the cred logo next to the word "SourceCred" is loaded properly
(i.e. svg-react-loader is integrated properly). Observe that there are
no build/compile errors from either `yarn start` or `yarn build`. Also,
observe that the UI looks passably nice, and that if the number of
elements in the entity lists is larger than can be displayed, the
sidebar pane scrolls independently.

The UI was tested in both Chrome and Firefox.

[odyssey-hackathon]: https://github.com/sourcecred/odyssey-hackathon
[scio]: https://sourcecred.io/odyssey-hackathon/

Thanks to @jmnemo, as the implementation is based on [his work].

[his work]: https://github.com/jmnemo/hackathon-event/
2019-05-06 18:15:39 +03:00
Dandelion Mané 5d3102e4db
Tests use yarn not npm (#1038)
Currently, our underlying test script uses npm rather than yarn to
execute the tests. This is awkward, because we use yarn everywhere else
in lieu of npm. It turns out that some setups have node available
without npm, and in such environments our tests fail with a cryptic
ENOENT error.

This changes the tests to use yarn instead.

Test plan: `yarn test --full` passes.

Thanks to @wpank for help uncovering this issue.
2019-01-07 14:38:21 -08:00
William Chargin 332e776317
deps: upgrade `flow-bin@^0.86.0` (#1002)
Summary:
There have been some breaking changes that require new type annotations,
which is a good thing: these prevent `any`-leakage.

Test Plan:
Run `yarn flow`.

wchargin-branch: flow-v0.86.0
2018-11-09 09:24:40 -08:00
William Chargin a653638e42
test: run Sharness verbosely on nightly jobs (#958)
Summary:
Tests that run only on nightly builds (`yarn test --full`) and fail only
on CI (not locally) are a bit more inconvenient to debug when they fail.
This patch makes the `yarn test --full` script print all the
intermediate output in Sharness tests, which can be helpful. We don’t do
this for `yarn test` simply because it generates a ton of spam even on
successful tests.

Test Plan:

    $ yarn test --full 2>&1 | wc -l
    1173

wchargin-branch: test-full-verbose
2018-10-31 11:06:48 -07:00
William Chargin c7ba89b807
license: relicense under MIT + Apache-2 (#896)
Summary:
All contributors to SourceCred have agreed to this more permissive
licensing option:

  - @decentralion: [link to comment][decentralion]
  - @wchargin: [link to comment][wchargin]
  - @claireandcode: [link to comment][claireandcode]

[decentralion]: https://github.com/sourcecred/sourcecred/issues/812#issuecomment-420817902
[wchargin]: https://github.com/sourcecred/sourcecred/issues/812#issuecomment-420819732
[claireandcode]: https://github.com/sourcecred/sourcecred/issues/812#issuecomment-424914639

Archive link to thread: <https://archive.fo/BH2v5>

Resolves #812.

Test Plan:
Note that the GitHub tree explorer correctly links from the README to
the individual license files.

wchargin-branch: license-dual-mit-apache2
2018-09-26 19:28:41 -07:00
Dandelion Mané 42cdfa4332
Upgrade jest and typings (#893)
Motivated by my desire for `.toMatchInlineSnapshot()`. Really we just
need and updated typing file for this, but I upgraded `jest` too to just
get us in a clean state.

Commit generated via:
```
yarn add --dev jest
flow-typed install jest@23.6.0
```

Test plan: `yarn test`
2018-09-25 18:48:54 -07:00
Dandelion Mané 7259233f82
Prepare to enable flow-type eslint rules (#848)
This commit upgrades the flow-type eslint plugin to latest, and writes
new rules into the eslintrc. To keep the diff clean, the rules are
disabled: I will turn them on individually (fixing errors) in followon
commits.

Test plan: `yarn test`.
Uncommenting the lines produces many lint errors (but the linter still operates as expected).
2018-09-17 14:11:39 -07:00
William Chargin 417cc231e9
deps: add `better-sqlite3` (#836)
Summary:
I selected this over the alternatives, `sqlite` and `sqlite3`, primarily
because its README explicitly acknowledges that using asynchronous APIs
for CPU-bound or serialized work units are worse than useless. To me,
this is a sign that the maintainer has his head on straight.

The many-fold performance increase over `sqlite` and `sqlite3` is nice
to have, too.

For now, we use my fork of the project, which includes a critical patch
to support private in-memory databases via SQLite’s standard `:memory:`
filepath. When this patch is merged upstream, we can move back to
mainline.

Test Plan:
The following session demonstrates the basic API and validates that the
install has completed successfully:

```js
const Database = require("better-sqlite3");
const db = new Database("/tmp/irrelevant", {memory: true});

db.prepare("CREATE TABLE pythagorean_triples (x, y, z)").run();
const insert = db.prepare("INSERT INTO pythagorean_triples VALUES (?, ?, ?)");
const get = db.prepare(
  "SELECT rowid, x * x + y * y AS xxyy, z * z AS zz FROM pythagorean_triples"
);

function print(x) {
  console.log(JSON.stringify(x));
}

print(insert.run(3, 4, 5));
print(get.all());
print(insert.run(5, 12, 13));
print(get.all());

db.prepare("DELETE FROM pythagorean_triples").run();
print(get.all());
```

It prints:

```js
{"changes":1,"lastInsertROWID":1}
[{"rowid":1,"xxyy":25,"zz":25}]
{"changes":1,"lastInsertROWID":2}
[{"rowid":1,"xxyy":25,"zz":25},{"rowid":2,"xxyy":169,"zz":169}]
[]
```

wchargin-branch: dep-better-sqlite3
2018-09-13 18:20:10 -07:00
William Chargin 513820c177
deps: upgrade `flow-bin@^0.80.0` (#791)
Summary:
This upgrade didn’t require fixing any new errors, but Flow is a good
dependency to keep on top of.

Test Plan:
Running `yarn flow` suffices.

wchargin-branch: flow-v0.80.0
2018-09-06 13:37:29 -07:00
William Chargin 92514ad559
deps: remove some unused packages (#789)
Summary:
Mostly Webpack loaders that have become unused through various config
changes.

Test Plan:
Check that these packages are not used anywhere except as transitive
dependencies:

```shell
$ git show --format= package.json |
>     sed '1,4d' | grep '^-' | cut -d\" -f2 | git grep -cf -
yarn.lock:3
```

Also, `yarn && yarn test --full` works, and `yarn start` works, and
`yarn backend && node ./bin/sourcecred.js load sourcecred/example-git`
works.

wchargin-branch: remove-unused-deps
2018-09-06 12:08:45 -07:00
William Chargin c48597651e
backend: invoke Webpack directly in package script (#782)
Summary:
This commit removes the `config/backend.js` script and replaces it with
a direct invocation of Webpack. This enables us to use command-line
arguments to Webpack, like `--output-path`.

Test Plan:
Note that `rm -rf bin; yarn backend` still works, and that the resulting
applications work (`node bin/sourcecred.js load`). Note that `yarn test`
and `yarn test --full` still work.

wchargin-branch: backend-webpack-direct
2018-09-05 12:28:27 -07:00
William Chargin 7e66886a70
dep: remove `eslint-loader` (#777)
Summary:
As of #775, this is no longer used.

Test Plan:
A `git grep eslint-loader` shows no results, and `yarn test --full`
passes.

wchargin-branch: remove-eslint-loader
2018-09-05 11:33:47 -07:00
William Chargin ddc93826be
Rename `makeWebpackConfig` to `webpack.config.web` (#770)
Summary:
The distinction was useful while `makeWebpackConfig` was being developed
(between #562 and #570), but is now confusing: we have a web config and
a backend config, and it is clearer if we name them as such.

Test Plan:
All of `yarn start`, `yarn build`, and `yarn test --full` work.

wchargin-branch: webpack-config-web
2018-09-04 21:45:10 -07:00
William Chargin 86c1b2e068
webpack: empty build dir instead of removing it (#768)
Test Plan:
Run `mkdir /tmp/out; cd /tmp/out; python -m SimpleHTTPServer`. In
another shell, run `./scripts/build_static_site.sh --target /tmp/out`.
Then, `curl localhost:8000`. Before this commit, this would have yielded
an `OSError` because the cwd of the Python process had been removed.
As of this commit, it works fine.

Also, run `git grep -c rimraf` and note only `yarn.lock:15`.

wchargin-branch: webpack-empty-build-directory
2018-09-04 21:44:09 -07:00
William Chargin 0a08783424
Remove OClif entirely (#745)
Test Plan:
Note that `yarn backend; node bin/sourcecred.js help` still works.
Note that `git grep -i oclif` returns no results.
Rejoice.

wchargin-branch: remove-oclif
2018-09-02 16:16:00 -07:00
William Chargin e71264f5cc
Replace `oclif` with `cli` (#744)
Summary:
This commit changes the CLI to use the code in `cli` instead of `oclif`.
A subsequent commit will remove the dependency on OClif altogether.
Resolves #580.

Test Plan:
Note that `yarn backend; node bin/sourcecred.js help` works. Note that
the documentation in the README is still correct.

wchargin-branch: cli-replace-oclif
2018-09-02 16:11:56 -07:00
William Chargin 7f81337d74
Store GitHub data gzipped at rest (#751)
Summary:
We store the relational view in `view.json.gz` instead of `view.json`,
taking advantage of the isomorphic `pako` library for gzip encoding and
decoding.

Sample space savings (note that post bodies are included; i.e., #747 has
not been applied):

       SAVE     OLD (B)     NEW (B) REPO
      89.7%       25326        2617 sourcecred/example-github
      82.9%     3257576      555948 sourcecred/sourcecred
      85.2%    11287621     1665884 ipfs/js-ipfs
      88.0%    20953425     2520358 gitcoinco/web
      84.4%    38196825     5951459 ipfs/go-ipfs
      84.9%   205770642    31101452 tensorflow/tensorflow

<details>
<summary>Script to generate space savings output</summary>

```shell
savings() {
    printf '% 7s % 11s % 11s %s\n' 'SAVE' 'OLD (B)' 'NEW (B)' 'REPO'
    for repo; do
        file="${SOURCECRED_DIRECTORY}/data/${repo}/github/view.json.gz"
        if ! [ -f "${file}" ]; then
            printf >&2 'warn: no such file %s\n' "${file}"
            continue
        fi
        script="$(sed -e 's/^ *//' <<EOF
            repo = '${repo}'
            pre_size = $(<"${file}" gzip -dc | wc -c)
            post_size = $(<"${file}" wc -c)
            percentage = '%0.1f%%' % (100 * (1 - post_size / pre_size))
            p = '% 7s % 11d % 11d %s' % (percentage, pre_size, post_size, repo)
            print(p)
EOF
        )"
        python3 -c "${script}"
    done
}
```

</details>

Closes #750.

Test Plan:
Comparing the raw old version with the decompressed new version shows
that they are identical:

```
$ <~/tmp/sourcecred/data/sourcecred/example-github/github/view.json \
> shasum -a 256 -
63853b9d3f918274aafacf5198787e18185a61b9c95faf640a1e61f5d11fa19f  -
$ <~/tmp/sourcecred/data/sourcecred/example-github/github/view.json.gz \
> gzip -dc | shasum -a 256
63853b9d3f918274aafacf5198787e18185a61b9c95faf640a1e61f5d11fa19f  -
```

Additionally, `yarn test --full` passes, and `yarn start` still loads
data and runs PageRank properly.

wchargin-branch: gzip-relational-view
2018-09-01 10:42:30 -07:00
William Chargin 0c2908dbfb
Retry GitHub queries with exponential backoff (#699)
Summary:
This patch adds independent exponential backoff to each individual
GitHub GraphQL query. We remove the fixed `GITHUB_DELAY_MS` delay before
each query in favor of this solution, which requires no additional
configuration (thus resolving a TODO in the process).

We use the NPM module `retry` with its default settings: namely, a
maximum of 10 retries with factor-2 backoff starting at 1000ms.
Empirically, it seems very unlikely that we should require much more
than 2 retries for a query. (See Test Plan for more details.)

This is both a short-term unblocker and a good kind of thing to have in
the long term.

Test Plan:
Note that `yarn test --full` passes, including `fetchGithubRepoTest.sh`.
Consider manual testing as follows.

Add `console.info` statements in `retryGithubFetch`, then load a large
repository like TensorFlow, and observe the output:

```shell
$ node bin/sourcecred.js load --plugin github tensorflow/tensorflow 2>&1 | ts -s '%.s'
0.252566 Fetching repo...
0.258422 Trying...
5.203014 Trying...
[snip]
1244.521197 Trying...
1254.848044 Will retry (n=1)...
1260.893334 Trying...
1271.547368 Trying...
1282.094735 Will retry (n=1)...
1283.349192 Will retry (n=2)...
1289.188728 Trying...
[snip]
1741.026869 Ensuring no more pages...
1742.139978 Creating view...
1752.023697 Stringifying...
1754.697116 Writing...
1754.697772 Done.
```

This took just under half an hour, with 264 queries total, of which:
  - 225 queries required 0 retries;
  - 38 queries required exactly 1 retry;
  - 1 query required exactly 2 retries; and
  - 0 queries required 3 or more retries.

wchargin-branch: github-backoff
2018-08-22 11:37:29 -07:00
William Chargin ad0e98ac2c
Add `createRelativeHistory` history implementation (#666)
Summary:
See #643 and the module docstring on `createRelativeHistory.js` for
context and explanation.

This patch adds `history@^3.0.0` as an explicit dependency—previously,
we were depending on it only implicitly through `react-router` (which
was fine then, but is not now). The dependency is chosen to match the
version specified in `react-router`’s `package.json`.

Test Plan:
Extensive unit tests included, with full coverage; `yarn test` suffices.

wchargin-branch: createRelativeHistory
2018-08-15 12:01:27 -07:00
William Chargin 3eb2b6eec6
Add a favicon (#637)
Summary:
In addition to the obvious benefit of having a favicon, this gets rid of
a 404 Not Found error on our home page, tremendously boosting our hacker
cred.

Test Plan:
The favicon is displayed in both `yarn start` and the static site (as a
result of the build script). The added build test fails before this
change.

wchargin-branch: add-favicon
2018-08-10 13:15:49 -07:00
William Chargin 8f2d2cd5cd
Remove service workers entirely (#635)
Summary:
This is a follow-up to #514, wherein we disabled new service workers and
instructed any existing service workers to self-destruct. (See that PR
for the rationale.) This commit removes them from our codebase entirely,
enabling us to slim down our build process and our build output.

Test Plan:
Running `yarn start` still works. Building the static site and exploring
it works, too.

wchargin-branch: remove-sw
2018-08-10 12:49:45 -07:00
William Chargin baa0cbff1b
Add `sharness` for shell-based testing (#597)
Summary:
We will shortly want to perform testing of shell scripts; it makes the
most sense to do so via the shell. We could roll our own testing
framework, but it makes more sense to use an existing one. By choosing
Sharness, we’re in good company: `go-ipfs` and `go-multihash` use it as
well, and it’s derived from Git’s testing library. I like it a lot.

For now, we need a dummy test file; our test runner will fail if there
are no tests to run. As soon as we have a real test, we can remove this.

This commit was generated by following the “per-project installation”
instructions at https://github.com/chriscool/sharness, and by
additionally including that repository’s `COPYING` file as
`SHARNESS_LICENSE`, with a header prepended. I considered instead adding
Sharness as a submodule, which is supported and has clear advantages
(e.g., you can update the thing), but opted to avoid the complexity of
submodules for now.

Test Plan:
Create the following tests in the `sharness` directory:

```shell
$ cat sharness/good.t
#!/bin/sh
test_description='demo of passing tests'
. ./sharness.sh
test_expect_success "look at me go" true
test_expect_success EXPENSIVE "this may take a while" 'sleep 2'
test_done
# vim: ft=sh
$ cat sharness/bad.t
#!/bin/sh
test_description='demo of failing tests'
. ./sharness.sh
test_expect_success "I don't feel so good" false
test_done
# vim: ft=sh
```

Note that `yarn sharness` and `yarn test` fail appropriately. Note that
`yarn sharness-full` fails appropriately after taking two extra seconds,
and `yarn test --full` runs the latter. Each failure message should
print the name of the failing test case, not just the suite name, and
should indicate that the passing tests passed.

Then, remove `sharness/bad.t`, and note that the above commands all
pass, with the `--full` variants still taking longer.

Finally, remove `sharness/good.t`, and note that the above commands all
pass (and all pass quickly).

wchargin-branch: add-sharness
2018-08-06 12:56:25 -07:00
William Chargin 48275590ba
Remove `clean-webpack-plugin` (#578)
Summary:
As of #577, this is no longer needed.

Test Plan:
Running `yarn && yarn test --full` suffices.

wchargin-branch: remove-clean-webpack-plugin
2018-07-31 19:10:03 -07:00
William Chargin 480bdf1bc7
Refine the build directory cleaning logic (#577)
Summary:
We were asking the `clean-webpack-plugin` to remove the `build/`
directory in all cases. However, Webpack accepts a command-line
parameter `--output-path`. When such a parameter is passed, we would be
removing the wrong directory.

The proper behavior is to remove “whatever the actual output path is”.
Webpack exposes this information, but it appears that the
`clean-webpack-plugin` does not take advantage of it. Therefore, this
commit includes a small Webpack plugin to do the right thing.

Test Plan:
Test that the behavior is correct when no output directory is specified:
```
mkdir -p build && touch build/wat && yarn build && ! [ -e build/wat ]
```

Test that the behavior is correct with an explicit `--output-path`:
```
outdir="$(mktemp -d)" && touch "${outdir}/wat" && \
    yarn build --output-path "${outdir}" && \
    ! [ -e "${outdir}/wat" ]
```

Test that the plugin refuses to remove the root directory:

```
! yarn build --output-path . && \
    sed -i '/path: /d' config/makeWebpackConfig.js && ! yarn build
```

(Feel free to comment out the actual `rimraf.sync` line in the plugin
when testing this.)

wchargin-branch: clean-actual-build-directory
2018-07-31 15:27:32 -07:00
William Chargin 3b5ad594bd
package.json: reorganize test commands (#571)
Summary:
Running `yarn test` (equiv. `npm test` or `npm run test`) now runs all
checks. It takes the place of the former `yarn travis`. This is more in
line with the expectation of a top-level `test` command: if it passes,
your code is good.

The `unit` command now runs Jest once, not in watch mode. It takes the
place of the former `ci-test`. To run tests in watch mode, run any of
the following:

  - `yarn unit --watch`, or
  - `npm run unit -- --watch`, or
  - `npm unit -- --watch`.

This behavior is more consistent with the standard behavior of commands
like `make test`. It is also empirically what @wchargin and
@decentralion want most of the time.

Test Plan:
Verify that each of the scripts `test`, `unit`, and `coverage` passes.
Verify that each of the aforementioned `--watch` invocations works.
Verify that `.travis.yml` has the correct `script:` command.

wchargin-branch: reorganize-test-command
2018-07-31 10:53:10 -07:00
William Chargin 0128df8c18
new-webpack: replace old scripts in `package.json` (#569)
Test Plan:
Run `yarn start` and note that everything checks out.

Run `yarn build && (cd build/ && python -m SimpleHTTPServer)` and note
that everything checks out, except that the static assets are of course
not included in the build.

wchargin-branch: webpack-replace-scripts
2018-07-30 18:10:54 -07:00
William Chargin b45ef739fe
new-webpack: clean `build/` before prod (#568)
Summary:
In our current system, we build by invoking `scripts/build.js`, which
begins by removing the `build/` directory. This behavior is nice,
because it prevents cross-contamination between builds. In this commit,
we add a plugin to achieve the same result from directly within Webpack.

Test Plan:
Run

```
mkdir -p ./build
touch ./build/wat
NODE_ENV=production node ./node_modules/.bin/webpack \
    --config config/makeWebpackConfig.js
```

and ensure that `./build/wat` does not exist after the build completes.

wchargin-branch: webpack-clean-build
2018-07-30 18:01:47 -07:00
William Chargin 873eca6350
Upgrade Flow to v0.76.0 (#546)
Summary:
In addition to a routine libdef update, we also need to work around a
particularly nasty new bug in Flow, which requires `any`-casts that are
even more unsafe than usual. That said, I think that it’s worth that
cost to remain up to date with Flow, so that we can amortize future such
issues.

Test Plan:
Running `yarn travis --full` passes.

wchargin-branch: upgrade-flow-v0.76.0
2018-07-27 15:54:59 -07:00