Commit Graph

780 Commits

Author SHA1 Message Date
Dandelion Mané 783a5e7d57
Add CHANGELOG.md (#670)
Also, update CONTRIBUTING.md to guide contributors to update the
changelog.

Test plan: Unnecessary
2018-08-15 15:20:59 -07:00
Dandelion Mané be79e3cb1c
Add some margin on the TableRow score column (#669)
The TableRow currently has some margin on the left, but not on the
right. This is visually unbalanced, especially when expanded so depth>0,
as the content on the right is at the very edge of the shaded rectangle.

This commit cleans that up a bit!

Test plan: Visual inspection (see screenshots in the pull request). I
don't think unit tests are necessary for small visual tweaks like this.
2018-08-15 15:10:29 -07:00
William Chargin 621a93851c
Resolve a relative path to the application root (#665)
Summary:
This is necessary for #643. If we’re serving `/prototype/index.html`, we
need to to use `..` to refer to the root of the site. This patch adds
`rootFromPath`, which performs the relevant transformation. (The
implementation is trivial, but figuring out exactly what the
specification should be was not!)

Test Plan:
Unit tests added; `yarn test` suffices.

wchargin-branch: rootFromPath
2018-08-15 13:03:19 -07:00
William Chargin c1997d041f
Add an `Assets` resolver (#664)
Summary:
This will enable clients to obtain the path to a static asset, even when
the app is not hosted at the root of a server, as outlined in #643.

This module will be used for simple assets (images, etc.) and API data
(fetches from `/api/**`) alike.

This supersedes #663. It includes the logic from that PR (`Assets`)
without the React-specific context bindings (`AssetsContext`).

Test Plan:
Unit tests included; `yarn test` suffices.

wchargin-branch: assets-resolver
2018-08-15 12:46:09 -07:00
William Chargin ad0e98ac2c
Add `createRelativeHistory` history implementation (#666)
Summary:
See #643 and the module docstring on `createRelativeHistory.js` for
context and explanation.

This patch adds `history@^3.0.0` as an explicit dependency—previously,
we were depending on it only implicitly through `react-router` (which
was fine then, but is not now). The dependency is chosen to match the
version specified in `react-router`’s `package.json`.

Test Plan:
Extensive unit tests included, with full coverage; `yarn test` suffices.

wchargin-branch: createRelativeHistory
2018-08-15 12:01:27 -07:00
Dandelion Mané 00bc9a9461
Add pluralized node names (#648)
This is preparation for #502 - we want to be able to describe groups of
nodes, e.g. "52 repositories", so we need a plural form for every node
name.

As a fly-by fix, I removed the parentheses around the node names in the
fallback adapter, as these proved to look ugly/inconsistent in the UI.

Test plan: `yarn test` is sufficient.
2018-08-15 11:58:12 -07:00
Dandelion Mané f100cd02cc
PagerankTable filter defaults to GitHub users (#653)
We humans tend to find information about humans more interesting than
information about commits or pulls. The UI should accomodate this by
defaulting to displaying GitHub user nodes in the cred explorer.

This is implemented as a new nullable argument to the PageRankTable. If
not present, then the filter defaults to showing all nodes. If the
default filter is present but doesn't match any available type, an error
is thrown.

Test plan: The new behavior is tested. Also, I checked it in the UI and
it works.

Closes #651
2018-08-15 11:44:44 -07:00
Dandelion Mané e3a4d1f2b9
ConnectionRow now shows the connection score (#658)
Previously, the ConnectionRow showed the score of the node that was the
source of the connection. I believe the UI will be more consistent and
useful if it instead shows the connection score, i.e. how important that
connection was to the node in scope. This combos well with PR #657.

Test plan: The change is very simple, and covered by unit tests. I also
verified the behavior by examining the cred explorer.
2018-08-14 15:40:49 -07:00
William Chargin c890fe03b4
Add Node’s `path.normalize` for the browser (#650)
Summary:
This function normalizes paths like `foo/bar//../baz` to `foo/baz`. The
implementation is directly copied from Node’s source code, which is
available under an MIT License.

I looked for a suitable NPM package, and rejected `path-normalize` and
`normalize-path`. (The former is closer and explicitly purports to be
the right thing, but actually isn’t.)

Test Plan:
Unit tests added, with full coverage except for one branch; I include a
proof that the branch is unreachable.

Tested on Node v8.9.4, Node v10.0.0, and Node v10.8.0. Tests pass in
each case. In the latter two cases, inconsistencies between the
implementation and the actual `path.posix.normalize` would cause a test
failure. In the former case, they do not. (All such cases verified.)

wchargin-branch: path-normalize
2018-08-14 13:04:53 -07:00
Dandelion Mané bf65860178
Change the PagerankTable component cycle (#657)
Currently, the PagerankTable creates components in the following
pattern:

```
NodeRow (depth=0)
 > ConnectionRowList (depth=1)
  > ConnectionRow (depth=2)
   > ConnectionRowList (depth=3)
```

This commit changes the cycle to the following:

```
NodeRow  (depth=0)
 > ConnectionRowList (depth=0)
  > ConnectionRow (depth=0)

NodeRow (padding=true, depth=1)
  > ConnectionRowList (depth=1)
    > ConnectionRow (depth=1)
```

This has some nice properties:

First, the context visually resets every time we return to a NodeRow, which
makes it feasible to change the score column to always have a context
dependent meaning:
- for a node row, the score is the score of that node.
- for a connection row, the score is the score contribution of that
connection
- (as of #502): for an aggregation row, the score is the score
contribution of that aggregation

We think this will be visually clear thanks to the padding around the
new NodeRow, along with the new color block indicating a new scope.

This design also ensures that every NodeRow has the full width available
to it (rather than getting crushed into a progressively smaller area of
the table), which will be very convenient for when we add renderers for
the nodes.

Thanks to @theliamcrawford as the idea for this came during a user study
with him.

Test plan:
The updated unit tests should be comprehensive. Also, try expanding some
rows in the cred explorer and verify that the behavior is as described.
2018-08-13 21:33:12 -07:00
Dandelion Mané 83969371d6
Tweak the cred explorer highlight color (#659)
William and I were experimenting, and felt that this color is slightly
more pleasing / harmonious with the rest of the site, and still quite
legibile.

Test plan: Examine the new UI, and conclude that the color choice is
harmonious and legible :). Screenshot included with the PR.

Paired with @wchargin
2018-08-13 21:27:21 -07:00
William Chargin c315af4dbb
Annotate adapter set constructor return types (#661)
Summary:
Due to <https://github.com/facebook/flow/issues/6400>, patches like the
following weren’t raising Flow errors:

```diff
diff --git a/src/app/adapters/adapterSet.test.js b/src/app/adapters/adapterSet.test.js
index 67dd3ed..ccc6ac6 100644
--- a/src/app/adapters/adapterSet.test.js
+++ b/src/app/adapters/adapterSet.test.js
@@ -77,6 +77,7 @@ describe("app/adapters/adapterSet", () => {
       const x = new TestStaticPluginAdapter();
       const fallback = new FallbackStaticAdapter();
       const sas = new StaticAdapterSet([x]);
+      sas.wat();
       return {x, fallback, sas};
     }
     it("errors if two plugins have the same name", () => {
```

A `flow type-at-pos` check indicated that the type of `sas` was indeed
inferred as `any`.

This patch applies the usual nonsensical fix. Better safe than spooky.

Test Plan:
The above patch now raises a Flow error.

wchargin-branch: annotate-adapterset-constructors
2018-08-13 21:27:02 -07:00
William Chargin 4081e0d008
Use trailing slashes for all routes (#660)
Summary:
See justification in the added unit test.

Test Plan:
Added unit test, with justification.

Also, `yarn sharness-full` passes, and `yarn start` still works.

wchargin-branch: route-trailing-slashes
2018-08-13 21:25:20 -07:00
Dandelion Mané 080edb380d
TableRows can create vertical padding (#656)
This commit adds the `showPadding` prop to `TableRow`s. If showPadding
is true, then the row will have vertical padding above the row, and
below the last child of the row. The padding will match the background
color of the given row. The padding is implemented as extra `tr`
elements that themselves contain empty tds.

Test plan: The new behavior is pretty thoroughly covered by new unit
tests. Additionally, if you want to see padding in the live UI, you can
apply the following (slightly contrived) diff.

```
diff --git a/src/app/credExplorer/pagerankTable/Connection.js b/src/app/credExplorer/pagerankTable/Connection.js
index 3a882cd..633525b 100644
--- a/src/app/credExplorer/pagerankTable/Connection.js
+++ b/src/app/credExplorer/pagerankTable/Connection.js
@@ -70,7 +70,7 @@ export class ConnectionRow extends React.PureComponent<ConnectionRowProps> {
         depth={depth}
         description={connectionView}
         connectionProportion={connectionProportion}
-        showPadding={false}
+        showPadding={depth % 3 === 0}
         score={sourceScore}
       >
         <ConnectionRowList
```
2018-08-13 19:11:19 -07:00
Dandelion Mané 7f18e389ea
Separate TableRow `depth` from `indent` (#655)
Currently, as we expand nodes or connections in the PagerankTable, the
rows both get more indented and attain a deeper color. Both of these
behaviors are controlled by the `depth` parameter.

We're going to switch the UI to a cyclic structure, where as you drill
down, once you get back to a particular node, the indentation resets to
base, but the color - which now indicates nested depth - continues to
change. This commit sets that change up, by splitting the behvaior into
two parameters: `depth`, which controls the color, and `indent`, which
controls the indentation level.

As a small additional tweak, the indentation formula is changed so that
buttons are always indented by 5 pixels. This results in a cleaner
display for nodes that have `depth>0` but `indent==0` (as the button
doesn't look squahsed against the color boundary).

Test plan:
The change is very simple; inspecting the updated snapshots should be
persuasive.
2018-08-13 18:07:13 -07:00
Dandelion Mané c5e20f9400
Factor out TableRow as a reusable abstraction (#652)
We currently have two components which create rows in our PagerankTable:
the `NodeRow` and `ConnectionRow`. Work on #502 will result in the
addition of a new one, the `AggregationRow`. It's time to stop
duplicating logic (and testing) of the shared behavior for these rows,
like depth-based styling, row expansion/collapse, etc. This commit pulls
all the common logic to rendering rows into a single, thoroughly tested
`TableRow` component.

There is one observable change in the UI: when a connection percentage
is not available (i.e. for NodeRows), we now leave the column empty
rather than placing a dash in the column. I think this is visually
cleaner.

Test plan: Unit tests pass, and this part of the code base is thoroughly
tested, so that's a pretty reliable indicator. I also poked around the
live PagerankTable in the cred explorer, just to be safe.
2018-08-13 12:31:16 -07:00
Dandelion Mané f1c5d3756d
Add `Trie.getLast` (#646)
When using Tries, we often want the last matching entry for the given
path, and to throw an error if one is not available. By adding this
method to the API, we avoid a lot of unnecessary repetition in the code
base.

Test plan: Unit tests pass. As this touches the untested WeightConfig,
I've also manually tested the weight config behavior.
2018-08-10 23:01:29 -07:00
William Chargin 00e2a67477
In client-side code, only import GraphQL types (#641)
Summary:
An `import *` was used for convenience, but this effects a value import
in addition to a type import. By exploding the wildcard import to
directly import the required types, we can shave off 2.3% of our
post-gzip bundle size (131.82 KB to 128.74 KB). It’s unfortunate that we
lose the namespacing, but c’est la vie.

Test Plan:
`yarn flow` suffices.

wchargin-branch: explode-wildcard-type-import
2018-08-10 22:26:28 -07:00
Dandelion Mané 233bec4f5e
Re-disable the Git plugin (#645)
Thanks to #642, it should now be safe to disable the Git plugin, reaping
the benefits described in #628, without causing the cred explorer to
crash (#631).

Test plan:
- `yarn travis --full` passes
- The full cred explorer works:
  - Running PageRank does not crash the explorer
  - Expanding a pull request does not crash the explorer
  - (After clearing state) the weight config doesn't show Git weights
  - The filter doesn't show Git nodes
2018-08-10 19:44:02 -07:00
Dandelion Mané 86ce26acb8
Improve weight config (#644)
This modifies WeightConfig to properly use the fallback node type, as
created in #640 and merged in #642. As an additional change, it now
displays type names, rather than the address parts. For example, the
issue type is now displayed as `Issue`, not
`["sourcecred", "github", "issue"]`.

The WeightConfig is an untested mess, and I will likely re-write it
entirely. (See a bevy of WeightConfig related issues: #604, #595, #588).
So, not too much effort was invested in keeping high code quality in
this commit.

Test plan: The weight config has no tests, so I manually tested:
- weights persist after page reload
- node weights influence cred attribution
- edge weights influence cred attribution
- edge directionality influences cred attribution
- weights have reasonably pretty description messages
2018-08-10 19:29:01 -07:00
Dandelion Mané 05c9f81cc0
Use AdapterSets in the cred explorer (#642)
This takes the code from #640 and puts it into production.

Test plan: Unit tests pass. The observable behavior in the cred explorer
is unchanged; i.e. the addition of the FallbackAdapter did not produce
new entries in the WeightConfig or in the Pagerank table options. The
WeightConfig is untested, so we don't have verification of that behavior
(other than that I tested it and am reporting it here). The
PagerankTable code is tested, and a snapshot would fail if another
option group had appeared.
2018-08-10 18:19:54 -07:00
Dandelion Mané 9edd7ac069
Add AdapterSet and FallbackAdapter (#640)
Issue #631 revealed that our current plugin-handling code is fragile -
we aren't robust to having nodes from a plugin without having that
plugin present in the frontend. This commit adds `StaticAdapterSet` and
`DynamicAdapterSet`, which are abstractions over finding the matching
plugin adapter or type for a node or edge. It's a robust abstraction,
because the adapter sets always include the `StaticFallbackAdapter` or
`DynamicFallbackAdapter`, which can match any node, so we'll never get
an error like #631 due to not having an adapter / type available.

Also relevant: #465

Test plan:
Unit tests included.
2018-08-10 18:08:49 -07:00
William Chargin 33763a9f35
Allow delay between GitHub queries (#626)
Summary:
This patch considers an environment variable `GITHUB_DELAY_MS`. If the
value is set to a positive integer, a delay of the given number of
milliseconds will be incurred before each query to GitHub. This is to
decrease the probability of being rate-limited; see #350 for details.
This in turn unblocks us to load SourceCred data for larger
repositories.

The use of an environment variable is something of a hack to get this
off the ground. See #638 for long-term plans.

Test Plan:
Run `time node ./bin/sourcecred.js load sourcecred/example-github` with
varying values for `GITHUB_DELAY_MS`. Note that with the variable unset,
set to zero, set to a negative number, or set to a non-numeric value,
the job completes quickly; when the delay is set to `5000`, the job
takes an extra five seconds.

wchargin-branch: delay-github-queries
2018-08-10 17:53:26 -07:00
Dandelion Mané 378f627a6f
Add `aggregateFlat` and `flattenAggregation` (#629)
For #502: The UI that I currently have in mind displays aggregations
grouped by connection type and node type together, rather than nested.
I think it will be cumbersome to have multiple hierarchical
levels of expansion.

To make that UI easy to write, this commit adds some logic for
flattening the hiearchical aggregation from #624. I add an extra
translation to flatten, rather than just having the logic produce nested
structures, because it's convenient to keep around the nested structure
in case I decide to implement the hierarchical UI instead. Once we have
solidified how we want the UI to behave, we might choose to simplify
this code.

Test plan: The implementation is rather simple. There are some unit
tests.
2018-08-10 17:05:59 -07:00
Dandelion Mané d8db763257
Move core plugin adapter code to its own directory (#632)
This commit creates the directory `src/app/adapters` and moves the
following three files into it:
- `src/app/pluginAdapter.js`
- `src/app/pluginAdapter.test.js`
- `src/app/defaultPlugins.js`

This is in preparation for a principled fix for #631, which will add a
base plugin and some logic that ensures it's always included.
2018-08-10 17:05:44 -07:00
William Chargin 3eb2b6eec6
Add a favicon (#637)
Summary:
In addition to the obvious benefit of having a favicon, this gets rid of
a 404 Not Found error on our home page, tremendously boosting our hacker
cred.

Test Plan:
The favicon is displayed in both `yarn start` and the static site (as a
result of the build script). The added build test fails before this
change.

wchargin-branch: add-favicon
2018-08-10 13:15:49 -07:00
William Chargin bb12f933a5
Check in SourceCred logo files (#636)
Summary:
The SVG was artisanally crafted by yours truly, and rasterized by the
accompanying script (which is fully deterministic, and also artisanal).

Test Plan:
Run `./src/assets/logo/rasterize.sh`, and note that the output is
unchanged.

wchargin-branch: add-logo
2018-08-10 13:01:36 -07:00
William Chargin 8f2d2cd5cd
Remove service workers entirely (#635)
Summary:
This is a follow-up to #514, wherein we disabled new service workers and
instructed any existing service workers to self-destruct. (See that PR
for the rationale.) This commit removes them from our codebase entirely,
enabling us to slim down our build process and our build output.

Test Plan:
Running `yarn start` still works. Building the static site and exploring
it works, too.

wchargin-branch: remove-sw
2018-08-10 12:49:45 -07:00
William Chargin 10930c42af
Use `hydrate` instead of `render` on the client (#634)
Summary:
This fixes the following warning on our development instance:

> Warning: render(): Calling ReactDOM.render() to hydrate
> server-rendered markup will stop working in React v17. Replace the
> ReactDOM.render() call with ReactDOM.hydrate() if you want React to
> attach to the server HTML.

We do in fact want to attach to the server HTML, so we apply the
suggested patch.

(The warning of course also applies to production, but warnings do not
appear in production.)

Test Plan:
Running `yarn start` shows that the above warning has disappeared, and
that the cred explorer still works. (Also, `yarn test --full` passes,
but that tells us effectively nothing because this code path is never
hit in tests: it only affects the HTML that is executed in the browser.
Erasing the entire module, leaving only `// @flow`, still lets tests
pass.)

wchargin-branch: migrate-to-hydrate
2018-08-10 11:54:14 -07:00
Dandelion Mané 885ff90f62
Revert "Disable the Git plugin (#628)" (#633)
This reverts commit 8c70f03122.

Context: This introduced a serious bug (#631), so we're reverting it to
get the codebase back in a working state. Meanwhile, I'll develop a
principled solution.

Test plan:
I rebuilt the backend, re-loaded a graph, and loaded it in the frontend.
PageRank, the cred explorer, and the weight config all work. Opening a
pull request does not trigger a crash.
2018-08-10 11:40:20 -07:00
Dandelion Mané bb59efbfe6
Implement core aggregation logic (#624)
This implements two methods:
`aggregateByNodeType` groups `scoredConnection`s by the specified
`NodeTypes`, along with summary statistics.

`aggregateByConnectionType` groups `scoredConnection`s by
`ConnectionType` at the top level, where `ConnectionType` includes
`EdgeType` and direction, (and also captures synthetic self-loops).
Then it also groups by `NodeType` within any aggregation.

This is progress towards #502.

Test plan: unit tests included.
2018-08-09 23:13:46 -07:00
Dandelion Mané 8c70f03122
Disable the Git plugin (#628)
See #627 for context.

Removing the Git plugin results in an enormous performance improvement.
In my testing on `metamask/metamask-extension`: before this change, load
took 23s, and PageRank took >9 mins and then crashed. After this change,
load+PageRank took 5s combined.

Test plan: All unit tests pass; loading new data from the CLI works; and
I poked around the UI to make sure there were no spurious references to
the Git plugin.

Note: This does not break backcompat, there's no need to regenerate any
already-loaded data.
2018-08-09 23:08:01 -07:00
Dandelion Mané 51acc25f12
Add type signatures for aggregation logic (#623)
This is the first real step towards #502.

Factoring this out because deciding the type signatures was non-trivial,
and the work was paired with @wchargin.

Test plan: `yarn test`
2018-08-09 22:37:12 -07:00
Dandelion Mané 74e00b0bfd
Split PagerankTable into smaller files (#630)
PagerankTable is getting a bit unwieldy, especially as #502 will need to
add a new pair of components. This commit splits the erstwise
PagerankTable.js into four files:

- `pagerankTable/shared`, shared utils and types
- `pagerankTable/Node`, the `NodeRow` and `NodeRowList`
- `pagerankTable/Connection`, the `ConnectionRow`, `ConnectionRowList`,
and `ConnectionView`
- `pagerankTable/Table`, the `PagerankTable` itself

This commit makes no logical changes; it is purely a reorganization.

Test plan: `yarn test`
2018-08-09 21:33:40 -07:00
Dandelion Mané dc13d460da
Display linear scores, normalized by the maximum (#625)
PageRank outputs scores as components in a probability distribution.
This means that most scores are very small numbers, e.g. 0.00003. This
doesn't make for a great UI (humans don't like thinking in tiny
decimals).

Our first attempt to come up with a more readable UI was to use log
scores; in #265 we displayed the log score alongside (arbitrarily)
`rawScore * 100` in the UI. The log scores were more usable, so we kept
them, with subsequent modifications. In the original version, all the
log scores were negative. In #466, we arbitrarily added 10 to the
scores, which made most scores look nicer, but introduced a meaningless
switch where scores counter-intuitively become negative after a certain
point. That was bad, so in #535 we started displaying negative log
scores. This is also counter-intuitive: it's weird that lower scores are
better, and it's not clear that a score of (say) 3 is 20x better than a
score of 6.

I think we need to do away with the log scores; people just don't think
about numbers logarithmically. This commit switches to linear scores,
normalized so that the largest score is always 1000. I've tried this out
on a few repos and demo'd it to people, and it seems much clearer.

Test plan: Some unit tests added; also, I launched the cred explorer and
experienced the change on several projects.
2018-08-09 14:26:08 -07:00
Dandelion Mané fb70152e7a
Add findEdgeType and findNodeType (#620)
These methods find the unique matching node or edge type for a node,
from a given pluginAdapter.

Test plan: unit tests
2018-08-07 18:43:26 -07:00
Dandelion Mané 123b85146e
Bundle the default plugins together (#621)
We manually import the set of plugins to be used by the app in a few
different places. That's a bad smell. This commit creates a centralized
import point instead.

Test plan: `yarn test`
2018-08-07 15:53:20 -07:00
Dandelion Mané 0cf5923bce
Add dispatch methods for plugin adapters (#619)
This commit adds some consistent and tested methods for getting the
appropriate plugin adapter for a given Node/Edge address. There are
methods for both static and dynamic adapters.

In the event that more than one plugin adapter matches the given
address, an error is thrown; likewise in the case where there is no
matching adapter.

Test plan: `yarn test`

Relevant to #465
2018-08-07 15:24:34 -07:00
William Chargin c02a81ff43
Expose a cache directory to plugins at load time (#616)
Summary:
The `node ./bin/sourcecred.js load` command invokes plugin code by
providing an output directory into which the plugin may store data.
As of this patch, it also provides a cache directory that the plugin may
use to store data that will not be available at runtime. For instance,
the Git plugin might choose to clone the repository herein, or the
GitHub plugin may choose to store partial GraphQL query results to deal
with interruptions. The contract is that the cache directory may be
removed at any time and that the plugin should continue to operate
normally.

Test Plan:
The build script has been updated and tested. Reverting the change to
the build script causes the newly added test to fail. (Each plugin has a
cache directory, though the cache directories are empty for now.)

wchargin-branch: create-plugin-cache
2018-08-07 13:10:15 -07:00
Dandelion Mané 2e6cc6fbfd
Remove the PULL_REQUEST_TEMPLATE.md (#618)
It's annoying that it always appends it to the end of the PR message
when we've uploaded a single commit. Since we have new contributors
relatively rarely, but we are slightly annoyed by it constantly, it's
better to remove it.

Test plan: not needed
2018-08-07 13:09:05 -07:00
Dandelion Mané bb2fed56ca
Revert "Move PluginAdapter to core (#615)" (#617)
This reverts commit 25b0124b56.

@wchargin had an extensive offline conversation about PluginAdapters,
and decided that for now we awnt to punt on figuring out the right
abstraction for having a "core"-scoped plugin adapter. Instead, we'll
keep on using plugin adapters as something of a kitchen sink where we
throw in all the per-plugin logic that we need to run the app.

This necessitates reverting #615 because we don't think that React
should be a dependency in core, but we will need the
DynamicPluginAdapter to have a type dependency on React so that we can
solve issues like #590.

Test plan: Yarn test suffices.
2018-08-07 13:04:33 -07:00
Dandelion Mané 25b0124b56
Move PluginAdapter to core (#615)
There's no reason for it to be in `app` - the concepts it contains are
core concepts, e.g. node types and graphs.

For now I'm leaving the `NodeType` and `EdgeType` interfaces with the
PluginAdapter. They may move later, but I'm relucant to clutter the
graph class more, and all I need for now is that they live in core.

Test plan: It's just a move/rename, so `yarn test` is amply sufficient.
2018-08-07 11:23:00 -07:00
William Chargin f448960105
Forbid accessing the `cache/` directory at runtime (#614)
Summary:
We plan to allow plugins to store permanent data in `$SC/data/` and
temporary, ephemeral, or intermediate data in `$SC/cache/`. The latter
subtree will be excluded from the static site at build time, so it
behooves us to also exclude it from the development environment.

Test Plan:
Run `yarn start`. Then,

```shell
$ root='localhost:8080/api/v1/data'
$ curl -sI "${root}/repositoryRegistry.json" | head -1
HTTP/1.1 200 OK
$ curl -sI "${root}/data/sourcecred/example-git/github/view.json" | head -1
HTTP/1.1 200 OK
$ curl -sI "${root}/cache" | head -1
HTTP/1.1 400 Bad Request
$ curl -sI "${root}/cache/" | head -1
HTTP/1.1 400 Bad Request
$ curl -sI "${root}/cache/foo" | head -1
HTTP/1.1 400 Bad Request
$ curl -sI "${root}/cache/foo/bar/baz" | head -1
HTTP/1.1 400 Bad Request
```

Also, check that the app still works.

wchargin-branch: exclude-cache-from-dev-server
2018-08-07 11:17:03 -07:00
William Chargin 7a4401e3ef
Remove the `start` CLI command and dependencies (#613)
Summary:
We never use the `node ./bin/sourcecred.js start` command. This command
contains an Express server to combine the static files with the build
output, which duplicates the logic in our Webpack config, which we
actually use (with `yarn start`). Once we actually want the command line
entry point to be a useful tool for end users, we can consider
reimplementing it the right way, whatever that may be. Until then, it’s
simply one more thing to keep in sync.

Test Plan:
Running `yarn test --full` passes; the `load` CLI command still works;
running `yarn start` still works.

wchargin-branch: remove-start
2018-08-07 11:00:09 -07:00
Dandelion Mané 9c781fcfee
Add `NodeTrie` and `EdgeTrie` classes (#612)
For #465, I'm planning to create an abstraction over NodeTypes and
EdgeTypes which traverses a hierarchy of types and aggregates/reduces
information across all the matching types for a given Node/Edge address.

To do that efficiently, we will want tries[1].

Thanks to @wchargin for helping me figure out how to implement this.

Test plan: Unit tests. The code is a little tricky so please review it
closely.

[1]: https://en.wikipedia.org/wiki/Trie
2018-08-06 19:56:25 -07:00
William Chargin 383f8d406e
deploy.sh: depend on build_static_site.sh (#611)
Summary:
This is a minimal patch to make the deploy script depend on the
separate, well-tested build script introduced in #592. The patch
suggests a number of improvements that can be made to the deploy script
itself, but I’m deferring those as they’re not on the critical path.
These are tracked at #610.

Test Plan:
Apply the below patch. Then run `./scripts/deploy.sh -n`, and verify
that the served output is correct, including the CNAME file. Verify that
when the `DEPLOY_CNAME_URL` default value is removed, the resulting
output does _not_ have a CNAME file.

Patch, for ease of building:

```diff
diff --git a/scripts/deploy.sh b/scripts/deploy.sh
index cddcece..8925e5b 100755
--- a/scripts/deploy.sh
+++ b/scripts/deploy.sh
@@ -23,7 +23,7 @@ main() {
     preview_dir=
     trap cleanup EXIT

-    ensure_clean_working_tree
+    #ensure_clean_working_tree
     build_and_deploy
 }

@@ -76,8 +76,7 @@ build_and_deploy() {
     "${sourcecred_repo}/scripts/build_static_site.sh" \
         --target "${static_site}" \
         ${DEPLOY_CNAME_URL:+--cname "${DEPLOY_CNAME_URL}"} \
-        --repo ipfs/js-ipfs \
-        --repo sourcecred/sourcecred \
+        --repo sourcecred/example-github \
         ;

     sourcecred_site="$(mktemp -d --suffix ".sourcecred-site")"
```

wchargin-branch: deploy-via-build
2018-08-06 19:00:35 -07:00
Dandelion Mané 943dea94f9
Pull out explicit NodeType and EdgeType (#608)
Will be useful for #502.

Test plan: `yarn flow`
2018-08-06 13:05:51 -07:00
William Chargin d7cb4c65fa
Create script to build static site (#592)
Summary:
Currently, we create the static site and deploy it all at once in
`scripts/deploy.sh`. This commit creates a new script that only builds
the static site. This has the advantage that it is easier/less scary to
change that script (because it can be tested without worrying about
deploying to a local test target), and that we can write automated tests
for it.

Test Plan:
Run `yarn sharness`; note that it completes very quickly. Then, in a
shell with your GitHub token exported, run `yarn sharness-full`. Expect
all tests to pass.

For a sanity check, you can run:

```shell
outdir="$(mktemp -d --suffix .sourcecred-site)"
./scripts/build_static_site.sh --target "${outdir}" \
    --cname sourcecred.io \
    --repo sourcecred/example-git \
    --repo sourcecred/example-github \
    ;
(cd "${outdir}" && python -m SimpleHTTPServer)
```

and ensure that <http://localhost:8000/> is as expected.

One test case that is not covered is the following: _if_ the actual app
somehow tries to emit a `CNAME` file at root, _and_ our script’s logic
to catch this is broken, then we will not catch this failure. I’ve
tested the logic manually by adding `>"${cname_file}"` after definition
of that variable, but I don’t see a good way to test it automatically,
without adding flags like `--but-actually-emit-cname-too` to the build.
The compound probability of this happening is sufficiently low that this
doesn’t bother me.

wchargin-branch: build-static-site-script
2018-08-06 13:05:40 -07:00
William Chargin baa0cbff1b
Add `sharness` for shell-based testing (#597)
Summary:
We will shortly want to perform testing of shell scripts; it makes the
most sense to do so via the shell. We could roll our own testing
framework, but it makes more sense to use an existing one. By choosing
Sharness, we’re in good company: `go-ipfs` and `go-multihash` use it as
well, and it’s derived from Git’s testing library. I like it a lot.

For now, we need a dummy test file; our test runner will fail if there
are no tests to run. As soon as we have a real test, we can remove this.

This commit was generated by following the “per-project installation”
instructions at https://github.com/chriscool/sharness, and by
additionally including that repository’s `COPYING` file as
`SHARNESS_LICENSE`, with a header prepended. I considered instead adding
Sharness as a submodule, which is supported and has clear advantages
(e.g., you can update the thing), but opted to avoid the complexity of
submodules for now.

Test Plan:
Create the following tests in the `sharness` directory:

```shell
$ cat sharness/good.t
#!/bin/sh
test_description='demo of passing tests'
. ./sharness.sh
test_expect_success "look at me go" true
test_expect_success EXPENSIVE "this may take a while" 'sleep 2'
test_done
# vim: ft=sh
$ cat sharness/bad.t
#!/bin/sh
test_description='demo of failing tests'
. ./sharness.sh
test_expect_success "I don't feel so good" false
test_done
# vim: ft=sh
```

Note that `yarn sharness` and `yarn test` fail appropriately. Note that
`yarn sharness-full` fails appropriately after taking two extra seconds,
and `yarn test --full` runs the latter. Each failure message should
print the name of the failing test case, not just the suite name, and
should indicate that the passing tests passed.

Then, remove `sharness/bad.t`, and note that the above commands all
pass, with the `--full` variants still taking longer.

Finally, remove `sharness/good.t`, and note that the above commands all
pass (and all pass quickly).

wchargin-branch: add-sharness
2018-08-06 12:56:25 -07:00
Dandelion Mané 00da630bb2
Rename Contributor & Contribution types (#607)
Consider the following types:

```
// Used to be called "Contributor"
export type Adjacency =
  | {|+type: "SYNTHETIC_LOOP"|}
  | {|+type: "IN_EDGE", +edge: Edge|}
  | {|+type: "OUT_EDGE", +edge: Edge|};

// Used to be called "Contribution"
export type Connection = {|
  +adjacency: Adjacency,
  // This `weight` is a conditional probability: given that you're at
  // the source of this connection's adjacency, what's the
  // probability that you travel along this connection to the target?
  +weight: Probability,
|};

// Used to be called "ScoredContribution"
export type ScoredConnection = {|
  +connection: Connection,
  +source: NodeAddressT,
  +sourceScore: number,
  +connectionScore: number,
|};
```

These types represent how a node's PagerankScore is influenced by its
connections in the markov chain. The previous names, "Contributor",
"Contribution" and "ScoredContribution", were quite confusing as
elsewhere in the project "contributon" means something that added value
to the project, and "contributor" means the author of contributions.
While these new names aren't necessarily much better a priori, in the
context of the project's vernacular they are much less confusing.

Test plan: It's just a rename, and `yarn test` passes.
2018-08-06 12:17:52 -07:00