Commit Graph

1055 Commits

Author SHA1 Message Date
Dandelion Mané bd28030caa
Ensure build failure emails (#339)
When the cron build fails, we want to make sure that we know about it.
This commit ensures that Travis emails @wchargin and @decentralion on
any build failure; our email filters will ensure that cron job failures
get noticed.

Based on documentation at: https://docs.travis-ci.com/user/notifications
2018-06-04 14:52:07 -07:00
Dandelion Mané 0c8ede57a2
Add v3 `Graph` nodes methods (#331)
This commit implements and tests the following methods on the `Graph`:
- `addNode`
- `removeNode`
- `hasNode`
- `nodes`

Test plan: Unit tests are included.

Paired with @wchargin
2018-06-04 14:47:27 -07:00
William Chargin 5b3b28c705
Fetch PR additions and deletions from GitHub (#340)
Summary:
Closes #336.

Test Plan:
Snapshots updated; changes are easily readable. Existing tests pass.
Running the cred explorer on the `sourcecred/sourcecred` graph shows
pull requests like `#124 (+73/−3): Update the README`, which is good.

wchargin-branch: pr-addition-deletion
2018-06-04 14:28:35 -07:00
Dandelion Mané 6585700c0c
Upgrade flow to 0.73 (#338)
It still passes :)

Test plan: Travis
2018-06-04 14:22:10 -07:00
William Chargin 3e83576422
Update `src/v1/` paths for CI cron (#337)
Summary:
Fixes breakage due to https://github.com/sourcecred/sourcecred/pull/327.

Test Plan:
`yarn travis --full` now passes.

wchargin-branch: src-v1-cron
2018-06-04 14:16:43 -07:00
Dandelion Mané 3ac051b16c
Upgrade prettier to 1.13 (#335)
Release notes: https://prettier.io/blog/2018/05/27/1.13.0.html
2018-06-04 13:02:17 -07:00
Dandelion Mané 754de16524
Reorganize Graph address-related code (#334)
This commit re-organizes our addressing code according to the following
conventions:
- All address code is implemented in `core/_address.js`, but clients
  should not import that file directly.
- Public address-related methods will be made available via the
  `Address` object in `core/graph.js`.
- The inaugural set of public address methods are:
  - `nodeAddress`, for constructing a new `NodeAddress` from parts
  - `edgeAddress`, for constructing a new `EdgeAddress` from parts
  - `toParts`, for turning an address back into parts
- The types `NodeAddress` and `EdgeAddress`are also exported from
  `graph.js`


Test plan: The tests cover both runtime and type properties.

Paired with @wchargin.

For more context and design discussions, see:
https://github.com/sourcecred/sourcecred/pull/334
2018-06-04 12:11:28 -07:00
Dandelion Mané 3e7776c245
Add basic Graph data structures (#330)
This very minor commit adds the basic data structures to the `Graph`
that will act as underlying storage for nodes and edges. This is similar
to how nodes and edges are stored in `Graph` v1 and v2, except much
simpler, because we now have string keys, no data stored with nodes, and
minimal data stored with edges.

Test plan: The only observable change is that the graph can now be
constructed without error. I added a unit test to verify this behavior
:) as well as that the Graph constructor properly returns an object of
type `Graph` and not an `any`.
(Context: https://github.com/facebook/flow/issues/6400)

Paired with @wchargin
2018-06-03 16:39:12 -07:00
Dandelion Mané 3acfefb904
Implement {to,from}{Node,Edge}Address (#329) (#333)
This commit implements Node/Edge addresses, and helper functions for
generating and manipulating them.

Test plan: Unit tests included.

Paired with @wchargin
2018-06-02 18:56:21 -07:00
Dandelion Mané aff4ddd4e3
Add a skeleton of the v3 Graph API (#328)
The newly added `v3/src/core/graph.js` module is the heart of our `v3`
refactor, which aims at having our `Graph` class be just a graph[1] and
not a key-value store to boot.

This commit adds a skeleton of our intended `Graph` API. We've mostly
ported features from our existing `v1` `Graph` class, but with a few
changes:

- `NodeAddress` and `EdgeAddress` are both strings under the hood, which
allows for better performance and use of native JS Maps and Sets.

- Addresses are now hierarchical paths, so the functionality previously expressed
through `pluginName` and `type` can now be encoded into the path
components.

- `NodeAddress` and `EdgeAddress` are distinct types, and have different
serializations, so they can't get mixed up at runtime.

- It is no longer possible to bind arbitrary payloads to nodes or edges.
The expectation is that external data stores will be responsible for
that.

Test plan: No code was implemented; flow and lint are sufficient.

Paired with @wchargin

[1]: Technically, it's a [quiver]

[quiver]: https://en.wikipedia.org/wiki/Quiver_(mathematics)
2018-06-02 17:52:45 -07:00
Dandelion Mané ba721a6fbb
Fork project to v1/ and v2/ in preparation for v3 (#327)
We want to reset some of our basic assumptions, and make `Graph` into a
pure graph implementation, rather than a hybrid graph and key-value
store.

This is a substantial rewrite, so we want to start from scratch in a v3/
directory and pull code into v3 as necessary. So that we can do this in
a relatively clean fashion, we're first moving the v1 and v2 code into
their own directories.

Paired with @wchargin

Test plan:  Travis, and `yarn backend`, `node bin/sourcecred.js start`.

Note that `yarn backend` and `node bin/sourcecred.js start` both use the
v1 versions. We'll migrate those (by changing paths.js) to v3 when
appropriate.
2018-06-01 17:17:44 -07:00
Dandelion Mané 797a2fbf9f
Remove `src/tools/loadCombinedGraph` (#326)
It was not used anywhere.

Test plan: Travis

Paired with @wchargin
2018-06-01 17:12:13 -07:00
William Chargin 40409f3151
Generate `bin/` in-place for cron CI (#325)
Summary:
This fixes a bug introduced in #317, which only occurred in the cron job
variant of the CI script (`yarn travis --full`): the two scripts run in
the cron job depend on `yarn backend` having previously written to the
`bin/` directory, but this is precisely what we wanted to prevent. To
fix this, we simply add an additional target for `yarn backend` during
the cron job. This is a little bit wasteful in that we compile the
backend applications twice, but it’s not a big deal because (a) it only
runs in cron jobs, so it won’t slow down normal builds, and (b) it only
takes about 5 seconds, anyway.

Test Plan:
Export a `GITHUB_TOKEN` and run `yarn travis --full`, which fails before
this change and passes after it.

wchargin-branch: cron-ci-overwrite-bin
2018-05-31 18:29:55 -07:00
William Chargin fbf0d65f76
Implement `toJSON` and `fromJSON` for GraphV2 (#323)
Summary:
The storage format is similar to the V1 storage format. Nodes are sorted
by address and stored in an ordered list. Edges have `srcIndex` and
`dstIndex`, keyed against the node order. Each node stores the name of
its plugin and the result of invoking `toJSON` on its payload.

Test Plan:
Unit tests added; run `yarn travis`. The single added snapshot is easy
to read and verify.

wchargin-branch: v2-json
2018-05-31 14:25:17 -07:00
William Chargin 6e442c2f0c
Use payload-JSON equality for `Node`s (#322)
Summary:
A `Node` includes a ref and a payload. We should say that two nodes at
the same address are equal iff their payloads are equal, and two
payloads at the same address are equal iff their JSON contents are
equal. Importantly, `Node` equality does not depend on the `ref`’s
internal structure: this structure includes a reference to the enclosing
graph, and so using this notion of node equality caused `equals` to
incorrectly return `false` when the inputs were two logically equal
graphs with different internal structure. (It also caused `equals` to be
very slow, performing a deep equality check on the graphs for every
node.)

Test Plan:
A unit test has been strengthened. It fails before this patch, and
passes afterward.

wchargin-branch: v2-node-payload-json-equality
2018-05-31 13:22:37 -07:00
William Chargin 5dcf32560d
Implement `copy` for GraphV2 (#321)
Summary:
Both the implementation and the tests here are pretty straightforward.
The only change from GraphV1 is that we should explicitly check that the
plugins are copied correctly, because graph equality does not check
this.

Test Plan:
New unit tests suffice.

wchargin-branch: copy-v2
2018-05-31 11:57:20 -07:00
William Chargin 760fd00427
Implement `mergeConservative` for GraphV2 (#320)
Summary:
I dropped the `edgeDecomposition` and `neighborhoodDecomposition` test
cases, because (a) we don’t have a canonical “interesting graph” on
which to run them, compounding that (b) it’s not clear how much value
they add.

Test Plan:
New unit tests suffice.

wchargin-branch: mergeConservative-v2
2018-05-31 11:39:58 -07:00
Dandelion Mané c7854c1154
Add the `NodeReference.neighbors` implementation (#319)
The implementation is adapted from our previous implementation, but has
been refactored to be more appropriate for our generator-function
approach.

The unit tests comprehensively explore a simple example graph, testing
the following cases:
- Direction filtering (IN/OUT/ANY/unspecified)
- Node types
- Edge types
- Self-loop edges
- Dangling edges
- Removed edges don't appear

Test plan: Carefully inspect the added unit tests.

Paired with @wchargin
2018-05-30 12:48:31 -07:00
William Chargin 8182bb340c
Add edges to the graph (#318)
Summary:
Based on code originally paired with @decentralion.

Test Plan:
Unit tests added. Running `yarn travis` is sufficient.

wchargin-branch: v2-edges
2018-05-29 18:47:04 -07:00
William Chargin f8242c8cab
Don’t erase the `bin/` folder in CI (#317)
Summary:
Previously, our CI script would run `yarn backend`, which has the
side-effect of erasing the `bin/` directory. By itself, this is not
great, but not awful. However, this frequently triggers a race condition
in Prettier, causing the `check-pretty` step of the build to fail. (More
details: https://github.com/prettier/prettier/issues/4468.)

This patch changes the CI script to build the backend scripts into a
temporary directory.

Test Plan:
Before applying this patch: `yarn backend` and then `yarn travis`. If
this consistently causes a Travis failure due to `check-pretty`, then
your machine can reproduce the race condition that we‛re trying to
eliminate. (Otherwise, you can try creating a bunch more Git history…
I’m not really sure what to say. It is a race condition, after all.)
Then, apply this patch, and repeat the above steps; note that the error
no longer occurs, and that the build output is to a temporary directory.

wchargin-branch: ci-preserve-bin
2018-05-29 15:40:42 -07:00
Dandelion Mané c0c79fd608
Refactor `addressFilterer` for using PluginFilter (#316)
Given that we have reified the type `PluginFilter` as a consistent way
to filter nodes and edges by plugin and type, we should also have
re-usable logic for that end.

This commit adds `addressFilterer`, which takes a `PluginFilter` and
returns a function that checks whether a given address matches the
filter. Right now, only the `nodes` function uses it.

For added safety, `addressFilterer` does runtime validation that the
required `plugin` property was set.

Test plan: See included unit tests.
2018-05-29 14:10:39 -07:00
Dandelion Mané 3b3564203c
Flow: enable `//$ExpectFlowError` (#315)
As of this commit, adding the comment `//$ExpectFlowError` in flow-typed
code asserts that the next line must cause a flow error. If it does, no
error or warning is generated. If it does not, then this produces a flow
warning, which is visible to developers running `yarn flow` and
additionally causes travis to fail.

Test plan:

- As committed, `yarn travis` passes.
- I added `//$ExpectFlowError` above some line of flow-checked code which does
not currently throw an error. Afterwards, `yarn travis` failed (and a
helpful message was displayed in console on running `yarn flow`)
- I added the following bad code into one of our files:
```javascript
//$ExpectFlowError
const foo: string = 3;
```
As expected, `yarn flow` and `yarn travis` both passed.
2018-05-29 13:56:36 -07:00
William Chargin 87b7df5957
Use a `Symbol` for DelegateNodeReference base ref (#313)
Summary:
It’s critical, even more so than usual, that the “base reference”
property of a `DelegateNodeReference` be a private property, because
this class is designed for inheritance. In ECMAScript 6, we can achieve
this by giving the property a `Symbol` key instead of a string key.
Unfortunately, Flow doesn’t know about `Symbol`s, so we need a few casts
through `any`, but they are localized to as small a scope as possible.

Test Plan:
Unit tests added. Note that they pass both before and after this change.

wchargin-branch: symbol-base-ref
2018-05-29 12:28:34 -07:00
William Chargin 6663c4f8ad
Fix `ensure-flow.sh` running under Node (#314)
Summary:
The use of `tee /dev/stderr` failed when running as a child process
under Node for some reason. (I haven’t been able to figure out why—it
works fine when run as a standalone script or when run as a child
process under Python.) This is also technically Linux-specific, so I’ve
changed it to use a process substitution. After looking around for a
bit, there doesn’t seem to be a way to do this in a way that is
portable, uses only POSIX shell features, and doesn’t create temporary
files all at the same time, so the script is now run under `bash`.

Test Plan:
Run `yarn travis` and note that the `ensure-flow.sh` output no longer
contains the line `tee: /dev/stderr: No such device or address`.

wchargin-branch: no-tee-devstderr
2018-05-29 12:20:53 -07:00
Dandelion Mané 8ab0598939
Graphv2: enable adding and retrieving nodes (#312)
This commit adds the following methods to the `Graph`:

* `addNode`
* `removeNode`
* `ref`
* `node`
* `nodes`
* `equals`

The graph now supports adding nodes, with thorough testing. Other
methods were implemented as necessary to test that `addNode` was
implemented properly.

Also, we've made a slight change to spec: `nodes` (and other filter
options) accept a `PluginFilter` object, which, if present, must
specify a plugin and may specify a type.

I've taken the opportunity to re-write the graph test code. Instead of
having a complicated `graphDemoData` file that creates a graph with many
different nodes, I've created an `examplePlugin` which makes it trivial
to instantiate new simple Foo and Bar nodes on the fly. Then, test cases
can construct a small graph that is clearly appropriate for whatever
functionality they are testing.

Test plan: Unit tests were added, travis passes.
2018-05-29 12:14:50 -07:00
William Chargin 13acbe1efd
Trim Flow’s server startup build output (#311)
Test Plan:
Run `yarn flow stop; yarn travis | cat` and note the absence of the
really long line that has ~2500 bytes of “Server is initializing”.

wchargin-branch: quiet-flow-server
2018-05-28 17:06:56 -07:00
Dandelion Mané 41ab7aa729
Create an explicit `PluginType` (#310)
Currently, when filtering by type (e.g. in neighbors), we require only a
type string. This is a design flaw, as if two plugins both define an
"ISSUE" type, either plugin may unexpectedly receive the other plugin's
nodes or edges.

We fix the flaw by explicitly binding the plugin name and type field
together as `PluginType`.

Test plan: Flow passes, and so does the following invocation to check
our design doc:
```
cat <(printf '// @flow\n') \
  <(awk -v RS='```[a-z]*' '(NR+1)%2' \
      src/core2/address_payload_unification_design.md) \
  | yarn -s flow check-contents
```
2018-05-28 15:36:38 -07:00
Dandelion Mané 7562453f02
Add plugin handler logic to Graph, with tests (#309)
- Add the `plugins` method to graph, and have the constructor take them
- Add in demo `PluginHandlers` to the `graphDemoData`
- Add unit tests for `plugins` methods

Test plan: Travis

Paired with: @wchargin
2018-05-28 15:29:18 -07:00
William Chargin 47c7e33ec2
Disable the `no-useless-constructor` lint rule (#308)
Summary:
In fact, a constructor that only calls `super` is useful: it specifies
the API for constructing an object of a given class without needing to
traverse its prototype chain. (That constructors are inherited at all is
arguably a design flaw.)

Test Plan:
None.

wchargin-branch: yes-useful-constructor
2018-05-28 15:01:28 -07:00
Dandelion Mané 630a6d6532
Add `DelegateNodeReference` to `graph` module (#307)
Also, rename `NodeDelegateReference` to `DelegateNodeReference` in the
design doc.

Test plan:
Didn't add tests, as the implementation is trivial. Flow passes.
2018-05-28 13:08:01 -07:00
Dandelion Mané 7078125c56
Create type signatures for new graph API (#306)
src/core2/graph.js has the types and Graph class scaffold as
described in address_payload_unification_design.md.
Also, we decided to modify the design doc to include a type parameter
for Edges (although we aer open to removing if it becomes cumbersome).

Test plan:
`yarn flow` passes, and the design doc still typechecks too.

Paired with @wchargin
2018-05-28 13:04:03 -07:00
Dandelion Mané c957e84da1
Update address_payload_unification_design (#305)
Since this document was originally written, we've iterated on the design
a bit. @wchargin and I have re-reviewed the doc, and we think it's now
up-to-date with our plans.

Test plan: I ran the following command to bring the code into my
clipboard:

```
❯ awk -v RS='```[a-z]*' '(NR+1)%2' src/core2/address_payload_unification_design.md | xsel -ib
```

Test plan:
Pasting this into https://flow.org/try produced no errors.

Paired with @wchargin
2018-05-28 12:08:36 -07:00
Dandelion Mané f22cf04e75
Setup `src/core2` as graph refactor staging area (#304)
In #190, @wchargin lays out an ambitious proposal for refactoring away
the graph's address-payload distinction. This has proven to be a
complicated refactor to land, so we are going to achieve it by forking
parts of the project into v2, updating incrementally in v2, and then
replacing original components with their v2 versions.

Test plan:
No new code added. `yarn travis` passes.

Paired with @wchargin
2018-05-28 10:52:28 -07:00
Dandelion Mané c68b78f959
Remove the artifact plugin (#303)
Given that we are undergoing a major world-changing refactor (#190), all
outstanding code needs to be refactored to use the new conventions. We
don't actually use the Artifact Plugin yet, and reading the code, it
needs non-trivial rewrites to be in sync with the new world.

Rather than maintain it now, I am deleting it; we can regain the context
when the time is ripe to setup and integrate the plugin.

Test plan: Travis passes. `yarn start` produces no references to the
artifact editor.
2018-05-25 19:29:36 -07:00
William Chargin f0fcf02791
Check for STOPSHIPs in CI (#301)
Summary:
Placing `STOPSHIP` or `stopship` (or any case variant) in any file
tracked by Git will now cause a `yarn travis` failure. If you need to
use this string, you can concatenate it as `"stop" + "ship"` or
equivalent.

Test Plan:
In `travis.js`, change `"check-stop" + "ships"` to `"check-stopships"`,
and note that this causes the build to fail with a nice message. Note
that this also causes `check-stopships.sh` to fail even when invoked
from an unrelated directory, like `src`.

wchargin-branch: check-stopships
2018-05-25 19:27:31 -07:00
William Chargin ab10e1746c
Remove `lint-staged` pre-commit hook (#300)
Summary:
This just slows down commits by a few seconds. We `check-pretty` in
Travis, so this doesn’t actually catch anything—and, anecdotally, it has
never caught anything for me because I automatically run `prettier` on
save and also (almost) always run Travis before pushing.

Test Plan:
Run `git commit --amend --no-edit` and note that it is now fast!

wchargin-branch: no-lint-on-commit
2018-05-25 19:25:43 -07:00
William Chargin 15f65fbff4
Simplify legacy “simple”/“advanced” tests in Graph (#302)
Summary:
Improves the result of the preceding commit to strip out cruft. Paired
with @decentralion.

Test Plan:
```shell
$ ! git grep -Fi -e simple -e advanced src/core/graph*
```
Also review `awk '/(describe|it)\(/' src/core/graph.test.js`.

wchargin-branch: simplify-simple-advanced
2018-05-25 19:25:23 -07:00
William Chargin d8fef6bf47
Fix tests for edge-induced graph disequality (#299)
Summary:
These tests have apparently been borked since the beginning. They were
testing properties about nodes instead of edges, had the wrong test
names, and had the LHS/RHS backward… this should make them more useful.

Test Plan:
Unit tests suffice, I suppose.

wchargin-branch: tests-for-missing-edges
2018-05-25 18:40:49 -07:00
William Chargin 853518fd60
Remove “simple graph” demo data (#298)
Summary:
We now use the “advanced” graph everywhere. Happily, all tests still
pass!

Starting from `:%s/\%(advanced\|simple\)MealGraph/mealGraph/gc`, only a
few tests had to be manually changed: in particular, the `#equals` tests
for “returns false when the ?HS has nodes missing in the ?HS” became
trivially incorrect by the above substitution, and were therefore
changed to use more appropriate inputs.

Paired with @decentralion.

Test Plan:
Existing unit tests suffice.

wchargin-branch: remove-simple-graph
2018-05-25 14:03:28 -07:00
William Chargin 719bf47156
Merge `merge`s (#297)
Summary:
We had three graph merging functions: `merge`, `mergeConservative`, and
`mergeManyConservative`. Of these, `merge` was never used outside of
code directly testing its behavior, and `mergeConservative` is a
strictly inferior version of `mergeManyConservative`. This commit
removes `merge` and `mergeConservative`, and renames
`mergeManyConservative` to `mergeConservative`.

Paired with @decentralion.

Test Plan:
Existing unit tests suffice; some useless tests pruned.

wchargin-branch: mmeerrggee
2018-05-25 13:55:44 -07:00
Dandelion Mané 418d046691
Remove Node/Edge polymorphism on Graph (#289)
As previously implemented, the Graph was polymorphic in its NodePayload
and EdgePayload. This was a lot of bookkeeping for very
little apparent benefit. In general, graphs may be constructed with
foreign plugins, so it's hard to do anything with this information.

In my experience, having the Graph polymorphism has never caught a bug,
and has led to lots of boilerplate code, especially typing `Graph<any,
any>`.  I view the fact that in #286 we added a new core `NodeReference`
concept, which always types its Graph as `<any, any>`, as strongly
suggestive that this was not going to provide any lasting value.

In this commit, I've removed the Graph polymorphism. Note how in many
cases where we were typing the graph, it provided no value, as evidenced
by the fact that the imported Node and Edge types were used no-where
else in the file other than in the Graph declaration.

Test plan:
Removing extra typing information is very unlikely to cause regressions.
`yarn flow` and `yarn lint` both pass.
2018-05-24 12:03:47 -07:00
William Chargin e20b794edc
Add design doc for address–payload unification (#296)
Summary:
See #190, and discussion on this PR, for context.

wchargin-branch: address-payload-unification-design
2018-05-24 11:37:04 -07:00
William Chargin 2b301f9159
Use indexed edges in graph internals (#295)
Summary:
This is an implementation-only, API-preserving change to the `Graph`
class. Edges’ `src` and `dst` attributes are now internally represented
as integer indices into a fixed ordering of nodes, which may depend on
non-logical properties such as insertion order. The graph’s serialized
form also now stores edges with integer `src`/`dst` keys, but the node
ordering is canonicalized so that two graphs are logically equal if and
only if their serialized forms are equal. This change substantially
reduces the rest storage space for graphs: the `sourcecred/sourcecred`
graph drops from 39MB to 30MB.

Currently, the graph will have to translate between integer indices and
full addresses at each client operation. This is not actually a big
performance regression, because it is just one more integer-index
dereference over the previous behavior, but it does indicate that the
optimization is not living up to its full potential. In subsequent
changes, the `NodeReference` class will be outfitted with facilities to
take advantage of the internal indexing; a long-term goal is that
roughly all operations should be able to be performed within the indexed
layer, and that translating between integers and addresses should only
happen at non-hot-path API boundaries.

This diff is considerably smaller and easier to read with `-w`.

Paired with @decentralion.

Test Plan:
I inspected the snapshots for general form, and manually verified that
the indices for one edge were correct (the MERGED_AS edge for the head
commit of the example repo). Other than that, existing unit tests mostly
suffice; one new unit test added.

wchargin-branch: graph-indexed-edges
2018-05-22 13:15:39 -07:00
Dandelion Mané 5a40bb0a30
Use versioning for `Graph` and `AddressMap` (#293)
This commit adds explicit versioning to the `Graph` and `AddressMap`
JSON representations, using the new `compat` module. This will make it
safer to change the serialization format for these classes.

(Note: I don't expect we'll add backcompat handlers for these classes
soon, but having versioning means that we can change the serialization
format in a way that breaks old data cleanly and explicitly, rather than
introducing undefined behavior.)

Test plan: The changes are slight, and well-captured by the snapshot
tests. Note that after this commit, the SourceCred commands will fail on
old data, so old data will need to be regenerated.
2018-05-21 11:47:05 -07:00
Dandelion Mané 1fc860bd56
Add data versioning utilities (#292)
This commit adds the `src/util/compat` module, which is responsible for
data compatibility. It provides two methods: `toCompat` and
`fromCompat`.

`toCompat` allows an object to be versioned with a type and version
string. Doing so wraps the object in an array whose first element has
`version` and `type` fields.

`fromCompat` allows loading compatibilized data. It takes a `type` and
`version` expectation, as well as optional processors for various data
versions. It throws an error if the data was not compatibilized, or if
it is the wrong type, or if it has an unsupported version.

The versioning is designed to be composable; see the final test block
for an example of loading a structure with nested versioning.

Test plan: Carefully inspect the included unit tests, and feel free to
propose more.

For context, see #280.
2018-05-21 11:21:05 -07:00
Dandelion Mané 180c3454af
Upgrade babel-plugin-flow-react-proptypes to 23 (#294)
Will allow us to use opaque types in #292 without breaking the build
in #293.

Test plan:
If travis passes, we're good.
2018-05-21 11:11:21 -07:00
Dandelion Mané 610a92a683
Run commands/graph with more heap available (#291)
`sourcecred graph` tends to die due to lack of heap during the Git
plugin. Node defaults to ~1.76GB of heap available, which is just not
that much. This commit uses the `--max_old_space_size` argument to
`node` to increase the limit. We use a default value of `8192`, and it's
configurable by the user via a flag.

This command is only available natively in `sourcecred graph`, because
that command turns on other node processes. For commands that run in
their original process, you would need to set the value yourself.
2018-05-16 20:07:40 -07:00
Dandelion Mané 64a8514cf8
markovChain should scale to 1m nodes (#290)
`markovChain.findStationaryDistribution` is currently written such that
it blows the function call stack size if it is called with >200k nodes.

This commit effects a small refactor so that the delta is computed in a
loop over elements in the distribution, rather than creating a massive
function arguments list. This results in the correct behavior.

Test plan:
We discussed offline and decided not to test it. Reasons:

- We could factor out `computeDelta` and test that function in
isolation, but I think this provides very, very little value. The class
of regression it protects against is almost precisely the case where
someone reverts this commit.

- We could test the entire `findStationaryDistribution` function, which
is more valuable, but requires a large, slow test case (~1 sec)

- This is a bug that is unlikely to re-occur or to slip by unnoticed if
it does. It's also unlikely to surface in future refactoring.
2018-05-16 19:27:41 -07:00
Dandelion Mané 1bd444a33b
Refactor Git plugin to use NodeReference (#288)
See #286 for context.

I also upgraded client code in src/app.

Test plan:
Unit tests are extensive, including testing that `get`, `ref`, and
constructors are overriden on every `GitReference` and `GitPorcelain`
type that is exposed to clients.

Paired with @wchargin
2018-05-15 19:10:30 -07:00
Dandelion Mané bb77c36626
Update GitHub to use NodeReference & NodePorcelain (#287)
See #286 for context. There are a few miscellaneous changes in
src/app/credExplorer to change clients to use the new API.

Test plan:
New unit test were added, and existing behavior is preserved. Most of
the functionality of the GitHub porcelain was already well tested.

Paired with @wchargin
2018-05-15 17:41:36 -07:00