sourcecred/CHANGELOG.md

14 lines
513 B
Markdown
Raw Normal View History

# Changelog
## [Unreleased]
Retry GitHub queries with exponential backoff (#699) Summary: This patch adds independent exponential backoff to each individual GitHub GraphQL query. We remove the fixed `GITHUB_DELAY_MS` delay before each query in favor of this solution, which requires no additional configuration (thus resolving a TODO in the process). We use the NPM module `retry` with its default settings: namely, a maximum of 10 retries with factor-2 backoff starting at 1000ms. Empirically, it seems very unlikely that we should require much more than 2 retries for a query. (See Test Plan for more details.) This is both a short-term unblocker and a good kind of thing to have in the long term. Test Plan: Note that `yarn test --full` passes, including `fetchGithubRepoTest.sh`. Consider manual testing as follows. Add `console.info` statements in `retryGithubFetch`, then load a large repository like TensorFlow, and observe the output: ```shell $ node bin/sourcecred.js load --plugin github tensorflow/tensorflow 2>&1 | ts -s '%.s' 0.252566 Fetching repo... 0.258422 Trying... 5.203014 Trying... [snip] 1244.521197 Trying... 1254.848044 Will retry (n=1)... 1260.893334 Trying... 1271.547368 Trying... 1282.094735 Will retry (n=1)... 1283.349192 Will retry (n=2)... 1289.188728 Trying... [snip] 1741.026869 Ensuring no more pages... 1742.139978 Creating view... 1752.023697 Stringifying... 1754.697116 Writing... 1754.697772 Done. ``` This took just under half an hour, with 264 queries total, of which: - 225 queries required 0 retries; - 38 queries required exactly 1 retry; - 1 query required exactly 2 retries; and - 0 queries required 3 or more retries. wchargin-branch: github-backoff
2018-08-22 18:37:29 +00:00
- Execute GraphQL queries with exponential backoff (#699)
Re-introduce a simplified git plugin (#685) This commit re-introduces the git plugin, now that it has been radically simplified as described in [1]. The new git plugin only has nodes for commits and only has commit has-parent edges. As compared to the version that was removed in #628, this plugin is far leaner. It doesn't bloat the graph (for `sourcecred/sourcecred`, the git plugin data is just 164k), and as such doesn't incur much performance penalty. Re-incorporating the git plugin also brings some tangible benefits. We already had git nodes in the graph, as the GitHub plugin attaches them to pull requests. Without any git plugin, these nodes are displayed as "uknown nodes" with ugly descriptions. Also, including a git plugin, even one that is very minimal, communicates to users that git is a source of information to SourceCred, and that they can expect more from it in the future. Note that this commit breaks backcompat for existing repositories that were locally loaded after #628. As such, it is best to `rm -rf $SOURCECRED_DIRECTORY` and start with fresh data. Also, due to a known bug in the WeightConfig, you should reset your browser's local storage. Test plan: After removing the SourceCred directory and the stale localStorage, the cred explorer nicely displays git commits, and connects them via has_parent edges. The NodeType filter allows filtering to commits as expected, and the WeightConfig shows node and edge weights for the Git plugin's nodes and edges. [1]: https://github.com/sourcecred/sourcecred/issues/627#issuecomment-413435447
2018-08-16 20:20:41 +00:00
- Introduce a simplified Git plugin that only tracks commits (#685)
- Rename cred explorer table columns (#680)
- Display version string in the app's footer
- Support hosting SourceCred instances at arbitrary gateways, not just
the root of a domain (#643)
- Aggregate over connection types in the cred explorer (#502)
- Start tracking changes in `CHANGELOG.md`
<!-- Please add new entries to the _top_ of this section. -->