751172ea77
Summary: Per #117, this is a first step toward at writing a pagination API that specifically targets our current GitHub query. For design details, see new module docs on `src/plugins/github/graphql.js`. This commit modifies the core GitHub query and thus the `example-repo.json` snapshot: we now request `endCursor` fields for all pagination info, and we request the `id` of the root `repository` field. The former is obviously necessary. The latter is necessary for the repository to be consistent with other nodes that offer connections as fields: we require an ID on the node containing the connection so that we can have random access to it in a continuation selector. Test Plan: Unit tests added. You can also try out the generated continuation queries for yourself: apply the patch below, run `yarn backend`, and then run the `fetchGithubRepo.js` script on `sourcecred/sourcecred`. This will output a nicely formatted query that you can paste directly into GitHub’s API explorer and execute. (Note that, because this patch is not fully polished, the query must be run against a repository that has a continuation for every node type: more pages of issues, PRs, comments, reviews, and review comments. This is due to an easy-but-annoying-to-fix bug in the patch, not in the code included in this commit.) <details> <summary>Patch for generating a continuations query</summary> ```diff diff --git a/src/plugins/github/fetchGithubRepo.js b/src/plugins/github/fetchGithubRepo.js index 789a20e..418c736 100644 --- a/src/plugins/github/fetchGithubRepo.js +++ b/src/plugins/github/fetchGithubRepo.js @@ -6,8 +6,13 @@ import fetch from "isomorphic-fetch"; -import {stringify, inlineLayout} from "../../graphql/queries"; -import {createQuery, createVariables} from "./graphql"; +import {stringify, inlineLayout, multilineLayout} from "../../graphql/queries"; +import { + continuationsFromQuery, + continuationQuery, + createQuery, + createVariables, +} from "./graphql"; /** * Scrape data from a GitHub repo using the GitHub API. @@ -66,8 +71,13 @@ function postQuery(payload, token) { if (x.errors) { return Promise.reject(x); } - ensureNoMorePages(x); - return Promise.resolve(x); + console.log( + stringify.body( + continuationQuery(Array.from(continuationsFromQuery(x.data))), + multilineLayout(" ") + ) + ); + throw new Error("STOPSHIP"); }); } diff --git a/src/plugins/github/graphql.js b/src/plugins/github/graphql.js index 9ea2592..9ead42b 100644 --- a/src/plugins/github/graphql.js +++ b/src/plugins/github/graphql.js @@ -39,11 +39,11 @@ import {build} from "../../graphql/queries"; * * [1]: https://developer.github.com/v4/guides/resource-limitations/#node-limit */ -export const PAGE_LIMIT = 100; -const PAGE_SIZE_ISSUES = 100; -const PAGE_SIZE_PRS = 100; -const PAGE_SIZE_COMMENTS = 20; -const PAGE_SIZE_REVIEWS = 10; +export const PAGE_LIMIT = 10; +const PAGE_SIZE_ISSUES = 10; +const PAGE_SIZE_PRS = 10; +const PAGE_SIZE_COMMENTS = 3; +const PAGE_SIZE_REVIEWS = 1; const PAGE_SIZE_REVIEW_COMMENTS = 10; /** @@ -340,6 +340,36 @@ function* continuationsFromReview( } } +/** + * Combine continuations into a query. + */ +export function continuationQuery( + continuations: $ReadOnlyArray<Continuation> +): Body { + const nonces: string[] = continuations.map((_, i) => `_n${String(i)}`); + const nonceToIndex = {}; + nonces.forEach((n, i) => { + nonceToIndex[n] = i; + }); + const b = build; + const query = b.query( + "Continuations", + [], + continuations.map((continuation, i) => + b.alias( + nonces[i], + b.field( + "node", + {id: b.literal(continuation.enclosingNodeId)}, + continuation.selections.slice() + ) + ) + ) + ); + const body = [query, ...createFragments()]; + return body; +} + /** * These fragments are used to construct the root query, and also to * fetch more pages of specific entity types. ``` </details> wchargin-branch: ad-hoc-pagination-continuations |
||
|---|---|---|
| config | ||
| experiments | ||
| flow-typed/npm | ||
| scripts | ||
| src | ||
| .flowconfig | ||
| .gitignore | ||
| .prettierignore | ||
| .prettierrc.json | ||
| .travis.yml | ||
| LICENSE | ||
| README.md | ||
| package.json | ||
| yarn.lock | ||
README.md
SourceCred
The open-source community provides an enormous amount of value to the world. However, open-source contributors go largely unrewarded and unrecognized. SourceCred aims to help that situation, by building tools that enable quantitatively measuring the value that open-source contributors provide to individual projects, and to the community as a whole.
SourceCred will create a "Cred Graph", which is a graph that shows how the contributions that compose open-source projects are related to and derive value from each other. From this, we'll be able to assign "cred" to users based on how valuable their contributions are. Cred will be assigned based on a mixture of objective data (e.g. references between GitHub pull requests) and subjective feedback (e.g. projects' own judgments on how important different contributions were).
If you'd like to contribute, please follow along with our issues, as we are using issues to coordinate development and design decisions. We also have a slack.