status-im
/
sourcecred
mirror of https://github.com/status-im/sourcecred.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
sourcecred/package.json

126 lines
3.4 KiB
JSON
Raw Normal View History

Frontend setup (#13) * Initial infrastructure for sourcecred-explorer. Setup npm, yarn, flow, jest using scaffolding from create-react-app. - used create-react-app to get the basic pieces - added support for flow via instructions in the create-react-app user guide - added flow-typed so that flow+jest will work well * Add some additional resources * Add index.css * Remove .DS_STORE per @wchargin's suggestion
2018-02-16 22:32:24 +00:00
{
"name": "sourcecred-explorer",
"version": "0.1.0",
"private": true,
"dependencies": {
Add style support to the artifacts app Test Plan: Note that the header, when rendered, is magenta. wchargin-branch: stylish-artifacts
2018-03-19 22:31:08 +00:00
"aphrodite": "^2.1.0",
Eject from create-react-app (#19) Summary: This commit was created by performing the following steps: - `rm -r node_modules` - `yarn install` - `yarn eject` - `rm -r node_modules` - `yarn install` on an environment wit Node v6.11.1. The final `rm -r node_modules && yarn install` ensures that the `yarn.lock` file is updated accordingly. Test Plan: `yarn test`, `yarn flow`, and `yarn start` all still do the right thing. wchargin-branch: eject
2018-02-17 21:28:47 +00:00
"autoprefixer": "7.1.6",
"babel-core": "6.26.0",
"babel-eslint": "7.2.3",
"babel-jest": "20.0.3",
"babel-loader": "7.1.2",
"babel-preset-react-app": "^3.1.1",
"babel-runtime": "6.26.0",
"case-sensitive-paths-webpack-plugin": "2.1.1",
"chalk": "1.1.3",
"css-loader": "0.28.7",
"dotenv": "4.0.0",
"dotenv-expand": "4.0.1",
"eslint": "4.10.0",
"eslint-config-react-app": "^2.1.0",
"eslint-loader": "1.9.0",
"eslint-plugin-flowtype": "2.39.1",
"eslint-plugin-import": "2.8.0",
"eslint-plugin-jsx-a11y": "5.1.1",
"eslint-plugin-react": "7.4.0",
"extract-text-webpack-plugin": "3.0.2",
"file-loader": "1.1.5",
Frontend setup (#13) * Initial infrastructure for sourcecred-explorer. Setup npm, yarn, flow, jest using scaffolding from create-react-app. - used create-react-app to get the basic pieces - added support for flow via instructions in the create-react-app user guide - added flow-typed so that flow+jest will work well * Add some additional resources * Add index.css * Remove .DS_STORE per @wchargin's suggestion
2018-02-16 22:32:24 +00:00
"flow-bin": "^0.65.0",
Eject from create-react-app (#19) Summary: This commit was created by performing the following steps: - `rm -r node_modules` - `yarn install` - `yarn eject` - `rm -r node_modules` - `yarn install` on an environment wit Node v6.11.1. The final `rm -r node_modules && yarn install` ensures that the `yarn.lock` file is updated accordingly. Test Plan: `yarn test`, `yarn flow`, and `yarn start` all still do the right thing. wchargin-branch: eject
2018-02-17 21:28:47 +00:00
"fs-extra": "3.0.1",
"html-webpack-plugin": "2.29.0",
Set up prettier as an autoformatter (#18) Setup prettier precommit, with yarn prettify to reformat. - Prettier runs as a precommit hook and fails if code is improperly formatted. - Run `yarn prettify` to reformat the project.
2018-02-18 09:59:28 +00:00
"husky": "^0.14.3",
Use isomorphic-fetch instead of node-fetch Summary: Paired with @dandelionmane. Test Plan: ``` $ CI=true yarn test $ yarn backend $ GITHUB_TOKEN="<your_token>" src/plugins/github/fetchGitHubRepoTest.sh ``` wchargin-branch: isomorphic-fetch
2018-03-20 00:01:41 +00:00
"isomorphic-fetch": "^2.2.1",
Eject from create-react-app (#19) Summary: This commit was created by performing the following steps: - `rm -r node_modules` - `yarn install` - `yarn eject` - `rm -r node_modules` - `yarn install` on an environment wit Node v6.11.1. The final `rm -r node_modules && yarn install` ensures that the `yarn.lock` file is updated accordingly. Test Plan: `yarn test`, `yarn flow`, and `yarn start` all still do the right thing. wchargin-branch: eject
2018-02-17 21:28:47 +00:00
"jest": "20.0.4",
Fetch GitHub repos using the GitHub v4 API (#75) Summary: It’s a whole new world of GraphQL! Our parser is now just a GraphQL query that asks for exactly what we want and dumps it to a file. The data exposed by the v4 API is also in a much nicer format than that of the v3 API, so this is pretty much a universal improvement. Currently, we do not handle pagination. We require that the repository in question have fewer than a fixed number of issues, and comments per issue, and reviews per PR, and review comments per PR, and so on. If this limit is exceeded, the script will fail-fast with a nice error message. To fix this, we’ll need to write a general-purpose pagination API that allows traversing cursors at any level of the query. Paired with @wchargin. Test Plan: Run $ GITHUB_TOKEN="your_token_here" src/backend/fetchGitHubRepoTest.sh and verify that it exits with 0. Note that if you change this script’s repository from `tiny-example-repository` to `sourcecred`, the script correctly fails and outputs a useful diff. wchargin-branch: github-v4-graphql
2018-03-15 21:56:25 +00:00
"json-stable-stringify": "^1.0.1",
Set up prettier as an autoformatter (#18) Setup prettier precommit, with yarn prettify to reformat. - Prettier runs as a precommit hook and fails if code is improperly formatted. - Run `yarn prettify` to reformat the project.
2018-02-18 09:59:28 +00:00
"lint-staged": "^6.1.1",
Convert in-memory Git repos to cred graphs (#169) Test Plan: This snapshot test is too unwieldy to actually read—it’s 1000 lines of opaque SHAs and thrice-stringified JSON objects—so it should be interpreted as a regression test only. The programmatic tests should suffice. wchargin-branch: wip-git-create-graph
2018-04-30 22:23:37 +00:00
"lodash.clonedeep": "^4.5.0",
Add an equality function for `Graph` (#61) Summary: We need this for testing graph equality: deep-equality is not sufficient because two graphs can be logically equal even if, say, two nodes are added in different orders. This commit adds a dependency on `lodash.isequal` for deep equality. Test Plan: New unit tests added. Run `yarn flow && yarn test`. wchargin-branch: graph-equals
2018-03-03 05:13:30 +00:00
"lodash.isequal": "^4.5.0",
Remove `address.sortedByAddress` (#161) Previously, the address module exported `sortedByAddress`, a utility function that sorts an array of `Addressable`s. This function was only used in test code. This commit replaces it with generic usage of `lodash.sortBy`. This reduces the API surface area of the module, and removes test-only code from the exported api. New dependency added: `lodash.sortby` https://www.npmjs.com/package/lodash.sortby
2018-04-27 21:29:49 +00:00
"lodash.sortby": "^4.7.0",
Eject from create-react-app (#19) Summary: This commit was created by performing the following steps: - `rm -r node_modules` - `yarn install` - `yarn eject` - `rm -r node_modules` - `yarn install` on an environment wit Node v6.11.1. The final `rm -r node_modules && yarn install` ensures that the `yarn.lock` file is updated accordingly. Test Plan: `yarn test`, `yarn flow`, and `yarn start` all still do the right thing. wchargin-branch: eject
2018-02-17 21:28:47 +00:00
"object-assign": "4.1.1",
"postcss-flexbugs-fixes": "3.2.0",
"postcss-loader": "2.0.8",
Set up prettier as an autoformatter (#18) Setup prettier precommit, with yarn prettify to reformat. - Prettier runs as a precommit hook and fails if code is improperly formatted. - Run `yarn prettify` to reformat the project.
2018-02-18 09:59:28 +00:00
"prettier": "^1.10.2",
Eject from create-react-app (#19) Summary: This commit was created by performing the following steps: - `rm -r node_modules` - `yarn install` - `yarn eject` - `rm -r node_modules` - `yarn install` on an environment wit Node v6.11.1. The final `rm -r node_modules && yarn install` ensures that the `yarn.lock` file is updated accordingly. Test Plan: `yarn test`, `yarn flow`, and `yarn start` all still do the right thing. wchargin-branch: eject
2018-02-17 21:28:47 +00:00
"promise": "8.0.1",
"raf": "3.4.0",
Frontend setup (#13) * Initial infrastructure for sourcecred-explorer. Setup npm, yarn, flow, jest using scaffolding from create-react-app. - used create-react-app to get the basic pieces - added support for flow via instructions in the create-react-app user guide - added flow-typed so that flow+jest will work well * Add some additional resources * Add index.css * Remove .DS_STORE per @wchargin's suggestion
2018-02-16 22:32:24 +00:00
"react": "^16.2.0",
Eject from create-react-app (#19) Summary: This commit was created by performing the following steps: - `rm -r node_modules` - `yarn install` - `yarn eject` - `rm -r node_modules` - `yarn install` on an environment wit Node v6.11.1. The final `rm -r node_modules && yarn install` ensures that the `yarn.lock` file is updated accordingly. Test Plan: `yarn test`, `yarn flow`, and `yarn start` all still do the right thing. wchargin-branch: eject
2018-02-17 21:28:47 +00:00
"react-dev-utils": "^5.0.0",
Frontend setup (#13) * Initial infrastructure for sourcecred-explorer. Setup npm, yarn, flow, jest using scaffolding from create-react-app. - used create-react-app to get the basic pieces - added support for flow via instructions in the create-react-app user guide - added flow-typed so that flow+jest will work well * Add some additional resources * Add index.css * Remove .DS_STORE per @wchargin's suggestion
2018-02-16 22:32:24 +00:00
"react-dom": "^16.2.0",
Eject from create-react-app (#19) Summary: This commit was created by performing the following steps: - `rm -r node_modules` - `yarn install` - `yarn eject` - `rm -r node_modules` - `yarn install` on an environment wit Node v6.11.1. The final `rm -r node_modules && yarn install` ensures that the `yarn.lock` file is updated accordingly. Test Plan: `yarn test`, `yarn flow`, and `yarn start` all still do the right thing. wchargin-branch: eject
2018-02-17 21:28:47 +00:00
"style-loader": "0.19.0",
"sw-precache-webpack-plugin": "0.11.4",
"url-loader": "0.6.2",
"webpack": "3.8.1",
"webpack-dev-server": "2.9.4",
"webpack-manifest-plugin": "1.3.2",
"whatwg-fetch": "2.0.3"
Frontend setup (#13) * Initial infrastructure for sourcecred-explorer. Setup npm, yarn, flow, jest using scaffolding from create-react-app. - used create-react-app to get the basic pieces - added support for flow via instructions in the create-react-app user guide - added flow-typed so that flow+jest will work well * Add some additional resources * Add index.css * Remove .DS_STORE per @wchargin's suggestion
2018-02-16 22:32:24 +00:00
},
"scripts": {
Set up prettier as an autoformatter (#18) Setup prettier precommit, with yarn prettify to reformat. - Prettier runs as a precommit hook and fails if code is improperly formatted. - Run `yarn prettify` to reformat the project.
2018-02-18 09:59:28 +00:00
"precommit": "npm run lint-staged",
"lint-staged": "lint-staged",
"prettify": "prettier --write '**/*.js'",
Setup travis CI testing (#58) Observe that it passed on this commit, but failed builds [#2: Break Tests][2], [#3: Break Flow][3], and [#4: Break Prettier][4]. Close #23 [2]: https://travis-ci.org/sourcecred/sourcecred/builds/348453195 [3]: https://travis-ci.org/sourcecred/sourcecred/builds/348454983 [4]: https://travis-ci.org/sourcecred/sourcecred/builds/348455387
2018-03-02 22:39:54 +00:00
"check-pretty": "prettier --list-different '**/*.js'",
Eject from create-react-app (#19) Summary: This commit was created by performing the following steps: - `rm -r node_modules` - `yarn install` - `yarn eject` - `rm -r node_modules` - `yarn install` on an environment wit Node v6.11.1. The final `rm -r node_modules && yarn install` ensures that the `yarn.lock` file is updated accordingly. Test Plan: `yarn test`, `yarn flow`, and `yarn start` all still do the right thing. wchargin-branch: eject
2018-02-17 21:28:47 +00:00
"start": "node scripts/start.js",
"build": "node scripts/build.js",
Configure Webpack for backend applications (#84) Summary: Running `yarn backend` will now bundle backend applications. They’ll be placed into the new `bin/` directory. This enables us to use ES6 modules with the standard syntax, Flow types, and all the other goodies that we’ve come to expect. A backend build takes about 2.5s on my laptop. Created by forking the prod configuration to a backend configuration and trimming it down appropriately. To test out the new changes, this commit changes `fetchGitHubRepo` and its driver to use the ES6 module system and Flow types, both of which are properly resolved. Test Plan: Run `yarn backend`. Then, you can directly run an entry point via ``` $ node bin/fetchAndPrintGitHubRepo.js sourcecred example-repo "${TOKEN}" ``` or invoke the standard test driver via ```shell $ GITHUB_TOKEN="${TOKEN}" src/backend/fetchGitHubRepoTest.sh ``` where `${TOKEN}` is your GitHub authentication token. wchargin-branch: webpack-backend
2018-03-19 05:43:23 +00:00
"backend": "node scripts/backend.js",
Eject from create-react-app (#19) Summary: This commit was created by performing the following steps: - `rm -r node_modules` - `yarn install` - `yarn eject` - `rm -r node_modules` - `yarn install` on an environment wit Node v6.11.1. The final `rm -r node_modules && yarn install` ensures that the `yarn.lock` file is updated accordingly. Test Plan: `yarn test`, `yarn flow`, and `yarn start` all still do the right thing. wchargin-branch: eject
2018-02-17 21:28:47 +00:00
"test": "node scripts/test.js --env=jsdom",
Implement a custom CI script (#189) Summary: This CI script accomplishes two tasks: 1. It speeds up our build by parallelizing where possible. 2. It opens the possibility for running Travis cron jobs. Currently, this script by default does the same amount of work as our current CI script. However, I’d like to move `yarn backend` into the list of basic actions: a backend build failure should fail CI. Note: this script is written to be executable directly by Node, so we can’t use Flow types with the standard syntax. Instead, we use the comment syntax: https://flow.org/en/docs/types/comments/ Test Plan: The following should pass with useful output: - `npm run travis` - `GITHUB_TOKEN="your_github_token" npm run travis -- --full` The following should fail with useful output: - `npm run travis -- --full` (fail) To test different failure modes, it can be helpful to add ```js {id: "doomed", cmd: ["false"], deps: []}, {id: "orphan", cmd: ["whoami"], deps: ["who", "are", "you"]}, ``` to the list of `basicTasks` in `travis.js`. To test performance: ```shell $ time node ./config/travis.js >/dev/null 2>/dev/null real 0m8.306s user 0m20.336s sys 0m1.364s $ time bash -c \ > 'npm run check-pretty && npm run lint && npm run flow && CI=1 npm run test' \ > >/dev/null 2>/dev/null real 0m12.427s user 0m13.752s sys 0m0.804s ``` A 50% savings is not bad at all—and the raw time saved should only improve from here on, as the individual steps start taking more time. wchargin-branch: custom-ci
2018-05-02 23:10:03 +00:00
"ci-test": "CI=1 npm run test",
Setup travis CI testing (#58) Observe that it passed on this commit, but failed builds [#2: Break Tests][2], [#3: Break Flow][3], and [#4: Break Prettier][4]. Close #23 [2]: https://travis-ci.org/sourcecred/sourcecred/builds/348453195 [3]: https://travis-ci.org/sourcecred/sourcecred/builds/348454983 [4]: https://travis-ci.org/sourcecred/sourcecred/builds/348455387
2018-03-02 22:39:54 +00:00
"flow": "flow",
Fix all lint errors, adding a lint CI step (#175) Test Plan: Run `yarn lint` and `yarn travis` and observe success. Add something that triggers a lint warning, like `const zzz = 3;`; re-run and observe failures. wchargin-branch: lint
2018-04-30 21:52:28 +00:00
"lint": "eslint src config --max-warnings 0",
Implement a custom CI script (#189) Summary: This CI script accomplishes two tasks: 1. It speeds up our build by parallelizing where possible. 2. It opens the possibility for running Travis cron jobs. Currently, this script by default does the same amount of work as our current CI script. However, I’d like to move `yarn backend` into the list of basic actions: a backend build failure should fail CI. Note: this script is written to be executable directly by Node, so we can’t use Flow types with the standard syntax. Instead, we use the comment syntax: https://flow.org/en/docs/types/comments/ Test Plan: The following should pass with useful output: - `npm run travis` - `GITHUB_TOKEN="your_github_token" npm run travis -- --full` The following should fail with useful output: - `npm run travis -- --full` (fail) To test different failure modes, it can be helpful to add ```js {id: "doomed", cmd: ["false"], deps: []}, {id: "orphan", cmd: ["whoami"], deps: ["who", "are", "you"]}, ``` to the list of `basicTasks` in `travis.js`. To test performance: ```shell $ time node ./config/travis.js >/dev/null 2>/dev/null real 0m8.306s user 0m20.336s sys 0m1.364s $ time bash -c \ > 'npm run check-pretty && npm run lint && npm run flow && CI=1 npm run test' \ > >/dev/null 2>/dev/null real 0m12.427s user 0m13.752s sys 0m0.804s ``` A 50% savings is not bad at all—and the raw time saved should only improve from here on, as the individual steps start taking more time. wchargin-branch: custom-ci
2018-05-02 23:10:03 +00:00
"travis": "node ./config/travis.js"
Eject from create-react-app (#19) Summary: This commit was created by performing the following steps: - `rm -r node_modules` - `yarn install` - `yarn eject` - `rm -r node_modules` - `yarn install` on an environment wit Node v6.11.1. The final `rm -r node_modules && yarn install` ensures that the `yarn.lock` file is updated accordingly. Test Plan: `yarn test`, `yarn flow`, and `yarn start` all still do the right thing. wchargin-branch: eject
2018-02-17 21:28:47 +00:00
},
Add "license" field to package.json (#52)
2018-03-02 19:31:12 +00:00
"license": "MIT",
Set up prettier as an autoformatter (#18) Setup prettier precommit, with yarn prettify to reformat. - Prettier runs as a precommit hook and fails if code is improperly formatted. - Run `yarn prettify` to reformat the project.
2018-02-18 09:59:28 +00:00
"lint-staged": {
"**/*.js": "prettier --list-different"
},
Eject from create-react-app (#19) Summary: This commit was created by performing the following steps: - `rm -r node_modules` - `yarn install` - `yarn eject` - `rm -r node_modules` - `yarn install` on an environment wit Node v6.11.1. The final `rm -r node_modules && yarn install` ensures that the `yarn.lock` file is updated accordingly. Test Plan: `yarn test`, `yarn flow`, and `yarn start` all still do the right thing. wchargin-branch: eject
2018-02-17 21:28:47 +00:00
"jest": {
"collectCoverageFrom": [
"src/**/*.{js,jsx,mjs}"
],
"setupFiles": [
"<rootDir>/config/polyfills.js"
],
"testMatch": [
"<rootDir>/src/**/__tests__/**/*.{js,jsx,mjs}",
"<rootDir>/src/**/?(*.)(spec|test).{js,jsx,mjs}"
],
"testEnvironment": "node",
"testURL": "http://localhost",
"transform": {
"^.+\\.(js|jsx|mjs)$": "<rootDir>/node_modules/babel-jest",
"^.+\\.css$": "<rootDir>/config/jest/cssTransform.js",
"^(?!.*\\.(js|jsx|mjs|css|json)$)": "<rootDir>/config/jest/fileTransform.js"
},
"transformIgnorePatterns": [
"[/\\\\]node_modules[/\\\\].+\\.(js|jsx|mjs)$"
],
"moduleNameMapper": {
"^react-native$": "react-native-web"
},
"moduleFileExtensions": [
"web.js",
"mjs",
"js",
"json",
"web.jsx",
"jsx",
"node"
]
},
"babel": {
Autogenerate PropTypes from Flow types (#20) Summary: Closes #17; see discussion there. This commit uses the `babel-plugin-flow-react-proptypes` package to automatically create PropType definitions from components that are typed with Flow. It simultaneously updates all of our existing components to be typed with Flow. As a result, we have both static and dynamic type checking. Test Plan: Note that `yarn test` and `yarn flow` report no errors, and that there are no prop validation errors at runtime with `yarn start`. Then, apply the following patch: ```diff diff --git a/explorer/src/UserExplorer.js b/explorer/src/UserExplorer.js index bb574cd..636a10d 100644 --- a/explorer/src/UserExplorer.js +++ b/explorer/src/UserExplorer.js @@ -18,7 +18,7 @@ export class UserExplorer extends Component<{ .sort((a,b) => b[1] - a[1]); const entries = sortedUserWeightTuples.map(authorWeight => { const [author, weight] = authorWeight; - return <UserEntry userId={author} weight={weight} key={author}/> + return <UserEntry userId={55} weight={weight} key={author}/> }); return <div className="user-explorer"> <h3> User Explorer </h3> ``` Note that `yarn test` fails (the `App.test.js` E2E rendering test), `yarn flow` fails, and there is a runtime prop validation error. wchargin-branch: autogenerate-proptypes
2018-02-17 21:30:16 +00:00
"plugins": [
"flow-react-proptypes"
],
Eject from create-react-app (#19) Summary: This commit was created by performing the following steps: - `rm -r node_modules` - `yarn install` - `yarn eject` - `rm -r node_modules` - `yarn install` on an environment wit Node v6.11.1. The final `rm -r node_modules && yarn install` ensures that the `yarn.lock` file is updated accordingly. Test Plan: `yarn test`, `yarn flow`, and `yarn start` all still do the right thing. wchargin-branch: eject
2018-02-17 21:28:47 +00:00
"presets": [
"react-app"
]
},
Autogenerate PropTypes from Flow types (#20) Summary: Closes #17; see discussion there. This commit uses the `babel-plugin-flow-react-proptypes` package to automatically create PropType definitions from components that are typed with Flow. It simultaneously updates all of our existing components to be typed with Flow. As a result, we have both static and dynamic type checking. Test Plan: Note that `yarn test` and `yarn flow` report no errors, and that there are no prop validation errors at runtime with `yarn start`. Then, apply the following patch: ```diff diff --git a/explorer/src/UserExplorer.js b/explorer/src/UserExplorer.js index bb574cd..636a10d 100644 --- a/explorer/src/UserExplorer.js +++ b/explorer/src/UserExplorer.js @@ -18,7 +18,7 @@ export class UserExplorer extends Component<{ .sort((a,b) => b[1] - a[1]); const entries = sortedUserWeightTuples.map(authorWeight => { const [author, weight] = authorWeight; - return <UserEntry userId={author} weight={weight} key={author}/> + return <UserEntry userId={55} weight={weight} key={author}/> }); return <div className="user-explorer"> <h3> User Explorer </h3> ``` Note that `yarn test` fails (the `App.test.js` E2E rendering test), `yarn flow` fails, and there is a runtime prop validation error. wchargin-branch: autogenerate-proptypes
2018-02-17 21:30:16 +00:00
"devDependencies": {
Use Enzyme to test ContributionList dynamics (#102) Summary: This is our first dynamic test of a React component! Enzyme looks pretty easy to use to me, for both snapshot tests and interaction simulation. In doing so, we catch a minor bug in the edge case where a contribution is not owned by any plugin (`colSpan`, not `colspan`). This edge case does not appear in the sample data, but it does appear in the test data, even prior to this commit. The previous renderer, `react-test-renderer`, appears not to surface this error. Furthermore, this bug did not cause any user-visible errors except a `console.error`. Test Plan: Inspect the snapshot file to make sure that it is reasonable. (The existing test case has its snapshot regenerated due to formatting differences between the two renderers.) To test that the browser error is fixed, render a contribution list on a GitHub graph but with an empty adapter set. One way to do this is to comment out line 7 of `standardAdapterSet.js`; alternately, you can use the React Dev Tools to select the `ContributionList` node, then run ```js $r.props.adapters.adapters = {}; $r.forceUpdate(); ``` Note subsequently that there is no console error and that the `<td>`s in question span three columns. wchargin-branch: contributionlist-dynamic-test
2018-03-22 00:35:17 +00:00
"babel-plugin-flow-react-proptypes": "^18.0.0",
"enzyme": "^3.3.0",
"enzyme-adapter-react-16": "^1.1.1",
Load Git repositories into memory (#139) Summary: In this newly added module, we load the structural state of a git repository into memory. We do not load into memory the contents of any blobs, so this is not enough information to perform any analysis requiring file diffing. However, it is sufficient to develop a notion of “this file was changed in this commit”, by simply diffing the trees. Test Plan: Unit tests added; `yarn test` suffices. Reading these snapshots is pretty easy, even though they’re filled with hashes: - First, read over the commit specifications on lines 69–83 of `loadRepository.test.js`, so you know what to expect. - In the snapshot file, keep handy the time-ordered list of commit SHAs at the bottom of the file, so that you know which commit SHA is which. - To verify that the large snapshot is correct: for each commit, read the corresponding tree object and make sure that the structure is correct. - To verify the small snapshot, just check that it’s the correct subset of the large snapshot. - If you want to verify that the SHA for a blob is correct, open a terminal and run `git hash-object -t blob --stdin`; then, enter the content of the blob and press `<C-d>`. The result is the blob SHA. To run a sanity-check on a large repository: apply the following patch: <details> <summary>Patch to print out statistics about loaded repository</summary> ```diff diff --git a/config/paths.js b/config/paths.js index d2f25fb..8fa2023 100644 --- a/config/paths.js +++ b/config/paths.js @@ -62,5 +62,6 @@ module.exports = { fetchAndPrintGithubRepo: resolveApp( "src/plugins/github/bin/fetchAndPrintGithubRepo.js" ), + loadRepository: resolveApp("src/plugins/git/loadRepository.js"), }, }; diff --git a/src/plugins/git/loadRepository.js b/src/plugins/git/loadRepository.js index a76b66c..9380941 100644 --- a/src/plugins/git/loadRepository.js +++ b/src/plugins/git/loadRepository.js @@ -106,3 +106,7 @@ function findTrees(git: GitDriver, rootTrees: Set<Hash>): Tree[] { } return result; } + +const result = loadRepository(...process.argv.slice(2)); +console.log("commits", result.commits.size); +console.log("trees", result.trees.size); ``` </details> Then, run `yarn backend` and put the following script in `test.sh`: <details> <summary>Contents for `test.sh`</summary> ```shell #!/bin/bash set -eu repo="$1" ref="$2" via_node() { node bin/loadRepository.js "${repo}" "${ref}" } via_git() ( cd "${repo}" printf 'commits ' git rev-list "${ref}" | wc -l printf 'trees ' git rev-list "${ref}" | while read -r commit; do git rev-parse "${commit}^{tree}" git ls-tree -rt "${commit}" \ | grep ' tree ' \ | cut -f 1 | cut -d ' ' -f 3 done | sort | uniq | wc -l ) echo printf 'Running directly via git...\n' time a="$(via_git)" echo printf 'Running Node script...\n' time b="$(via_node)" diff -u <(cat <<<"${a}") <(cat <<<"${b}") ``` </details> Finally, run `./test.sh /path/to/some/repo origin/master`, and verify that it exits successfully (zero diff). Here are some timing results on SourceCred and TensorBoard: - SourceCred: 0.973s via Node, 0.327s via git. - TensorBoard: 30.836s via Node, 6.895s via git. For TensorFlow, running via git takes 7m33.995s. Running via Node fails with an out-of-memory error after 39 minutes, with 10GB RAM and 4GB swap. See details below. <details> <summary> Full timing details, commit SHAs, and OOM error message </summary> ``` + ./test.sh /home/wchargin/git/sourcecred 01634aabcca3756b38e13aaf2f451cfbda2ad5ea Running directly via git... real 0m0.327s user 0m0.016s sys 0m0.052s Running Node script... real 0m0.973s user 0m0.268s sys 0m0.176s + ./test.sh /home/wchargin/git/tensorboard 7aa1ab9d60671056b8811b7099eec08650f2e4fd Running directly via git... real 0m6.895s user 0m0.600s sys 0m0.832s Running Node script... real 0m30.836s user 0m3.216s sys 0m10.588s + ./test.sh /home/wchargin/git/tensorflow 968addadfd4e4f5688eedc31f92a9066329ff6a7 Running directly via git... real 7m33.995s user 5m21.124s sys 1m5.476s Running Node script... FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory 1: node::Abort() [node] 2: 0x121a2cc [node] 3: v8::Utils::ReportOOMFailure(char const*, bool) [node] 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node] 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [node] 6: v8::internal::DeoptimizationInputData::New(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [node] 7: v8::internal::compiler::CodeGenerator::PopulateDeoptimizationData(v8::internal::Handle<v8::internal::Code>) [node] 8: v8::internal::compiler::CodeGenerator::FinalizeCode() [node] 9: v8::internal::compiler::PipelineImpl::FinalizeCode() [node] 10: v8::internal::compiler::PipelineCompilationJob::FinalizeJobImpl() [node] 11: v8::internal::Compiler::FinalizeCompilationJob(v8::internal::CompilationJob*) [node] 12: v8::internal::OptimizingCompileDispatcher::InstallOptimizedFunctions() [node] 13: v8::internal::Runtime_TryInstallOptimizedCode(int, v8::internal::Object**, v8::internal::Isolate*) [node] 14: 0x12dc8b08463d ``` </details> wchargin-branch: load-git-repositories # Please enter the commit message for your changes. Lines starting # with '#' will be kept; you may remove them yourself if you want to. # An empty message aborts the commit. # # Date: Mon Apr 23 23:02:14 2018 -0700 # # HEAD detached at origin/wchargin-load-git-repositories # Changes to be committed: # modified: package.json # new file: src/plugins/git/__snapshots__/loadRepository.test.js.snap # new file: src/plugins/git/loadRepository.js # new file: src/plugins/git/loadRepository.test.js # # Untracked files: # out # runtests.sh # src/plugins/artifact/editor/ArtifactSetInput.js # src/plugins/git/repository.js # test.sh # todo #
2018-04-24 20:57:10 +00:00
"enzyme-to-json": "^3.3.3",
"mkdirp": "^0.5.1",
Exclude node modules from backend build (#211) Setup following directions from [webpack-node-externals] [webpack-node-externals]: https://www.npmjs.com/package/webpack-node-externals This unblocks #210. Test plan: `yarn backend` still succeeds, and the binary scripts still work. The resultant binaries are much smaller, as seen below (note build time is the same). before: ``` ❯ yarn backend yarn run v1.5.1 $ node scripts/backend.js Building backend applications... Compiled successfully. File sizes after gzip: 231.37 KB bin/printCombinedGraph.js 199.5 KB bin/fetchAndPrintGithubRepo.js 46.41 KB bin/cloneAndPrintGitGraph.js 21.48 KB bin/createExampleRepo.js 17.71 KB bin/loadAndPrintGitRepository.js Build completed; results in 'bin'. Done in 4.46s. ``` after: ``` ❯ yarn backend yarn run v1.5.1 $ node scripts/backend.js Building backend applications... Compiled successfully. File sizes after gzip: 27.78 KB bin/printCombinedGraph.js 12.73 KB bin/cloneAndPrintGitGraph.js 12.41 KB bin/fetchAndPrintGithubRepo.js 6.03 KB bin/loadAndPrintGitRepository.js 5.52 KB bin/createExampleRepo.js Build completed; results in 'bin'. Done in 4.28s. ```
2018-05-04 23:31:39 +00:00
"tmp": "^0.0.33",
"webpack-node-externals": "^1.7.2"
Frontend setup (#13) * Initial infrastructure for sourcecred-explorer. Setup npm, yarn, flow, jest using scaffolding from create-react-app. - used create-react-app to get the basic pieces - added support for flow via instructions in the create-react-app user guide - added flow-typed so that flow+jest will work well * Add some additional resources * Add index.css * Remove .DS_STORE per @wchargin's suggestion
2018-02-16 22:32:24 +00:00
}
}