2018-02-16 22:32:24 +00:00
|
|
|
{
|
2018-05-05 07:04:42 +00:00
|
|
|
"name": "sourcecred",
|
2018-02-16 22:32:24 +00:00
|
|
|
"version": "0.1.0",
|
|
|
|
|
"private": true,
|
|
|
|
|
"dependencies": {
|
2018-05-05 02:28:37 +00:00
|
|
|
"@oclif/command": "^1.4.20",
|
|
|
|
|
"@oclif/config": "^1.6.17",
|
|
|
|
|
"@oclif/plugin-help": "^1.2.10",
|
2018-05-09 17:22:48 +00:00
|
|
|
"@tensorflow/tfjs-core": "^0.9.1",
|
2018-03-19 22:31:08 +00:00
|
|
|
"aphrodite": "^2.1.0",
|
2018-02-17 21:28:47 +00:00
|
|
|
"autoprefixer": "7.1.6",
|
|
|
|
|
"babel-core": "6.26.0",
|
|
|
|
|
"babel-eslint": "7.2.3",
|
|
|
|
|
"babel-jest": "20.0.3",
|
|
|
|
|
"babel-loader": "7.1.2",
|
Target latest Node for backend applications (#213)
Summary:
Consequently, Babel won’t transform classes to their roughly equivalent
ES5 counterparts, etc.
Test Plan:
Create `src/classy.js` with `class X {}; console.log(X);`. Then, add a
build target for `classy: resolveApp("src/classy.js"),` in `paths.js`.
Use `yarn backend` and inspect the contents of `bin/classy.js`; in
particular, look at the definition of `X` (whatever the argument to
`console.log` is). Before this commit, the result will be a big
complicated mess. After this commit, it will be `class X {}`.
Note also that `yarn travis --full` passes, indicating that the two
manual tests, which call out to the utilities in `bin/`, still work.
wchargin-branch: target-node
2018-05-05 02:22:39 +00:00
|
|
|
"babel-plugin-transform-es2015-for-of": "^6.23.0",
|
2018-02-17 21:28:47 +00:00
|
|
|
"babel-preset-react-app": "^3.1.1",
|
|
|
|
|
"babel-runtime": "6.26.0",
|
|
|
|
|
"case-sensitive-paths-webpack-plugin": "2.1.1",
|
|
|
|
|
"chalk": "1.1.3",
|
|
|
|
|
"css-loader": "0.28.7",
|
|
|
|
|
"dotenv": "4.0.0",
|
|
|
|
|
"dotenv-expand": "4.0.1",
|
|
|
|
|
"eslint": "4.10.0",
|
|
|
|
|
"eslint-config-react-app": "^2.1.0",
|
|
|
|
|
"eslint-loader": "1.9.0",
|
|
|
|
|
"eslint-plugin-flowtype": "2.39.1",
|
|
|
|
|
"eslint-plugin-import": "2.8.0",
|
|
|
|
|
"eslint-plugin-jsx-a11y": "5.1.1",
|
|
|
|
|
"eslint-plugin-react": "7.4.0",
|
2018-05-08 03:05:52 +00:00
|
|
|
"express": "^4.16.3",
|
2018-02-17 21:28:47 +00:00
|
|
|
"extract-text-webpack-plugin": "3.0.2",
|
|
|
|
|
"file-loader": "1.1.5",
|
2018-02-16 22:32:24 +00:00
|
|
|
"flow-bin": "^0.65.0",
|
2018-02-17 21:28:47 +00:00
|
|
|
"fs-extra": "3.0.1",
|
|
|
|
|
"html-webpack-plugin": "2.29.0",
|
2018-02-18 09:59:28 +00:00
|
|
|
"husky": "^0.14.3",
|
2018-03-20 00:01:41 +00:00
|
|
|
"isomorphic-fetch": "^2.2.1",
|
2018-02-17 21:28:47 +00:00
|
|
|
"jest": "20.0.4",
|
2018-03-15 21:56:25 +00:00
|
|
|
"json-stable-stringify": "^1.0.1",
|
2018-02-18 09:59:28 +00:00
|
|
|
"lint-staged": "^6.1.1",
|
2018-04-30 22:23:37 +00:00
|
|
|
"lodash.clonedeep": "^4.5.0",
|
2018-03-03 05:13:30 +00:00
|
|
|
"lodash.isequal": "^4.5.0",
|
2018-04-27 21:29:49 +00:00
|
|
|
"lodash.sortby": "^4.7.0",
|
2018-02-17 21:28:47 +00:00
|
|
|
"object-assign": "4.1.1",
|
|
|
|
|
"postcss-flexbugs-fixes": "3.2.0",
|
|
|
|
|
"postcss-loader": "2.0.8",
|
2018-02-18 09:59:28 +00:00
|
|
|
"prettier": "^1.10.2",
|
2018-02-17 21:28:47 +00:00
|
|
|
"promise": "8.0.1",
|
|
|
|
|
"raf": "3.4.0",
|
2018-02-16 22:32:24 +00:00
|
|
|
"react": "^16.2.0",
|
2018-02-17 21:28:47 +00:00
|
|
|
"react-dev-utils": "^5.0.0",
|
2018-02-16 22:32:24 +00:00
|
|
|
"react-dom": "^16.2.0",
|
2018-05-08 19:02:17 +00:00
|
|
|
"react-router-dom": "^4.2.2",
|
2018-02-17 21:28:47 +00:00
|
|
|
"style-loader": "0.19.0",
|
|
|
|
|
"sw-precache-webpack-plugin": "0.11.4",
|
|
|
|
|
"url-loader": "0.6.2",
|
|
|
|
|
"webpack": "3.8.1",
|
|
|
|
|
"webpack-dev-server": "2.9.4",
|
|
|
|
|
"webpack-manifest-plugin": "1.3.2",
|
|
|
|
|
"whatwg-fetch": "2.0.3"
|
2018-02-16 22:32:24 +00:00
|
|
|
},
|
|
|
|
|
"scripts": {
|
2018-02-18 09:59:28 +00:00
|
|
|
"precommit": "npm run lint-staged",
|
|
|
|
|
"lint-staged": "lint-staged",
|
|
|
|
|
"prettify": "prettier --write '**/*.js'",
|
2018-03-02 22:39:54 +00:00
|
|
|
"check-pretty": "prettier --list-different '**/*.js'",
|
2018-02-17 21:28:47 +00:00
|
|
|
"start": "node scripts/start.js",
|
|
|
|
|
"build": "node scripts/build.js",
|
2018-03-19 05:43:23 +00:00
|
|
|
"backend": "node scripts/backend.js",
|
2018-02-17 21:28:47 +00:00
|
|
|
"test": "node scripts/test.js --env=jsdom",
|
Implement a custom CI script (#189)
Summary:
This CI script accomplishes two tasks:
1. It speeds up our build by parallelizing where possible.
2. It opens the possibility for running Travis cron jobs.
Currently, this script by default does the same amount of work as our
current CI script. However, I’d like to move `yarn backend` into the
list of basic actions: a backend build failure should fail CI.
Note: this script is written to be executable directly by Node, so we
can’t use Flow types with the standard syntax. Instead, we use the
comment syntax: https://flow.org/en/docs/types/comments/
Test Plan:
The following should pass with useful output:
- `npm run travis`
- `GITHUB_TOKEN="your_github_token" npm run travis -- --full`
The following should fail with useful output:
- `npm run travis -- --full` (fail)
To test different failure modes, it can be helpful to add
```js
{id: "doomed", cmd: ["false"], deps: []},
{id: "orphan", cmd: ["whoami"], deps: ["who", "are", "you"]},
```
to the list of `basicTasks` in `travis.js`.
To test performance:
```shell
$ time node ./config/travis.js >/dev/null 2>/dev/null
real 0m8.306s
user 0m20.336s
sys 0m1.364s
$ time bash -c \
> 'npm run check-pretty && npm run lint && npm run flow && CI=1 npm run test' \
> >/dev/null 2>/dev/null
real 0m12.427s
user 0m13.752s
sys 0m0.804s
```
A 50% savings is not bad at all—and the raw time saved should only
improve from here on, as the individual steps start taking more time.
wchargin-branch: custom-ci
2018-05-02 23:10:03 +00:00
|
|
|
"ci-test": "CI=1 npm run test",
|
2018-03-02 22:39:54 +00:00
|
|
|
"flow": "flow",
|
2018-04-30 21:52:28 +00:00
|
|
|
"lint": "eslint src config --max-warnings 0",
|
Implement a custom CI script (#189)
Summary:
This CI script accomplishes two tasks:
1. It speeds up our build by parallelizing where possible.
2. It opens the possibility for running Travis cron jobs.
Currently, this script by default does the same amount of work as our
current CI script. However, I’d like to move `yarn backend` into the
list of basic actions: a backend build failure should fail CI.
Note: this script is written to be executable directly by Node, so we
can’t use Flow types with the standard syntax. Instead, we use the
comment syntax: https://flow.org/en/docs/types/comments/
Test Plan:
The following should pass with useful output:
- `npm run travis`
- `GITHUB_TOKEN="your_github_token" npm run travis -- --full`
The following should fail with useful output:
- `npm run travis -- --full` (fail)
To test different failure modes, it can be helpful to add
```js
{id: "doomed", cmd: ["false"], deps: []},
{id: "orphan", cmd: ["whoami"], deps: ["who", "are", "you"]},
```
to the list of `basicTasks` in `travis.js`.
To test performance:
```shell
$ time node ./config/travis.js >/dev/null 2>/dev/null
real 0m8.306s
user 0m20.336s
sys 0m1.364s
$ time bash -c \
> 'npm run check-pretty && npm run lint && npm run flow && CI=1 npm run test' \
> >/dev/null 2>/dev/null
real 0m12.427s
user 0m13.752s
sys 0m0.804s
```
A 50% savings is not bad at all—and the raw time saved should only
improve from here on, as the individual steps start taking more time.
wchargin-branch: custom-ci
2018-05-02 23:10:03 +00:00
|
|
|
"travis": "node ./config/travis.js"
|
2018-02-17 21:28:47 +00:00
|
|
|
},
|
2018-03-02 19:31:12 +00:00
|
|
|
"license": "MIT",
|
2018-02-18 09:59:28 +00:00
|
|
|
"lint-staged": {
|
|
|
|
|
"**/*.js": "prettier --list-different"
|
|
|
|
|
},
|
2018-02-17 21:28:47 +00:00
|
|
|
"jest": {
|
|
|
|
|
"collectCoverageFrom": [
|
|
|
|
|
"src/**/*.{js,jsx,mjs}"
|
|
|
|
|
],
|
|
|
|
|
"setupFiles": [
|
|
|
|
|
"<rootDir>/config/polyfills.js"
|
|
|
|
|
],
|
|
|
|
|
"testMatch": [
|
|
|
|
|
"<rootDir>/src/**/__tests__/**/*.{js,jsx,mjs}",
|
|
|
|
|
"<rootDir>/src/**/?(*.)(spec|test).{js,jsx,mjs}"
|
|
|
|
|
],
|
|
|
|
|
"testEnvironment": "node",
|
|
|
|
|
"testURL": "http://localhost",
|
|
|
|
|
"transform": {
|
|
|
|
|
"^.+\\.(js|jsx|mjs)$": "<rootDir>/node_modules/babel-jest",
|
|
|
|
|
"^.+\\.css$": "<rootDir>/config/jest/cssTransform.js",
|
|
|
|
|
"^(?!.*\\.(js|jsx|mjs|css|json)$)": "<rootDir>/config/jest/fileTransform.js"
|
|
|
|
|
},
|
|
|
|
|
"transformIgnorePatterns": [
|
|
|
|
|
"[/\\\\]node_modules[/\\\\].+\\.(js|jsx|mjs)$"
|
|
|
|
|
],
|
|
|
|
|
"moduleNameMapper": {
|
|
|
|
|
"^react-native$": "react-native-web"
|
|
|
|
|
},
|
|
|
|
|
"moduleFileExtensions": [
|
|
|
|
|
"web.js",
|
|
|
|
|
"mjs",
|
|
|
|
|
"js",
|
|
|
|
|
"json",
|
|
|
|
|
"web.jsx",
|
|
|
|
|
"jsx",
|
|
|
|
|
"node"
|
|
|
|
|
]
|
|
|
|
|
},
|
|
|
|
|
"babel": {
|
2018-02-17 21:30:16 +00:00
|
|
|
"plugins": [
|
|
|
|
|
"flow-react-proptypes"
|
|
|
|
|
],
|
2018-02-17 21:28:47 +00:00
|
|
|
"presets": [
|
2018-05-05 02:19:45 +00:00
|
|
|
"./config/babel"
|
2018-02-17 21:28:47 +00:00
|
|
|
]
|
|
|
|
|
},
|
2018-02-17 21:30:16 +00:00
|
|
|
"devDependencies": {
|
Use Enzyme to test ContributionList dynamics (#102)
Summary:
This is our first dynamic test of a React component! Enzyme looks pretty
easy to use to me, for both snapshot tests and interaction simulation.
In doing so, we catch a minor bug in the edge case where a contribution
is not owned by any plugin (`colSpan`, not `colspan`). This edge case
does not appear in the sample data, but it does appear in the test data,
even prior to this commit. The previous renderer, `react-test-renderer`,
appears not to surface this error. Furthermore, this bug did not cause
any user-visible errors except a `console.error`.
Test Plan:
Inspect the snapshot file to make sure that it is reasonable. (The
existing test case has its snapshot regenerated due to formatting
differences between the two renderers.)
To test that the browser error is fixed, render a contribution list on a
GitHub graph but with an empty adapter set. One way to do this is to comment out line 7 of
`standardAdapterSet.js`; alternately, you can use the React Dev Tools to
select the `ContributionList` node, then run
```js
$r.props.adapters.adapters = {};
$r.forceUpdate();
```
Note subsequently that there is no console error and that the `<td>`s in
question span three columns.
wchargin-branch: contributionlist-dynamic-test
2018-03-22 00:35:17 +00:00
|
|
|
"babel-plugin-flow-react-proptypes": "^18.0.0",
|
|
|
|
|
"enzyme": "^3.3.0",
|
|
|
|
|
"enzyme-adapter-react-16": "^1.1.1",
|
Load Git repositories into memory (#139)
Summary:
In this newly added module, we load the structural state of a git
repository into memory. We do not load into memory the contents of any
blobs, so this is not enough information to perform any analysis
requiring file diffing. However, it is sufficient to develop a notion of
“this file was changed in this commit”, by simply diffing the trees.
Test Plan:
Unit tests added; `yarn test` suffices. Reading these snapshots is
pretty easy, even though they’re filled with hashes:
- First, read over the commit specifications on lines 69–83 of
`loadRepository.test.js`, so you know what to expect.
- In the snapshot file, keep handy the time-ordered list of commit
SHAs at the bottom of the file, so that you know which commit SHA is
which.
- To verify that the large snapshot is correct: for each commit, read
the corresponding tree object and make sure that the structure is
correct.
- To verify the small snapshot, just check that it’s the correct
subset of the large snapshot.
- If you want to verify that the SHA for a blob is correct, open a
terminal and run `git hash-object -t blob --stdin`; then, enter the
content of the blob and press `<C-d>`. The result is the blob SHA.
To run a sanity-check on a large repository: apply the following patch:
<details>
<summary>Patch to print out statistics about loaded repository</summary>
```diff
diff --git a/config/paths.js b/config/paths.js
index d2f25fb..8fa2023 100644
--- a/config/paths.js
+++ b/config/paths.js
@@ -62,5 +62,6 @@ module.exports = {
fetchAndPrintGithubRepo: resolveApp(
"src/plugins/github/bin/fetchAndPrintGithubRepo.js"
),
+ loadRepository: resolveApp("src/plugins/git/loadRepository.js"),
},
};
diff --git a/src/plugins/git/loadRepository.js b/src/plugins/git/loadRepository.js
index a76b66c..9380941 100644
--- a/src/plugins/git/loadRepository.js
+++ b/src/plugins/git/loadRepository.js
@@ -106,3 +106,7 @@ function findTrees(git: GitDriver, rootTrees: Set<Hash>): Tree[] {
}
return result;
}
+
+const result = loadRepository(...process.argv.slice(2));
+console.log("commits", result.commits.size);
+console.log("trees", result.trees.size);
```
</details>
Then, run `yarn backend` and put the following script in `test.sh`:
<details>
<summary>Contents for `test.sh`</summary>
```shell
#!/bin/bash
set -eu
repo="$1"
ref="$2"
via_node() {
node bin/loadRepository.js "${repo}" "${ref}"
}
via_git() (
cd "${repo}"
printf 'commits '
git rev-list "${ref}" | wc -l
printf 'trees '
git rev-list "${ref}" |
while read -r commit; do
git rev-parse "${commit}^{tree}"
git ls-tree -rt "${commit}" \
| grep ' tree ' \
| cut -f 1 | cut -d ' ' -f 3
done | sort | uniq | wc -l
)
echo
printf 'Running directly via git...\n'
time a="$(via_git)"
echo
printf 'Running Node script...\n'
time b="$(via_node)"
diff -u <(cat <<<"${a}") <(cat <<<"${b}")
```
</details>
Finally, run `./test.sh /path/to/some/repo origin/master`, and verify
that it exits successfully (zero diff). Here are some timing results on
SourceCred and TensorBoard:
- SourceCred: 0.973s via Node, 0.327s via git.
- TensorBoard: 30.836s via Node, 6.895s via git.
For TensorFlow, running via git takes 7m33.995s. Running via Node fails
with an out-of-memory error after 39 minutes, with 10GB RAM and 4GB
swap. See details below.
<details>
<summary>
Full timing details, commit SHAs, and OOM error message
</summary>
```
+ ./test.sh /home/wchargin/git/sourcecred 01634aabcca3756b38e13aaf2f451cfbda2ad5ea
Running directly via git...
real 0m0.327s
user 0m0.016s
sys 0m0.052s
Running Node script...
real 0m0.973s
user 0m0.268s
sys 0m0.176s
+ ./test.sh /home/wchargin/git/tensorboard 7aa1ab9d60671056b8811b7099eec08650f2e4fd
Running directly via git...
real 0m6.895s
user 0m0.600s
sys 0m0.832s
Running Node script...
real 0m30.836s
user 0m3.216s
sys 0m10.588s
+ ./test.sh /home/wchargin/git/tensorflow 968addadfd4e4f5688eedc31f92a9066329ff6a7
Running directly via git...
real 7m33.995s
user 5m21.124s
sys 1m5.476s
Running Node script...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [node]
2: 0x121a2cc [node]
3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [node]
6: v8::internal::DeoptimizationInputData::New(v8::internal::Isolate*, int, v8::internal::PretenureFlag) [node]
7: v8::internal::compiler::CodeGenerator::PopulateDeoptimizationData(v8::internal::Handle<v8::internal::Code>) [node]
8: v8::internal::compiler::CodeGenerator::FinalizeCode() [node]
9: v8::internal::compiler::PipelineImpl::FinalizeCode() [node]
10: v8::internal::compiler::PipelineCompilationJob::FinalizeJobImpl() [node]
11: v8::internal::Compiler::FinalizeCompilationJob(v8::internal::CompilationJob*) [node]
12: v8::internal::OptimizingCompileDispatcher::InstallOptimizedFunctions() [node]
13: v8::internal::Runtime_TryInstallOptimizedCode(int, v8::internal::Object**, v8::internal::Isolate*) [node]
14: 0x12dc8b08463d
```
</details>
wchargin-branch: load-git-repositories
# Please enter the commit message for your changes. Lines starting
# with '#' will be kept; you may remove them yourself if you want to.
# An empty message aborts the commit.
#
# Date: Mon Apr 23 23:02:14 2018 -0700
#
# HEAD detached at origin/wchargin-load-git-repositories
# Changes to be committed:
# modified: package.json
# new file: src/plugins/git/__snapshots__/loadRepository.test.js.snap
# new file: src/plugins/git/loadRepository.js
# new file: src/plugins/git/loadRepository.test.js
#
# Untracked files:
# out
# runtests.sh
# src/plugins/artifact/editor/ArtifactSetInput.js
# src/plugins/git/repository.js
# test.sh
# todo
#
2018-04-24 20:57:10 +00:00
|
|
|
"enzyme-to-json": "^3.3.3",
|
|
|
|
|
"mkdirp": "^0.5.1",
|
2018-05-04 23:31:39 +00:00
|
|
|
"tmp": "^0.0.33",
|
|
|
|
|
"webpack-node-externals": "^1.7.2"
|
2018-05-05 02:28:37 +00:00
|
|
|
},
|
|
|
|
|
"oclif": {
|
|
|
|
|
"commands": "./bin/commands",
|
|
|
|
|
"bin": "sourcecred",
|
|
|
|
|
"plugins": [
|
|
|
|
|
"@oclif/plugin-help"
|
|
|
|
|
]
|
2018-02-16 22:32:24 +00:00
|
|
|
}
|
|
|
|
|
}
|
