Previously, if any individual plugin failed to load, the load command
would print an error, but return exit code 0.
As of this commit, the load command returns 1 if any plugin(s) fail to
load, and prints the name(s) of the offending plugins.
Test plan: CLI doesn't have automated testing, so induce failure in an
instance by e.g. setting the SOURCECRED_GITHUB_TOKEN="" in an instance
which uses the GitHub plugin.
Observe that the full error message is still printed to console, and
that the command returns exit code 1, and that if more than one plugins
fail that the message is printed nicely showing all the failed plugins.
Also observe that load's behavior on successful loads is unchanged.
Previously this recheck had a usecase: give priority to categories
that are used as Initiative source of truth. This model was abandoned
and so the priority doesn't make much sense either.
Instead use cache clearing to guarantee recent data when you need it.
This adds a user type to the ledger. Each user is tracked by a UUID, and
has a human-specified and renameable username. Each user also
corresponds to a node in the graph, and has a list of aliase addresses
that should be merged into it using `graph.contractNodes`.
It's important that we track this in the ledger because the ledger needs
to reconcile when aliases are added or removed.
Test plan: Unit tests added. Quite a simple module.
This changes the Discord plugin so that, by default, it adds every
Reaction to the graph, so they will all mint Cred.
If custom reaction weights are added, they will act as multiplicative
modifiers on the Cred minting.
Test plan: I've tested this in production, by deploying SourceCred's own
Cred from this patch. The Discord plugin has no unit tests.
This is the dual to #1858; it allows the cli/graph command to choose to
load only specific plugins.
This is helpful if, for example, you load an instance where you don't
have the credentials to re-load one of the plugins, but want to update
the other plugins. Then you can load and graph the plugins for which you
do have data, but leave the graph for the inaccessible plugin
un-touched.
Test plan: Manual testing, as with #1858 and for similar rationale.
Summary:
A new utility module defines an opaque UUID type. Clients can generate
random UUIDs and convert them to and from strings.
We roll our own module here rather than using the `uuid` NPM module
because that module only exposes the 32-character hyphen-separated
string form of UUID, which takes up nearly 50% more space. The usual
approach these days is to use fixed-length base64 strings without
padding. For instance, [YouTube video IDs are of this form][yt].
This module takes care not to generate IDs that might confuse
command-line argument parsers, and tries not to generate IDs that might
contain English curse words. The implementation uses isomorphic
libraries for RNG and base64 conversion because the relevant Node and
browser APIs are not aligned.
[yt]: https://webapps.stackexchange.com/a/101153
Test Plan:
Unit tests included, with full coverage.
wchargin-branch: util-uuid
Now, if a `weights.json` file is present in the `config/` folder, its
weights will be loaded and used in computing the credResult.
Test plan: We don't yet have proper unit testing for the CLI, so I added
weights to the snapshot. The snapshot weights are pretty silly; 32x for
Discourse posts and 32x for GitHub bots. Load the UI via `yarn start`,
and observe that these weights were persisted in the CredResult.
If the instance wants to specify custom TimelineCredParams, e.g. a
custom alpha value, they may now write a partial TimelineCredParams file
as `config/params.json`.
Test plan: We don't yet have proper unit testing for the CLI, so I added
this case to the test-instance. Using the new frontend, I've verified
that the custom alpha value is correctly reflected in the output data
(use `yarn start`, open weight config, and view alpha).
Prior to this commit, `yarn start` would fail if a development instance
path is not provided. However, now that we package a test instance
inside of our snapshots, we can use that as the default dev instance
instead.
Test plan: `yarn start` with no arguments or environment variables now
loads the test instance, both when run from repository root, and when
cd'd into a subdirectory.
This commit adds a major piece of missing functionality to the new UI:
the ability to view weights, change parameters, and then re-compute
Cred. The code has mostly been copied from the v2 UI.
Test plan: Manual testing around changing weights and params, and
recomputing Cred.
This will be needed for letting the UI recompute Cred scores with new
weights / params (which is a vital feature!).
Test plan: Not tested :'( will sanity check via frontend integration.
This changes the instance system structure so that all the
plugin-specific configs are organized under
`config/plugins/$OWNER/$NAME` instead of `config/$OWNER/$NAME`. I think
this is a somewhat clearer structure; since `config/` will hold other
files (e.g. `weights.json` or `params.json`), I think it's cleaner if
everything plugin-specific is under it's own clearly scoped folder. This
avoids potential confusion if we ever have plugins with very
generic-named organizations, e.g. "config".
Test plan: The test instance has been updated, and the sharness test
loading still works.
Summary:
The `sourcecred site` command now sets up a static site for a cred
instance, which may be deployed to (e.g.) GitHub Pages or IPFS.
This implementation sets up a directory structure where the site is
contained in a `site/` subdirectory whose top-level entries are also
symbolically linked from the root of the instance. Thus:
```
my-instance/
sourcecred.json
config/
output/
site/
index.html
favicon.png
static/
static/js/...
index.html -> site/index.html
favicon.png -> site/favicon.png
static -> site/static
```
The purpose of this is to enable the instance to be served directly as a
GitHub Pages site without the user having to navigate to a URL that
includes a `site/` path component.
It’s not a perfect solution. The top level of the instance is cluttered.
If we have any web pages other than the root directory, then they’ll
appear at top level, too. If we change the structure, we’ll need to
teach `sourcecred site` to clean up vestiges of old structures. We also
won’t be able to have any pages called `config` or `output` or `cache`
due to namespace collision. But we think that it’s at least a reasonable
stopgap, and doesn’t incur much conceptual overhead.
This isn’t tested or officially supported on Windows.
Paired with @decentralion.
Test Plan:
Run `yarn build` to build the frontend and backend, then `cd` into a
SourceCred instance and run `node "${OLDPWD}/bin/sourcecred.js site`.
Note that a `site/` directory is created, and when the static site is
served it works from either `site/` or the instance root. Note that this
all works when actually deployed to GitHub Pages as well.
wchargin-branch: cli-site
Summary:
Follow-up to #1903. In particular, this should fix the Docker build.
Test Plan:
Running `git grep 'yarn backend'` no longer turns up examples.
wchargin-branch: build-update-yarn-backend-refs
Summary:
Previously, we used `backend` to build the backend and `build` to build
the frontend, which was a bit confusing. Now, `build:backend` and
`build:frontend` build the respective components, and `build` builds
both in parallel.
Test Plan:
Run `rm -r bin build` and then `yarn build`, and note that the `bin` and
`build` directories are recreated.
wchargin-branch: build-unified-cmd
This commit removes the `core/project.js` type. It was a sort of
"kitchen sink" config file where we just threw any random thing. This
was not a good approach to config, so now we let each plugin provide
it's own config format, and we no longer need (or want) the project
type.
I removed a test from the compatIO module, although we'll probably
remove that too ere long.
Test plan: `yarn test`.
This adds a simple sharness snapshot test for the new instance system,
in the model of the old [test_load_example_github.t][old].
I've setup a very simple test instance with the GitHub and Discourse
plugins, and we verify that the output generated by running `load`,
`graph`, and `score` in succession is stable. (Cache is not
persisted.) This is a nice sanity check to verify that nothing ever gets
totally broken; we'll still want to add unit testing for more specific
features and edge cases.
Test plan:
`yarn test --full` passes.
If you `rm -rf sharness/test-instance/output`, then `yarn test --full`
fails.
If you then run `./scripts/update_snapshots.sh`, then the output
directory will be restored; afterwards `yarn test --full` passes again.
To verify that the snapshots are valid, you can test them with the
frontend:
`yarn start --instance sharness/__snapshots__/test-instance`
If you are actually debugging this script, rather than using
`yarn test --full` you'll want to invoke
`(cd sharness; ./load_test_instance.t -l -v)`
[old]: https://github.com/sourcecred/sourcecred/blob/v0.6.0/sharness/test_load_example_github.t
This commit removes old logic related to loading projects in the context
of "data directories". In particular, almost all of the `src/backend`
directory is removed (and I suspect the remaining files should come out
too; definitely the compat_io module is superseded by the new Parser
based approach). The old style loaders are also removed, as they are
rendered redundant by the new CliPlugin interface.
Test plan: `yarn flow` passes, and none of the load-bearing (ha ha)
parts of the current setup are impacted, i.e. there are no changes to
the CLI. Thus, we may be confident that everything is still working.
This commit removes (both) legacy explorer UIs, since they have been
replaced as of #1894.
Test plan: No code that is currently in use has been touched, `yarn test`
passes. Clean deletes are easy.
This commit integrates a new version of the cred explorer, which is
compatible with the new instance-based cred. It combines some of the
benefits of the v1 explorer (shows cred source decomposition) and the v2
explorer (shows cred-over-time for users). It's still very basic, and it
lacks an important feature of both v1 and v2 in that it doesn't allow
recomputing cred scores with new weights.
Nonetheless, it's a big improvement over having no functional ui at all!
Test plan: Manually test with a real instance, e.g. via
`yarn start --instance $PATH`
This adds `inflows` and `outflows` methods to the CredView.
The CredView module is known-temporary because it is built on top of
TimelineCred, which is going to be replaced by CredRank. Also, it's
extremely hard to write proper tests for, because TimelineCred is rotten
and doesn't have clearly defined semantics on small, easily tested
graphs.
Therefore, I haven't written unit tests for this; I have already used it
in downstream UIs and it does show consistent and intuitive results, so
I am reasonably confident that it works.
I expect this will stay in the codebase for 1-3 months before being
replaced by new (and thoroughly tested) views built over CredRank's
output data.
Test plan: Downstream usage in commits, e.g. see UI in:
9b9ec4bf1d
Summary:
By making the `Compatible<T>` type definition transparent, it becomes
valid to pass a compat value to the `parse` or `parseOrThrow` method of
a `Combo.Parser`. Ideally, we’d be able to more precisely specify the
relationship (`instance Json t => Json (Compatible t)`), or failing that
at least uniformly require that `T: JsonObject` and keep the type
definition opaque, but the former isn’t possible in Flow and the latter
is [blocked on a Flow bug][1], so this will have to do for now.
Fixes#1875.
[1]: https://github.com/facebook/flow/issues/8405
Test Plan:
Running `git grep 1875` no longer turns up anything relevant.
wchargin-branch: compat-expose-jsonobject
Summary:
This is more general: a mutable type is a subtype of its corresponding
read-only type. Using the covariant form lets structures like `string[]`
be subtypes of `JsonObject`, as `T[]` is not (in general) a subtype of
`(T | U)[]` but is a subtype of `$ReadOnlyArray<T | U>`.
Test Plan:
Demonstration type-level tests added.
wchargin-branch: jsonobject-readonly
We need the `--version` flag, both for general goodness, and because our
Docker publish pipeline depends on it. While I was at it, I
re-integrated the `help` command as well, along with some unit testing.
Test plan: `yarn test` passes; run `yarn backend` and then run the
command both without args (to see a dummy help message), or with the
`--help` or `help` args, or with `--version` args. Unit tests cover all
thsi behavior.
This moves all of the `cli2` command files into the `cli` directory,
since we are making the instance CLI the canonical CLI.
Test plan:
`yarn test --full` passes; `git grep cli2` returns no hits.
This commit removes the old CLI, which is being replaced by the new
instance-system based approach (currently called cli2).
This also removes sharness tests on the old CLI, as well as the
associated snapshots. This is a little unfortunate since the GitHub
snapshot did provide some validation against changes to the plugin; we'd
do well to re-integrate such a system when adding testing to cli2.
Test plan: `yarn test --full` passes. Since this is just a removal,
there's not much that can go wrong.
Summary:
One weird hack needs to be slightly amended; everything else still works
given recent changes to suppression comments.
Test Plan:
Running `yarn flow` passes as is, but fails if a suppression comment is
removed.
wchargin-branch: flow-v0.127.0
Summary:
The latest versions of Flow restrict suppression comments to the fixed
names `$FlowExpectedError` or `$FlowFixMe`. This patch replaces uses of
our custom suppression comment with `$FlowExpectedError`, as we don’t
use suppressions to mean “fix me; need to add types later” but instead
as expected type errors in tests (almost exclusively).
Generated by changing the suppression in `.flowconfig` and then running:
```
git ls-files -z '*.js' |
xargs -0 sed -i -e 's/\$ExpectFlowError/$FlowExpectedError/g'
```
To check the changes in non-test files, run
```
git show ':!*.test.js'
```
and note that all changes are in fact expected errors rather than TODOs.
Test Plan:
Check that `yarn flow` passes as is, but fails if a `$FlowExpectedError`
comment is deleted.
wchargin-branch: flow-standard-suppressions
Summary:
This removes a SourceCred-specific hack because the upstream issue has
been fixed. Removing the hack is important because it uses a suppression
comment and thus hampers upgrading to latest Flow.
The libdef portion of this change was generated by removing the old file
and then running `flow-typed install express@4.17.1`.
Test Plan:
That `yarn flow` passes suffices.
wchargin-branch: libdefs-express
Now that we're making the new UI the canonical one, we can remove
references to calling it `build2` or `start2`.
Test plan: `yarn start --instance $PATH` and `yarn build` both work.
`yarn test --full` passes.
This commit removes the old homepage entirely. This is a prelude
to removing the react-router dependency, which is needed to unblock work
on the initiatives editor.
Because the homepage is gone, there's now no frontend included with
SourceCred. As such, we should merge #1873 alongside this one, so that
our README doesn't give any patently false information to our users.
Test plan: `yarn test` passes. `yarn start` and `yarn build` are no
longer commands. (`yarn start2` and `yarn build2` will be renamed
later).
This adds a proper parse for core.Weights, and modifies the `fromJSON`
method to use that parser.
This means we now have runtime validation that all Weights files are
actually valid.
Note: I made a tweak to the eslint config so as to allow `_`s in names
when followed by a sequence of numbers, so that types like `Data_0_1_0`
(for including version strings) do not trigger lint failures.
Test plan: All existing tests pass, and I added a new test to validate
that we keep supporting the 0.2.0 format going forward.
Our README.md has a hodge-podge of information about using SourceCred.
Much of it is about to go entirely out of date because we are killing
the old CLI and frontend.
As such, I'm cleaning out and greatly simplifying the README, so we
won't give our users inaccurate information. I didn't make an effort at
replacing the documentation, that will come afterwards once we're ready
to document the instance system.
Creates a class to handle the implementation
of fetching data from the Discord server and
saving it to the local mirror.
Test plan:
Unit tests added.
paired with @beanow
This adds a fromRaw method to the Address module so that we may properly
parse serialized addresses. It has a type signature of `string =>
Address`, and throws an error if the string is not a valid address.
I basically copied the implementation out of the `assertValid` method. I
considered deduplicating them (i.e. having `assertValid` simply call
`fromRaw`), but `assertValid` had some extra logic around accepting a
prefix for the error message, and it felt simpler to simply copy+paste
rather than trying to wrap it.
Test plan:
I added unit tests; `yarn test` passes.
Summary:
An optional second parameter to `C.dict` can now be used to refine keys
to a subtype of string: e.g.,
C.dict(C.number, C.fmap(C.string, NodeAddress.fromRaw))
is a `C.Parser<{[NodeAddressT]: number}>`. Before this change, `C.dict`
could only parse objects whose key type ranged literally over `string`.
Note that the argument order is `C.dict(valueParser, keyParser)`, not
the other way around. This is because `C.dict(valueParser)` is the more
general use case (you _need_ a value parser), and it’s simpler to have
the optional argument be at the end. (Analogously, the `forEach` methods
on `Array`s and `Map`s take a callback accepting `(value, key)`.)
Test Plan:
Unit tests included, retaining full coverage. Existing tests for the
experimental Discord plugin, which uses `C.dict`, also still pass.
wchargin-branch: combo-dict-key
This adds two methods to cli2/common.js:
- loadJson, which loads a JSON file from disk and then parses it
- loadJsonWithDefault, which loads a JSON file from disk and parses it,
or returns a default value if the file is not present.
Both methods are well documented and well tested.
Test plan: `yarn test`; see included unit tests which are thorough.
This adds Combo parsing support to the compatible module. Now, rather
than writing `fromJSON` methods which implicitly take any, we can
instead write typesafe `parseJSON` methods which will parse compatible
headers, and then choose a version-appropriate parser.
Test plan: Added unit tests; `yarn test` passes.
Just a more sensible name since it's not a homepage of any sort.
Test plan: Flow passes, `yarn build2` and `yarn start2` both succeed.
Also: `git grep homepage2` gets no hits.
