Commit Graph

1273 Commits

Author SHA1 Message Date
William Chargin 32f83ad676
ci: fast-fail on forked PRs with no credentials (#1522)
Summary:
PRs created from forks don’t have credentials when running CI. This
commit causes the `test-full` job (which requires credentials) to fail
fast with a helpful error message.

Test Plan:
Push distinct versions of this commit to a fork and to the main
repository, and open pull requests for each. Note that the tests pass
from the main repository, but fail with a nice message from the fork:

![Screenshot of expected fast-fail behavior][ss]

The “team member pushes to trusted branch” workflow has already been
successfully exercised for #1521.

[ss]: https://user-images.githubusercontent.com/4317806/71707839-b782ab00-2da1-11ea-8aa9-7d8720538a87.png

wchargin-branch: forked-pr-fail-fast
2020-01-07 21:03:23 -08:00
Robin van Boven fa00751cd4
CI: remove broken tag deploy (#1513)
See #1512 for full context.

Short explanation:
Because the job wants to run only on tag pushes, but requires
the `test` job (which doesn't run on tag pushes), the job
will never run.
2020-01-08 00:01:02 +01:00
Robin van Boven 3cfb338336
Discourse: add mirror findUsername (#1528)
Gets the username of a user, if it exists.
Helpful for fixing capitalization issues such as #1479,
and verifying the user exists for reference detection.
2020-01-07 22:25:09 +01:00
Robin van Boven 6c15372d9f
CI: separate node10 and 12 cache (#1527)
Previously both node versions would share the same cache.
This caused one of the two versions to always rebuild
the `better-sqlite3` package, costing about 1 min per job.

Now we're using a different cache key for each version,
rebuilding a cached `better-sqlite3` should no longer be
necessary.
2020-01-07 22:16:39 +01:00
Robin van Boven 03d525810d
Initiatives: implement DiscourseInitiativeRepository (#1483)
Uses the Discourse mirror data and parsing function to create an
InitiativeRepository implementation. Which we can use for createGraph.
2020-01-07 14:43:45 +01:00
Robin van Boven 105912e498
Initiatives: create Initiative from discourse topics (#1480)
This adds the Discourse specific semantics on top of the basic template parsing.
2020-01-07 14:21:54 +01:00
Robin van Boven 4ab58a09b5
Initiatives: implement "cooked HTML" template parsing (#1478)
Will return a partial Initiative, or throw when the template isn't matched.
2020-01-07 14:15:26 +01:00
Robin van Boven b05cc84f2e
Initiatives: implement basic createGraph (#1477) 2020-01-07 14:07:27 +01:00
Robin van Boven 579b01ed46
UniRef: adding TranslatingReferenceDetector implementation (#1511)
The TranslatingReferenceDetector is an abstraction particularly useful for the
Initiatives reference detector. Which should use the Discourse reference
detector as it's base and translate the node address of the returned discourse
topic to the initiative's node address.
2020-01-07 13:48:25 +01:00
Robin van Boven 953cbfaac2
UniRef: add CascadingReferenceDetector implementation (#1510)
The CascadingReferenceDetector is an abstraction we can use for combining multiple ReferenceDetector instances and giving them a priority.
2020-01-07 13:39:21 +01:00
Robin van Boven 821be0b46e
UniRef: add MappedReferenceDetector implementation (#1509)
The current reference detection implementation internal to the GitHub plugin
uses a map similar to this. This class being near to that makes it easy to adopt.
It's also very simple to use for tests.
2020-01-07 13:26:53 +01:00
Robin van Boven 4bdc7a57b7
UniRef: Declare ReferenceDetector interface (#1508)
The core declaration of the ReferenceDetector interface.

Reason I'm adding an index.js file is to allow (core) classes that implement
this interface to have separate files, while keeping redundancy out of the
import statements.
2020-01-07 12:27:19 +01:00
William Chargin cdb360f6f9
docs: add note about full test suite credentials (#1524)
Summary:
Contributors who open PRs from a fork will need to have their commits
“blessed” by a core team member before the `test-full` CI job will run
successfully. This commit explains that to ward off any confusion.

Test Plan:
This workflow was recently exercised for #1521, successfully.

wchargin-branch: contributing-test-full
2020-01-05 11:18:54 -08:00
Robin van Boven b860f31a19
Use createProject to set default values for Project (#1492)
Creation of new Project instances is spread out across the code.
So whenever there's a change in it's format, the PR is cluttered
with adding a logical default value in many places. It means
our default values might be inconsistent as well.

For example #1385 adds many `identities: [],` lines.
A similar situation would happen with the planned Initiatives
plugin, adding many `initiatives: null,` lines.

Using this function we can manage what default values to add
from a central place. Avoiding noise and code churn.
2020-01-03 15:11:41 +01:00
Robin van Boven b7b93d2a8d
Add unit tests for projectFromJSON's upgrade support (#1519) 2020-01-03 14:57:46 +01:00
Robin van Boven fb16530b8f
Update projectFromJSON & upgrade function signatures (#1518)
This creates better flow type coverage for the upgrading
from older Project types feature.

Note projectFromJSON's function signature changes like
this:

- (Compatible<Project>) => Project
+ (Compatible<any>) => Project

And that makes sense, because we use this function to
validate an object we parsed from JSON at runtime. It
could actually be anything.

Added benefit is that is makes writing unit tests possible.
Because now will flow not throw a type error when we provide
something other than Compatible<Project> as input, to test
upgrading or validation functionality.

Note that the underlying utility fromCompat already uses
Compatible<any> for the same object.
2020-01-03 14:50:20 +01:00
Robin van Boven 89529ea4fe
Define previous Project type versions (#1517)
These are important to accurately add types to function
signatures of validating and upgrading logic.
2020-01-03 14:36:01 +01:00
burrrata 5b9455cf9d Add instructions to set up the GitHub API Token (#1521)
The README explains how to set the SOURCECRED_GITHUB_TOKEN, but later in
the Docker section. People who aren't using Docker will follow the
initial installation instructions. This commit adds the instructions to
set that up when users first install and set up SourceCred.
2020-01-02 20:17:31 -08:00
Dandelion Mané 507091a976
Allow `while(true)` loops (#1495)
The eslint no-constant-condition rule disallows while(true) loops,
since the true is a constant condition. However, I find the allowed
alternative (`for (;;)`) less readable, so I am adding the sub-rule that
allows constant conditions for loops.

Test plan: A followon commit uses a while(true) loop, and, assuming this
patch is applied, it does not result in a lint error.

Co-authored-by: Robin van Boven <497556+Beanow@users.noreply.github.com>
2019-12-28 15:58:43 -08:00
Robin van Boven 8f32912270
Initiatives: define internal datatype Initiative (#1417) 2019-12-23 00:43:36 +01:00
Robin van Boven 1b0eb483ce
Initiatives: create plugin declaration (#1416) 2019-12-23 00:36:57 +01:00
Robin van Boven 5727a831a9
Security: force update yarn version in Dockerfile (#1503)
Note, unless you used the SourceCred Docker image's bundled
npm or yarn to install your own package.json dependencies,
you were not vulnerable. Otherwise the same risk applies as
[in this NPM blog][1].

You can patch the vulnerability by using the latest Docker image
using `docker pull sourcecred/sourcecred` as soon as this commit
is included in the latest release.

## Commit details

In a [recent security issue][1] found in NPM and Yarn, handling
binary file installation has changed. Quoting from there:

> The bin script linking libraries in use in npm v6.13.4 were
> updated such that, when installing binary entries of top-level
> globally installed packages, they will only overwrite existing
> binary files if they are currently installed on behalf of the
> same package being installed.  For example, npm install –global
> foo could overwrite /usr/local/bin/foo if and only if
> /usr/local/bin/foo is currently a link to a previously installed
> version of foo.

In our case, we specifically want this behavior in our Dockerfile.
The node:12 base image comes with an NPM and Yarn version installed.
We're using npm i -g yarn@<version> to upgrade the yarn installation
to a predictable minimum, should we have an older version from the
base image. But since they're from different installation sources,
it causes an error as it would overwrite the yarn binary that wasn't
previously owned by npm install.

Our own package.json or yarn.lock did not appear to have any risk
of exploitation. However since we bundle our image with npm and yarn,
people using our image could in theory use it to install their own
packages. Meaning we should include the fixed npm and yarn versions
to protect users in such a scenario.

[1]: https://blog.npmjs.org/post/189618601100/binary-planting-with-the-npm-cli
2019-12-22 21:42:03 +01:00
greenkeeper[bot] e68f99bf00 Update jest-fetch-mock to the latest version 🚀 (#1494)
* chore(package): update jest-fetch-mock to version 3.0.0

* chore(package): update lockfile yarn.lock
2019-12-22 13:23:19 -06:00
burrrata 02515de550 Fix broken README link (#1507)
The link to the prototype was outdated.

Test plan: Manually check the link.
2019-12-22 13:20:23 -06:00
greenkeeper[bot] 0aea1ff13b Update flow-bin to the latest version 🚀 (#1489)
* chore(package): update flow-bin to version 0.114.0

* chore(package): update lockfile yarn.lock
2019-12-16 14:34:16 -06:00
Robin van Boven f35d7e088f
Chore: update packages & bugfix new versions (#1487)
* chore(package): yarn upgrade

Updates all packages within version range.

* Bugfix update stacktrace matching code

The stacktrace has changed, most likely due to
a babel plugin updating. It now seems based on
the name of the `handlingErrors` argument
instead of the variable name storing the
anonymous function.

* Bugfix update react-router patch version

By updating the react packages, warnings were
logged about unsafe componentWillMount usage.
These warnings tripped a unit test.
react-router was the cause of these, so this
update avoids getting the warnings.
2019-12-10 20:52:55 +01:00
Robin van Boven a41eb71949
GitHub: assume installation token length of 40 (#1486)
In the documentation 16 characters were displayed.
But testing showed we're typically seeing 40.

Fixes #1474
2019-12-10 18:45:54 +01:00
greenkeeper[bot] ef3e1d6c48 Update eslint-plugin-import to the latest version 🚀 (#1484)
* chore(package): update eslint-plugin-import to version 2.19.0

* chore(package): update lockfile yarn.lock
2019-12-09 15:19:42 +01:00
greenkeeper[bot] 7951d3adae Update file-loader to the latest version 🚀 (#1468)
* chore(package): update file-loader to version 5.0.2

Closes #1468

* chore(package): update lockfile yarn.lock
2019-12-09 15:07:58 +01:00
greenkeeper[bot] 13fa8e78a6 Update url-loader to the latest version 🚀 (#1470)
* chore(package): update url-loader to version 3.0.0

* chore(package): update lockfile yarn.lock
2019-12-09 14:57:52 +01:00
Robin van Boven b4a0cd5ec7
Discourse: remove update mode 1 (#1482) 2019-12-09 13:00:47 +01:00
Robin van Boven c209c40e08
Discourse: update error handling of fetch (#1481)
- Have "topic" reflect actual method name.
- Add missing 403 and 429 test for likes.
- Preemptively change method used for headers,
  as .post will be obsolete after refactor.
2019-12-09 12:54:02 +01:00
Robin van Boven 32a1db3010
Discourse: default to update mode 2 (#1465) 2019-12-03 20:11:28 +01:00
greenkeeper[bot] c5d7d27459 Update flow-bin to the latest version 🚀 (#1476)
* chore(package): update flow-bin to version 0.113.0

* chore(package): update lockfile yarn.lock
2019-12-03 12:51:25 -06:00
Robin van Boven f30723b96d
Discourse: add update mode 2 (#1464) 2019-12-02 20:37:19 +01:00
Robin van Boven 890489c0d2
Discourse: make MockFetcher API similar to real forum (#1473)
This extends the MockFetcher in the tests
to provide new semantics update mode 2 relies on.
They're based on the below changes to the Fetcher:

- add categoryId and bumpedMs to Topic data #1454
- make topicWithPosts fetch all posts #1455
- add categoryDefinitionTopicIds to fetcher #1456
- implement topicsBumpedSince in fetcher #1457

Particularly because the addition of two new concepts
(categories and category definition topics), the API of
the MockFetcher got rather convoluted. This refactor
makes it behave a lot more like you'd be familiar with
within Discourse.

Such as, creating a topic creates it's opening post
as a side effect. Instead of a post with an unknown
topic ID creating a topic as a side effect.
And creating a category creates it's category
definition topic as a side effect.

Also, we're being a lot more explicit, using objects
instead of positional arguments.
2019-12-02 20:29:18 +01:00
Robin van Boven c521acc145
Discourse: scope mirror tests as being "mode 1" (#1463)
This is to prepare for mode 2 being tested side-by-side.

The normalizeMode1Topics function enforces bumpedMs is not
updated for mode 1 tests.

Additionally describe "update semantics" is redundant,
as the mirror has no other function than update.
2019-12-02 20:24:49 +01:00
Robin van Boven 3ceb4fb7fa
GitHub: update token validation function (#1471)
Previously an inline check was used for this.
It only accepted the personal access token format.
This adds installation tokens as requested in #1461.

With more complex logic, we'd benefit from tests.
Therefore it's a separate function with a test suite.
2019-11-29 11:53:07 +01:00
Robin van Boven 984c6bbe9f
Add transifex-integration bot (#1469)
See 9d48a5fca6
as an example of the bot acting as a user.
2019-11-29 11:46:02 +01:00
Robin van Boven f2e1775c20
Add github-actions bot (#1466)
See https://github.com/sourcecred/sourcecred-action/pull/6
as an example of the bot acting as a user.
2019-11-29 11:40:37 +01:00
greenkeeper[bot] 811d22aa61 Update eslint-plugin-flowtype to the latest version 🚀 (#1467)
* chore(package): update eslint-plugin-flowtype to version 4.5.0

* chore(package): update lockfile yarn.lock
2019-11-28 18:54:52 -06:00
Robin van Boven a9e89b9f32
Discourse: move update steps to separate functions (#1462)
Makes no functional changes, it simply splits the update into separate
functions so it can be switched out for another implementation.
2019-11-26 11:55:39 +01:00
Robin van Boven 1e643d012f
Discourse: add SyncHeads to the repository (#1460)
This tracks the local state for new mirroring logic.
2019-11-26 11:47:16 +01:00
Robin van Boven f6bc91ce5f
Discourse: add replaceTopicTransaction method to repository (#1459)
Idempotent insert/replace of a Topic, including all it's Posts.
Note: this will insert new posts, update existing posts and delete
old posts. As these are separate queries, we use a transaction here.

This is to be used in the new update logic, which also fetches
all posts of a topic when the topic is loaded. In particular
this allows post editing, which is important for wiki's such as
those used for the initiative system.
2019-11-26 11:35:04 +01:00
Robin van Boven 7deb0a3205
Discourse: adds bumpedMsForTopic and topicsInCategories queries (#1458)
bumpedMsForTopic
For the given topic ID, retrieves the bumpedMs value.
Returns null, when the topic wasn't found.
Used by the new update code as a fallback value when making API
calls that don't contain the bumpedMs field.

topicsInCategories
Finds the TopicIds of topics that have one of the categoryIds as
it's category.
Useful to find out which topics a set of categories contains.
For example to implement the `recheckTopicsInCategories` mirror
option, or to locate topics for the initiative plugin.
2019-11-26 11:02:27 +01:00
Robin van Boven 564fd89b1e
Discourse: implement topicsBumpedSince in fetcher (#1457) 2019-11-20 13:27:13 +01:00
Robin van Boven 51e3eb8c25
Discourse: add categoryDefinitionTopicIds to fetcher (#1456) 2019-11-16 14:04:45 +01:00
Robin van Boven 623c362246
Discourse: make topicWithPosts fetch all posts (#1455)
Previously it would only consider page 1.
Now we're walking through all pages, as this
is a much more effective way of discovering
all posts.
2019-11-16 13:59:18 +01:00
Robin van Boven 23f1db6ce4
Discourse: add categoryId and bumpedMs to Topic data (#1454)
As not all API calls return bumpedMs, make a new type
to show the distinction.
2019-11-16 13:52:32 +01:00
Robin van Boven e79cca6c6c
Discourse: add mirror options to 0.4.0 projects (#1451)
N.b. this is an alternative to #1433, removing
multi-server support for discourse.
2019-11-16 13:46:09 +01:00