status-im/swarms
2
0
Fork 0
mirror of https://github.com/status-im/swarms.git synced 2025-01-20 23:39:37 +00:00
Code Issues Projects Releases Wiki Activity

Split the core team proposal into 3: the team doc, core improvements + core networking (#339)

Browse Source
This commit is contained in:
Igor Mandrigin 2018-12-13 19:02:45 +01:00 committed by GitHub
parent 0bf757cd5b
commit 292d42bd05
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 404 additions and 214 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

249
ideas/310-core-team-proposal.md
View File

@ -1,13 +1,22 @@
---
title: Status Core Team
id: #310-core-team
title: Core Team
status: research
lead conributor: Igor
contributors:
- Adam
- Andrea MP
- Pedro
- Roman
- Dmitry S
budget:
- actual: xxx
- estimate: yyy
- currency: ETH/USD/SNT
---
###### tags: `proposal` `core` `SNIP` `SNIP-core`
# Status Core proposal
## Summary
# Core Team
## Summary and Goal(s)
Status Core is responsible for implementing a decentralized, reliable Ethereum reference client for Mobile on Android and iOS. This client should provide a reliable communication medium and ability to access the open decentralized web.
Core team also sets up the design direction for the mobile app.
@ -17,43 +26,31 @@ Desktop is out of scope, as is protocol research. Whitepaper use case implementa
Core needs an accountable, passionate get-shit-done mentality and drive, in contrast to the meandering blame-shifting.
## Budget
Headcount-based. 8 people.
The swarm also has a discretionary budget for bounty based development using Gitcoin, in order to parallelize development.
Assuming average organizational compensation, this would be roughly 69k/m. Possibly more based on performance, etc.
A self-imposed ceiling of 100k is proposed for budget bucketing (~10%) for the team as a whole.
Related adjacent activities such as security, devops, marketing, QA, etc will be assumed to be provided and funded by the Gmbh until further notice.
## Core participants
- Adam
- Andrea MP
- Chad (PM)
- DmitryS
- Igor (tech lead)
## Additional contibutors
- Andrei M (Design)
- Pedro
- Roman
- Serhy (QA)
- Serhy/Anna/Nastya(QA)
## Scope
See also: [team and responsibilities](https://notes.status.im/k0xVSu6nRvS5p0UAWV__BA?view)
Core should focus on problems/features that require multidisciplinary understanding and influence the application architecture. With the highest priority of making Status a usable application for consumption.
For simplicity, the responsibilities are split into 2 categories:
* **maintenance** - that means this effort is continuous and doesn't have any end date or exit criteria (publishing a mobile release is a good example, we don't stop)
* **project** - something that has a clear exit criteria, with a few iterations, after which the task is considered completed (for instance, when LES option is there and works)
### Active Swarms
* [Network Incentivization Swarm](./313-core-networking.md)
* [Core Improvements Swarm](./315-core-improvements.md)
## Process (aka How We Work?)
## Communications
(required)
`status channel`:[`#status-core`](http://get.status.im/chat/public/status-core).
`sync frequency`: Weekly Sync, Monday 10am CET
`meeting notes`: [this running doc on notes.status.im](https://notes.status.im/sm7x7tPpQESkRwu7YNFzSQ)
### Projects
### Process
Weekly sync-up calls
- goals for the week
- focus area, progress
Each project that is currently active should have an umbrella issue that should
link all the other issues. It contains a full description of the project and is
@ -68,187 +65,11 @@ Each project should contain **Vision**, **Scope** and **Tradeoffs**.
2. Scope: limiting conditions that we empose on ourselves when implementing the project (what it is doing and what it is NOT doing).
3. Tradeoffs: a lot of projects can contain tradeoffs that make is feasible to implement. These should be documented there.
See https://github.com/status-im/status-react/issues/6757 as an example of a similar issue.
See [status-react#6757](https://github.com/status-im/status-react/issues/6757) as an example of a similar issue.
The projects list should have links to umbrella issues for each projects.
### Meetings & Philosophy
Process can be revisited through retrospective. Probably, the process should go to a github repository as a markdown file and using PRs to update it.
## Copyright
Taking as little of the parallel projects as possible. For example, we have to do maintenance anyway, so adding one other project as a focus area should be okay.
Weekly sync-up calls
- goals for the week
- focus area, progress
- post-meeting report in a Discuss thread
Post-project retrospective with post-mortem
- when the project milestone is done/failed
- when we are deciding to switch to another focus area
[DRIs](https://www.forbes.com/sites/quora/2012/10/02/how-well-does-apples-directly-responsible-individual-dri-model-work-in-practice/#38840682194c) for focus areas / maintenance tasks:
- single point of contact
- focused on one thing (one can be a DRI only to one focus area/maintenance task)
# Core: Maintenance
#### 1. Battery usage
#### 2. Application performance profiling
#### 3. Mobile Releases
- [Mobile release guide](https://notes.status.im/mobile-release-guide)
- [Mobile release notes](https://notes.status.im/mobile-release-notes)
- Upgrade policy
- *User’s data.* User should be able to successfully upgrade preserving all his/her data from 0.9.19 to any beta release.
- *Messaging/Transactions.* User should be able to successfully interact (make transactions, chat, etc) with users that have 0.9.20 or later release.
#### 4. Product Analytics
Product analytics provides insight into how Status is being used. Currently we have 3 key metrics delivering insight:
- App downloads
- Public message
- ENS registrations
The core team is responsible for maintaining the reporting of these metrics at [status.im/analytics](https://status.im/analytics/)
# Core: Projects - Active
#### Message Reliability Improvements
The goal of this project is:
* to make messaging extremely reliable;
* to be able to prove that;
* to be able to show that information to a user (especially it is important to
communicate if the message isn't delivered).
See more and up-to-date info in the Epic: https://github.com/status-im/status-react/issues/6757
# Core: Projects - Backlog
This backlog is organized in a prioritized way. The higher the project the more important it is.
When promoting the project, it is also important to make it more detailed.
#### "Save Password" on Android
The goal is to achieve feature-parity between iOS and Android in terms of
storing the password.
Epic: https://github.com/status-im/status-react/issues/5793
#### Support LES and ULC
Add an option to use LES and ULC in our application. It should work on Ropsten and Mainnet with working discovery.
Status can host clusters for LES and ULC, for both testing and reliability purposes, but the usage of LES/ULC shouldn't be limited to that.
See more and up-to-date info in the Epic: https://github.com/status-im/status-react/issues/6905
#### Spam filtering & DDoS Protection
We lack the way of protecting our network from spam, especially on the network
layer where a bot can relatively easily spam channels or at least waste users'
traffic.
#### LES/ULC: Economic incentives
> [dmitry] is it about proposal from Jacek?
> [igorm] probably, I'm not sure. it isn't the priority unless LES/ULC is just functional.
#### VipNode support
#### Implement EIP-681
https://eips.ethereum.org/EIPS/eip-681
> A standard way of representing various transactions, especially payment requests in Ethers and ERC #20 tokens as URLs.
#### Reduce unnecessary bandwidth usage
https://github.com/status-im/status-go/issues/1266
Make sure that we identify and fix all the unnecessary bandwidth usage cases. We need to be able to test the usage regressions in the future, so these issues don't happen anymore.
Currently, it looks like our application uses too much bandwidth.
**Why?**
1. We don't have bandwidth tests in place.
2. Mailserver requests seems to be very expensive in bandwidth and we probably receive a lot of duplicate messages.
3. We don't know all the places where networking is used, so we can't measure bandwidth client-side (we have to only rely on iOS/Android settings to check that).
#### Support multi-device experience
We need to be able to see the same message history on multiple devices (desktop + mobile is the common scenario).
#### Improve mechanisms of storing/using sensitive information on mobile
**Problems in this area**
1. Whisper uses the same keypairs as the blockchain, exposing one of them leads to losing funds.
2. It is possible (and it happened before) to leak sensitive data through logs and debugging information.
3. Unsafe defaults in some publicly available builds in the application.
4. Sensitive information is used directly (accessible to most of the app components), whereas the best practices is to provide minimal APIs to do whatever needs to be done w/o exposing data (see Secure Enclave data signing)
5. Only a single protection mechanism for sensitive data (should be at least two).
6. (potentially) Storing sensitive information which we don't have to store.
#### Improved message compatibility with other versions and other clients (using the *current* version of the protocol)
**Problems**
1. It is too easy to change protocol (message format, etc) in a PR w/o necessary oversight.
2. There are no tests to detect that message format or semantics didn't change after PR is merged.
3. We don't have any procedures and best practices on how to introduce changes to the current protocol.
4. We don't have any procedures and best practices on how to handle breaking changes gracefully and coordinate them between the clients.
5. We don't have any procedures, tools and best practices to handle breaking changes in mailservers.
> [pedro] Same thing for mailservers. Just recently there was a change to status-go that made it incompatible with the deployed eth.beta mailservers. We have no way to deal with backward compatibility, and having mailserver pinned makes it especially problematic.
> [igor] added this part too
#### Reduce metadata leakage
#### Audited private group chat and PFS
#### Reproducible builds
**Problems**
1. Right now it's hard to tell exactly which commit of any dependency was used to build a particular release
2. Users don't have guarantees that the release they downloaded matches the official release (i.e. hasn't been tampered with).
3. A user should be able to build the sources on his own machine and obtain exactly the same binaries.
#### Integrate with `libp2p`
#### Better discovery
1. Support/integrate geth initiatives
#### Better censorship resistance
1. Being able to update bootnodes list if they are blocked
#### Push Notifications v2
https://ideas.status.im/ideas/086-push-notif-v2/README
---
# Outscoped Projects (to other teams)
#### 14. Better CI experience
To DevOps team?
Not sure if the core team responsibility (igorm).
This is one of the problems that everyone depends on. If/when our CI is unstable, essentially, the whole dev team cannot work.
> [???] Although I agree that this is mission critical, I would like to see this offloaded to a different team, it requires a specific skillset (which we might have in the core-team, but not only) and not many dependencies, also our plate is quite full as it is.
> [igorm] Agree, I just want to make sure it will be prioritized by some team, and asap.
**Problems**
1. Issues with builds scripts on macOS builders (not isolated)
2. Way too many network requests on each build process (if any of these fails, all the build procedure fails).
3. No retries of commonly unstable steps (again, when it fails, everything fails).
4. No decent caching/reusing of artifacts (that makes the build process very slow).
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

138
ideas/313-core-networking.md Normal file
View File

@ -0,0 +1,138 @@
---
id: #313-core-network-incentives
title: Core Network Incentives Swarm Proposal
status: research
contributors:
- igor
- andrea mp
budget:
- actual: xxx
- estimate: yyy
- currency: ETH/USD/SNT
---
# Core Networking Swarm proposal
## Summary and Goal(s)
Research and develop framework and incentivization structure for people
deploying and using Status nodes according to [the whitepaper](https://status.im/whitepaper.pdf).
One important vision aspect is that the app continues to work if all nodes in Status-hosted cluster is down.
### 1. Offline messaging
This one is the primary topic of research.
The goal is:
* to give the ability for the people ("providers") to run and provide offline messaging to users of the Status app.
- cryptoeconomic incentives (SNT use-case);
- lowering the boundaries to install status node (docs, install scripts, etc);
* to users of the Status app to discover and use offline messaging nodes from providers.
- receiving messages reliabily;
- being able to choose the best mailservers and knowing the compromises.
* preserve security and privacy;
### 2. Push Notifications
Distributed push-notification market. Researching ways on how can users pick PN providers.
### Additional contributors
- oskart
- dmitrys
- andrei m (design)
- a smart contract dev
## Communication
(required)
`status channel (same as swarm id)`: [#313-core-network-incentives](https://get.status.im/chat/public/313-core-networking)
`sync frequency`: Weekly Sync Mondays, 11:00 CET
## Research
(required)
`timebox research (approx)`
Research deadline: January 31th
(optional)
`eips`, `competitors`, `existing research`, `wireframes`, `spike (PoC)`
https://1337alliance.com/
https://github.com/status-im/swarms/tree/master/ideas/168-paid-master-nodes
https://status.im/whitepaper.pdf
protocol work and leveraging data sync data, eg
### 1. Offline Messaging
The goal of this research is to research the ways on how people can run Status
nodes and how we can guarantee good behaviour in the network.
Areas of research:
- viability of ideas;
- cryptoeconomics and incentivization for good behaviour;
- security and privacy aspects;
- UX research for both discovering nodes providing archives and running them;
- MVP and transition using the current protocol;
- risks assesments (what are known risks and unknown risks in implementing this
idea).
Outcomes:
- a document with the full research;
- a document with MVP proposal;
- list of risks for implementation;
- (optional) a few PoCs;
### 2. Push notifications
Areas of research:
- viability of ideas and their techincal implementation;
- cryptoeconomics and incentivization for good behaviour;
- security and privacy aspects;
- UX research for both discovering PN providers and hosting them;
- MVP and transition using the current protocol;
- risks assesments (what are known risks and unknown risks in implementing this
idea).
Outcomes:
- a document with the full research;
- a document with MVP proposal;
- list of risks for implementation;
- (optional) a few PoCs;
## Specification
> do after `Research`
(required)
`timebox specification (approx)`
(optional)
`user stories`, `architecture`, `designs`, `PoC`
## Implementation
(required)
`timebox implementation (approx)`
> do after `Specification`
> All swarm contributors should test and break the implementation, especially developers
(required)
`document progress`
(optional)
`townhall demo`
## Maintenance
`lead-contributor`,`post-mortems`
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

231
ideas/315-core-improvements.md Normal file
View File

@ -0,0 +1,231 @@
---
id: #313-core-improvements
title: Core Improvements Swarm
status: research
lead contributor: igor
contributors:
- adam
- dmitry s
- roman
- pedro
budget:
- actual: xxx
- estimate: yyy
- currency: ETH/USD/SNT
---
# Core Improvements Swarm
## Summary and Goal(s)
Status Core Improvements swarm's goal is the preparation of Status Mobile Client to be released publicly (AppStore/GP release tracks).
A few key areas that we need to focus on:
* all the critical security and privacy issues are resolved (including audit);
- a security and privacy document is delivered (it lists guarantees we make around security and privacy; as well as ways we are falling short)
* all the critical tech debt issues are resolved;
* key UX problems are solved in two areas:
- adoption issues;
- retention issues.
The work includes releasing internal "dogfood" builds as well as betas in preparation to the public release.
### Additional contributors
- Andrea MP
- Andrei M (Design)
- Serhy/Anna/Nastya(QA)
- Rachel/Hester (PO)
## Communication
(required)
`status channel (same as Core Team)`: [`#status-core`](https://get.status.im/chat/public/status-core)
`sync frequency`: Weekly Sync Mondays, 10:00 CET (the regular "Core Team" standup)
`meeting notes`: [this running doc on notes.status.im](https://notes.status.im/sm7x7tPpQESkRwu7YNFzSQ)
# Research
### 0. Meta (what this team will be doing?)
Completion by December, 19th (1 week)
Goals:
- Figure out the list of tasks and top UX issues that needs to be fixed
- Rank them by priority (what should we do first)
### 3. Improve Security using native views
Completion by 21/12/2018
Goals:
- Figure out how we can use native code (Java/ObjC) to short-circuit sensitive text views
to Status-Go, so attacks like [event-stream incident](https://blog.npmjs.org/post/180565383195/details-about-the-event-stream-incident) are impossible.
## Specification
### 2. "Save Password" on Android
* Timeline - January, 14th
The goal is to achieve feature-parity between iOS and Android in terms of
storing the password.
Epic: https://github.com/status-im/status-react/issues/5793
- [ ] Threat modelling (STRIDE) + document about that.
## Implementation
### 0. Meta
* Timeline - EOQ2 2019, continuous process. Each backlog item discovered by
research will have it's own deadline.
Setup a regular meetings with the stakeholders to adjust the backlog.
`document progress` - keep the backlog and the state up to date with this
document.
Projects from should be put in the correct state to see the progress, etc.
### 1. Message Reliability Improvements
* Timeline - January, 11
The goal of this project is:
* to make messaging extremely reliable;
* to be able to prove that;
* to be able to show that information to a user (especially it is important to
communicate if the message isn't delivered).
`document progress`: see the epic https://github.com/status-im/status-react/issues/6757
### 2. "Save Password" on Android
* Timeline - January, 21th
Epic: https://github.com/status-im/status-react/issues/5793
- [ ] Implement according to the spec
- [ ] Audit the implementation
## Maintenance
### 1. Battery usage
### 2. Application performance profiling
### 3. Mobile Releases
- [Mobile release guide](https://notes.status.im/mobile-release-guide)
- [Mobile release notes](https://notes.status.im/mobile-release-notes)
- Upgrade policy
* *User’s data.* User should be able to successfully upgrade preserving all his/her data from 0.9.19 to any beta release.
* *Messaging/Transactions.* User should be able to successfully interact (make transactions, chat, etc) with users that have 0.9.20 or later release.
## Backlog
This backlog is organized in a prioritized way. The higher the project the more important it is.
When promoting the project, it is also important to make it more detailed.
### Improve secure storage of geth private keys
1. Get rid of `react-native-keychain` and it's dependencies.
2. Patch `go-ethereum`, so AndroidKeyStore and iOS KeyChain (+optional Secure Enclave encryption) are used to store private keys.
### Split Whisper and Ethereum keys
### Improved message compatibility with other versions and other clients (using the *current* version of the protocol)
**Problems**
1. It is too easy to change protocol (message format, etc) in a PR w/o necessary oversight.
2. There are no tests to detect that message format or semantics didn't change after PR is merged.
3. We don't have any procedures and best practices on how to introduce changes to the current protocol.
4. We don't have any procedures and best practices on how to handle breaking changes gracefully and coordinate them between the clients.
5. We don't have any procedures, tools and best practices to handle breaking changes in mailservers.
### Spam filtering & DDoS Protection
We lack the way of protecting our network from spam, especially on the network
layer where a bot can relatively easily spam channels or at least waste users'
traffic.
### Reproducible builds
**Problems**
1. Right now it's hard to tell exactly which commit of any dependency was used to build a particular release
2. Users don't have guarantees that the release they downloaded matches the official release (i.e. hasn't been tampered with).
3. A user should be able to build the sources on his own machine and obtain exactly the same binaries.
### Reduce unnecessary bandwidth usage
https://github.com/status-im/status-go/issues/1266
Make sure that we identify and fix all the unnecessary bandwidth usage cases. We need to be able to test the usage regressions in the future, so these issues don't happen anymore.
Currently, it looks like our application uses too much bandwidth.
**Why?**
1. We don't have bandwidth tests in place.
2. Mailserver requests seems to be very expensive in bandwidth and we probably receive a lot of duplicate messages.
3. We don't know all the places where networking is used, so we can't measure bandwidth client-side (we have to only rely on iOS/Android settings to check that).
### Improve mechanisms of storing/using sensitive information on mobile
**Problems in this area**
1. It is possible (and it happened before) to leak sensitive data through logs and debugging information.
2. Unsafe defaults in some publicly available builds in the application.
3. Sensitive information is used directly (accessible to most of the app components), whereas the best practices is to provide minimal APIs to do whatever needs to be done w/o exposing data (see Secure Enclave data signing)
4. (potentially) Storing sensitive information which we don't have to store.
### Support LES and ULC
Add an option to use LES and ULC in our application. It should work on Ropsten and Mainnet with working discovery.
Status can host clusters for LES and ULC, for both testing and reliability purposes, but the usage of LES/ULC shouldn't be limited to that.
See more and up-to-date info in the Epic: https://github.com/status-im/status-react/issues/6905
### Reduce metadata leakage
### Audited private group chat and PFS
### Integrate with `libp2p`
### Better discovery
1. Support/integrate geth initiatives
### Be able to update bootnodes list if they are blocked
1. Being able to update bootnodes list if they are blocked
### Push Notifications v2
https://ideas.status.im/ideas/086-push-notif-v2/README
### LES/ULC: Economic incentives
> [dmitry] is it about proposal from Jacek?
> [igorm] probably, I'm not sure. it isn't the priority unless LES/ULC is just functional.
### VipNode support
### Implement EIP-681
https://eips.ethereum.org/EIPS/eip-681
> A standard way of representing various transactions, especially payment requests in Ethers and ERC #20 tokens as URLs.
## Copyright
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).