Commit Graph

180 Commits

Author SHA1 Message Date
Shivek Khurana 79bf90e990
👮🏽‍♂️ Update validators and add a new event (#2189)
* Add extra event to capture other type of navigations, allow empty screen name, rename cofx to get rid of clj ns, update tests

* Some view ids are greater than 16 characters. Made it 32 to be safe.

* Tab navigation events occur outside nav, add a new validator for them

* Remove navigate to cofx event, capture screens on will focus, get rid of enum and make valid screens a string less than 32 characters

* Run make generate

* Fix test

* Bump version to 0.75.1
2021-04-12 17:55:53 +05:30
Andrea Maria Piana d22182ea82 Ban user from community
This commit introduces a new endpoint to ban a user from a community.

Messages from this user will be ignored.
2021-03-24 09:29:21 +01:00
Shivek Khurana 1097b14a7f
🎭 📊 Anonymous Metrics V0 (#2170)
* Migrations in place, how to run them?

* Remove down migrations and touch database.go

* Database and Database Test package in place, added functions to get and store app metrics

* make generate output

* Minor bug fix on app metrics insert and select

* Add a validation layer to restrict what can be saved in the database

* Make validation more terse, throw error if schema doesn't exist, expose appmetrics service

* service updates

* Compute all errors before sending them out

* Trying to bring a closjure to appmetrics go

* Expose appmetrics via an api, skip fancy

* Address value as Jason Dawt Rawmasage to ease parsing

* Introduce a buffered chan with magic cap of 8 to minimize writes to DB. Tests for service and API. Also expose GetAppMetrics function.

* Lint issues

* Remove autoincrement, undo waku.json changes, fix error being shadowed, return nil where nil ought to be returned, get rid of buffered channel

* Bump migration number

* Fix API factory usage

* Add comment re:json.RawMessage instead of strings

* Get rid of test vars, throw save error inside the loop

* Update version

Co-authored-by: Samuel Hawksby-Robinson <samuel@samyoul.com>
2021-03-17 18:09:28 +05:30
Andrea Maria Piana f75f7bb738 Expand confirmations to private group chat messages
This commit expands the confirmation mechanism to allow private group
chat messages to be confirmed:

Changes:

- Added a separate table for message confirmations as group chat
messages have same messageID but multiple datasyncID
- Removed DataSyncID from raw message (I haven't removed the column name
as it can't be done in sqlite without copying over the table)
2021-03-16 13:41:14 +01:00
Roman Volosovskyi a7f2e94999
[wallet] Store account's last known balance and nonce 2021-03-15 16:18:12 +02:00
Andrea Maria Piana a1b3e3a772
Clean topics that we don't listen to
There was a bug on status-react where it would save filters that were
not listened to.
This commit adds a task to clean up those filters as they might result
in long syncing times.

This commit also returns topics/ranges/mailserves from messenger in
order to make the initialization of the app simpler and start moving
logic to status-go.

It also removes whisper from vendor.
2021-01-26 09:39:57 +01:00
Andrea Maria Piana 6f088bb5c5 Fix communities migration
In some instances the communities migration would be skipped but not
marked as `dirty`.

This commit addresses the issue by:
- Making sure that if dirty is set the migration is not skipped but
replayed
- If the version is on the communities migration and dirty is false, we
check for the presence of the communities table. If not present we
replay the communities migration.
- Make community_id field in user_messages nullable

It also removes all the `down` migration, as we can't use them
effectively, as explained in the README.md added.
2021-01-08 08:43:16 +01:00
Samuel Hawksby-Robinson e8d883edb6 bindata for profile_image migrations 2020-12-17 14:10:00 +01:00
Jonathan Rainville b2d8f7f416
feat: add bookmarks table for the browser bookmarks
plus get bookmark icon using a lib
2020-11-16 13:31:34 -05:00
Andrea Maria Piana 401d18e558
Enabled push notifications by default and add fleet 2020-08-27 18:54:06 +02:00
Andrea Maria Piana 4b8739a8bc
Polish up and address review feedback 2020-07-27 08:51:28 +02:00
Andrea Maria Piana 3a84afd0f1 Add replies to messages
Currently replies to messages are handled in status-react.
This causes some issues with the fact that sometimes replies might come
out of order, they might be offloaded to the database etc.

This commit changes the behavior so that status-go always returns the
replies, and in case a reply comes out of order (first the reply, later
the message being replied to), it will include in the messages the
updated message.

It also adds some fields (RTL,Replace,LineCount) to the database which
were not previously saved, resulting in some potential bugs.

The method that we use to pull replies is currently a bit naive, we just
pull all the message again from the database, but has the advantage of
being simple. It will go through performance testing to make sure
performnace are acceptable, if so I think it's reasonable to avoid some
complexity.
2020-04-16 15:51:28 +02:00
Andrea Maria Piana a5874fdb21 Add content type for group chats 2020-03-25 09:29:08 +01:00
Andrea Maria Piana de3c33f009 Add appearance setting 2020-03-25 07:42:52 +01:00
Jakub Sokołowski db9173492b update rendezvous nodes in configs
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-03-17 15:47:07 +01:00
Andrea Maria Piana 849492fda9
Add statusUpdate code (#1861)
* Add status-option code

This commits changes the behavior of waku introducing a new status-code,
`2`, that replaces the current single options codes.

* linting
2020-02-21 15:48:53 +01:00
Jakub Sokołowski 86f54c3f5f remove eth.beta fleet as it has been decomissioned
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-02-21 14:48:47 +01:00
Volodymyr Kozieiev 19fc448c85
pending-stickers field added to settings, unit-tests fixed (#1830) 2020-02-13 13:48:04 +02:00
Adam Babik 44aa313981
Make shhext and protocol work with Waku (#1777)
This change makes shhext and protocol submodule work with Waku and Whisper.
2020-01-13 20:17:30 +01:00
Andrea Maria Piana 88a1d0111e
Add Commands (#1731)
This commit adds handling of Transaction commands.
2020-01-10 19:59:01 +01:00
yenda 024f30f0b9 Put settings in a table with schema (#1746)
Move settings table schema from a key-value store to a one row table with many columns.

We now save the first row with initial data in saveAccountAndLogin and follow up saveSetting calls are only saving one setting at a time.

Co-authored-by: Adam Babik <a.babik@designfortress.com>
2019-12-27 10:58:25 +01:00
André Medeiros 0571f561f0
APIs to Get and Create custom Tokens (#1717) 2019-12-10 12:31:08 -05:00
Andrea Maria Piana fd49b0140e
Move to protobuf for Message type (#1706)
* Use a single Message type `v1/message.go` and `message.go` are the same now, and they embed `protobuf.ChatMessage`

* Use `SendChatMessage` for sending chat messages, this is basically the old `Send` but a bit more flexible so we can send different message types (stickers,commands), and not just text.

* Remove dedup from services/shhext. Because now we process in status-protocol, dedup makes less sense, as those messages are going to be processed anyway, so removing for now, we can re-evaluate if bringing it to status-go or not.

* Change the various retrieveX method to a single one:
`RetrieveAll` will be processing those messages that it can process (Currently only `Message`), and return the rest in `RawMessages` (still transit). The format for the response is:
`Chats`: -> The chats updated by receiving the message
`Messages`: -> The messages retrieved (already matched to a chat)
`Contacts`: -> The contacts updated by the messages
`RawMessages` -> Anything else that can't be parsed, eventually as we move everything to status-protocol-go this will go away.
2019-12-05 17:25:34 +01:00
Andrea Maria Piana b27779aa4e
Add Mailserver Request Gaps service (#1596)
This commits add some endpoints to store mailserver request gaps,
and provides 4 endpoints:

```
AddMailserverRequestGaps(gaps []MailserverRequestGap) error
GetMailserverRequestGaps(chatID string) []MailserverRequestGap
DeleteMailserverRequestGaps(ids []string) error
DeleteMailserverRequestGapsByChatID(id string) error
```
2019-09-04 12:04:17 +02:00
Adam Babik d7f454fe4c
Add mailservers service (#1595) 2019-09-04 08:25:33 +02:00
Andrea Maria Piana cef7f367ab Add topic negotiation
This commit add topic negotiation to the protocol.

On receiving a message from a client with version >= 1, we will generate
a shared key using Diffie-Hellman. We will record also which
installationID has sent us a message.

This key will be passed back to the above layer, which will then use to
start listening to a whisper topic (the `chat` namespace has no
knowledge of whisper).

When sending a message to a set of InstallationIDs, we check whether we
have agreed on a topic with all of them, and if so, we will send on this
separate topic, otherwise we fallback on discovery.

This change is backward compatible, as long as there is no downgrade of
the app on the other side.

A few changes:

* Factored out the DB in a separate namespace as now it is
being used by multiple services (TopicService and EncryptionService).

* Factored out multidevice management in a separate namespace

* Moved all the test to test the whole protoocl rather than just the encryption service

* Moved all the filter management in status-go
2019-06-19 17:38:45 +02:00
Adam Babik c9e99c432d migrate to go 1.12 and go modules 2019-06-12 13:12:00 +02:00
Andrea Maria Piana 4ab08629f6 Add postgres
This commits adds support for postgres database.
Currently two fields are stored: the bloom filter and the topic.
Only the bloom filter is actually used to query, but potentially we will
use also the topic in the future, so easier to separate it now in order
to avoid a migration.
2019-05-15 11:01:34 +02:00
Andrea Maria Piana 3292538b41
Allow multiple bundles, change n devices & bundle refresh (#1331)
Change to support sending multiple bundles, as needed for group chats,
limit number of devices to 3 as already done in the UI and refresh
bundle daily.
2018-12-21 11:07:25 +01:00
Andrea Maria Piana 1f6cccd0fc
Add enabling/disabling of installations (#1264)
This commit adds a list new table, installations, which is used to keep
track of which installation are active for a given identity key.

In general, we limit the number of installation that we keep
synchronized to 5, to avoid excessive usage of resources.

Any installation coming from our own identity, will have to be manually
enabled, otherwise we trust the other peer has correctly paired their
devices.

We use a timestamp to decide which installations to keep synchronized as
a logical clock would have make the creation of the bundle more
complicated, but this can always be converted to a logical clock at
later stages without breaking compatibility.
2018-11-06 09:05:32 +01:00
Andrea Maria Piana ee3c05c79b
Change handling of skipped/deleted keys & add version (#1261)
- Skipped keys

The purpose of limiting the number of skipped keys generated is to avoid a dos
attack whereby an attacker would send a large N, forcing the device to
compute all the keys between currentN..N .

Previously the logic for handling skipped keys was:

- If in the current receiving chain there are more than maxSkip keys,
throw an error

This is problematic as in long-lived session dropped/unreceived messages starts
piling up, eventually reaching the threshold (1000 dropped/unreceived
messages).

This logic has been changed to be more inline with signals spec, and now
it is:

- If N is > currentN + maxSkip, throw an error

The purpose of limiting the number of skipped keys stored is to avoid a dos
attack whereby an attacker would force us to store a large number of
keys, filling up our storage.

Previously the logic for handling old keys was:

- Once you have maxKeep ratchet steps, delete any key from
currentRatchet - maxKeep.

This, in combination with the maxSkip implementation, capped the number of stored keys to
maxSkip * maxKeep.

The logic has been changed to:

- Keep a maximum of MaxMessageKeysPerSession

and additionally we delete any key that has a sequence number <
currentSeqNum - maxKeep

- Version

We check now the version of the bundle so that when we get a bundle from
the same installationID with a higher version, we mark the previous
bundle as expired and use the new bundle the next time a message is sent
2018-11-05 20:00:04 +01:00
Andrea Maria Piana e4ba365b8a
Add bundles.added signal & pairing endpoint (#1237) 2018-10-16 12:31:05 +02:00
Adam Babik 8aef7c4f15
tune CLI for better docs experience (#1207) 2018-09-21 16:09:31 +02:00
Dmitry a11d970112 Update rendezvous nodes accordingle to our inventory
Rendezvous node identities were changed. Probably, previous private keys were
removed due to replacament of the VM. Additional node is from HK DC.
2018-09-20 09:46:59 +03:00
Pedro Pombeiro 99d7bc207d
Only keep two random static nodes from fleets.status.im (#1196) 2018-09-19 17:08:54 +02:00
Pedro Pombeiro 4e7ae20eae
Add script to update fleets static JSON. Closes #1188 (#1189) 2018-09-14 10:46:34 +02:00
Pedro Pombeiro 3d00af7fa3
Streamline configuration in status-go. Part of #1180 (#1183)
- Replace command line flags with `-c` config flag. Part of #1180
- Convert node config private keys to hex-encoded string versions.
- Remove `GenerateConfig` from library.
- Remove unused `FirebaseConfig` from library.
- Fix loading of `config/status-chain-genesis.json` in non-dev machines.
2018-09-13 18:31:29 +02:00
Kishan Sagathiya 0f5ad2545d Issue #765 Read configuration without go-bindata (#1158)
In the current codebase if you want to use different keys (or any
other configuration) for tests it is required to place them at a
certain path in static folder, bundle them with go-bindata and only
then run tests. This is simply unnecessary.

This commit instead directs loads configurations from file without
relying on go-bindata
2018-08-21 14:46:10 +02:00
Pedro Pombeiro ebc77374b2 Remove unused `test.sol` file
- File is no longer used after jail code was removed
- Ensure that `go-bindata` is set up on `make setup`
2018-08-15 14:52:40 +02:00
Ivan Daniluk 801053a17d
Remove jail package (#1106) 2018-07-24 12:05:21 +02:00
Dmitry Shulyak b120587c9b Update bindata 2018-04-09 15:42:36 +02:00
Dmitry Shulyak 88f881c83e
Rewrite config as a native golang structure and remove it from static (#762) 2018-03-26 15:52:20 +03:00
Adam Babik 8b94ac00bb
Update mainnet dev cluster config (#745) 2018-03-22 15:58:28 +01:00
Alex Kohler 4a01a395b4 Move status-go over to go-ethereum/log #717 (#727) 2018-03-20 19:35:28 +01:00
Frank Mueller dc4a0a98ed Fix setting of bootstrap nodes (#744)
Fix dropped bootstrap nodes setting and rename of cluster config and separation of nodes
2018-03-20 16:05:21 +02:00
Frank Mueller 8e42014671
Static peers are now also optionally configurable (#738)
* Rename bootnode to static peers

Also add some groupings between types.

* Remove not needed genesisHash

* More cleanup of bootnodes with static peers

* Add option of cluster configuration file

* New generated bindata.go

* Changes after npm install

* Add argument for cluster configuration file

* Add test for dynamic cluster loading

Not yet sure with name "cluster config".

* Solved conflicts

* Renaming of static peers

* Remove static peers population

* Missing argument for config

* Renaming of static peers to boot nodes for consistency

* Fix of name change

* Cluster config is now cluster data

* Load static nodes from configuration

* Final renaming of var for file content
2018-03-19 17:22:09 +01:00
Pedro Pombeiro ec83efc26a
Add two Whisper v6 full nodes to `staticpeers.json` (#721) 2018-03-02 16:48:45 +01:00
Pedro Pombeiro 6cdea4ef97
Update codebase to leverage Whisper v6 (#703)
* Update project to use Whisper v6. Part of #638

* Revert "Add patch to downgrade usage of Whisper v6 to v5 in some geth 1.8.1 vendor files. Part of #665" - this reverts commit 6aefb4c8fd02dbcfffac6b69e8bb22b13ef86b6b.

* Enable light mode on Whisper v6 for non-mail servers. Part of #638

* Fix race condition in whisperv6/peer.go. Part of #665 (PR already accepted upstream for 1.8.2)

* Update bootnode addresses in staticnodes.json. Part of #638

* Add `shh.lightclient` flag and tests for bloom filter setting logic. Part of #638

* Move MakeTestNodeConfig to utils. Part of #638

* Reduce PoW in `whisper_jail_test.go` to fix flaky test. Part of #638
2018-03-02 10:25:30 +01:00
Adam Babik 0a15c9ff9a
Configure mainnet nodes (#701)
Add mainnet static peers to static/config/staticpeers.json.
2018-02-26 19:36:04 +01:00
Pedro Pombeiro 134d01fcc1
Fix incorrect path in Makefile and run `make generate`. Part of #692 2018-02-22 12:19:31 +01:00
Igor Mandrigin c06d58addd Remove `notifications` package from go-ethereum. 2018-02-22 11:10:21 +01:00
Ivan Daniluk f93cd81d83
Upgrade to geth 1.7.3 and add geth patches (#492) 2018-01-15 21:26:41 +01:00
Evgeny Danilenko 919ab285d3 add password for MailServer encryption (#494) 2017-12-07 17:58:11 +01:00
Adam Babik ac76e4731f
Prepare wnode-status to support MailServer (#477) 2017-12-04 17:11:14 +01:00
perissology f0beeb3829 fix failing tests in PRs from other repos, fixes #459 (#461)
Fix failing tests for users w/o access to the ACCOUNT_PASSWORD env variable. Disable e2e public network tests in Travis pull requests. Exclude lib dir from unit tests.
2017-11-20 10:21:30 -08:00
Evgeny Danilenko 209d5fed74 Issue/fix public testnet tests #424 (#438)
* Accounts are now encrypted with a strong passphrase.
* The encrypted passphrase is stored in .travis.yml
* Public testnet tests are now run after tests on a private net
2017-11-07 12:46:11 -05:00
Adam Babik 987cdd6221 Update Ropsten CHT to number 478 (#428) 2017-10-29 16:44:44 +03:00
Eugene 4aaeeb6ebf
Extract notification into separete package and interface 2017-10-22 20:39:51 +03:00
Adam Babik ad9a8777a8 fix node unit tests (#416)
Fix tests that broke after introducing #414
2017-10-19 16:45:14 +03:00
Adam Babik 2e9302c631 removed CHT root from Rinkeby (#414)
CHT roots for Rinkeby do not work as their presence prevents status-go from even starting synchronization.
2017-10-19 15:02:42 +03:00
screwyprof 26fcfda87c Introduced a private Ethereum network for testing purposes (#397)
1. A new private testing network (StatusChain) is introduced.
2. An example test case implemented (TestSendEtherOnStatusChainTx)
3. Genesis file added static/config/status-chain-genesis.json
2017-10-17 00:54:56 +03:00
Ivan Tomilov 88294b29be Removed homestead bootnodes (#394)
Remove homestead bootnodes from cht.json. Their existence reflected the reality incorrectly as we don't have any homestead bootnodes in our cluster.
2017-10-12 12:46:41 +03:00
Adam Babik 90acfedf7a Rebase geth 1.7.0 (#353) 2017-10-10 12:38:49 +03:00
Ivan Tomilov fc8f59e121 Fixed eth_sendTransaction routing to the local node (#351)
* Fixed eth_sendTransaction routing to the local node
* Add local RPC handlers for eth_accounts and eth_sendTransaction
2017-09-25 18:04:40 +02:00
Ivan Tomilov 5f19c9cd0a Implemented sendAsync for js commands with a callback (#321)
Geth js commands coming through jail with a callback will now be executed truly asynchronously blocking jail only when an actual interaction with VM is performed.

Technically, it registers a new handler jeth.sendAsync which executes functions with callbacks asynchronously.

Changes include:

1. Send and SendAsync now use cell.VM instead of otto.Otto providing proper locking.
2. Unmarshalling in ExecuionPolicy.ExecuteWithClient is now done into var result interface{} instead of var result json.RawMessage because test case 0 of TestJailWhisper failed providing byte codes instead of 5.0.
3. Due to the asynchronous nature of web3 calls new weird timeouts in tests have been introduced. They may fail sometimes but I gave up trying to implement a more reliable and readable solution.
2017-09-18 15:13:32 +03:00
Ivan Tomilov c85525e139 Extended testGetDefaultConfig to support Rinkeby (#241)
This pull requests only extends testGetDefaultConfig to run against Rinkeby as well and fixes TestJailWhisper.
2017-09-08 15:32:02 +03:00
Ivan Tomilov 9463d3843a Updated CHT (#279)
Syncing for Ropsten should now start from something around block 1617921 and from 839680 for Rinkeby.
2017-09-07 10:54:33 +03:00
Adam Babik a4d2fdfa4a cht: remove downloading CHT from GH but embed it (#270) 2017-08-25 10:56:54 +01:00
Ivan Tomilov ebd77aabe2 Merging bug/whisper-on-geth1.6.1 (#236) which acts like develop
* static: updates Whisper test (to work with Geth 1.6.1)
* jail: VM persistence implemented
* jail: sendMessage/showSuggestions minor fixes (to be squashed)
* node: CHT and boot nodes auto-load implemented
* Replaced CHT data file from farazdagi's to tiabc's
* Rewrote config_test.go using testify having reduced it twice in size
* Increased SyncTime and panic timeout in tests
* Fixed test - remove go default test to testify/suite (#207)
* Add flag setup for RPCEnabled and add comment (#225)
* jail: register method handlers before running initial js in jail (#226)
* Console Jail Mod #179 (#228)
* Added ./statusd-data into .gitignore
* Increased log level for the test node from INFO to ERROR
* Add call to loop.Run to evaluate all setTimeout/setIntervals methods. (#208)
* Rebase onto geth1.6.7 (#232)
* Got back sync duration from 60s to 30s, updated bindata.go
2017-08-04 23:14:17 +07:00
Victor Farazdagi bc3ea62eca config: add CHT root URL, fixes #170 2017-05-27 23:42:03 +03:00
Victor Farazdagi 35eb15fed7 api: make is completely sequential 2017-05-25 15:34:13 +03:00
Victor Farazdagi 349103de1a major refactoring: node manager, tx queue, accounts, jail
- node: signals and node reset, fixes #152
- tests update (testify is used)
- node manager refactored, race conditions fixed
- node wrapper has been removed, we rely on go-ethereum node now
- tx queue refactored
- jail refactored
- backend and API introduced
2017-05-24 00:29:20 +03:00
Victor Farazdagi b0c3489d22 params: ropsten dev/prod config updated, fixes #167 2017-05-17 15:47:39 +03:00
Victor Farazdagi 82fe888853 whisper5: minor fixes 2017-05-16 11:02:34 +03:00
Victor Farazdagi 4f27a79d83 config: homestead and rinkeby confg + allow dev/prod modes 2017-05-16 10:51:11 +03:00
Victor Farazdagi 88394690c2 cmd/statusd: faucet sub-command added, refs #159 2017-05-16 10:51:11 +03:00
Victor Farazdagi 9a7caa6b58 config: expose bootstrap cluster config, closes #139 2017-05-16 10:51:11 +03:00
Victor Farazdagi 8876fd04d3 whisper: JavaScript tests to cover updated Whisper API 2017-05-14 02:34:08 +03:00
Victor Farazdagi ecea845d88 rebase: minor fixes to integrate with Geth 1.6.1 2017-05-03 16:00:36 +03:00
Victor Farazdagi 21344af199 geth, jail: embed static resources, closes #131 2017-04-28 11:50:12 +03:00