Commit Graph

268 Commits

Author SHA1 Message Date
RichΛrd 192ca6e319
feat: add function to free memory (#2344) 2021-08-31 15:17:44 -04:00
Samuel Hawksby-Robinson 78892847ee Removed unnecessary dependency from logutil package
util packages should have a few dependencies as possible, particularly from within the same repo, to prevent the application from creating import loops

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-19 11:35:40 +02:00
Andrea Maria Piana a4701f41ee linting and remove unused code 2021-07-20 10:57:38 +02:00
Andrea Maria Piana d84da4f515 Fix some tests and restore private/public RPC client 2021-07-20 10:57:38 +02:00
Andrea Maria Piana 4b0daeb47b Move services to status-node
Move all the services to status-node, as some of them were there, some
of them were in the geth backend and scattered around.
2021-07-20 10:57:38 +02:00
Andrea Maria Piana 188eabef0b Add populate db script
This commit adds a populate db script, used by QA, but also very useful
to test Login/Logout flows.
2021-07-20 10:57:38 +02:00
RichΛrd 40359f9c1b
go-waku integration (#2247)
* Adding wakunode module
* Adding wakuv2 fleet files
* Add waku fleets to update-fleet-config script
* Adding config items for waku v2
* Conditionally start waku v2 node depending on config
* Adapting common code to use go-waku
* Setting log level to info
* update dependencies
* update fleet config to use WakuNodes instead of BootNodes
* send and receive messages
* use hash returned when publishing a message
* add waku store protocol
* trigger signal after receiving store messages
* exclude linting rule SA1019 to check deprecated packages
2021-06-16 16:19:45 -04:00
Andrea Maria Piana d50fee6bb2 Handle connection state 2021-05-21 07:22:58 +02:00
Andrea Maria Piana cf6ef3171a Add mailserver logic 2021-05-21 07:22:58 +02:00
Andrea Maria Piana c55659b4f6
Add community notifications (#2160)
This commit introduces the following changes:

- `local-notifications` require as body an interface complying with
`json.Marshaler`
- removed unmarshaling of `Notifications` as not used (we only Marshal
notifications)
- `protocol/messenger.go` creates directly a `Notification` instead of
having an intermediate format
- add community notifications on request to join
- move parsing of text in status-go for notifications
2021-03-31 18:23:45 +02:00
Andrea Maria Piana 84db2fb472
Revert "Revert "Expand Local Notifications to support multiple Notification types (#2100)""
This reverts commit 5887337b88.
2021-02-05 16:48:45 +02:00
Andrea Maria Piana 5887337b88
Revert "Expand Local Notifications to support multiple Notification types (#2100)"
This reverts commit 46157dc4dc.
2021-01-26 09:40:05 +01:00
Andrea Maria Piana a1b3e3a772
Clean topics that we don't listen to
There was a bug on status-react where it would save filters that were
not listened to.
This commit adds a task to clean up those filters as they might result
in long syncing times.

This commit also returns topics/ranges/mailserves from messenger in
order to make the initialization of the app simpler and start moving
logic to status-go.

It also removes whisper from vendor.
2021-01-26 09:39:57 +01:00
Samuel Hawksby-Robinson 46157dc4dc
Expand Local Notifications to support multiple Notification types (#2100)
* Initial work on expanding Local Notifications

Adding functionality to support multiple notification types in Notification.Body. Currently have a bug that I think is caused by a the jsonMarshal func not working as intented, need to resolve this next before proceeding

* Fixed json.Marshaller issue and implemented json.Unmarshaller

* Tweak errors, go convention is errors don't begin with capital letters

* Added notificationMessageBody with un/marshalling

Also removed the Body interface

* Added check for bodyType mismatch

* Implement building and sending new message notifications

* Refactor to remove cycle imports

* Resolved linting issue ... Hopefully

* Resolving an implicit memory aliasing in a for loop

* version bump

* Added Notification.Category consts
2021-01-12 14:28:27 +00:00
Andrea Maria Piana 401d18e558
Enabled push notifications by default and add fleet 2020-08-27 18:54:06 +02:00
Jakub Sokołowski 3996aa7853 node-canary: switch to using Waku instead of Whisper
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-08-26 09:25:54 +02:00
Andrea Maria Piana d09eaeacb8 Allow node to start a push notification server
This commit allows a node to start a push notification server.
If the config is set it will start a messenger with a corresponding pn
server.
2020-08-20 10:54:33 +02:00
Andrea Maria Piana cbaf523b0e Address feedback 2020-06-03 11:07:04 +02:00
Andrea Maria Piana de698e64a3 Fix export & handle unsafe.pointer types 2020-06-03 11:07:04 +02:00
Andrea Maria Piana b8d64cbb1e Generate bindings dynamically
Before we had two directories `lib/` and `mobile/` that generated
respectively the bindings for desktop and ios/android.

This needed to be kept in sync and there was a fair amount of code
duplication, plus some missing methods on one or the other side.

I have made a change so the whole `lib/` namespace is generated by
parsing the `AST` of `mobile`, and bindings are generated before
compiling.
2020-06-03 11:07:04 +02:00
Adam Babik 6fdeb9b115
Fix listen flag (#1897) 2020-03-10 13:40:35 +01:00
Adam Babik b33ad8147d
Add example to demonstrate Whisper-Waku bridge (#1875) 2020-02-26 20:36:25 +01:00
Jakub Sokołowski 6b5e97d148 statusd: use eth.prod fleet by default
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-02-11 16:11:41 +01:00
Adam Babik 8b61d92d58
Remove protocol and eth-node submodules (#1835) 2020-02-10 12:22:37 +01:00
Pedro Pombeiro 741f43cb0c Fix wrong field being logged in `EnvelopesMonitor` 2020-01-23 17:19:10 +01:00
Adam Babik 79b8112f89
Split shhext into shhext and wakuext (#1803) 2020-01-20 21:56:06 +01:00
Adam Babik 37a508a97b
Integrate Waku service (#1759) 2020-01-08 12:12:23 +01:00
Pedro Pombeiro c8a911ebd1 Use goimports instead of gofmt 2020-01-06 10:17:23 +01:00
Adam Babik 549fe06339
fix linter gocyclo error for bootnode 2019-12-06 10:59:29 +01:00
Jakub Sokołowski 99a5a4d7b7 add -genkey flag to bootnode for making new keys
also add -writeaddress for getting the public node address

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-12-04 13:44:05 +01:00
Pedro Pombeiro 26b6d7c36a Create a home submodule for Eth node bridges- Rename StatusBackend to GethStatusBackend 2019-11-27 17:02:09 +01:00
Pedro Pombeiro ed5a5c154d
Move to monorepo structure (#1684)
Move to a monorepo structure with submodules

- Rename status-protocol-go to status-go/protocol
2019-11-21 17:19:22 +01:00
Jakub Sokołowski 46cd6e5118 add missing keystore init for node-canary
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-10-30 11:53:58 +01:00
Pedro Pombeiro c874960215
Use status-protocol-go/bridge/geth (#1638)
* Use status-protocol-go/bridge/geth
2019-10-14 09:53:38 +02:00
Jakub Sokołowski 4fe317917e create a /metrics endpoint for Prometheus
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-10-11 10:55:08 +02:00
Jakub Sokołowski 294399916e add script for easier setup of mailserver
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-10-10 11:27:32 +02:00
Pedro Pombeiro a2f106e4c5
Update for status-protocol-go API changes (#1627) 2019-10-09 16:22:53 +02:00
Adam Babik 26880b83d7
Upgrade geth to 1.9.5 and Whisper (#1617) 2019-10-04 17:21:24 +02:00
Adam Babik 3dbcb6a19e
Initialize keystore in statusd (#1592)
This reverts commit 0b403ce25b because otherwise the CI fails.
2019-08-31 16:55:05 +02:00
Adam Babik 1c33220834 update linter 2019-06-12 13:12:00 +02:00
Dmitry Shulyak 442a12e996
Removing discovery-proxy from sources (#1445)
It was going to be used as proxy from discv5 to rendezvous for les nodes.
We are not using it at the moment, and not clear when we will be using it
and if we are going to use it all.

Additionally it has some memory leaks that need to be fixed, so it is better
to remove it for now and restore once/if we will need it again.
2019-04-18 10:39:55 +03:00
Dmitry eeaf669006 Add goerli testnet 2019-03-19 10:56:09 +02:00
Dmitry Shulyak cf8f20b23c Use lumberjack library to rotate logs and add config that enables it (#1399) 2019-03-01 14:37:13 +01:00
Adam Babik d20b5dc3b3
mailserver: build pruning into WMailServer (#1342)
MailServer pruning was implemented as a separate command but that required stopping a mail server and executing the command manually. This change builds pruning into MailServer and can be set using MailServerDataRetention in WhisperConfig.
2019-01-23 14:32:45 +01:00
Dmitry Shulyak 863d07f798 Accept status service config directly from json input (#1337) 2019-01-17 13:56:22 +01:00
Adam Babik fd48ad8b29
geth: upgrade geth to 1.8.21 (#1350) 2019-01-16 15:42:00 +01:00
Adam Babik b1f9030177
update to geth v1.8.20 (#1327) 2018-12-19 11:02:07 +01:00
Adam Babik 913dbfca9b
Support Mail Server data synchronization (#1302)
These changes add a support for syncing data between two Mail Servers. It does not give an easy way to start syncing. This will be solved in the next PR.
2018-12-06 10:48:28 +01:00
Dmitry Shulyak a609b468fe
Mail peer store and connection manager (#1295)
This change implements connection manager that monitors 3 types of events:
1. update of the selected mail servers
2. disconnect from a mail server
3. errors for requesting mail history

When selected mail servers provided we will try to connect with as many as possible, and later disconnect the surplus. For example if we want to connect with one mail server and 3 were selected, we try to connect with all (3), and later disconnect with 2. It will to establish connection with live mail server faster.

If mail server disconnects we will choose any other mail server from the list of selected. Unless we have only one mail server. In such case we don't have any other choice and we will leave things as is.

If request for history was expired we will disconnect such peer and try to find another one. We will follow same rules as described above.

We will have two components that will rely on this logic:
1. requesting history

If target peer is provided we will use that peer, otherwise we will request history from any selected mail server that is connected at the time of request.

2. confirmation from selected mail server

Confirmation from any selected mail server will bee used to send a feedback that envelope was sent.

I will add several extensions, but probably in separate PRs:
1. prioritize connection with mail server that was used before reboot
2. disconnect from mail servers if history request wasn't expired but failed.
3. wait some time in RequestsMessage RPC to establish connection with any mail server

Currently this feature is hidden, as certain changes will be necessary in status-react. 

partially implements: https://github.com/status-im/status-go/issues/1285
2018-12-05 15:57:05 +02:00
Pedro Pombeiro d3c8079075 Update mailserver canary to be compatible with geth 1.8.17 upgrade 2018-11-20 08:59:36 +01:00
Adam Babik 04ba4d44e8
fix collecting p2p/Peers metric (#1269) 2018-11-12 10:27:09 +01:00
Pedro Pombeiro 5d0ed0303e Add timeout parameter to node-canary 2018-11-09 17:02:55 +01:00
Pedro Pombeiro 87f1af0743 Ensure that `NewNodeConfigWithDefaults` builds a valid configuration 2018-11-09 17:02:55 +01:00
Ivan Daniluk 4d5f808085
Add IPC path command line flag (#1229) 2018-10-11 12:29:59 +02:00
Ivan Daniluk 802197879a
Remove statusd-cli (#1231) 2018-10-10 00:57:53 +03:00
Dmitry Shulyak fbd3c9ca80
Limit number of proxied records and proxy only recently observed nodes (#1227)
* Limit number of proxied records and proxy only recently observed nodes

* Explicitly verify that limit is preserved and records removed eventually
2018-10-08 08:24:39 +03:00
Adam Babik fc54e0b06c
make node-canary working again (#1221)
* removed fleet validation (no fleet is allowed)
2018-10-04 12:40:26 +02:00
Dmitry 5b551c67fa Use status-im whisper/whisperv6 module everywhere 2018-10-03 09:27:51 +03:00
Adam Babik 298939b5fd
add networkid as switch between ropsten/mainnet is often (#1209) 2018-09-26 08:49:31 +02:00
Andrea Maria Piana dcaf8caed0
Add x3dh key exchange (#1127)
* Add x3dh key exchange

* Encrypt using the double ratchet

* Multi device with auto-pairing

* Add pfs enabled flag
2018-09-24 20:07:34 +02:00
Adam Babik 8aef7c4f15
tune CLI for better docs experience (#1207) 2018-09-21 16:09:31 +02:00
Pedro Pombeiro 5d43eac31c
Add missing changes to node-canary. Part of #1202 (#1205) 2018-09-20 19:15:35 +02:00
Pedro Pombeiro 518234803a
Add support for testing static nodes to node-canary. Closes #1202 (#1203) 2018-09-20 13:25:32 +02:00
Adam Babik fd99487328
fix updating peers limit if LES is enabled in CLI (#1199) 2018-09-19 20:29:42 +02:00
Pedro Pombeiro 452b7e2198
Disable listening on port 30303 for the canary (#1198) 2018-09-19 14:45:43 +02:00
Pedro Pombeiro 3d00af7fa3
Streamline configuration in status-go. Part of #1180 (#1183)
- Replace command line flags with `-c` config flag. Part of #1180
- Convert node config private keys to hex-encoded string versions.
- Remove `GenerateConfig` from library.
- Remove unused `FirebaseConfig` from library.
- Fix loading of `config/status-chain-genesis.json` in non-dev machines.
2018-09-13 18:31:29 +02:00
Dmitry 0068917ecb Run statusd not in a standalone mode by default 2018-08-31 12:45:44 +03:00
Adam Babik 776248a42f
Prevent RendezvousNodes to be overwritten by empty cli flag (#1174) 2018-08-27 16:10:58 +02:00
Dmitry Shulyak 6858662d59
Proxy records found in ethereum discovery v5 (#1165) 2018-08-24 06:17:32 +03:00
Adam Babik cf21f981f7
Fix rendezvous loop when discovery protocol is stopped; add advertiseaddr flag (#1151) 2018-08-17 08:25:55 +02:00
Sebastian Delgado 4afd9e6c6c Remove transactions queue 1027 (#1125)
Remove `PendingSignRequests` queue from the sign module.

This closes #1027 by removing the pending sign requests queue dependency from the SendTransaction, SignMessage and Recover.
2018-08-16 13:37:53 +02:00
Pedro Pombeiro 9587d030a6 mailserver-canary: Disable fleet specification 2018-08-08 11:21:13 +02:00
Pedro Pombeiro fb1e30c5f4 Disable NTP sync on the remaining e2e tests 2018-08-07 18:02:42 +02:00
Pedro Pombeiro 840b5b64d2 Implement mailserver canary service. Closes #1086 2018-08-07 17:17:59 +02:00
Adam Babik 408ba5a7e6
Configure eth.staging fleet (#1111)
Provide a way to run a status node with a different fleet.
2018-07-25 16:03:35 +02:00
Dmitry eeca435064 Add rendezvous implementation for discovery interface
Update vendor

Integrate rendezvous into status node

Add a test with failover using rendezvous

Use multiple servers in client

Use discovery V5 by default and test that node can be started with rendezvous discovet

Fix linter

Update rendezvous client to one with instrumented stream

Address feedback

Fix test with updated topic limits

Apply several suggestions

Change log to debug for request errors because we continue execution

Remove web3js after rebase

Update rendezvous package
2018-07-25 15:10:57 +03:00
Adam Babik 38a60135b2 Add asymmetric key support for MailServer requests (#1075)
* add Asymmetric Key support for MailServer requests

* remove deprecated notice

* fix linter

* refactoring Whisper config related to MailServer

* fix race condition
2018-07-04 11:30:57 +02:00
Adrià Cidre 237aeea7b2
Centralize leveldb.OpenFile allowing Recover corrupted db everywhere (#1072) 2018-07-02 10:42:16 +02:00
Andrea Franz 385f3b3377
collect process metrics (#1051)
*  collect process metrics

* collect process metrics in a different goroutine
2018-06-25 17:21:04 +02:00
Adrià Cidre 5bbfabde94
New debug_postSync endpoint (#1046)
* [#ISSUE-1041] New debug_postconfirm endpoint

* move debug service api inside shhext service
2018-06-25 15:27:17 +02:00
Adrià Cidre 56cc3d20da
First step to remove StatusAPI (#1034) 2018-06-19 09:49:24 +02:00
Pedro Pombeiro 354e23aaf5 Fix issues reported by lint. Part of #1017 2018-06-14 13:52:51 +02:00
Adrià Cidre 765fb69d89
Add option to disable ntp sync to statusd (#1031) 2018-06-14 10:56:01 +02:00
Adrià Cidre 8141657ad7
[#1018] Add a flag to disable shell colors and move verbose logs to debug level (#1020) 2018-06-12 09:49:38 +02:00
Adrià Cidre d5be8c525d
[#856] move geth subpackages to root level (#1007) 2018-06-08 13:29:50 +02:00
Adam Babik 8ce6efc749
Make statusd distinct from StatusIM client (#1008) 2018-06-08 11:20:30 +02:00
Dmitry 68966af049 Allow to set key hex as a flag 2018-06-07 13:53:04 +03:00
Adrià Cidre c82d5635b6
Allow gas and gas input as argument on signing TTXX (#1000)
* Allow gas and gas input as argument on signing TTXX

* Two different bindings ApproveSignRequest and ApproveSignRequestWithArgs
2018-06-06 09:35:32 +02:00
Dmitry Shulyak 5323af7da2 Fill new bootnode dht using already running bootnodes 2018-05-30 10:45:19 +03:00
Frank Mueller c41ae530c3
Add parameter for API modules (#997) 2018-05-28 18:37:09 +02:00
Dmitry Shulyak ed1cdf0418
Custom status bootnodes (#968)
This change makes invalidation mechanism more aggressive. With a primary goal to invalidate short living nodes faster. In current setup any node that became known in terms of discovery will stay in this state until it will fail to respond to 5 queries. Removing them earlier from a table allows to reduce latency for finding required nodes.

The second change, one adds a version for discovery, separates status dht from ethereum dht.
After we rolled out discovery it became obvious that our boot nodes became spammed with irrelevant nodes. And this made discovery process very long, for example with separate dht discovery takes ~2s, with mutual dht - it can take 1m-10m and there is still no guarantee to find a max amount of peers, cause status nodes is a very small part of whole ethereum infra.

In my understanding, we don't need to be a part of ethereum dht, and lower latency is way more important for us.

Closes: #941
Partially closes: #960 (960 requires futher investigations on devices)
2018-05-18 16:43:07 +03:00
Andrea Franz 4317b8f687
add cmd/statusd-prune (#957)
add mailserver cleaner

use memstorage for leveldb in tests

avoid write if batch size is 0

add comments

add cmd/statusd-prune

rmeove batch size var in prune method

validate range values

pass only flag name to missingFlag

refactor Cleaner.prune method

update batch not to be a pointer

removed extra batch counter increment

don't increment counter if batch returns errors

add README
2018-05-17 19:24:00 +02:00
Ivan Daniluk 4b17ea1ced
Exit when extra/errorneous flags are set (#965) 2018-05-17 16:03:23 +03:00
Adam Babik 6da469140f
Refactor in-app metrics (#963)
Remove Prometheus and other metric types and use metrics package from go-ethereum.
2018-05-16 17:36:59 +02:00
Dmitry Shulyak 375d5ec8c3
Explicitly disable discovery (#886)
* Make it possible to explicitly disable discovery

Discovery will be disabled in following cases:
- if there are not bootnodes - v5 server will be disabled
because there is no point in running it
- if user defined in config NoDiscovery=true this value will be preserved
even if we have bootnodes

So, basically discovery will be always enabled by default on mobile, unless
it is explicitly specified otherwise.

When statusd is used current behavior is that discovery is disabled by default.
I kept it in this change, but it would be better to change it.

* Fix leftovers

* Add wait group to peer pool to protect from races with p2p.Server

* Change fields only when all goroutines finished

* Turn off discovery after topic searches are stopped

* Don't set period to nil to avoid race with SearchTopic

* Close period chan only when all writers are finished
2018-05-10 14:45:51 +03:00
Andrea Franz f7dac0bcc4
add ipcfile flag to specify IPC file path (#896) 2018-05-10 01:59:05 +02:00
Andrea Franz c673148bf4
add `-status` flag to enable the Status service (#902)
* add `-status` flag to enable the Status service

* remove status from default APIModules and add it only from statusd if specified

* remove AddAPIModule method

* allow -status flag values to be http or ipc
2018-05-08 23:57:29 +02:00
Frank Mueller c85e0802eb
Remove mention of DevMode (#869)
* Remove mention of DevMode

* Remove prodMode
2018-04-26 19:59:57 +02:00
Andrea Franz 5f45f6498d
Add LogEnabled attribute to NodeConfig to, used in the call from status-react (#866)
* add LogEnabled attribute to NodeConfig, used in the call from status-react

* fix use of OverrideRootLogin in tests

* enable logger in tests based on LogLevel

* move LogEnabled before LogFile
2018-04-26 18:28:42 +02:00
Adam Babik 5f075eeaba
Enable logs in tests (#852)
Pass `-log` option to `go test` in order to change the log level. By default, it's `INFO`.
2018-04-22 18:50:34 +02:00