Commit Graph

2115 Commits

Author SHA1 Message Date
Samuel Hawksby-Robinson 07e46714f0
Anon Metrics Broadcast (#2198)
* Protobufs and adapters

* Added basic anon metric service and config init

* Added fibonacci interval incrementer

* Added basic Client.Start func and integrated interval incrementer

* Added new processed field to app metrics table

* Added id column to app metrics table

* Added migration clean up

* Added appmetrics GetUnprocessed and SetToProcessedByIDs and tests

There was a wierd bug where metrics in the db that did not explicitly insert a  value would be NULL, so could not be found by . In addition I've added a new primary id field to the app_metrics table so that updates could be done against very specific metric rows.

* Updated adaptors and db to handle proto_id

I need a way to distinguish individual metric items from each other so that I can ignore the ones that have been seen before.

* Moved incrementer into dedicated file

* Resolve incrementer test fail

* Finalised the main loop functionality

* Implemented delete loop framework

* Updated adaptors file name

* Added delete loop delay and quit, and tweak on RawMessage gen

* Completed delete loop logic

* Added DBLock to prevent deletion during mainLoop

* Added postgres DB connection, integrated into anonmetrics.Server

* Removed proto_id from SQL migration and model

* Integrated postgres with Server and updated adaptors

* Function name update

* Added sample config files for client and server

* Fixes and testing for low level e2e

* make generate

* Fix lint

* Fix for receiving an anonMetricBatch not in server mode

* Postgres test fixes

* Tidy up, make vendor and make generate

* delinting

* Fixing database tests

* Attempted fix of does:  cannot open `does' (No such file or directory)
not:   cannot open `not' (No such file or directory)
exist: cannot open `exist' (No such file or directory) error on sql resource loas

* Moved all anon metric postgres migration logic and sources into a the protocol/anonmetrics package or sub packages. I don't know if this will fix the does:  cannot open `does' (No such file or directory)
not:   cannot open `not' (No such file or directory)
exist: cannot open `exist' (No such file or directory) error that happens in Jenkins but this could work

* Lint for the lint god

* Why doesn't the linter list all its problems at once?

* test tweaks

* Fix for wakuV2 change

* DB reset change

* Fix for postgres db migrations fails

* More robust implementation of postgres test setup and teardown

* Added block for anon metrics functionality

* Version Bump to 0.84.0

* Added test to check anon metrics broadcast is deactivated

* Protobufs and adapters

* Added basic anon metric service and config init

* Added new processed field to app metrics table

* Added id column to app metrics table

* Added migration clean up

* Added appmetrics GetUnprocessed and SetToProcessedByIDs and tests

There was a wierd bug where metrics in the db that did not explicitly insert a  value would be NULL, so could not be found by . In addition I've added a new primary id field to the app_metrics table so that updates could be done against very specific metric rows.

* Updated adaptors and db to handle proto_id

I need a way to distinguish individual metric items from each other so that I can ignore the ones that have been seen before.

* Added postgres DB connection, integrated into anonmetrics.Server

* Removed proto_id from SQL migration and model

* Integrated postgres with Server and updated adaptors

* Added sample config files for client and server

* Fix lint

* Fix for receiving an anonMetricBatch not in server mode

* Postgres test fixes

* Tidy up, make vendor and make generate

* Moved all anon metric postgres migration logic and sources into a the protocol/anonmetrics package or sub packages. I don't know if this will fix the does:  cannot open `does' (No such file or directory)
not:   cannot open `not' (No such file or directory)
exist: cannot open `exist' (No such file or directory) error that happens in Jenkins but this could work
2021-09-01 13:02:18 +01:00
Roman Volosovskyi 8d7f26eb60
Register public and community chats for push notifications after creation 2021-09-01 12:30:42 +03:00
Anthony Laibe 048511cc99 feat: Add trait to collectible
Build a custom unmarshaller for field that can be int or string

see: https://docs.opensea.io/reference/asset-object
2021-09-01 09:56:26 +02:00
RichΛrd 192ca6e319
feat: add function to free memory (#2344) 2021-08-31 15:17:44 -04:00
RichΛrd 3463853f5f
fix: empty wakuv2 host (#2342) 2021-08-31 12:27:02 -04:00
Roman Volosovskyi 0c0e02e93a
Return counter for messages with mention which are marked as seen 2021-08-31 11:49:40 +03:00
RichΛrd 719a303b05
feat: wakuV2 bandwidth stats (#2325) 2021-08-30 17:35:37 -04:00
Jakub Sokołowski 034f3bfec3 get_enode.sh: --retry-connrefused instead --retry-all-errors
The `--retry-all-errors` flag was added only in Curl `7.71.0`:
https://github.com/curl/curl/commit/b995bb58

So it fails on older distrubutions with:
```
curl: option --retry-all-errors: is unknown
curl: try 'curl --help' or 'curl --manual' for more information
```

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-30 14:24:29 +02:00
RichΛrd 842ba229df
feat: add function to obtain bloomfilter (#2334) 2021-08-26 16:25:43 -04:00
RichΛrd fe086b2fdd
fix: allow empty category id when sorting chats and fix error messages (#2301)
* fix: allow empty category id when sorting chats and fix error messages
* fix: unordered maps
2021-08-25 12:52:28 -04:00
andrey 20363e32ef fix activechats ordering 2021-08-25 15:10:26 +02:00
Andrea Maria Piana 83394e0ed1
Add chatID to removed messages 2021-08-25 12:21:40 +03:00
Jakub Sokołowski 5c408f8ce4 compose/systemd: fix completion of make targets
Something goes wrong when parsing and GRN definition is treated as target.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-24 16:45:47 +02:00
Jakub Sokołowski 06587bd1bc get_enode.sh: add retries to account for slow start
Containers can be slow to start, but still want that enode.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-24 16:45:47 +02:00
Jakub Sokołowski b631b93bd1 compose: upgrade image, fix RPC listening address
Also move data dir to folder named after contaienr.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-24 16:45:47 +02:00
Jakub Sokołowski d2b379bb27
Bump version to 0.84.0
This release does not contain a breaking change. The intention behind
incrementing the minor version is to make a clean break from broken
versions and make it easier to remember the name of recent working
release.

This specifically targets the following fix:
https://github.com/status-im/status-go/pull/2331

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-24 15:42:28 +02:00
Andrea Maria Piana 2c96475aac [Fixes: #2328] Fixes iterator on leveldb
Leveldb did not support topics only queries.
This commit changes the behavior so that now we use the topics map if
present.
2021-08-24 15:40:40 +02:00
Jakub Sokołowski cd2b53643d rcp.sh: curl wrapper for easier RPC calls
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-24 11:23:25 +02:00
Jakub Sokołowski e4540a0853 add 3 instead of 2 static nodes to default configs
Each fleet has 3 data centers, so it makes sense to use 3.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-24 11:23:25 +02:00
Jakub Sokołowski 315f63ea43 gen_config.sh: default HTTPHost to localhost
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-24 11:23:25 +02:00
Jakub Sokołowski b805d0b9f6 gen_config.sh: add MaxPeers default of 50
I think this setting should be more explicit and visible in the config.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-24 11:23:25 +02:00
Jakub Sokołowski 714eb416d2 metrics: fix p2p_peers_count metrics labels
Because `p2p.Server.Name()` function started returning an abbreviated
name of the node this broke the parsing of the name in order to fill in
the labels for `p2p_peers_count` metric, resulting in metrics like this:
```sh
 > curl -sS localhost:9090/metrics | grep '^p2p_peers_count'
p2p_peers_count{platform="v0.79.0",type="Statusd",version="unknown"} 3
```
Caused by value returned from `Name()` to look like this:
```
Statusd/v0.79.0/linu...
```
By using `Fullname()` we are sure we are pasing all the segments.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-24 11:07:39 +02:00
Brian Sztamfater 0e54d04e67
Delete activity center notifications for deleted messages (#2319) 2021-08-23 11:23:55 -03:00
Anthony Laibe 8f9c644dae
feat: fetch assets from opensea (#2320) 2021-08-20 15:53:24 -04:00
Anthony Laibe dbd34ae7eb
feat: Re-enable unfurling for tenor (#2302) 2021-08-20 15:13:10 -04:00
andrey ab08042f21 preload chats 2021-08-20 15:29:09 +02:00
Roman Volosovskyi 45212b0823
[status-im/status-react#12450] Add push notification for deleted message 2021-08-20 11:26:46 +03:00
Andrea Maria Piana c51f9b800c
Bump version to 0.83.14 2021-08-20 10:09:04 +02:00
Andrea Maria Piana 482518d179 Fix last message deletion 2021-08-20 10:07:14 +02:00
saledjenic b8959e3f66
feature(@status-go/chat): implement search results for communities, channels (#2303)
AllMessagesFromChatsAndCommunitiesWhichMatchTerm method added which returns all messages which
match the search term, if they belong to either any chat from the chatIds array or any channel of
any community from communityIds array.

This api point is necessary for desktop issue 2934.

Fixes: #2934
2021-08-19 15:47:03 -04:00
Jakub Sokołowski 2b566db298 drop AdvertiseAddr from default configuration
It appears it is being used by Rendezvous, and if that protocol is not
being used there is no need to set `AdvertiseAddr` either.

I also adjusted all the `Makefile`s to not depend on `PUBLIC_IP` variable.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-19 16:53:31 +02:00
Roman Volosovskyi d1a581b18c
Fix eip1559 contract transactions 2021-08-19 13:29:54 +03:00
Jakub Sokołowski b22a261bc2 using zap.NewNop() ignores configured log level
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-19 11:35:40 +02:00
Samuel Hawksby-Robinson 78892847ee Removed unnecessary dependency from logutil package
util packages should have a few dependencies as possible, particularly from within the same repo, to prevent the application from creating import loops

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-19 11:35:40 +02:00
andrey bc63fa606e delete imported key 2021-08-18 12:56:53 +02:00
RichΛrd f6dc6f752a
use 23/WAKU2-TOPICS format for content topics (#2281) 2021-08-17 11:27:28 -04:00
andrey fc16588cf1 [#11468] Hide/show option for accounts in the wallet 2021-08-17 11:40:37 +02:00
Ismael 1efe023dd7 Add unfurling for Twitter links
Fix lint issues

Wrap json.Unmarshall() errors in urls.go

Bump version 0.83.9 -> 0.83.10
2021-08-16 20:03:10 +02:00
Andrea Maria Piana 7dfeda1511 Bump version to 0.83.9 2021-08-16 11:32:20 +02:00
RichΛrd dae447466f
fix: delete messages when the signal arrives before the OG msg (#2310)
* fix: delete messages when the signal arrives before the OG msg
* fix: code review
2021-08-13 13:10:32 -04:00
Jakub Sokołowski 5ac8b873b8 config/cli/fleet: disable Rendezvous, drop nodes
Since we've broken LibP2P communication with our Rendezvous nodes
somewhere after `v0.79.11` we should not enable it by defaults since it
generates errors like these:
```
failed to dial 16Uiu2...d687e4: all dials failed
failed to negotiate security protocol: protocol not supported
```
Since normal bootnodes are defined as well as a list of static nodes
there should not be a need for using Rendezvous as well.

Resolves: https://github.com/status-im/status-go/issues/2309

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-13 16:22:17 +02:00
Jakub Sokołowski 5caf22e35a scripts/gen_config.sh: drop WhisperConfig, limit API modules
I've disabled Whisper in the config since we do not use it anymore.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-13 16:22:17 +02:00
Roman Volosovskyi c78eac8d94
Add MaxFeePerGas and MaxPriorityFeePerGas to TransferView 2021-08-12 19:42:13 +03:00
Roman Volosovskyi 9f9f00b7a2
Adjust HashTransaction to eip1559
This commit fixed eip1559 tx signing on keycard.
2021-08-12 18:47:56 +03:00
RichΛrd 8a7fd22ceb
add index on user_messages to speed-up marking all messages as read (#2304) 2021-08-11 08:26:15 -04:00
Anthony Laibe ef014e25e2
feat: add gif recent and favorite to settings (#2296) 2021-08-09 08:51:05 -04:00
Samuel Hawksby-Robinson 89251e8416
Sync Communities (#2253)
* Added community sync protobuf

* Updated community sync send logic

* Integrated syncCommunity handling

* Added synced_at field and tidied up some other logic

* persistence testing

* Added testing and join functionality

* Fixed issue with empty scan params

* Finshed persistence tests for new db funcs

* Midway debug of description not persisting after sync

* Resolved final issues and tidied up

* Polish

* delint

* Fix error not handled on SetPrivateKey

* fix infinite loop, again

* Added muted option and test fix

* Added Muted to syncing functions, not just in persistence

* Fix bug introduced with Muted property

* Added a couple of notes for future devs

* Added most of the sync RequestToJoin functionality

Tests need to be completed and tests are giving some errors

* Finished tests for getJoinedAndPending

* Added note

* Resolving lint

* Fix of protobuf gen bug

* Fixes to community sync tests

* Fixes to test

* Continued fix of e2e

* Final fix to e2e testing

* Updated migration position

* resolve missing import

* Apparently the linter spellchecks

* Fix bug from  #2276 merge

* Bug fix for leaving quirkiness

* Addressed superfluous MessengerResponse field

* Addressed feedback

* VERSION bump
2021-08-06 16:40:23 +01:00
saledjenic c4a71f813a
feature(@status-go/chat): implement search on sqlcipher (status-go side) (#2282)
* feature(@status-go/chat): implement search on sqlcipher (status-go side)

Searching messages by some term for a specific channel is added on the side of status-go.

Fixes: #2912

* Linting

Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2021-08-04 15:31:44 -04:00
RichΛrd 0aab3e238b
chore: bump go-waku (#2292) 2021-08-03 16:40:22 -04:00
RichΛrd facad9f07e
feat: upload/download rate for waku v1 messages (#2286)
* feat: upload/download rate for waku v1 messages

* reorganize code

* fix failing test
2021-08-03 15:27:15 -04:00