Commit Graph

194 Commits

Author SHA1 Message Date
shashankshampi 14dcd29eee test_: Code Migration from status-cli-tests
author shashankshampi <shashank.sanket1995@gmail.com> 1729780155 +0530
committer shashankshampi <shashank.sanket1995@gmail.com> 1730274350 +0530

test: Code Migration from status-cli-tests
fix_: functional tests (#5979)

* fix_: generate on test-functional

* chore(test)_: fix functional test assertion

---------

Co-authored-by: Siddarth Kumar <siddarthkay@gmail.com>

feat(accounts)_: cherry-pick Persist acceptance of Terms of Use & Privacy policy (#5766) (#5977)

* feat(accounts)_: Persist acceptance of Terms of Use & Privacy policy (#5766)

The original GH issue https://github.com/status-im/status-mobile/issues/21113
came from a request from the Legal team. We must show to Status v1 users the new
terms (Terms of Use & Privacy Policy) right after they upgrade to Status v2
from the stores.

The solution we use is to create a flag in the accounts table, named
hasAcceptedTerms. The flag will be set to true on the first account ever
created in v2 and we provide a native call in mobile/status.go#AcceptTerms,
which allows the client to persist the user's choice in case they are upgrading
(from v1 -> v2, or from a v2 older than this PR).

This solution is not the best because we should store the setting in a separate
table, not in the accounts table.

Related Mobile PR https://github.com/status-im/status-mobile/pull/21124

* fix(test)_: Compare addresses using uppercased strings

---------

Co-authored-by: Icaro Motta <icaro.ldm@gmail.com>

test_: restore account (#5960)

feat_: `LogOnPanic` linter (#5969)

* feat_: LogOnPanic linter

* fix_: add missing defer LogOnPanic

* chore_: make vendor

* fix_: tests, address pr comments

* fix_: address pr comments

fix(ci)_: remove workspace and tmp dir

This ensures we do not encounter weird errors like:
```
+ ln -s /home/jenkins/workspace/go_prs_linux_x86_64_main_PR-5907 /home/jenkins/workspace/go_prs_linux_x86_64_main_PR-5907@tmp/go/src/github.com/status-im/status-go
ln: failed to create symbolic link '/home/jenkins/workspace/go_prs_linux_x86_64_main_PR-5907@tmp/go/src/github.com/status-im/status-go': File exists
script returned exit code 1
```

Signed-off-by: Jakub Sokołowski <jakub@status.im>

chore_: enable windows and macos CI build (#5840)

- Added support for Windows and macOS in CI pipelines
- Added missing dependencies for Windows and x86-64-darwin
- Resolved macOS SDK version compatibility for darwin-x86_64

The `mkShell` override was necessary to ensure compatibility with the newer
macOS SDK (version 11.0) for x86_64. The default SDK (10.12) was causing build failures
because of the missing libs and frameworks. OverrideSDK creates a mapping from
the default SDK in all package categories to the requested SDK (11.0).

fix(contacts)_: fix trust status not being saved to cache when changed (#5965)

Fixes https://github.com/status-im/status-desktop/issues/16392

cleanup

added logger and cleanup

review comments changes

fix_: functional tests (#5979)

* fix_: generate on test-functional

* chore(test)_: fix functional test assertion

---------

Co-authored-by: Siddarth Kumar <siddarthkay@gmail.com>

feat(accounts)_: cherry-pick Persist acceptance of Terms of Use & Privacy policy (#5766) (#5977)

* feat(accounts)_: Persist acceptance of Terms of Use & Privacy policy (#5766)

The original GH issue https://github.com/status-im/status-mobile/issues/21113
came from a request from the Legal team. We must show to Status v1 users the new
terms (Terms of Use & Privacy Policy) right after they upgrade to Status v2
from the stores.

The solution we use is to create a flag in the accounts table, named
hasAcceptedTerms. The flag will be set to true on the first account ever
created in v2 and we provide a native call in mobile/status.go#AcceptTerms,
which allows the client to persist the user's choice in case they are upgrading
(from v1 -> v2, or from a v2 older than this PR).

This solution is not the best because we should store the setting in a separate
table, not in the accounts table.

Related Mobile PR https://github.com/status-im/status-mobile/pull/21124

* fix(test)_: Compare addresses using uppercased strings

---------

Co-authored-by: Icaro Motta <icaro.ldm@gmail.com>

test_: restore account (#5960)

feat_: `LogOnPanic` linter (#5969)

* feat_: LogOnPanic linter

* fix_: add missing defer LogOnPanic

* chore_: make vendor

* fix_: tests, address pr comments

* fix_: address pr comments

chore_: enable windows and macos CI build (#5840)

- Added support for Windows and macOS in CI pipelines
- Added missing dependencies for Windows and x86-64-darwin
- Resolved macOS SDK version compatibility for darwin-x86_64

The `mkShell` override was necessary to ensure compatibility with the newer
macOS SDK (version 11.0) for x86_64. The default SDK (10.12) was causing build failures
because of the missing libs and frameworks. OverrideSDK creates a mapping from
the default SDK in all package categories to the requested SDK (11.0).

fix(contacts)_: fix trust status not being saved to cache when changed (#5965)

Fixes https://github.com/status-im/status-desktop/issues/16392

test_: remove port bind

chore(wallet)_: move route execution code to separate module

chore_: replace geth logger with zap logger (#5962)

closes: #6002

feat(telemetry)_: add metrics for message reliability (#5899)

* feat(telemetry)_: track message reliability

Add metrics for dial errors, missed messages,
missed relevant messages, and confirmed delivery.

* fix_: handle error from json marshal

chore_: use zap logger as request logger

iterates: status-im/status-desktop#16536

test_: unique project per run

test_: use docker compose v2, more concrete project name

fix(codecov)_: ignore folders without tests

Otherwise Codecov reports incorrect numbers when making changes.
https://docs.codecov.com/docs/ignoring-paths

Signed-off-by: Jakub Sokołowski <jakub@status.im>

test_: verify schema of signals during init; fix schema verification warnings (#5947)

fix_: update defaultGorushURL (#6011)

fix(tests)_: use non-standard port to avoid conflicts

We have observed `nimbus-eth2` build failures reporting this port:
```json
{
  "lvl": "NTC",
  "ts": "2024-10-28 13:51:32.308+00:00",
  "msg": "REST HTTP server could not be started",
  "topics": "beacnde",
  "address": "127.0.0.1:5432",
  "reason": "(98) Address already in use"
}
```
https://ci.status.im/job/nimbus-eth2/job/platforms/job/linux/job/x86_64/job/main/job/PR-6683/3/

Signed-off-by: Jakub Sokołowski <jakub@status.im>

fix_: create request logger ad-hoc in tests

Fixes `TestCall` failing when run concurrently.

chore_: configure codecov (#6005)

* chore_: configure codecov

* fix_: after_n_builds
2024-10-30 14:49:26 +05:30
Igor Sirotin c1dd9397f7
refactor_: remove generated files from source control (#5878)
* fix_: gitignore generated files

* chore_: delete generated files

* fix_: update go generate instructions

* feat(Makefile)_: clean-generated-files target

* feat(Makefile)_: `generate` target

* fix(Makefile)_: dependent generate target

* ci_: run generate, update docker file deps

* fix(Makefile)_: remove `clean-generated-files` target

* fix(Makefile)_: simpler GO_GENERATE_CMD arg

* fix_: temp workspace GO_GENERATE_FAST_DIR
2024-10-03 20:59:44 +01:00
frank 38308d48f2
feat_: log on panic (#5849)
* feat_: log error and stacktrace when panic in goroutine

* test_: add test TestSafeGo

* chore_: rename logAndCall to call

* chore_: rename SafeGo to Go

* chore_: make lint-fix

* chore_: use t.Cleanup

* chore_: Revert "chore_: use t.Cleanup"

This reverts commit 4eb420d179cc0e208e84c13cb941e6b3d1ed9819.

* chore_: Revert "chore_: make lint-fix"

This reverts commit fcc995f157e671a4229b47419c3a0e4004b5fdab.

* chore_: Revert "chore_: rename SafeGo to Go"

This reverts commit a6d73d6df583f313032d79aac62f66328039cb55.

* chore_: Revert "chore_: rename logAndCall to call"

This reverts commit 8fbe993bedb9fbba67349a44f151e2dd5e3bc4cc.

* chore_: Revert "test_: add test TestSafeGo"

This reverts commit a1fa91839f3960398980c6bf456e6462ec944819.

* chore_: Revert "feat_: log error and stacktrace when panic in goroutine"

This reverts commit f612dd828fa2ce410d0e806fe773ecbe3e86a68a.

* feat_: log error and stacktrace when panic in goroutine

* chore_: make lint-fix

* chore_: rename logAndCall to call

* chore_: renaming LogOnPanic

* chore_: update rest goroutine function calls

* chore_: make lint-fix
2024-09-27 06:37:32 +08:00
Azanul Haque 3ca2d6eb23
fix_: Orphaned generated code (#4086)
Signed-off-by: Azanul <azanulhaque@gmail.com>
2024-09-03 21:32:23 +01:00
Igor Sirotin f660be0daa
chore!: simplify local pairing API (#5643)
* chore!: switch to use CreateAccount, fix tests

* chore_: move GenerateInstallationID to multidevice

* chore!: clean up ReceiverConfig

* chore_: use DefaultKeystoreRelativePath

* chore_: cleanup

* fix_: import api

* chore_: lint fix

* chore_: remove unused import
2024-08-14 21:10:19 +01:00
frank f123e98179
feat: fallback pairing seed (#5614)
* feat(pairing)!: Add extra parameters and remove v2 compatibility

This commit includes the following changes:

I have added a flag to maintain 2.29 compatibility.

Breaking change in connection string

The local pairing code that was parsing the connection string had a few non-upgradable features:

It was strictly checking the number of parameters, throwing an error if the number was different. This made it impossible to add parameters to it without breaking.
It was strictly checking the version number. This made increasing the version number impossible as older client would just refuse to connect.
The code has been changed so that:

Two parameters have been added, installation-id and key-uid. Those are needed for the fallback flow.
I have also removed version from the payload, since it wasn't used.

This means that we don't support v1 anymore. V2 parsing is supported . Going forward there's a clear strategy on how to update the protocol (append parameters, don't change existing one).

https://www.youtube.com/watch?v=oyLBGkS5ICk Is a must watch video for understanding the strategy

Changed MessengerResponse to use internally a map of installations rather than an array (minor)
Just moving towards maps as arrays tend to lead to subtle bugs.

Moved pairing methods to messenger_pairing.go
Just moved some methods

Added 2 new methods for the fallback flow
FinishPairingThroughSeedPhraseProcess
https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R29

EnableAndSyncInstallation

https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R18

Flow for clients
Client A1 is logged in
Client A2 is logged out

Client A1 shows a QR code
Client A2 scans a QR code
If connection fails on A2, the user will be prompted to enter a seed phrase.
If the generated account matches the key-uid from the QR code, A2 should call FinishPairingThroughSeedPhraseProcess with the installation id passed in the QR code. This will send installation information over waku. The user should be shown its own installation id and prompted to check the other device.
Client A1 will receive new installation data through waku, if they are still on the qr code page, they should show a popup to the user showing the received installation id, and a way to Enable and Sync, which should call the EnableAndSyncInstallation endpoint. This should finish the fallback syncing flow.
Current issues
Currently I haven't tested that all the data is synced after finishing the flow. I see that the two devices are paired correctly, but for example the DisplayName is not changed on the receiving device. I haven't had time to look into it further.

* test_: add more test for connection string parser

* fix_: fix panic when parse old connection string

* test_: add comments for TestMessengerPairAfterSeedPhrase

* fix_: correct error description

* feat_:rename FinishPairingThroughSeedPhraseProcess to EnableInstallationAndPair

* fix_: delete leftover

* fix_: add UniqueKey method

* fix_: unify the response for InputConnectionStringForBootstrapping

* fix_: remove fields installationID and keyUID in GethStatusBackend

* fix_: rename messenger_pairing to messenger_pairing_and_syncing

---------

Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2024-07-30 17:14:05 +08:00
Andrea Maria Piana 9a59d6a459 feat(metrics)_: add centralized metrics
This commit adds support for centralized metrics.
There are two providers as of now, and we haven't quite decided which
one to go for, so for the time being both are supported.
It also introduces a new endpoint InitializeApplication that replaces
OpenAccounts
2024-07-11 10:05:31 +01:00
Patryk Osmaczko cb20c4c64a feat(communities)_: introduce bloom filter members list
iterates: status-im/status-desktop#15064
2024-07-04 17:54:29 +02:00
Mikhail Rogachev 1362d10620
feat_: implement connector service (#5375) (#5403) 2024-06-24 07:29:40 -07:00
Igor Sirotin 19875ed9b5
fix: archive decryption fail (#5076)
* fix_: delay archive import if decryption failed

* chore_: minor cleanup

* chore_: test archive import of encrypted channel

* test(TestImportDecryptedArchiveMessages)_: first iteration

* feat_: GetHashRatchetMessagesCountForGroup

* chore_: log community description and shard info message ids

* test(TestImportDecryptedArchiveMessages)_: cleanup

* fix_: rebase issues

* chore_: remove temporal test

* test(TestImportDecryptedArchiveMessages)_: cleanup

* chore_: lint fix

* fix(TestImportDecryptedArchiveMessages)_: ForceMembersReevaluation
2024-05-27 13:49:09 +01:00
Vitaly Vlasov 20d3a7b69b fix_: Use UnixMilli() 2024-05-20 14:39:46 +03:00
Igor Sirotin d8a5665871 fix_: encryption.GetCurrentTime implementation 2024-05-20 14:39:46 +03:00
Igor Sirotin da12c52596 fix_: loading hash_ratchet_encryption from database checks group_id 2024-05-20 14:39:46 +03:00
Mikhail Rogachev 6da423fc71
feat: Add an expiration and periodical publishing for community grants (#5024)
* feat_: add periodical publishing for community grants
feat_: Validate grant when receiving it
feat_: add expiration for grants
feat_: add test for grants expiration
fix_: move grants test to profile showcase, fix a few bugs
* feat_: use one group mesage to update grants
* chore_: review fixes
2024-04-17 16:53:51 +02:00
Mikhail Rogachev 1211b86151
chore_: update pb.go files according protoc version from nix (#5052) 2024-04-12 19:04:00 +02:00
Andrea Maria Piana 2311da2475 Port back migrations from release branch 2024-03-14 13:27:12 +00:00
Andrea Maria Piana 67dfff2324 Add index to hash ratchet & cache processed description 2024-03-13 11:35:11 +00:00
Igor Sirotin 7f671f7632
chore: make generate with custom `-modtime` (#4896) 2024-03-08 09:25:17 +00:00
Igor Sirotin 571f30777e
chore: make generate with nix-shell (#4853) 2024-03-01 13:58:24 +00:00
Igor Sirotin 9fa396e797
chore: nix-shell make generate (#4824) 2024-02-27 19:35:43 +00:00
Patryk Osmaczko eeaff0d3e3 fix: sync&backup channels encryption keys
fixes: status-im/status-desktop#13356
2024-02-26 16:41:30 +01:00
Mykhailo Prakhov 3959948c4c
chore: fix ban/unban flow and delete all messages feature (#4743)
feat: delete all messages for banned member and ban/unban AC notifications
2024-02-22 11:25:13 +01:00
Igor Sirotin add46fbda0
chore: `make generate` with nix-shell (#4745) 2024-02-16 12:48:27 +00:00
Mikhail Rogachev 3ea2002904
Feat: Modify grants for profile showcase use (#4694)
* chore: regenerate proto binaries

* feat: remove unused grant code for community's canPost
2024-02-11 15:11:04 +03:00
Andrea Maria Piana 605fe40e32 Fix encryption metadata issues #4613
This commit fixes a few issues with communities encryption:

Key distribution was disconnected from the community description, this created a case where the key would arrive after the community description and that would result in the client thinking that it was kicked.
To overcome this, we added a message that signals the user that is kicked. Also, we distribute the key with the community description so that there's no more issues with timing.
This is a bit expensive for large communities, and it will require some further optimizations.

Key distribution is now also connected to the request to join response, so there are no timing issues.

Fixes an issue with key distribution (race condition) where the community would be modified before being compared, resulting in a comparison of two identical communities, which would result in no key being distributed. This commit only partially address the issue.
2024-02-07 10:25:41 +00:00
Michal Iskierko 73a5189398 feat(BridgeMessage): Add new type of chat message content: BridgeMessage
BridgeMessage is a type of chat message content which will be sent from  Matterbridge.
It contains fields:
- bridge name - depends on the used bridge, eg. "discord", "slack", etc...
- user name - username the message was received from
- content - message content
- user avatar
- message id
- parent message id - used in case of replies

Message is saved to a separated table: bridge_messages, similarly to discord messages.
The user_messages table is untouched.
bridge_messages table contains user_messages_id in order to join with user_messages table.

Issue #13098
2024-01-30 12:14:54 +01:00
Mikhail Rogachev fb98ee93ce
Correct profile showcase tokens and collectibles (#4511)
* feat: add profile showcase collectibles & assets missing fileds

* feat: resolve collectible identification issue

* feat: add validation for collectible visibility relative account

* feat: separate profile showcase assets on verified and unverified tokens

* fix: make chainId uint64, comment collectible account check

* chore: re-generate protobuf binaries with right protoc version

* Update protocol/messenger_profile_showcase.go

Co-authored-by: Igor Sirotin <sirotin@status.im>
2024-01-25 20:48:27 +04:00
Andrea Maria Piana e65760ca85 Add basic peersyncing
This commit adds basic syncing capabilities with peers if they are both
online.

It updates the work done on MVDS, but I decided to create the code in
status-go instead, since it's very tight to the application (similarly
the code that was the inspiration for mvds, bramble, is all tight
together at the database level).

I reused parts of the protobufs.

The flow is:

1) An OFFER message is sent periodically with a bunch of message-ids and
   group-ids.
2) Anyone can REQUEST some of those messages if not present in their
   database.

3) The peer will then send over those messages.

It's disabled by default, but I am planning to add a way to set up the
flags.
2024-01-23 12:46:17 +00:00
Ibrahem Khalil 436d229856
Add last opened at key for community (#4514) 2024-01-21 12:55:14 +02:00
Dario Gabriel Lipicar 0d2c3cef7c feat: implement token management settings for collectibles 2024-01-18 11:46:35 -03:00
Ibrahem Khalil e1b52f9eae
Add joined at key to communities (#4513) 2024-01-09 20:36:47 +02:00
Godfrain Jacques 9d8b55e3d9
This PR (status-go) fixes #12821 (#4516)
- Add album_images column to the user_messages table
- Migrate the database
2024-01-08 10:18:57 -08:00
Patryk Osmaczko 1d3c618fb4 feat: encrypt `CommunityDescription` fields
Extended `CommunityDescription` with a `privateData` map. This map
associates each hash ratchet `key_id` and `seq_no` with an encrypted
`CommunityDescription`. Each encrypted instance includes only data
requiring encryption.

closes: status-im/status-desktop#12851
closes: status-im/status-desktop#12852
closes: status-im/status-desktop#12853
2023-12-22 18:17:06 +01:00
Mykhailo Prakhov 134137f9c5
feat: implemented a mechanism to retrieve shard information for a given community ID (#4499)
* feat: a mechanism to retrieve shard information for a given community ID
2023-12-22 13:37:37 +01:00
Igor Sirotin f8236fb555
fix: link previews public keys (#4272) 2023-11-09 16:24:35 +00:00
frank 054356fed2
remove sync AC state (#4241) 2023-11-02 23:07:03 +08:00
Mykhailo Prakhov eb437e9d8d
feat: kick all members after ownership change and auto-accept after sharing the address (#4187)
feat: kick all members after the ownership change and auto-accept after sharing the address
2023-10-31 15:20:40 +01:00
Andrea Maria Piana 360c2d50d4 Fix issues with out of order encryption and batched messages
This commit fixes 3 issues:

1) In some cases, the hash ratchet was not correctly found
2) Out of order messages were not processed correctly as the wrong error
   was returned
3) Batched non datasync messages were not processed correctly

Fixes: #4170
2023-10-30 10:11:32 +00:00
frank 06a9fe4f81
fix: sync decision on join community request (#4190)
* fix: sync community request decision

* make generate

* simplify test

* make generate
2023-10-26 12:17:18 +08:00
Patryk Osmaczko b321f28c9d refactor: update community access enum naming 2023-10-25 17:34:37 +02:00
Andrea Maria Piana 23f71c1125 Fix encryption id && rekey with a single message
This commit changes the format of the encryption id to be based off 3
things:

1) The group id
2) The timestamp
3) The actual key

Previously this was solely based on the timestamp and the group id, but
this might lead to conflicts. Moreover the format of the key was an
uint32 and so it would wrap periodically.

The migration is a bit tricky, so first we cleared the cache of keys,
that's easier than migrating, and second we set the new field hash_id to
the concatenation of group_id / key_id.
This might lead on some duplication in case keys are re-received, but it
should not have an impact on the correctness of the code.

I have added 2 tests covering compatibility between old/new clients, as
this should not be a breaking change.

It also adds a new message to rekey in a single go, instead of having to
send multiple messages
2023-10-24 20:48:54 +01:00
frank 624996a7e9
only sync decisions/state of AC notifications (#3979) 2023-10-22 17:41:20 +08:00
Patryk Osmaczko 0881d8cdb0 Add queue for processing community description & configurable control node 2023-10-17 21:00:24 +01:00
Igor Sirotin aded258ccb
feature: Unfurl status links (#4033) 2023-10-13 13:25:34 +01:00
Patryk Osmaczko 53423e58ba fix: use proper migrations for protocol's test database
- use `appdatabse.DbInitializer{}` in tests to ensure consistent migrations

- remove protocol's open database functions due to improper
  initialization caused by missing node config migration

- introduce `PushNotificationServerConfig` to resolve cyclic dependency
  issues
2023-10-03 15:11:58 +02:00
Mykhailo Prakhov db9adb631f
feat: share requests to join and revealed addresses with privileged roles (#3951)
- share requests to join with new privileged roles during reevaluating member role
- share requests to join with new members, joined the community as TOKEN_MASTER, ADMIN
- share requests to join revealed addresses to ADMINS and TOKEN_MASTERS
- refactor common test functionality to make them more predictable
- removed unused CommunityToken protobuf
2023-09-20 10:37:46 +02:00
frank 6bcf5f1289
Removed settings.Usernames and reactivate PreferredName (#3964)
* sync preferred name;
remove settings.usernames

* update account name when handle settings.preferred_name from backup message

* fix Error:Field validation for 'KeycardPairingDataFile' failed on the 'required' tag

* bump version

* rebase
2023-09-16 09:20:23 +08:00
Ibrahem Khalil 22ec4ac586
Link previews not working on message edit (#3974) 2023-08-31 10:21:25 +03:00
frank 38415849cd fix generate_handlers_template.txt 2023-08-28 14:17:33 +01:00
Mykhailo Prakhov 86d969727f
feat: share CommunityTokens between privileged users (#3931)
feat: sync CommunityTokens between privileged users
feat: restore PrivilegeLevel for synchronized CommunityToken
2023-08-22 19:48:42 +02:00