Commit Graph

727 Commits

Author SHA1 Message Date
Lorenzo Delgado ff619de037
chore(utils): remove unused types and methods 2022-09-12 15:29:52 +02:00
G 01cd201f58
PoC implementation of Waku Pairing and Secure Transfer (#1117)
* feat(noise): add PoC implementation for WakuPairing and Secure Transfer
2022-09-12 02:23:14 +02:00
Lorenzo Delgado 30f0e19d79
fix(store): add a table index to improve message insertion time 2022-09-09 15:04:48 +02:00
Hanno Cornelius f4e9d39524
chore: always include node ENR in response to info requests (#1128) 2022-09-08 14:25:02 +01:00
Lorenzo Delgado d857c4a421
chore(wakunode): collocate wakuswitch file to the depending modules 2022-09-08 11:02:50 +02:00
Hanno Cornelius 7f23bdf29f
chore: update nwaku submodules (#1123)
* chore: update submodules

* fix: SIGTERM ambiguity

* fix: ambiguous RNGs

* fix: ContentType is no longer a string

* fix: more fixes related to ambiguous RNGs

* fix: start all protocols

* chore: also update nim-eth

* chore: important new fixes in nim-libp2p

* fix: more changes related to RNG. Some reversion to reflect nim-eth update

* fix: breaking changes in nim-eth submodule

* fix: start protocols in tests

* fix: chat2bridge protocols async mounting

* fix: v1 test compilation

* fix: rln test compilation

* fix: remove confusing keys qualifier for the same HmacDrbgContext
2022-09-07 16:31:27 +01:00
Lorenzo Delgado 3f7aa8c6e4
fix(waku-store): added an index to improve messages query time 2022-09-05 15:18:47 +02:00
Lorenzo Delgado 21e9f25276
refactor(wakunode2): deprecate non-async protocol api methods 2022-09-02 10:17:14 +02:00
Lorenzo Delgado 4f3a3da084
chore(waku-store): improve logging and query traceability 2022-09-02 10:14:58 +02:00
Lorenzo Delgado 982fb08c77
refactor(wakunode2): add convenience methods for adding protocol peers 2022-09-01 17:02:58 +02:00
Aaryamann Challani ea62e2b7ce
chore(noise): minor refactor of default checks of keypairs (#1104)
* chore(noise): readable default checks for keypairs

* chore(noise): fix naming of arg in exists proc

* chore(noise): fix naming to make it clearer

* chore(noise): inline and remove result usage

* chore(noise): move the isDefault function up, replace defaultyVar with value

* chore(noise): inline style

* chore(noise): static the default check, remove inline pragma
2022-09-01 14:31:48 +05:30
Bulat-Ziganshin 80aeebc61c
Fixed handling of bindParam(uint32) in sqlite.nim (#1014) 2022-08-31 14:50:05 +02:00
Aaryamann Challani ad7f9ed5bf
chore(lightpush): fix minor typo (#1100) 2022-08-31 17:00:10 +05:30
Lorenzo Delgado 6d0168bff2
feat(rest): add waku filter openapi rest api specification file 2022-08-31 13:24:35 +02:00
Aaryamann Challani ed53bcdec6
chore(logging): Periodic logging for additional metrics (#1094)
* chore: enable metrics logging by default

* feat(logging): add additional logging metrics, change period to 30 seconds

* chore(logging): s/var/let

* chore(logging): import filter directly

* chore(logging): add store/lightpush peer count, fix style
2022-08-31 15:23:28 +05:30
Sanaz Taheri Boshrooyeh 0deedf0d7a
fix|feat(rln-relay|rln-chat2): Handling on-chain transactions errors and informing chat2 users (#1093)
If any error occurs during the on-chain registration, it will be handled; the chat2 users will get informed about the cause and the installation of rln-relay gets discarded and the chat program continues with no crashing.
2022-08-30 10:59:02 -07:00
Lorenzo Delgado ecf4ba1167
feat(message-cache): make the topic cache generic 2022-08-30 15:57:45 +02:00
Lorenzo Delgado 791ce6d222 fix(rest): encode waku message payload in base64 2022-08-30 14:53:04 +02:00
Lorenzo Delgado 842cfb3476 refactor(waku-filter): remove wakunode2 waku_filter exports 2022-08-29 10:14:35 +02:00
Lorenzo Delgado c964aea885 refactor(waku-filter): waku filter protocol code reorganisation 2022-08-26 22:17:56 +02:00
Hanno Cornelius caa47896c1
fix: peer manager dial logic when dialing an existing peer (#1090) 2022-08-26 11:20:55 +02:00
Sanaz Taheri Boshrooyeh bc29619679
feat(Rln relay): Improves rln-chat2 interface, displays links to the registration tx, warns about spam messages (#1082)
* warns  about publishing spam messages, display tx hash on Goerli

* passes registration handler
2022-08-24 15:47:06 -07:00
Hanno Cornelius 11158c3ea2
fix(setup): do not attempt connecting to empy dynamic bootstrap node list (#1088) 2022-08-24 17:44:43 +02:00
Sanaz Taheri Boshrooyeh 8a6b1cfba6
feat(Rln-Relay): Consolidates the input paths of rln credential, chat2 reads rln credential from file only (#1080)
* Consolidates the input paths of rln credential, accepts credential file only

* fixes a typo
2022-08-18 10:35:02 -07:00
Lorenzo Delgado 73df74f8c6 refactor(waku-store): remove wakunode2 waku_store exports 2022-08-18 17:32:47 +02:00
Sanaz Taheri Boshrooyeh f373f8bc56
chore(RLN-Relay): onchain rln-relay chat2 testnet editorial feadbacks
Signed-off-by: sanaz <staheri14@ku.edu.tr>

adds a newline


fixe a minor spacing issue
2022-08-17 16:55:49 -07:00
Lorenzo Delgado 7a30e485ec refactor(waku-store): deprecate waku store callback methods 2022-08-17 17:22:56 +02:00
G 2c4730c850
fix(rln): add zerokit rln to chat2 + fix static group members (#1077) 2022-08-17 12:11:46 +02:00
Lorenzo Delgado 2cb7123df8 fix(waku-store): integrate the predicate into the WHERE clause instead of calling it on each result line (#1026) 2022-08-12 12:48:45 +02:00
Lorenzo Delgado 4f93510fc9 refactor(waku-store): inherit from message_store in store_queue 2022-08-11 10:15:20 +02:00
Lorenzo Delgado 613812023e refactor(waku-lightpush): remove wakunode2 waku_lightpush exports 2022-08-10 19:44:40 +02:00
Lorenzo Delgado d201a37330 refactor(waku-lightpush): waku lightpush protocol code reorganization 2022-08-10 12:30:34 +02:00
G 2aeaba48e4
feat(noise): add padding to transport messages (#1056)
* feat(noise): add padding to transport messages (handshakes+write+read)

* fix(noise): address reviewers' comments

* fix(noise): update padding block size const name
2022-08-07 16:20:34 +02:00
G 8a14b5b243
Zerokit RLN integration in nwaku: alternative approach with inline compilation flag (#1060)
* refactor(rln): add inline compilation flags for RLN zerokit

* update submodule

update submodule

* fix(rln): fix wrong merge conflict resolution

* fix(rln): align hardcoded values in light of vacp2p/zerokit/issues/28
2022-08-05 22:58:19 +02:00
Keshav Gupta 115982f5c3
Persisting rln credentials (#1037)
* Persisting Credentials implemented by writing json of keypair and rlnindex to files

* Removing testing files and ignores

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Daniel Kaiser <git@kais3r.de>

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Daniel Kaiser <git@kais3r.de>

* Comments

* Comments

* Security warning in comments

* Redundant echos. Omitting unused variables.

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Limit line lengths using line breaks and indents

* Membership file paths declared as const

* Const fix

* Rln Credentials Merged.
Reading credentials from file abstracted away.

* Spaces

* Spaces

* Dangling constants removed. Comments position.

* Import sequence.

* Path as argument to readPersistentKeys.
Spaces in comments

* readPersistentKeys test

* Debug and info

* Index check in test

* Update tests/v2/test_waku_rln_relay.nim

Co-authored-by: G. <28568419+s1fr0@users.noreply.github.com>

* Abstracted writeRlnCredentials.
Fix var name in test.

* Usage of writeRlnCredentials in test

* Dnsclient?

* Test reverted to direct call to writeFile.
Abstrated writePersistentRlnCredentials usage causing error,
with readPersistentRlnCredentials

* Indentation

* Revert "Dnsclient?"

This reverts commit 3f4dba1a0b.

* Reverting abstraction of writing..
..persiting credential

Co-authored-by: Daniel Kaiser <git@kais3r.de>
Co-authored-by: Keshav Gupta <keshav.pg@hotmail.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: G. <28568419+s1fr0@users.noreply.github.com>
2022-08-05 12:48:01 +02:00
G 32f91cb26b
Noise: split Noise submodule in smaller submodules (#979)
* refactor(noise): split Noise submodule in smaller submodules
2022-08-04 10:47:00 +02:00
Lorenzo Delgado 888f7cb312 refactor(waku-store): major code reorganization, move StoreQueue to message_store folder 2022-08-01 13:02:40 +02:00
Sanaz Taheri Boshrooyeh 79459d29ce
Feat(Rln relay): enables relay-mode only (#1051)
* makes eth private key optional

* makes eth account private key field optional

* clarifies a comment
2022-07-28 11:45:29 -07:00
Lorenzo Delgado e7ba81ff88 refactor(wakunode2): Rename waku_setup.nim to wakunode2_setup.nim 2022-07-19 15:09:23 +02:00
Lorenzo Delgado 10c237dc5f refactor(wakunode2): Split wankunode2 SQL migrations setup to its own file 2022-07-19 15:09:23 +02:00
Lorenzo Delgado e16747137e refactor(wakunode2): Split wankunode2 JSON-RPC setup to its own file 2022-07-19 15:09:23 +02:00
Lorenzo Delgado 538ee12a25 refactor(wakunode2): Split wankunode2 metrics setup to its own file 2022-07-19 15:09:23 +02:00
Lorenzo Delgado 732f853f55 feat(rest): Add HTTP REST API (#727). Integrate REST API with the Waku V2 node 2022-07-19 11:02:43 +02:00
Sanaz Taheri Boshrooyeh daaca4a4a3
Feat(Rln relay): Enables and integrates onchain rln into chat2 (#1029)
* adds wakunode2_types

* removes unused imports

* adds dynamic rln relay config options

* deletes unused imports

* replaces static rln-relay with both modes

* introduces spamHandler parameter to mountRlnRelay

* some debugging info

* introduces the config option

* adds the eth priv key to the register proc

* refactors the code by adding eth private key to the relevant procs

* revises config descriptions

* minor wording fix

* make a minor edit

* further edits

* adds Eth private key to the config options

* calls spam handler when to rln credential is given

* defaults to none for the spam handler

* updates the membership fee to match the contract

* fixes an issue for event subscription

* adds eth private key to the transactions made in the tests

* adds eth private key for the test of register process

* adds comments

* deletes lingering echo

* displays registration info

* removes an echo

* removes Goerli testnet from the config descriptions

* removes an excess space
2022-07-14 12:23:52 +02:00
Sanaz Taheri Boshrooyeh a40686e26e
Feat (Rln relay): enabling signed tx (#1023) 2022-06-30 18:05:38 -07:00
Lorenzo Delgado 993781a137 chore: remove unused imports 2022-06-30 00:35:18 +02:00
Sanaz Taheri Boshrooyeh 5c7baf87e5
Chore(rln-relay): adds more documentation for rln nim bindings (#1016)
* annotates rln nim bindings for more clarity

* adds pointer to the rln construct

* revises further

* addresses comments

* revises proc description of verify

* removes AuthObj
2022-06-29 13:45:56 -07:00
Daniel Kaiser 3b60ea70c0
perf(store): remove vacuum (#1025) 2022-06-29 12:06:42 +02:00
Sanaz Taheri Boshrooyeh 3e20127933
Chore(rln-relay): decouples rln-relay and wakunode2 modules (#1019)
* adds wakunode2_types

* removes unused imports
2022-06-27 12:35:26 -07:00
Hanno Cornelius 9be64a55b6
fix: bridge loses connection to v1 (#1020)
* fix: bridge loses connection to v1

* Remove magic numbers in checks
2022-06-27 14:16:15 +02:00
Daniel Kaiser a2749c4a52
fix(store): limit dbPageSize (#1018) 2022-06-23 17:05:48 +02:00
Lorenzo Delgado cf1a7c3074 feat(rest): Add HTTP REST API (#727). Add Relay REST API 2022-06-23 10:12:38 +02:00
Lorenzo Delgado d2feb7c763 feat(rest): Add HTTP REST API (#727). Amend Debug REST API client 2022-06-22 20:30:44 +02:00
Richard Ramos 17ebb32331 fix: encode protobuffers using proto3 version 2022-06-21 17:21:03 -04:00
Sanaz Taheri Boshrooyeh 6c8ab0ab0f
Feat(rln-relay): enables two modes of onchain and offchain (#992)
* adds the contract handler file

* adds integration test for the group listening

* adds groupManagement proc

* deletes rln relay contract handler file

* brings back all the tests

* replaces toUINT256 with getIdCommitment proc

* replaces individual futures with an array of futures

* adds code documentation

* asyncSpawn instead of await

* adds untitest for toIDCommitment and toUInt256

* reorganizes the test and add rlnInstance

* mounts handleGroupUpdates on the rln peer

* asyncSpawn to await

* implements toIDCommitment

* updates the unittest

* improves the code documentation

* removes unused tests

* WIP

* uncomments the tests

* defines a new mountRlnRelayStatic proc

* splits mountRlnRelay into two procs for dynamic and static group management

* adds a config for off-chain and on-chain rln-relay

* runs dynamic or static mode of rln-relay based on the input config

* adds Eth private key and account configs

* reads Eth private key and account the configs

* comments put the second register proc

* add proper call to the rlnrelay dynamic mode

* adds todo

* adds new rln relay configs

* splits register into two procs

* makes eth account private key non-optional

* removes getIdCommitment and edits the register proc

* removes getIdCommitment calls

* uncomments the commented tests

* fixes a bug

* removes contract deployment for the offchain test

* fixes a bug, edits comments

* removes custom types  without proper parsing and serialization routines from the configs

* fixes a bug

* switches to stew byte utils

* removes log decoding

* WIP

* updates register proc

* edits test titles

* removes eth private key config

* changes the output of register proc to return the registered index

* integrates the registration process into mountRlnRelayDynamic

* integration test for the register proc

* brings back the onchain tests

* updates comments

* cleans up

* disambiguates the Address type namespace

* fixes type ambiguities

* adds default values for rln key and index

* updates config descriptions

* adds type conversion from hex to MembershipKeyPair

* adds more code documentation

* passed the group value instead of option  to the mount proc

* fix a bug

* a minor input type fix for rln chat2

* groups let declarations

* adds default values for addresses

* logs registered keys
2022-06-17 15:00:19 -07:00
Lorenzo Delgado 76e9a98841 feat(rest): Add HTTP REST API (#727). Debug API POC 2022-06-14 18:20:15 +02:00
Daniel Kaiser 5445303a23
feat(store): sqlite only store (#982)
* feat(sqlite): parameterized prep statement exec
  - from nim-eth
* feat(store): sql-only-store
  - add getPage(...) to sql-message-store
  - acts as separate store impl that can be activated with
    `--sqlite-store=true`
* test(store): new test-suite for sql message store
* test(store): waku_store test SQL-store compatible
* perf(store): avoid builing an additional B-tree in sql query
  - use `receiverTime` as the last ORDER BY column to fully utilize the
    primary key sorting
feat(store): retention time
  - retention time in seconds can be specified via `--sqlite-retention-time=`
2022-06-13 19:59:53 +02:00
Hanno Cornelius 6a09ed2c88
feat(enr): add discv5 UDP port to node's main discoverable ENR (#996) 2022-06-09 17:46:21 +02:00
Daniel Kaiser 271a2c750b
chore: adjust dyn bootstrap error message (#993) 2022-06-08 22:23:13 +02:00
Hanno Cornelius 0847eec6e0
chore: reduce unnecessary logging (#991) 2022-06-08 11:20:18 +02:00
Hanno Cornelius 1b29dcda7b
chore: disable swap by default (#989) 2022-06-08 09:16:45 +02:00
Hanno Cornelius 4e9215a920
chore: more regular keepalive pings (#990) 2022-06-07 22:44:58 +02:00
G c259f4cbaa
feat(noise): add Noise Handshake processing and after-handshake encryption (#934)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* feat(noise): add Noise Handshake State Machine and primitives

* feat(noise): add Handshake processing and after handshake encryption

* feat(noise): code updated according to #933

* fix(noise): export HandshakeResult

* feat(noise): add pre message validation utility

* chore(noise): improve documentation

* fix(noise): address reviewers comments
2022-06-03 21:12:41 +02:00
Hanno Cornelius d5a04a2719
chore: update submodules (#987) 2022-06-01 11:49:41 +02:00
Hanno Cornelius 37325c8f48
fix(sqlite): memory leak due to undisposed prepared statements (#981) 2022-05-31 12:47:38 +02:00
Sanaz Taheri Boshrooyeh 1ac029025a
Feat (Rln relay): adds utility procs to listen to the registration events emitted by the rln membership contract (#976)
* adds the contract handler file

* adds integration test for the group listening

* adds groupManagement proc

* deletes rln relay contract handler file

* brings back all the tests

* replaces toUINT256 with getIdCommitment proc

* replaces individual futures with an array of futures

* adds code documentation

* asyncSpawn instead of await

* adds untitest for toIDCommitment and toUInt256

* reorganizes the test and add rlnInstance

* mounts handleGroupUpdates on the rln peer

* asyncSpawn to await

* implements toIDCommitment

* updates the unittest

* improves the code documentation

* removes unused tests

* removes registration of the dynamic group management handler

* adds a comment

* adds a comment

* adds a TODO

* removes getIdCommitment
2022-05-30 12:14:07 -07:00
Lorenzo Delgado 32d230b474 feat(store): ability to decouple store from relay (#937) 2022-05-23 20:42:48 +02:00
G 710ce1315a
feat(noise): add Noise Handshake State Machine and primitives (#933)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* feat(noise): add Noise Handshake State Machine and primitives

* Fixed header

* refactor(noise): add comments, restyle code

* refactor(noise): address reviewer's comments

* feat(noise): add Noise Handshake State Machine and primitives

* refactor(noise): rebase

* refactor(noise): refactor and add documentation

* fix(noise): fixed trace parameter

* update submodules

* fix(noise): remove echo, add stdout.write in print handshake

* refactor(noise): add Noise state machine overview

* Revert "update submodules"

This reverts commit 9fc162dff8.

* update submodules

* fix Jenkins caching

* Revert "update submodules"

This reverts commit 1927d9dd26.

* revert nim-eth submodule update

* fix(noise): missing camelCase

* feat(noise): adding unit tests for Noise State machine primitives

* fix(noise): address reviewers' comments

* fix(noise): use expect for error handling in unit tests

* fix(noise): revert to implicit nonce declaration in CipherState initialization
2022-05-20 15:51:36 +02:00
Hanno Cornelius 984f3179a7
chore: update submodules (#974) 2022-05-20 12:57:19 +02:00
Daniel Kaiser 6a60458135
chore(migration): avoid unnecessarily calling migration (#967) 2022-05-18 21:44:53 +02:00
Hanno Cornelius 185485c336
docs: generate and configure nodekey (#957) 2022-05-18 15:00:30 +02:00
Daniel Kaiser 59b242fcad
fix(dns4name): dns4name + unset port resolved (#965) 2022-05-18 10:15:03 +02:00
Daniel Kaiser 41cdb8ba26
feat(info): add --version cmd flag (#970) 2022-05-17 22:11:07 +02:00
Daniel Kaiser 51cbc39d49
feat(config): add config file support (#953)
including

* bump vendor/nim-confutils
* add nim-toml-serialization submodule
* feat(config): add example config file
* add new feature to CHANGELOG
2022-05-17 17:48:08 +02:00
Hanno Cornelius bd70a1b7bb
Disable GossipSub PX by default (#958) 2022-05-13 18:18:09 +02:00
Sanaz Taheri Boshrooyeh 47a9fcfd77
chore|feat (waku-rln-relay): modules reorganization|Initial test for capturing events using nim-web3 (#941)
* first edition

* adds the full test scenario

* fixes typos

* fixes a bug in the supplied command

* further edits the description

* displays the chat prompt after spam detection

* updates changelog

* minor wording fix

* adds a new test file for onchain rln relay

* adds the Event proc

* adds one working example of event subscription

* defines a new unitt test for event subscription

* adds the new test file

* cleans up the code

* adds a working event subscription for faucet contract

* wip

* makes faucet test conditional

* updates contract byte codes

* adds a working test for event subscription and cleans up the tests

* fixes case

* adss toUInt256 unit function

* enables the tests

* fixes a bug

* undo commented tests

* cleans up the test

* logs the pk

* removes excess entry in the changelog

* fixes spacing

* comments

* removes unused test codes

* adds the conditional compilation for onchain tests

* uncomments offchain tests

* removes onchain tests

* reorganizes the code and moves the rln contract data into a separate module

* deletes txt files

* beautifies the code

* beautifies the code

* removes an excess line

* more formatting fixes

* minor fix

* updates the case of membership fee const

* renames compare to diff

* renames time to e

* edits the number of arguments of the send proc

* fixes a comment alignment

* fixes indentation

* fixed id style

* splits check from condition

* fixes a naming mismatch
2022-05-10 14:09:18 -07:00
Franck R b2eb992bd3
Fix typo (#960) 2022-05-10 12:49:12 +02:00
Daniel Kaiser 5e85eecd14
perf(store): add index on receiverTimestamp (#947) 2022-04-23 10:49:11 +02:00
G cb11c192d6
feat(noise): add support to Waku Payload V2 (#932)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* Fixed header

* refactor(noise): add comments, restyle code

* refactor(noise): address reviewer's comments

* Update nim-eth

* fix(noise): moved strutils in std
2022-04-12 13:50:51 +02:00
Hanno Cornelius 450e0494f1
fix(store): timestamp validity checks (#876) 2022-04-11 14:58:18 +02:00
G d2fccb5220
feat(noise): add support to encrypted/unencrypted public keys (#931)
* feat(noise): adding ChaChaPoly encryption

* test(noise): add test for ChaChaPoly Encryption/decryption

* style(noise): converted variables to camelCase

* feat(noise): add support to Noise public keys

* Fixed header

* refactor(noise): address reviewer's comments

* refactor(noise): small fixes/comments

* refactor(noise): add empty plaintext/ciphertext handling + string encryption test

* refactor(noise): address reviewer's comments
2022-04-06 14:37:02 +02:00
G 45b1e10647
feat(noise): add support to ChaChaPoly encryption (#930)
* feat(noise): adding ChaChaPoly encryption

* fix(noise): restored waku2 tests

* fix(noise): missing comma

* Fixed header

* fix(noise): made comment about empty ChaChaPolyKey clearer

* refactor(noise): address reviewer's comments

* test(noise): add test for ChaChaPoly Encryption/decryption

* refactor(noise): removed libp2p specific debug tools usage

* style(noise): converted variables to camelCase
2022-04-04 17:46:51 +02:00
Daniel Kaiser ecdde23add
feat(store): info message after delete (#929) 2022-03-30 22:55:39 +02:00
Daniel Kaiser a9c31b4b53
feat(store): vacuum after delete (#928) 2022-03-30 22:54:37 +02:00
Daniel Kaiser 2f13aff663
feat(jsonrpc): add call returning git version (#919) 2022-03-30 17:17:21 +02:00
Daniel Kaiser ec0a46b65b
feat(jsonrpc): add ENR to debug_v1_info call (#918) 2022-03-30 15:10:15 +02:00
Hanno Cornelius 52de0e434f
feat(relay): add support for gossipsub peer exchange (#911) 2022-03-29 10:09:48 +02:00
Daniel Kaiser d0cf3ed1f9
fix(store): capacity limit for persistent message store (#900)
* fix(store): limit capacity of persistent message store

fixes performance issue of loading a large message table by deleting
old messages in case of an overflow

* test(store): message capacity limit
2022-03-28 20:27:14 +02:00
Daniel Kaiser be8860c2fc
fix(discv5): info print ENR containing discv5 udp port (#913) 2022-03-28 10:29:19 +02:00
Sanaz Taheri Boshrooyeh c9e82873f1
Increase rln-relay logging (#908)
* updates default rln-relay content topic

* adds further debug and trace level logs

* converts a debug level log to the trace level
2022-03-24 10:47:40 -07:00
Hanno Cornelius 06f20af7d4
chore: update submodules (#905) 2022-03-23 15:20:07 +02:00
Hanno Cornelius 994ac49472
Fix: timestamp type could overflow before conversion (#892) 2022-03-18 12:04:11 +02:00
Daniel Kaiser e1146b2df8
discv5 dns bootstrap (#888) 2022-03-17 17:33:17 +01:00
Hanno Cornelius 266d6a7e80
fix: prevent empty store response due to blocked walker (#897) 2022-03-17 17:18:59 +02:00
Sanaz Taheri Boshrooyeh 2d6a2bec47
Updates the type of the `proof` field of wakumessage to `RateLimitProof` (#893)
* permanently changes the wakumessage proof field to RateLimitProof

* updates an old comment
2022-03-16 13:51:18 -07:00
Daniel Kaiser e6fdbdb42a
add RAM requirement to README (#889) 2022-03-16 17:13:23 +01:00
Hanno Cornelius eb02e4411b
Remove content topic dimension from message metrics (#891) 2022-03-15 14:39:32 +02:00
Hanno Cornelius ea72a65ef8
Increase store and store queue logging (#887) 2022-03-11 07:26:15 +02:00
Hanno Cornelius 9165720f63
Bridge improvements (#875) 2022-03-08 11:48:17 +01:00
Sanaz Taheri Boshrooyeh 2f73667a96
Fixes getNanosecondTime() time conversion (#878)
* fixes conversion from sec to nanosec

* uses snake case for power of 10 for readability
2022-03-07 16:56:39 +01:00
Hanno Cornelius 06fbb875c8
Update submodules (#873)
* Update submodules

* Fix MsgIDProvider
2022-03-02 13:24:48 +01:00
Daniel Kaiser 2972a50035
discv5 stage1 testnet (#863) 2022-03-01 17:12:23 +01:00
Daniel Kaiser dbe76d29ce
discv5 stage1 beta test (#862)
* update vendor/nim-eth submodule

point to the selectable `protocol-id` feature branch

* use d5waku as protocol-id

Use compiletime flag to change the discv5 protocol-id to "d5waku" allowing
nim-waku to span a separate discv5 network

* waku discv5 first beta stage

  - start discv5 during node start
  - added config options for discv5

* fix: pass to tcp port also when specifying ext ip

* bump nim-eth

Co-authored-by: ksr <kaiserd@users.noreply.github.com>
2022-03-01 15:11:56 +01:00
Hanno Cornelius 6be0fb233a
Add bridge docs and minor improvements (#867)
* Add bridge docs and minor improvements

* Clarify that bridging is bidirectional
2022-02-28 18:16:50 +01:00
Hanno Cornelius 20efdd7255
Store data model updates and fixes (#864) 2022-02-28 17:29:01 +01:00
G c27086e186
Moved PB Timestamp index to 10 (#861) 2022-02-23 08:19:52 +01:00
Hanno Cornelius 0b7b033550
Add docs on configuring domain and dns disc (#856) 2022-02-21 16:23:55 +01:00
Hanno Cornelius 2fd5b6c4ea
Domain name config for `nwaku` node (#852) 2022-02-18 12:10:38 +01:00
G dd2ed886a3
Update WakuStoreCodec string to beta4 (#855)
* Fix waku store version to beta4

* Updated changelog
2022-02-18 07:23:10 +01:00
G 21cac6d491
Refactoring timestamps (#842)
* Refactor timestamps type from float64 to int64 (milliseconds resolution)

* Revert epochs to float64

* Update 00002_addSenderTimeStamp.up.sql

* Update quicksim2.nim

* Add files via upload

* Delete 00003_convertTimestampsToInts.up.sql

* Add files via upload

* Rename 00003_convertTimestampsToInts.up.sql to 00003_addTimestampsToInts.up.sql

* Delete 00003_addTimestampsToInts.up.sql

* Rln-relay integration into chat2 (#835)

* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* removes duplicate epoch time calculation

* updates log level to trace

* updates default rln-relay content topic

* adds support for empty content topics

* updates changelog

* changelog updates

* removes a commented code block

* updates addRLNRelayValidator string doc

* Squashed commit of the following:

commit bc36c99ab2
Merge: dc2b2946 5a77d6e2
Author: G <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 01:10:06 2022 +0100

    Merge branch 'master' into int64-timestamps-ns

commit dc2b294667
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:24:45 2022 +0100

    Fix

commit f97b95a036
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:13:18 2022 +0100

    Missing import

commit 060c4f8d64
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:10:36 2022 +0100

    Fixed typo

commit 08ca99b6f6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:59:20 2022 +0100

    Time util file

commit 2b5c360746
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:33:20 2022 +0100

    Moved time utility functions to utils/time

commit fdaf121f08
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:10:25 2022 +0100

    Fix comment

commit c7e06ab4e7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:04:13 2022 +0100

    Restore previous migration script

commit 80282db1d7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:54:15 2022 +0100

    Typo

commit b9d67f89b0
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:49:29 2022 +0100

    Added utilities to get int64 nanosecond, microsecond, millisecond time resolution from float

commit 0130d496e6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:36:35 2022 +0100

    Switched to nanoseconds support.

* Update CHANGELOG.md

* Create 00003_convertTimestampsToInt64.up.sql

Migration script

* Moved migration script to right location

* Update waku_rln_relay_utils.nim

* Update waku_rln_relay_utils.nim

* Addressed reviewers' comments

* Update default fleet metrics dashboard (#844)

* Fix

* No need for float

* Aligning master to changes in PR

* Further fixes

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2022-02-17 16:00:15 +01:00
Hanno Cornelius bb3e59454e
Store performance improvements (#849) 2022-02-17 11:00:45 +01:00
Sanaz Taheri Boshrooyeh d851d48424
Updates rln-chat2 interface (#846)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* WIP

* removes duplicate epoch time calculation

* converts echo to log

* invokes handler

* bug fix

* prints calculated epoch

* changes the format of printed epoch

* updates log levels

* logs the input buffer supplied to the generate_proof

* replaces echos with logs

* changes log level to trace

* resets the log level of chat2 to INFO

* upgrades log level to debug

* exports toRLNSignal and adds a doc string

* updates log level

* enables all test2

* removes an echo statement

* modifies a comment

* further updates on the log level

* a minor update

* invokes the spam handler when provided

* checks for payload version

* deletes a redundant check

* deletes a rendant check

* updates default rln-relay cht2 content topic

* adds a todo and log

* changes the case of testnet content topic

* removes a flaky check
2022-02-16 14:52:21 -08:00
Hanno Cornelius e1254c20ae
Support for dns `multiaddrs` (#851)
* Support for dns addrs

* Tests and minor improvements
2022-02-16 17:12:09 +01:00
Franck R 5a890b5f90
Update websocket documentation to use domain certificates (#848) 2022-02-15 17:07:55 +11:00
Sanaz Taheri Boshrooyeh 5a77d6e2a6
Rln-relay integration into chat2 (#835)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* removes duplicate epoch time calculation

* updates log level to trace

* updates default rln-relay content topic

* adds support for empty content topics

* updates changelog

* changelog updates

* removes a commented code block

* updates addRLNRelayValidator string doc
2022-02-04 15:58:27 -08:00
Sanaz Taheri Boshrooyeh a5f8b90c23
Enables RLN compilation under CI env var (#786)
* compiles RLN if CI is defined

* minor ineffective change

* minor

* removes excess space

* resolves namespace conflicts

* moves pubsub import under compiler flag

* disables rln on-chain tests

* brings back the on-chain tests and fixes a name mismatch

* adds a debug log message

* minor formating

* kills ganache-cli procces on windows

* wip: adds a line to kill ganache-cli on windows

* disables rln-relay tests that depend on ganache-cli

* disables ganache-cli installation

* introduces onchain rln flag

* fixes a flaky test

* enables onchain test in non-windows CIs

* moves variables around

it seems detected_os is undefined, hence the compiler flag onchain_rln, which is conditioned to the os, does not get added

* limits onchain rln to macOS
2022-01-28 13:57:17 -08:00
Hanno Cornelius 4421b8de00
Improve node metrics (#831)
* Improve node metrics

* Add support for empty content topics as a separate label
2022-01-26 12:02:57 +01:00
Sanaz Taheri Boshrooyeh dbccecc562
Resolves naming conflict (#821)
* resolves namespace conflicts

* moves pubsub import under compiler flag
2022-01-21 10:55:36 -08:00
Daniel Kaiser b3a7722297
result aggregation in resume API (#816) 2022-01-18 23:05:41 +01:00
Hanno Cornelius 95d2e8bf71
Round receiver time (#815) 2022-01-17 19:37:06 +01:00
Hanno Cornelius 3e9ca25abc
Update submodules (#814) 2022-01-14 10:25:01 +01:00
Hanno Cornelius c422866a49
Improved `store` query performance (#812) 2022-01-11 14:32:09 +01:00
Hanno Cornelius f3ecdb80ea
Remove node peerInfo copy (#801) 2022-01-10 16:07:35 +01:00
Hanno Cornelius 62dbb3d0f3
Add MessageIdProvider for `WakuRelay` (#803)
* Add MessageIdProvider

* Fix unit test. Changelog.
2022-01-10 15:07:01 +01:00
Hanno Cornelius 685d43b2b3
Ensure store response never exceeds `MaxPageSize` (#811)
* Never exceed maxPageSize, even with no specified pagination

* fix test

* Changelog

* Forgot to stage everything...

* Account for timing difference in Windows
2022-01-07 15:01:23 +01:00
Hanno Cornelius 4e6960ad26
Fix incorrect store read buffer limit (#800)
* Fix store read buffer limit

* Changelog
2022-01-06 13:42:37 +01:00
Hanno Cornelius eb5dcadd45
Added protocol message counters (#804)
* Added protocol message counters

* Fix how stored messages are counted
2022-01-06 12:23:25 +01:00
rshiv b2273fff9a
Handles stale clients in FILTER protocol (#782)
* Handles stale clients

Signed-off-by: rshiv <reeshav96@gmail.com>

* adds test

Signed-off-by: rshiv <reeshav96@gmail.com>

* removes failed client from subscriber list

Signed-off-by: rshiv <reeshav96@gmail.com>

* adds filter timeout config

Signed-off-by: rshiv <reeshav96@gmail.com>

* reverts peer removal logic

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolve ci issues

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comment

Signed-off-by: rshiv <reeshav96@gmail.com>

* solves review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* ChangeLog update

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comment

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update CHANGELOG.md

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* fixes review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* handles CI issues

Signed-off-by: rshiv <reeshav96@gmail.com>

* tries to solve test CI

Signed-off-by: rshiv <reeshav96@gmail.com>

* solves CI issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves ci issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves CI issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2021-12-08 18:35:47 +00:00
rshiv ce0607b2eb
resolves readme (#796)
Signed-off-by: rshiv <reeshav96@gmail.com>
2021-12-08 16:04:23 +00:00
Hanno Cornelius 9eb532171e
Fix empty scripts condition. Logging. (#795) 2021-12-07 13:06:59 +01:00
Hanno Cornelius dd256e3bd2
Add support for RFC31 ENR (#789) 2021-12-06 20:51:37 +01:00
Richard Ramos 1217e25245 fix: display success message only if messages are loaded from the db 2021-12-03 09:35:40 -04:00
Richard Ramos 2b6922cdaa fix: return db error message 2021-12-03 09:35:40 -04:00
Hanno Cornelius 7ef51edfa2
Update announced addrs in switch (#784) 2021-11-30 10:12:09 +01:00
rshiv d49c95f6a1
publish addresses from announced address sequence. (#775)
* publish multiple addresses

Signed-off-by: rshiv <reeshav96@gmail.com>

* publish from announced address

Signed-off-by: rshiv <reeshav96@gmail.com>

* change in json

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes format

Signed-off-by: rshiv <reeshav96@gmail.com>

* makes listenstr a sequence

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes the wakuinfo Field

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes the test

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes the tests variable

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes test

Signed-off-by: rshiv <reeshav96@gmail.com>
2021-11-24 15:10:29 +00:00
Sanaz Taheri Boshrooyeh fda5128cc1
WIP: Waku-rln-relay: Message validation and double signaling detection (#769)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds trace log for the valid messages

* brings the log trace one line up
2021-11-23 14:48:40 -08:00
rshiv 53b21be2b5
Resolve jenkins windows error (#777)
* try different key

Signed-off-by: rshiv <reeshav96@gmail.com>

* key change

Signed-off-by: rshiv <reeshav96@gmail.com>

* cert change

Signed-off-by: rshiv <reeshav96@gmail.com>

* add newline to key

Signed-off-by: rshiv <reeshav96@gmail.com>

* exception change

Signed-off-by: rshiv <reeshav96@gmail.com>

* exception log

Signed-off-by: rshiv <reeshav96@gmail.com>

* add debug log

Signed-off-by: rshiv <reeshav96@gmail.com>
2021-11-23 11:40:43 +00:00
rshiv ac163932ad
Adds readme for websockets (#768)
* test_fleet

Signed-off-by: rshiv <reeshav96@gmail.com>

* add readme for websockets

Signed-off-by: rshiv <reeshav96@gmail.com>

* add readme

Signed-off-by: rshiv <reeshav96@gmail.com>

* add readme

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update waku/v2/README.md

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/README.md

Co-authored-by: Jakub <i+github@always.fail>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: Jakub <i+github@always.fail>
2021-11-16 10:51:05 +00:00
rshiv 6cad482ca6
Publish multiple address when websocket enabled. (#766)
* publish multiple nodes

Signed-off-by: rshiv <reeshav96@gmail.com>

* Print multiple addresses if wsenabled

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* review changes

Signed-off-by: rshiv <reeshav96@gmail.com>

* review changes

Signed-off-by: rshiv <reeshav96@gmail.com>

* code review changes

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* review fixes

Signed-off-by: rshiv <reeshav96@gmail.com>

* websocket address fix

Signed-off-by: rshiv <reeshav96@gmail.com>
2021-11-15 13:29:18 +00:00
Hanno Cornelius 067478d725
Proposal: ENR filtering on Waku-field in `discv5` (#763) 2021-11-12 15:10:54 +01:00
rshiv d2f7076b53
Secure websocket (#759)
* add config

Signed-off-by: rshiv <reeshav96@gmail.com>

* secure websocket integration

Signed-off-by: rshiv <reeshav96@gmail.com>

* secure websocket debug

Signed-off-by: rshiv <reeshav96@gmail.com>

* config change

Signed-off-by: rshiv <reeshav96@gmail.com>

* working secure websocket changes

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update submodules (#761)

* Release v0.6 (#760)

* Read from path

Signed-off-by: rshiv <reeshav96@gmail.com>

* Tests for secure websockets

Signed-off-by: rshiv <reeshav96@gmail.com>

* CI failure fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* path resolver CI

Signed-off-by: rshiv <reeshav96@gmail.com>

* self review fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update examples/v2/config_chat2.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update waku/v2/utils/peers.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/node/wakunode2.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update tests/v2/test_wakunode.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* handle review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
2021-11-10 12:05:36 +00:00
Hanno Cornelius 6d81f1ee55
Update submodules (#761) 2021-11-04 15:46:38 +01:00
Hanno Cornelius ee8ff014f4
Fix: store dimensioning issues (#758) 2021-11-03 11:59:51 +01:00
rshiv d1e06fa17a
Waku v2 Websocket support (#732)
* add config for ws support

* code clean up

* code clean up

* Integrate both ws and tcp transport

* change ws default port

* resolve review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* Unit test debug

Signed-off-by: rshiv <reeshav96@gmail.com>

* Websocket tests addition

Signed-off-by: rshiv <reeshav96@gmail.com>

* websocket failure unittest addition

Signed-off-by: rshiv <reeshav96@gmail.com>

* Commented testcase related to incorrect helper function

Signed-off-by: rshiv <reeshav96@gmail.com>

* Add wireAddr validation

Signed-off-by: rshiv <reeshav96@gmail.com>

* CLoses issue 756

Signed-off-by: rshiv <reeshav96@gmail.com>

* Websocket tests addition

Signed-off-by: rshiv <reeshav96@gmail.com>

* Minor Code clean up

* review comment fixes

Signed-off-by: rshiv <reeshav96@gmail.com>

* Review comment fix

* Update tests/v2/test_wakunode.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* Update tests/v2/test_wakunode.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Changelog update

Signed-off-by: rshiv <reeshav96@gmail.com>

* chat2 var change

Signed-off-by: rshiv <reeshav96@gmail.com>

* test flag changes

Signed-off-by: rshiv <reeshav96@gmail.com>

Co-authored-by: oskarth <ot@oskarthoren.com>
2021-11-02 10:29:11 +00:00
Hanno Cornelius f85434e072
Discv5 POC integration (#748) 2021-11-01 19:02:39 +01:00
Sanaz Taheri Boshrooyeh dbbc0f750b
RLN-Relay: Adds fix to the hash, proofGen and proofVerify procs (to support arbitrary messages) (#753)
* wip: updates the hash interface

* updates hash unittests

* exposes appendLength

* updates proof gen interface and the unittests

* enables test, fixes proofGen and verify utils and unittest, beautifies the code

* removes int size from mem index type

* enables all the rln tests

* adds documentation of two private procs

* documentation for the appendLength

* minor clean up

* adds clarificaltion on the endianness of input length

* Delete .DS_Store
2021-10-26 14:42:24 -07:00
Sanaz Taheri Boshrooyeh 6efba0dc56
Integrates proof generation and verification into wakunode2 (#735)
* WIP

* WIP: fixes a bug

* adds test for static group formation

* adds static group creation when rln-relay is enabled

* adds createStatic group

* wip: adds group formation to mount rlnrelay

* adds createMembershipList utility function

* adds doc strings and todos

* cleans up the code and add comments

* defaults createRLNInstance depth argument to 32

* renames Depth

* distinguishes between onchain and offchain modes

* updates index boundaries

* updates log levels

* updates docstring

* updates log level of displayed membership keys

* relocates a todo

* activates all the tests

* fixes some comments and todos

* extracts some utils procs for better debugging

* adds todo

* moves calculateMerkleRoot and toMembersipKeyPairs to the rln utils

* makes calls to the utils functions

* adds unit test for createMembershipList

* adds unittest for toMembershipKeyPairs and calcMerkleRoot

* cleans up the code and fixes tree root value

* reverts an unwanted change

* minor

* adds comments and cleans up the code

* updates config message

* adds more comments

* fixes a minor value mismatch

* edits the size of group

* minor rewording

* defines a const var for the group keys

* replaces the sequence literal with the StaticGroupKeys const

* adds a rudimentary unittest

* adds todos

* adds more comment

* replaces uint with MembeshipIndex type

* fixes rln relay mem index config message

* adds rln relay setup proc

* decouples relay and rln-relay

* uses MemIndexType instead of uint

* brings back the rlnRelayEnabled flag to mountRlnRelay

* deletes commented codes

* adds rln relay topic validator inside updates rln relay mounting procedure

* adds rln-relay-pubsub-topic cli option

* adds a static rln-relay topic

* deletes rlnrelayEnabled argument

* adds pubsub topic for rln-relay

* deletes static pubsub topic

* mounts relay before rlnrelay in the tests

* logs rln relay pubsub topic

* cleans up the code

* edits rlnrelay setup

* uninitializes the input parameter of rlnrelay setup

* adds comments

* removes unused comments

* compiles addRLNRelayValidtor when RLN compilation flag is set

* adds comment about topic validator

* minor

* mode modifications on the description of add validator

* adds pubsubtopic field to wakuRlnRelay type

* WIP: shaping the test

* Checks whether rln relay pubsub topic is within the supported topics of relay protocol

* minor

* WIP: unit test for actual proof

* fixes a bug

* removes a redundant proc

* refines the test for actual proof

* breaks lines to 80 chars

* defines NonSpamProof type

* adds a return

* defines Epoch type

* WIP: proof gen

* implements actual proof gen

* adds proto enc and init

* adds notes about proof structure

* adds NonSpamProof to wakumessage

* adds proof gen

* WIP: non working tests for protobuf

* fixes the protobuf encoding issue

* discards the output of copyFrom

* WIP: hash unittest and proofVrfy and ProofGen

* integrates proofVrfy

* uses toBuffer inside the hash proc

* adds comment

* fixes a bug

* removes proof field initialization

* cleans up the test

* generalizes input from byte seq to byte openArray

* adds toBuffer

* adds a bad test

* cleans up unused tests

* adds integration test

* adds comments

* cleans up

* adds description to the integration test

* adds test for unhappy path

* tides up the tests

* tides up hash unit test

* renames a few var

* uses a const for wku rln relay pubsub topic

* minor refinement

* deletes an obsolete comment

* comment revision

* adds comments

* cleans up and adds docstrings

* profGen returns proofRes instead of proof

* removes extra sleepAsync

* fixes two bugs

* returns reject when proof is not verified\

* addresses comments

* adds comments

* links to rln doc

* more comments

* fixes space format

* uncomments v2 tests

* dnsclient branch update

* undo branch update

* minor spacing fix

* makes proof field conditional
2021-10-19 17:37:29 -07:00
Hanno Cornelius 165e235158
Make maximum connections configurable (#731) 2021-10-12 14:48:48 +02:00
Pascal Precht 83f71ae905
Remove `udpPort` CLI option from waku v1 and v2 nodes (#728) 2021-10-12 13:43:01 +02:00
rshiv 5f76d90b7f Revert "wss multiaddress switch support"
This reverts commit ac76e2f189.
2021-10-06 18:03:00 +01:00
rshiv ac76e2f189 wss multiaddress switch support 2021-10-06 17:30:55 +01:00
Hanno Cornelius 50189a21a9
Update submodules: differentiate between local and remote `PeerInfo` (#730)
* Update submodules

* Remove PeerInfo where necessary
2021-10-06 14:29:08 +02:00
Sanaz Taheri Boshrooyeh 696f41e12a
Rln-Relay: Change tree depth to 20 (#726)
* adds instructions on how to generate parameters for a different tree size

* updates the depth of tree

* modifies the Merkle root

* updates parameters for the new tree size

* fixes failing tests

* updates the comment
2021-09-28 12:03:53 -07:00
Sanaz Taheri Boshrooyeh 645832e9fd
Restructuring Rln-Relay mounting process (#722)
* replaces uint with MembeshipIndex type

* fixes rln relay mem index config message

* adds rln relay setup proc

* decouples relay and rln-relay

* uses MemIndexType instead of uint

* brings back the rlnRelayEnabled flag to mountRlnRelay

* deletes commented codes

* adds rln relay topic validator inside updates rln relay mounting procedure

* adds rln-relay-pubsub-topic cli option

* adds a static rln-relay topic

* deletes rlnrelayEnabled argument

* adds pubsub topic for rln-relay

* deletes static pubsub topic

* mounts relay before rlnrelay in the tests

* logs rln relay pubsub topic

* cleans up the code

* edits rlnrelay setup

* uninitializes the input parameter of rlnrelay setup

* adds comments

* removes unused comments

* compiles addRLNRelayValidtor when RLN compilation flag is set

* adds comment about topic validator

* minor

* mode modifications on the description of add validator

* Checks whether rln relay pubsub topic is within the supported topics of relay protocol

* minor

* addresses comments

membeshipindex to membershipindex,
adds default value for rln topic
adds missing return
2021-09-24 10:32:45 -07:00
Sanaz Taheri Boshrooyeh 4895be61ec
Off-chain group construction and management (#718)
* WIP

* WIP: fixes a bug

* adds test for static group formation

* adds static group creation when rln-relay is enabled

* adds createStatic group

* wip: adds group formation to mount rlnrelay

* adds createMembershipList utility function

* adds doc strings and todos

* cleans up the code and add comments

* defaults createRLNInstance depth argument to 32

* renames Depth

* distinguishes between onchain and offchain modes

* updates index boundaries

* updates log levels

* updates docstring

* updates log level of displayed membership keys

* relocates a todo

* activates all the tests

* fixes some comments and todos

* extracts some utils procs for better debugging

* adds todo

* moves calculateMerkleRoot and toMembersipKeyPairs to the rln utils

* makes calls to the utils functions

* adds unit test for createMembershipList

* adds unittest for toMembershipKeyPairs and calcMerkleRoot

* cleans up the code and fixes tree root value

* reverts an unwanted change

* minor

* adds comments and cleans up the code

* updates config message

* adds more comments

* fixes a minor value mismatch

* edits the size of group

* minor rewording

* defines a const var for the group keys

* replaces the sequence literal with the StaticGroupKeys const

* converts var to let when applicable

* replaces hardcoded value with well-defined constants

* moves createMembershipList to the rln relay utils module

* renames HashSize to HashHexSize

* minor updates on the comments

* reorganizes the consts

* indicates that rlnRelayMemIndex is an experimental option

* fixes a type conversion bug

* revises the unittest of "mount waku rln-relay off-chain"

* clarifies the use of index

* updates a docstring

* removes redundant constants and capitalize all of them

* deletes the ETH_CLIENT const from the test file

* renames a few vars for the sake of clarity

* reorganizes unittest into blocks of execution, debug messages, and checks

* adds more comments

* more comments and clarifications

* cleans up the tests

* minor

* adds a minor fix

* replaces a var usage with let

* fixes a bug
2021-09-17 10:31:25 -07:00
Hanno Cornelius a335a40a21
Fix network simulations (#719)
* Fix/tweak network simulations

* fix terminology
2021-09-15 10:01:09 +02:00
Sanaz Taheri Boshrooyeh b7998de09d
RLN-Relay static group construction (#708)
* tests rln instance as pointer

* test insertion

* tests deletion

* tests proof generation and verification

* updates rln instance type and the rln api

* deletes old API

* removes temporary tests

* deletes unused codes

* Delete settings.json

* reverts the changes in tests v2

* removes an old comment

* adds member insertion and deletion

* adds getMerkleRoot and unit tests

* makes insertMember argument  a value type

* adds static group creation

* adds rln field to WakuRLNRelay type, and enables static group construction inside mountRlnRelay

* renames self to membershipKeyPair and removes key generation from mountRlnRelay

* renames sk,pk tp idKey and idCommitment

* updates mountRlnRelay arguments name

* logs created keys

* uncomments the key generation and adds explainer about it

* enables all the  tests

* adds comments to the rln relay types

* logs error message for the arguments that are not set
2021-08-26 16:14:51 -07:00
Hanno Cornelius 9a45aa7055
Fix: bridge encoding and missing timestamps (#697)
* Fix: bridge should not re-encode pre-encoded payload

* Test payload encoding, fix content topic

* Minor improvements

* Fix broken unit test
2021-08-26 14:24:00 +02:00
Hanno Cornelius 0db4107ae2
Integrate DNS resolution (#709)
* Update submodule

* Add DNS Client library

* Fork and update dnsclient submodule

* Integrate DNS resolution
2021-08-25 13:57:35 +02:00
Sanaz Taheri Boshrooyeh c222d83bcc
Enables passing RLN object across different function calls (#705)
* tests rln instance as pointer

* test insertion

* tests deletion

* tests proof generation and verification

* updates rln instance type and the rln api

* deletes old API

* removes temporary tests

* deletes unused codes

* Delete settings.json

* reverts the changes in tests v2

* removes an old comment
2021-08-24 12:25:29 -07:00
Hanno Cornelius 4aae5c55ac
Remove unnecessary ENR caching (#700) 2021-08-17 10:30:17 +02:00
Hanno Cornelius 36b9176569
Waku v2 Node Discovery via DNS: Integration (#690) 2021-08-12 10:51:38 +02:00
Hanno Cornelius 1ebcb266cb
Revert PRUNE backoff period to default (#694) 2021-08-12 09:57:19 +02:00
Sanaz Taheri Boshrooyeh 027cd7f95d
Simplify pagination (#689)
* initial simplification

* simplify forward pagination

* simplifies the backward pagination part

* removes old paginate proc

* revises comments, cleans up the code

* comments on initial paging query

* defines temporary variable for the paginate output
2021-08-04 15:24:36 -07:00
Hanno Cornelius 6b1e0c3079
Patch v0.5.1 - fix multiple protocol IDs in persistent storage (#687) 2021-07-27 08:48:56 +02:00
Hanno Cornelius 87b98a2460
Refactor wakunode2 setup (#675)
* Refactor wakunode2 setup
2021-07-22 11:46:54 +02:00
Hanno Cornelius 7be949ed67
Update submodules (#682) 2021-07-22 10:43:41 +02:00
Sanaz Taheri Boshrooyeh 71a532a414
Adds timestamp to waku relay message (#681)
* adds timestamp to WakuRelayMessage

* converts float to float64

* generates timestamp using epochTime()

* tides up imports and generates timestamps using epochTime()

* updates changelog

* adds timestamp to the post_waku_v2_relay_v1_message query parameters
2021-07-21 11:05:10 -07:00
Hanno Cornelius b87984af46
Fix topic conversion in bridge (#679) 2021-07-21 14:22:40 +02:00
Hanno Cornelius d44eab7cd7
Fix compilation with `rln` defined (#680)
* Fix compilation with rln defined
2021-07-21 11:37:10 +02:00
Sanaz Taheri Boshrooyeh 52b99f37af
Fixes the order of fields of History Response protobuf (#676)
* shifts history response field numbers by 1

* adds changelog

* minor rewording

* minor
2021-07-20 11:51:32 -07:00
oskarth 108599a1ed
Whisper tests (#678)
* Remove leftover comment

* Import Whisper tests

* Update copyright to 2018-2021

* Fix whisper import paths

* Whisper tests and Makefile target
2021-07-19 17:54:36 +08:00
oskarth 75ee287ca7
Adds `{.push raises: [Defect].}` to waku_swap (#677)
Doesn't try to deal with errors, just being explicit about them.
2021-07-19 16:49:27 +08:00
oskarth d3368324fe
Move Whisper types and protocol from nim-eth to this repo (#663)
* Move Whisper type and protocol from nim-eth

- Import whisper_protocol and whisper_types
- Change import locations as appropriate

* Explicit errors in wakunode1
2021-07-19 16:14:37 +08:00
Sanaz Taheri Boshrooyeh 32a8ffb867
Cleaning imports of the waku store module (#673)
* cleans up store modules

* removes endians2 as an unused imports

* reorganizes imports

* fixes an ambiguous import

* adds comments re imports order

* adds message_store
2021-07-16 15:28:35 -07:00
Kim De Mey 4b9b654655
Add top level push raises Defect to protocol v1 (#670) 2021-07-16 23:13:04 +02:00
Hanno Cornelius 13cf7380bb
General refactoring: `nim-waku` (#671)
* General Track 1 refactoring
2021-07-16 17:13:36 +02:00
Sanaz Taheri Boshrooyeh a41b9ac80b
Refactor waku protocols and utils (#667)
* refactors waku_filter

* refactors waku_lightpush

* refactors waku_swap

* refactor namespacing.nim

* refactor peers

* refactors requests

* adds top level {.push raises: [Defect].}

* log scope for rln relay

* cleans up comments

* removes comments

* comments out raise Defect

* defines temp var then pass to constructor

* Update waku/v2/protocol/waku_rln_relay/rln.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* Update waku/v2/protocol/waku_swap/waku_swap.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* explains the potential exception in waku_swap

* creates temp var before return

* adjusts spaces

* adds line breaks, temp vars and fixes format

* removes type declaration

* fixes and indentation issue

* adjusts spacing

* adjusts line <80ch

* formating improvement

Co-authored-by: oskarth <ot@oskarthoren.com>
2021-07-15 11:25:52 -07:00
Hanno Cornelius 92f2d5a7f5
Refactor wakunode2.nim (#664)
* Refactor wakunode2.nim

* Remove empty raises

* Some formatting improvements
2021-07-14 19:58:46 +02:00
Sanaz Taheri Boshrooyeh c107072d9a
Waku store: explicit return instead of result (#665)
* replaces result with returns

* fixes a missing proc call

* removes unused comments
2021-07-14 08:59:42 -07:00
Sanaz Taheri Boshrooyeh 07586153f9
Adding error message to the History Responses (#660)
* adds error field to history response

* unittest for protobuf enc/dec

* renames HistoryResponseError to Store Error and simplifies pagination logic

* reverts the naming of StoreError to HistoryResponseError

* fixes a unittest for invalid cursor

* renames OK to NONE
2021-07-13 12:01:21 -07:00
Hanno Cornelius 2869e71865
Simplify wakunode2 subscriptions model (#662) 2021-07-13 09:18:51 +02:00
Sanaz Taheri Boshrooyeh cf7b8faf27
History response pagination using sender-generated timestamp (#657)
* enables pagination based on sender timestamp

* uncomments a test

* bumps up version number

* updates migration script

* clean up

* unpdates changelog

* undo removal of receiver timestamp

* updates message_storage

* uses epochTime()

* minor

* removes a comment

* removes receiver timestamp deletion migration script

* fixes formatting issues

* fixes a bad field name

* fixes field issue

* adjusts spacing
2021-07-07 16:56:20 -07:00
Sanaz Taheri Boshrooyeh cd703c6c52
Integrating pagination into the resume proc (#653)
* adds queryFromWithPaging

* adds unit test for paged queryFrom

* adds a new resume with paging capability

* unittest for resumePaging

* fixes the bug

* implementes a second version of query with paging

* integrates the second version of resume

* clean up

* adds the error message

* fixes a format issue

* renames variables

* defines DefaultPageSize

* adds a todo

* updates a docstring

* gets the pagesize as proc input

* updates unittest description

* removes unused gcsafe pragma

* better var naming

* updates changelog

* updates changelog

* updates the TODO

* more debug logs
2021-07-02 11:37:58 -07:00
Hanno Cornelius 734fe0795f
Add support for multiple protocol IDs (#652) 2021-07-02 10:49:41 +02:00
Hanno Cornelius 52b741ac11
Support for stable version of relay protocol (#651) 2021-06-29 16:29:04 +02:00
Sanaz Taheri Boshrooyeh 81b207624b
Add db migration support for peer store (#648)
* moves migration paths to the migration_types module

* moves USER_VERSION to the migration_types module

* updates migration execution condition to cover the peer store

* adds a message to the store if it is inserted to the db

* adds debug messages and checks migration path recursively

* deletes unused debug messages

* deletes commented codes

* clean up

* fixes a bug

* renames the first migration file of the message store
2021-06-25 14:06:56 -07:00
Ebube Sered Ud 6079ae8525
Fix issues with Peer reference in Cheque Functions (#635)
* Add function for getting peer Id

* Modify Handle Cheque and Send Cheque function

* Include PeerInfo in debit and credit functions

* Minor changes to queryWithAccounting function
2021-06-22 21:55:01 +01:00
Sanaz Taheri Boshrooyeh 30a7a7a4fe
Filter out duplicate messages in the resume procedure (#600)
* adds timestamp to waku message store impl

* stores timestamp as int64

* adds untitest

* stores timestamp as seq of bytes

* minor

* re-orders unittest

* changes receiver timestamp to float64

* unit test for receiver timestamps

* adds comments

* reorder a few lines

* updates changelog

* more updates on changelog

* WIP: deduplicate

* add debug messages

* more debugging logs

* add timestamp to the message store

* converts resume result to uint64, removes some logs

* docstring for isDuplicate

* increments `added` before insertion into the db

* unit test for deduplication

* removes float to/from Bytes procs

* deletes extra spaces

* returns failed instead of the error

* minor

* minor

* adds changelog
2021-06-22 11:30:12 +08:00
Michael Bradley, Jr c5da7c8fcd Remove unused import of ./storage/sqlite from wakunode2
In the `when isMainModule:` section of wakunode2.nim, the module
`./storage/message/waku_message_store` is imported, which in turn imports
waku's sqlite module and also the sqlite3_abi module from `vendor/`.

Outside of the `when isMainModule:` section, imports from waku's sqlite module
are not used in wakunode2. However, importing it at the top-level results in
the C code underlying sqlite3_abi to be compiled into whatever executable one
is building, and this is a problem if nim-waku is being consumed together with
another library, e.g. nim-sqlcipher, which links against a different build of
sqlite (sqlcipher being a superset of sqlite). The linker step fails with
reports of duplicate symbols for sqlite.
2021-06-18 13:56:37 -05:00
Ebube Sered Ud c1763362d7
Add Issuer to Cheque Object (#627)
* Add Issuer to Cheque Object closes  #626

* Minor changes

* Fix typo in comment

* Improve docs

* Fix Typo In Comment

* Minor changes

* Minor changes
2021-06-18 16:53:51 +01:00
Hanno Cornelius 2f53c2c60b
Feat/bridge topic mapping (#633)
* Use namespaced content topics for bridge
2021-06-18 12:30:24 +02:00
Ebube Sered Ud eb49d3148c
Improve Swap Account Balance Metrics to make use of a Histogram (#628)
Modify metrics description

Add more buckets to the swap account metrics
2021-06-17 16:36:44 +01:00
Hanno Cornelius 9ee0c2ecb3
Make bridge topics configurable (#629)
* Make bridge topics configurable

* Less ambiguous config naming
2021-06-17 12:59:28 +02:00
Sanaz Taheri Boshrooyeh ddf93814fe
Persisting Waku message timestamp & implementing DB migration & convert receiver timestamp data type to float64 (#607)
* adds timestamp to waku message store impl

* stores timestamp as int64

* adds untitest

* stores timestamp as seq of bytes

* minor

* re-orders unittest

* changes receiver timestamp to float64

* unit test for receiver timestamps

* adds comments

* reorder a few lines

* updates changelog

* more updates on changelog

* WIP

* WIP

* adds migration

* more debug messages

* passes the path to the migration scripts from message store module

* adds migration result type

* replaces migrationScripts with migrationScriptsResult

* adds path calculation to the message store

* removes some tests binary file

* removes redundant imports

* comments out user_version assignment in sqlite  init

* more descriptive err messages

* clean up test file

* more info logs

* minor code format

* removes a todo

* minor updates

* remove a binary file

* unit tests for migration utils

* adds split script

* integrates split query to handle scripts with multiple commands

* updates migration script for v1

* updates the v1 migration script

* update user version

* updates script

* fixes a few bugs on the splitScript

* more debug logs

* adds float64 parameter support to sqlite3

* change in timestamp type in the script

* deletes float64 toBytes utils

* enables storage of timestamp as a real number in the sqlite db

* bump up script index

* comment edits

* removes migrate unit test

* adds todo and docstring

* updates changelog

* removes an unused item in .gitignore

* minor

* updates changelog

* organizes imports

* cleans up imports

* WIP

* updates script


fixes a few bugs on the splitScript


more debug logs


adds float64 parameter support to sqlite3


change in timestamp type in the script


deletes float64 toBytes utils

* edits migration util test

* remove an empty test file

* includes migration utils tests in

* deletes unused codes

* tides up imports

* adds range based filter to the filterMigrationScripts

* renames procs: removes Migration

* tides up imports

* edits docstring

* edits docstring

* edits docstring

* removes unused imports

* more clean up

* groups std imports

* updates changelog

* adds docstring for setUserVersion

* adds unittest for the migrate

* Update waku/v2/node/storage/message/waku_message_store.nim

Co-authored-by: RichΛrd <info@richardramos.me>

* Update waku/v2/node/storage/sqlite.nim

Co-authored-by: RichΛrd <info@richardramos.me>

* Update waku/v2/node/storage/sqlite.nim

Co-authored-by: RichΛrd <info@richardramos.me>

* removes split scripts

* fixes a naming issue

* fixes a bug

* fixes a typo

* adds a log re updated user_version

* fixes a proc naming mismatch

* fixes naming mismatch

* more descriptive var names

* adds migration script of the first user version

* moves migration to after persistMessages flag is checked

* deletes unused comment

* fixes a bug

* brings back split script

* adds unit tests for split scripts

* runs scripts one command at a time

* deletes a commented line

* relocates the migrate proc to sqlite.nim

* adds unit test for filter scripts

* adds filterScripts unittest testing varying zero-prefixed user versions

* minor

Co-authored-by: RichΛrd <info@richardramos.me>
2021-06-16 13:23:55 -07:00
Hanno Cornelius a044c6a82c
Add ping protocol support and use for keep-alive (#621)
* Add ping protocol support and use for keep-alive

* Separate WakuPeerStore from libp2p PeerStore

* Revert silly test timeout
2021-06-15 10:55:47 +02:00
Kim De Mey e2ea94dc71
Remove the need for pcre (#623) 2021-06-15 09:51:09 +02:00
Ebube Sered Ud 76d4b89dbb
Add mode to waku swap (#601)
* Add mode to waku swap

* Modify swap test  and policy function

* Make rln optional (dependencies and compilation) (#602)

* makes rlnlib target optional

* makes the rln-related targets conditioned to the RLN command line flag

* adds the rln compiler flag condition

* adds the rln compiler flag condition to the rln module

* wakunode2 rln conditional compilation

* updates wakunode2 test

* updates waku rln relay tests

* removing the rln test from the default imports of the v2 tests

* imports rln module conditionally

* removes the rln flag condition from the rln module

* separates rln data types from its procs

* adds the import statement

* brings back the contract def to the rln utils

* adds rln module import to the rln unit tests

* clean up and reorganization

* adds a todo

* minor edits on a comment

* Fix indentation and typo

* Add Swap Config Object and set default config value on mount

* Minor changes

* Fix issues with grammar in code comment

* Create init procedure for setting default values in SwapConfig

* Minor changes to mountSwap call in wakunode2

* Improve Docs on Swap Mode

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2021-06-15 10:06:36 +08:00
Kim De Mey 4ca59bcce0
Use chronosStrictException for Waku v1 code (#609)
* Use chronosStrictException for Waku v1 code

And accompanying changes and clean-up

* Fix chronosStrictException changes for wakubridge
2021-06-14 14:40:08 +02:00
Oskar Thorén be717ce5f3
Improve libwaku and C wrappers (#613)
- Move stuff to Makefile
- Move libwaku to root
- Make examples compile for C and Go
- Update README
2021-06-13 20:50:10 +08:00
Hanno Cornelius a8da4dfd6b
Fix: metrics-server use after submodule update (#610)
* Fix: metrics-server use after submodule update

* Remove insecure check
2021-06-10 16:18:41 +02:00
Hanno Cornelius 9e0b73bd18
Update submodules (#594) 2021-06-09 16:37:08 +02:00
Sanaz Taheri Boshrooyeh 2fe53c334c
Make rln optional (dependencies and compilation) (#602)
* makes rlnlib target optional

* makes the rln-related targets conditioned to the RLN command line flag

* adds the rln compiler flag condition

* adds the rln compiler flag condition to the rln module

* wakunode2 rln conditional compilation

* updates wakunode2 test

* updates waku rln relay tests

* removing the rln test from the default imports of the v2 tests

* imports rln module conditionally

* removes the rln flag condition from the rln module

* separates rln data types from its procs

* adds the import statement

* brings back the contract def to the rln utils

* adds rln module import to the rln unit tests

* clean up and reorganization

* adds a todo

* minor edits on a comment
2021-06-08 11:56:32 -07:00
Ebube Sered Ud e2185caf27
Add Swap account Metrics to Grafana (#585)
* Fix gcsafe issues with log account metrics function

Signed-off-by: kripsonud <kripsonud@gmail.com>

* Change Line ending to LF

* Minor changes to waku_swap

* Mount keepalive on chat2bridge and bridge (#591)

* Light push beta bump (#590)

Closes https://github.com/status-im/nim-waku/issues/565

* Filter and lightpush showcase (#593)

* Filter and lightpush showcase

* Fix log error

* Add swap account panel to grafana dashboard template

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-06-06 16:15:18 +01:00
Hanno Cornelius b615c6aa8b
Filter and lightpush showcase (#593)
* Filter and lightpush showcase

* Fix log error
2021-06-04 12:02:47 +02:00
Oskar Thorén 2568af3b98
Light push beta bump (#590)
Closes https://github.com/status-im/nim-waku/issues/565
2021-06-03 13:29:04 +08:00
Hanno Cornelius 83f546ede4
Mount keepalive on chat2bridge and bridge (#591) 2021-06-02 16:54:38 +02:00
Hanno Cornelius 2b571e205b
Fix keepalive for connected peers (#588)
* Fix keepalive for connected peers

* Remove comment
2021-06-02 09:53:34 +02:00
Hanno Cornelius ee757308c6
Fix: No keep-alive for backoff peers (#584) 2021-06-01 09:55:46 +02:00
Sanaz Taheri Boshrooyeh 8f741de483
A quick fix to the master issue (#579)
* moves resume to after the relay is mounted

* Update waku/v2/node/wakunode2.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2021-05-28 11:41:29 -07:00
Ebube Sered Ud cc5f55dbbb
Bump swap protocol to beta1 (#578)
* Bump swap protocol to beta1
2021-05-28 17:13:17 +01:00
Sanaz Taheri Boshrooyeh f75f7c4751
Add resume to waku node api (#564)
* activates resume

* adds unit test

* add store condition

* mode debug info

* updates wakunode resume api and its unittest

* adds todo

* adds documentation

* edits resume documentation

* further edits on the docs

* removes a todo

* fixes a bug

* add resume to waku node api

* further updates on the node.md

* updates the changelog

* minor

* removes return type

* adds a brief desc of the resume to node.md
2021-05-26 12:33:22 -07:00
Ebube Sered Ud e6d7e3a2b4
Improve SWAP logging and enable by default (#549)
* Enable Swap by default
* Add PeerId and amount to Logs
* Add Policy function in swap protocol
* Modified Changelog to reflect changes to swap config
* Commented out test for updating account state after a cheque has been sent
2021-05-26 11:05:56 +01:00
Hanno Cornelius e0d8d8f468
Minor bridge improvements (#563) 2021-05-24 13:19:33 +02:00
Sanaz Taheri Boshrooyeh c9e94cba41
Enable time-based history query in the store rpc api (#560)
* adds time based query

* updates the changelog
2021-05-21 10:36:24 -07:00
Hanno Cornelius 2691b3e506
Duplicate filtering for bridge (#556) 2021-05-20 18:03:56 +02:00
Sanaz Taheri Boshrooyeh 9b38e5c893
Ft-store: Peer management (#548)
* adds an optional list of peers to the resume proc

WIP working


adds unit tests

* cleans up and adds comments

* adds unittest for queryFrom

* converts queryLoop to a private func

* elaborates on the peer selection of the resume proc

* minor format correction

* returns the status of the resume call as a Result object

* updates unittest based on the new return type

* defines QueryResult type
2021-05-19 12:28:09 -07:00
Sanaz Taheri Boshrooyeh 89909d281f
Fault tolerant store protocol: resume message history (#539)
* adds resume function

* unittest for findLastSeen

* fixes a bug in find last seen unit test

* argument type change to indexed waku message

* relocates findLastSeen

* adds unit test for resume

* adds comments

* adds a TODO

* adds offset

* cleans up

* modifies some of tests title and deletes a duplicate unit test

* more condition on running resume()

* adds more detailed docstring and simplifies if statement

* Update waku/v2/protocol/waku_store/waku_store.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* new doc string

* Update waku/v2/protocol/waku_store/waku_store.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* fixes a bug

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-05-13 14:21:46 -07:00
Hanno Cornelius 1ae767206d
Remove deprecated config items (#545) 2021-05-12 12:55:09 +02:00
Hanno Cornelius 50f2235bfc
Fix nim-waku config inconsistencies (#543)
* Fix nim-waku config inconsistencies

* Revert staticnode to singular form

* Fix reference to deprecated config item

* Revert nodekey config item to be consistent with Waku v1
2021-05-11 17:07:26 +02:00
Hanno Cornelius c5e7580149
Add chat2-matterbridge (#532)
* Add chat2-matterbridge

* Update waku/common/README.md

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-05-06 15:43:43 +02:00
Sanaz Taheri Boshrooyeh 7c5df3379b
Storing waku message version inside the store db (#530)
* WIP

* stores version in db, edits error messages

* fixes a bug

* reverts edit messages

* converts uint32 storage to int64

* unit tests for various version numbers

* minor comments edit

* removes debugging messages

* updates changelog
2021-05-05 15:51:36 -07:00
Ebube Sered Ud a6bb3f65e1
Fix redundant use of content topics (#528)
made resulting changes to waku_filter

Made changes to wakunode2, filter_api and waku_filter

Update waku v2 test scripts referencing Content Topics

Update ContentFilter in chat example

Remove unneccesary loops from filter api
closes #496

Apply keep-alive for chat2 (#525)

Makes the arguments of the store jsonrpc api optional (#526)

* makes pubsubTopic filter optional

* makes contentFilter optional

* append Option

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Update Changelog with changes to ContentFilter

Fix indentation and code semantics

Enables perssist-message flag in the store protocol for wakunode2 (#519)

* enables perssistmessage flag

* disables in memory storage when persist-messages is false

* adds the persistMessages input to the mountStore

* defaults the store flag to true

* adds the missing argument

* persists messages in memory conditioned to the persistMessages flag

* adds persistmessages flag to the config_bridge

* defaults persistmessages to true

* defaults the store flag to true and persist-messages to false

* updates store.md

* updates chat2 instructions about --store flag

* removes --store flag from chat2 command execution

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Fix: light-mode relay for all light protocols (#529)

* Fix: light-mode relay for all light protocols

* Clear up confusing use of overloaded concepts

Fix ContentFilter Schema in wakunode test script

Enables perssist-message flag in the store protocol for wakunode2 (#519)

* enables perssistmessage flag

* disables in memory storage when persist-messages is false

* adds the persistMessages input to the mountStore

* defaults the store flag to true

* adds the missing argument

* persists messages in memory conditioned to the persistMessages flag

* adds persistmessages flag to the config_bridge

* defaults persistmessages to true

* defaults the store flag to true and persist-messages to false

* updates store.md

* updates chat2 instructions about --store flag

* removes --store flag from chat2 command execution

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Fix: light-mode relay for all light protocols (#529)

* Fix: light-mode relay for all light protocols

* Clear up confusing use of overloaded concepts

Fix resulting issues after merge
2021-05-05 09:34:40 +01:00
Hanno Cornelius 36ff537612
Fix: light-mode relay for all light protocols (#529)
* Fix: light-mode relay for all light protocols

* Clear up confusing use of overloaded concepts
2021-05-04 14:11:41 +02:00
Sanaz Taheri Boshrooyeh c0d858af1d
Enables perssist-message flag in the store protocol for wakunode2 (#519)
* enables perssistmessage flag

* disables in memory storage when persist-messages is false

* adds the persistMessages input to the mountStore

* defaults the store flag to true

* adds the missing argument

* persists messages in memory conditioned to the persistMessages flag

* adds persistmessages flag to the config_bridge

* defaults persistmessages to true

* defaults the store flag to true and persist-messages to false

* updates store.md

* updates chat2 instructions about --store flag

* removes --store flag from chat2 command execution

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-05-03 14:30:52 -07:00
Sanaz Taheri Boshrooyeh 7b6d93d13b
Makes the arguments of the store jsonrpc api optional (#526)
* makes pubsubTopic filter optional

* makes contentFilter optional

* append Option

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-05-03 12:25:28 -07:00
Hanno Cornelius f457db9e0a
Add optional keepAlive for relay peers (#523) 2021-04-30 14:07:46 +02:00
Hanno Cornelius 967db6d610
Allow relay=false for light filter nodes (#518) 2021-04-29 12:54:31 +08:00
Sanaz Taheri Boshrooyeh 7e9dac67c8
Integrates pubsub topic filter in the store json rpc api (#514)
* replaces topics with seq of ContentFilters

* update topics to contentFilter

* updates the contentFilter  structure

 one content  topic per content filter instead of a sequence of topics

* updates store json rpc api

* renames ContentFilter to HistoryContentFilter

* unit test for a query with several content filters

* makes shortcut for store api

* updates chat2

* clean up

* renames topic to contentTopic

* adds pubsub topic to the history query

updates message store interface to return the pubsub topic
updates waku message store implementation
updates database schema to hold pubsub topi per waku message

* clarifies the use of content topic in store api

* clarifies the use of contentTopic in the init method of HistoryContentFilter

* simplifies the test and add comments

* lowers the field number of pubsub topic in historyQuery  protobuf

* captures an empty contentFilter case

* test pubsub topic filter for the entire history and no message match

* demoves duplicates

* adds TODO

* fix a broken comment line

* updates waku store codec

* swaps the order of pubsub topic and content topic in protobuf

* Update waku/v2/protocol/waku_store/waku_store_types.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* updates the pubsub topic to the default value

* bumps protocol id

* moves the comment close to IndexedWakuMessage

* adds checks to the store put method

* makes table title a constant variable and retitles the table to Message

* updates the changelog

* minor update

* minor

* adds pubsubtopic to the store json rpc api

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-28 19:14:03 +08:00
Oskar Thorén 0d5e14f83c
Libwaku spike start (#510)
* libwaku init e2e

* basic cstring check

* wip
2021-04-28 19:13:34 +08:00
Sanaz Taheri Boshrooyeh 50a54cb4ee
Enabling pubsub topic filter in history queries (#492)
* replaces topics with seq of ContentFilters

* update topics to contentFilter

* updates the contentFilter  structure

 one content  topic per content filter instead of a sequence of topics

* updates store json rpc api

* renames ContentFilter to HistoryContentFilter

* unit test for a query with several content filters

* makes shortcut for store api

* updates chat2

* clean up

* renames topic to contentTopic

* adds pubsub topic to the history query

updates message store interface to return the pubsub topic
updates waku message store implementation
updates database schema to hold pubsub topi per waku message

* clarifies the use of content topic in store api

* clarifies the use of contentTopic in the init method of HistoryContentFilter

* simplifies the test and add comments

* lowers the field number of pubsub topic in historyQuery  protobuf

* captures an empty contentFilter case

* test pubsub topic filter for the entire history and no message match

* demoves duplicates

* adds TODO

* fix a broken comment line

* updates waku store codec

* swaps the order of pubsub topic and content topic in protobuf

* Update waku/v2/protocol/waku_store/waku_store_types.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* updates the pubsub topic to the default value

* bumps protocol id

* moves the comment close to IndexedWakuMessage

* adds checks to the store put method

* makes table title a constant variable and retitles the table to Message

* updates the changelog

* minor update

* minor

* beta2 to beta3

* minor

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-27 16:52:24 -07:00
Sanaz Taheri Boshrooyeh 7bb58d089d
Fix store protocol identifier (#507)
* beta1 to beta2

* removing Lightpush message return success test to spot windows err

* bring back the Lightpush test
2021-04-27 15:37:50 -07:00
Oskar Thorén 13ac035e5a
Lightpush protocol cont (#506)
* lightpush conf and mount with relay from node

* mount lightpush after relay

* lightpush relay integration wip

* lightpush node integrate and test node

* clean
2021-04-24 12:56:37 +08:00
Sanaz Taheri Boshrooyeh a04643e9a2
Updates store json rpc api (#499)
* updates the store jsonrpc api and its usage across the codebase

* deletes unused codes

* deletes commented codes

* minor

* spacing issue
2021-04-23 14:14:40 -07:00
Oskar Thorén d7de633793
Light push protocol initial cut (#503)
* Init lightpush

* Lightpush types

* light push encode and decode

* lightpush protocol more

* light push test draft

* lightpush compiles

* lightpush fix tests etc

* cleanup

* pubsubTopic -> pubSubTopic
2021-04-23 14:56:04 +08:00
Hanno Cornelius f1a6027158
Fix filter notifications (#501) 2021-04-22 15:45:13 +02:00
Hanno Cornelius f0eadfec13
Add persistent backoff for peers (#497) 2021-04-21 11:36:56 +02:00
Sanaz c997860397
undo the last unwanted commit 2021-04-20 17:06:26 -07:00
Sanaz f7fbc28780
updates protocol identifier 2021-04-20 16:37:11 -07:00
Oskar Thorén 5f1bab146d
Log more in store protocol (#493)
Makes it easier to see what is going on
2021-04-20 09:59:14 +08:00
Sanaz Taheri Boshrooyeh fb2ea06a4f
Update store protocol interface: add history content filter (#488)
* replaces topics with seq of ContentFilters

* update topics to contentFilter

* updates the contentFilter  structure

 one content  topic per content filter instead of a sequence of topics

* updates store json rpc api

* renames ContentFilter to HistoryContentFilter

* unit test for a query with several content filters

* makes shortcut for store api

* updates chat2

* clean up

* renames topic to contentTopic

* clarifies the use of content topic in store api

* clarifies the use of contentTopic in the init method of HistoryContentFilter
2021-04-19 10:38:30 -07:00
Hanno Cornelius 192d4dd66b
Add backoff period before subscribing to persistent peers (#491) 2021-04-16 11:57:45 +02:00
Hanno Cornelius 900d53f9df
Implement graceful shutdown for `wakunode2` and `chat2` (#490)
* Implement graceful shutdown

* Consistent use of QuitSuccess
2021-04-15 10:18:14 +02:00
Hanno Cornelius 17cfc32c4f
Add default handler for all topics (#483)
Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-13 07:56:49 +02:00
Sanaz Taheri Boshrooyeh 286482ea32
Handling initial paging requests over message history of size one (#479)
* handles initial paging query for history of size one

* adds two more tests to fetch the entire history in the initial paging request

* increases the pageSize
2021-04-12 10:29:09 -07:00
Sanaz Taheri Boshrooyeh 38b7a257c5
Enabling time window history query (#454)
* adds time based query

* handling temporal history queries

* more tests for corner cases

* edits tests comments

* further comments

* updates the time query logic

queries with zero-size window will result in an empty response

* returns no messages for invalid time queries

* comment update

* converts contentTopics from int to string

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-09 16:40:34 -07:00
Hanno Cornelius 58dd431779
Fix Windows CI (#467)
* Force chronological message history

* Fix start/end index computation

* Sleep before RPC call

* Lower default max cache sizes on APIs

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-09 13:47:24 -07:00
Sanaz Taheri Boshrooyeh 6a4fcaa671
Fix the issue in the order of fields of HistoryQuery (#470)
Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-09 09:50:51 -07:00
Sanaz Taheri Boshrooyeh 5f0e6201ed
Change page size type to uint64 (#469)
* changes pageSize to uint64

* WIP

* reworks tests and indexing proc to match uint64 pageSize

* minor simplification

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-09 18:04:21 +08:00
RichΛrd b5f1becace
fix: direction encoding (#464)
* fix: direction encoding

* fix: decoding pagingInfo direction
2021-04-09 11:09:20 +08:00
Hanno Cornelius 5747ff5be0
Change contentTopic to string (#463)
* Change contentTopic to string

* Missed a spot

* Try to fix Windows CI

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-08 17:55:19 +08:00
Sanaz Taheri Boshrooyeh d49cb53b56
Updates the field number of proof and timestamp (#457)
* updates the fields numbers of proof and timestamp

* clarifies that proof is not part of waku message specs

* Update waku/v2/protocol/waku_message.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* fixes indentation

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-07 11:08:46 -07:00
Sanaz Taheri Boshrooyeh a0dba2fea4
adds startTime and endTime to the HistoryQuery (#451) 2021-04-02 15:53:28 -07:00
Sanaz Taheri Boshrooyeh 192c2c7ea0
Add timestamp field to the waku message (#450)
* adds timestamp field

* unit test for timestamp

* adds comments

* updates test title
2021-04-02 14:28:46 -07:00
Hanno Cornelius 683577f045
Start relay protocol on wakunode2 (#446)
* Start relay protocol on wakunode2
2021-04-01 11:37:14 +02:00
Sanaz Taheri Boshrooyeh e7c21c2f74
Rln-relay zkp module Nim bindings (#427)
* entirely replaces the prior rln header, the var variables are changed to ptr

* updates the unittest of key_gen

* adds test for update_next_member

* updates membershipKeyGen internals and prototype

* adds createRLNInstance

* adds helpers methods

* adds generateKeyPairBuffer

* cleans up the test and adds comments

* renames  merkleTreeDepth to d

* fixes a buf re decoding the keys into sk and pk

* adds getSKPK proc

* unifies key gen helper procs, adds todos

* comments out the createRLNInstance

* refactors the code based on the updated createRLNInstance interface

* adds the test for the verify proc

* fixes a variable name and replaces random key gen with the real key gen

* tests a simple hash

* adds get_root method

* fixes the data pointer issue and adds the proof breakdown

* adds rln

* adds unit tests for Merkle tree

* adds a sample hash test

* fixes the hash bug and comments out unused part of proof gen test

* cleans up the proof gent test

* replaces unsafeAddr with addr

* fixes an issue in key gen

* updates rln submodule

* fixes the verification problem

* adds a failed test

* replaces an old test scenario with a new one

* handles createRLNInstance output

* working createRLNInstance2

* refactors the code by replacing the old createRLNInstance

* renames createRLNInstance2

* adds documentation and reorganizes rln.nim

* replace echo with debug, renames vars, adds a bad proof test

* minor

* minor

* edits var names

* adds one more check

* adds one more test to the hash

* enforcing exception handling

* adds pacman -Sy

* removes update:true

* activates update
2021-03-31 17:39:27 -07:00
Hanno Cornelius aa9a74c60b
Fix nil error when starting wakunode2 with store/filter node (#441) 2021-03-30 15:33:19 +02:00
Kim De Mey d1c1a0ca13
Bump chronos and fix exception tracking issues (#436)
* Bump nim-chronos and fix exception tracking issues

* Bump other Nim submodules to latest

* Fix repeatMessage properly through proc type fix in nim-eth

Also add and use unittest2 through testutils to avoid extra
annotations.
2021-03-26 10:52:04 +01:00
Hanno Cornelius 5102576234
Integrate persistent peer storage (#437)
* Integrate persistent peer storage
2021-03-26 10:49:51 +02:00
Hanno Cornelius 7a732e7cc6
Added persistent storage for peers (#435) 2021-03-25 10:37:11 +02:00
Sanaz Taheri Boshrooyeh 29d69b98cb
Adds Merkle tree Nim bindings (#430)
* entirely replaces the prior rln header, the var variables are changed to ptr

* updates the unittest of key_gen

* adds test for update_next_member

* updates membershipKeyGen internals and prototype

* adds createRLNInstance

* adds helpers methods

* adds generateKeyPairBuffer

* cleans up the test and adds comments

* renames  merkleTreeDepth to d

* fixes a buf re decoding the keys into sk and pk

* adds getSKPK proc

* unifies key gen helper procs, adds todos

* comments out the createRLNInstance

* refactors the code based on the updated createRLNInstance interface

* adds the test for the verify proc

* fixes a variable name and replaces random key gen with the real key gen

* tests a simple hash

* adds get_root method

* fixes the data pointer issue and adds the proof breakdown

* adds rln

* adds unit tests for Merkle tree

* deletes unnecessary comments

* updates createRLNInstance to return bool indicating the success of call

* updates create RLN Instance interface

* minor

* clean up

* removes unused imports

* adds documentation

* adds comments

* adds byteutils

* removes extra spaces

* updates rln submodule

* deletes genSKPK

* fixes a bug in membershipKeyGen

* unsafeAddr to addr

* Update waku/v2/protocol/waku_rln_relay/rln.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* clean up

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-03-24 10:26:56 -07:00
Hanno Cornelius a30d6a8aae
Improvement/bridge improvements (#429)
* General test, import and log improvements

* Bridge improvements
2021-03-23 10:04:51 +02:00
Hanno Cornelius 2fe6935623
Fix marshalling of message payloads (#428)
Fix marshalling of message payloads
2021-03-22 17:13:56 +02:00
Hanno Cornelius 62b824c387
Add WakuBridge and test (#426)
* Add WakuBridge and test

* Fix test
2021-03-22 11:44:45 +02:00
Hanno Cornelius 4ae1cd4737
Add message passing to waku bridge script (#425) 2021-03-18 12:01:27 +02:00
Hanno Cornelius 0050f9da59
Add hook on newly added envelope (#423) 2021-03-17 11:49:06 +02:00
Hanno Cornelius 714bbebbc1
Update submodules (#424)
Update submodules and fix issues
2021-03-17 10:52:27 +02:00
Sanaz Taheri Boshrooyeh d64aadc8ad
Basic proof gen and vrfy (#421)
* adds the proof field to the WakuMessage

* adds a basic zkProof api

* adds proof gen to the publish proc

* relocates the proofGen and proofVrfy to rln_relay_utils

* wip: test of proof gen

* adds a procedure to pass rln-relay message validator, adds the proof gen

* tests the proof gen and verify

* relocates zkp API

* adds documentation and todos

* adds todos and documentations

* removes an unnecessary comment

* adds todos
2021-03-16 11:18:40 -07:00
Hanno Cornelius a575c44934
Fix Relay API: handle messages on previously subscribed topics (#413) 2021-03-11 09:48:59 +02:00
Hanno Cornelius e6b26cc059
Add post method to Admin API for ad-hoc peer dialing (#412)
* Add post method to Admin API

* Updated CHANGELOG
2021-03-11 09:05:39 +02:00
Hanno Cornelius 55096cbd1d
Improved description for config items (#410) 2021-03-10 09:03:22 +02:00
Hanno Cornelius 398b701f4f
Minor fixes for chat2 and store (#407) 2021-03-08 09:45:10 +02:00
Hanno Cornelius cc962f2dd5
Further chat2 improvements (#405) 2021-03-04 09:19:21 +02:00
Sanaz Taheri Boshrooyeh 168f8d1e1d
Adds rln as a submodule (#402)
* adds rln as submodule

* removes the duplicate rln submodule

* adds rln

* Delete waku.nims
2021-03-02 12:57:48 -08:00
Oskar Thorén 0e40bf6e92
Swap redeem cheque and update accounting state accordingly (#397)
* Redeem cheque when recevived

* Check balance, ish

* conditional update accounting after redeem cheque

* Add override for account update state w/o cheque

* Update waku/v2/protocol/waku_swap/waku_swap.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
2021-03-01 13:55:20 +08:00
Oskar Thorén f85597a7cb
Better error handling with Results from interactions with swap-contracts-module (#393)
* Fix balance JSON resp

* Resultify swap contract calls for error handling

* More resultify and fixup tests

* More results

* resultify waku swap handling

* re-enable waku swap test

should not crash now, even if it fails to settle (tested separately)
2021-02-25 10:47:48 +08:00
Oskar Thorén b730d79438
Waku Swap: sign and send cheque (#386)
* fix balance test

* move waku swap contracts into own module

* Move more to separate waku swap contracts

* move erc20address

* Improve cmd/task util

* fix redeem part too

* sign cheque hardcode

* add signature field, send cheque

* echo->debug

* disable waku swap test

it calls contracts, which isnt integrated
2021-02-23 13:15:06 +08:00
Sanaz Taheri Boshrooyeh e152ed349f
Integrate registration into wakunode and Enable signed registration transactions (#385)
* cleans up imported modules

* adds uploadContract proc and consolidates rln-relay test files

* deletes test_rln_relay_wrappers

* deletes wrappers tests

* adds waku_rln_relay_utils

* adds the unit test for the membership key generation

* adds the key generation procedure

* adds unit test of key gen proc

* adds RLNRelayPeer data type

* adds the register proc

* adds the register proc unit test

* minor

* edits registration test

* adds comments to the registration unit test and relocates some constants

* defines constants variables for membership contract inputs and adds todos

* fixes a typo

* enables signed transaction

* adds registration into waku rln relay

* adds a TODO

* adds debug notes

* WIP add registration test

* integrates rln relay inputs into the mountRelay proc

* minor

* adds mountRelay

* renames RlnRelayPeer to WakuRlnRelay

* adds debugging notes

* changes the test title

* adds rln to gitignore

* cleans up tests, fixes a bug

* exposes mountRlnrelay, adds some comments

* fixes a bug

* deletes async pragma from mountRelay

* exposes the wakuRlnRelay field

* adds bash command to delete any existing rln directory before cloning the rln repo

* removes -v verbose flag from rln removal

* replaces await with WaitFor

* embed mountRlnRelay inside mountRelay
2021-02-22 09:40:02 -08:00
Sanaz Taheri Boshrooyeh 57a9447972
Rln relay registration (#378)
* cleans up imported modules

* adds uploadContract proc and consolidates rln-relay test files

* deletes test_rln_relay_wrappers

* deletes wrappers tests

* adds waku_rln_relay_utils

* adds the unit test for the membership key generation

* adds the key generation procedure

* adds unit test of key gen proc

* adds RLNRelayPeer data type

* adds the register proc

* adds the register proc unit test

* minor

* edits registration test

* adds comments to the registration unit test and relocates some constants

* defines constants variables for membership contract inputs and adds todos

* fixes a typo

* adds rln to gitignore

* moves all the types to the top

* adds contracts inputs to the utils module

* adds a TODO

* adds some comments

* removes newline! adds some comments
2021-02-19 11:44:18 -08:00
Sanaz Taheri Boshrooyeh b7ce32cf4b
Rln relay key generation (#383)
* cleans up imported modules

* adds uploadContract proc and consolidates rln-relay test files

* deletes test_rln_relay_wrappers

* deletes wrappers tests

* adds waku_rln_relay_utils

* adds the unit test for the membership key generation

* adds the key generation procedure

* adds unit test of key gen proc

* removes extra spaces

* Update tests/v2/test_waku_rln_relay.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* deletes the newline

* adds rln to gitignore

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-02-18 14:59:10 -08:00
Sanaz Taheri Boshrooyeh f64c701821
rlnlib dependency (#376)
* adds rlnlib dependency

* clones the full-node branch of rln repository

* moves rlnlib dependency to the deps target

* changes rlnlib path

* adds test_rln_relay_wrappers to the v2 tests

* loads rlnlib with different extensions based on the OS

* deletes unnecessary imports

* changes rlnlib extension to .exe for windows

* minor

* list files in rln/target/debug

* edits rln lib name for windows

* removes unnecessary commands

* edits some comments

* fetches specific commit of rln library
2021-02-12 12:12:10 -08:00
Hanno Cornelius 3b6db72287
Feat/pm connection tracking (#377)
* Track connectedness state in peer manager
2021-02-12 10:53:52 +02:00
Hanno Cornelius bf0eab4a48
Removed local peer sets for filter, swap and store (#375) 2021-02-11 10:58:25 +02:00
Sanaz Taheri Boshrooyeh 0c60e6537f
Nim wrappers for Rln lib (Key and Params generation) (#373)
* adds the initial rln.nim wrapper

* adds test_rln_wrapper

* WIP: initial test (not working)

* changes RLNBn256 data type

* updates parameters type

* adds the wrappers and fixes the bugs

* adds tests

* replaces echo with debug
2021-02-10 12:51:41 -08:00
Hanno Cornelius 1b96ee0adb
Integrate peer manager with Swap and Store (#370) 2021-02-09 10:31:38 +02:00
Hanno Cornelius 1d9e3afaa4
Peer manager: improvements; waku_filter integration (#368)
* Integrate peer manager with waku_filter

* Changelog and misc PR suggestions
2021-02-08 11:17:20 +02:00
Hanno Cornelius 1f5c3cc621
Feat/peer manager improvements (#367)
* Add managed peers to Admin API result

* Deal with dial failures
2021-02-05 12:49:11 +02:00
Hanno Cornelius 080d82dd9a
Added basic peer manager (#364)
* Added basic peer manager
2021-02-04 12:32:58 +02:00
Hanno Cornelius 9faff49e5b
Remove stale rpc code & repair scripts (#365)
* Remove stale rpc code & repair scripts
2021-02-04 10:39:55 +02:00
Hanno Cornelius 41fe4395a8
Bump submodules (#363)
* Bump submodules

* GossipSub refactor: Rem async from sub/unsub
2021-02-02 13:33:59 +02:00
Hanno Cornelius 886b458ff5
Added waku metrics (#356) 2021-01-29 10:42:41 +02:00
Hanno Cornelius ed979a7d60
Fix network simulation and remote peer ID CLI argument parsing (#351)
* Fix network simulation and remote peer ID CLI argument parsing
2021-01-25 13:03:52 +02:00
Hanno Cornelius e8dd014079
Extract MessageStore interface (#350) 2021-01-22 11:39:16 +02:00
Hanno Cornelius 76c0d2b839
Prefer announced addrs in listenStr (#348)
* Prefer announced addrs in listenStr
2021-01-15 09:37:20 +02:00
Sanaz Taheri Boshrooyeh cc1fc15d38
Adds rln-relay flag (#347)
* adds rlnrelay flag

* minor edit

* adds the input name
2021-01-08 09:56:58 -08:00
Sanaz Taheri Boshrooyeh 96f95ed0b6
Test for the Rln relay/membership management contract deployment (#312)
* enhancement/cleanup-warnings (#290)


Accounting: Move flag to config and add basic test for accounting state (#284)

* Accounting: Add config flag (default to false)

Also fix bug where query function is called twice when flag is set.

* Accounting: test state update after store req
Refactor: waku_types separation of concern (trial with waku_swap) (#282)

* waku_types refactor: Accounting types to waku_swap

* waku_types refactor: Separate waku_swap_types file

* Fix import path for waku_swap
adds the test file for waku rln relay


WIP: adds some initial code


commented out the rln-relay test code


refactor/db-message-store-split (#280)

* fixes

* fic

* fix

* fix

* fix

* fixed

* fix

* fixes

* fixes, using sqlite lib fully

* fix

* Update sqlite.nim
Waku v2 JSON-RPC REST API: Store protocol proof of concept (#263)

* Waku V2 history query POC

* Fix folder structure

* Improve test clarity

* Improve imports, returns and some naming

* Changed naming conventions. Refactor & improve.

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Refactor types store (#291)

* Refactor: Move waku_store into its own folder

* Refactor: Move waku store types to new home (WIP)

* Refactor: Fix errors and recursive imports

* Fix rebase errors

* Refactor: More rebase import fixes
Accounting credit receiving node (#292)

* Accounting WIP: Swap access through Store

* Fix order bug and comment scenario + typo

* WIP

* Accounting: Account for receiving store node

- Turn accountFor function into credit and debit
- Misc formatting

* Accounting: Fix bugs related to mount and test

* Accounting: Simplify query signature

We already have a ref to wakuSwap through wakuStore now.

* Resolve rebase issues
enhancement/remove-flood (#295)

* removes flood

* rm

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Updated submodules (#303)


add Jenkinsfile and some Makefile targets for CI (#302)

Signed-off-by: Jakub Sokołowski <jakub@status.im>

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Accounting: End to end with Cheque PoC (#304)


Added some basic debug and relay json-rpc calls (#305)


Uncomment and call empty rln test

adds the membership contract to the test file


adds a test utils for contract deployment


increases the gas limit for group management contract

* removes some files

* WIP: adds the interface of membership contract

* uncomments the interface of getNumber contract

* adds tests for interaction with the membership contract

* send is replaced with call

* replaces call with send

* enhancement/cleanup-warnings (#290)


Accounting: Move flag to config and add basic test for accounting state (#284)

* Accounting: Add config flag (default to false)

Also fix bug where query function is called twice when flag is set.

* Accounting: test state update after store req
Refactor: waku_types separation of concern (trial with waku_swap) (#282)

* waku_types refactor: Accounting types to waku_swap

* waku_types refactor: Separate waku_swap_types file

* Fix import path for waku_swap
adds the test file for waku rln relay


WIP: adds some initial code


commented out the rln-relay test code


refactor/db-message-store-split (#280)

* fixes

* fic

* fix

* fix

* fix

* fixed

* fix

* fixes

* fixes, using sqlite lib fully

* fix

* Update sqlite.nim
Waku v2 JSON-RPC REST API: Store protocol proof of concept (#263)

* Waku V2 history query POC

* Fix folder structure

* Improve test clarity

* Improve imports, returns and some naming

* Changed naming conventions. Refactor & improve.

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Refactor types store (#291)

* Refactor: Move waku_store into its own folder

* Refactor: Move waku store types to new home (WIP)

* Refactor: Fix errors and recursive imports

* Fix rebase errors

* Refactor: More rebase import fixes
Accounting credit receiving node (#292)

* Accounting WIP: Swap access through Store

* Fix order bug and comment scenario + typo

* WIP

* Accounting: Account for receiving store node

- Turn accountFor function into credit and debit
- Misc formatting

* Accounting: Fix bugs related to mount and test

* Accounting: Simplify query signature

We already have a ref to wakuSwap through wakuStore now.

* Resolve rebase issues
enhancement/remove-flood (#295)

* removes flood

* rm

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Updated submodules (#303)


add Jenkinsfile and some Makefile targets for CI (#302)

Signed-off-by: Jakub Sokołowski <jakub@status.im>

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Accounting: End to end with Cheque PoC (#304)


Added some basic debug and relay json-rpc calls (#305)


Uncomment and call empty rln test

adds the membership contract to the test file


adds a test utils for contract deployment


increases the gas limit for group management contract

* removes some files

* WIP: adds the interface of membership contract

* uncomments the interface of getNumber contract

* adds tests for interaction with the membership contract

* send is replaced with call

* replaces call with send

* adds a todo

* removes empty lines

* adds some comments

* adds a todo

* adds the solidity code of the NumberContract

* fixes a bug: replaces call with send

* removes store test comments

* removes excess spaces
2021-01-07 12:34:24 -08:00
Oskar Thorén 37a3a4fadd
Refactor: Put waku_type content where it belongs (#333)
- Separate out types and methods into appropriate module
- Add util folder for small utils used in multiple independent places
- Update paths
2021-01-06 17:35:05 +08:00
Oskar Thorén f860c0dda4
Refactor: waku_types -> waku_filter (#331)
Also create a folder with README for filter protocol.
2021-01-05 12:52:10 +08:00
Dean Eigenmann 3242b0020b
fix/message-notification-descriptive (#327)
* moved

* added docs

* Update message_notifier.nim

* Update waku_types.nim
2021-01-04 12:57:26 +01:00
Hanno Cornelius 8203f30099
Integrate API and add tutorial (#330) 2020-12-24 10:02:30 +02:00