Commit Graph

741 Commits

Author SHA1 Message Date
Aaryamann Challani bdb120d842
feat(chat2): metrics server + metrics logging (#1279)
* feat(rln-relay): periodically log metrics

* fix(chat2): enable metrics by default in chat2 config

* test(chat2): metrics, gc compilation error

* chore(metrics): make metrics a util, and import into wakunode2 apps dir

* fix(metrics): raise error

* fix(metrics): gc error

* fix(wakunode2): remove setup_metrics

* chore(metrics): waku utils metrics

* fix(metrics): create waku_metrics in node dir

* fix(metrics): log scope

* fix(chat2): disable metrics server by default

* fix(utils): collectorAsF64 proc def

* fix(metrics): store metrics path
2022-10-21 14:03:36 +05:30
Lorenzo Delgado e5c3aa560e
feat(store): add waku store client module 2022-10-20 18:09:40 +02:00
Hanno Cornelius a12a359b79
fix: store cursors should be exclusive and match a DB item (#1263)
* fix(store): use exclusive cursor

* fix(store): read correct column length

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2022-10-20 13:24:40 +03:00
Lorenzo Delgado 757804c9ee
chore(wakunode2): move wakunode2 to apps directory 2022-10-18 12:35:26 -05:00
Lorenzo Delgado 5046a4b3da
refactor(node): split wakunode into waku_node and wakunode2 2022-10-18 09:05:53 -05:00
Lorenzo Delgado b139692b05
refactor(wakunode2): move config random private key generation to a procedure (#1273) 2022-10-18 07:06:54 -05:00
Aaryamann Challani 9a6a7334ef
feat(rln-relay): periodically log metrics (#1262)
* feat(rln-relay): periodically log metrics

* chore(rln-relay): log rln metrics immediately

* fix(metrics): calculation etc

* chore(metrics): parseAndAccumulate into collector utils

* fix(metrics): use standard logging interval
2022-10-18 17:07:44 +05:30
Lorenzo Delgado 5fbc537cbf
chore: move apps and tools to their own directory 2022-10-12 14:41:25 -05:00
G a2a5a6ef01
refactor(rln): display byte arrays in hex as uints and fix Kilic's proofVerify proc (#1259)
* refactor(rln): convert 32 bytes arrays to hex as uints

* fix(rln): fix Kilic's proofVerify proc to take stub validRoots

* refactor(rln): use input's length when convert to/from hex

* chore(rln): print more commitments inHex in debug logs
2022-10-12 10:38:48 +02:00
Aaryamann Challani 04154ab90f
chore(rln-relay): make account address optional (#1258)
* chore(rln-relay): make account address optional

* fix(rln-relay): onchain test

* chore(chat2): update docs, and handle error case in chat2

* fix(rln-relay): handle registration better
2022-10-12 07:48:11 +05:30
G f282621a1c
fix(rln): set maxEpochGap as maximum epoch gap (#1257) 2022-10-10 21:50:16 +02:00
G 586d549113
fix(rln): fix encoding to LE for idcommitments (#1256) 2022-10-10 19:35:43 +02:00
Aaryamann Challani 3a80fae634
test(rln-relay): setup heartbeat ws subscription and log when connection drops (#1253)
* test(rln-relay): setup heartbeat ws subscription and log when connection drops

* fix(rln-relay): quantity

* fix(rln-relay): allow fromBlock to be passed to sub proc

* chore(rln-relay): cleaner logs

* fix(rln-relay): typo
2022-10-10 17:55:12 +05:30
Aaryamann Challani f8c4d9aebc
chore(rln-relay): additional logging (#1243)
* chore(rln-relay): additional logging

* fix: whitespace
2022-10-08 12:14:52 +05:30
G 3e1e7cc74f
feat(RLN): Add verify against multiple roots (#1250)
* update zerokit submodule

* fix(rln): allow Buffer creation on empty arrays

* feat(rln): use verify_with_root

* fix(rln): update test to work with verify_with_root
2022-10-07 21:24:54 +02:00
Aaryamann Challani 8a8c745e30
chore(rln-relay): disable merkle root validation (#1246)
* chore(rln-relay): disable merkle root validation

* brings back validation of the root but without invalidating the message

* adds the tracking issue

Co-authored-by: sanaz <staheri14@ku.edu.tr>
2022-10-06 13:17:01 -07:00
Lorenzo Delgado 1cb3f4b7a4
fix(store): fix store validity timestamp variance constant (#1244) 2022-10-06 16:07:12 +01:00
Hanno Cornelius 8a184e208c
chore: publish nwaku version to metric (#1234) 2022-10-06 13:06:54 +01:00
Lorenzo Delgado 2c975597e1
fix(store): fix waku store resume tests 2022-10-05 17:58:24 +02:00
Lorenzo Delgado 14abdef796
chore(store): improve database migrations logging 2022-10-05 11:45:44 +02:00
G f9c05aae52
chore(rln): Update zerokit submodule and RLN APIs (#1209)
* chore(rln): update zerokit submodule and APIs

* feat(rln): add new_circuit_from_data zerokit API

* chore(rln): update zerokit module
2022-10-05 00:06:33 +02:00
Sanaz Taheri Boshrooyeh fe3a3136a3
feat(rln-relay): adds rln identifier to the RateLimitProof (#1219)
* adds rln identifier to the RateLimitProof

* enables rln-relay compilation for wakunode2

* unifies the two RateLimitProof types

* a minor update

Co-authored-by: G <28568419+s1fr0@users.noreply.github.com>
2022-10-04 13:20:44 -07:00
Hanno Cornelius ad628b5b52
fix: store config without db-path leads SIGSEGV (#1218) 2022-10-04 14:34:05 +01:00
Aaryamann Challani 787464487b
chore(rln-relay): namespaced flags (#1204)
* chore(rln-relay): namespaced flags

* chore(rln-relay): shorten flag size

* fix(rln-relay): consistent config naming

* chore(rln-relay): consistency with address configs

* fix(rln-relay): config
2022-10-03 13:25:56 -07:00
Lorenzo Delgado fa6a44eb3c
fix(store): Move page info logic to waku store protocol 2022-10-03 17:36:17 +02:00
Lorenzo Delgado adf1dab34f
fix(store): update sqlite db table primary key and decouple queries module from pagination types 2022-10-03 15:23:39 +02:00
Richard Ramos 8af3438814
fix: add lastEpoch when using zerokit, and missing return (#1210) 2022-10-03 11:48:01 +02:00
Lorenzo Delgado c61e88b352
refactor(store): simplify store queue query implementation 2022-10-03 08:40:01 +02:00
Aaryamann Challani d614e4504b
feat(rln-relay): metrics (#1185)
* feat(rln-relay): metrics

* fix(rln-relay): invalid token

* fix(rln-relay): return outside time

* style(rln-relay): multiline proc def

* fix(rln-relay): oserror

* fix(rln-relay): Exception should not be raised

* Revert "fix(rln-relay): oserror"

This reverts commit edcf39c0088ca526ca50256d856eee20d87a3c37.

* Revert "fix(rln-relay): Exception should not be raised"

This reverts commit c8d09b055314a83bb4a94900a47e899773cb47df.

* fix(rln-relay): revert exception removal

* feat(rln-relay): granular tracking

* fix(rln-relay): observe only on histogram, convert mount to hist

* chore(rln-relay): add additional metrics

* fix(rln-relay): missing import

* fix(rln-relay): template should expand only when times is defined

* fix(rln-relay): follow prometheus naming style

* fix(rln-relay): explicitly assert that the root window size should be less than 10

* fix(rln-relay): outstanding issues with infinite labels, minor refactor

* fix(rln-relay): typo

* fix(rln-relay): screaming case to pascal case

* fix(rln-relay): typos

* fix(rln-relay): imports

* fix(rln-relay): remove dependence on times, make durations as gauges

* fix(rln-relay): import order
2022-09-30 18:13:42 +05:30
Lorenzo Delgado 612023fab0
chore(store): collocate pagination and message store types 2022-09-28 13:36:05 +02:00
Lorenzo Delgado 5ee4a00765
refactor(store): decouple message store queue from pagination index type (#1187) 2022-09-27 21:10:11 +02:00
Lorenzo Delgado d7d51dc4c1
fix(store): rename store timestamp sqlite column 2022-09-27 11:47:25 +02:00
Aaryamann Challani 37cd8b593b
feat(rln-relay): multiple acceptable roots (#1177)
* feat(rln-relay): multiple acceptable roots

* fix(rln-relay): make sure onchain handler uses correct proc

* fix(rln-relay): typo

* style(rln-relay): convert const to pascalcase

* chore(rln-relay): address pr review

* fix(rln-relay): add require to test

* fix(rln-relay): add proc desc

* fix(rln-relay): ensure that group id commitments were added correctly

* fix(rln-relay): potential mem leak

* style(rln-relay): comments

* fix(rln-relay): magic number ambiguity

* chore(rln-relay): comment

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
2022-09-27 10:10:04 +05:30
Lorenzo Delgado 1b4a9e5dfd
fix(store): move insert criteria logic to waku store protocol module 2022-09-26 11:50:15 +02:00
Sanaz Taheri Boshrooyeh 489ca1b01e
chore(rln-relay): specifies the address type for the eth-client-address config parameter (#1176) 2022-09-23 21:57:28 -07:00
Hanno Cornelius 6b86c4a22a
fix: fail to start switch with ext ip (#1181) 2022-09-23 15:27:15 +01:00
Hanno Cornelius ca1718c504
fix: signing key for wakuv2 DNS lists has changed (#1174) 2022-09-22 14:51:17 +02:00
Lorenzo Delgado aebda46a53
chore(store): move constants to retention policies modules 2022-09-22 11:17:38 +02:00
Lorenzo Delgado 5e3a75c56e
refactor(wakunode): remove deprecated non-async methods 2022-09-21 18:27:40 +02:00
Lorenzo Delgado 11832411a0
feat(store): execute retention policies periodically (#1155) 2022-09-21 11:32:59 +02:00
Hanno Cornelius efe82eade2
fix: no need to start protocols twice (#1162)
* fix: no need to start protocols twice

* fix: relay should still be explicitly started

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2022-09-20 21:02:20 +01:00
Aaryamann Challani e8d77306c8
feat(rln-relay): validate the merkle root in the RateLimitProof (#1158)
* feat(rln-relay): validate the merkle root in the RateLimitProof

* style|chore: replace snake case usage, convert proofVerify to Result[bool,string]

* chore(rln-relay): remove unhandled usage of result returned

* fix(rln-relay): typos

* chore(rln-relay): validate root before proof verification

* fix(rln-relay): cmp

* fix(rln-relay): generic rln relay result

* style(rln-relay): readability

* fix(rln-relay): use patterns for testing, requires instead of checks

* fix(rln-relay): move root validation to message validation

* fix(rln-relay): typo

* fix(rln-relay): typo

* chore(rln-relay): address pr review
2022-09-20 18:38:05 +05:30
kaiserd 371016b04f
feat: waku peer exchange (RFC34) (#1152) 2022-09-20 13:03:34 +02:00
Lorenzo Delgado 4a9f6f3c5e
refactor(store): protocol code refactoring and api reorganization 2022-09-20 11:39:52 +02:00
Hanno Cornelius 9be3d296be
fix: nodes in AddressBook are not necessarily connected (#1161) 2022-09-20 10:00:40 +01:00
Lorenzo Delgado f73477be6e
refactor(store): invert retention policies dependency on message store interface 2022-09-16 12:55:22 +02:00
Lorenzo Delgado 706c355b18
refactor(store): extend message store interface with store manipulation methods 2022-09-15 21:12:13 +02:00
Lorenzo Delgado 70fdd85066
refactor(store): don't cache the message count in sqlite message store 2022-09-15 18:13:30 +02:00
Lorenzo Delgado 87479a603b feat(store): run sqlite database vacuum at node start 2022-09-14 21:40:17 +02:00
Lorenzo Delgado 89069aa34f
fix(store): follow good practices in store insert and query metrics (#1150) 2022-09-14 13:40:11 +01:00
Lorenzo Delgado f7b4e8f57d refactor(wakunode): remove waku rln relay export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado ddbc57bc88 refactor(wakunode): remove waku message export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado c2c2d30cdf refactor(wakunode): remove libp2p builders export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado 2912a45433 refactor(wakunode): remove waku relay export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado fdd57c7bce refactor(wakunode): remove waku swap export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado a4860d92a0 feat(store): add query and insertion time metrics 2022-09-13 19:36:11 +02:00
Lorenzo Delgado 728e298dd7 fix(store): support stored messages metric for persistent store 2022-09-13 19:36:11 +02:00
Lorenzo Delgado 9561f12172 feat(store): add support for getting stored messages count 2022-09-13 19:36:11 +02:00
Lorenzo Delgado 83cffd50ed refactor(store): reorganise waku store protocol is sqqlite-only logic 2022-09-13 19:36:11 +02:00
Lorenzo Delgado d49cd1348d
refactor(store): remove deprecated waku message store type 2022-09-13 13:36:04 +02:00
Aaryamann Challani fdcc9824a2
feat(store): Allow messages to be marked as "don't store" (#1119)
* feat(store): init allow messages to be marked as ephemeral

* feat(store): replace ephemeral with ttl model

* Revert "feat(store): replace ephemeral with ttl model"

This reverts commit 4398c61c919dda8f014b4fcc0679c573c74854e4.

* fix(store): flags and test

* chore(store): remove storeTTL

* chore(store): remove unused utils

* fix: conflict

* fix(store): make test more brief
2022-09-13 16:07:06 +05:30
Aaryamann Challani 4745c7872c
fix(ci): macos builds (#1145)
* chore: revert json rpc

* chore(ci): cleanup

* chore(ci): remove windows from build matrix

* Revert "chore(ci): remove windows from build matrix"

This reverts commit 1f1d33b5f4.
2022-09-13 15:13:56 +05:30
Lorenzo Delgado 051f5db9af
refactor(store): rename to sqlite store and introduce retention policies interface 2022-09-12 16:49:01 +02:00
Lorenzo Delgado e7ebd190a3
refactor(utils): split message digest calculation form index calculation 2022-09-12 15:51:06 +02:00
Lorenzo Delgado ff619de037
chore(utils): remove unused types and methods 2022-09-12 15:29:52 +02:00
G 01cd201f58
PoC implementation of Waku Pairing and Secure Transfer (#1117)
* feat(noise): add PoC implementation for WakuPairing and Secure Transfer
2022-09-12 02:23:14 +02:00
Lorenzo Delgado 30f0e19d79
fix(store): add a table index to improve message insertion time 2022-09-09 15:04:48 +02:00
Hanno Cornelius f4e9d39524
chore: always include node ENR in response to info requests (#1128) 2022-09-08 14:25:02 +01:00
Lorenzo Delgado d857c4a421
chore(wakunode): collocate wakuswitch file to the depending modules 2022-09-08 11:02:50 +02:00
Hanno Cornelius 7f23bdf29f
chore: update nwaku submodules (#1123)
* chore: update submodules

* fix: SIGTERM ambiguity

* fix: ambiguous RNGs

* fix: ContentType is no longer a string

* fix: more fixes related to ambiguous RNGs

* fix: start all protocols

* chore: also update nim-eth

* chore: important new fixes in nim-libp2p

* fix: more changes related to RNG. Some reversion to reflect nim-eth update

* fix: breaking changes in nim-eth submodule

* fix: start protocols in tests

* fix: chat2bridge protocols async mounting

* fix: v1 test compilation

* fix: rln test compilation

* fix: remove confusing keys qualifier for the same HmacDrbgContext
2022-09-07 16:31:27 +01:00
Lorenzo Delgado 3f7aa8c6e4
fix(waku-store): added an index to improve messages query time 2022-09-05 15:18:47 +02:00
Lorenzo Delgado 21e9f25276
refactor(wakunode2): deprecate non-async protocol api methods 2022-09-02 10:17:14 +02:00
Lorenzo Delgado 4f3a3da084
chore(waku-store): improve logging and query traceability 2022-09-02 10:14:58 +02:00
Lorenzo Delgado 982fb08c77
refactor(wakunode2): add convenience methods for adding protocol peers 2022-09-01 17:02:58 +02:00
Aaryamann Challani ea62e2b7ce
chore(noise): minor refactor of default checks of keypairs (#1104)
* chore(noise): readable default checks for keypairs

* chore(noise): fix naming of arg in exists proc

* chore(noise): fix naming to make it clearer

* chore(noise): inline and remove result usage

* chore(noise): move the isDefault function up, replace defaultyVar with value

* chore(noise): inline style

* chore(noise): static the default check, remove inline pragma
2022-09-01 14:31:48 +05:30
Bulat-Ziganshin 80aeebc61c
Fixed handling of bindParam(uint32) in sqlite.nim (#1014) 2022-08-31 14:50:05 +02:00
Aaryamann Challani ad7f9ed5bf
chore(lightpush): fix minor typo (#1100) 2022-08-31 17:00:10 +05:30
Lorenzo Delgado 6d0168bff2
feat(rest): add waku filter openapi rest api specification file 2022-08-31 13:24:35 +02:00
Aaryamann Challani ed53bcdec6
chore(logging): Periodic logging for additional metrics (#1094)
* chore: enable metrics logging by default

* feat(logging): add additional logging metrics, change period to 30 seconds

* chore(logging): s/var/let

* chore(logging): import filter directly

* chore(logging): add store/lightpush peer count, fix style
2022-08-31 15:23:28 +05:30
Sanaz Taheri Boshrooyeh 0deedf0d7a
fix|feat(rln-relay|rln-chat2): Handling on-chain transactions errors and informing chat2 users (#1093)
If any error occurs during the on-chain registration, it will be handled; the chat2 users will get informed about the cause and the installation of rln-relay gets discarded and the chat program continues with no crashing.
2022-08-30 10:59:02 -07:00
Lorenzo Delgado ecf4ba1167
feat(message-cache): make the topic cache generic 2022-08-30 15:57:45 +02:00
Lorenzo Delgado 791ce6d222 fix(rest): encode waku message payload in base64 2022-08-30 14:53:04 +02:00
Lorenzo Delgado 842cfb3476 refactor(waku-filter): remove wakunode2 waku_filter exports 2022-08-29 10:14:35 +02:00
Lorenzo Delgado c964aea885 refactor(waku-filter): waku filter protocol code reorganisation 2022-08-26 22:17:56 +02:00
Hanno Cornelius caa47896c1
fix: peer manager dial logic when dialing an existing peer (#1090) 2022-08-26 11:20:55 +02:00
Sanaz Taheri Boshrooyeh bc29619679
feat(Rln relay): Improves rln-chat2 interface, displays links to the registration tx, warns about spam messages (#1082)
* warns  about publishing spam messages, display tx hash on Goerli

* passes registration handler
2022-08-24 15:47:06 -07:00
Hanno Cornelius 11158c3ea2
fix(setup): do not attempt connecting to empy dynamic bootstrap node list (#1088) 2022-08-24 17:44:43 +02:00
Sanaz Taheri Boshrooyeh 8a6b1cfba6
feat(Rln-Relay): Consolidates the input paths of rln credential, chat2 reads rln credential from file only (#1080)
* Consolidates the input paths of rln credential, accepts credential file only

* fixes a typo
2022-08-18 10:35:02 -07:00
Lorenzo Delgado 73df74f8c6 refactor(waku-store): remove wakunode2 waku_store exports 2022-08-18 17:32:47 +02:00
Sanaz Taheri Boshrooyeh f373f8bc56
chore(RLN-Relay): onchain rln-relay chat2 testnet editorial feadbacks
Signed-off-by: sanaz <staheri14@ku.edu.tr>

adds a newline


fixe a minor spacing issue
2022-08-17 16:55:49 -07:00
Lorenzo Delgado 7a30e485ec refactor(waku-store): deprecate waku store callback methods 2022-08-17 17:22:56 +02:00
G 2c4730c850
fix(rln): add zerokit rln to chat2 + fix static group members (#1077) 2022-08-17 12:11:46 +02:00
Lorenzo Delgado 2cb7123df8 fix(waku-store): integrate the predicate into the WHERE clause instead of calling it on each result line (#1026) 2022-08-12 12:48:45 +02:00
Lorenzo Delgado 4f93510fc9 refactor(waku-store): inherit from message_store in store_queue 2022-08-11 10:15:20 +02:00
Lorenzo Delgado 613812023e refactor(waku-lightpush): remove wakunode2 waku_lightpush exports 2022-08-10 19:44:40 +02:00
Lorenzo Delgado d201a37330 refactor(waku-lightpush): waku lightpush protocol code reorganization 2022-08-10 12:30:34 +02:00
G 2aeaba48e4
feat(noise): add padding to transport messages (#1056)
* feat(noise): add padding to transport messages (handshakes+write+read)

* fix(noise): address reviewers' comments

* fix(noise): update padding block size const name
2022-08-07 16:20:34 +02:00
G 8a14b5b243
Zerokit RLN integration in nwaku: alternative approach with inline compilation flag (#1060)
* refactor(rln): add inline compilation flags for RLN zerokit

* update submodule

update submodule

* fix(rln): fix wrong merge conflict resolution

* fix(rln): align hardcoded values in light of vacp2p/zerokit/issues/28
2022-08-05 22:58:19 +02:00
Keshav Gupta 115982f5c3
Persisting rln credentials (#1037)
* Persisting Credentials implemented by writing json of keypair and rlnindex to files

* Removing testing files and ignores

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Daniel Kaiser <git@kais3r.de>

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Daniel Kaiser <git@kais3r.de>

* Comments

* Comments

* Security warning in comments

* Redundant echos. Omitting unused variables.

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Limit line lengths using line breaks and indents

* Membership file paths declared as const

* Const fix

* Rln Credentials Merged.
Reading credentials from file abstracted away.

* Spaces

* Spaces

* Dangling constants removed. Comments position.

* Import sequence.

* Path as argument to readPersistentKeys.
Spaces in comments

* readPersistentKeys test

* Debug and info

* Index check in test

* Update tests/v2/test_waku_rln_relay.nim

Co-authored-by: G. <28568419+s1fr0@users.noreply.github.com>

* Abstracted writeRlnCredentials.
Fix var name in test.

* Usage of writeRlnCredentials in test

* Dnsclient?

* Test reverted to direct call to writeFile.
Abstrated writePersistentRlnCredentials usage causing error,
with readPersistentRlnCredentials

* Indentation

* Revert "Dnsclient?"

This reverts commit 3f4dba1a0b.

* Reverting abstraction of writing..
..persiting credential

Co-authored-by: Daniel Kaiser <git@kais3r.de>
Co-authored-by: Keshav Gupta <keshav.pg@hotmail.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: G. <28568419+s1fr0@users.noreply.github.com>
2022-08-05 12:48:01 +02:00
G 32f91cb26b
Noise: split Noise submodule in smaller submodules (#979)
* refactor(noise): split Noise submodule in smaller submodules
2022-08-04 10:47:00 +02:00
Lorenzo Delgado 888f7cb312 refactor(waku-store): major code reorganization, move StoreQueue to message_store folder 2022-08-01 13:02:40 +02:00
Sanaz Taheri Boshrooyeh 79459d29ce
Feat(Rln relay): enables relay-mode only (#1051)
* makes eth private key optional

* makes eth account private key field optional

* clarifies a comment
2022-07-28 11:45:29 -07:00
Lorenzo Delgado e7ba81ff88 refactor(wakunode2): Rename waku_setup.nim to wakunode2_setup.nim 2022-07-19 15:09:23 +02:00
Lorenzo Delgado 10c237dc5f refactor(wakunode2): Split wankunode2 SQL migrations setup to its own file 2022-07-19 15:09:23 +02:00
Lorenzo Delgado e16747137e refactor(wakunode2): Split wankunode2 JSON-RPC setup to its own file 2022-07-19 15:09:23 +02:00
Lorenzo Delgado 538ee12a25 refactor(wakunode2): Split wankunode2 metrics setup to its own file 2022-07-19 15:09:23 +02:00
Lorenzo Delgado 732f853f55 feat(rest): Add HTTP REST API (#727). Integrate REST API with the Waku V2 node 2022-07-19 11:02:43 +02:00
Sanaz Taheri Boshrooyeh daaca4a4a3
Feat(Rln relay): Enables and integrates onchain rln into chat2 (#1029)
* adds wakunode2_types

* removes unused imports

* adds dynamic rln relay config options

* deletes unused imports

* replaces static rln-relay with both modes

* introduces spamHandler parameter to mountRlnRelay

* some debugging info

* introduces the config option

* adds the eth priv key to the register proc

* refactors the code by adding eth private key to the relevant procs

* revises config descriptions

* minor wording fix

* make a minor edit

* further edits

* adds Eth private key to the config options

* calls spam handler when to rln credential is given

* defaults to none for the spam handler

* updates the membership fee to match the contract

* fixes an issue for event subscription

* adds eth private key to the transactions made in the tests

* adds eth private key for the test of register process

* adds comments

* deletes lingering echo

* displays registration info

* removes an echo

* removes Goerli testnet from the config descriptions

* removes an excess space
2022-07-14 12:23:52 +02:00
Sanaz Taheri Boshrooyeh a40686e26e
Feat (Rln relay): enabling signed tx (#1023) 2022-06-30 18:05:38 -07:00
Lorenzo Delgado 993781a137 chore: remove unused imports 2022-06-30 00:35:18 +02:00
Sanaz Taheri Boshrooyeh 5c7baf87e5
Chore(rln-relay): adds more documentation for rln nim bindings (#1016)
* annotates rln nim bindings for more clarity

* adds pointer to the rln construct

* revises further

* addresses comments

* revises proc description of verify

* removes AuthObj
2022-06-29 13:45:56 -07:00
Daniel Kaiser 3b60ea70c0
perf(store): remove vacuum (#1025) 2022-06-29 12:06:42 +02:00
Sanaz Taheri Boshrooyeh 3e20127933
Chore(rln-relay): decouples rln-relay and wakunode2 modules (#1019)
* adds wakunode2_types

* removes unused imports
2022-06-27 12:35:26 -07:00
Hanno Cornelius 9be64a55b6
fix: bridge loses connection to v1 (#1020)
* fix: bridge loses connection to v1

* Remove magic numbers in checks
2022-06-27 14:16:15 +02:00
Daniel Kaiser a2749c4a52
fix(store): limit dbPageSize (#1018) 2022-06-23 17:05:48 +02:00
Lorenzo Delgado cf1a7c3074 feat(rest): Add HTTP REST API (#727). Add Relay REST API 2022-06-23 10:12:38 +02:00
Lorenzo Delgado d2feb7c763 feat(rest): Add HTTP REST API (#727). Amend Debug REST API client 2022-06-22 20:30:44 +02:00
Richard Ramos 17ebb32331 fix: encode protobuffers using proto3 version 2022-06-21 17:21:03 -04:00
Sanaz Taheri Boshrooyeh 6c8ab0ab0f
Feat(rln-relay): enables two modes of onchain and offchain (#992)
* adds the contract handler file

* adds integration test for the group listening

* adds groupManagement proc

* deletes rln relay contract handler file

* brings back all the tests

* replaces toUINT256 with getIdCommitment proc

* replaces individual futures with an array of futures

* adds code documentation

* asyncSpawn instead of await

* adds untitest for toIDCommitment and toUInt256

* reorganizes the test and add rlnInstance

* mounts handleGroupUpdates on the rln peer

* asyncSpawn to await

* implements toIDCommitment

* updates the unittest

* improves the code documentation

* removes unused tests

* WIP

* uncomments the tests

* defines a new mountRlnRelayStatic proc

* splits mountRlnRelay into two procs for dynamic and static group management

* adds a config for off-chain and on-chain rln-relay

* runs dynamic or static mode of rln-relay based on the input config

* adds Eth private key and account configs

* reads Eth private key and account the configs

* comments put the second register proc

* add proper call to the rlnrelay dynamic mode

* adds todo

* adds new rln relay configs

* splits register into two procs

* makes eth account private key non-optional

* removes getIdCommitment and edits the register proc

* removes getIdCommitment calls

* uncomments the commented tests

* fixes a bug

* removes contract deployment for the offchain test

* fixes a bug, edits comments

* removes custom types  without proper parsing and serialization routines from the configs

* fixes a bug

* switches to stew byte utils

* removes log decoding

* WIP

* updates register proc

* edits test titles

* removes eth private key config

* changes the output of register proc to return the registered index

* integrates the registration process into mountRlnRelayDynamic

* integration test for the register proc

* brings back the onchain tests

* updates comments

* cleans up

* disambiguates the Address type namespace

* fixes type ambiguities

* adds default values for rln key and index

* updates config descriptions

* adds type conversion from hex to MembershipKeyPair

* adds more code documentation

* passed the group value instead of option  to the mount proc

* fix a bug

* a minor input type fix for rln chat2

* groups let declarations

* adds default values for addresses

* logs registered keys
2022-06-17 15:00:19 -07:00
Lorenzo Delgado 76e9a98841 feat(rest): Add HTTP REST API (#727). Debug API POC 2022-06-14 18:20:15 +02:00
Daniel Kaiser 5445303a23
feat(store): sqlite only store (#982)
* feat(sqlite): parameterized prep statement exec
  - from nim-eth
* feat(store): sql-only-store
  - add getPage(...) to sql-message-store
  - acts as separate store impl that can be activated with
    `--sqlite-store=true`
* test(store): new test-suite for sql message store
* test(store): waku_store test SQL-store compatible
* perf(store): avoid builing an additional B-tree in sql query
  - use `receiverTime` as the last ORDER BY column to fully utilize the
    primary key sorting
feat(store): retention time
  - retention time in seconds can be specified via `--sqlite-retention-time=`
2022-06-13 19:59:53 +02:00
Hanno Cornelius 6a09ed2c88
feat(enr): add discv5 UDP port to node's main discoverable ENR (#996) 2022-06-09 17:46:21 +02:00
Daniel Kaiser 271a2c750b
chore: adjust dyn bootstrap error message (#993) 2022-06-08 22:23:13 +02:00
Hanno Cornelius 0847eec6e0
chore: reduce unnecessary logging (#991) 2022-06-08 11:20:18 +02:00
Hanno Cornelius 1b29dcda7b
chore: disable swap by default (#989) 2022-06-08 09:16:45 +02:00
Hanno Cornelius 4e9215a920
chore: more regular keepalive pings (#990) 2022-06-07 22:44:58 +02:00
G c259f4cbaa
feat(noise): add Noise Handshake processing and after-handshake encryption (#934)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* feat(noise): add Noise Handshake State Machine and primitives

* feat(noise): add Handshake processing and after handshake encryption

* feat(noise): code updated according to #933

* fix(noise): export HandshakeResult

* feat(noise): add pre message validation utility

* chore(noise): improve documentation

* fix(noise): address reviewers comments
2022-06-03 21:12:41 +02:00
Hanno Cornelius d5a04a2719
chore: update submodules (#987) 2022-06-01 11:49:41 +02:00
Hanno Cornelius 37325c8f48
fix(sqlite): memory leak due to undisposed prepared statements (#981) 2022-05-31 12:47:38 +02:00
Sanaz Taheri Boshrooyeh 1ac029025a
Feat (Rln relay): adds utility procs to listen to the registration events emitted by the rln membership contract (#976)
* adds the contract handler file

* adds integration test for the group listening

* adds groupManagement proc

* deletes rln relay contract handler file

* brings back all the tests

* replaces toUINT256 with getIdCommitment proc

* replaces individual futures with an array of futures

* adds code documentation

* asyncSpawn instead of await

* adds untitest for toIDCommitment and toUInt256

* reorganizes the test and add rlnInstance

* mounts handleGroupUpdates on the rln peer

* asyncSpawn to await

* implements toIDCommitment

* updates the unittest

* improves the code documentation

* removes unused tests

* removes registration of the dynamic group management handler

* adds a comment

* adds a comment

* adds a TODO

* removes getIdCommitment
2022-05-30 12:14:07 -07:00
Lorenzo Delgado 32d230b474 feat(store): ability to decouple store from relay (#937) 2022-05-23 20:42:48 +02:00
G 710ce1315a
feat(noise): add Noise Handshake State Machine and primitives (#933)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* feat(noise): add Noise Handshake State Machine and primitives

* Fixed header

* refactor(noise): add comments, restyle code

* refactor(noise): address reviewer's comments

* feat(noise): add Noise Handshake State Machine and primitives

* refactor(noise): rebase

* refactor(noise): refactor and add documentation

* fix(noise): fixed trace parameter

* update submodules

* fix(noise): remove echo, add stdout.write in print handshake

* refactor(noise): add Noise state machine overview

* Revert "update submodules"

This reverts commit 9fc162dff8.

* update submodules

* fix Jenkins caching

* Revert "update submodules"

This reverts commit 1927d9dd26.

* revert nim-eth submodule update

* fix(noise): missing camelCase

* feat(noise): adding unit tests for Noise State machine primitives

* fix(noise): address reviewers' comments

* fix(noise): use expect for error handling in unit tests

* fix(noise): revert to implicit nonce declaration in CipherState initialization
2022-05-20 15:51:36 +02:00
Hanno Cornelius 984f3179a7
chore: update submodules (#974) 2022-05-20 12:57:19 +02:00
Daniel Kaiser 6a60458135
chore(migration): avoid unnecessarily calling migration (#967) 2022-05-18 21:44:53 +02:00
Hanno Cornelius 185485c336
docs: generate and configure nodekey (#957) 2022-05-18 15:00:30 +02:00
Daniel Kaiser 59b242fcad
fix(dns4name): dns4name + unset port resolved (#965) 2022-05-18 10:15:03 +02:00
Daniel Kaiser 41cdb8ba26
feat(info): add --version cmd flag (#970) 2022-05-17 22:11:07 +02:00
Daniel Kaiser 51cbc39d49
feat(config): add config file support (#953)
including

* bump vendor/nim-confutils
* add nim-toml-serialization submodule
* feat(config): add example config file
* add new feature to CHANGELOG
2022-05-17 17:48:08 +02:00
Hanno Cornelius bd70a1b7bb
Disable GossipSub PX by default (#958) 2022-05-13 18:18:09 +02:00
Sanaz Taheri Boshrooyeh 47a9fcfd77
chore|feat (waku-rln-relay): modules reorganization|Initial test for capturing events using nim-web3 (#941)
* first edition

* adds the full test scenario

* fixes typos

* fixes a bug in the supplied command

* further edits the description

* displays the chat prompt after spam detection

* updates changelog

* minor wording fix

* adds a new test file for onchain rln relay

* adds the Event proc

* adds one working example of event subscription

* defines a new unitt test for event subscription

* adds the new test file

* cleans up the code

* adds a working event subscription for faucet contract

* wip

* makes faucet test conditional

* updates contract byte codes

* adds a working test for event subscription and cleans up the tests

* fixes case

* adss toUInt256 unit function

* enables the tests

* fixes a bug

* undo commented tests

* cleans up the test

* logs the pk

* removes excess entry in the changelog

* fixes spacing

* comments

* removes unused test codes

* adds the conditional compilation for onchain tests

* uncomments offchain tests

* removes onchain tests

* reorganizes the code and moves the rln contract data into a separate module

* deletes txt files

* beautifies the code

* beautifies the code

* removes an excess line

* more formatting fixes

* minor fix

* updates the case of membership fee const

* renames compare to diff

* renames time to e

* edits the number of arguments of the send proc

* fixes a comment alignment

* fixes indentation

* fixed id style

* splits check from condition

* fixes a naming mismatch
2022-05-10 14:09:18 -07:00
Franck R b2eb992bd3
Fix typo (#960) 2022-05-10 12:49:12 +02:00
Daniel Kaiser 5e85eecd14
perf(store): add index on receiverTimestamp (#947) 2022-04-23 10:49:11 +02:00
G cb11c192d6
feat(noise): add support to Waku Payload V2 (#932)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* Fixed header

* refactor(noise): add comments, restyle code

* refactor(noise): address reviewer's comments

* Update nim-eth

* fix(noise): moved strutils in std
2022-04-12 13:50:51 +02:00
Hanno Cornelius 450e0494f1
fix(store): timestamp validity checks (#876) 2022-04-11 14:58:18 +02:00
G d2fccb5220
feat(noise): add support to encrypted/unencrypted public keys (#931)
* feat(noise): adding ChaChaPoly encryption

* test(noise): add test for ChaChaPoly Encryption/decryption

* style(noise): converted variables to camelCase

* feat(noise): add support to Noise public keys

* Fixed header

* refactor(noise): address reviewer's comments

* refactor(noise): small fixes/comments

* refactor(noise): add empty plaintext/ciphertext handling + string encryption test

* refactor(noise): address reviewer's comments
2022-04-06 14:37:02 +02:00
G 45b1e10647
feat(noise): add support to ChaChaPoly encryption (#930)
* feat(noise): adding ChaChaPoly encryption

* fix(noise): restored waku2 tests

* fix(noise): missing comma

* Fixed header

* fix(noise): made comment about empty ChaChaPolyKey clearer

* refactor(noise): address reviewer's comments

* test(noise): add test for ChaChaPoly Encryption/decryption

* refactor(noise): removed libp2p specific debug tools usage

* style(noise): converted variables to camelCase
2022-04-04 17:46:51 +02:00
Daniel Kaiser ecdde23add
feat(store): info message after delete (#929) 2022-03-30 22:55:39 +02:00
Daniel Kaiser a9c31b4b53
feat(store): vacuum after delete (#928) 2022-03-30 22:54:37 +02:00
Daniel Kaiser 2f13aff663
feat(jsonrpc): add call returning git version (#919) 2022-03-30 17:17:21 +02:00
Daniel Kaiser ec0a46b65b
feat(jsonrpc): add ENR to debug_v1_info call (#918) 2022-03-30 15:10:15 +02:00
Hanno Cornelius 52de0e434f
feat(relay): add support for gossipsub peer exchange (#911) 2022-03-29 10:09:48 +02:00
Daniel Kaiser d0cf3ed1f9
fix(store): capacity limit for persistent message store (#900)
* fix(store): limit capacity of persistent message store

fixes performance issue of loading a large message table by deleting
old messages in case of an overflow

* test(store): message capacity limit
2022-03-28 20:27:14 +02:00
Daniel Kaiser be8860c2fc
fix(discv5): info print ENR containing discv5 udp port (#913) 2022-03-28 10:29:19 +02:00
Sanaz Taheri Boshrooyeh c9e82873f1
Increase rln-relay logging (#908)
* updates default rln-relay content topic

* adds further debug and trace level logs

* converts a debug level log to the trace level
2022-03-24 10:47:40 -07:00
Hanno Cornelius 06f20af7d4
chore: update submodules (#905) 2022-03-23 15:20:07 +02:00
Hanno Cornelius 994ac49472
Fix: timestamp type could overflow before conversion (#892) 2022-03-18 12:04:11 +02:00
Daniel Kaiser e1146b2df8
discv5 dns bootstrap (#888) 2022-03-17 17:33:17 +01:00
Hanno Cornelius 266d6a7e80
fix: prevent empty store response due to blocked walker (#897) 2022-03-17 17:18:59 +02:00
Sanaz Taheri Boshrooyeh 2d6a2bec47
Updates the type of the `proof` field of wakumessage to `RateLimitProof` (#893)
* permanently changes the wakumessage proof field to RateLimitProof

* updates an old comment
2022-03-16 13:51:18 -07:00
Daniel Kaiser e6fdbdb42a
add RAM requirement to README (#889) 2022-03-16 17:13:23 +01:00
Hanno Cornelius eb02e4411b
Remove content topic dimension from message metrics (#891) 2022-03-15 14:39:32 +02:00
Hanno Cornelius ea72a65ef8
Increase store and store queue logging (#887) 2022-03-11 07:26:15 +02:00
Hanno Cornelius 9165720f63
Bridge improvements (#875) 2022-03-08 11:48:17 +01:00
Sanaz Taheri Boshrooyeh 2f73667a96
Fixes getNanosecondTime() time conversion (#878)
* fixes conversion from sec to nanosec

* uses snake case for power of 10 for readability
2022-03-07 16:56:39 +01:00
Hanno Cornelius 06fbb875c8
Update submodules (#873)
* Update submodules

* Fix MsgIDProvider
2022-03-02 13:24:48 +01:00
Daniel Kaiser 2972a50035
discv5 stage1 testnet (#863) 2022-03-01 17:12:23 +01:00
Daniel Kaiser dbe76d29ce
discv5 stage1 beta test (#862)
* update vendor/nim-eth submodule

point to the selectable `protocol-id` feature branch

* use d5waku as protocol-id

Use compiletime flag to change the discv5 protocol-id to "d5waku" allowing
nim-waku to span a separate discv5 network

* waku discv5 first beta stage

  - start discv5 during node start
  - added config options for discv5

* fix: pass to tcp port also when specifying ext ip

* bump nim-eth

Co-authored-by: ksr <kaiserd@users.noreply.github.com>
2022-03-01 15:11:56 +01:00
Hanno Cornelius 6be0fb233a
Add bridge docs and minor improvements (#867)
* Add bridge docs and minor improvements

* Clarify that bridging is bidirectional
2022-02-28 18:16:50 +01:00
Hanno Cornelius 20efdd7255
Store data model updates and fixes (#864) 2022-02-28 17:29:01 +01:00
G c27086e186
Moved PB Timestamp index to 10 (#861) 2022-02-23 08:19:52 +01:00
Hanno Cornelius 0b7b033550
Add docs on configuring domain and dns disc (#856) 2022-02-21 16:23:55 +01:00
Hanno Cornelius 2fd5b6c4ea
Domain name config for `nwaku` node (#852) 2022-02-18 12:10:38 +01:00
G dd2ed886a3
Update WakuStoreCodec string to beta4 (#855)
* Fix waku store version to beta4

* Updated changelog
2022-02-18 07:23:10 +01:00
G 21cac6d491
Refactoring timestamps (#842)
* Refactor timestamps type from float64 to int64 (milliseconds resolution)

* Revert epochs to float64

* Update 00002_addSenderTimeStamp.up.sql

* Update quicksim2.nim

* Add files via upload

* Delete 00003_convertTimestampsToInts.up.sql

* Add files via upload

* Rename 00003_convertTimestampsToInts.up.sql to 00003_addTimestampsToInts.up.sql

* Delete 00003_addTimestampsToInts.up.sql

* Rln-relay integration into chat2 (#835)

* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* removes duplicate epoch time calculation

* updates log level to trace

* updates default rln-relay content topic

* adds support for empty content topics

* updates changelog

* changelog updates

* removes a commented code block

* updates addRLNRelayValidator string doc

* Squashed commit of the following:

commit bc36c99ab2
Merge: dc2b2946 5a77d6e2
Author: G <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 01:10:06 2022 +0100

    Merge branch 'master' into int64-timestamps-ns

commit dc2b294667
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:24:45 2022 +0100

    Fix

commit f97b95a036
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:13:18 2022 +0100

    Missing import

commit 060c4f8d64
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:10:36 2022 +0100

    Fixed typo

commit 08ca99b6f6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:59:20 2022 +0100

    Time util file

commit 2b5c360746
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:33:20 2022 +0100

    Moved time utility functions to utils/time

commit fdaf121f08
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:10:25 2022 +0100

    Fix comment

commit c7e06ab4e7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:04:13 2022 +0100

    Restore previous migration script

commit 80282db1d7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:54:15 2022 +0100

    Typo

commit b9d67f89b0
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:49:29 2022 +0100

    Added utilities to get int64 nanosecond, microsecond, millisecond time resolution from float

commit 0130d496e6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:36:35 2022 +0100

    Switched to nanoseconds support.

* Update CHANGELOG.md

* Create 00003_convertTimestampsToInt64.up.sql

Migration script

* Moved migration script to right location

* Update waku_rln_relay_utils.nim

* Update waku_rln_relay_utils.nim

* Addressed reviewers' comments

* Update default fleet metrics dashboard (#844)

* Fix

* No need for float

* Aligning master to changes in PR

* Further fixes

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2022-02-17 16:00:15 +01:00
Hanno Cornelius bb3e59454e
Store performance improvements (#849) 2022-02-17 11:00:45 +01:00
Sanaz Taheri Boshrooyeh d851d48424
Updates rln-chat2 interface (#846)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* WIP

* removes duplicate epoch time calculation

* converts echo to log

* invokes handler

* bug fix

* prints calculated epoch

* changes the format of printed epoch

* updates log levels

* logs the input buffer supplied to the generate_proof

* replaces echos with logs

* changes log level to trace

* resets the log level of chat2 to INFO

* upgrades log level to debug

* exports toRLNSignal and adds a doc string

* updates log level

* enables all test2

* removes an echo statement

* modifies a comment

* further updates on the log level

* a minor update

* invokes the spam handler when provided

* checks for payload version

* deletes a redundant check

* deletes a rendant check

* updates default rln-relay cht2 content topic

* adds a todo and log

* changes the case of testnet content topic

* removes a flaky check
2022-02-16 14:52:21 -08:00
Hanno Cornelius e1254c20ae
Support for dns `multiaddrs` (#851)
* Support for dns addrs

* Tests and minor improvements
2022-02-16 17:12:09 +01:00
Franck R 5a890b5f90
Update websocket documentation to use domain certificates (#848) 2022-02-15 17:07:55 +11:00
Sanaz Taheri Boshrooyeh 5a77d6e2a6
Rln-relay integration into chat2 (#835)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* removes duplicate epoch time calculation

* updates log level to trace

* updates default rln-relay content topic

* adds support for empty content topics

* updates changelog

* changelog updates

* removes a commented code block

* updates addRLNRelayValidator string doc
2022-02-04 15:58:27 -08:00
Sanaz Taheri Boshrooyeh a5f8b90c23
Enables RLN compilation under CI env var (#786)
* compiles RLN if CI is defined

* minor ineffective change

* minor

* removes excess space

* resolves namespace conflicts

* moves pubsub import under compiler flag

* disables rln on-chain tests

* brings back the on-chain tests and fixes a name mismatch

* adds a debug log message

* minor formating

* kills ganache-cli procces on windows

* wip: adds a line to kill ganache-cli on windows

* disables rln-relay tests that depend on ganache-cli

* disables ganache-cli installation

* introduces onchain rln flag

* fixes a flaky test

* enables onchain test in non-windows CIs

* moves variables around

it seems detected_os is undefined, hence the compiler flag onchain_rln, which is conditioned to the os, does not get added

* limits onchain rln to macOS
2022-01-28 13:57:17 -08:00
Hanno Cornelius 4421b8de00
Improve node metrics (#831)
* Improve node metrics

* Add support for empty content topics as a separate label
2022-01-26 12:02:57 +01:00
Sanaz Taheri Boshrooyeh dbccecc562
Resolves naming conflict (#821)
* resolves namespace conflicts

* moves pubsub import under compiler flag
2022-01-21 10:55:36 -08:00
Daniel Kaiser b3a7722297
result aggregation in resume API (#816) 2022-01-18 23:05:41 +01:00
Hanno Cornelius 95d2e8bf71
Round receiver time (#815) 2022-01-17 19:37:06 +01:00
Hanno Cornelius 3e9ca25abc
Update submodules (#814) 2022-01-14 10:25:01 +01:00
Hanno Cornelius c422866a49
Improved `store` query performance (#812) 2022-01-11 14:32:09 +01:00
Hanno Cornelius f3ecdb80ea
Remove node peerInfo copy (#801) 2022-01-10 16:07:35 +01:00
Hanno Cornelius 62dbb3d0f3
Add MessageIdProvider for `WakuRelay` (#803)
* Add MessageIdProvider

* Fix unit test. Changelog.
2022-01-10 15:07:01 +01:00
Hanno Cornelius 685d43b2b3
Ensure store response never exceeds `MaxPageSize` (#811)
* Never exceed maxPageSize, even with no specified pagination

* fix test

* Changelog

* Forgot to stage everything...

* Account for timing difference in Windows
2022-01-07 15:01:23 +01:00
Hanno Cornelius 4e6960ad26
Fix incorrect store read buffer limit (#800)
* Fix store read buffer limit

* Changelog
2022-01-06 13:42:37 +01:00
Hanno Cornelius eb5dcadd45
Added protocol message counters (#804)
* Added protocol message counters

* Fix how stored messages are counted
2022-01-06 12:23:25 +01:00
rshiv b2273fff9a
Handles stale clients in FILTER protocol (#782)
* Handles stale clients

Signed-off-by: rshiv <reeshav96@gmail.com>

* adds test

Signed-off-by: rshiv <reeshav96@gmail.com>

* removes failed client from subscriber list

Signed-off-by: rshiv <reeshav96@gmail.com>

* adds filter timeout config

Signed-off-by: rshiv <reeshav96@gmail.com>

* reverts peer removal logic

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolve ci issues

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comment

Signed-off-by: rshiv <reeshav96@gmail.com>

* solves review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* ChangeLog update

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comment

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update CHANGELOG.md

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* fixes review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* handles CI issues

Signed-off-by: rshiv <reeshav96@gmail.com>

* tries to solve test CI

Signed-off-by: rshiv <reeshav96@gmail.com>

* solves CI issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves ci issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves CI issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2021-12-08 18:35:47 +00:00
rshiv ce0607b2eb
resolves readme (#796)
Signed-off-by: rshiv <reeshav96@gmail.com>
2021-12-08 16:04:23 +00:00
Hanno Cornelius 9eb532171e
Fix empty scripts condition. Logging. (#795) 2021-12-07 13:06:59 +01:00
Hanno Cornelius dd256e3bd2
Add support for RFC31 ENR (#789) 2021-12-06 20:51:37 +01:00
Richard Ramos 1217e25245 fix: display success message only if messages are loaded from the db 2021-12-03 09:35:40 -04:00
Richard Ramos 2b6922cdaa fix: return db error message 2021-12-03 09:35:40 -04:00
Hanno Cornelius 7ef51edfa2
Update announced addrs in switch (#784) 2021-11-30 10:12:09 +01:00
rshiv d49c95f6a1
publish addresses from announced address sequence. (#775)
* publish multiple addresses

Signed-off-by: rshiv <reeshav96@gmail.com>

* publish from announced address

Signed-off-by: rshiv <reeshav96@gmail.com>

* change in json

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes format

Signed-off-by: rshiv <reeshav96@gmail.com>

* makes listenstr a sequence

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes the wakuinfo Field

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes the test

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes the tests variable

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes test

Signed-off-by: rshiv <reeshav96@gmail.com>
2021-11-24 15:10:29 +00:00
Sanaz Taheri Boshrooyeh fda5128cc1
WIP: Waku-rln-relay: Message validation and double signaling detection (#769)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds trace log for the valid messages

* brings the log trace one line up
2021-11-23 14:48:40 -08:00
rshiv 53b21be2b5
Resolve jenkins windows error (#777)
* try different key

Signed-off-by: rshiv <reeshav96@gmail.com>

* key change

Signed-off-by: rshiv <reeshav96@gmail.com>

* cert change

Signed-off-by: rshiv <reeshav96@gmail.com>

* add newline to key

Signed-off-by: rshiv <reeshav96@gmail.com>

* exception change

Signed-off-by: rshiv <reeshav96@gmail.com>

* exception log

Signed-off-by: rshiv <reeshav96@gmail.com>

* add debug log

Signed-off-by: rshiv <reeshav96@gmail.com>
2021-11-23 11:40:43 +00:00
rshiv ac163932ad
Adds readme for websockets (#768)
* test_fleet

Signed-off-by: rshiv <reeshav96@gmail.com>

* add readme for websockets

Signed-off-by: rshiv <reeshav96@gmail.com>

* add readme

Signed-off-by: rshiv <reeshav96@gmail.com>

* add readme

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update waku/v2/README.md

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/README.md

Co-authored-by: Jakub <i+github@always.fail>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: Jakub <i+github@always.fail>
2021-11-16 10:51:05 +00:00
rshiv 6cad482ca6
Publish multiple address when websocket enabled. (#766)
* publish multiple nodes

Signed-off-by: rshiv <reeshav96@gmail.com>

* Print multiple addresses if wsenabled

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* review changes

Signed-off-by: rshiv <reeshav96@gmail.com>

* review changes

Signed-off-by: rshiv <reeshav96@gmail.com>

* code review changes

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* review fixes

Signed-off-by: rshiv <reeshav96@gmail.com>

* websocket address fix

Signed-off-by: rshiv <reeshav96@gmail.com>
2021-11-15 13:29:18 +00:00
Hanno Cornelius 067478d725
Proposal: ENR filtering on Waku-field in `discv5` (#763) 2021-11-12 15:10:54 +01:00
rshiv d2f7076b53
Secure websocket (#759)
* add config

Signed-off-by: rshiv <reeshav96@gmail.com>

* secure websocket integration

Signed-off-by: rshiv <reeshav96@gmail.com>

* secure websocket debug

Signed-off-by: rshiv <reeshav96@gmail.com>

* config change

Signed-off-by: rshiv <reeshav96@gmail.com>

* working secure websocket changes

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update submodules (#761)

* Release v0.6 (#760)

* Read from path

Signed-off-by: rshiv <reeshav96@gmail.com>

* Tests for secure websockets

Signed-off-by: rshiv <reeshav96@gmail.com>

* CI failure fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* path resolver CI

Signed-off-by: rshiv <reeshav96@gmail.com>

* self review fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update examples/v2/config_chat2.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update waku/v2/utils/peers.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/node/wakunode2.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update tests/v2/test_wakunode.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* handle review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
2021-11-10 12:05:36 +00:00
Hanno Cornelius 6d81f1ee55
Update submodules (#761) 2021-11-04 15:46:38 +01:00
Hanno Cornelius ee8ff014f4
Fix: store dimensioning issues (#758) 2021-11-03 11:59:51 +01:00
rshiv d1e06fa17a
Waku v2 Websocket support (#732)
* add config for ws support

* code clean up

* code clean up

* Integrate both ws and tcp transport

* change ws default port

* resolve review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* Unit test debug

Signed-off-by: rshiv <reeshav96@gmail.com>

* Websocket tests addition

Signed-off-by: rshiv <reeshav96@gmail.com>

* websocket failure unittest addition

Signed-off-by: rshiv <reeshav96@gmail.com>

* Commented testcase related to incorrect helper function

Signed-off-by: rshiv <reeshav96@gmail.com>

* Add wireAddr validation

Signed-off-by: rshiv <reeshav96@gmail.com>

* CLoses issue 756

Signed-off-by: rshiv <reeshav96@gmail.com>

* Websocket tests addition

Signed-off-by: rshiv <reeshav96@gmail.com>

* Minor Code clean up

* review comment fixes

Signed-off-by: rshiv <reeshav96@gmail.com>

* Review comment fix

* Update tests/v2/test_wakunode.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* Update tests/v2/test_wakunode.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Changelog update

Signed-off-by: rshiv <reeshav96@gmail.com>

* chat2 var change

Signed-off-by: rshiv <reeshav96@gmail.com>

* test flag changes

Signed-off-by: rshiv <reeshav96@gmail.com>

Co-authored-by: oskarth <ot@oskarthoren.com>
2021-11-02 10:29:11 +00:00
Hanno Cornelius f85434e072
Discv5 POC integration (#748) 2021-11-01 19:02:39 +01:00
Sanaz Taheri Boshrooyeh dbbc0f750b
RLN-Relay: Adds fix to the hash, proofGen and proofVerify procs (to support arbitrary messages) (#753)
* wip: updates the hash interface

* updates hash unittests

* exposes appendLength

* updates proof gen interface and the unittests

* enables test, fixes proofGen and verify utils and unittest, beautifies the code

* removes int size from mem index type

* enables all the rln tests

* adds documentation of two private procs

* documentation for the appendLength

* minor clean up

* adds clarificaltion on the endianness of input length

* Delete .DS_Store
2021-10-26 14:42:24 -07:00
Sanaz Taheri Boshrooyeh 6efba0dc56
Integrates proof generation and verification into wakunode2 (#735)
* WIP

* WIP: fixes a bug

* adds test for static group formation

* adds static group creation when rln-relay is enabled

* adds createStatic group

* wip: adds group formation to mount rlnrelay

* adds createMembershipList utility function

* adds doc strings and todos

* cleans up the code and add comments

* defaults createRLNInstance depth argument to 32

* renames Depth

* distinguishes between onchain and offchain modes

* updates index boundaries

* updates log levels

* updates docstring

* updates log level of displayed membership keys

* relocates a todo

* activates all the tests

* fixes some comments and todos

* extracts some utils procs for better debugging

* adds todo

* moves calculateMerkleRoot and toMembersipKeyPairs to the rln utils

* makes calls to the utils functions

* adds unit test for createMembershipList

* adds unittest for toMembershipKeyPairs and calcMerkleRoot

* cleans up the code and fixes tree root value

* reverts an unwanted change

* minor

* adds comments and cleans up the code

* updates config message

* adds more comments

* fixes a minor value mismatch

* edits the size of group

* minor rewording

* defines a const var for the group keys

* replaces the sequence literal with the StaticGroupKeys const

* adds a rudimentary unittest

* adds todos

* adds more comment

* replaces uint with MembeshipIndex type

* fixes rln relay mem index config message

* adds rln relay setup proc

* decouples relay and rln-relay

* uses MemIndexType instead of uint

* brings back the rlnRelayEnabled flag to mountRlnRelay

* deletes commented codes

* adds rln relay topic validator inside updates rln relay mounting procedure

* adds rln-relay-pubsub-topic cli option

* adds a static rln-relay topic

* deletes rlnrelayEnabled argument

* adds pubsub topic for rln-relay

* deletes static pubsub topic

* mounts relay before rlnrelay in the tests

* logs rln relay pubsub topic

* cleans up the code

* edits rlnrelay setup

* uninitializes the input parameter of rlnrelay setup

* adds comments

* removes unused comments

* compiles addRLNRelayValidtor when RLN compilation flag is set

* adds comment about topic validator

* minor

* mode modifications on the description of add validator

* adds pubsubtopic field to wakuRlnRelay type

* WIP: shaping the test

* Checks whether rln relay pubsub topic is within the supported topics of relay protocol

* minor

* WIP: unit test for actual proof

* fixes a bug

* removes a redundant proc

* refines the test for actual proof

* breaks lines to 80 chars

* defines NonSpamProof type

* adds a return

* defines Epoch type

* WIP: proof gen

* implements actual proof gen

* adds proto enc and init

* adds notes about proof structure

* adds NonSpamProof to wakumessage

* adds proof gen

* WIP: non working tests for protobuf

* fixes the protobuf encoding issue

* discards the output of copyFrom

* WIP: hash unittest and proofVrfy and ProofGen

* integrates proofVrfy

* uses toBuffer inside the hash proc

* adds comment

* fixes a bug

* removes proof field initialization

* cleans up the test

* generalizes input from byte seq to byte openArray

* adds toBuffer

* adds a bad test

* cleans up unused tests

* adds integration test

* adds comments

* cleans up

* adds description to the integration test

* adds test for unhappy path

* tides up the tests

* tides up hash unit test

* renames a few var

* uses a const for wku rln relay pubsub topic

* minor refinement

* deletes an obsolete comment

* comment revision

* adds comments

* cleans up and adds docstrings

* profGen returns proofRes instead of proof

* removes extra sleepAsync

* fixes two bugs

* returns reject when proof is not verified\

* addresses comments

* adds comments

* links to rln doc

* more comments

* fixes space format

* uncomments v2 tests

* dnsclient branch update

* undo branch update

* minor spacing fix

* makes proof field conditional
2021-10-19 17:37:29 -07:00
Hanno Cornelius 165e235158
Make maximum connections configurable (#731) 2021-10-12 14:48:48 +02:00
Pascal Precht 83f71ae905
Remove `udpPort` CLI option from waku v1 and v2 nodes (#728) 2021-10-12 13:43:01 +02:00
rshiv 5f76d90b7f Revert "wss multiaddress switch support"
This reverts commit ac76e2f189.
2021-10-06 18:03:00 +01:00
rshiv ac76e2f189 wss multiaddress switch support 2021-10-06 17:30:55 +01:00
Hanno Cornelius 50189a21a9
Update submodules: differentiate between local and remote `PeerInfo` (#730)
* Update submodules

* Remove PeerInfo where necessary
2021-10-06 14:29:08 +02:00
Sanaz Taheri Boshrooyeh 696f41e12a
Rln-Relay: Change tree depth to 20 (#726)
* adds instructions on how to generate parameters for a different tree size

* updates the depth of tree

* modifies the Merkle root

* updates parameters for the new tree size

* fixes failing tests

* updates the comment
2021-09-28 12:03:53 -07:00
Sanaz Taheri Boshrooyeh 645832e9fd
Restructuring Rln-Relay mounting process (#722)
* replaces uint with MembeshipIndex type

* fixes rln relay mem index config message

* adds rln relay setup proc

* decouples relay and rln-relay

* uses MemIndexType instead of uint

* brings back the rlnRelayEnabled flag to mountRlnRelay

* deletes commented codes

* adds rln relay topic validator inside updates rln relay mounting procedure

* adds rln-relay-pubsub-topic cli option

* adds a static rln-relay topic

* deletes rlnrelayEnabled argument

* adds pubsub topic for rln-relay

* deletes static pubsub topic

* mounts relay before rlnrelay in the tests

* logs rln relay pubsub topic

* cleans up the code

* edits rlnrelay setup

* uninitializes the input parameter of rlnrelay setup

* adds comments

* removes unused comments

* compiles addRLNRelayValidtor when RLN compilation flag is set

* adds comment about topic validator

* minor

* mode modifications on the description of add validator

* Checks whether rln relay pubsub topic is within the supported topics of relay protocol

* minor

* addresses comments

membeshipindex to membershipindex,
adds default value for rln topic
adds missing return
2021-09-24 10:32:45 -07:00
Sanaz Taheri Boshrooyeh 4895be61ec
Off-chain group construction and management (#718)
* WIP

* WIP: fixes a bug

* adds test for static group formation

* adds static group creation when rln-relay is enabled

* adds createStatic group

* wip: adds group formation to mount rlnrelay

* adds createMembershipList utility function

* adds doc strings and todos

* cleans up the code and add comments

* defaults createRLNInstance depth argument to 32

* renames Depth

* distinguishes between onchain and offchain modes

* updates index boundaries

* updates log levels

* updates docstring

* updates log level of displayed membership keys

* relocates a todo

* activates all the tests

* fixes some comments and todos

* extracts some utils procs for better debugging

* adds todo

* moves calculateMerkleRoot and toMembersipKeyPairs to the rln utils

* makes calls to the utils functions

* adds unit test for createMembershipList

* adds unittest for toMembershipKeyPairs and calcMerkleRoot

* cleans up the code and fixes tree root value

* reverts an unwanted change

* minor

* adds comments and cleans up the code

* updates config message

* adds more comments

* fixes a minor value mismatch

* edits the size of group

* minor rewording

* defines a const var for the group keys

* replaces the sequence literal with the StaticGroupKeys const

* converts var to let when applicable

* replaces hardcoded value with well-defined constants

* moves createMembershipList to the rln relay utils module

* renames HashSize to HashHexSize

* minor updates on the comments

* reorganizes the consts

* indicates that rlnRelayMemIndex is an experimental option

* fixes a type conversion bug

* revises the unittest of "mount waku rln-relay off-chain"

* clarifies the use of index

* updates a docstring

* removes redundant constants and capitalize all of them

* deletes the ETH_CLIENT const from the test file

* renames a few vars for the sake of clarity

* reorganizes unittest into blocks of execution, debug messages, and checks

* adds more comments

* more comments and clarifications

* cleans up the tests

* minor

* adds a minor fix

* replaces a var usage with let

* fixes a bug
2021-09-17 10:31:25 -07:00
Hanno Cornelius a335a40a21
Fix network simulations (#719)
* Fix/tweak network simulations

* fix terminology
2021-09-15 10:01:09 +02:00
Sanaz Taheri Boshrooyeh b7998de09d
RLN-Relay static group construction (#708)
* tests rln instance as pointer

* test insertion

* tests deletion

* tests proof generation and verification

* updates rln instance type and the rln api

* deletes old API

* removes temporary tests

* deletes unused codes

* Delete settings.json

* reverts the changes in tests v2

* removes an old comment

* adds member insertion and deletion

* adds getMerkleRoot and unit tests

* makes insertMember argument  a value type

* adds static group creation

* adds rln field to WakuRLNRelay type, and enables static group construction inside mountRlnRelay

* renames self to membershipKeyPair and removes key generation from mountRlnRelay

* renames sk,pk tp idKey and idCommitment

* updates mountRlnRelay arguments name

* logs created keys

* uncomments the key generation and adds explainer about it

* enables all the  tests

* adds comments to the rln relay types

* logs error message for the arguments that are not set
2021-08-26 16:14:51 -07:00
Hanno Cornelius 9a45aa7055
Fix: bridge encoding and missing timestamps (#697)
* Fix: bridge should not re-encode pre-encoded payload

* Test payload encoding, fix content topic

* Minor improvements

* Fix broken unit test
2021-08-26 14:24:00 +02:00
Hanno Cornelius 0db4107ae2
Integrate DNS resolution (#709)
* Update submodule

* Add DNS Client library

* Fork and update dnsclient submodule

* Integrate DNS resolution
2021-08-25 13:57:35 +02:00
Sanaz Taheri Boshrooyeh c222d83bcc
Enables passing RLN object across different function calls (#705)
* tests rln instance as pointer

* test insertion

* tests deletion

* tests proof generation and verification

* updates rln instance type and the rln api

* deletes old API

* removes temporary tests

* deletes unused codes

* Delete settings.json

* reverts the changes in tests v2

* removes an old comment
2021-08-24 12:25:29 -07:00
Hanno Cornelius 4aae5c55ac
Remove unnecessary ENR caching (#700) 2021-08-17 10:30:17 +02:00
Hanno Cornelius 36b9176569
Waku v2 Node Discovery via DNS: Integration (#690) 2021-08-12 10:51:38 +02:00
Hanno Cornelius 1ebcb266cb
Revert PRUNE backoff period to default (#694) 2021-08-12 09:57:19 +02:00
Sanaz Taheri Boshrooyeh 027cd7f95d
Simplify pagination (#689)
* initial simplification

* simplify forward pagination

* simplifies the backward pagination part

* removes old paginate proc

* revises comments, cleans up the code

* comments on initial paging query

* defines temporary variable for the paginate output
2021-08-04 15:24:36 -07:00
Hanno Cornelius 6b1e0c3079
Patch v0.5.1 - fix multiple protocol IDs in persistent storage (#687) 2021-07-27 08:48:56 +02:00
Hanno Cornelius 87b98a2460
Refactor wakunode2 setup (#675)
* Refactor wakunode2 setup
2021-07-22 11:46:54 +02:00
Hanno Cornelius 7be949ed67
Update submodules (#682) 2021-07-22 10:43:41 +02:00
Sanaz Taheri Boshrooyeh 71a532a414
Adds timestamp to waku relay message (#681)
* adds timestamp to WakuRelayMessage

* converts float to float64

* generates timestamp using epochTime()

* tides up imports and generates timestamps using epochTime()

* updates changelog

* adds timestamp to the post_waku_v2_relay_v1_message query parameters
2021-07-21 11:05:10 -07:00
Hanno Cornelius b87984af46
Fix topic conversion in bridge (#679) 2021-07-21 14:22:40 +02:00
Hanno Cornelius d44eab7cd7
Fix compilation with `rln` defined (#680)
* Fix compilation with rln defined
2021-07-21 11:37:10 +02:00
Sanaz Taheri Boshrooyeh 52b99f37af
Fixes the order of fields of History Response protobuf (#676)
* shifts history response field numbers by 1

* adds changelog

* minor rewording

* minor
2021-07-20 11:51:32 -07:00
oskarth 108599a1ed
Whisper tests (#678)
* Remove leftover comment

* Import Whisper tests

* Update copyright to 2018-2021

* Fix whisper import paths

* Whisper tests and Makefile target
2021-07-19 17:54:36 +08:00
oskarth 75ee287ca7
Adds `{.push raises: [Defect].}` to waku_swap (#677)
Doesn't try to deal with errors, just being explicit about them.
2021-07-19 16:49:27 +08:00
oskarth d3368324fe
Move Whisper types and protocol from nim-eth to this repo (#663)
* Move Whisper type and protocol from nim-eth

- Import whisper_protocol and whisper_types
- Change import locations as appropriate

* Explicit errors in wakunode1
2021-07-19 16:14:37 +08:00
Sanaz Taheri Boshrooyeh 32a8ffb867
Cleaning imports of the waku store module (#673)
* cleans up store modules

* removes endians2 as an unused imports

* reorganizes imports

* fixes an ambiguous import

* adds comments re imports order

* adds message_store
2021-07-16 15:28:35 -07:00
Kim De Mey 4b9b654655
Add top level push raises Defect to protocol v1 (#670) 2021-07-16 23:13:04 +02:00
Hanno Cornelius 13cf7380bb
General refactoring: `nim-waku` (#671)
* General Track 1 refactoring
2021-07-16 17:13:36 +02:00
Sanaz Taheri Boshrooyeh a41b9ac80b
Refactor waku protocols and utils (#667)
* refactors waku_filter

* refactors waku_lightpush

* refactors waku_swap

* refactor namespacing.nim

* refactor peers

* refactors requests

* adds top level {.push raises: [Defect].}

* log scope for rln relay

* cleans up comments

* removes comments

* comments out raise Defect

* defines temp var then pass to constructor

* Update waku/v2/protocol/waku_rln_relay/rln.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* Update waku/v2/protocol/waku_swap/waku_swap.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* explains the potential exception in waku_swap

* creates temp var before return

* adjusts spaces

* adds line breaks, temp vars and fixes format

* removes type declaration

* fixes and indentation issue

* adjusts spacing

* adjusts line <80ch

* formating improvement

Co-authored-by: oskarth <ot@oskarthoren.com>
2021-07-15 11:25:52 -07:00
Hanno Cornelius 92f2d5a7f5
Refactor wakunode2.nim (#664)
* Refactor wakunode2.nim

* Remove empty raises

* Some formatting improvements
2021-07-14 19:58:46 +02:00
Sanaz Taheri Boshrooyeh c107072d9a
Waku store: explicit return instead of result (#665)
* replaces result with returns

* fixes a missing proc call

* removes unused comments
2021-07-14 08:59:42 -07:00
Sanaz Taheri Boshrooyeh 07586153f9
Adding error message to the History Responses (#660)
* adds error field to history response

* unittest for protobuf enc/dec

* renames HistoryResponseError to Store Error and simplifies pagination logic

* reverts the naming of StoreError to HistoryResponseError

* fixes a unittest for invalid cursor

* renames OK to NONE
2021-07-13 12:01:21 -07:00
Hanno Cornelius 2869e71865
Simplify wakunode2 subscriptions model (#662) 2021-07-13 09:18:51 +02:00
Sanaz Taheri Boshrooyeh cf7b8faf27
History response pagination using sender-generated timestamp (#657)
* enables pagination based on sender timestamp

* uncomments a test

* bumps up version number

* updates migration script

* clean up

* unpdates changelog

* undo removal of receiver timestamp

* updates message_storage

* uses epochTime()

* minor

* removes a comment

* removes receiver timestamp deletion migration script

* fixes formatting issues

* fixes a bad field name

* fixes field issue

* adjusts spacing
2021-07-07 16:56:20 -07:00
Sanaz Taheri Boshrooyeh cd703c6c52
Integrating pagination into the resume proc (#653)
* adds queryFromWithPaging

* adds unit test for paged queryFrom

* adds a new resume with paging capability

* unittest for resumePaging

* fixes the bug

* implementes a second version of query with paging

* integrates the second version of resume

* clean up

* adds the error message

* fixes a format issue

* renames variables

* defines DefaultPageSize

* adds a todo

* updates a docstring

* gets the pagesize as proc input

* updates unittest description

* removes unused gcsafe pragma

* better var naming

* updates changelog

* updates changelog

* updates the TODO

* more debug logs
2021-07-02 11:37:58 -07:00
Hanno Cornelius 734fe0795f
Add support for multiple protocol IDs (#652) 2021-07-02 10:49:41 +02:00
Hanno Cornelius 52b741ac11
Support for stable version of relay protocol (#651) 2021-06-29 16:29:04 +02:00