Commit Graph

594 Commits

Author SHA1 Message Date
Alvaro Revuelta 700dbbb7fd
refactor(networking): peermanager refactor and cleanups (#1539)
* refactor(networking): use addServicePeer where needed + add metrics
2023-02-27 10:24:31 -07:00
Lorenzo Delgado 8258415804
fix(jsonrpc): use padded-base64 for payload encoding 2023-02-23 09:31:06 +01:00
Aaryamann Challani b6a011733d
chore(rln-relay): bump zerokit and update ffi (#1571)
fix(chat2): compilation
2023-02-22 19:47:12 +05:30
Hanno Cornelius 59203c7453
fix: allow higher resolution timestamps (#1570)
* fix: allow higher resolution timestamps

* fix: higher precision requires coarser test
2023-02-21 14:52:30 +02:00
Lorenzo Delgado f7584dfc49
feat(protobuf): added error wrappers for invalid length validation 2023-02-20 15:03:32 +01:00
Lorenzo Delgado 9258914057
fix(jsonrpc): encode waku messages payload in base64 (#1139) 2023-02-14 09:19:06 +01:00
Alvaro Revuelta 28e5bcf8b5
fix(networking): fix flaky peerstore test (#1554) 2023-02-13 18:10:20 +01:00
Lorenzo Delgado 0a15ce48d8
refactor(rest): deep code and tests reorganization 2023-02-13 15:22:24 +01:00
Lorenzo Delgado 9a40ad6400
fix: remove unused imports 2023-02-13 15:02:34 +01:00
Lorenzo Delgado 6eb13b9a0b
fix(tests): do not specify port number 2023-02-13 12:27:49 +01:00
Lorenzo Delgado 6782398989
feat(testlib): added global tests rng and waku2 sub-module 2023-02-13 11:43:49 +01:00
Lorenzo Delgado a7f0a7f704
fix(node): remove subscription to waku default network topic (#1545) 2023-02-10 17:55:47 +01:00
Alvaro Revuelta d6887ccdb0
chore: bump nim-libp2p 444b837 (#1546)
* chore: bump nim-libp2p 444b837

* fix: update test with libp2p mods
2023-02-10 16:44:06 +01:00
Lorenzo Delgado 274101af43
refactor(relay): improve waku_relay apis and add tests 2023-02-10 15:17:50 +01:00
Lorenzo Delgado 2f390ce884
refactor(jsonrpc): deep code and tests reorganization 2023-02-10 10:43:16 +01:00
Lorenzo Delgado a853bf52f0
refactor(wakubridge): split message compatibility from wakubridge 2023-02-10 09:54:47 +01:00
Alvaro Revuelta 9c3e5f5bc5
refactor(px): refactor peer exchange + tests (#1527) 2023-02-09 16:59:29 +01:00
G cdc09aeeb4
feat(credentials): store and processing generic app credentials (#1466)
* feat(credentials): store and processing generic app credentials

* feat(credentials): separate module; minimal tests

* more work

* feat(credentials): check presence of idCredential in keystore and add only new membership groups

* feat(credential): refactor, new data structure, dynamic add credential, filter

* feat(credential): add filter, get credentials

* feat(credential): encode/decode utility

* feat(credential): sort groups, test credential retrieval/group merging

* fix(credential): remove unnecessary order in sort

* fix(credentials): fix vendor commits

* fix(credential/rln): embed credential module in rln relay

* feat(credentials/rln): use credentials API in rln-relay to store/read credentials

* refactor(credentials): implement hasKeys for JsonNode

* fix(credentials): restore connectToNodes call

* refactor(credentials): remove unnecessary imports

* refactor(credentials): add Res suffix to results

* refactor(credential): moved save json to separate proc; added comments

* feat(credentials): use appInfo

* refactor(keystore): refactor code in a more structured module; address reviewers

* fix(keystore): fix indentation
2023-02-08 16:26:23 +01:00
Lorenzo Delgado 00f4ce4cbc
refactor(relay): improve wakuy relay api 2023-02-08 16:09:59 +01:00
Alvaro Revuelta 55bac8dedf
feat: add WakuMessage validation in gossipsub (#1537) 2023-02-07 17:42:17 +01:00
Lorenzo Delgado 7cbb0bb28f
chore(tests): moved common test suites to its own folder 2023-02-07 17:18:41 +01:00
Aaryamann Challani 9ddf0fe1e2
feat(discv5): advertise custom multiaddresses (#1512)
* feat(discv5): allow custom multiaddr advertisement in discv5

feat(discv5): allow custom multiaddr advertisement in discv5

feat(discv5): move discv5 setup from wakunode2 to waku_node

fix(waku_node): def param

test(discv5): test for ext multiaddr

fix(discv5): address comments

fix(discv5): address comments

fix(wakunode2): discoveryconfig in var before init

fix(discv5): pass multiaddr to discv5 directly

fix(discv5): make multiaddrs optional

fix(test): discv5 init

fix(discv5): split discv5 mounting from waku_node

chore(discv5): s/WakuAddressMetadata/WakuNodeAddrMeta/g

* fix(waku_node): 1.25 max conns

* fix(discv5): s/WakuNodeAddrMeta/NetConfig/g

* fix(discv5): address reviews

* fix(discv5): smaller try-catches

* fix(discv5): missing arg

* fix: compile error
2023-02-07 18:36:50 +05:30
Lorenzo Delgado 7f2ea1caeb
refactor(utils): moved noise/compat payload encryption code to utils 2023-02-07 10:45:25 +01:00
Aaryamann Challani c9bc774895
fix(rng): create rng only within isMainModule blocks (#1523)
* fix(rng): create rng only within isMainModule blocks

* fix(rng): wakucanary rng

* fix(rng): address comments
2023-02-06 17:23:05 +05:30
Lorenzo Delgado 19abd51976
chore: remove unused test fixture file 2023-02-06 11:06:45 +01:00
Lorenzo Delgado 8ba36e2dfa
chore(peer_manager): added peer_manager barrel module 2023-02-06 10:03:30 +01:00
Hanno Cornelius f36d708e76
fix: improve filter test race condition (#1529)
* fix: improve filter test race condition

* fix: missed one sleep
2023-02-03 10:00:26 +02:00
Alvaro Revuelta da7592bcc7
feat(networking): prune peers from peerstore exceeding capacity (#1513)
* feat(networking): prune peers from peerstore

* chore: add comments

* feat(networking): fix comments

* Add tests
2023-01-31 13:24:49 +01:00
Lorenzo Delgado c5286340d5
test(store): extend wankunode's waku store test coverage 2023-01-30 15:21:51 +01:00
Lorenzo Delgado a590575a6a
refactor(tests): group waku store test suite files 2023-01-27 14:31:58 +01:00
Alvaro Revuelta 93a07babf2
fix(rpc): fix out of bound when empty multiadd (#1507) 2023-01-26 12:05:25 +01:00
Aaryamann Challani d09ec815ce
feat(wakunode): advertise custom multiaddresses (#1509)
* feat(wakunode2): ability to advertise custom multiaddresses

* test(wakunode): test the feature

* fix(wakunode): remove rln diff

* revert(rln-relay): change that creeped into the diff

* fix(wakunode): move extMultiaddrs closer to nat

* fix(waku_node): idiomatic default arg

* fix(config): shortened validation

* fix(wakunode): discoverable via discv5 and dnsdisc
2023-01-26 15:48:30 +05:30
Alvaro Revuelta ea4703e9a2
feat(networking): add service slots to peer manager (#1473) 2023-01-26 10:20:20 +01:00
Lorenzo Delgado 95d31b3ed3
fix(archive): reverse the db query results in the waku archive front-end 2023-01-26 10:19:58 +01:00
Alvaro Revuelta 028efc8547
feat(networking): add backoff period after failed dial (#1462)
* feat(networking): add exponential backoff when dialing relay peers

* feat(networking): fix tests

* revert withTimeout

* feat(networking): refactor tests

* feat(networking): improve logs + ping using switch

* feat(networking): fix backoff bug + fix tests

* feat(networking): fix comments
2023-01-23 21:24:46 +01:00
Alvaro Revuelta fd1ec4a74a
feat(networking): add relay connectivity loop (#1482)
* feat(networking): add relay connectivity loop

* Add unit tests

* feat(networking): fix comments

* Fix lnsd comments
2023-01-18 15:17:56 +01:00
Aaryamann Challani 605cf1c38c
feat(rln-relay): abstract group management into its own api (#1465)
* feat(rln-relay): group manager api

* fix(rln-relay): static gm, method async works now

* fix(rln-relay): cb naming, ensure merkle root changed

* chore(rln-relay): static group manager fully tested

* chore(rln-relay): split ffi into its own module, split conversion_utils

* chore(rln-relay): refactor onchain group

* fix(rln-relay): throw error if privatekey is malformed

* chore(rln-relay): convert methods to procs, remove warnings, formatting

* chore(rln-relay): remove comment

* style(rln-relay): fmt test

* feat(rln-relay): complete onchain group manager. pending tests

* fix(rln-relay): onchain implementation works now

* fix(rln-relay): reg index

* fix(rln): imports

* fix(rln-relay): revert method to proc conv

* fix(rln-relay): s/ffi/rln

* fix(rln-relay): remove rln/ from gitignore

* fix(rln-relay): s/ffi/rln
2023-01-16 18:26:18 +05:30
Lorenzo Delgado 3ee87d1848
feat(archive): respond with error on more than 10 content topics query 2023-01-11 12:19:59 +01:00
Hanno Cornelius 76bb7520f3
chore: update submodules (#1480)
* chore: switch to nim-eth master branch

* chore: updated submodules

* fix: submodule without error handling

* fix: autonat client
2023-01-11 08:06:30 +02:00
Alvaro Revuelta 333ce0a2e1
fix(rpc): return error if peer cant be connected (#1476) 2023-01-09 14:15:03 +01:00
G 226b44c86d
refactor(rln): use zerokit extended_key_gen in place of key_gen (#1459)
* chore(rln-relay): refactor mounting procedure

* fix(rln-relay): tests

* fix(rln-relay|chat2): update mounting proc in chat2, fix args to mountRlnRelay

* refactor(rln): use zerokit extended_key_gen in place of key_gen

refactor(rln): rename membershipKeyPair to identityCredential

update zerokit submodule

refactor(rln): use extended_key_gen; rebrand keypairs to idCredentials

refactor(rln): rename mountRlnRelay memKeyPair argument

* fix(rln): fix wrong field

* fix(rln): add () to toHex

Co-authored-by: Aaryamann Challani <43716372+rymnc@users.noreply.github.com>

Co-authored-by: rymnc <43716372+rymnc@users.noreply.github.com>
2022-12-14 12:28:09 +01:00
Aaryamann Challani 33e9d8b9b2
chore(rln-relay): refactor mounting procedure (#1457)
* chore(rln-relay): refactor mounting procedure

* fix(rln-relay): tests

* fix(rln-relay|chat2): update mounting proc in chat2, fix args to mountRlnRelay

* fix(rln-relay): remove stale todo
2022-12-13 14:56:24 +05:30
Aaryamann Challani f08680045e
chore(rln-relay): address laundry list for tests (#1452)
* chore(rln-relay): s/var/let

* chore(rln-relay): s/var/let

* chore(rln-relay): s/check/require

* chore(rln-relay): s/check/require

* fix(rln-relay): s/check/require
2022-12-07 22:47:08 +05:30
Lorenzo Delgado e4c10ed661
feat(wakunode2): support log format format selection 2022-12-07 12:30:32 +01:00
Alvaro Revuelta 8cce06b8a2
feat(peerstore): store peer direction (#1424)
* feat(peerstore): store peer direction

* feat(peerstore): add getPeersByDirection function + tests

* feat(peerstore): set out own MaxConnectionsPerPeer to 1

* feat(peermanager): add metric for inbound/outbound peers
2022-11-29 17:35:25 +01:00
Hanno Cornelius 08cbe5216a
fix: waku bridge test timing (#1429)
* fix: waku bridge test timing

* fix: shorten timing more
2022-11-28 22:07:21 +02:00
Hanno Cornelius 8d69846aa4
feat: enable AutoNAT and libp2p circuit relay (#1425)
* feat: enable libp2p circuit relay

* feat: enable libp2p autonat
2022-11-28 16:56:01 +02:00
Aaryamann Challani 63fe1351f5
feat(ci): run experimental features in diff ci (#1410)
* feat(ci): run experimental features in diff ci

* chore(ci): add EXPERIMENTAL flag

* chore(makefile): make compilation cleaner

* fix(rln-relay): remove onchain_rln flag

* fix(makefile): remove experimental_rln flag

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2022-11-25 16:03:19 +05:30
Alvaro Revuelta 43fd11b4dc
refactor: reuse nim-libp2p peerstore + move peermanager logic (#1383)
* refactor: reuse nim-libp2p peerstore + move peermanager logic

* refactor: fix comments

* refactor: modify reconnectPeers and unittest

* feat(apps): new flag for peerStoreCapacity

* fix(examples): fix example2 target

* refactor: fix comments
2022-11-24 14:11:23 +01:00
Lorenzo Delgado 089e2ae1e8
test(archive): randomize archive driver query test suites' fixtures 2022-11-24 00:58:17 +01:00
Lorenzo Delgado 6811af328c
feat(archive): add waku archive implementation 2022-11-23 10:08:00 +01:00
Lorenzo Delgado 0188ec1a9e
feat(archive): add waku archive implementation 2022-11-22 19:40:24 +01:00
Lorenzo Delgado 299ee3eb5d
refactor(rln): decouple rln types from waku message type 2022-11-22 18:29:43 +01:00
Alvaro Revuelta 1e8e60caed
fix(tests): add missing import for tests (#1375) 2022-11-22 08:13:51 +01:00
Aaryamann Challani 853995e17f
chore(rln-relay): barrel imports (#1386)
Co-authored-by: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2022-11-21 15:15:03 +01:00
Lorenzo Delgado 59c18ab5c4
feat(store): add support for history query handler 2022-11-21 11:16:57 +01:00
Lorenzo Delgado e85b5cbae1
chore(store): remove waku swap integration from store protocol 2022-11-21 09:36:41 +01:00
Lorenzo Delgado f89e6869cb
fix(store): waku store rpc codec support optional fields 2022-11-17 20:40:08 +01:00
Lorenzo Delgado 39bf289f43
refactor(protocols): decouple protocol rpc types from api types 2022-11-16 20:02:38 +01:00
Aaryamann Challani 07833ce313
feat(rln-relay): process blocks atomically (#1349)
* test(rln-relay): atomic block processing

* fix(rln-relay): use correct starting index

* fix(rln-relay): args

* fix(rln-relay): append length

* fix(rln-relay): tests, remove insertMember

* fix(rln-relay): camelCase, cleanup

* fix(rln-relay): actually process per block

* fix(rln-relay): clean up

* chore(gitignore): Update .gitignore

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Sanaz Taheri <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Sanaz Taheri <35961250+staheri14@users.noreply.github.com>

* fix(rln-relay): args

* fix(rln-relay): add prefix def

* fix(rln-relay): make test cleaner

* chore(rln-relay): apply suggestions

Co-authored-by: G. <28568419+s1fr0@users.noreply.github.com>
Co-authored-by: Lorenzo Delgado <lorenzo@status.im>

* chore(rln-relay): add member order check

* test(rln-relay): batch insert in tests

* fix(rln-relay): test batching

* fix(rln-relay): toSeq the HSlice

* fix(rln-relay): naming

* fix(rln-relay): add insertMember back

* fix(rln-relay): serialize util, address review

* fix(rln-relay): add atomicity desc

* fix(rln-relay): inHex

* fix(rln-relay): explicit proc def

* fix(rln-relay): indexGap condition

* fix(rln-relay): func sig

* fix(rln-relay): onchain test

* fix(rln-relay): use asyncSpawn vs asyncCheck

* fix(rln-relay): do not explicitly insert into the index

* fix(rln-relay): condition, semantics

* fix(rln-relay): index must be 1

* chore(rln-relay): line br

* fix(rln-relay): missing return ok(true)

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
Co-authored-by: Sanaz Taheri <35961250+staheri14@users.noreply.github.com>
Co-authored-by: G. <28568419+s1fr0@users.noreply.github.com>
2022-11-10 22:28:31 +05:30
Alvaro Revuelta 7917e05d9d
chore(networkmonitor): tool to discover and provide metrics on peers (#1290)
* chore(networkmonitor): initial prototype

* chore(networkmonitor): add cli, metrics and PoC

* feat(utils): add supportsCapability function + tests

* feat(utils): add supportedCapabilites function

* chore(networkmonitor): add metrics with enr/ip/capabilities

* chore(networkmonitor): refactor + tests

* chore(networkmonitor): add discovered timestamp

* chore(networkmonitor): add metrics on connected nodes

* chore(networkmonitor): new flags + utils file + readme

* chore(networkmonitor): add user-agent metrics

* chore(networkmonitor): connect only to randomly discovered peers

* chore(networkmonitor): get location of peer using ip

* chore(networkmonitor): expose peer metrics with simple rest server

* chore(networkmonitor): update README

* chore(networkmonitor): fix wakunode2 to waku_node

* chore(networkmonitor): fix import order

* chore(networkmonitor): fix comments + refactor + pushraises

* chore(networkmonitor): refactor + handle exceptions

* chore(networkmonitor): fix makefile after rebase

* chore(networkmonitor): address review comments 1

* chore(networkmonitor): add nim.cfg
2022-11-10 10:29:34 +01:00
G 63137f3e2a
refactor(rln): remove kilic lib (#1365)
* refactor(rln): remove kilic lib

* chore(rln): address reviewerS
2022-11-09 19:45:04 +01:00
Lorenzo Delgado b07cdb1841
refactor(store): decouple waku store public api types from rpc types 2022-11-09 18:50:18 +01:00
Alvaro Revuelta 054dc61763
refactor: continue pubsub/content types started in #1352 (#1362)
* refactor: continue gossip/content topic refactor started in #1352

* refactor: enforce using pubsubTopic instead of topic
2022-11-09 15:00:11 +01:00
Lorenzo Delgado 4acc611e47
refactor: unify pubsub topic type and default value 2022-11-09 09:55:47 +01:00
Lorenzo Delgado c4f9813ab3
refactor(queue_store): rename queue_store module and simplify api 2022-11-08 16:10:23 +01:00
G 17d71faf67
chore(rln): update ganache (#1347)
* chore: build rln dependecies only when building v2

* chore(rln): deprecate ganache-cli; move installation from makefile to test + uninstall; gracefully terminate ganache

* chore: add package.json to gitignore

* fix(rln): move ganache package to build folder and remove package once uninstalled

* refactor(rln): (un)install/run/stop ganache with separate procs

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
Co-authored-by: Aaryamann Challani <43716372+rymnc@users.noreply.github.com>
2022-11-08 14:28:11 +01:00
Lorenzo Delgado bcc6c32287
chore: rename protocols rpc_codec procs from init to decode 2022-11-07 16:24:16 +01:00
Lorenzo Delgado 2c2ce20c4e
chore: build rln dependecies only when building v2 2022-11-07 09:14:21 +01:00
Lorenzo Delgado 4509f4f361
refactor(message_store): move message_store to node module 2022-11-04 19:48:22 +01:00
Lorenzo Delgado cb1f4d46f4
chore: remove unused imports 2022-11-04 10:52:08 +01:00
Lorenzo Delgado 0569beadbe
refactor(peer_manager): move peer_store under peer_manager module 2022-11-04 09:40:13 +01:00
Aaryamann Challani 53e8979aa9
chore(rln-relay): remove all raises and replace with Result types (#1321)
* chore(rln-relay): remove all raises and replace with Result types

* chore(rln-relay): s/var/let

* chore(rln-relay): s/isOk/isOk()

* fix(rln-relay): proc def comments about the result type

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2022-11-04 08:30:42 +05:30
Lorenzo Delgado d1df046c87
feat(wakunode2): support configuration via environment variables 2022-11-03 17:58:48 +01:00
Lorenzo Delgado 85d2842f75
feat(testlib): mark commented test cases as skipped 2022-11-03 17:58:28 +01:00
Lorenzo Delgado 1330f7745f
chore: randomize waku tcp ports in tests 2022-11-03 14:47:56 +01:00
Lorenzo Delgado 1d3943febb
refactor(sqlite): move migrations logic to sqlite common module 2022-11-02 16:36:33 +01:00
Lorenzo Delgado 8fee1b9bed
feat(filter): split waku filter protocol into server and client 2022-11-02 11:59:58 +01:00
Lorenzo Delgado 1c46b61402
refactor(peer-exchange): move peer management to waku_node module 2022-11-02 09:45:21 +01:00
Aaryamann Challani cd73029a0c
feat(rln-relay): track last seen event (#1296)
* feat(rln-relay): track last seen event

* fix(rln-relay): clean up subscribeToMemberRegistrations proc

* fix(rln-relay): tests

* fix(rln-relay): unnecessary try-except

* fix(rln-relay): proc descriptions, logging

Co-authored-by: G <28568419+s1fr0@users.noreply.github.com>
2022-11-01 08:15:34 +05:30
Aaryamann Challani 24d288ccb4
fix(rln-relay): RangeDefect test (#1318)
* fix(rln-relay): RangeDefect test

* fix(rln-relay): any overflow

* fix(rln-relay): dont use int64 when its absolute value is being used anyway

* chore(rln): rename proc to be more meaningful about operation

* fix(rln): remove unused imports
2022-11-01 05:55:39 +05:30
Lorenzo Delgado 0de5a122ef
chore(common): move sqlite module to common module 2022-10-28 20:13:41 +02:00
Lorenzo Delgado ee86d190d6
refactor(node): use store client in waku_node 2022-10-28 20:11:28 +02:00
Lorenzo Delgado ed9b2b59b2
chore: remove legacy unused test suite 2022-10-28 17:44:46 +02:00
Lorenzo Delgado 2c1ad9f760
refactor(node): use lightpush client in waku_node 2022-10-28 16:30:01 +02:00
Alvaro Revuelta d479aacec5
feat(wakunode2): add nwaku agent-string to switch (#1302) 2022-10-28 15:12:06 +02:00
Lorenzo Delgado 979f1d397a
feat(testlib): add enable/disable test debug utils 2022-10-28 15:04:29 +02:00
Hanno Cornelius 16c85db43c
chore: update submodules and bump Nim to 1.6.6 (#1307)
* chore: update submodules

* fix: libp2p now provides callback to update peer addrs

* fix: breaking change in EthereumNode constructor

* fix: contentType type has changed (again)

* fix: explicit future type

* fix: nim 1.6.6 error handling requirements

* fix: missed a spot - peer info addrs in sim2

* fix: help compiler a bit here
2022-10-28 12:51:46 +03:00
G ea8d72188e
feat: add keyfile support for RLN credentials secure storage (#1285)
* feat(utils): add keyfile + test

* feat(keyfile): make id and version field optional

* feat(rln): enable input password for keyfile encryption

* refactor(keyfile): store multiple keyfiles in single file

* refactor(rln): write and read rln credentials proc using keyfiles + test update

* feat(keyfile): allow skip successful decryption to iterate among multiple keyfiles + test

* fix(rln): add rln-relay-cred-password in wakunode2 config

* chore(keyfile): remove unnencessary echo

* refactor(rln/kesyore): address reviewers' comments

* refactor(rln/keyfile): use defer instead of try/finally

* feat(rln): address reviewers comments; return all successful decryptions

* chore(keyfile): add comments; changed name to loadKeyfiles

* feat(keyfile): chmod 600 for keyfiles

* fix(rln): fix errors raised by readRlnCredentials

* feat(keyfile): add nim-eth tests; address reviewers comments

* fix(keyfile/rln): better error handling

* refactor(keyfile): refactoring nim-eth tests

* fix(keyfile): fix wrong index

* fix(keyfile): fix wrong password generation in test

* feat(keyfile): add wrong mac test

Co-authored-by: Aaryamann Challani <43716372+rymnc@users.noreply.github.com>
2022-10-28 11:13:05 +02:00
Lorenzo Delgado 62328f0abd
feat(wakunode2): simplify wakunode2 config and decouple peer persistence 2022-10-28 00:05:02 +02:00
Lorenzo Delgado 7e7bba4a98
feat(lightpush): add waku lightpush protocol client 2022-10-25 14:55:31 +02:00
Lorenzo Delgado 9b59052bcd
chore: extract test procedures and constants into a common module 2022-10-21 15:01:39 +02:00
Lorenzo Delgado e5c3aa560e
feat(store): add waku store client module 2022-10-20 18:09:40 +02:00
Lorenzo Delgado 89c75f414f
chore(store): disable temporarily the store resume flaky test case 2022-10-20 14:56:28 +02:00
Hanno Cornelius a12a359b79
fix: store cursors should be exclusive and match a DB item (#1263)
* fix(store): use exclusive cursor

* fix(store): read correct column length

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2022-10-20 13:24:40 +03:00
Lorenzo Delgado 5046a4b3da
refactor(node): split wakunode into waku_node and wakunode2 2022-10-18 09:05:53 -05:00
Lorenzo Delgado 5fbc537cbf
chore: move apps and tools to their own directory 2022-10-12 14:41:25 -05:00
G a2a5a6ef01
refactor(rln): display byte arrays in hex as uints and fix Kilic's proofVerify proc (#1259)
* refactor(rln): convert 32 bytes arrays to hex as uints

* fix(rln): fix Kilic's proofVerify proc to take stub validRoots

* refactor(rln): use input's length when convert to/from hex

* chore(rln): print more commitments inHex in debug logs
2022-10-12 10:38:48 +02:00
Aaryamann Challani 04154ab90f
chore(rln-relay): make account address optional (#1258)
* chore(rln-relay): make account address optional

* fix(rln-relay): onchain test

* chore(chat2): update docs, and handle error case in chat2

* fix(rln-relay): handle registration better
2022-10-12 07:48:11 +05:30
G 3e1e7cc74f
feat(RLN): Add verify against multiple roots (#1250)
* update zerokit submodule

* fix(rln): allow Buffer creation on empty arrays

* feat(rln): use verify_with_root

* fix(rln): update test to work with verify_with_root
2022-10-07 21:24:54 +02:00
Lorenzo Delgado 2c975597e1
fix(store): fix waku store resume tests 2022-10-05 17:58:24 +02:00
Lorenzo Delgado 14abdef796
chore(store): improve database migrations logging 2022-10-05 11:45:44 +02:00
Lorenzo Delgado fa6a44eb3c
fix(store): Move page info logic to waku store protocol 2022-10-03 17:36:17 +02:00
Lorenzo Delgado adf1dab34f
fix(store): update sqlite db table primary key and decouple queries module from pagination types 2022-10-03 15:23:39 +02:00
Lorenzo Delgado c61e88b352
refactor(store): simplify store queue query implementation 2022-10-03 08:40:01 +02:00
Aaryamann Challani d614e4504b
feat(rln-relay): metrics (#1185)
* feat(rln-relay): metrics

* fix(rln-relay): invalid token

* fix(rln-relay): return outside time

* style(rln-relay): multiline proc def

* fix(rln-relay): oserror

* fix(rln-relay): Exception should not be raised

* Revert "fix(rln-relay): oserror"

This reverts commit edcf39c0088ca526ca50256d856eee20d87a3c37.

* Revert "fix(rln-relay): Exception should not be raised"

This reverts commit c8d09b055314a83bb4a94900a47e899773cb47df.

* fix(rln-relay): revert exception removal

* feat(rln-relay): granular tracking

* fix(rln-relay): observe only on histogram, convert mount to hist

* chore(rln-relay): add additional metrics

* fix(rln-relay): missing import

* fix(rln-relay): template should expand only when times is defined

* fix(rln-relay): follow prometheus naming style

* fix(rln-relay): explicitly assert that the root window size should be less than 10

* fix(rln-relay): outstanding issues with infinite labels, minor refactor

* fix(rln-relay): typo

* fix(rln-relay): screaming case to pascal case

* fix(rln-relay): typos

* fix(rln-relay): imports

* fix(rln-relay): remove dependence on times, make durations as gauges

* fix(rln-relay): import order
2022-09-30 18:13:42 +05:30
Lorenzo Delgado 612023fab0
chore(store): collocate pagination and message store types 2022-09-28 13:36:05 +02:00
Lorenzo Delgado 5ee4a00765
refactor(store): decouple message store queue from pagination index type (#1187) 2022-09-27 21:10:11 +02:00
Lorenzo Delgado d7d51dc4c1
fix(store): rename store timestamp sqlite column 2022-09-27 11:47:25 +02:00
Aaryamann Challani 37cd8b593b
feat(rln-relay): multiple acceptable roots (#1177)
* feat(rln-relay): multiple acceptable roots

* fix(rln-relay): make sure onchain handler uses correct proc

* fix(rln-relay): typo

* style(rln-relay): convert const to pascalcase

* chore(rln-relay): address pr review

* fix(rln-relay): add require to test

* fix(rln-relay): add proc desc

* fix(rln-relay): ensure that group id commitments were added correctly

* fix(rln-relay): potential mem leak

* style(rln-relay): comments

* fix(rln-relay): magic number ambiguity

* chore(rln-relay): comment

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
2022-09-27 10:10:04 +05:30
Lorenzo Delgado 1b4a9e5dfd
fix(store): move insert criteria logic to waku store protocol module 2022-09-26 11:50:15 +02:00
Lorenzo Delgado aebda46a53
chore(store): move constants to retention policies modules 2022-09-22 11:17:38 +02:00
Lorenzo Delgado 5e3a75c56e
refactor(wakunode): remove deprecated non-async methods 2022-09-21 18:27:40 +02:00
Lorenzo Delgado 11832411a0
feat(store): execute retention policies periodically (#1155) 2022-09-21 11:32:59 +02:00
Aaryamann Challani e8d77306c8
feat(rln-relay): validate the merkle root in the RateLimitProof (#1158)
* feat(rln-relay): validate the merkle root in the RateLimitProof

* style|chore: replace snake case usage, convert proofVerify to Result[bool,string]

* chore(rln-relay): remove unhandled usage of result returned

* fix(rln-relay): typos

* chore(rln-relay): validate root before proof verification

* fix(rln-relay): cmp

* fix(rln-relay): generic rln relay result

* style(rln-relay): readability

* fix(rln-relay): use patterns for testing, requires instead of checks

* fix(rln-relay): move root validation to message validation

* fix(rln-relay): typo

* fix(rln-relay): typo

* chore(rln-relay): address pr review
2022-09-20 18:38:05 +05:30
kaiserd 371016b04f
feat: waku peer exchange (RFC34) (#1152) 2022-09-20 13:03:34 +02:00
Lorenzo Delgado 4a9f6f3c5e
refactor(store): protocol code refactoring and api reorganization 2022-09-20 11:39:52 +02:00
Lorenzo Delgado f73477be6e
refactor(store): invert retention policies dependency on message store interface 2022-09-16 12:55:22 +02:00
Lorenzo Delgado ddbc57bc88 refactor(wakunode): remove waku message export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado c2c2d30cdf refactor(wakunode): remove libp2p builders export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado 2912a45433 refactor(wakunode): remove waku relay export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado fdd57c7bce refactor(wakunode): remove waku swap export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado d49cd1348d
refactor(store): remove deprecated waku message store type 2022-09-13 13:36:04 +02:00
Aaryamann Challani fdcc9824a2
feat(store): Allow messages to be marked as "don't store" (#1119)
* feat(store): init allow messages to be marked as ephemeral

* feat(store): replace ephemeral with ttl model

* Revert "feat(store): replace ephemeral with ttl model"

This reverts commit 4398c61c919dda8f014b4fcc0679c573c74854e4.

* fix(store): flags and test

* chore(store): remove storeTTL

* chore(store): remove unused utils

* fix: conflict

* fix(store): make test more brief
2022-09-13 16:07:06 +05:30
Lorenzo Delgado c171e3dfcd
refactor(wakunode): split wakunode relay tests into a new file 2022-09-12 14:51:52 +02:00
G 01cd201f58
PoC implementation of Waku Pairing and Secure Transfer (#1117)
* feat(noise): add PoC implementation for WakuPairing and Secure Transfer
2022-09-12 02:23:14 +02:00
Lorenzo Delgado 797c82b030
refactor(wakunode): split wakunode store tests into a new file 2022-09-09 15:54:16 +02:00
Lorenzo Delgado 5fd2eb08ed
refactor(wakunode): split wakunode filter tests into a new file 2022-09-09 12:17:17 +02:00
Lorenzo Delgado e2590b588e
refactor(wakunode): split wakunode lightpush tests into a new file 2022-09-08 23:18:59 +02:00
Hanno Cornelius 7f23bdf29f
chore: update nwaku submodules (#1123)
* chore: update submodules

* fix: SIGTERM ambiguity

* fix: ambiguous RNGs

* fix: ContentType is no longer a string

* fix: more fixes related to ambiguous RNGs

* fix: start all protocols

* chore: also update nim-eth

* chore: important new fixes in nim-libp2p

* fix: more changes related to RNG. Some reversion to reflect nim-eth update

* fix: breaking changes in nim-eth submodule

* fix: start protocols in tests

* fix: chat2bridge protocols async mounting

* fix: v1 test compilation

* fix: rln test compilation

* fix: remove confusing keys qualifier for the same HmacDrbgContext
2022-09-07 16:31:27 +01:00
Lorenzo Delgado fbaf295ac1
refactor(wakunode): split wakunode rln-relay tests into a new file 2022-09-06 20:43:46 +02:00
Lorenzo Delgado ab373a9abd
refactor(wakunode): split peer utils tests into a new file 2022-09-06 16:05:06 +02:00
Sanaz Taheri Boshrooyeh 0deedf0d7a
fix|feat(rln-relay|rln-chat2): Handling on-chain transactions errors and informing chat2 users (#1093)
If any error occurs during the on-chain registration, it will be handled; the chat2 users will get informed about the cause and the installation of rln-relay gets discarded and the chat program continues with no crashing.
2022-08-30 10:59:02 -07:00
Lorenzo Delgado ecf4ba1167
feat(message-cache): make the topic cache generic 2022-08-30 15:57:45 +02:00
Lorenzo Delgado 791ce6d222 fix(rest): encode waku message payload in base64 2022-08-30 14:53:04 +02:00
Lorenzo Delgado 842cfb3476 refactor(waku-filter): remove wakunode2 waku_filter exports 2022-08-29 10:14:35 +02:00
Lorenzo Delgado c964aea885 refactor(waku-filter): waku filter protocol code reorganisation 2022-08-26 22:17:56 +02:00
Lorenzo Delgado 73df74f8c6 refactor(waku-store): remove wakunode2 waku_store exports 2022-08-18 17:32:47 +02:00
Lorenzo Delgado 7a30e485ec refactor(waku-store): deprecate waku store callback methods 2022-08-17 17:22:56 +02:00
G 2c4730c850
fix(rln): add zerokit rln to chat2 + fix static group members (#1077) 2022-08-17 12:11:46 +02:00
Lorenzo Delgado 2cb7123df8 fix(waku-store): integrate the predicate into the WHERE clause instead of calling it on each result line (#1026) 2022-08-12 12:48:45 +02:00
Lorenzo Delgado d201a37330 refactor(waku-lightpush): waku lightpush protocol code reorganization 2022-08-10 12:30:34 +02:00
Lorenzo Delgado 1575312a0c fix(waku-store): fix missing import issue 2022-08-10 11:43:31 +02:00
Lorenzo Delgado 0c56d3792a refactor(waku-store): rename store queue test suite 2022-08-10 11:43:31 +02:00
Lorenzo Delgado feda542fdb refactor(waku-store): split waku store test suite 2022-08-10 11:43:31 +02:00
G 2aeaba48e4
feat(noise): add padding to transport messages (#1056)
* feat(noise): add padding to transport messages (handshakes+write+read)

* fix(noise): address reviewers' comments

* fix(noise): update padding block size const name
2022-08-07 16:20:34 +02:00
G 8a14b5b243
Zerokit RLN integration in nwaku: alternative approach with inline compilation flag (#1060)
* refactor(rln): add inline compilation flags for RLN zerokit

* update submodule

update submodule

* fix(rln): fix wrong merge conflict resolution

* fix(rln): align hardcoded values in light of vacp2p/zerokit/issues/28
2022-08-05 22:58:19 +02:00
Keshav Gupta 115982f5c3
Persisting rln credentials (#1037)
* Persisting Credentials implemented by writing json of keypair and rlnindex to files

* Removing testing files and ignores

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Daniel Kaiser <git@kais3r.de>

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Daniel Kaiser <git@kais3r.de>

* Comments

* Comments

* Security warning in comments

* Redundant echos. Omitting unused variables.

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Limit line lengths using line breaks and indents

* Membership file paths declared as const

* Const fix

* Rln Credentials Merged.
Reading credentials from file abstracted away.

* Spaces

* Spaces

* Dangling constants removed. Comments position.

* Import sequence.

* Path as argument to readPersistentKeys.
Spaces in comments

* readPersistentKeys test

* Debug and info

* Index check in test

* Update tests/v2/test_waku_rln_relay.nim

Co-authored-by: G. <28568419+s1fr0@users.noreply.github.com>

* Abstracted writeRlnCredentials.
Fix var name in test.

* Usage of writeRlnCredentials in test

* Dnsclient?

* Test reverted to direct call to writeFile.
Abstrated writePersistentRlnCredentials usage causing error,
with readPersistentRlnCredentials

* Indentation

* Revert "Dnsclient?"

This reverts commit 3f4dba1a0b.

* Reverting abstraction of writing..
..persiting credential

Co-authored-by: Daniel Kaiser <git@kais3r.de>
Co-authored-by: Keshav Gupta <keshav.pg@hotmail.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: G. <28568419+s1fr0@users.noreply.github.com>
2022-08-05 12:48:01 +02:00
G 32f91cb26b
Noise: split Noise submodule in smaller submodules (#979)
* refactor(noise): split Noise submodule in smaller submodules
2022-08-04 10:47:00 +02:00
Lorenzo Delgado b0d4e25984 refactor(waku-store): reorganise pagination test cases 2022-08-02 18:48:30 +02:00
Lorenzo Delgado 888f7cb312 refactor(waku-store): major code reorganization, move StoreQueue to message_store folder 2022-08-01 13:02:40 +02:00
Lorenzo Delgado a54e25972f chore: remove test suite compiled binary artifact 2022-08-01 12:00:53 +02:00
Sanaz Taheri Boshrooyeh 79459d29ce
Feat(Rln relay): enables relay-mode only (#1051)
* makes eth private key optional

* makes eth account private key field optional

* clarifies a comment
2022-07-28 11:45:29 -07:00
Lorenzo Delgado 732f853f55 feat(rest): Add HTTP REST API (#727). Integrate REST API with the Waku V2 node 2022-07-19 11:02:43 +02:00
Sanaz Taheri Boshrooyeh a40686e26e
Feat (Rln relay): enabling signed tx (#1023) 2022-06-30 18:05:38 -07:00
Lorenzo Delgado 993781a137 chore: remove unused imports 2022-06-30 00:35:18 +02:00
Sanaz Taheri Boshrooyeh 3e20127933
Chore(rln-relay): decouples rln-relay and wakunode2 modules (#1019)
* adds wakunode2_types

* removes unused imports
2022-06-27 12:35:26 -07:00
Hanno Cornelius 9be64a55b6
fix: bridge loses connection to v1 (#1020)
* fix: bridge loses connection to v1

* Remove magic numbers in checks
2022-06-27 14:16:15 +02:00
Kim De Mey 6f37c7eba4
Run test_shh test as this code is used in Waku v1 (#1011) 2022-06-23 20:59:49 +02:00
Lorenzo Delgado cf1a7c3074 feat(rest): Add HTTP REST API (#727). Add Relay REST API 2022-06-23 10:12:38 +02:00
Lorenzo Delgado d2feb7c763 feat(rest): Add HTTP REST API (#727). Amend Debug REST API client 2022-06-22 20:30:44 +02:00
Sanaz Taheri Boshrooyeh 6c8ab0ab0f
Feat(rln-relay): enables two modes of onchain and offchain (#992)
* adds the contract handler file

* adds integration test for the group listening

* adds groupManagement proc

* deletes rln relay contract handler file

* brings back all the tests

* replaces toUINT256 with getIdCommitment proc

* replaces individual futures with an array of futures

* adds code documentation

* asyncSpawn instead of await

* adds untitest for toIDCommitment and toUInt256

* reorganizes the test and add rlnInstance

* mounts handleGroupUpdates on the rln peer

* asyncSpawn to await

* implements toIDCommitment

* updates the unittest

* improves the code documentation

* removes unused tests

* WIP

* uncomments the tests

* defines a new mountRlnRelayStatic proc

* splits mountRlnRelay into two procs for dynamic and static group management

* adds a config for off-chain and on-chain rln-relay

* runs dynamic or static mode of rln-relay based on the input config

* adds Eth private key and account configs

* reads Eth private key and account the configs

* comments put the second register proc

* add proper call to the rlnrelay dynamic mode

* adds todo

* adds new rln relay configs

* splits register into two procs

* makes eth account private key non-optional

* removes getIdCommitment and edits the register proc

* removes getIdCommitment calls

* uncomments the commented tests

* fixes a bug

* removes contract deployment for the offchain test

* fixes a bug, edits comments

* removes custom types  without proper parsing and serialization routines from the configs

* fixes a bug

* switches to stew byte utils

* removes log decoding

* WIP

* updates register proc

* edits test titles

* removes eth private key config

* changes the output of register proc to return the registered index

* integrates the registration process into mountRlnRelayDynamic

* integration test for the register proc

* brings back the onchain tests

* updates comments

* cleans up

* disambiguates the Address type namespace

* fixes type ambiguities

* adds default values for rln key and index

* updates config descriptions

* adds type conversion from hex to MembershipKeyPair

* adds more code documentation

* passed the group value instead of option  to the mount proc

* fix a bug

* a minor input type fix for rln chat2

* groups let declarations

* adds default values for addresses

* logs registered keys
2022-06-17 15:00:19 -07:00
Lorenzo Delgado 76e9a98841 feat(rest): Add HTTP REST API (#727). Debug API POC 2022-06-14 18:20:15 +02:00
Daniel Kaiser 5445303a23
feat(store): sqlite only store (#982)
* feat(sqlite): parameterized prep statement exec
  - from nim-eth
* feat(store): sql-only-store
  - add getPage(...) to sql-message-store
  - acts as separate store impl that can be activated with
    `--sqlite-store=true`
* test(store): new test-suite for sql message store
* test(store): waku_store test SQL-store compatible
* perf(store): avoid builing an additional B-tree in sql query
  - use `receiverTime` as the last ORDER BY column to fully utilize the
    primary key sorting
feat(store): retention time
  - retention time in seconds can be specified via `--sqlite-retention-time=`
2022-06-13 19:59:53 +02:00
G c259f4cbaa
feat(noise): add Noise Handshake processing and after-handshake encryption (#934)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* feat(noise): add Noise Handshake State Machine and primitives

* feat(noise): add Handshake processing and after handshake encryption

* feat(noise): code updated according to #933

* fix(noise): export HandshakeResult

* feat(noise): add pre message validation utility

* chore(noise): improve documentation

* fix(noise): address reviewers comments
2022-06-03 21:12:41 +02:00
Hanno Cornelius d5a04a2719
chore: update submodules (#987) 2022-06-01 11:49:41 +02:00
Sanaz Taheri Boshrooyeh 1ac029025a
Feat (Rln relay): adds utility procs to listen to the registration events emitted by the rln membership contract (#976)
* adds the contract handler file

* adds integration test for the group listening

* adds groupManagement proc

* deletes rln relay contract handler file

* brings back all the tests

* replaces toUINT256 with getIdCommitment proc

* replaces individual futures with an array of futures

* adds code documentation

* asyncSpawn instead of await

* adds untitest for toIDCommitment and toUInt256

* reorganizes the test and add rlnInstance

* mounts handleGroupUpdates on the rln peer

* asyncSpawn to await

* implements toIDCommitment

* updates the unittest

* improves the code documentation

* removes unused tests

* removes registration of the dynamic group management handler

* adds a comment

* adds a comment

* adds a TODO

* removes getIdCommitment
2022-05-30 12:14:07 -07:00
Lorenzo Delgado 32d230b474 feat(store): ability to decouple store from relay (#937) 2022-05-23 20:42:48 +02:00
G 710ce1315a
feat(noise): add Noise Handshake State Machine and primitives (#933)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* feat(noise): add Noise Handshake State Machine and primitives

* Fixed header

* refactor(noise): add comments, restyle code

* refactor(noise): address reviewer's comments

* feat(noise): add Noise Handshake State Machine and primitives

* refactor(noise): rebase

* refactor(noise): refactor and add documentation

* fix(noise): fixed trace parameter

* update submodules

* fix(noise): remove echo, add stdout.write in print handshake

* refactor(noise): add Noise state machine overview

* Revert "update submodules"

This reverts commit 9fc162dff8.

* update submodules

* fix Jenkins caching

* Revert "update submodules"

This reverts commit 1927d9dd26.

* revert nim-eth submodule update

* fix(noise): missing camelCase

* feat(noise): adding unit tests for Noise State machine primitives

* fix(noise): address reviewers' comments

* fix(noise): use expect for error handling in unit tests

* fix(noise): revert to implicit nonce declaration in CipherState initialization
2022-05-20 15:51:36 +02:00
Hanno Cornelius 984f3179a7
chore: update submodules (#974) 2022-05-20 12:57:19 +02:00
Sanaz Taheri Boshrooyeh 47a9fcfd77
chore|feat (waku-rln-relay): modules reorganization|Initial test for capturing events using nim-web3 (#941)
* first edition

* adds the full test scenario

* fixes typos

* fixes a bug in the supplied command

* further edits the description

* displays the chat prompt after spam detection

* updates changelog

* minor wording fix

* adds a new test file for onchain rln relay

* adds the Event proc

* adds one working example of event subscription

* defines a new unitt test for event subscription

* adds the new test file

* cleans up the code

* adds a working event subscription for faucet contract

* wip

* makes faucet test conditional

* updates contract byte codes

* adds a working test for event subscription and cleans up the tests

* fixes case

* adss toUInt256 unit function

* enables the tests

* fixes a bug

* undo commented tests

* cleans up the test

* logs the pk

* removes excess entry in the changelog

* fixes spacing

* comments

* removes unused test codes

* adds the conditional compilation for onchain tests

* uncomments offchain tests

* removes onchain tests

* reorganizes the code and moves the rln contract data into a separate module

* deletes txt files

* beautifies the code

* beautifies the code

* removes an excess line

* more formatting fixes

* minor fix

* updates the case of membership fee const

* renames compare to diff

* renames time to e

* edits the number of arguments of the send proc

* fixes a comment alignment

* fixes indentation

* fixed id style

* splits check from condition

* fixes a naming mismatch
2022-05-10 14:09:18 -07:00
G cb11c192d6
feat(noise): add support to Waku Payload V2 (#932)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* Fixed header

* refactor(noise): add comments, restyle code

* refactor(noise): address reviewer's comments

* Update nim-eth

* fix(noise): moved strutils in std
2022-04-12 13:50:51 +02:00
Hanno Cornelius 450e0494f1
fix(store): timestamp validity checks (#876) 2022-04-11 14:58:18 +02:00
G d2fccb5220
feat(noise): add support to encrypted/unencrypted public keys (#931)
* feat(noise): adding ChaChaPoly encryption

* test(noise): add test for ChaChaPoly Encryption/decryption

* style(noise): converted variables to camelCase

* feat(noise): add support to Noise public keys

* Fixed header

* refactor(noise): address reviewer's comments

* refactor(noise): small fixes/comments

* refactor(noise): add empty plaintext/ciphertext handling + string encryption test

* refactor(noise): address reviewer's comments
2022-04-06 14:37:02 +02:00
G 45b1e10647
feat(noise): add support to ChaChaPoly encryption (#930)
* feat(noise): adding ChaChaPoly encryption

* fix(noise): restored waku2 tests

* fix(noise): missing comma

* Fixed header

* fix(noise): made comment about empty ChaChaPolyKey clearer

* refactor(noise): address reviewer's comments

* test(noise): add test for ChaChaPoly Encryption/decryption

* refactor(noise): removed libp2p specific debug tools usage

* style(noise): converted variables to camelCase
2022-04-04 17:46:51 +02:00
Hanno Cornelius 52de0e434f
feat(relay): add support for gossipsub peer exchange (#911) 2022-03-29 10:09:48 +02:00
Daniel Kaiser d0cf3ed1f9
fix(store): capacity limit for persistent message store (#900)
* fix(store): limit capacity of persistent message store

fixes performance issue of loading a large message table by deleting
old messages in case of an overflow

* test(store): message capacity limit
2022-03-28 20:27:14 +02:00
Hanno Cornelius 994ac49472
Fix: timestamp type could overflow before conversion (#892) 2022-03-18 12:04:11 +02:00
Daniel Kaiser e1146b2df8
discv5 dns bootstrap (#888) 2022-03-17 17:33:17 +01:00
Hanno Cornelius 9165720f63
Bridge improvements (#875) 2022-03-08 11:48:17 +01:00
Hanno Cornelius 20efdd7255
Store data model updates and fixes (#864) 2022-02-28 17:29:01 +01:00
Hanno Cornelius 2fd5b6c4ea
Domain name config for `nwaku` node (#852) 2022-02-18 12:10:38 +01:00
G 21cac6d491
Refactoring timestamps (#842)
* Refactor timestamps type from float64 to int64 (milliseconds resolution)

* Revert epochs to float64

* Update 00002_addSenderTimeStamp.up.sql

* Update quicksim2.nim

* Add files via upload

* Delete 00003_convertTimestampsToInts.up.sql

* Add files via upload

* Rename 00003_convertTimestampsToInts.up.sql to 00003_addTimestampsToInts.up.sql

* Delete 00003_addTimestampsToInts.up.sql

* Rln-relay integration into chat2 (#835)

* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* removes duplicate epoch time calculation

* updates log level to trace

* updates default rln-relay content topic

* adds support for empty content topics

* updates changelog

* changelog updates

* removes a commented code block

* updates addRLNRelayValidator string doc

* Squashed commit of the following:

commit bc36c99ab2
Merge: dc2b2946 5a77d6e2
Author: G <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 01:10:06 2022 +0100

    Merge branch 'master' into int64-timestamps-ns

commit dc2b294667
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:24:45 2022 +0100

    Fix

commit f97b95a036
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:13:18 2022 +0100

    Missing import

commit 060c4f8d64
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:10:36 2022 +0100

    Fixed typo

commit 08ca99b6f6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:59:20 2022 +0100

    Time util file

commit 2b5c360746
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:33:20 2022 +0100

    Moved time utility functions to utils/time

commit fdaf121f08
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:10:25 2022 +0100

    Fix comment

commit c7e06ab4e7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:04:13 2022 +0100

    Restore previous migration script

commit 80282db1d7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:54:15 2022 +0100

    Typo

commit b9d67f89b0
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:49:29 2022 +0100

    Added utilities to get int64 nanosecond, microsecond, millisecond time resolution from float

commit 0130d496e6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:36:35 2022 +0100

    Switched to nanoseconds support.

* Update CHANGELOG.md

* Create 00003_convertTimestampsToInt64.up.sql

Migration script

* Moved migration script to right location

* Update waku_rln_relay_utils.nim

* Update waku_rln_relay_utils.nim

* Addressed reviewers' comments

* Update default fleet metrics dashboard (#844)

* Fix

* No need for float

* Aligning master to changes in PR

* Further fixes

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2022-02-17 16:00:15 +01:00
Hanno Cornelius bb3e59454e
Store performance improvements (#849) 2022-02-17 11:00:45 +01:00
Sanaz Taheri Boshrooyeh d851d48424
Updates rln-chat2 interface (#846)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* WIP

* removes duplicate epoch time calculation

* converts echo to log

* invokes handler

* bug fix

* prints calculated epoch

* changes the format of printed epoch

* updates log levels

* logs the input buffer supplied to the generate_proof

* replaces echos with logs

* changes log level to trace

* resets the log level of chat2 to INFO

* upgrades log level to debug

* exports toRLNSignal and adds a doc string

* updates log level

* enables all test2

* removes an echo statement

* modifies a comment

* further updates on the log level

* a minor update

* invokes the spam handler when provided

* checks for payload version

* deletes a redundant check

* deletes a rendant check

* updates default rln-relay cht2 content topic

* adds a todo and log

* changes the case of testnet content topic

* removes a flaky check
2022-02-16 14:52:21 -08:00
Hanno Cornelius e1254c20ae
Support for dns `multiaddrs` (#851)
* Support for dns addrs

* Tests and minor improvements
2022-02-16 17:12:09 +01:00
Sanaz Taheri Boshrooyeh 5a77d6e2a6
Rln-relay integration into chat2 (#835)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* removes duplicate epoch time calculation

* updates log level to trace

* updates default rln-relay content topic

* adds support for empty content topics

* updates changelog

* changelog updates

* removes a commented code block

* updates addRLNRelayValidator string doc
2022-02-04 15:58:27 -08:00
Sanaz Taheri Boshrooyeh a5f8b90c23
Enables RLN compilation under CI env var (#786)
* compiles RLN if CI is defined

* minor ineffective change

* minor

* removes excess space

* resolves namespace conflicts

* moves pubsub import under compiler flag

* disables rln on-chain tests

* brings back the on-chain tests and fixes a name mismatch

* adds a debug log message

* minor formating

* kills ganache-cli procces on windows

* wip: adds a line to kill ganache-cli on windows

* disables rln-relay tests that depend on ganache-cli

* disables ganache-cli installation

* introduces onchain rln flag

* fixes a flaky test

* enables onchain test in non-windows CIs

* moves variables around

it seems detected_os is undefined, hence the compiler flag onchain_rln, which is conditioned to the os, does not get added

* limits onchain rln to macOS
2022-01-28 13:57:17 -08:00
Hanno Cornelius 4421b8de00
Improve node metrics (#831)
* Improve node metrics

* Add support for empty content topics as a separate label
2022-01-26 12:02:57 +01:00
Daniel Kaiser d45e3c8f7a
stop sockets after use in test_waku_store.nim (#829)
* stop sockets after use

- this frees resources (closes sockets)

* rename switch and proto

- in asyncTest "resume message history" to avoid confusion.

Co-authored-by: ksr <kaiserd@users.noreply.github.com>
2022-01-25 10:31:14 +08:00
Daniel Kaiser b3a7722297
result aggregation in resume API (#816) 2022-01-18 23:05:41 +01:00
Hanno Cornelius 3e9ca25abc
Update submodules (#814) 2022-01-14 10:25:01 +01:00
Hanno Cornelius f3ecdb80ea
Remove node peerInfo copy (#801) 2022-01-10 16:07:35 +01:00
Hanno Cornelius 62dbb3d0f3
Add MessageIdProvider for `WakuRelay` (#803)
* Add MessageIdProvider

* Fix unit test. Changelog.
2022-01-10 15:07:01 +01:00
Hanno Cornelius 685d43b2b3
Ensure store response never exceeds `MaxPageSize` (#811)
* Never exceed maxPageSize, even with no specified pagination

* fix test

* Changelog

* Forgot to stage everything...

* Account for timing difference in Windows
2022-01-07 15:01:23 +01:00
rshiv b2273fff9a
Handles stale clients in FILTER protocol (#782)
* Handles stale clients

Signed-off-by: rshiv <reeshav96@gmail.com>

* adds test

Signed-off-by: rshiv <reeshav96@gmail.com>

* removes failed client from subscriber list

Signed-off-by: rshiv <reeshav96@gmail.com>

* adds filter timeout config

Signed-off-by: rshiv <reeshav96@gmail.com>

* reverts peer removal logic

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolve ci issues

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comment

Signed-off-by: rshiv <reeshav96@gmail.com>

* solves review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* ChangeLog update

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comment

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update CHANGELOG.md

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* fixes review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* handles CI issues

Signed-off-by: rshiv <reeshav96@gmail.com>

* tries to solve test CI

Signed-off-by: rshiv <reeshav96@gmail.com>

* solves CI issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves ci issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves CI issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2021-12-08 18:35:47 +00:00
Hanno Cornelius dd256e3bd2
Add support for RFC31 ENR (#789) 2021-12-06 20:51:37 +01:00
Hanno Cornelius 7ef51edfa2
Update announced addrs in switch (#784) 2021-11-30 10:12:09 +01:00
rshiv d49c95f6a1
publish addresses from announced address sequence. (#775)
* publish multiple addresses

Signed-off-by: rshiv <reeshav96@gmail.com>

* publish from announced address

Signed-off-by: rshiv <reeshav96@gmail.com>

* change in json

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes format

Signed-off-by: rshiv <reeshav96@gmail.com>

* makes listenstr a sequence

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes the wakuinfo Field

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes the test

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes the tests variable

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes test

Signed-off-by: rshiv <reeshav96@gmail.com>
2021-11-24 15:10:29 +00:00
Sanaz Taheri Boshrooyeh fda5128cc1
WIP: Waku-rln-relay: Message validation and double signaling detection (#769)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds trace log for the valid messages

* brings the log trace one line up
2021-11-23 14:48:40 -08:00
rshiv 53b21be2b5
Resolve jenkins windows error (#777)
* try different key

Signed-off-by: rshiv <reeshav96@gmail.com>

* key change

Signed-off-by: rshiv <reeshav96@gmail.com>

* cert change

Signed-off-by: rshiv <reeshav96@gmail.com>

* add newline to key

Signed-off-by: rshiv <reeshav96@gmail.com>

* exception change

Signed-off-by: rshiv <reeshav96@gmail.com>

* exception log

Signed-off-by: rshiv <reeshav96@gmail.com>

* add debug log

Signed-off-by: rshiv <reeshav96@gmail.com>
2021-11-23 11:40:43 +00:00
Hanno Cornelius 067478d725
Proposal: ENR filtering on Waku-field in `discv5` (#763) 2021-11-12 15:10:54 +01:00
rshiv d2f7076b53
Secure websocket (#759)
* add config

Signed-off-by: rshiv <reeshav96@gmail.com>

* secure websocket integration

Signed-off-by: rshiv <reeshav96@gmail.com>

* secure websocket debug

Signed-off-by: rshiv <reeshav96@gmail.com>

* config change

Signed-off-by: rshiv <reeshav96@gmail.com>

* working secure websocket changes

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update submodules (#761)

* Release v0.6 (#760)

* Read from path

Signed-off-by: rshiv <reeshav96@gmail.com>

* Tests for secure websockets

Signed-off-by: rshiv <reeshav96@gmail.com>

* CI failure fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* path resolver CI

Signed-off-by: rshiv <reeshav96@gmail.com>

* self review fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update examples/v2/config_chat2.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update waku/v2/utils/peers.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/node/wakunode2.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update tests/v2/test_wakunode.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* handle review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
2021-11-10 12:05:36 +00:00
Hanno Cornelius 6d81f1ee55
Update submodules (#761) 2021-11-04 15:46:38 +01:00
Hanno Cornelius ee8ff014f4
Fix: store dimensioning issues (#758) 2021-11-03 11:59:51 +01:00
rshiv d1e06fa17a
Waku v2 Websocket support (#732)
* add config for ws support

* code clean up

* code clean up

* Integrate both ws and tcp transport

* change ws default port

* resolve review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* Unit test debug

Signed-off-by: rshiv <reeshav96@gmail.com>

* Websocket tests addition

Signed-off-by: rshiv <reeshav96@gmail.com>

* websocket failure unittest addition

Signed-off-by: rshiv <reeshav96@gmail.com>

* Commented testcase related to incorrect helper function

Signed-off-by: rshiv <reeshav96@gmail.com>

* Add wireAddr validation

Signed-off-by: rshiv <reeshav96@gmail.com>

* CLoses issue 756

Signed-off-by: rshiv <reeshav96@gmail.com>

* Websocket tests addition

Signed-off-by: rshiv <reeshav96@gmail.com>

* Minor Code clean up

* review comment fixes

Signed-off-by: rshiv <reeshav96@gmail.com>

* Review comment fix

* Update tests/v2/test_wakunode.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* Update tests/v2/test_wakunode.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Changelog update

Signed-off-by: rshiv <reeshav96@gmail.com>

* chat2 var change

Signed-off-by: rshiv <reeshav96@gmail.com>

* test flag changes

Signed-off-by: rshiv <reeshav96@gmail.com>

Co-authored-by: oskarth <ot@oskarthoren.com>
2021-11-02 10:29:11 +00:00
Hanno Cornelius f85434e072
Discv5 POC integration (#748) 2021-11-01 19:02:39 +01:00
Sanaz Taheri Boshrooyeh dbbc0f750b
RLN-Relay: Adds fix to the hash, proofGen and proofVerify procs (to support arbitrary messages) (#753)
* wip: updates the hash interface

* updates hash unittests

* exposes appendLength

* updates proof gen interface and the unittests

* enables test, fixes proofGen and verify utils and unittest, beautifies the code

* removes int size from mem index type

* enables all the rln tests

* adds documentation of two private procs

* documentation for the appendLength

* minor clean up

* adds clarificaltion on the endianness of input length

* Delete .DS_Store
2021-10-26 14:42:24 -07:00
Sanaz Taheri Boshrooyeh 6efba0dc56
Integrates proof generation and verification into wakunode2 (#735)
* WIP

* WIP: fixes a bug

* adds test for static group formation

* adds static group creation when rln-relay is enabled

* adds createStatic group

* wip: adds group formation to mount rlnrelay

* adds createMembershipList utility function

* adds doc strings and todos

* cleans up the code and add comments

* defaults createRLNInstance depth argument to 32

* renames Depth

* distinguishes between onchain and offchain modes

* updates index boundaries

* updates log levels

* updates docstring

* updates log level of displayed membership keys

* relocates a todo

* activates all the tests

* fixes some comments and todos

* extracts some utils procs for better debugging

* adds todo

* moves calculateMerkleRoot and toMembersipKeyPairs to the rln utils

* makes calls to the utils functions

* adds unit test for createMembershipList

* adds unittest for toMembershipKeyPairs and calcMerkleRoot

* cleans up the code and fixes tree root value

* reverts an unwanted change

* minor

* adds comments and cleans up the code

* updates config message

* adds more comments

* fixes a minor value mismatch

* edits the size of group

* minor rewording

* defines a const var for the group keys

* replaces the sequence literal with the StaticGroupKeys const

* adds a rudimentary unittest

* adds todos

* adds more comment

* replaces uint with MembeshipIndex type

* fixes rln relay mem index config message

* adds rln relay setup proc

* decouples relay and rln-relay

* uses MemIndexType instead of uint

* brings back the rlnRelayEnabled flag to mountRlnRelay

* deletes commented codes

* adds rln relay topic validator inside updates rln relay mounting procedure

* adds rln-relay-pubsub-topic cli option

* adds a static rln-relay topic

* deletes rlnrelayEnabled argument

* adds pubsub topic for rln-relay

* deletes static pubsub topic

* mounts relay before rlnrelay in the tests

* logs rln relay pubsub topic

* cleans up the code

* edits rlnrelay setup

* uninitializes the input parameter of rlnrelay setup

* adds comments

* removes unused comments

* compiles addRLNRelayValidtor when RLN compilation flag is set

* adds comment about topic validator

* minor

* mode modifications on the description of add validator

* adds pubsubtopic field to wakuRlnRelay type

* WIP: shaping the test

* Checks whether rln relay pubsub topic is within the supported topics of relay protocol

* minor

* WIP: unit test for actual proof

* fixes a bug

* removes a redundant proc

* refines the test for actual proof

* breaks lines to 80 chars

* defines NonSpamProof type

* adds a return

* defines Epoch type

* WIP: proof gen

* implements actual proof gen

* adds proto enc and init

* adds notes about proof structure

* adds NonSpamProof to wakumessage

* adds proof gen

* WIP: non working tests for protobuf

* fixes the protobuf encoding issue

* discards the output of copyFrom

* WIP: hash unittest and proofVrfy and ProofGen

* integrates proofVrfy

* uses toBuffer inside the hash proc

* adds comment

* fixes a bug

* removes proof field initialization

* cleans up the test

* generalizes input from byte seq to byte openArray

* adds toBuffer

* adds a bad test

* cleans up unused tests

* adds integration test

* adds comments

* cleans up

* adds description to the integration test

* adds test for unhappy path

* tides up the tests

* tides up hash unit test

* renames a few var

* uses a const for wku rln relay pubsub topic

* minor refinement

* deletes an obsolete comment

* comment revision

* adds comments

* cleans up and adds docstrings

* profGen returns proofRes instead of proof

* removes extra sleepAsync

* fixes two bugs

* returns reject when proof is not verified\

* addresses comments

* adds comments

* links to rln doc

* more comments

* fixes space format

* uncomments v2 tests

* dnsclient branch update

* undo branch update

* minor spacing fix

* makes proof field conditional
2021-10-19 17:37:29 -07:00
Hanno Cornelius 165e235158
Make maximum connections configurable (#731) 2021-10-12 14:48:48 +02:00
Hanno Cornelius 50189a21a9
Update submodules: differentiate between local and remote `PeerInfo` (#730)
* Update submodules

* Remove PeerInfo where necessary
2021-10-06 14:29:08 +02:00
Sanaz Taheri Boshrooyeh 696f41e12a
Rln-Relay: Change tree depth to 20 (#726)
* adds instructions on how to generate parameters for a different tree size

* updates the depth of tree

* modifies the Merkle root

* updates parameters for the new tree size

* fixes failing tests

* updates the comment
2021-09-28 12:03:53 -07:00
Sanaz Taheri Boshrooyeh 645832e9fd
Restructuring Rln-Relay mounting process (#722)
* replaces uint with MembeshipIndex type

* fixes rln relay mem index config message

* adds rln relay setup proc

* decouples relay and rln-relay

* uses MemIndexType instead of uint

* brings back the rlnRelayEnabled flag to mountRlnRelay

* deletes commented codes

* adds rln relay topic validator inside updates rln relay mounting procedure

* adds rln-relay-pubsub-topic cli option

* adds a static rln-relay topic

* deletes rlnrelayEnabled argument

* adds pubsub topic for rln-relay

* deletes static pubsub topic

* mounts relay before rlnrelay in the tests

* logs rln relay pubsub topic

* cleans up the code

* edits rlnrelay setup

* uninitializes the input parameter of rlnrelay setup

* adds comments

* removes unused comments

* compiles addRLNRelayValidtor when RLN compilation flag is set

* adds comment about topic validator

* minor

* mode modifications on the description of add validator

* Checks whether rln relay pubsub topic is within the supported topics of relay protocol

* minor

* addresses comments

membeshipindex to membershipindex,
adds default value for rln topic
adds missing return
2021-09-24 10:32:45 -07:00
Sanaz Taheri Boshrooyeh 4895be61ec
Off-chain group construction and management (#718)
* WIP

* WIP: fixes a bug

* adds test for static group formation

* adds static group creation when rln-relay is enabled

* adds createStatic group

* wip: adds group formation to mount rlnrelay

* adds createMembershipList utility function

* adds doc strings and todos

* cleans up the code and add comments

* defaults createRLNInstance depth argument to 32

* renames Depth

* distinguishes between onchain and offchain modes

* updates index boundaries

* updates log levels

* updates docstring

* updates log level of displayed membership keys

* relocates a todo

* activates all the tests

* fixes some comments and todos

* extracts some utils procs for better debugging

* adds todo

* moves calculateMerkleRoot and toMembersipKeyPairs to the rln utils

* makes calls to the utils functions

* adds unit test for createMembershipList

* adds unittest for toMembershipKeyPairs and calcMerkleRoot

* cleans up the code and fixes tree root value

* reverts an unwanted change

* minor

* adds comments and cleans up the code

* updates config message

* adds more comments

* fixes a minor value mismatch

* edits the size of group

* minor rewording

* defines a const var for the group keys

* replaces the sequence literal with the StaticGroupKeys const

* converts var to let when applicable

* replaces hardcoded value with well-defined constants

* moves createMembershipList to the rln relay utils module

* renames HashSize to HashHexSize

* minor updates on the comments

* reorganizes the consts

* indicates that rlnRelayMemIndex is an experimental option

* fixes a type conversion bug

* revises the unittest of "mount waku rln-relay off-chain"

* clarifies the use of index

* updates a docstring

* removes redundant constants and capitalize all of them

* deletes the ETH_CLIENT const from the test file

* renames a few vars for the sake of clarity

* reorganizes unittest into blocks of execution, debug messages, and checks

* adds more comments

* more comments and clarifications

* cleans up the tests

* minor

* adds a minor fix

* replaces a var usage with let

* fixes a bug
2021-09-17 10:31:25 -07:00
Sanaz Taheri Boshrooyeh b7998de09d
RLN-Relay static group construction (#708)
* tests rln instance as pointer

* test insertion

* tests deletion

* tests proof generation and verification

* updates rln instance type and the rln api

* deletes old API

* removes temporary tests

* deletes unused codes

* Delete settings.json

* reverts the changes in tests v2

* removes an old comment

* adds member insertion and deletion

* adds getMerkleRoot and unit tests

* makes insertMember argument  a value type

* adds static group creation

* adds rln field to WakuRLNRelay type, and enables static group construction inside mountRlnRelay

* renames self to membershipKeyPair and removes key generation from mountRlnRelay

* renames sk,pk tp idKey and idCommitment

* updates mountRlnRelay arguments name

* logs created keys

* uncomments the key generation and adds explainer about it

* enables all the  tests

* adds comments to the rln relay types

* logs error message for the arguments that are not set
2021-08-26 16:14:51 -07:00
Hanno Cornelius 9a45aa7055
Fix: bridge encoding and missing timestamps (#697)
* Fix: bridge should not re-encode pre-encoded payload

* Test payload encoding, fix content topic

* Minor improvements

* Fix broken unit test
2021-08-26 14:24:00 +02:00
Sanaz Taheri Boshrooyeh c222d83bcc
Enables passing RLN object across different function calls (#705)
* tests rln instance as pointer

* test insertion

* tests deletion

* tests proof generation and verification

* updates rln instance type and the rln api

* deletes old API

* removes temporary tests

* deletes unused codes

* Delete settings.json

* reverts the changes in tests v2

* removes an old comment
2021-08-24 12:25:29 -07:00
Hanno Cornelius 4aae5c55ac
Remove unnecessary ENR caching (#700) 2021-08-17 10:30:17 +02:00
Hanno Cornelius 36b9176569
Waku v2 Node Discovery via DNS: Integration (#690) 2021-08-12 10:51:38 +02:00
Hanno Cornelius 6b1e0c3079
Patch v0.5.1 - fix multiple protocol IDs in persistent storage (#687) 2021-07-27 08:48:56 +02:00
Sanaz Taheri Boshrooyeh 71a532a414
Adds timestamp to waku relay message (#681)
* adds timestamp to WakuRelayMessage

* converts float to float64

* generates timestamp using epochTime()

* tides up imports and generates timestamps using epochTime()

* updates changelog

* adds timestamp to the post_waku_v2_relay_v1_message query parameters
2021-07-21 11:05:10 -07:00
Hanno Cornelius b87984af46
Fix topic conversion in bridge (#679) 2021-07-21 14:22:40 +02:00
oskarth 108599a1ed
Whisper tests (#678)
* Remove leftover comment

* Import Whisper tests

* Update copyright to 2018-2021

* Fix whisper import paths

* Whisper tests and Makefile target
2021-07-19 17:54:36 +08:00
oskarth d3368324fe
Move Whisper types and protocol from nim-eth to this repo (#663)
* Move Whisper type and protocol from nim-eth

- Import whisper_protocol and whisper_types
- Change import locations as appropriate

* Explicit errors in wakunode1
2021-07-19 16:14:37 +08:00
Hanno Cornelius 13cf7380bb
General refactoring: `nim-waku` (#671)
* General Track 1 refactoring
2021-07-16 17:13:36 +02:00
Hanno Cornelius 92f2d5a7f5
Refactor wakunode2.nim (#664)
* Refactor wakunode2.nim

* Remove empty raises

* Some formatting improvements
2021-07-14 19:58:46 +02:00
Sanaz Taheri Boshrooyeh 07586153f9
Adding error message to the History Responses (#660)
* adds error field to history response

* unittest for protobuf enc/dec

* renames HistoryResponseError to Store Error and simplifies pagination logic

* reverts the naming of StoreError to HistoryResponseError

* fixes a unittest for invalid cursor

* renames OK to NONE
2021-07-13 12:01:21 -07:00
Hanno Cornelius 2869e71865
Simplify wakunode2 subscriptions model (#662) 2021-07-13 09:18:51 +02:00
Sanaz Taheri Boshrooyeh cf7b8faf27
History response pagination using sender-generated timestamp (#657)
* enables pagination based on sender timestamp

* uncomments a test

* bumps up version number

* updates migration script

* clean up

* unpdates changelog

* undo removal of receiver timestamp

* updates message_storage

* uses epochTime()

* minor

* removes a comment

* removes receiver timestamp deletion migration script

* fixes formatting issues

* fixes a bad field name

* fixes field issue

* adjusts spacing
2021-07-07 16:56:20 -07:00
Sanaz Taheri Boshrooyeh cd703c6c52
Integrating pagination into the resume proc (#653)
* adds queryFromWithPaging

* adds unit test for paged queryFrom

* adds a new resume with paging capability

* unittest for resumePaging

* fixes the bug

* implementes a second version of query with paging

* integrates the second version of resume

* clean up

* adds the error message

* fixes a format issue

* renames variables

* defines DefaultPageSize

* adds a todo

* updates a docstring

* gets the pagesize as proc input

* updates unittest description

* removes unused gcsafe pragma

* better var naming

* updates changelog

* updates changelog

* updates the TODO

* more debug logs
2021-07-02 11:37:58 -07:00
Hanno Cornelius 52b741ac11
Support for stable version of relay protocol (#651) 2021-06-29 16:29:04 +02:00
Sanaz Taheri Boshrooyeh 81b207624b
Add db migration support for peer store (#648)
* moves migration paths to the migration_types module

* moves USER_VERSION to the migration_types module

* updates migration execution condition to cover the peer store

* adds a message to the store if it is inserted to the db

* adds debug messages and checks migration path recursively

* deletes unused debug messages

* deletes commented codes

* clean up

* fixes a bug

* renames the first migration file of the message store
2021-06-25 14:06:56 -07:00
Sanaz Taheri Boshrooyeh 30a7a7a4fe
Filter out duplicate messages in the resume procedure (#600)
* adds timestamp to waku message store impl

* stores timestamp as int64

* adds untitest

* stores timestamp as seq of bytes

* minor

* re-orders unittest

* changes receiver timestamp to float64

* unit test for receiver timestamps

* adds comments

* reorder a few lines

* updates changelog

* more updates on changelog

* WIP: deduplicate

* add debug messages

* more debugging logs

* add timestamp to the message store

* converts resume result to uint64, removes some logs

* docstring for isDuplicate

* increments `added` before insertion into the db

* unit test for deduplication

* removes float to/from Bytes procs

* deletes extra spaces

* returns failed instead of the error

* minor

* minor

* adds changelog
2021-06-22 11:30:12 +08:00
Hanno Cornelius 2f53c2c60b
Feat/bridge topic mapping (#633)
* Use namespaced content topics for bridge
2021-06-18 12:30:24 +02:00
Hanno Cornelius 9ee0c2ecb3
Make bridge topics configurable (#629)
* Make bridge topics configurable

* Less ambiguous config naming
2021-06-17 12:59:28 +02:00
Sanaz Taheri Boshrooyeh ddf93814fe
Persisting Waku message timestamp & implementing DB migration & convert receiver timestamp data type to float64 (#607)
* adds timestamp to waku message store impl

* stores timestamp as int64

* adds untitest

* stores timestamp as seq of bytes

* minor

* re-orders unittest

* changes receiver timestamp to float64

* unit test for receiver timestamps

* adds comments

* reorder a few lines

* updates changelog

* more updates on changelog

* WIP

* WIP

* adds migration

* more debug messages

* passes the path to the migration scripts from message store module

* adds migration result type

* replaces migrationScripts with migrationScriptsResult

* adds path calculation to the message store

* removes some tests binary file

* removes redundant imports

* comments out user_version assignment in sqlite  init

* more descriptive err messages

* clean up test file

* more info logs

* minor code format

* removes a todo

* minor updates

* remove a binary file

* unit tests for migration utils

* adds split script

* integrates split query to handle scripts with multiple commands

* updates migration script for v1

* updates the v1 migration script

* update user version

* updates script

* fixes a few bugs on the splitScript

* more debug logs

* adds float64 parameter support to sqlite3

* change in timestamp type in the script

* deletes float64 toBytes utils

* enables storage of timestamp as a real number in the sqlite db

* bump up script index

* comment edits

* removes migrate unit test

* adds todo and docstring

* updates changelog

* removes an unused item in .gitignore

* minor

* updates changelog

* organizes imports

* cleans up imports

* WIP

* updates script


fixes a few bugs on the splitScript


more debug logs


adds float64 parameter support to sqlite3


change in timestamp type in the script


deletes float64 toBytes utils

* edits migration util test

* remove an empty test file

* includes migration utils tests in

* deletes unused codes

* tides up imports

* adds range based filter to the filterMigrationScripts

* renames procs: removes Migration

* tides up imports

* edits docstring

* edits docstring

* edits docstring

* removes unused imports

* more clean up

* groups std imports

* updates changelog

* adds docstring for setUserVersion

* adds unittest for the migrate

* Update waku/v2/node/storage/message/waku_message_store.nim

Co-authored-by: RichΛrd <info@richardramos.me>

* Update waku/v2/node/storage/sqlite.nim

Co-authored-by: RichΛrd <info@richardramos.me>

* Update waku/v2/node/storage/sqlite.nim

Co-authored-by: RichΛrd <info@richardramos.me>

* removes split scripts

* fixes a naming issue

* fixes a bug

* fixes a typo

* adds a log re updated user_version

* fixes a proc naming mismatch

* fixes naming mismatch

* more descriptive var names

* adds migration script of the first user version

* moves migration to after persistMessages flag is checked

* deletes unused comment

* fixes a bug

* brings back split script

* adds unit tests for split scripts

* runs scripts one command at a time

* deletes a commented line

* relocates the migrate proc to sqlite.nim

* adds unit test for filter scripts

* adds filterScripts unittest testing varying zero-prefixed user versions

* minor

Co-authored-by: RichΛrd <info@richardramos.me>
2021-06-16 13:23:55 -07:00
Hanno Cornelius a044c6a82c
Add ping protocol support and use for keep-alive (#621)
* Add ping protocol support and use for keep-alive

* Separate WakuPeerStore from libp2p PeerStore

* Revert silly test timeout
2021-06-15 10:55:47 +02:00
Ebube Sered Ud 76d4b89dbb
Add mode to waku swap (#601)
* Add mode to waku swap

* Modify swap test  and policy function

* Make rln optional (dependencies and compilation) (#602)

* makes rlnlib target optional

* makes the rln-related targets conditioned to the RLN command line flag

* adds the rln compiler flag condition

* adds the rln compiler flag condition to the rln module

* wakunode2 rln conditional compilation

* updates wakunode2 test

* updates waku rln relay tests

* removing the rln test from the default imports of the v2 tests

* imports rln module conditionally

* removes the rln flag condition from the rln module

* separates rln data types from its procs

* adds the import statement

* brings back the contract def to the rln utils

* adds rln module import to the rln unit tests

* clean up and reorganization

* adds a todo

* minor edits on a comment

* Fix indentation and typo

* Add Swap Config Object and set default config value on mount

* Minor changes

* Fix issues with grammar in code comment

* Create init procedure for setting default values in SwapConfig

* Minor changes to mountSwap call in wakunode2

* Improve Docs on Swap Mode

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2021-06-15 10:06:36 +08:00
Kim De Mey 4ca59bcce0
Use chronosStrictException for Waku v1 code (#609)
* Use chronosStrictException for Waku v1 code

And accompanying changes and clean-up

* Fix chronosStrictException changes for wakubridge
2021-06-14 14:40:08 +02:00
Hanno Cornelius 9e0b73bd18
Update submodules (#594) 2021-06-09 16:37:08 +02:00
Sanaz Taheri Boshrooyeh 2fe53c334c
Make rln optional (dependencies and compilation) (#602)
* makes rlnlib target optional

* makes the rln-related targets conditioned to the RLN command line flag

* adds the rln compiler flag condition

* adds the rln compiler flag condition to the rln module

* wakunode2 rln conditional compilation

* updates wakunode2 test

* updates waku rln relay tests

* removing the rln test from the default imports of the v2 tests

* imports rln module conditionally

* removes the rln flag condition from the rln module

* separates rln data types from its procs

* adds the import statement

* brings back the contract def to the rln utils

* adds rln module import to the rln unit tests

* clean up and reorganization

* adds a todo

* minor edits on a comment
2021-06-08 11:56:32 -07:00
Hanno Cornelius 2b571e205b
Fix keepalive for connected peers (#588)
* Fix keepalive for connected peers

* Remove comment
2021-06-02 09:53:34 +02:00
Sanaz Taheri Boshrooyeh f75f7c4751
Add resume to waku node api (#564)
* activates resume

* adds unit test

* add store condition

* mode debug info

* updates wakunode resume api and its unittest

* adds todo

* adds documentation

* edits resume documentation

* further edits on the docs

* removes a todo

* fixes a bug

* add resume to waku node api

* further updates on the node.md

* updates the changelog

* minor

* removes return type

* adds a brief desc of the resume to node.md
2021-05-26 12:33:22 -07:00
Ebube Sered Ud e6d7e3a2b4
Improve SWAP logging and enable by default (#549)
* Enable Swap by default
* Add PeerId and amount to Logs
* Add Policy function in swap protocol
* Modified Changelog to reflect changes to swap config
* Commented out test for updating account state after a cheque has been sent
2021-05-26 11:05:56 +01:00
Oskar Thorén d99e9251b5
Re-enable lightpush tests and fix Windows CI failure (#572)
* Enable lightpush test again

- Stop nodes differently
- Bump port numbers to possibly get rid of Windows leaked resources

* Lightpush test mount relay w/o relaying messaging
2021-05-25 16:31:59 +08:00
Sanaz Taheri Boshrooyeh c9e94cba41
Enable time-based history query in the store rpc api (#560)
* adds time based query

* updates the changelog
2021-05-21 10:36:24 -07:00
Hanno Cornelius 2691b3e506
Duplicate filtering for bridge (#556) 2021-05-20 18:03:56 +02:00
Sanaz Taheri Boshrooyeh 9b38e5c893
Ft-store: Peer management (#548)
* adds an optional list of peers to the resume proc

WIP working


adds unit tests

* cleans up and adds comments

* adds unittest for queryFrom

* converts queryLoop to a private func

* elaborates on the peer selection of the resume proc

* minor format correction

* returns the status of the resume call as a Result object

* updates unittest based on the new return type

* defines QueryResult type
2021-05-19 12:28:09 -07:00
Sanaz Taheri Boshrooyeh 89909d281f
Fault tolerant store protocol: resume message history (#539)
* adds resume function

* unittest for findLastSeen

* fixes a bug in find last seen unit test

* argument type change to indexed waku message

* relocates findLastSeen

* adds unit test for resume

* adds comments

* adds a TODO

* adds offset

* cleans up

* modifies some of tests title and deletes a duplicate unit test

* more condition on running resume()

* adds more detailed docstring and simplifies if statement

* Update waku/v2/protocol/waku_store/waku_store.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* new doc string

* Update waku/v2/protocol/waku_store/waku_store.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* fixes a bug

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-05-13 14:21:46 -07:00
Sanaz Taheri Boshrooyeh 7c5df3379b
Storing waku message version inside the store db (#530)
* WIP

* stores version in db, edits error messages

* fixes a bug

* reverts edit messages

* converts uint32 storage to int64

* unit tests for various version numbers

* minor comments edit

* removes debugging messages

* updates changelog
2021-05-05 15:51:36 -07:00