Commit Graph

674 Commits

Author SHA1 Message Date
Prem Chaitanya Prathi 9b05d48318
Feat : handle dynamic peer topic sub unsub (#751)
* feat: handle dynamic peer join and leave a pubSub topic

Co-authored-by: richΛrd <info@richardramos.me>


---------

Co-authored-by: richΛrd <info@richardramos.me>
2023-09-19 11:35:29 +05:30
harsh jain a650469fae
feat: use CommonService in peerConnector (#737)
* feat(CommonService): add channel and use commonService in discv5

* fix: add mutex to PushToChan

* fix: remove generic functionality

* feat: use CommonService in peerConnector

* fix: remove generic functionality

* nit: add error log
2023-09-19 07:39:39 +07:00
Prem Chaitanya Prathi dd5dc7a9c8
fix: change mutex to rwLock and fix relay unsubscribe (#749)
* fix: change mutex to rwLock and fix relay unsubscribe

* chore: modify relay test to cover unsubscribe and few more relay functions
2023-09-18 18:48:16 +05:30
harsh jain a5f9ee5ad8
feat(CommonService): add channel and use commonService in discv5 (#735)
* feat(CommonService): add channel and use commonService in discv5

* fix: add mutex to PushToChan

* fix: remove generic functionality
2023-09-18 16:41:40 +07:00
Richard Ramos 2060c8c837 feat: allow running rln with no credentials 2023-09-15 10:18:44 -04:00
Richard Ramos 22b097817e feat(rln-relay): use zerokit 0.3.4 to obtain number of leaves set in merkle tree 2023-09-15 10:13:02 -04:00
Richard Ramos e1a84aab0e chore(rln-relay): docs and docker 2023-09-14 14:30:48 -04:00
Prem Chaitanya Prathi bfc3083fb1
feat: update WakuPeerStore to store pubSubTopics for a peer (#734)
* feat: update peerStore to store pubSubTopics a peer is linked to

* derive pubSubTopics from ENR

* New API to retrieve peers by shard information
2023-09-14 20:30:06 +05:30
Richard Ramos 55bc21c604 feat(rln-relay): isReady 2023-09-14 10:47:27 -04:00
Richard Ramos 7beaa3f029 feat(rln-relay): ensure execution order for pubsub validators 2023-09-14 09:38:28 -04:00
Richard Ramos ab7e45c735 feat: make RLN available by default 2023-09-14 09:03:18 -04:00
Prem Chaitanya Prathi 8df69279ee
chore:remove duplicate declaration of relay protocol const (#736) 2023-09-14 14:06:08 +05:30
harsh jain 4b1c188cf0
feat: add common protocol design (#724)
* feat: add common protocol design

* fix: remove redundant vars

* fix: use AppDesign's ctx

* refactor: relay, add AppDesign

* feat: changes for suggestions

* test: commonService start/stop execution

* fix: lint error

* nit: add comments
2023-09-13 12:18:44 +07:00
Richard Ramos 2aea2f5ca5 fix: time calculation for fetching events 2023-09-12 12:09:30 -04:00
Richard Ramos 75486f8226 chore(rln-relay): add logs for fetching events 2023-09-12 11:22:07 -04:00
Richard Ramos dcc87cf24f chore(lint): adds `make lint-full` target and fixes some linting errors 2023-09-12 09:54:34 -04:00
Prem Chaitanya Prathi db25b307e2
Fix/filterv2 (#722)
* fix: updated filterv2 protocol as per rfc, make pubsub topic optional

* chore: make broadcaster optional in filter client

* reverting optional pubSub topic in Filter.MessagePush

* fix: revert pubSubTopic as optional and add higher level validation

* Update waku/v2/protocol/filter/client.go

Co-authored-by: richΛrd <info@richardramos.me>

---------

Co-authored-by: richΛrd <info@richardramos.me>
2023-09-12 18:04:43 +05:30
Richard Ramos db3a2d53b3 fix: log enr only after it has been setup 2023-09-08 17:37:23 -04:00
Richard Ramos cc28267951 feat: append RLN proofs when posting messages in REST/RPC 2023-09-08 14:42:33 -04:00
Richard Ramos 18efd2c737 chore(rln-relay): clean up nullifier table every MaxEpochGap 2023-09-08 14:13:12 -04:00
Richard Ramos 793c059ea7 fix: loop counter for writing multiaddresses in enr 2023-09-08 13:45:11 -04:00
Richard Ramos be982d8617 chore: add tls/ws to address factory 2023-09-08 12:25:14 -04:00
Richard Ramos 8783cd2f45 chore(rln-relay): use the only key from keystore if only 1 exists 2023-09-08 12:05:32 -04:00
harsh jain 08cabab41f
fix: and optimising fetching membership events (#706)
* fix: and optimising fetching membership events

* fix: start from lastProcessedBlock+1

* test: fetching membership logic

* refactor: usage of rlnInstance,rootTracker,groupManager

rlnInstance, rootTrack were previously created while creating rlnRelay
but were assigned to groupManager on Start of rlnRelay. This created
unncessary dependency of passing them to static and dynamic group
manager.
Web3Config uses interface EthClientI for client, so that we can pass
mock client for testing MembershipFetcher.

* fix: failing test

* fix: lint error

* fix: account for PR suggestions

* fix: failing race test

* fix: dont' increase fromBlock on error

* nit: fix naming and add comments
2023-09-07 23:23:48 +07:00
Prem Chaitanya Prathi 092811658e
fix: changes as per review comments in #714 and fix test (#717) 2023-09-07 15:01:31 +05:30
Siddarth Kumar 7badb4a37b
Fix panic in peer manager (#714)
* fix: slice error for index out of bounds

this commit fixes slicing `notConnectedPeers` array with 0 to  a negative value of `numPeersToConnect` by first checking if `numPeersToConnect` are greater than 0
2023-09-07 12:06:43 +05:30
Richard Ramos def745cfba chore(rln-relay): confirm that the provided credential is correct using onchain query 2023-09-06 10:17:46 -04:00
Prem Chaitanya Prathi d13b1f0aa3
autosharding content-topic config (#696)
* chore: add shard choice simulation test

* feat: add new flags for pubsub and contentTopics and deprecate topic flag

* chore: remove store-resume-peer config and comment out functionality until redesign of store is done

* chore: fix code to use contentTopics value

* fix: use default waku topic only if no other topics are provided in the config
2023-09-06 10:07:21 +05:30
Richard Ramos f9179cd116 feat: store and retrieve valid merkle roots in RLN db 2023-09-04 10:51:40 -04:00
Richard Ramos f9ed8d973c fix(rln-relay): sync from deployed block number 2023-09-04 10:30:37 -04:00
Richard Ramos 84fa332e1c fix: allow mixing named and static shards 2023-09-04 10:10:49 -04:00
Richard Ramos 5fcfbb9897 feat: use rln registry contract 2023-09-04 10:02:14 -04:00
Richard Ramos ab3f21f209 chore: compile smart contracts and generate types 2023-09-04 10:02:14 -04:00
richΛrd 8aa1c4a39b
refactor(filter): unsubscribe waitgroup, execute async, and guard against calling functions while the protocol is not started (#692)
* refactor(filter): unsubscribe waitgroup and async
* refactor: verify started state for doing filter operations
2023-09-04 09:53:51 -04:00
Richard Ramos e8bd38a023 fix: add back to peerstore dns4 addresses removed by go-libp2p identify 2023-09-01 16:53:07 -04:00
Richard Ramos 9c0bebc859 refactor: use a map to store credentials instead of an array 2023-08-31 15:42:45 -04:00
Richard Ramos 0854edaf3d refactor: keystore as a data type 2023-08-31 15:42:45 -04:00
harsh jain 229fb7a970
refactor: peerManager test (#683)
* refactor: peerManager test

* fix: break down and fix peerManager test

* refactor: codeClimate linting for peerManager/ConnectionGater
2023-08-30 21:33:57 +07:00
harsh jain accd9ff3e3
feat: add serviceSlot ds (#684)
* feat: add serviceSlot ds

* test: service slot

* nit: codeClimate related changs

* nit: based on comments

* nit: dont' run getPeers for WakuRelayIDv200
2023-08-30 18:57:22 +07:00
harsh jain 4a546d12d3
fix: dont send to subscriptions that don't have any ContentTopic (#695) 2023-08-30 17:35:08 +07:00
richΛrd 4c52149fac
chore: log succesful message pushes (#694)
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
2023-08-30 10:48:06 +05:30
Andrea Maria Piana 8ad08d6b04
Check nils when deliting subscription (#691) 2023-08-29 12:53:39 +01:00
Prem Chaitanya Prathi 67d0b77147
chore: fix frequent logging from poll (#688) 2023-08-28 14:05:59 +05:30
harsh jain 467d1b2ca5
refactor: peerConnector (#665)
* refactor: peerConnector

* fix: code climate and dont waitOn subscriptions PeerData

* fix: check in peerConnector is on outRelay connections

* fix: introduced bug in peerConnector
2023-08-28 13:47:48 +07:00
richΛrd 09eb8ed19b
fix(discv5): threadsafe peerCh (#687) 2023-08-28 11:15:26 +05:30
Richard Ramos 041dc4070a fix(filterV2): requestID and log request type 2023-08-25 13:05:04 -04:00
harsh jain 8b73eb8ae3
refactor(WakuPeerStore): nit origin (#685) 2023-08-25 19:36:06 +04:00
harsh jain bfbc50eb22
fix: missed passing protocols to addPeer (#682) 2023-08-25 14:42:10 +04:00
Prem Chaitanya Prathi 44d3ef6d78
feat: autosharding core logic (#669)
* feat: autosharding core logic
2023-08-25 09:55:38 +05:30
Richard Ramos cb3f5da322 feat: rln metrics 2023-08-24 11:26:37 -04:00
Richard Ramos ddb08adbbd chore: set limit to 5000 to not choke eth1 node and start rln before relay 2023-08-24 10:48:22 -04:00
Richard Ramos 624fe1bb2f chore(rln): run rln in all relay pubsubtopics + remove cli flags 2023-08-24 10:48:22 -04:00
Richard Ramos 5422af8130 fix(rln-relay): RLN DB should be aware of chain and contract address 2023-08-24 10:38:59 -04:00
Richard Ramos 7e36f91a5a fix(rln-relay): flush_interval incorrectly set 2023-08-24 10:38:28 -04:00
Richard Ramos f62ba67a92 chore: update smart contracts and documentation 2023-08-24 10:37:54 -04:00
Richard Ramos 0b943caaa8 refactor: remove credential registering from waku 2023-08-24 10:37:54 -04:00
Richard Ramos f088e49075 feat: create `generate-rln-credentials` subcommand 2023-08-24 10:37:54 -04:00
Richard Ramos 8cc92dfdef chore(rln-relay): rename keystore application to `waku-rln-relay` 2023-08-23 15:38:05 -04:00
Prem Chaitanya Prathi bc06867cc9
chore: utils tests (#661)
* chore: add tests for utils

* chore:delete unused code

* fix: ipv6 validation issue
2023-08-22 19:18:43 +05:30
Prem Chaitanya Prathi 8a9c4d68e3
feat: register for local node's reachability change (#651)
* feat:Register for reachability change and log status

Co-authored-by: richΛrd <info@richardramos.me>

---------

Co-authored-by: richΛrd <info@richardramos.me>
2023-08-20 18:06:35 +05:30
Richard Ramos 2818c3d394 refactor: add libp2p metrics, and make each protocol responsable for defining its own metrics 2023-08-18 13:56:43 -04:00
richΛrd 387ee5f9ac
feat: merkle tree persistence
- use atomic operations API
- resume onchain sync from persisted tree db
- close eth client and db connection appropriately
- pass in the path to the tree db
- fix nwaku compatibility issues
2023-08-18 09:59:37 -04:00
Prem Chaitanya Prathi 287d7a014e
fix: fix relay peer calculation (#650) 2023-08-17 18:26:20 +05:30
Prem Chaitanya Prathi f263be4a74
fix: race condition in peermgt initialization reported by race detector (#646) 2023-08-16 17:55:58 +05:30
Prem Chaitanya Prathi 06f027b1a9
feat: implement relay connectivity loop (#642)
* feat: implement relay conenctivity loop

* chore: fix codeclimate issues

* Apply suggestions from code review

Co-authored-by: richΛrd <info@richardramos.me>

* chore:address review comments

---------

Co-authored-by: richΛrd <info@richardramos.me>
2023-08-15 06:57:51 +05:30
Vitaliy Vlasov 419adcb6a8
chore(filter2): test updates 2023-08-14 16:29:00 -04:00
Richard Ramos c320b38cbe feat(rln-relay): use atomic operations API 2023-08-12 07:44:07 -04:00
Richard Ramos da45dab68e
fix: nwaku / go-waku rlnKeystore compat 2023-08-11 11:54:24 -04:00
Richard Ramos 9e52e09dd5 chore: keepAlive will not immediatly disconnect peers when waking up from sleep, but do it only if pinging the peer fails 2023-08-10 11:16:02 -04:00
Richard Ramos e0e4a2fa87 refactor: remove unused function and simplify code related to creating db and migrations 2023-08-10 10:37:22 -04:00
Richard Ramos ee2af4646c feat: sqlite vacuum and optional migrations 2023-08-10 10:29:40 -04:00
Prem Chaitanya Prathi 9f45d271ac
feat: support serviceslots in peermanager (#631)
* feat: support peermanager serviceslots and update store protocol to use serviceslots

* fix: lint errors in test code

* fix: error in nix build due to vendor sha change

* fix: set host in peermanager even if relay is disabled

* chore: fix codeclimate issues

* chore: using common filterPeer function to avoid duplication

* feat:use service slots in other service protocols

* chore: fix codeclimate issues

* chore: move AddPeer to peermanager

* Apply suggestions from code review

Co-authored-by: richΛrd <info@richardramos.me>

* chore:address review comments

* feat: implement RemovePeer #638

* chore: fix test failure

* Support for multiple slots for service peers
Adding discovered peers also moved to peer manager

---------

Co-authored-by: richΛrd <info@richardramos.me>
2023-08-10 18:28:22 +05:30
Prem Chaitanya Prathi af7471d158
fix:during unsubscribeAll, remove peer without any checks 2023-08-07 15:01:44 +05:30
Prem Chaitanya Prathi baeaa1dd55
fix: data race in peer connector 2023-08-07 14:42:48 +05:30
Prem Chaitanya Prathi 20040f2e9b
fix: data race in filter unsubscribe 2023-08-05 08:03:19 +05:30
Prem Chaitanya Prathi 22398b2868
fix:race in filter peer subscription map iteration and deletion 2023-08-05 08:00:04 +05:30
Prem Chaitanya Prathi 884f65ecdf
fix: noise protocol channel close race condition 2023-08-04 17:19:53 +05:30
Prem Chaitanya Prathi 42b2c0bc6d
fix: race condition in rendezvous test code 2023-08-04 16:37:45 +05:30
Prem Chaitanya Prathi 34de2941c7
Feat/peer manager (#596)
* chore: refactor existing code into peer maanger package

* feat: move peer connection related code into peer manager

* feat: in relay peer connection pruning

* feat: add max-connections CLI flag and limit outRelayPeers based on max-connections #621

* tested both in and out relay connection limits

Co-authored-by: richΛrd <info@richardramos.me>

* Review comment, use context to pause connectivity loop during node shutdown.

Co-authored-by: richΛrd <info@richardramos.me>

* address review comments


---------

Co-authored-by: richΛrd <info@richardramos.me>
2023-08-03 21:51:15 +05:30
Richard Ramos 33344c2ae0 refactor: use uint instead of int for RLN indexes 2023-08-03 11:17:50 -04:00
Richard Ramos a4b038730c feat(rln-relay): pass in index to keystore credentials 2023-08-03 11:17:50 -04:00
Prem Chaitanya Prathi e6aaf9c852
chore: fix comment on test 2023-08-02 11:24:50 +05:30
Richard Ramos 55ec2347a8 fix: RLN
- subscription to topic is not required for registering RLN validator
- check for duplication msg within same epoch was incorrect
- added an example on how to use RLN
2023-08-01 09:19:38 -04:00
Richard Ramos 8b3f42310d fix: code review 2023-08-01 09:17:11 -04:00
Richard Ramos 76961f2bd8 fix: code review 2023-08-01 09:17:11 -04:00
Richard Ramos 1abd476d3e refactor: use backoff strategy to connect to random rendezvous points 2023-08-01 09:17:11 -04:00
Richard Ramos 88d52d6417 fix: set concurrent dials to 5 2023-07-31 12:01:05 -04:00
Richard Ramos dc08c44804 feat: add filters for discv5 2023-07-31 10:45:45 -04:00
Richard Ramos ad9236fd6b refactor: move RPC and REST servers to waku cmd 2023-07-28 08:03:00 -04:00
Richard Ramos a422add209 chore: add missing comments to functions 2023-07-23 07:01:42 -04:00
Richard Ramos bc6a305759 chore: extract `EncapsulatePeerID` 2023-07-07 12:04:05 -04:00
Richard Ramos cfe28d4698 fix: add artificial delay to discv5 iterator 2023-07-07 12:04:05 -04:00
Richard Ramos a124c48b4e fix: add peerID to multiaddr 2023-07-07 12:04:05 -04:00
Richard Ramos 55a7d7b332 fix: reduce timeout and increase minimum backoff time 2023-07-07 12:04:05 -04:00
Richard Ramos 6d9018cb11 refactor: connect to discovered peers linearly 2023-07-07 12:04:05 -04:00
Richard Ramos dbd94ebb81 fix: add p2p-circuit component to multiaddresses that use circuit-relay 2023-07-07 12:04:05 -04:00
Richard Ramos 8cac406a4c fix: warning message format when using static shards and named shards at same time 2023-07-07 10:35:12 -04:00
Richard Ramos b26859fc6a refactor: make discovery connector subscribe to discovery services 2023-07-07 09:47:46 -04:00
Richard Ramos 5ca26ef897 fix: allow wakunodes to be restarted without error 2023-07-07 09:47:46 -04:00
Richard Ramos 6ece3c483b feat: subscribe to shards with --topic and refactors
- Shards in same cluster are advertised on ENR
- Store().Resume() was moved to app layer
- NoDefaultWakuTopic() was removed since it's the app that must determine whether it subscribes to the default waku topic
- Removed `Publish` from WakuNode (not really used and easy to implement in app layer if needed)
2023-07-06 11:18:29 -04:00
Richard Ramos 7dd02067f8 fix: unsubscribe from all peers unless specified and check err code when full node has no subscription 2023-07-05 15:54:31 -04:00
Richard Ramos fb3447f3d8
fix: adding relay protocol to feature check function of gossipsub 2023-06-29 09:29:08 -04:00
Richard Ramos b8ab99ba08 fix: set wakurelay at index 0 in gossipsub protocol list 2023-06-28 14:06:16 -04:00
Richard Ramos cd358c7bd6 refactor: rendezvous 2023-06-28 09:01:41 -04:00
Richard Ramos cd335f8168 fix: handle scenarios where localnode has no shard set 2023-06-26 11:03:16 -04:00
Richard Ramos eba4aa43e5 feat: find discv5 peers with shards 2023-06-26 11:03:16 -04:00
Richard Ramos 0381b92531 feat(c-bindings): filterv2 2023-06-26 11:03:01 -04:00
Vitaliy Vlasov f0f3543df8 Modify isOnline setting when LightMode is on and Relay is off 2023-06-22 15:38:40 -04:00
Richard Ramos be766a8636 feat(discv5): find peers with predicate 2023-06-15 08:54:40 -04:00
Richard Ramos c380faa6ae fix: code review and disconnect bad peers 2023-06-09 14:57:43 -04:00
Richard Ramos 875d132c1e feat: gossipsub scoring 2023-06-09 14:57:43 -04:00
Richard Ramos 76186e5477 refactor: store ENR 2023-06-09 14:57:43 -04:00
Richard Ramos 52ac8e3740 refactor: various
- Limit inbound connections to 10 per IP
- Expose gossipsub parameters on WakuRelay
- New peerstore
2023-06-09 14:57:43 -04:00
Richard Ramos b0c094b0b6 feat: more utils functions to setup a relay shards field on the enr 2023-06-05 16:03:14 -04:00
Richard Ramos 1bffa35dfa fix: enodeToMultiaddress when using ipv6
Fixes #560
2023-06-01 14:02:18 -04:00
Richard Ramos 1c75c89790 refactor: remove unneeded `Connect` as dns4 addresses are being resolved now 2023-06-01 14:02:08 -04:00
Richard Ramos f18f219086 fix(rendezvous) - interop with nwaku 2023-06-01 13:31:21 -04:00
Richard Ramos 10c2e20910
fix: signatures 2023-05-26 11:18:00 -04:00
Richard Ramos 2c17e20d99
fix: neither encoding nor address comparison are needed to protect a topic 2023-05-26 10:42:25 -04:00
Richard Ramos 189b2ed120
revert: "refactor: use an address instead of public key for the node setup"
This reverts commit 38a9fc4b19.
2023-05-22 17:03:40 -04:00
Richard Ramos b66c19ad8f
fix: set autorelay interval to 2 seconds 2023-05-18 12:10:37 -04:00
frank 898f433d2c
fix: dead loop after calling WakuNode.Stop() 2023-05-18 11:22:19 -04:00
Richard Ramos 8d6b2cd721 refactor: add `--ext-ip` and `--dns4-domain-name` values to the list of node multiaddressess 2023-05-15 17:39:58 -04:00
Richard Ramos 9594e54d36 feat: use circuit relay in service node 2023-05-14 13:10:56 -04:00
Richard Ramos ceed9c7d59
feat: v0.6.0 2023-05-12 12:27:14 -04:00
Richard Ramos 25562d6240 feat: add peer connection notif channel and allow dialing peers with the peer.AddrInfo 2023-05-12 09:09:08 -04:00
Richard Ramos 38a9fc4b19 refactor: use an address instead of public key for the node setup 2023-05-12 09:09:08 -04:00
Richard Ramos aed730c634 feat: use addresses in signed topic validator 2023-05-12 09:09:08 -04:00
Richard Ramos fa61e58d3b fix: do not block ctx.Done() on peer discovery 2023-05-10 08:09:44 -04:00
Richard Ramos e14c55c263 refactor: enr builder 2023-05-09 17:30:26 -04:00
Richard Ramos d9a12bf079 fix: gossipsub parameters, and cancel() execution in wakunode2 2023-05-09 16:42:24 -04:00
Richard Ramos 279524f100 feat(filter): get subscription list 2023-05-08 17:37:15 -04:00
Richard Ramos ac75160817
fix: use 5s timeout in discv5 iterator 2023-05-08 12:29:18 -04:00
Richard Ramos 9b7ad40b06 feat: ext-ip 2023-05-08 11:59:46 -04:00
harsh-98 38741caca0 fix: constrainted env add wait to TestRelaySubscription 2023-05-08 08:31:19 -04:00
harsh-98 bf8c856bc3 fix: use buffered channel as needed by legacy_filter, filter 2023-05-08 08:31:19 -04:00
harsh-98 128999b763 fix: make test work without buffered channel from relay.Subcribe 2023-05-08 08:31:19 -04:00
harsh-98 60edf95c48 fix: broadcaster shouldn't block if channels are not listened to 2023-05-08 08:31:19 -04:00
harsh-98 210597f7e0 fix: to test for new Broadcaster 2023-05-08 08:31:19 -04:00
harsh-98 31c8035589 feat: safe broadcaster 2023-05-08 08:31:19 -04:00
harsh-98 02fdf916d8 refactor: sync.atomic has Boolean type 2023-05-08 08:27:44 -04:00
harsh-98 e391fe6a2f refactor(discv5): lock-free via atomics 2023-05-08 08:27:44 -04:00
Richard Ramos 46500b0de9 fix: handle duplicate filter id removal 2023-05-05 13:52:58 -04:00
Richard Ramos 231469b441 feat: validate message timestamp in signed topic validator 2023-05-05 13:52:51 -04:00
harsh-98 f94b071a84 nit: remove redundant field 2023-05-05 11:41:10 -04:00
harsh-98 e6d9f6b1d8 nit: change to simpleLRU 2023-05-05 11:41:10 -04:00
harsh-98 c890b1fee8 refactor(peerExchange): use lru for storing peers 2023-05-05 11:41:10 -04:00
Richard Ramos c15f4ada56 feat: add message size and published/received time to debug log 2023-05-04 16:22:11 -04:00
Richard Ramos 8711a08df6 fix: code review 2023-05-04 13:25:45 -04:00
Richard Ramos f11b82d94a fix: use deterministic signatures RFC6979 2023-05-04 13:25:45 -04:00