status-im
/
go-waku
mirror of https://github.com/status-im/go-waku.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,134 Commits 36 Branches 19 Tags 79 MiB
Commit Graph

868 Commits

Author SHA1 Message Date
richΛrd 48acff4a5c
feat: add warning about bootnodes not supporting shards (#848) 2023-10-30 19:20:13 -04:00
Richard Ramos cf82f66d12
revert(#820): msg digest matches msg hash 2023-10-30 16:25:21 -04:00
richΛrd 4584bb4324
refactor: validate protobuffer for store (#841) 2023-10-30 12:55:36 -04:00
richΛrd 38202e7a2e
refactor: publish API for relay and lightpush (#845) 2023-10-30 12:30:25 -04:00
harsh jain ddf188bbf8
feat: remove named topic (#844) 
* feat: remove named topic

* fix: update examples

* Update library/mobile/api.go
 2023-10-30 21:56:26 +07:00
richΛrd 279752344f
chore: print a message periodically indicating that VACUUM is still being executed (#838) 2023-10-30 09:22:50 -04:00
richΛrd db222a24ef
fix(c-bindings): userdata (#785) 2023-10-28 19:37:53 -04:00
Roman Zajic fc3b2f76d5
chore(filter v2): test updates (#811) 
---------

Co-authored-by: Richard Ramos <info@richardramos.me>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
 2023-10-28 19:27:01 -04:00
harsh jain 0868f5d4dd
feat: add filter v2 rpc (#798) 
* feat: add filter v2 rpc

ping, subscribe/unsubscribe and unsubscribeAll.



* test(filterRest): pingFailure, subscribe-ping, unsubscribe and unsubscribeAll
 2023-10-27 06:21:50 +07:00
richΛrd c58d0f51e4
fix: race condition in peer connector / manager interaction (#837) 2023-10-25 21:25:56 -04:00
harsh jain 0ba8b2caeb
fix(filter-subscribe): params.selectedPeer not set (#836) 
* fix(filter-subscribe): params.selectedPeer not set
 2023-10-26 05:06:44 +07:00
richΛrd 2701a38b2a
refactor: validate protobuffer for filter (#833) 2023-10-25 08:49:25 -04:00
Prem Chaitanya Prathi 18c16de94e
fix: modify store test to not depend on order of msgs (#832) 
* fix: modify store test to not depend on order of msgs

Co-authored-by: richΛrd <info@richardramos.me>

---------

Co-authored-by: richΛrd <info@richardramos.me>
 2023-10-25 03:41:00 +05:30
richΛrd 94f18c537c refactor: validate protobuffers for lightpush and relay (#824) 2023-10-24 17:48:22 -04:00
Richard Ramos fa51d10b4b fix: stream close/reset (#823) 2023-10-24 17:48:22 -04:00
Prem Chaitanya Prathi a3c3aab44e
fix: propagate relay subscribe opt properly (#829) 2023-10-25 02:11:42 +05:30
richΛrd 077df2fbb6
fix: do not disconnect peers in `TestWakuMetadataRequest` (#826) 2023-10-24 16:01:11 -04:00
Richard Ramos 9161c4f7fe fix: stream closing/reset 2023-10-20 20:30:23 -04:00
Richard Ramos 519fa2977a refactor: rename `connOpt` to `stream` 2023-10-20 20:30:23 -04:00
Richard Ramos 19ba25ffcb feat: metadata protocol 2023-10-20 20:30:23 -04:00
richΛrd 4181655b7a
fix: msg digest matches msg hash (#820) 2023-10-20 16:25:54 -04:00
Prem Chaitanya Prathi b5be83a02e
feat : autoshard relay api (#807) 
* fix: using relay without bcaster should consume and drop messages

* update relay api usage

* move subscription to broadcaster

* move filter logic under subscription

* Support more than 1 relay subscription for a pubSubTopic

* modify relay Publish API to derive pubSubTopic based on autosharding

* implement relay RPC methods for autosharding

* remove relay msgChannel and relay on pubsub buffersize for subscription

Co-authored-by: richΛrd <info@richardramos.me>

* handle relay subscribe with noConsumer and address issue reported in code review

* chore: reorg relay code

---------

Co-authored-by: richΛrd <info@richardramos.me>
 2023-10-21 01:26:18 +05:30
Prem Chaitanya Prathi ac9d826b03
fix: handle empty content topics in filter subcribe and unsubscribe (#812) 
* fix: handle empty content topics in filter subcribe and unsubscribe
 2023-10-18 01:23:40 +05:30
richΛrd bfee9964f6
fix: setFallbackIP with reported libp2p addr (#808) 2023-10-17 10:33:50 -04:00
Richard Ramos d4abe15634 fix: addr update 2023-10-17 09:00:30 -04:00
Richard Ramos ee94581d0a feat: prefer circuit relay addr over ws, and update cache with enrs with newer seq number 2023-10-17 08:29:24 -04:00
Richard Ramos 9a0cf85ae1 fix: add ws to enr 2023-10-17 08:29:24 -04:00
Prem Chaitanya Prathi 6955d01498
Update peer selection options for light protocols (#787) 
* Update peer selection options for lightPush

* Update peer selection options for filter

* migrate peer selection functionality from peer manager

Co-authored-by: richΛrd <info@richardramos.me>

---------

Co-authored-by: richΛrd <info@richardramos.me>
 2023-10-16 22:12:01 +05:30
harsh-98 2ef7e732dd nit: remove testing var 2023-10-13 10:35:11 +07:00
harsh-98 b5802adf5b fix: use NewQueries from db utils 2023-10-13 10:35:11 +07:00
harsh-98 5d0692b339 test(store): add fixture for sqlite and postgres 2023-10-13 10:35:11 +07:00
harsh-98 d268b2e403 fix: limit can't be negative for postgres 2023-10-13 10:35:11 +07:00
harsh-98 2f9f304762 test: for postgres and jenkins postgres setup 2023-10-13 10:35:11 +07:00
Richard Ramos 7826e31f14 fix(store): query time comparison and max rows per page 2023-10-12 10:58:55 -04:00
richΛrd 3aa477cbc6
fix: return errors in FilterSubscribeOption (#794) 2023-10-07 16:19:53 -04:00
richΛrd dcc828749f
fix: use https for cloning submodules instead of ssh (#780) 
* fix: use https for cloning submodules instead of ssh
* chore: move to libs/

This simplifies the checkout process for github actions
 2023-09-30 13:57:49 -04:00
Prem Chaitanya Prathi 47c961dcbb
feat: update lightpush API for autosharding (#774) 
* feat: update lightpush API to make pubSubTopic optional as per autosharding

* Extract contentFilter and subscriptions out of filter to reuse in relay (#779)

* chore: extract contentFilter outside filter package

* chore: move subscription outside of filter so that it can be modified and reused for relay

* Feat: filter select peer for sharding (#783)

* update selectPeer to support pubsubTopic based selection
 2023-09-29 10:43:25 +05:30
Prem Chaitanya Prathi dfd104dbac
Chore: test cov improvement (#784) 
* add few tests to increase coverage

* consider other packages coverage while running unit tests
 2023-09-29 04:40:43 +05:30
Anton Iakimov 7f466c1d99
update wakuv2 fleet DNS discovery enrtree (#775) 
https://github.com/status-im/infra-misc/issues/171
 2023-09-28 08:34:30 -04:00
Prem Chaitanya Prathi 388f56b43f
feat: Sharded peer management - Relay (#764) 
* feat: connect/disconnect to peers based on node topic sub/unsub

* feat: maintain healty relay connections per pubSubTopic

Co-authored-by: richΛrd <info@richardramos.me>

* chore: add config to limit peerstore capacity (#770)
 2023-09-27 12:16:37 +05:30
richΛrd d324234c81
fix(filter2): add requestID to pings and remove unneeded log (#776) 2023-09-26 12:27:29 -04:00
Vitaliy Vlasov 16ec22596e feat: change UnsubscribeWithSubscription so that it's single sub-specific 
Also merge FilterSubscribe and FilterUnsubscribe options/params
 2023-09-22 17:53:33 +03:00
Vitaliy Vlasov e0ba66791d fix: only allow adding unique topics 2023-09-22 16:47:36 +03:00
Prem Chaitanya Prathi 9f4754dcae
fix: panic during removePubSubTopic (#765) 2023-09-22 17:36:45 +05:30
richΛrd d317b294a0
fix: panic when removing pubsub topic (#759) 2023-09-21 15:03:19 +05:30
richΛrd 003c90fba8
feat: force unreachability (#753) 
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
 2023-09-20 12:24:16 +05:30
Prem Chaitanya Prathi 054bdae1de
Feat/autoshard filter (#723) 
* feat: update filter client to support autosharding

* chore: add filter tests for autoshard

* chore:update filter API docs for autosharding

* chore: docs changes to indicate sharding impact on pubSubTopic

* fix: handle partial errors during subscribe and return failed content-topic details
 2023-09-20 11:26:55 +05:30
Vitaliy Vlasov 81638fe111
Use PubsubTopic naming; enforce unique ContentTopics (#750) 2023-09-19 18:22:11 +05:30
harsh jain 3d8d435502
test(store): make queries (#752) 
* test(store): make queries

* test: most recent timestamp and count of msg

* nit: add comment for pageSize+1
 2023-09-19 13:28:11 +07:00
Prem Chaitanya Prathi 9b05d48318
Feat : handle dynamic peer topic sub unsub (#751) 
* feat: handle dynamic peer join and leave a pubSub topic

Co-authored-by: richΛrd <info@richardramos.me>


---------

Co-authored-by: richΛrd <info@richardramos.me>
 2023-09-19 11:35:29 +05:30
harsh jain a650469fae
feat: use CommonService in peerConnector (#737) 
* feat(CommonService): add channel and use commonService in discv5

* fix: add mutex to PushToChan

* fix: remove generic functionality

* feat: use CommonService in peerConnector

* fix: remove generic functionality

* nit: add error log
 2023-09-19 07:39:39 +07:00
Prem Chaitanya Prathi dd5dc7a9c8
fix: change mutex to rwLock and fix relay unsubscribe (#749) 
* fix: change mutex to rwLock and fix relay unsubscribe

* chore: modify relay test to cover unsubscribe and few more relay functions
 2023-09-18 18:48:16 +05:30
harsh jain a5f9ee5ad8
feat(CommonService): add channel and use commonService in discv5 (#735) 
* feat(CommonService): add channel and use commonService in discv5

* fix: add mutex to PushToChan

* fix: remove generic functionality
 2023-09-18 16:41:40 +07:00
Richard Ramos 2060c8c837 feat: allow running rln with no credentials 2023-09-15 10:18:44 -04:00
Richard Ramos 22b097817e feat(rln-relay): use zerokit 0.3.4 to obtain number of leaves set in merkle tree 2023-09-15 10:13:02 -04:00
Richard Ramos e1a84aab0e chore(rln-relay): docs and docker 2023-09-14 14:30:48 -04:00
Prem Chaitanya Prathi bfc3083fb1
feat: update WakuPeerStore to store pubSubTopics for a peer (#734) 
* feat: update peerStore to store pubSubTopics a peer is linked to

* derive pubSubTopics from ENR

* New API to retrieve peers by shard information
 2023-09-14 20:30:06 +05:30
Richard Ramos 55bc21c604 feat(rln-relay): isReady 2023-09-14 10:47:27 -04:00
Richard Ramos 7beaa3f029 feat(rln-relay): ensure execution order for pubsub validators 2023-09-14 09:38:28 -04:00
Richard Ramos ab7e45c735 feat: make RLN available by default 2023-09-14 09:03:18 -04:00
Prem Chaitanya Prathi 8df69279ee
chore:remove duplicate declaration of relay protocol const (#736) 2023-09-14 14:06:08 +05:30
harsh jain 4b1c188cf0
feat: add common protocol design (#724) 
* feat: add common protocol design

* fix: remove redundant vars

* fix: use AppDesign's ctx

* refactor: relay, add AppDesign

* feat: changes for suggestions

* test: commonService start/stop execution

* fix: lint error

* nit: add comments
 2023-09-13 12:18:44 +07:00
Richard Ramos 2aea2f5ca5 fix: time calculation for fetching events 2023-09-12 12:09:30 -04:00
Richard Ramos 75486f8226 chore(rln-relay): add logs for fetching events 2023-09-12 11:22:07 -04:00
Richard Ramos dcc87cf24f chore(lint): adds `make lint-full` target and fixes some linting errors 2023-09-12 09:54:34 -04:00
Prem Chaitanya Prathi db25b307e2
Fix/filterv2 (#722) 
* fix: updated filterv2 protocol as per rfc, make pubsub topic optional

* chore: make broadcaster optional in filter client

* reverting optional pubSub topic in Filter.MessagePush

* fix: revert pubSubTopic as optional and add higher level validation

* Update waku/v2/protocol/filter/client.go

Co-authored-by: richΛrd <info@richardramos.me>

---------

Co-authored-by: richΛrd <info@richardramos.me>
 2023-09-12 18:04:43 +05:30
Richard Ramos db3a2d53b3 fix: log enr only after it has been setup 2023-09-08 17:37:23 -04:00
Richard Ramos cc28267951 feat: append RLN proofs when posting messages in REST/RPC 2023-09-08 14:42:33 -04:00
Richard Ramos 18efd2c737 chore(rln-relay): clean up nullifier table every MaxEpochGap 2023-09-08 14:13:12 -04:00
Richard Ramos 793c059ea7 fix: loop counter for writing multiaddresses in enr 2023-09-08 13:45:11 -04:00
Richard Ramos be982d8617 chore: add tls/ws to address factory 2023-09-08 12:25:14 -04:00
Richard Ramos 8783cd2f45 chore(rln-relay): use the only key from keystore if only 1 exists 2023-09-08 12:05:32 -04:00
harsh jain 08cabab41f
fix: and optimising fetching membership events (#706) 
* fix: and optimising fetching membership events

* fix: start from lastProcessedBlock+1

* test: fetching membership logic

* refactor: usage of rlnInstance,rootTracker,groupManager

rlnInstance, rootTrack were previously created while creating rlnRelay
but were assigned to groupManager on Start of rlnRelay. This created
unncessary dependency of passing them to static and dynamic group
manager.
Web3Config uses interface EthClientI for client, so that we can pass
mock client for testing MembershipFetcher.

* fix: failing test

* fix: lint error

* fix: account for PR suggestions

* fix: failing race test

* fix: dont' increase fromBlock on error

* nit: fix naming and add comments
 2023-09-07 23:23:48 +07:00
Prem Chaitanya Prathi 092811658e
fix: changes as per review comments in #714 and fix test (#717) 2023-09-07 15:01:31 +05:30
Siddarth Kumar 7badb4a37b
Fix panic in peer manager (#714) 
* fix: slice error for index out of bounds

this commit fixes slicing `notConnectedPeers` array with 0 to  a negative value of `numPeersToConnect` by first checking if `numPeersToConnect` are greater than 0
 2023-09-07 12:06:43 +05:30
Richard Ramos def745cfba chore(rln-relay): confirm that the provided credential is correct using onchain query 2023-09-06 10:17:46 -04:00
Prem Chaitanya Prathi d13b1f0aa3
autosharding content-topic config (#696) 
* chore: add shard choice simulation test

* feat: add new flags for pubsub and contentTopics and deprecate topic flag

* chore: remove store-resume-peer config and comment out functionality until redesign of store is done

* chore: fix code to use contentTopics value

* fix: use default waku topic only if no other topics are provided in the config
 2023-09-06 10:07:21 +05:30
Richard Ramos f9179cd116 feat: store and retrieve valid merkle roots in RLN db 2023-09-04 10:51:40 -04:00
Richard Ramos f9ed8d973c fix(rln-relay): sync from deployed block number 2023-09-04 10:30:37 -04:00
Richard Ramos 84fa332e1c fix: allow mixing named and static shards 2023-09-04 10:10:49 -04:00
Richard Ramos 5fcfbb9897 feat: use rln registry contract 2023-09-04 10:02:14 -04:00
Richard Ramos ab3f21f209 chore: compile smart contracts and generate types 2023-09-04 10:02:14 -04:00
richΛrd 8aa1c4a39b
refactor(filter): unsubscribe waitgroup, execute async, and guard against calling functions while the protocol is not started (#692) 
* refactor(filter): unsubscribe waitgroup and async
* refactor: verify started state for doing filter operations
 2023-09-04 09:53:51 -04:00
Richard Ramos e8bd38a023 fix: add back to peerstore dns4 addresses removed by go-libp2p identify 2023-09-01 16:53:07 -04:00
Richard Ramos 9c0bebc859 refactor: use a map to store credentials instead of an array 2023-08-31 15:42:45 -04:00
Richard Ramos 0854edaf3d refactor: keystore as a data type 2023-08-31 15:42:45 -04:00
harsh jain 229fb7a970
refactor: peerManager test (#683) 
* refactor: peerManager test

* fix: break down and fix peerManager test

* refactor: codeClimate linting for peerManager/ConnectionGater
 2023-08-30 21:33:57 +07:00
harsh jain accd9ff3e3
feat: add serviceSlot ds (#684) 
* feat: add serviceSlot ds

* test: service slot

* nit: codeClimate related changs

* nit: based on comments

* nit: dont' run getPeers for WakuRelayIDv200
 2023-08-30 18:57:22 +07:00
harsh jain 4a546d12d3
fix: dont send to subscriptions that don't have any ContentTopic (#695) 2023-08-30 17:35:08 +07:00
richΛrd 4c52149fac
chore: log succesful message pushes (#694) 
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
 2023-08-30 10:48:06 +05:30
Andrea Maria Piana 8ad08d6b04
Check nils when deliting subscription (#691) 2023-08-29 12:53:39 +01:00
Prem Chaitanya Prathi 67d0b77147
chore: fix frequent logging from poll (#688) 2023-08-28 14:05:59 +05:30
harsh jain 467d1b2ca5
refactor: peerConnector (#665) 
* refactor: peerConnector

* fix: code climate and dont waitOn subscriptions PeerData

* fix: check in peerConnector is on outRelay connections

* fix: introduced bug in peerConnector
 2023-08-28 13:47:48 +07:00
richΛrd 09eb8ed19b
fix(discv5): threadsafe peerCh (#687) 2023-08-28 11:15:26 +05:30
Richard Ramos 041dc4070a fix(filterV2): requestID and log request type 2023-08-25 13:05:04 -04:00
harsh jain 8b73eb8ae3
refactor(WakuPeerStore): nit origin (#685) 2023-08-25 19:36:06 +04:00
harsh jain bfbc50eb22
fix: missed passing protocols to addPeer (#682) 2023-08-25 14:42:10 +04:00
Prem Chaitanya Prathi 44d3ef6d78
feat: autosharding core logic (#669) 
* feat: autosharding core logic
 2023-08-25 09:55:38 +05:30
Richard Ramos cb3f5da322 feat: rln metrics 2023-08-24 11:26:37 -04:00
Richard Ramos ddb08adbbd chore: set limit to 5000 to not choke eth1 node and start rln before relay 2023-08-24 10:48:22 -04:00
Richard Ramos 624fe1bb2f chore(rln): run rln in all relay pubsubtopics + remove cli flags 2023-08-24 10:48:22 -04:00
Richard Ramos 5422af8130 fix(rln-relay): RLN DB should be aware of chain and contract address 2023-08-24 10:38:59 -04:00
Richard Ramos 7e36f91a5a fix(rln-relay): flush_interval incorrectly set 2023-08-24 10:38:28 -04:00
Richard Ramos f62ba67a92 chore: update smart contracts and documentation 2023-08-24 10:37:54 -04:00
Richard Ramos 0b943caaa8 refactor: remove credential registering from waku 2023-08-24 10:37:54 -04:00
Richard Ramos f088e49075 feat: create `generate-rln-credentials` subcommand 2023-08-24 10:37:54 -04:00
Richard Ramos 8cc92dfdef chore(rln-relay): rename keystore application to `waku-rln-relay` 2023-08-23 15:38:05 -04:00
Prem Chaitanya Prathi bc06867cc9
chore: utils tests (#661) 
* chore: add tests for utils

* chore:delete unused code

* fix: ipv6 validation issue
 2023-08-22 19:18:43 +05:30
Prem Chaitanya Prathi 8a9c4d68e3
feat: register for local node's reachability change (#651) 
* feat:Register for reachability change and log status

Co-authored-by: richΛrd <info@richardramos.me>

---------

Co-authored-by: richΛrd <info@richardramos.me>
 2023-08-20 18:06:35 +05:30
Richard Ramos 2818c3d394 refactor: add libp2p metrics, and make each protocol responsable for defining its own metrics 2023-08-18 13:56:43 -04:00
richΛrd 387ee5f9ac
feat: merkle tree persistence 
- use atomic operations API
- resume onchain sync from persisted tree db
- close eth client and db connection appropriately
- pass in the path to the tree db
- fix nwaku compatibility issues
 2023-08-18 09:59:37 -04:00
Prem Chaitanya Prathi 287d7a014e
fix: fix relay peer calculation (#650) 2023-08-17 18:26:20 +05:30
Prem Chaitanya Prathi f263be4a74
fix: race condition in peermgt initialization reported by race detector (#646) 2023-08-16 17:55:58 +05:30
Prem Chaitanya Prathi 06f027b1a9
feat: implement relay connectivity loop (#642) 
* feat: implement relay conenctivity loop

* chore: fix codeclimate issues

* Apply suggestions from code review

Co-authored-by: richΛrd <info@richardramos.me>

* chore:address review comments

---------

Co-authored-by: richΛrd <info@richardramos.me>
 2023-08-15 06:57:51 +05:30
Vitaliy Vlasov 419adcb6a8
chore(filter2): test updates 2023-08-14 16:29:00 -04:00
Richard Ramos c320b38cbe feat(rln-relay): use atomic operations API 2023-08-12 07:44:07 -04:00
Richard Ramos da45dab68e
fix: nwaku / go-waku rlnKeystore compat 2023-08-11 11:54:24 -04:00
Richard Ramos 9e52e09dd5 chore: keepAlive will not immediatly disconnect peers when waking up from sleep, but do it only if pinging the peer fails 2023-08-10 11:16:02 -04:00
Richard Ramos e0e4a2fa87 refactor: remove unused function and simplify code related to creating db and migrations 2023-08-10 10:37:22 -04:00
Richard Ramos e56f54252f fix: postgresql VACUUM FULL is the equivalent to sqlite VACUUM 2023-08-10 10:29:40 -04:00
Richard Ramos 05d8cfdc65 feat: postgresql vacuum 2023-08-10 10:29:40 -04:00
Richard Ramos ee2af4646c feat: sqlite vacuum and optional migrations 2023-08-10 10:29:40 -04:00
Prem Chaitanya Prathi 9f45d271ac
feat: support serviceslots in peermanager (#631) 
* feat: support peermanager serviceslots and update store protocol to use serviceslots

* fix: lint errors in test code

* fix: error in nix build due to vendor sha change

* fix: set host in peermanager even if relay is disabled

* chore: fix codeclimate issues

* chore: using common filterPeer function to avoid duplication

* feat:use service slots in other service protocols

* chore: fix codeclimate issues

* chore: move AddPeer to peermanager

* Apply suggestions from code review

Co-authored-by: richΛrd <info@richardramos.me>

* chore:address review comments

* feat: implement RemovePeer #638

* chore: fix test failure

* Support for multiple slots for service peers
Adding discovered peers also moved to peer manager

---------

Co-authored-by: richΛrd <info@richardramos.me>
 2023-08-10 18:28:22 +05:30
Prem Chaitanya Prathi af7471d158
fix:during unsubscribeAll, remove peer without any checks 2023-08-07 15:01:44 +05:30
Prem Chaitanya Prathi baeaa1dd55
fix: data race in peer connector 2023-08-07 14:42:48 +05:30
Prem Chaitanya Prathi 20040f2e9b
fix: data race in filter unsubscribe 2023-08-05 08:03:19 +05:30
Prem Chaitanya Prathi 22398b2868
fix:race in filter peer subscription map iteration and deletion 2023-08-05 08:00:04 +05:30
Prem Chaitanya Prathi 884f65ecdf
fix: noise protocol channel close race condition 2023-08-04 17:19:53 +05:30
Prem Chaitanya Prathi 42b2c0bc6d
fix: race condition in rendezvous test code 2023-08-04 16:37:45 +05:30
Prem Chaitanya Prathi 34de2941c7
Feat/peer manager (#596) 
* chore: refactor existing code into peer maanger package

* feat: move peer connection related code into peer manager

* feat: in relay peer connection pruning

* feat: add max-connections CLI flag and limit outRelayPeers based on max-connections #621

* tested both in and out relay connection limits

Co-authored-by: richΛrd <info@richardramos.me>

* Review comment, use context to pause connectivity loop during node shutdown.

Co-authored-by: richΛrd <info@richardramos.me>

* address review comments


---------

Co-authored-by: richΛrd <info@richardramos.me>
 2023-08-03 21:51:15 +05:30
Richard Ramos 33344c2ae0 refactor: use uint instead of int for RLN indexes 2023-08-03 11:17:50 -04:00
Richard Ramos a4b038730c feat(rln-relay): pass in index to keystore credentials 2023-08-03 11:17:50 -04:00
Prem Chaitanya Prathi e6aaf9c852
chore: fix comment on test 2023-08-02 11:24:50 +05:30
Richard Ramos 55ec2347a8 fix: RLN 
- subscription to topic is not required for registering RLN validator
- check for duplication msg within same epoch was incorrect
- added an example on how to use RLN
 2023-08-01 09:19:38 -04:00
Richard Ramos 8b3f42310d fix: code review 2023-08-01 09:17:11 -04:00
Richard Ramos 76961f2bd8 fix: code review 2023-08-01 09:17:11 -04:00
Richard Ramos 1abd476d3e refactor: use backoff strategy to connect to random rendezvous points 2023-08-01 09:17:11 -04:00
Richard Ramos 88d52d6417 fix: set concurrent dials to 5 2023-07-31 12:01:05 -04:00
Richard Ramos dc08c44804 feat: add filters for discv5 2023-07-31 10:45:45 -04:00
Richard Ramos ad9236fd6b refactor: move RPC and REST servers to waku cmd 2023-07-28 08:03:00 -04:00
Richard Ramos a422add209 chore: add missing comments to functions 2023-07-23 07:01:42 -04:00
Richard Ramos bc6a305759 chore: extract `EncapsulatePeerID` 2023-07-07 12:04:05 -04:00
Richard Ramos cfe28d4698 fix: add artificial delay to discv5 iterator 2023-07-07 12:04:05 -04:00
Richard Ramos a124c48b4e fix: add peerID to multiaddr 2023-07-07 12:04:05 -04:00
Richard Ramos 55a7d7b332 fix: reduce timeout and increase minimum backoff time 2023-07-07 12:04:05 -04:00
Richard Ramos 6d9018cb11 refactor: connect to discovered peers linearly 2023-07-07 12:04:05 -04:00
Richard Ramos dbd94ebb81 fix: add p2p-circuit component to multiaddresses that use circuit-relay 2023-07-07 12:04:05 -04:00
Richard Ramos 8cac406a4c fix: warning message format when using static shards and named shards at same time 2023-07-07 10:35:12 -04:00
Richard Ramos ca297713fa refactor: move app code to cmd/waku 2023-07-07 10:10:21 -04:00
Richard Ramos b26859fc6a refactor: make discovery connector subscribe to discovery services 2023-07-07 09:47:46 -04:00
Richard Ramos 5ca26ef897 fix: allow wakunodes to be restarted without error 2023-07-07 09:47:46 -04:00
Prem Chaitanya Prathi 97f02361d4
Persistence: Addressing code-climate issues to increase maintainability score (#592) 
* chore:fix minor issues reported by code-climate

* chore: reduce code duplication

* chore: update vendorSHA for nix build due to recent updates to dependencies
 2023-07-07 07:08:23 +05:30
Richard Ramos 6ece3c483b feat: subscribe to shards with --topic and refactors 
- Shards in same cluster are advertised on ENR
- Store().Resume() was moved to app layer
- NoDefaultWakuTopic() was removed since it's the app that must determine whether it subscribes to the default waku topic
- Removed `Publish` from WakuNode (not really used and easy to implement in app layer if needed)
 2023-07-06 11:18:29 -04:00
Richard Ramos 7dd02067f8 fix: unsubscribe from all peers unless specified and check err code when full node has no subscription 2023-07-05 15:54:31 -04:00
Richard Ramos 0822fdb280 chore: use pgx instead of lib/pq since it's not actively maintained 2023-07-03 11:16:05 -04:00
Richard Ramos fb3447f3d8
fix: adding relay protocol to feature check function of gossipsub 2023-06-29 09:29:08 -04:00
Richard Ramos b8ab99ba08 fix: set wakurelay at index 0 in gossipsub protocol list 2023-06-28 14:06:16 -04:00
Richard Ramos cd358c7bd6 refactor: rendezvous 2023-06-28 09:01:41 -04:00
Richard Ramos cd335f8168 fix: handle scenarios where localnode has no shard set 2023-06-26 11:03:16 -04:00
Richard Ramos eba4aa43e5 feat: find discv5 peers with shards 2023-06-26 11:03:16 -04:00
Richard Ramos 0381b92531 feat(c-bindings): filterv2 2023-06-26 11:03:01 -04:00
Vitaliy Vlasov f0f3543df8 Modify isOnline setting when LightMode is on and Relay is off 2023-06-22 15:38:40 -04:00
Richard Ramos be766a8636 feat(discv5): find peers with predicate 2023-06-15 08:54:40 -04:00
Richard Ramos c380faa6ae fix: code review and disconnect bad peers 2023-06-09 14:57:43 -04:00
Richard Ramos 875d132c1e feat: gossipsub scoring 2023-06-09 14:57:43 -04:00
Richard Ramos 76186e5477 refactor: store ENR 2023-06-09 14:57:43 -04:00
Richard Ramos 52ac8e3740 refactor: various 
- Limit inbound connections to 10 per IP
- Expose gossipsub parameters on WakuRelay
- New peerstore
 2023-06-09 14:57:43 -04:00
Richard Ramos b0c094b0b6 feat: more utils functions to setup a relay shards field on the enr 2023-06-05 16:03:14 -04:00
Richard Ramos 1bffa35dfa fix: enodeToMultiaddress when using ipv6 
Fixes #560
 2023-06-01 14:02:18 -04:00
Richard Ramos 1c75c89790 refactor: remove unneeded `Connect` as dns4 addresses are being resolved now 2023-06-01 14:02:08 -04:00
Richard Ramos f18f219086 fix(rendezvous) - interop with nwaku 2023-06-01 13:31:21 -04:00
Richard Ramos 9207dbcb4a
fix: addPeers 2023-06-01 08:26:03 -04:00
Richard Ramos 10c2e20910
fix: signatures 2023-05-26 11:18:00 -04:00
Richard Ramos 2c17e20d99
fix: neither encoding nor address comparison are needed to protect a topic 2023-05-26 10:42:25 -04:00
Richard Ramos 39c97c5d89 feat: set resource limits 2023-05-24 11:46:32 -04:00
Richard Ramos 189b2ed120
revert: "refactor: use an address instead of public key for the node setup" 
This reverts commit 38a9fc4b19.
 2023-05-22 17:03:40 -04:00
Richard Ramos b66c19ad8f
fix: set autorelay interval to 2 seconds 2023-05-18 12:10:37 -04:00
frank 898f433d2c
fix: dead loop after calling WakuNode.Stop() 2023-05-18 11:22:19 -04:00
Richard Ramos 8d6b2cd721 refactor: add `--ext-ip` and `--dns4-domain-name` values to the list of node multiaddressess 2023-05-15 17:39:58 -04:00
Richard Ramos 30b9fac6a7
fix: dns > ext ip 2023-05-15 11:51:46 -04:00
Richard Ramos 9594e54d36 feat: use circuit relay in service node 2023-05-14 13:10:56 -04:00
Richard Ramos ceed9c7d59
feat: v0.6.0 2023-05-12 12:27:14 -04:00
Richard Ramos 25562d6240 feat: add peer connection notif channel and allow dialing peers with the peer.AddrInfo 2023-05-12 09:09:08 -04:00
Richard Ramos 38a9fc4b19 refactor: use an address instead of public key for the node setup 2023-05-12 09:09:08 -04:00
Richard Ramos aed730c634 feat: use addresses in signed topic validator 2023-05-12 09:09:08 -04:00
Richard Ramos fa61e58d3b fix: do not block ctx.Done() on peer discovery 2023-05-10 08:09:44 -04:00
Richard Ramos e14c55c263 refactor: enr builder 2023-05-09 17:30:26 -04:00
Richard Ramos d9a12bf079 fix: gossipsub parameters, and cancel() execution in wakunode2 2023-05-09 16:42:24 -04:00
Richard Ramos 279524f100 feat(filter): get subscription list 2023-05-08 17:37:15 -04:00
Richard Ramos ac75160817
fix: use 5s timeout in discv5 iterator 2023-05-08 12:29:18 -04:00
Richard Ramos 9b7ad40b06 feat: ext-ip 2023-05-08 11:59:46 -04:00
harsh-98 38741caca0 fix: constrainted env add wait to TestRelaySubscription 2023-05-08 08:31:19 -04:00
harsh-98 bf8c856bc3 fix: use buffered channel as needed by legacy_filter, filter 2023-05-08 08:31:19 -04:00
harsh-98 128999b763 fix: make test work without buffered channel from relay.Subcribe 2023-05-08 08:31:19 -04:00
harsh-98 60edf95c48 fix: broadcaster shouldn't block if channels are not listened to 2023-05-08 08:31:19 -04:00
harsh-98 210597f7e0 fix: to test for new Broadcaster 2023-05-08 08:31:19 -04:00
harsh-98 31c8035589 feat: safe broadcaster 2023-05-08 08:31:19 -04:00
harsh-98 02fdf916d8 refactor: sync.atomic has Boolean type 2023-05-08 08:27:44 -04:00
harsh-98 e391fe6a2f refactor(discv5): lock-free via atomics 2023-05-08 08:27:44 -04:00
Richard Ramos 46500b0de9 fix: handle duplicate filter id removal 2023-05-05 13:52:58 -04:00
Richard Ramos 231469b441 feat: validate message timestamp in signed topic validator 2023-05-05 13:52:51 -04:00
harsh-98 f94b071a84 nit: remove redundant field 2023-05-05 11:41:10 -04:00
harsh-98 e6d9f6b1d8 nit: change to simpleLRU 2023-05-05 11:41:10 -04:00
harsh-98 c890b1fee8 refactor(peerExchange): use lru for storing peers 2023-05-05 11:41:10 -04:00
Richard Ramos c15f4ada56 feat: add message size and published/received time to debug log 2023-05-04 16:22:11 -04:00
harsh-98 caeba39b21 fix: only last discoveryUrl set discoveredNodes 2023-05-04 13:27:54 -04:00
Richard Ramos 8711a08df6 fix: code review 2023-05-04 13:25:45 -04:00
Richard Ramos f11b82d94a fix: use deterministic signatures RFC6979 2023-05-04 13:25:45 -04:00
Richard Ramos 837a0f2708 chore: add unit test for signed validator and --protected-topic-flag 2023-05-04 13:25:45 -04:00
Richard Ramos 88ecf728a3 fix: rename `gowaku_` metrics to `waku_` 2023-04-27 09:58:06 -04:00
Richard Ramos 21a28e9b53 feat: record message size 2023-04-27 09:58:06 -04:00
Richard Ramos fad1a98f77 chore: run onchain tests 2023-04-26 12:40:15 -04:00
Richard Ramos 360a44830a
fix(rln): onchain tests 2023-04-26 12:12:10 -04:00
Richard Ramos 7f276d572f
fix: retrieve membership fee from contract 2023-04-26 11:56:56 -04:00
Richard Ramos 24587e57aa refactor: handle max records exceeded 2023-04-25 10:55:11 -04:00
Richard Ramos 966cbba4c4 feat: utils for handling shards in enr 2023-04-25 10:54:59 -04:00
Richard Ramos c23eac64fb feat(filterv2): ignore messages received from peers not subscribed to 2023-04-25 10:46:50 -04:00
Richard Ramos 9ed8f16a0a feat: add metrics to archive protocol and discv5 2023-04-20 10:04:20 -04:00
Richard Ramos 68b615a87e feat: improve metrics for filter, lightpush, dns discovery and store protocols 2023-04-20 08:57:22 -04:00
Richard Ramos 2c3566377a refactor: inject host on node start 2023-04-17 11:09:05 -04:00
Richard Ramos fbd58df2e8 refactor: initialize broadcaster on Start 2023-04-17 10:19:08 -04:00
Richard Ramos c559932931 chore: update flags 2023-04-17 10:07:15 -04:00
Richard Ramos 0b01e4bb16 refactor: rename filter to legacy filter and set filterv2 as filter 2023-04-17 10:07:15 -04:00
Richard Ramos c3ef173b2c feat: handle chain forks 2023-04-17 10:02:09 -04:00
Richard Ramos 5de3d9f619 feat(rest): store 2023-04-17 10:01:32 -04:00
Richard Ramos c690b6c050 feat: dos protected topic relay msgs based on meta field 2023-04-17 09:56:13 -04:00
Richard Ramos e74f60f173 feat: enable webstransport 2023-04-14 12:34:47 -04:00
Richard Ramos 801ec92c11 refactor: loading credentials and chat2 2023-04-11 10:39:07 -04:00
Richard Ramos 42c0e123d9 refactor: credentials 2023-04-11 10:39:07 -04:00
Richard Ramos 04c90657cd refactor: dynamic rln 2023-04-11 10:39:07 -04:00
harsh-98 6747603a73 refactor: fetch membership events 2023-04-10 11:39:15 -04:00
Richard Ramos 41691a44e5 fix: remove message cache on relay and fix proof validation 2023-04-04 14:20:41 -04:00
Richard Ramos 9c5d1e88b1 refactor: proof generation and merkleroot tracking 2023-04-04 14:20:41 -04:00
Richard Ramos be09f3f550 refactor: static RLN relay 2023-04-04 14:20:41 -04:00
Richard Ramos 6796936f5e chore: upgrade RLN dep 2023-04-04 14:20:41 -04:00
harsh-98 70ddd48274 refactor: setting nodeKey via GOWAKU-NODEKEY env 2023-04-04 06:09:52 -04:00
Richard Ramos 98e14cf0f6
chore: add debug log to identify when peer is discovered 2023-03-31 08:44:01 -04:00
Richard Ramos 7c56ceb139 chore: remove swap protocol 2023-03-30 12:12:41 -04:00
Richard Ramos 56785d8aa7 fix: limit number of content topics per request and timeout message pushes 2023-03-30 12:10:04 -04:00
Richard Ramos dd9261c981 fix: simplify code by using mutex instead of channels 2023-03-30 12:00:19 -04:00
Richard Ramos c5faf8b9e9 fix: race condition deleting items from cache 2023-03-30 12:00:19 -04:00
Richard Ramos 144dfa5b7b feat: add WakuMessage validation in gossipsub 
This also stores the waku message in a cache to avoid having to decode it twice
 2023-03-30 12:00:19 -04:00
Richard Ramos ca20eb4a79 feat: connect to discovered peers 2023-03-29 08:59:36 -04:00
Richard Ramos 2b30726c14 feat: rendezvous client 2023-03-29 08:59:36 -04:00
RichΛrd 8826e2df66
feat: rendezvous server 2023-03-09 11:48:25 -04:00
Richard Ramos 63bb4509bf feat(filterv2): ping 2023-03-08 12:26:50 -04:00
Richard Ramos 4b52983fc4 fix: limit number of subscribers and criteria 2023-03-08 09:51:26 -04:00
Richard Ramos aeddc7848a refactor: namespaced pubsub topics 2023-03-08 09:10:14 -04:00
Richard Ramos 3bba1a86f1 feat(message): added waku message deterministic hashing 2023-03-08 09:08:08 -04:00
Richard Ramos a19a3b5a79 feat: add meta field to WakuMessage 2023-03-04 13:27:36 -04:00