9 Commits

Author SHA1 Message Date
Arseniy Klempner
69406bf1f0
reintroduce and deprecate named sharding (#1751)
Co-authored-by: danisharora099 <danisharora099@gmail.com>
2024-01-10 13:04:30 +05:30
danisharora099
2bc3735e4d
feat: add support for autosharded pubsub topics
tests: use a generator for sharded pubsub topics
set pubsub topic in encoder/decoder based on sharding type
add function for grouping content topics by pubsub topic
add autosharding config to create options
add autoshard rpc endpoints to nwaku and use in tests
set autoshard pubsub topics in all protocols
fix rebase with static sharding
removes unused function
remove console logs
remove autosharding from ShardInfo, add to EncoderOptions
fix enr and encoder/decoder options
test that same application/version hashes to same shard index
update comment on shard field
fix spelling of autosharding
fix content topic protocol in tests
add sharding type alias and function to determine topic in encoders/decoders
move DefaultPubsubTopic from core to interfaces
2023-12-21 10:03:22 -08:00
Danish Arora
4cf2ffefa7
feat!: add support for sharded pubsub topics & remove support for named pubsub topics (#1697)
* merge branches

* tests: use a generator for sharded pubsub topics

* fix namespace edge case

* move shardInfo to pubsubTopic logic in waku.ts

* simplify encoder/decoder creation logic + update tests

* sharding utils: add error handling

* remove redundant test util

* baseprotocol: create abstraction for initialising pubsub topics

* fix: `createDecoder` interface

* filter: createSubscription takes shardInfo instead of pubsubTopicStr

* fix: sharding utils for error handling

* SingleShardInfo: use a new interface instead of omitting and rename namespace

* change redundant namespaces
2023-11-28 15:57:18 +05:30
Danish Arora
0f7d63ef93
feat: Logger with log levels (#1672)
* setup a custom Logger with log level support

* refactor codebase for to use new Logger with log levels

* disallow usage of `debug` directly / only allow usage in/through custom Logger

* remove `debug` from logger
2023-10-20 16:36:47 +05:30
Danish Arora
9fe04d85f3
chore: use pubsubTopic/pubsubTopics across the codebase (#1653)
* chore: change all references of pubSubTopic to pubsubTopic

* change references of pubSubTopics to pubsubTopics

* flag words in cspell
2023-10-16 12:52:32 +05:30
Florin Barbu
347cbfa08a
chore: run tests in parallel (#1655)
* run tests in parallel

* small fixes

* small fixes

* fix setup of nodes

* fix relay tests

* fix Static Sharding: Running Nodes tests

* try with 5 threads

* try with 6 threads

* use startWithRetries as default start

* revert to 6

* set 10 jobs

* revert to back to 6

* add CI info in readme
2023-10-13 12:36:43 +03:00
Florin Barbu
f1d8d6097f
chore(static sharding): add tests for multiple pubsub topics (#1624)
* new tests for static sharding

* store tests

* small fixes after ci run

* small fixes after ci run

* small fixes after ci run

* multiple pubsubtopics on nwaku node
2023-10-02 15:07:00 +05:30
Danish Arora
a3c45b6e1a
feat(static-sharding)!: allow multiple pubSubTopics (#1586)
* `ProtocolCreateOptions` now has `pubSubTopic` as `pubSubTopic[]`

* chore: update encoder & decoder to support `PubSubTopic`

* feat(protocols): allow multiple `PubSubTopic[]`

* feat(relay): allow multiple `PubSubTopic[]`

* chore(tests): update for new API

* chore: minor fixes

* chore: make store more robust

* fix(relay): correctly set types

* chore(address comments): update terminology around configured pubsub topics

* chore(address comments): minor refactoring

* chore(relay): split `subscribe` into smaller functions for readability & modularity

* chore(address comments): refactor `waitForGossipSubPeerInMesh`

* chore(store): only allow to query one `pubSubTopic`

* fix: `store` bug

* feat(tests): add some basic tests

* sharding utils

* address comments

* feat(relay): re-add API for `getMeshPeers`

* update error message

Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>

* refactor for new API

* feat: simplify handling of observers (#1614)

* refactor: simplify handling of observers

* refactor: Remove redundant PubSubTopic from Observer

* use `??` instead of `||`

* update `pubsubTopic` to `pubSubTopic`

* update `interval` typo

* change occurence of `pubsubTopic` to `pubSubTopic`

* relay: rm `getAllMeshPeers` and make `pubSubTopics` public

* relay: use `push_or_init_map` and move to `utils`

* fix: update API for tests

* fix: relay waitForRemotePeer

---------

Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>
2023-09-27 15:28:07 +05:30
fbarbu15
e284c78701
chore: new lightpush tests (#1571)
* new lightpush tests

* fixes after CI run

* split tests into 2 files

* small fix

* address code review comments

* small fix after CI run

---------

Co-authored-by: Danish Arora <35004822+danisharora099@users.noreply.github.com>
2023-09-19 15:51:03 +05:30