Nodes returned by DNS Discovery are not guaranteed to be reachable.
Hence, setting an upfront limit based on sought capability does not provide
any guarantees that such capability should be reached.
The discovery's mechanism role is to find Waku 2 nodes. As many as possible.
It is then the role of the peer manager to decide:
- whether to attempt connecting to the node based on advertised capabilities
- retain connection to the node based on actual mounted protocols.
We still want to prevent infinite loops, hence the `maxGet` parameter.
Also, there was a dichotomy between code tested, and code actually used by libp2p peer discovery, now resolved.
# Conflicts:
# packages/discovery/src/dns/constants.ts
These are high level configuration values, that need to be consistent within a given execution. Setting them as default value at such a low level is a footgun risk.
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).
This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.
It also included various back and forth conversions between shards, pubsub topics, etc.
With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.
Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.
# Conflicts:
# packages/core/src/lib/connection_manager/connection_manager.ts
# packages/core/src/lib/metadata/metadata.ts
# packages/interfaces/src/metadata.ts
# packages/interfaces/src/sharding.ts
# packages/relay/src/create.ts
# packages/sdk/src/filter/filter.ts
# packages/sdk/src/filter/types.ts
# packages/sdk/src/light_push/light_push.spec.ts
# packages/tests/tests/sharding/auto_sharding.spec.ts
# packages/tests/tests/sharding/static_sharding.spec.ts
# Conflicts:
# packages/sdk/src/store/store.ts
For an edge node, there is no such thing as a "pubsub topic configuration". An edge node should be able to operate for any possible shard, and it is a per-protocol matter (eg send message with light push).
A relay node do subscribe to shards, but in this case, even metadata protocol does not need to advertise them, this is already handled by gossipsub.
Only service node should advertise their shards via metadata protocol, which is out of scope for js-waku.
# Conflicts:
# packages/interfaces/src/connection_manager.ts
* add FF for auto recovery
* implement connection locking, connection maintenance, auto recovery, bootstrap connections maintenance and fix bootstrap peers dropping
* add ut for peer manager changes
* implement UT for Connection Limiter
* increase connection maintenance interval
* update e2e test
* chore: add ABI for PriceCalculator
* chore: rename LINEA_CONTRACT to RLN_CONTRACT
* chore: add price calculator & test
* fix: import
* chore: convert e2e test to unit
* fix: test
* make dialer use dial queue
* skip undiable peers
* cover new cases in tests
* expose dialer config at connection manager
* update tests with new config
* add more tests
* chore: idCommitmentBigInt validates against contract Q
* chore: fix linting
* chore: add log
* chore: rename Q and make sync
* fix: test
* chore: remove stubbed contract test
* chore: hardcode default constant for Q
* use non deprecated sha256
* chore: use full 32 bytes for bigint
* chore: all storage in LE, but smart contract interactions in BE
* chore: remove references to idCOmmitmentBigInt in Identity
* chore: don't fetch Q from contract
* chore: ByteUtils as a class
* chore: store Identity in BE, convert during Keystore
* chore: add IDCommitmentBigInt part of Identity
* chore: minor improvements
* chore: switch idTrapdoor to LE
* chore: add logs
* chore: rename `DEFAULT_Q` to `RLN_Q`
* chore: rm spec test
* chore: improve modulo logging
* fix(tests): add IDCommitmentBigInt
* implement new peer manager, use in lightPush, improve retry manager and fix retry bug
* fix unsubscribe issue
* remove not needed usage of pubsub, use peer manager in store sdk
* chore: remove deprecated filter implementation
* update tests
* update next filter for new peer manager
* skip IReceiver test, remove unused utility
* remove comment
* fix typo
* remove old connection based peer manager
* update types, export, and edge case for light push
* add retry manager tests
* add new peer manager tests
* refactor tests
* use peer manager events in filter and check for pubsub topic as well
* update test names
* address comments
* unskip Filter e2e test
* address more comments, remove duplication
* skip CI test
* update after merge
* move to peer:idenfity and peer:disconnect events, improve mapping in filter subscriptions
* update tests
* add logs and change peer manager time lock to 10s
* feat: implement shard retrieval for store and improve set store peers usage
* remove log
* remove only, improve condition
* implement smarter way to retrieve peers
* up tests
* update mock
* address nits, add target to eslint, revert to es2022
* remove IBaseProtocol
* fix references, interfaces and integration
* fix ci
* up mock
* up lock
* add mock for local storage
* add missing prop, fix tests
* up lock
