Commit Graph

330 Commits

Author SHA1 Message Date
Danish Arora b99f828cfb
chore: minor refactoring for protocols (#1762)
* remove: unnecessary function

* remove: test

* update doc

* add: tests

* revert unintended change

* fix: ping return

* fix(tests): stub connections

* log warning instead of throwing an error
2024-01-11 17:25:47 +05:30
fryorcraken aaa16f3ece
chore: release master (#1721) 2024-01-10 23:27:39 +01:00
Arseniy Klempner 69406bf1f0
reintroduce and deprecate named sharding (#1751)
Co-authored-by: danisharora099 <danisharora099@gmail.com>
2024-01-10 13:04:30 +05:30
Danish Arora c81872ae0e
supress info log for tests (#1763) 2024-01-02 16:30:18 +05:30
Danish Arora 9983549e8b
chore: use connected peers instead of discovered peers for protocols (#1758)
* switch to use connected peers instead of discovered peers for protocols

* switch to use connected peers instead of discovered peers for protocols
2024-01-02 15:49:31 +05:30
danisharora099 2bc3735e4d
feat: add support for autosharded pubsub topics
tests: use a generator for sharded pubsub topics
set pubsub topic in encoder/decoder based on sharding type
add function for grouping content topics by pubsub topic
add autosharding config to create options
add autoshard rpc endpoints to nwaku and use in tests
set autoshard pubsub topics in all protocols
fix rebase with static sharding
removes unused function
remove console logs
remove autosharding from ShardInfo, add to EncoderOptions
fix enr and encoder/decoder options
test that same application/version hashes to same shard index
update comment on shard field
fix spelling of autosharding
fix content topic protocol in tests
add sharding type alias and function to determine topic in encoders/decoders
move DefaultPubsubTopic from core to interfaces
2023-12-21 10:03:22 -08:00
dependabot[bot] 411b76059c
chore(deps)(deps-dev): bump @types/debug from 4.1.10 to 4.1.12 (#1750)
Bumps [@types/debug](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/debug) from 4.1.10 to 4.1.12.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/debug)

---
updated-dependencies:
- dependency-name: "@types/debug"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-06 00:10:49 +03:00
Danish Arora 9ac2a3f363
feat: metadata protocol (#1732)
* add proto

* add rpc and interfaces

* add protocol implementation

* update faulty proto def

* add rpc and interfaces

* refactor implementation & write test

* setup the metadata protocol as a service

* fix cases where metadata service needs to be undefined

* remove redundant catch block

* remove addressed TODO

* update import path

* log errors

* remove redundant code from handling incoming metadata request

* update tests

* add test to check for active connections

* change expects

* save remote peer's shard info after successful connection
2023-12-05 19:26:52 +05:30
dependabot[bot] 66440ae383
chore(deps)(deps-dev): bump rollup from 3.29.2 to 4.6.0 (#1745)
Bumps [rollup](https://github.com/rollup/rollup) from 3.29.2 to 4.6.0.
- [Release notes](https://github.com/rollup/rollup/releases)
- [Changelog](https://github.com/rollup/rollup/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rollup/rollup/compare/v3.29.2...v4.6.0)

---
updated-dependencies:
- dependency-name: rollup
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-30 18:29:12 +05:30
Danish Arora 7ce642c2cc
chore: fix encoder/decoder creation (#1742)
* fix encoder creation

* bump nwaku to v0.22.0 (#1741)
2023-11-29 17:30:56 +05:30
dependabot[bot] b36c3e2bd6
chore(deps)(deps): bump fast-check from 3.13.1 to 3.14.0 (#1739)
Bumps [fast-check](https://github.com/dubzzz/fast-check/tree/HEAD/packages/fast-check) from 3.13.1 to 3.14.0.
- [Release notes](https://github.com/dubzzz/fast-check/releases)
- [Changelog](https://github.com/dubzzz/fast-check/blob/main/packages/fast-check/CHANGELOG.md)
- [Commits](https://github.com/dubzzz/fast-check/commits/v3.14.0/packages/fast-check)

---
updated-dependencies:
- dependency-name: fast-check
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Danish Arora <35004822+danisharora099@users.noreply.github.com>
2023-11-28 17:02:37 +01:00
Danish Arora 4cf2ffefa7
feat!: add support for sharded pubsub topics & remove support for named pubsub topics (#1697)
* merge branches

* tests: use a generator for sharded pubsub topics

* fix namespace edge case

* move shardInfo to pubsubTopic logic in waku.ts

* simplify encoder/decoder creation logic + update tests

* sharding utils: add error handling

* remove redundant test util

* baseprotocol: create abstraction for initialising pubsub topics

* fix: `createDecoder` interface

* filter: createSubscription takes shardInfo instead of pubsubTopicStr

* fix: sharding utils for error handling

* SingleShardInfo: use a new interface instead of omitting and rename namespace

* change redundant namespaces
2023-11-28 15:57:18 +05:30
dependabot[bot] 9ede6eb8cc
chore(deps)(deps-dev): bump @types/uuid from 9.0.6 to 9.0.7 (#1735)
Bumps [@types/uuid](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/uuid) from 9.0.6 to 9.0.7.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/uuid)

---
updated-dependencies:
- dependency-name: "@types/uuid"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-27 19:10:17 +01:00
Arseniy Klempner 1d0e2ace7f
feat: track node connection state (#1719)
Co-authored-by: chair <29414216+chair28980@users.noreply.github.com>
Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
2023-11-27 12:44:49 +01:00
dependabot[bot] affdc265b8
chore(deps)(deps): bump chai and @types/chai (#1731)
Bumps [chai](https://github.com/chaijs/chai) and [@types/chai](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/chai). These dependencies needed to be updated together.

Updates `chai` from 4.3.8 to 4.3.10
- [Release notes](https://github.com/chaijs/chai/releases)
- [Changelog](https://github.com/chaijs/chai/blob/4.x.x/History.md)
- [Commits](https://github.com/chaijs/chai/compare/v4.3.8...v4.3.10)

Updates `@types/chai` from 4.3.6 to 4.3.11
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/chai)

---
updated-dependencies:
- dependency-name: chai
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: "@types/chai"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-27 12:26:31 +01:00
dependabot[bot] 9744417d69
chore(deps)(deps-dev): bump @rollup/plugin-commonjs (#1730)
Bumps [@rollup/plugin-commonjs](https://github.com/rollup/plugins/tree/HEAD/packages/commonjs) from 25.0.4 to 25.0.7.
- [Changelog](https://github.com/rollup/plugins/blob/master/packages/commonjs/CHANGELOG.md)
- [Commits](https://github.com/rollup/plugins/commits/commonjs-v25.0.7/packages/commonjs)

---
updated-dependencies:
- dependency-name: "@rollup/plugin-commonjs"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-27 12:11:15 +01:00
dependabot[bot] 0670c2990d
chore(deps)(deps): bump it-all from 3.0.3 to 3.0.4 (#1727)
Bumps [it-all](https://github.com/achingbrain/it) from 3.0.3 to 3.0.4.
- [Release notes](https://github.com/achingbrain/it/releases)
- [Commits](https://github.com/achingbrain/it/compare/it-all-v3.0.3...it-all-v3.0.4)

---
updated-dependencies:
- dependency-name: it-all
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-24 02:20:49 +03:00
Danish Arora 3166a5135e
chore!: change all instances of `PubSubTopic` to `PubsubTopic` (#1703)
* rename all PubSub patterns

* feat: forbid identifiers with camelcase pubSub (#1709)

---------

Co-authored-by: Arseniy Klempner <adklempner@gmail.com>
2023-11-14 18:52:52 +03:00
dependabot[bot] 535a748ae9
chore(deps)(deps-dev): bump @types/uuid from 9.0.4 to 9.0.6 (#1706)
Bumps [@types/uuid](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/uuid) from 9.0.4 to 9.0.6.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/uuid)

---
updated-dependencies:
- dependency-name: "@types/uuid"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-07 01:17:38 +05:30
fryorcraken 34a25c2f40
chore: release master (#1669) 2023-11-02 23:32:59 +01:00
fryorcraken b7dc3d7576
fix: measure total message size (#1643)
Network message limitations are imposed on the whole message, not just
the payload.

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
2023-10-26 13:14:47 +02:00
Vaclav Pavlin b3864f8772
fix: handle all empty responses in filter (#1688) 2023-10-26 10:46:08 +02:00
Florin Barbu 80a33b9c2b
chore: allure test reporting (#1668)
* allure test reporting

* make reports only for the main job

* fail a test for demo

* fail a test for demo

* revert

* fail a test for demo

* revert

* deploy to allure-jswaku

* use PAT

* add pat as secret

* check if pat exists

* debug secrets issue

* remove debug steps

* debug1

* debug2

* debug3

* debug4

* set reports for all tests

* fix after ci run

* self review

---------

Co-authored-by: Sasha <118575614+weboko@users.noreply.github.com>
2023-10-23 17:53:56 +03:00
Danish Arora 491366bd4f
fix: filter subscription with `pubsubTopic1` and decoder with `pubsubTopic2` (#1675)
* add a test for a failing case

* add error handling & update test

* remove only

* rename test
2023-10-23 14:17:02 +05:30
Danish Arora 81b2bf46ab
allow filtering based on log levels (#1677) 2023-10-23 13:26:35 +05:30
dependabot[bot] f82add16ab
chore(deps)(deps-dev): bump @rollup/plugin-node-resolve (#1641)
Bumps [@rollup/plugin-node-resolve](https://github.com/rollup/plugins/tree/HEAD/packages/node-resolve) from 15.2.1 to 15.2.3.
- [Changelog](https://github.com/rollup/plugins/blob/master/packages/node-resolve/CHANGELOG.md)
- [Commits](https://github.com/rollup/plugins/commits/node-resolve-v15.2.3/packages/node-resolve)

---
updated-dependencies:
- dependency-name: "@rollup/plugin-node-resolve"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Danish Arora <35004822+danisharora099@users.noreply.github.com>
2023-10-23 13:20:20 +05:30
dependabot[bot] 1ca39fce42
chore(deps)(deps-dev): bump @types/debug from 4.1.8 to 4.1.10 (#1673)
Bumps [@types/debug](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/debug) from 4.1.8 to 4.1.10.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/debug)

---
updated-dependencies:
- dependency-name: "@types/debug"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Danish Arora <35004822+danisharora099@users.noreply.github.com>
2023-10-23 13:12:46 +05:30
Danish Arora 1892f5093d
fix: don't dial discovered peers if have already been attempted dial (#1657)
* don't dial peers if they exist in PeerStore already

* fix(tests): bugs & add types to dispatch event

* fix: more tests

* fix: dial validation

* update doc & reduce delay

* refactor test

* use -1 instead of Infinity

* fix comment

* fix rebase
2023-10-20 18:16:48 +05:30
Danish Arora 0f7d63ef93
feat: Logger with log levels (#1672)
* setup a custom Logger with log level support

* refactor codebase for to use new Logger with log levels

* disallow usage of `debug` directly / only allow usage in/through custom Logger

* remove `debug` from logger
2023-10-20 16:36:47 +05:30
fryorcraken 6bad4ea7d1
feat: fail early when trying to send empty payload (#1642)
* refactor: rename as this method does not check empty payloads

* feat: fail early when trying to send empty payload

---------

Co-authored-by: Sasha <oleksandr@status.im>
2023-10-17 00:14:45 +02:00
fryorcraken 0e4ceb6ac9
chore: release master (#1568) 2023-10-16 16:18:34 +02:00
Danish Arora 9fe04d85f3
chore: use `pubsubTopic`/`pubsubTopics` across the codebase (#1653)
* chore: change all references of pubSubTopic to pubsubTopic

* change references of pubSubTopics to pubsubTopics

* flag words in cspell
2023-10-16 12:52:32 +05:30
Danish Arora b8d006bee7
chore: refactor `ConnectionManager` for readability & maintainability (#1658)
* restructure shouldDialPeer for readability & maintainability:

* fix return & add logs

* fix: check on dialPeer instead of attemptDial

* rm: console log
2023-10-13 18:08:37 +05:30
Danish Arora e6527e9ab1
chore: add error handling for an uncaught dial attempt (#1660) 2023-10-13 17:33:42 +05:30
Danish Arora 1d60c4ba44
feat: enable pinging connected peers by default (#1647)
* enable pinging peers by default

* handle ping failure
2023-10-11 18:01:42 +05:30
Danish Arora 124a29ebba
feat(static-sharding): filter peer connections per shards (#1626)
* add interface for `ShardInfo`

* enr: add deserialization logic & setup getters

* add sharding related utils

* utils: add shard<-> bytes conversion helpers

* pass `pubSubTopics` to `Waku`

* add `rs`/`rsv` details during discovery

* connection-manager: discard irrelevant peers

* add tests for static sharding - peer exchange

* update `ConnectionManager` tests to account for topic validity

* add js suffix to import

* address some comments

* move shardInfo encoding to ENR

* test: update for new API

* enr: add tests for serialisation & deserialisation

* address comment

* update test

* move getPeershardInfo to ConnectionManager and return ShardInfo instead of bytes

* update encoding and decoding relay shards to also factor for shards>64

* relay shard encoding decoding: use DataView and verbose spec tests

* improve tests for relay shard encoding decoding

* rm: only

* improve log message for unconfigured pubsub topic

* minor improvement

* fix: buffer <> Uint8array problems with shard decoding

* fix: test

* rm: only
2023-10-10 20:18:02 +05:30
Danish Arora b10c46b910
fix(store)!: use `pubSubTopic` from `DecodedMessage` for `createCursor` (#1640)
* fix!(store): Cursor: use pubsubtopic from Message

* add control to check cursor topic should match decoder

* fix
2023-10-09 21:25:38 +05:30
dependabot[bot] d02838164f
chore(deps)(deps): bump fast-check from 3.13.0 to 3.13.1 (#1629)
Bumps [fast-check](https://github.com/dubzzz/fast-check/tree/HEAD/packages/fast-check) from 3.13.0 to 3.13.1.
- [Release notes](https://github.com/dubzzz/fast-check/releases)
- [Changelog](https://github.com/dubzzz/fast-check/blob/main/packages/fast-check/CHANGELOG.md)
- [Commits](https://github.com/dubzzz/fast-check/commits/v3.13.1/packages/fast-check)

---
updated-dependencies:
- dependency-name: fast-check
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Danish Arora <35004822+danisharora099@users.noreply.github.com>
2023-10-02 22:28:51 +05:30
dependabot[bot] fa6510040c
chore(deps)(deps): bump p-event from 5.0.1 to 6.0.0 (#1610)
Bumps [p-event](https://github.com/sindresorhus/p-event) from 5.0.1 to 6.0.0.
- [Release notes](https://github.com/sindresorhus/p-event/releases)
- [Commits](https://github.com/sindresorhus/p-event/compare/v5.0.1...v6.0.0)

---
updated-dependencies:
- dependency-name: p-event
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Danish Arora <35004822+danisharora099@users.noreply.github.com>
2023-10-02 10:29:45 +05:30
Danish Arora a3c45b6e1a
feat(static-sharding)!: allow multiple pubSubTopics (#1586)
* `ProtocolCreateOptions` now has `pubSubTopic` as `pubSubTopic[]`

* chore: update encoder & decoder to support `PubSubTopic`

* feat(protocols): allow multiple `PubSubTopic[]`

* feat(relay): allow multiple `PubSubTopic[]`

* chore(tests): update for new API

* chore: minor fixes

* chore: make store more robust

* fix(relay): correctly set types

* chore(address comments): update terminology around configured pubsub topics

* chore(address comments): minor refactoring

* chore(relay): split `subscribe` into smaller functions for readability & modularity

* chore(address comments): refactor `waitForGossipSubPeerInMesh`

* chore(store): only allow to query one `pubSubTopic`

* fix: `store` bug

* feat(tests): add some basic tests

* sharding utils

* address comments

* feat(relay): re-add API for `getMeshPeers`

* update error message

Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>

* refactor for new API

* feat: simplify handling of observers (#1614)

* refactor: simplify handling of observers

* refactor: Remove redundant PubSubTopic from Observer

* use `??` instead of `||`

* update `pubsubTopic` to `pubSubTopic`

* update `interval` typo

* change occurence of `pubsubTopic` to `pubSubTopic`

* relay: rm `getAllMeshPeers` and make `pubSubTopics` public

* relay: use `push_or_init_map` and move to `utils`

* fix: update API for tests

* fix: relay waitForRemotePeer

---------

Co-authored-by: fryorcraken <110212804+fryorcraken@users.noreply.github.com>
2023-09-27 15:28:07 +05:30
Sasha d9e4bcbe3f
feat: add Firefox and Webkit to karma (#1598)
* add Firefox and Webkit to karma testing

* try adding install to CI

* add display var

* try container prop

* try FirefoxHeadless

* set env var differently

* apply for all configs

* use root karma config

* fix relative link in karma config

* add log

* return as before

* fix path pattern

* move karma to the root

* try moving tsconfig

* return ts config

* use container for playwright

* add FF for firefox and chrome

* use command for playwright

* add retries

* remove retries
2023-09-22 17:48:07 +02:00
Danish Arora 8f9faf4aed
chore: upgrade engine to 18 (#1604) 2023-09-22 14:37:34 +05:30
Sasha 020463c358
chore: update typescript (#1528)
* update typescript, define in root package.json

* update typedoc and fix typedoc comments

* update dev version of tsconfig

* set module and resolution for tsconfig dev

* bump ts plugins

* set order of typedoc generation

* turn off warnings as errors setting
2023-09-21 10:57:37 +02:00
fryorcraken.eth fb37c89e40
fix: catch top level exception when preemptively creating streams 2023-09-21 12:39:00 +10:00
fryorcraken.eth 053b6545ad
feat!: return `REMOTE_PEER_REJECTED` if remote peer rejected the message 2023-09-21 11:56:02 +10:00
fryorcraken.eth a31b6e472e
refactor: Remove nest of try/catch in favor of sequential try/catch 2023-09-21 11:56:02 +10:00
fryorcraken.eth b696a89572
fix: catch stream creation promise rejection for `lightPush.send` 2023-09-21 11:56:02 +10:00
dependabot[bot] 4bce7295e0
chore(deps)(deps-dev): bump rollup from 3.29.0 to 3.29.2 (#1577)
Bumps [rollup](https://github.com/rollup/rollup) from 3.29.0 to 3.29.2.
- [Release notes](https://github.com/rollup/rollup/releases)
- [Changelog](https://github.com/rollup/rollup/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rollup/rollup/compare/v3.29.0...v3.29.2)

---
updated-dependencies:
- dependency-name: rollup
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 21:33:23 +05:30
dependabot[bot] 05ea276c4f
chore(deps)(deps-dev): bump karma from 6.4.1 to 6.4.2 (#1576)
Bumps [karma](https://github.com/karma-runner/karma) from 6.4.1 to 6.4.2.
- [Release notes](https://github.com/karma-runner/karma/releases)
- [Changelog](https://github.com/karma-runner/karma/blob/master/CHANGELOG.md)
- [Commits](https://github.com/karma-runner/karma/compare/v6.4.1...v6.4.2)

---
updated-dependencies:
- dependency-name: karma
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 21:20:28 +05:30
Sasha d049ebbc34
fix: throw error when no response (#1567) 2023-09-14 13:35:10 +02:00