nwaku/CHANGELOG.md

427 lines
25 KiB
Markdown
Raw Normal View History

2022-03-31 09:16:20 +00:00
## 2021-03-31 v0.9
2022-03-08 10:48:17 +00:00
2022-03-31 09:16:20 +00:00
Release highlights:
- Support for Peer Exchange (PX) when a peer prunes a [`11/WAKU2-RELAY`](https://rfc.vac.dev/spec/11/) mesh due to oversubscription. This can significantly increase mesh stability.
- Improved start-up times through managing the size of the underlying persistent message storage.
- New websocket connections are no longer blocked due to parsing failures in other connections.
The full list of changes is below.
2022-03-08 10:48:17 +00:00
### Features
2022-03-31 09:16:20 +00:00
2022-03-18 21:31:34 +00:00
- Support for bootstrapping [`33/WAKU-DISCV5`](https://rfc.vac.dev/spec/33) via [DNS discovery](https://rfc.vac.dev/spec/10/#discovery-methods)
2022-03-31 09:16:20 +00:00
- Support for GossipSub [Peer Exchange](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#prune-backoff-and-peer-exchange)
- Support for TOML config files via `--config-file=<path/to/config.toml>`
2022-03-18 21:31:34 +00:00
2022-03-08 10:48:17 +00:00
### Changes
- Waku v1 <> v2 bridge now supports DNS `multiaddrs`
- Waku v1 <> v2 bridge now validates content topics before attempting to bridge a message from Waku v2 to Waku v1
2022-03-31 09:16:20 +00:00
- Persistent message storage now auto deletes messages once over specified `--store-capacity`. This can significantly improve node start-up times.
- Renamed Waku v1 <> v2 bridge `make` target and binary to `wakubridge`
- Increased `store` logging to assist with debugging
- Increased `rln-relay` logging to assist with debugging
- Message metrics no longer include the content topic as a dimension to keep Prometheus metric cardinality under control
- Waku v2 `toy-chat` application now sets the sender timestamp when creating messages
- The type of the `proof` field of the `WakuMessage` is changed to `RateLimitProof`
- Added method to the JSON-RPC API that returns the git tag and commit hash of the binary
- The node's ENR is now included in the JSON-RPC API response when requesting node info
2022-03-08 10:48:17 +00:00
### Fixes
2022-03-31 09:16:20 +00:00
- Fixed incorrect conversion of seconds to nanosecond timestamps
- Fixed store queries blocking due to failure in resource clean up
- Fixed underlying issue where new websocket connections are blocked due to parsing failures in other connections
- Fixed failure to log the ENR necessary for a discv5 connection to the node
2022-03-08 10:48:17 +00:00
### Docs
2022-03-31 09:16:20 +00:00
- Added [RAM requirements](https://github.com/status-im/nim-waku/tree/ee96705c7fbe4063b780ac43b7edee2f6c4e351b/waku/v2#wakunode) to `wakunode2` build instructions
- Added [tutorial](https://github.com/status-im/nim-waku/blob/ee96705c7fbe4063b780ac43b7edee2f6c4e351b/docs/tutorial/rln-chat2-live-testnet.md) on communicating with waku2 test fleets via the chat2 `toy-chat` application in spam-protected mode using [`17/WAKU-RLN-RELAY`](https://rfc.vac.dev/spec/17/).
- Added a [section on bug reporting](https://github.com/status-im/nim-waku/blob/ee96705c7fbe4063b780ac43b7edee2f6c4e351b/README.md#bugs-questions--features) to `wakunode2` README
- Fixed broken links in the [JSON-RPC API Tutorial](https://github.com/status-im/nim-waku/blob/5ceef37e15a15c52cbc589f0b366018e81a958ef/docs/tutorial/jsonrpc-api.md)
This release supports the following [libp2p protocols](https://docs.libp2p.io/concepts/protocols/):
| Protocol | Spec status | Protocol id |
| ---: | :---: | :--- |
| [`11/WAKU2-RELAY`](https://rfc.vac.dev/spec/11/) | `stable` | `/vac/waku/relay/2.0.0` |
| [`12/WAKU2-FILTER`](https://rfc.vac.dev/spec/12/) | `draft` | `/vac/waku/filter/2.0.0-beta1` |
| [`13/WAKU2-STORE`](https://rfc.vac.dev/spec/13/) | `draft` | `/vac/waku/store/2.0.0-beta4` |
| [`18/WAKU2-SWAP`](https://rfc.vac.dev/spec/18/) | `draft` | `/vac/waku/swap/2.0.0-beta1` |
| [`19/WAKU2-LIGHTPUSH`](https://rfc.vac.dev/spec/19/) | `draft` | `/vac/waku/lightpush/2.0.0-beta1` |
The Waku v1 implementation is stable but not under active development.
2022-03-08 10:48:17 +00:00
2022-03-03 10:04:34 +00:00
## 2021-03-03 v0.8
Rln-relay integration into chat2 (#835) * adds ProofMetadata * adds EPOCH_INTERVAL * adds messageLog field * adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo * adds unit test for toEpoch and fromEpoch * adds unit test for Epoch comparison * adds result codes for updateLog * adds unit test for update log * renames epoch related consts * modifies updateLog with new return type and new logic of spam detection * adds unit text for the modified updateLog * changes max epoch gap type size * splits updateLog into two procs isSpam and updateLog * updates unittests * fixes a bug, returns false when the message is not spam * renames messageLog to nullifierLog * renames isSpam to hasDuplicate * updates the rln validator, adds comments * adds appendRLNProof proc plus some code beatification * unit test for validate message * adds unhappy test to validateMessage unit test * renames EPOCH_UNIT_SECONDS * renames MAX_CLOCK_GAP_SECONDS * WIP: integration test * fixes compile errors * sets a real epoch value * updates on old unittests * adds comments to the rln relay tests * adds more comments * makes rln import conditional * adds todos * adds more todos * adds rln-relay mount process into chat2 * further todos * logs contentTopic * introduces rln relay configs * changes default pubsub topic * adds contentTopic config * imports rln relay dependencies * consolidates imports * removes module identifier from ContentTopic * adds contentTopic field * adds contentTopic argument to mountRlnRelay calls * appends rln proof to chat2 messages * changes the default chat2 contentTopic * adds missing content topic fields * fixes a bug * adds a new logic about empty content topics * appends proof only when rln flag is active * removes unnecessary todos * fixes an indentation issue * adds log messages * verifies the proof against the concatenation of msg payload and content topic * a bug fix * removes duplicate epoch time calculation * updates log level to trace * updates default rln-relay content topic * adds support for empty content topics * updates changelog * changelog updates * removes a commented code block * updates addRLNRelayValidator string doc
2022-02-04 23:58:27 +00:00
Release highlights:
2022-03-03 10:04:34 +00:00
- Working demonstration and integration of [`17/WAKU-RLN-RELAY`](https://rfc.vac.dev/spec/17/) in the Waku v2 `toy-chat` application
- Beta support for ambient peer discovery using [a version of Discovery v5](https://github.com/vacp2p/rfc/pull/487)
- A fix for the issue that caused a `store` node to run out of memory after serving a number of historical queries
- Ability to configure a `dns4` domain name for a node and resolve other dns-based `multiaddrs`
Rln-relay integration into chat2 (#835) * adds ProofMetadata * adds EPOCH_INTERVAL * adds messageLog field * adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo * adds unit test for toEpoch and fromEpoch * adds unit test for Epoch comparison * adds result codes for updateLog * adds unit test for update log * renames epoch related consts * modifies updateLog with new return type and new logic of spam detection * adds unit text for the modified updateLog * changes max epoch gap type size * splits updateLog into two procs isSpam and updateLog * updates unittests * fixes a bug, returns false when the message is not spam * renames messageLog to nullifierLog * renames isSpam to hasDuplicate * updates the rln validator, adds comments * adds appendRLNProof proc plus some code beatification * unit test for validate message * adds unhappy test to validateMessage unit test * renames EPOCH_UNIT_SECONDS * renames MAX_CLOCK_GAP_SECONDS * WIP: integration test * fixes compile errors * sets a real epoch value * updates on old unittests * adds comments to the rln relay tests * adds more comments * makes rln import conditional * adds todos * adds more todos * adds rln-relay mount process into chat2 * further todos * logs contentTopic * introduces rln relay configs * changes default pubsub topic * adds contentTopic config * imports rln relay dependencies * consolidates imports * removes module identifier from ContentTopic * adds contentTopic field * adds contentTopic argument to mountRlnRelay calls * appends rln proof to chat2 messages * changes the default chat2 contentTopic * adds missing content topic fields * fixes a bug * adds a new logic about empty content topics * appends proof only when rln flag is active * removes unnecessary todos * fixes an indentation issue * adds log messages * verifies the proof against the concatenation of msg payload and content topic * a bug fix * removes duplicate epoch time calculation * updates log level to trace * updates default rln-relay content topic * adds support for empty content topics * updates changelog * changelog updates * removes a commented code block * updates addRLNRelayValidator string doc
2022-02-04 23:58:27 +00:00
The full list of changes is below.
### Features
2022-03-03 10:04:34 +00:00
- [`17/WAKU-RLN-RELAY`](https://rfc.vac.dev/spec/17/) implementation now supports spam-protection for a specific combination of `pubsubTopic` and `contentTopic` (available under the `rln` compiler flag).
- [`17/WAKU-RLN-RELAY`](https://rfc.vac.dev/spec/17/) integrated into chat2 `toy-chat` (available under the `rln` compiler flag)
- Added support for resolving dns-based `multiaddrs`
2022-03-03 10:04:34 +00:00
- A Waku v2 node can now be configured with a domain name and `dns4` `multiaddr`
- Support for ambient peer discovery using [`33/WAKU-DISCV5`](https://github.com/vacp2p/rfc/pull/487)
Rln-relay integration into chat2 (#835) * adds ProofMetadata * adds EPOCH_INTERVAL * adds messageLog field * adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo * adds unit test for toEpoch and fromEpoch * adds unit test for Epoch comparison * adds result codes for updateLog * adds unit test for update log * renames epoch related consts * modifies updateLog with new return type and new logic of spam detection * adds unit text for the modified updateLog * changes max epoch gap type size * splits updateLog into two procs isSpam and updateLog * updates unittests * fixes a bug, returns false when the message is not spam * renames messageLog to nullifierLog * renames isSpam to hasDuplicate * updates the rln validator, adds comments * adds appendRLNProof proc plus some code beatification * unit test for validate message * adds unhappy test to validateMessage unit test * renames EPOCH_UNIT_SECONDS * renames MAX_CLOCK_GAP_SECONDS * WIP: integration test * fixes compile errors * sets a real epoch value * updates on old unittests * adds comments to the rln relay tests * adds more comments * makes rln import conditional * adds todos * adds more todos * adds rln-relay mount process into chat2 * further todos * logs contentTopic * introduces rln relay configs * changes default pubsub topic * adds contentTopic config * imports rln relay dependencies * consolidates imports * removes module identifier from ContentTopic * adds contentTopic field * adds contentTopic argument to mountRlnRelay calls * appends rln proof to chat2 messages * changes the default chat2 contentTopic * adds missing content topic fields * fixes a bug * adds a new logic about empty content topics * appends proof only when rln flag is active * removes unnecessary todos * fixes an indentation issue * adds log messages * verifies the proof against the concatenation of msg payload and content topic * a bug fix * removes duplicate epoch time calculation * updates log level to trace * updates default rln-relay content topic * adds support for empty content topics * updates changelog * changelog updates * removes a commented code block * updates addRLNRelayValidator string doc
2022-02-04 23:58:27 +00:00
### Changes
2022-03-03 10:04:34 +00:00
- Metrics: now monitoring content topics and the sources of new connections
- Metrics: improved default fleet monitoring dashboard
- Introduced a `Timestamp` type (currently an alias for int64).
- All timestamps changed to nanosecond resolution.
2022-03-08 10:48:17 +00:00
- `timestamp` field number in WakuMessage object changed from `4` to `10`
2022-03-03 10:04:34 +00:00
- [`13/WAKU2-STORE`](https://rfc.vac.dev/spec/13/) identifier updated to `/vac/waku/store/2.0.0-beta4`
- `toy-chat` application now uses DNS discovery to connect to existing fleets
Refactoring timestamps (#842) * Refactor timestamps type from float64 to int64 (milliseconds resolution) * Revert epochs to float64 * Update 00002_addSenderTimeStamp.up.sql * Update quicksim2.nim * Add files via upload * Delete 00003_convertTimestampsToInts.up.sql * Add files via upload * Rename 00003_convertTimestampsToInts.up.sql to 00003_addTimestampsToInts.up.sql * Delete 00003_addTimestampsToInts.up.sql * Rln-relay integration into chat2 (#835) * adds ProofMetadata * adds EPOCH_INTERVAL * adds messageLog field * adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo * adds unit test for toEpoch and fromEpoch * adds unit test for Epoch comparison * adds result codes for updateLog * adds unit test for update log * renames epoch related consts * modifies updateLog with new return type and new logic of spam detection * adds unit text for the modified updateLog * changes max epoch gap type size * splits updateLog into two procs isSpam and updateLog * updates unittests * fixes a bug, returns false when the message is not spam * renames messageLog to nullifierLog * renames isSpam to hasDuplicate * updates the rln validator, adds comments * adds appendRLNProof proc plus some code beatification * unit test for validate message * adds unhappy test to validateMessage unit test * renames EPOCH_UNIT_SECONDS * renames MAX_CLOCK_GAP_SECONDS * WIP: integration test * fixes compile errors * sets a real epoch value * updates on old unittests * adds comments to the rln relay tests * adds more comments * makes rln import conditional * adds todos * adds more todos * adds rln-relay mount process into chat2 * further todos * logs contentTopic * introduces rln relay configs * changes default pubsub topic * adds contentTopic config * imports rln relay dependencies * consolidates imports * removes module identifier from ContentTopic * adds contentTopic field * adds contentTopic argument to mountRlnRelay calls * appends rln proof to chat2 messages * changes the default chat2 contentTopic * adds missing content topic fields * fixes a bug * adds a new logic about empty content topics * appends proof only when rln flag is active * removes unnecessary todos * fixes an indentation issue * adds log messages * verifies the proof against the concatenation of msg payload and content topic * a bug fix * removes duplicate epoch time calculation * updates log level to trace * updates default rln-relay content topic * adds support for empty content topics * updates changelog * changelog updates * removes a commented code block * updates addRLNRelayValidator string doc * Squashed commit of the following: commit bc36c99ab202d07baa0a5f0100bd10d1d76fdfa1 Merge: dc2b2946 5a77d6e2 Author: G <28568419+s1fr0@users.noreply.github.com> Date: Sat Feb 5 01:10:06 2022 +0100 Merge branch 'master' into int64-timestamps-ns commit dc2b294667bb5770cc32b93cc560638cf5ce7087 Author: s1fr0 <28568419+s1fr0@users.noreply.github.com> Date: Sat Feb 5 00:24:45 2022 +0100 Fix commit f97b95a036a197938df38a5adaea46fca778016d Author: s1fr0 <28568419+s1fr0@users.noreply.github.com> Date: Sat Feb 5 00:13:18 2022 +0100 Missing import commit 060c4f8d64e1b6e7c0593540fa8fa7f4cadf6df7 Author: s1fr0 <28568419+s1fr0@users.noreply.github.com> Date: Sat Feb 5 00:10:36 2022 +0100 Fixed typo commit 08ca99b6f692d3df6d4c7c2312c7cada05fc0041 Author: s1fr0 <28568419+s1fr0@users.noreply.github.com> Date: Fri Feb 4 23:59:20 2022 +0100 Time util file commit 2b5c360746990936dec256e90d08dae3c3e35a94 Author: s1fr0 <28568419+s1fr0@users.noreply.github.com> Date: Fri Feb 4 23:33:20 2022 +0100 Moved time utility functions to utils/time commit fdaf121f089aa011855303cc8dd1ce52aec506ad Author: s1fr0 <28568419+s1fr0@users.noreply.github.com> Date: Fri Feb 4 23:10:25 2022 +0100 Fix comment commit c7e06ab4e7618d9a3fe8aa744dd48bf3f7d8754c Author: s1fr0 <28568419+s1fr0@users.noreply.github.com> Date: Fri Feb 4 23:04:13 2022 +0100 Restore previous migration script commit 80282db1d79df676255d4b8e6e09d9f8a2b00fd3 Author: s1fr0 <28568419+s1fr0@users.noreply.github.com> Date: Fri Feb 4 22:54:15 2022 +0100 Typo commit b9d67f89b0eea11a8362dbb10b5f9d6894343352 Author: s1fr0 <28568419+s1fr0@users.noreply.github.com> Date: Fri Feb 4 22:49:29 2022 +0100 Added utilities to get int64 nanosecond, microsecond, millisecond time resolution from float commit 0130d496e694a01cfc9eeb90b7cbc77764490bf9 Author: s1fr0 <28568419+s1fr0@users.noreply.github.com> Date: Fri Feb 4 22:36:35 2022 +0100 Switched to nanoseconds support. * Update CHANGELOG.md * Create 00003_convertTimestampsToInt64.up.sql Migration script * Moved migration script to right location * Update waku_rln_relay_utils.nim * Update waku_rln_relay_utils.nim * Addressed reviewers' comments * Update default fleet metrics dashboard (#844) * Fix * No need for float * Aligning master to changes in PR * Further fixes Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com> Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2022-02-17 15:00:15 +00:00
Rln-relay integration into chat2 (#835) * adds ProofMetadata * adds EPOCH_INTERVAL * adds messageLog field * adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo * adds unit test for toEpoch and fromEpoch * adds unit test for Epoch comparison * adds result codes for updateLog * adds unit test for update log * renames epoch related consts * modifies updateLog with new return type and new logic of spam detection * adds unit text for the modified updateLog * changes max epoch gap type size * splits updateLog into two procs isSpam and updateLog * updates unittests * fixes a bug, returns false when the message is not spam * renames messageLog to nullifierLog * renames isSpam to hasDuplicate * updates the rln validator, adds comments * adds appendRLNProof proc plus some code beatification * unit test for validate message * adds unhappy test to validateMessage unit test * renames EPOCH_UNIT_SECONDS * renames MAX_CLOCK_GAP_SECONDS * WIP: integration test * fixes compile errors * sets a real epoch value * updates on old unittests * adds comments to the rln relay tests * adds more comments * makes rln import conditional * adds todos * adds more todos * adds rln-relay mount process into chat2 * further todos * logs contentTopic * introduces rln relay configs * changes default pubsub topic * adds contentTopic config * imports rln relay dependencies * consolidates imports * removes module identifier from ContentTopic * adds contentTopic field * adds contentTopic argument to mountRlnRelay calls * appends rln proof to chat2 messages * changes the default chat2 contentTopic * adds missing content topic fields * fixes a bug * adds a new logic about empty content topics * appends proof only when rln flag is active * removes unnecessary todos * fixes an indentation issue * adds log messages * verifies the proof against the concatenation of msg payload and content topic * a bug fix * removes duplicate epoch time calculation * updates log level to trace * updates default rln-relay content topic * adds support for empty content topics * updates changelog * changelog updates * removes a commented code block * updates addRLNRelayValidator string doc
2022-02-04 23:58:27 +00:00
### Fixes
2022-03-03 10:04:34 +00:00
- Fixed underlying bug that caused occasional failures when reading the certificate for secure websockets
- Fixed `store` memory usage issues when responding to history queries
2022-03-03 10:04:34 +00:00
### Docs
2022-03-03 10:04:34 +00:00
- Documented [use of domain certificates](https://github.com/status-im/nim-waku/tree/2972a5003568848164033da3fe0d7f52a3d54824/waku/v2#enabling-websocket) for secure websockets
- Documented [how to configure a `dns4` domain name](https://github.com/status-im/nim-waku/tree/2972a5003568848164033da3fe0d7f52a3d54824/waku/v2#using-dns-discovery-to-connect-to-existing-nodes) for a node
- Clarified [use of DNS discovery](https://github.com/status-im/nim-waku/tree/2972a5003568848164033da3fe0d7f52a3d54824/waku/v2#using-dns-discovery-to-connect-to-existing-nodes) and provided current URLs for discoverable fleet nodes
- Added [tutorial](https://github.com/status-im/nim-waku/blob/2972a5003568848164033da3fe0d7f52a3d54824/docs/tutorial/rln-chat2-local-test.md) on using [`17/WAKU-RLN-RELAY`](https://rfc.vac.dev/spec/17/) with the chat2 `toy-chat` application
- Added [tutorial](https://github.com/status-im/nim-waku/blob/2972a5003568848164033da3fe0d7f52a3d54824/docs/tutorial/bridge.md) on how to configure and a use a [`15/WAKU-BRIDGE`](https://rfc.vac.dev/spec/15/)
This release supports the following [libp2p protocols](https://docs.libp2p.io/concepts/protocols/):
| Protocol | Spec status | Protocol id |
| ---: | :---: | :--- |
| [`11/WAKU2-RELAY`](https://rfc.vac.dev/spec/11/) | `stable` | `/vac/waku/relay/2.0.0` |
| [`12/WAKU2-FILTER`](https://rfc.vac.dev/spec/12/) | `draft` | `/vac/waku/filter/2.0.0-beta1` |
| [`13/WAKU2-STORE`](https://rfc.vac.dev/spec/13/) | `draft` | `/vac/waku/store/2.0.0-beta4` |
| [`18/WAKU2-SWAP`](https://rfc.vac.dev/spec/18/) | `draft` | `/vac/waku/swap/2.0.0-beta1` |
| [`19/WAKU2-LIGHTPUSH`](https://rfc.vac.dev/spec/19/) | `draft` | `/vac/waku/lightpush/2.0.0-beta1` |
The Waku v1 implementation is stable but not under active development.
Rln-relay integration into chat2 (#835) * adds ProofMetadata * adds EPOCH_INTERVAL * adds messageLog field * adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo * adds unit test for toEpoch and fromEpoch * adds unit test for Epoch comparison * adds result codes for updateLog * adds unit test for update log * renames epoch related consts * modifies updateLog with new return type and new logic of spam detection * adds unit text for the modified updateLog * changes max epoch gap type size * splits updateLog into two procs isSpam and updateLog * updates unittests * fixes a bug, returns false when the message is not spam * renames messageLog to nullifierLog * renames isSpam to hasDuplicate * updates the rln validator, adds comments * adds appendRLNProof proc plus some code beatification * unit test for validate message * adds unhappy test to validateMessage unit test * renames EPOCH_UNIT_SECONDS * renames MAX_CLOCK_GAP_SECONDS * WIP: integration test * fixes compile errors * sets a real epoch value * updates on old unittests * adds comments to the rln relay tests * adds more comments * makes rln import conditional * adds todos * adds more todos * adds rln-relay mount process into chat2 * further todos * logs contentTopic * introduces rln relay configs * changes default pubsub topic * adds contentTopic config * imports rln relay dependencies * consolidates imports * removes module identifier from ContentTopic * adds contentTopic field * adds contentTopic argument to mountRlnRelay calls * appends rln proof to chat2 messages * changes the default chat2 contentTopic * adds missing content topic fields * fixes a bug * adds a new logic about empty content topics * appends proof only when rln flag is active * removes unnecessary todos * fixes an indentation issue * adds log messages * verifies the proof against the concatenation of msg payload and content topic * a bug fix * removes duplicate epoch time calculation * updates log level to trace * updates default rln-relay content topic * adds support for empty content topics * updates changelog * changelog updates * removes a commented code block * updates addRLNRelayValidator string doc
2022-02-04 23:58:27 +00:00
2022-01-18 13:14:41 +00:00
## 2021-01-19 v0.7
2022-01-18 13:14:41 +00:00
Release highlights:
- Support for secure websockets.
- Ability to remove unreachable clients in a `filter` node.
- Several fixes to improve `store` performance and decrease query times. Query time for large stores decreased from longer than 8 min to under 100 ms.
- Fix for a long-standing bug that prevented proper database migration in some deployed Docker containers.
The full list of changes is below.
### Features
2022-01-18 13:14:41 +00:00
- Support for secure websocket transport
### Changes
2022-01-18 13:14:41 +00:00
- Filter nodes can now remove unreachable clients
- The WakuInfo `listenStr` is deprecated and replaced with a sequence of `listenAddresses` to accommodate multiple transports
- Removed cached `peerInfo` on local node. Rely on underlying libp2p switch instead
- Metrics: added counters for protocol messages
2022-01-18 13:14:41 +00:00
- Waku v2 node discovery now supports [`31/WAKU2-ENR`](https://rfc.vac.dev/spec/31/)
- resuming the history via `resume` now takes the answers of all peers in `peerList` into consideration and consolidates them into one deduplicated list
### Fixes
2022-01-18 13:14:41 +00:00
- Fixed database migration failure in the Docker image
- All `HistoryResponse` messages are now auto-paginated to a maximum of 100 messages per response
2022-01-18 13:14:41 +00:00
- Increased maximum length for reading from a libp2p input stream to allow largest possible protocol messages, including `HistoryResponse` messages at max size
- Significantly improved `store` node query performance
- Implemented a GossipSub `MessageIdProvider` for `11/WAKU2-RELAY` messages instead of relying on the unstable default
- Receiver timestamps for message indexing in the `store` now have consistent millisecond resolution
This release supports the following [libp2p protocols](https://docs.libp2p.io/concepts/protocols/):
| Protocol | Spec status | Protocol id |
| ---: | :---: | :--- |
| [`17/WAKU-RLN-RELAY`](https://rfc.vac.dev/spec/17/) | `raw` | `/vac/waku/waku-rln-relay/2.0.0-alpha1` |
| [`11/WAKU2-RELAY`](https://rfc.vac.dev/spec/11/) | `stable` | `/vac/waku/relay/2.0.0` |
| [`12/WAKU2-FILTER`](https://rfc.vac.dev/spec/12/) | `draft` | `/vac/waku/filter/2.0.0-beta1` |
| [`13/WAKU2-STORE`](https://rfc.vac.dev/spec/13/) | `draft` | `/vac/waku/store/2.0.0-beta3` |
| [`18/WAKU2-SWAP`](https://rfc.vac.dev/spec/18/) | `draft` | `/vac/waku/swap/2.0.0-beta1` |
| [`19/WAKU2-LIGHTPUSH`](https://rfc.vac.dev/spec/19/) | `draft` | `/vac/waku/lightpush/2.0.0-beta1` |
The Waku v1 implementation is stable but not under active development.
2021-11-04 15:32:49 +00:00
## 2021-11-05 v0.6
2021-07-23 10:32:46 +00:00
2021-11-04 15:32:49 +00:00
Some useful features and fixes in this release, include:
- two methods for Waku v2 node discovery
- support for unsecure websockets, which paves the way for native browser usage
- a fix for `nim-waku` store nodes running out of memory due to store size: the number of stored messages can now easily be configured
- a fix for densely connected nodes refusing new connections: the maximum number of allowed connections can now easily be configured
- support for larger message sizes (up from 64kb to 1Mb per message)
2021-11-04 15:32:49 +00:00
The full list of changes is below.
### Features
- Waku v2 node discovery via DNS following [EIP-1459](https://eips.ethereum.org/EIPS/eip-1459)
2021-11-01 18:02:39 +00:00
- Waku v2 node discovery via [Node Discovery v5](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md)
### Changes
2021-11-04 15:32:49 +00:00
- Pagination of historical queries are now simplified
- GossipSub [prune backoff period](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#prune-backoff-and-peer-exchange) is now the recommended 1 minute
- Bridge now uses content topic format according to [23/WAKU2-TOPICS](https://rfc.vac.dev/spec/23/)
- Better internal differentiation between local and remote peer info
- Maximum number of libp2p connections is now configurable
2021-11-04 15:32:49 +00:00
- `udp-port` CLI option has been removed for binaries where it's not used
- Waku v2 now supports unsecure WebSockets
- Waku v2 now supports larger message sizes of up to 1 Mb by default
- Further experimental development of [RLN for spam protection](https://rfc.vac.dev/spec/17/).
These changes are disabled by default under a compiler flag. Changes include:
- Per-message rate limit proof defined
- RLN proof generation and verification integrated into Waku v2
- RLN tree depth changed from 32 to 20
- Support added for static membership group formation
#### Docs
2021-11-04 15:32:49 +00:00
- Added [contributor guidelines](https://github.com/status-im/nim-waku/blob/master/docs/contributors/waku-fleets.md) on Waku v2 fleet monitoring and management
- Added [basic tutorial](https://github.com/status-im/nim-waku/blob/master/docs/tutorial/dns-disc.md) on using Waku v2 DNS-based discovery
### Fixes
2021-11-04 15:32:49 +00:00
- Bridge between `toy-chat` and matterbridge now shows correct announced addresses
- Bridge no longer re-encodes already encoded payloads when publishing to V1
- Bridge now populates WakuMessage timestamps when publishing to V2
2021-11-04 15:32:49 +00:00
- Store now has a configurable maximum number of stored messages
- Network simulations for Waku v1 and Waku v2 are runnable again
This release supports the following [libp2p protocols](https://docs.libp2p.io/concepts/protocols/):
| Protocol | Spec status | Protocol id |
| ---: | :---: | :--- |
| [`17/WAKU-RLN`](https://rfc.vac.dev/spec/17/) | `raw` | `/vac/waku/waku-rln-relay/2.0.0-alpha1` |
| [`11/WAKU2-RELAY`](https://rfc.vac.dev/spec/11/) | `stable` | `/vac/waku/relay/2.0.0` |
| [`12/WAKU2-FILTER`](https://rfc.vac.dev/spec/12/) | `draft` | `/vac/waku/filter/2.0.0-beta1` |
| [`13/WAKU2-STORE`](https://rfc.vac.dev/spec/13/) | `draft` | `/vac/waku/store/2.0.0-beta3` |
| [`18/WAKU2-SWAP`](https://rfc.vac.dev/spec/18/) | `draft` | `/vac/waku/swap/2.0.0-beta1` |
| [`19/WAKU2-LIGHTPUSH`](https://rfc.vac.dev/spec/19/) | `draft` | `/vac/waku/lightpush/2.0.0-beta1` |
The Waku v1 implementation is stable but not under active development.
## 2021-07-26 v0.5.1
This patch release contains the following fix:
- Support for multiple protocol IDs when reconnecting to previously connected peers:
A bug in `v0.5` caused clients using persistent peer storage to only support the mounted protocol ID.
This is a patch release that is fully backwards-compatible with release `v0.5`.
It supports the same [libp2p protocols](https://docs.libp2p.io/concepts/protocols/):
| Protocol | Spec status | Protocol id |
| ---: | :---: | :--- |
| [`17/WAKU-RLN`](https://rfc.vac.dev/spec/17/) | `raw` | `/vac/waku/waku-rln-relay/2.0.0-alpha1` |
| [`11/WAKU2-RELAY`](https://rfc.vac.dev/spec/11/) | `stable` | `/vac/waku/relay/2.0.0` |
| [`12/WAKU2-FILTER`](https://rfc.vac.dev/spec/12/) | `draft` | `/vac/waku/filter/2.0.0-beta1` |
| [`13/WAKU2-STORE`](https://rfc.vac.dev/spec/13/) | `draft` | `/vac/waku/store/2.0.0-beta3` |
| [`18/WAKU2-SWAP`](https://rfc.vac.dev/spec/18/) | `draft` | `/vac/waku/swap/2.0.0-beta1` |
| [`19/WAKU2-LIGHTPUSH`](https://rfc.vac.dev/spec/19/) | `draft` | `/vac/waku/lightpush/2.0.0-beta1` |
The Waku v1 implementation is stable but not under active development.
2021-07-23 10:32:46 +00:00
## 2021-07-23 v0.5
Persisting Waku message timestamp & implementing DB migration & convert receiver timestamp data type to float64 (#607) * adds timestamp to waku message store impl * stores timestamp as int64 * adds untitest * stores timestamp as seq of bytes * minor * re-orders unittest * changes receiver timestamp to float64 * unit test for receiver timestamps * adds comments * reorder a few lines * updates changelog * more updates on changelog * WIP * WIP * adds migration * more debug messages * passes the path to the migration scripts from message store module * adds migration result type * replaces migrationScripts with migrationScriptsResult * adds path calculation to the message store * removes some tests binary file * removes redundant imports * comments out user_version assignment in sqlite init * more descriptive err messages * clean up test file * more info logs * minor code format * removes a todo * minor updates * remove a binary file * unit tests for migration utils * adds split script * integrates split query to handle scripts with multiple commands * updates migration script for v1 * updates the v1 migration script * update user version * updates script * fixes a few bugs on the splitScript * more debug logs * adds float64 parameter support to sqlite3 * change in timestamp type in the script * deletes float64 toBytes utils * enables storage of timestamp as a real number in the sqlite db * bump up script index * comment edits * removes migrate unit test * adds todo and docstring * updates changelog * removes an unused item in .gitignore * minor * updates changelog * organizes imports * cleans up imports * WIP * updates script fixes a few bugs on the splitScript more debug logs adds float64 parameter support to sqlite3 change in timestamp type in the script deletes float64 toBytes utils * edits migration util test * remove an empty test file * includes migration utils tests in * deletes unused codes * tides up imports * adds range based filter to the filterMigrationScripts * renames procs: removes Migration * tides up imports * edits docstring * edits docstring * edits docstring * removes unused imports * more clean up * groups std imports * updates changelog * adds docstring for setUserVersion * adds unittest for the migrate * Update waku/v2/node/storage/message/waku_message_store.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * removes split scripts * fixes a naming issue * fixes a bug * fixes a typo * adds a log re updated user_version * fixes a proc naming mismatch * fixes naming mismatch * more descriptive var names * adds migration script of the first user version * moves migration to after persistMessages flag is checked * deletes unused comment * fixes a bug * brings back split script * adds unit tests for split scripts * runs scripts one command at a time * deletes a commented line * relocates the migrate proc to sqlite.nim * adds unit test for filter scripts * adds filterScripts unittest testing varying zero-prefixed user versions * minor Co-authored-by: RichΛrd <info@richardramos.me>
2021-06-16 20:23:55 +00:00
This release contains the following:
### Features
2021-07-23 10:32:46 +00:00
- Support for keep-alives using [libp2p ping protocol](https://docs.libp2p.io/concepts/protocols/#ping).
- DB migration for the message and peer stores.
- Support for multiple protocol IDs. Mounted protocols now match versions of the same protocol that adds a postfix to the stable protocol ID.
Persisting Waku message timestamp & implementing DB migration & convert receiver timestamp data type to float64 (#607) * adds timestamp to waku message store impl * stores timestamp as int64 * adds untitest * stores timestamp as seq of bytes * minor * re-orders unittest * changes receiver timestamp to float64 * unit test for receiver timestamps * adds comments * reorder a few lines * updates changelog * more updates on changelog * WIP * WIP * adds migration * more debug messages * passes the path to the migration scripts from message store module * adds migration result type * replaces migrationScripts with migrationScriptsResult * adds path calculation to the message store * removes some tests binary file * removes redundant imports * comments out user_version assignment in sqlite init * more descriptive err messages * clean up test file * more info logs * minor code format * removes a todo * minor updates * remove a binary file * unit tests for migration utils * adds split script * integrates split query to handle scripts with multiple commands * updates migration script for v1 * updates the v1 migration script * update user version * updates script * fixes a few bugs on the splitScript * more debug logs * adds float64 parameter support to sqlite3 * change in timestamp type in the script * deletes float64 toBytes utils * enables storage of timestamp as a real number in the sqlite db * bump up script index * comment edits * removes migrate unit test * adds todo and docstring * updates changelog * removes an unused item in .gitignore * minor * updates changelog * organizes imports * cleans up imports * WIP * updates script fixes a few bugs on the splitScript more debug logs adds float64 parameter support to sqlite3 change in timestamp type in the script deletes float64 toBytes utils * edits migration util test * remove an empty test file * includes migration utils tests in * deletes unused codes * tides up imports * adds range based filter to the filterMigrationScripts * renames procs: removes Migration * tides up imports * edits docstring * edits docstring * edits docstring * removes unused imports * more clean up * groups std imports * updates changelog * adds docstring for setUserVersion * adds unittest for the migrate * Update waku/v2/node/storage/message/waku_message_store.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * removes split scripts * fixes a naming issue * fixes a bug * fixes a typo * adds a log re updated user_version * fixes a proc naming mismatch * fixes naming mismatch * more descriptive var names * adds migration script of the first user version * moves migration to after persistMessages flag is checked * deletes unused comment * fixes a bug * brings back split script * adds unit tests for split scripts * runs scripts one command at a time * deletes a commented line * relocates the migrate proc to sqlite.nim * adds unit test for filter scripts * adds filterScripts unittest testing varying zero-prefixed user versions * minor Co-authored-by: RichΛrd <info@richardramos.me>
2021-06-16 20:23:55 +00:00
### Changes
2021-07-23 10:32:46 +00:00
- Bridge topics are now configurable.
- The `resume` Nim API now eliminates duplicates messages before storing them.
- The `resume` Nim API now fetches historical messages in page sequence.
- Added support for stable version of `relay` protocol, with protocol ID `/vac/waku/relay/2.0.0`.
- Added optional `timestamp` to `WakuRelayMessage`.
- Removed `PCRE` as a prerequisite for building Waku v1 and Waku v2.
- Improved [`swap`](https://rfc.vac.dev/spec/18/) metrics.
Persisting Waku message timestamp & implementing DB migration & convert receiver timestamp data type to float64 (#607) * adds timestamp to waku message store impl * stores timestamp as int64 * adds untitest * stores timestamp as seq of bytes * minor * re-orders unittest * changes receiver timestamp to float64 * unit test for receiver timestamps * adds comments * reorder a few lines * updates changelog * more updates on changelog * WIP * WIP * adds migration * more debug messages * passes the path to the migration scripts from message store module * adds migration result type * replaces migrationScripts with migrationScriptsResult * adds path calculation to the message store * removes some tests binary file * removes redundant imports * comments out user_version assignment in sqlite init * more descriptive err messages * clean up test file * more info logs * minor code format * removes a todo * minor updates * remove a binary file * unit tests for migration utils * adds split script * integrates split query to handle scripts with multiple commands * updates migration script for v1 * updates the v1 migration script * update user version * updates script * fixes a few bugs on the splitScript * more debug logs * adds float64 parameter support to sqlite3 * change in timestamp type in the script * deletes float64 toBytes utils * enables storage of timestamp as a real number in the sqlite db * bump up script index * comment edits * removes migrate unit test * adds todo and docstring * updates changelog * removes an unused item in .gitignore * minor * updates changelog * organizes imports * cleans up imports * WIP * updates script fixes a few bugs on the splitScript more debug logs adds float64 parameter support to sqlite3 change in timestamp type in the script deletes float64 toBytes utils * edits migration util test * remove an empty test file * includes migration utils tests in * deletes unused codes * tides up imports * adds range based filter to the filterMigrationScripts * renames procs: removes Migration * tides up imports * edits docstring * edits docstring * edits docstring * removes unused imports * more clean up * groups std imports * updates changelog * adds docstring for setUserVersion * adds unittest for the migrate * Update waku/v2/node/storage/message/waku_message_store.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * removes split scripts * fixes a naming issue * fixes a bug * fixes a typo * adds a log re updated user_version * fixes a proc naming mismatch * fixes naming mismatch * more descriptive var names * adds migration script of the first user version * moves migration to after persistMessages flag is checked * deletes unused comment * fixes a bug * brings back split script * adds unit tests for split scripts * runs scripts one command at a time * deletes a commented line * relocates the migrate proc to sqlite.nim * adds unit test for filter scripts * adds filterScripts unittest testing varying zero-prefixed user versions * minor Co-authored-by: RichΛrd <info@richardramos.me>
2021-06-16 20:23:55 +00:00
#### General refactoring
2021-07-23 10:32:46 +00:00
- Refactored modules according to [Nim best practices](https://hackmd.io/1imOGULZRsed2HpgmzGleA).
- Simplified the [way protocols get notified](https://github.com/status-im/nim-waku/issues/574) of new messages.
- Refactored `wakunode2` setup into 6 distinct phases with improved logging and error handling.
- Moved `Whisper` types and protocol from the `nim-eth` module to `nim-waku`.
Persisting Waku message timestamp & implementing DB migration & convert receiver timestamp data type to float64 (#607) * adds timestamp to waku message store impl * stores timestamp as int64 * adds untitest * stores timestamp as seq of bytes * minor * re-orders unittest * changes receiver timestamp to float64 * unit test for receiver timestamps * adds comments * reorder a few lines * updates changelog * more updates on changelog * WIP * WIP * adds migration * more debug messages * passes the path to the migration scripts from message store module * adds migration result type * replaces migrationScripts with migrationScriptsResult * adds path calculation to the message store * removes some tests binary file * removes redundant imports * comments out user_version assignment in sqlite init * more descriptive err messages * clean up test file * more info logs * minor code format * removes a todo * minor updates * remove a binary file * unit tests for migration utils * adds split script * integrates split query to handle scripts with multiple commands * updates migration script for v1 * updates the v1 migration script * update user version * updates script * fixes a few bugs on the splitScript * more debug logs * adds float64 parameter support to sqlite3 * change in timestamp type in the script * deletes float64 toBytes utils * enables storage of timestamp as a real number in the sqlite db * bump up script index * comment edits * removes migrate unit test * adds todo and docstring * updates changelog * removes an unused item in .gitignore * minor * updates changelog * organizes imports * cleans up imports * WIP * updates script fixes a few bugs on the splitScript more debug logs adds float64 parameter support to sqlite3 change in timestamp type in the script deletes float64 toBytes utils * edits migration util test * remove an empty test file * includes migration utils tests in * deletes unused codes * tides up imports * adds range based filter to the filterMigrationScripts * renames procs: removes Migration * tides up imports * edits docstring * edits docstring * edits docstring * removes unused imports * more clean up * groups std imports * updates changelog * adds docstring for setUserVersion * adds unittest for the migrate * Update waku/v2/node/storage/message/waku_message_store.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * removes split scripts * fixes a naming issue * fixes a bug * fixes a typo * adds a log re updated user_version * fixes a proc naming mismatch * fixes naming mismatch * more descriptive var names * adds migration script of the first user version * moves migration to after persistMessages flag is checked * deletes unused comment * fixes a bug * brings back split script * adds unit tests for split scripts * runs scripts one command at a time * deletes a commented line * relocates the migrate proc to sqlite.nim * adds unit test for filter scripts * adds filterScripts unittest testing varying zero-prefixed user versions * minor Co-authored-by: RichΛrd <info@richardramos.me>
2021-06-16 20:23:55 +00:00
#### Docs
2021-07-23 10:32:46 +00:00
- Added [database migration tutorial](https://github.com/status-im/nim-waku/blob/master/docs/tutorial/db-migration.md).
- Added [tutorial to setup `websockify`](https://github.com/status-im/nim-waku/blob/master/docs/tutorial/websocket.md).
Persisting Waku message timestamp & implementing DB migration & convert receiver timestamp data type to float64 (#607) * adds timestamp to waku message store impl * stores timestamp as int64 * adds untitest * stores timestamp as seq of bytes * minor * re-orders unittest * changes receiver timestamp to float64 * unit test for receiver timestamps * adds comments * reorder a few lines * updates changelog * more updates on changelog * WIP * WIP * adds migration * more debug messages * passes the path to the migration scripts from message store module * adds migration result type * replaces migrationScripts with migrationScriptsResult * adds path calculation to the message store * removes some tests binary file * removes redundant imports * comments out user_version assignment in sqlite init * more descriptive err messages * clean up test file * more info logs * minor code format * removes a todo * minor updates * remove a binary file * unit tests for migration utils * adds split script * integrates split query to handle scripts with multiple commands * updates migration script for v1 * updates the v1 migration script * update user version * updates script * fixes a few bugs on the splitScript * more debug logs * adds float64 parameter support to sqlite3 * change in timestamp type in the script * deletes float64 toBytes utils * enables storage of timestamp as a real number in the sqlite db * bump up script index * comment edits * removes migrate unit test * adds todo and docstring * updates changelog * removes an unused item in .gitignore * minor * updates changelog * organizes imports * cleans up imports * WIP * updates script fixes a few bugs on the splitScript more debug logs adds float64 parameter support to sqlite3 change in timestamp type in the script deletes float64 toBytes utils * edits migration util test * remove an empty test file * includes migration utils tests in * deletes unused codes * tides up imports * adds range based filter to the filterMigrationScripts * renames procs: removes Migration * tides up imports * edits docstring * edits docstring * edits docstring * removes unused imports * more clean up * groups std imports * updates changelog * adds docstring for setUserVersion * adds unittest for the migrate * Update waku/v2/node/storage/message/waku_message_store.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * removes split scripts * fixes a naming issue * fixes a bug * fixes a typo * adds a log re updated user_version * fixes a proc naming mismatch * fixes naming mismatch * more descriptive var names * adds migration script of the first user version * moves migration to after persistMessages flag is checked * deletes unused comment * fixes a bug * brings back split script * adds unit tests for split scripts * runs scripts one command at a time * deletes a commented line * relocates the migrate proc to sqlite.nim * adds unit test for filter scripts * adds filterScripts unittest testing varying zero-prefixed user versions * minor Co-authored-by: RichΛrd <info@richardramos.me>
2021-06-16 20:23:55 +00:00
#### Schema
2021-07-23 10:32:46 +00:00
- Updated the `Message` table of the persistent message store:
- Added `senderTimestamp` column.
- Renamed the `timestamp` column to `receiverTimestamp` and changes its type to `REAL`.
Persisting Waku message timestamp & implementing DB migration & convert receiver timestamp data type to float64 (#607) * adds timestamp to waku message store impl * stores timestamp as int64 * adds untitest * stores timestamp as seq of bytes * minor * re-orders unittest * changes receiver timestamp to float64 * unit test for receiver timestamps * adds comments * reorder a few lines * updates changelog * more updates on changelog * WIP * WIP * adds migration * more debug messages * passes the path to the migration scripts from message store module * adds migration result type * replaces migrationScripts with migrationScriptsResult * adds path calculation to the message store * removes some tests binary file * removes redundant imports * comments out user_version assignment in sqlite init * more descriptive err messages * clean up test file * more info logs * minor code format * removes a todo * minor updates * remove a binary file * unit tests for migration utils * adds split script * integrates split query to handle scripts with multiple commands * updates migration script for v1 * updates the v1 migration script * update user version * updates script * fixes a few bugs on the splitScript * more debug logs * adds float64 parameter support to sqlite3 * change in timestamp type in the script * deletes float64 toBytes utils * enables storage of timestamp as a real number in the sqlite db * bump up script index * comment edits * removes migrate unit test * adds todo and docstring * updates changelog * removes an unused item in .gitignore * minor * updates changelog * organizes imports * cleans up imports * WIP * updates script fixes a few bugs on the splitScript more debug logs adds float64 parameter support to sqlite3 change in timestamp type in the script deletes float64 toBytes utils * edits migration util test * remove an empty test file * includes migration utils tests in * deletes unused codes * tides up imports * adds range based filter to the filterMigrationScripts * renames procs: removes Migration * tides up imports * edits docstring * edits docstring * edits docstring * removes unused imports * more clean up * groups std imports * updates changelog * adds docstring for setUserVersion * adds unittest for the migrate * Update waku/v2/node/storage/message/waku_message_store.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * removes split scripts * fixes a naming issue * fixes a bug * fixes a typo * adds a log re updated user_version * fixes a proc naming mismatch * fixes naming mismatch * more descriptive var names * adds migration script of the first user version * moves migration to after persistMessages flag is checked * deletes unused comment * fixes a bug * brings back split script * adds unit tests for split scripts * runs scripts one command at a time * deletes a commented line * relocates the migrate proc to sqlite.nim * adds unit test for filter scripts * adds filterScripts unittest testing varying zero-prefixed user versions * minor Co-authored-by: RichΛrd <info@richardramos.me>
2021-06-16 20:23:55 +00:00
#### API
2021-07-23 10:32:46 +00:00
- Added optional `timestamp` to [`WakuRelayMessage`](https://rfc.vac.dev/spec/16/#wakurelaymessage) on JSON-RPC API.
Persisting Waku message timestamp & implementing DB migration & convert receiver timestamp data type to float64 (#607) * adds timestamp to waku message store impl * stores timestamp as int64 * adds untitest * stores timestamp as seq of bytes * minor * re-orders unittest * changes receiver timestamp to float64 * unit test for receiver timestamps * adds comments * reorder a few lines * updates changelog * more updates on changelog * WIP * WIP * adds migration * more debug messages * passes the path to the migration scripts from message store module * adds migration result type * replaces migrationScripts with migrationScriptsResult * adds path calculation to the message store * removes some tests binary file * removes redundant imports * comments out user_version assignment in sqlite init * more descriptive err messages * clean up test file * more info logs * minor code format * removes a todo * minor updates * remove a binary file * unit tests for migration utils * adds split script * integrates split query to handle scripts with multiple commands * updates migration script for v1 * updates the v1 migration script * update user version * updates script * fixes a few bugs on the splitScript * more debug logs * adds float64 parameter support to sqlite3 * change in timestamp type in the script * deletes float64 toBytes utils * enables storage of timestamp as a real number in the sqlite db * bump up script index * comment edits * removes migrate unit test * adds todo and docstring * updates changelog * removes an unused item in .gitignore * minor * updates changelog * organizes imports * cleans up imports * WIP * updates script fixes a few bugs on the splitScript more debug logs adds float64 parameter support to sqlite3 change in timestamp type in the script deletes float64 toBytes utils * edits migration util test * remove an empty test file * includes migration utils tests in * deletes unused codes * tides up imports * adds range based filter to the filterMigrationScripts * renames procs: removes Migration * tides up imports * edits docstring * edits docstring * edits docstring * removes unused imports * more clean up * groups std imports * updates changelog * adds docstring for setUserVersion * adds unittest for the migrate * Update waku/v2/node/storage/message/waku_message_store.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * removes split scripts * fixes a naming issue * fixes a bug * fixes a typo * adds a log re updated user_version * fixes a proc naming mismatch * fixes naming mismatch * more descriptive var names * adds migration script of the first user version * moves migration to after persistMessages flag is checked * deletes unused comment * fixes a bug * brings back split script * adds unit tests for split scripts * runs scripts one command at a time * deletes a commented line * relocates the migrate proc to sqlite.nim * adds unit test for filter scripts * adds filterScripts unittest testing varying zero-prefixed user versions * minor Co-authored-by: RichΛrd <info@richardramos.me>
2021-06-16 20:23:55 +00:00
### Fixes
2021-07-23 10:32:46 +00:00
- Conversion between topics for the Waku v1 <-> v2 bridge now follows the [RFC recommendation](https://rfc.vac.dev/spec/23/).
- Fixed field order of `HistoryResponse` protobuf message: the field numbers of the `HistoryResponse` are shifted up by one to match up the [13/WAKU2-STORE](https://rfc.vac.dev/spec/13/) specs.
This release supports the following [libp2p protocols](https://docs.libp2p.io/concepts/protocols/):
| Protocol | Spec status | Protocol id |
| ---: | :---: | :--- |
| [`17/WAKU-RLN`](https://rfc.vac.dev/spec/17/) | `raw` | `/vac/waku/waku-rln-relay/2.0.0-alpha1` |
| [`11/WAKU2-RELAY`](https://rfc.vac.dev/spec/11/) | `stable` | `/vac/waku/relay/2.0.0` |
| [`12/WAKU2-FILTER`](https://rfc.vac.dev/spec/12/) | `draft` | `/vac/waku/filter/2.0.0-beta1` |
| [`13/WAKU2-STORE`](https://rfc.vac.dev/spec/13/) | `draft` | `/vac/waku/store/2.0.0-beta3` |
| [`18/WAKU2-SWAP`](https://rfc.vac.dev/spec/18/) | `draft` | `/vac/waku/swap/2.0.0-beta1` |
| [`19/WAKU2-LIGHTPUSH`](https://rfc.vac.dev/spec/19/) | `draft` | `/vac/waku/lightpush/2.0.0-beta1` |
The Waku v1 implementation is stable but not under active development.
Persisting Waku message timestamp & implementing DB migration & convert receiver timestamp data type to float64 (#607) * adds timestamp to waku message store impl * stores timestamp as int64 * adds untitest * stores timestamp as seq of bytes * minor * re-orders unittest * changes receiver timestamp to float64 * unit test for receiver timestamps * adds comments * reorder a few lines * updates changelog * more updates on changelog * WIP * WIP * adds migration * more debug messages * passes the path to the migration scripts from message store module * adds migration result type * replaces migrationScripts with migrationScriptsResult * adds path calculation to the message store * removes some tests binary file * removes redundant imports * comments out user_version assignment in sqlite init * more descriptive err messages * clean up test file * more info logs * minor code format * removes a todo * minor updates * remove a binary file * unit tests for migration utils * adds split script * integrates split query to handle scripts with multiple commands * updates migration script for v1 * updates the v1 migration script * update user version * updates script * fixes a few bugs on the splitScript * more debug logs * adds float64 parameter support to sqlite3 * change in timestamp type in the script * deletes float64 toBytes utils * enables storage of timestamp as a real number in the sqlite db * bump up script index * comment edits * removes migrate unit test * adds todo and docstring * updates changelog * removes an unused item in .gitignore * minor * updates changelog * organizes imports * cleans up imports * WIP * updates script fixes a few bugs on the splitScript more debug logs adds float64 parameter support to sqlite3 change in timestamp type in the script deletes float64 toBytes utils * edits migration util test * remove an empty test file * includes migration utils tests in * deletes unused codes * tides up imports * adds range based filter to the filterMigrationScripts * renames procs: removes Migration * tides up imports * edits docstring * edits docstring * edits docstring * removes unused imports * more clean up * groups std imports * updates changelog * adds docstring for setUserVersion * adds unittest for the migrate * Update waku/v2/node/storage/message/waku_message_store.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * Update waku/v2/node/storage/sqlite.nim Co-authored-by: RichΛrd <info@richardramos.me> * removes split scripts * fixes a naming issue * fixes a bug * fixes a typo * adds a log re updated user_version * fixes a proc naming mismatch * fixes naming mismatch * more descriptive var names * adds migration script of the first user version * moves migration to after persistMessages flag is checked * deletes unused comment * fixes a bug * brings back split script * adds unit tests for split scripts * runs scripts one command at a time * deletes a commented line * relocates the migrate proc to sqlite.nim * adds unit test for filter scripts * adds filterScripts unittest testing varying zero-prefixed user versions * minor Co-authored-by: RichΛrd <info@richardramos.me>
2021-06-16 20:23:55 +00:00
2021-06-10 15:06:21 +00:00
## 2021-06-03 v0.4
This release contains the following:
### Features
2021-06-10 15:06:21 +00:00
- Initial [`toy-chat` implementation](https://rfc.vac.dev/spec/22/)
### Changes
2021-06-10 15:06:21 +00:00
- The [toy-chat application](https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md) can now perform `lightpush` and request content-filtered messages from remote peers.
2021-06-10 15:06:21 +00:00
- The [toy-chat application](https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md) now uses default content topic `/toy-chat/2/huilong/proto`
- Improve `toy-chat` [briding to matterbridge]((https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md#bridge-messages-between-chat2-and-matterbridge))
- Improve [`swap`](https://rfc.vac.dev/spec/18/) logging and enable soft mode by default
- Content topics are no longer in a redundant nested structure
- Improve error handling
#### API
2021-06-10 15:06:21 +00:00
- [JSON-RPC Store API](https://rfc.vac.dev/spec/16): Added an optional time-based query to filter historical messages.
- [Nim API](https://github.com/status-im/nim-waku/blob/master/docs/api/v2/node.md): Added `resume` method.
2021-06-10 15:06:21 +00:00
### Fixes
2021-06-10 15:06:21 +00:00
- Connections between nodes no longer become unstable due to keep-alive errors if mesh grows large
2021-06-10 15:06:21 +00:00
- Re-enable `lightpush` tests and fix Windows CI failure
The [Waku v2 suite of protocols](https://rfc.vac.dev/) are still in a raw/draft state.
This release supports the following [libp2p protocols](https://docs.libp2p.io/concepts/protocols/):
| Protocol | Spec status | Protocol id |
| ---: | :---: | :--- |
| [`17/WAKU-RLN`](https://rfc.vac.dev/spec/17/) | `raw` | `/vac/waku/waku-rln-relay/2.0.0-alpha1` |
| [`11/WAKU2-RELAY`](https://rfc.vac.dev/spec/11/) | `draft` | `/vac/waku/relay/2.0.0-beta2` |
| [`12/WAKU2-FILTER`](https://rfc.vac.dev/spec/12/) | `draft` | `/vac/waku/filter/2.0.0-beta1` |
| [`13/WAKU2-STORE`](https://rfc.vac.dev/spec/13/) | `draft` | `/vac/waku/store/2.0.0-beta3` |
| [`18/WAKU2-SWAP`](https://rfc.vac.dev/spec/18/) | `draft` | `/vac/waku/swap/2.0.0-beta1` |
| [`19/WAKU2-LIGHTPUSH`](https://rfc.vac.dev/spec/19/) | `draft` | `/vac/waku/lightpush/2.0.0-beta1` |
The Waku v1 implementation is stable but not under active development.
## 2021-05-11 v0.3
This release contains the following:
### Features
- Start of [`RLN relay` implementation](https://rfc.vac.dev/spec/17/)
- Start of [`swap` implementation](https://rfc.vac.dev/spec/18/)
- Start of [fault-tolerant `store` implementation](https://rfc.vac.dev/spec/21/)
- Initial [`bridge` implementation](https://rfc.vac.dev/spec/15/) between Waku v1 and v2 protocols
- Initial [`lightpush` implementation](https://rfc.vac.dev/spec/19/)
- A peer manager for `relay`, `filter`, `store` and `swap` peers
- Persistent storage for peers: A node with this feature enabled will now attempt to reconnect to `relay` peers after a restart. It will respect the gossipsub [PRUNE backoff](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md#prune-backoff-and-peer-exchange) period before attempting to do so.
- `--persist-peers` CLI option to persist peers in local storage
- `--persist-messages` CLI option to store historical messages locally
- `--keep-alive` CLI option to maintain a stable connection to `relay` peers on idle topics
- A CLI chat application ([`chat2`](https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md)) over Waku v2 with [bridging to matterbridge](https://github.com/status-im/nim-waku/blob/master/docs/tutorial/chat2.md#bridge-messages-between-chat2-and-matterbridge)
### Changes
- Enable `swap` protocol by default and improve logging
#### General refactoring
- Split out `waku_types` types into the right place; create `utils` folder.
- Change type of `contentTopic` in [`ContentFilter`](https://rfc.vac.dev/spec/12/#protobuf) to `string`.
- Replace sequence of `contentTopics` in [`ContentFilter`](https://rfc.vac.dev/spec/12/#protobuf) with a single `contentTopic`.
- Add `timestamp` field to [`WakuMessage`](https://rfc.vac.dev/spec/14/#payloads).
- Ensure CLI config parameters use a consistent naming scheme. Summary of changes [here](https://github.com/status-im/nim-waku/pull/543).
#### Docs
Several clarifications and additions aimed at contributors, including
- information on [how to query Status test fleet](https://github.com/status-im/nim-waku/blob/master/docs/faq.md) for node addresses,
- [how to view logs](https://github.com/status-im/nim-waku/blob/master/docs/contributors/cluster-logs.md), and
- [how to update submodules](https://github.com/status-im/nim-waku/blob/master/docs/contributors/git-submodules.md).
#### Schema
- Add `Message` table to the persistent message store. This table replaces the old `messages` table. It has two additional columns, namely
- `pubsubTopic`, and
- `version`.
- Add `Peer` table for persistent peer storage.
#### API
- [JSON-RPC Admin API](https://rfc.vac.dev/spec/16): Added a [`post` method](https://rfc.vac.dev/spec/16/#post_waku_v2_admin_v1_peers) to connect to peers on an ad-hoc basis.
- [Nim API](https://github.com/status-im/nim-waku/blob/master/docs/api/v2/node.md): PubSub topic `subscribe` and `unsubscribe` no longer returns a future (removed `async` designation).
- [`HistoryQuery`](https://rfc.vac.dev/spec/13/#historyquery): Added `pubsubTopic` field. Message history can now be filtered and queried based on the `pubsubTopic`.
- [`HistoryQuery`](https://rfc.vac.dev/spec/13/#historyquery): Added support for querying a time window by specifying start and end times.
### Fixes
- Running nodes can now be shut down gracefully
- Content filtering now works on any PubSub topic and not just the `waku` default.
- Nodes can now mount protocols without supporting `relay` as a capability
The [Waku v2 suite of protocols](https://rfc.vac.dev/) are still in a raw/draft state.
This release supports the following [libp2p protocols](https://docs.libp2p.io/concepts/protocols/):
| Protocol | Spec status | Protocol id |
| ---: | :---: | :--- |
| [`17/WAKU-RLN`](https://rfc.vac.dev/spec/17/) | `raw` | `/vac/waku/waku-rln-relay/2.0.0-alpha1` |
| [`18/WAKU2-SWAP`](https://rfc.vac.dev/spec/18/) | `raw` | `/vac/waku/swap/2.0.0-alpha1` |
| [`19/WAKU2-LIGHTPUSH`](https://rfc.vac.dev/spec/19/) | `raw` | `/vac/waku/lightpush/2.0.0-alpha1` |
| [`11/WAKU2-RELAY`](https://rfc.vac.dev/spec/11/) | `draft` | `/vac/waku/relay/2.0.0-beta2` |
| [`12/WAKU2-FILTER`](https://rfc.vac.dev/spec/12/) | `draft` | `/vac/waku/filter/2.0.0-beta1` |
| [`13/WAKU2-STORE`](https://rfc.vac.dev/spec/13/) | `draft` | `/vac/waku/store/2.0.0-beta3` |
The Waku v1 implementation is stable but not under active development.
2021-01-06 06:49:28 +00:00
## 2021-01-05 v0.2
This release contains the following changes:
- Calls to `publish` a message on `wakunode2` now `await` instead of `discard` dispatched [`WakuRelay`](https://github.com/vacp2p/specs/blob/master/specs/waku/v2/waku-relay.md) procedures.
- [`StrictNoSign`](https://github.com/libp2p/specs/tree/master/pubsub#message-signing) enabled.
- Add JSON-RPC API for external access to `wakunode2` functionality:
- Admin API retrieves information about peers registered on the `wakunode2`.
- Debug API exposes debug information about a `wakunode2`.
- Filter API saves bandwidth by allowing light nodes to filter for specific content.
- Private API enables symmetric or asymmetric cryptography to encrypt/decrypt message payloads.
- Relay API allows basic pub/sub functionality.
- Store API retrieves historical messages.
- Add tutorial on how to use JSON-RPC API.
- Refactor: Move `waku_filter` protocol into its own module.
2021-01-06 06:49:28 +00:00
The Waku v2 implementation, and [most protocols it consist of](https://specs.vac.dev/specs/waku/),
are still in a draft/beta state. The Waku v1 implementation is stable but not under active development.
## 2020-11-30 v0.1
Initial beta release.
This release contains:
- A Nim implementation of the [Waku v1 protocol](https://specs.vac.dev/waku/waku.html).
- A Nim implementation of the [Waku v2 protocol](https://specs.vac.dev/specs/waku/v2/waku-v2.html).
- CLI applications `wakunode` and `wakunode2` that allows you to run a Waku v1 or v2 node.
- Examples of Waku v1 and v2 usage.
- Various tests of above.
Currenty the Waku v2 implementation, and [most protocols it consist of](https://specs.vac.dev/specs/waku/),
are in a draft/beta state. The Waku v1 implementation is stable but not under active development.
Feedback welcome!