Commit Graph

91 Commits

Author SHA1 Message Date
Sanaz Taheri Boshrooyeh 30a7a7a4fe
Filter out duplicate messages in the resume procedure (#600)
* adds timestamp to waku message store impl

* stores timestamp as int64

* adds untitest

* stores timestamp as seq of bytes

* minor

* re-orders unittest

* changes receiver timestamp to float64

* unit test for receiver timestamps

* adds comments

* reorder a few lines

* updates changelog

* more updates on changelog

* WIP: deduplicate

* add debug messages

* more debugging logs

* add timestamp to the message store

* converts resume result to uint64, removes some logs

* docstring for isDuplicate

* increments `added` before insertion into the db

* unit test for deduplication

* removes float to/from Bytes procs

* deletes extra spaces

* returns failed instead of the error

* minor

* minor

* adds changelog
2021-06-22 11:30:12 +08:00
Hanno Cornelius 9e0b73bd18
Update submodules (#594) 2021-06-09 16:37:08 +02:00
Sanaz Taheri Boshrooyeh 2fe53c334c
Make rln optional (dependencies and compilation) (#602)
* makes rlnlib target optional

* makes the rln-related targets conditioned to the RLN command line flag

* adds the rln compiler flag condition

* adds the rln compiler flag condition to the rln module

* wakunode2 rln conditional compilation

* updates wakunode2 test

* updates waku rln relay tests

* removing the rln test from the default imports of the v2 tests

* imports rln module conditionally

* removes the rln flag condition from the rln module

* separates rln data types from its procs

* adds the import statement

* brings back the contract def to the rln utils

* adds rln module import to the rln unit tests

* clean up and reorganization

* adds a todo

* minor edits on a comment
2021-06-08 11:56:32 -07:00
Sanaz Taheri Boshrooyeh f75f7c4751
Add resume to waku node api (#564)
* activates resume

* adds unit test

* add store condition

* mode debug info

* updates wakunode resume api and its unittest

* adds todo

* adds documentation

* edits resume documentation

* further edits on the docs

* removes a todo

* fixes a bug

* add resume to waku node api

* further updates on the node.md

* updates the changelog

* minor

* removes return type

* adds a brief desc of the resume to node.md
2021-05-26 12:33:22 -07:00
Oskar Thorén d99e9251b5
Re-enable lightpush tests and fix Windows CI failure (#572)
* Enable lightpush test again

- Stop nodes differently
- Bump port numbers to possibly get rid of Windows leaked resources

* Lightpush test mount relay w/o relaying messaging
2021-05-25 16:31:59 +08:00
Ebube Sered Ud a6bb3f65e1
Fix redundant use of content topics (#528)
made resulting changes to waku_filter

Made changes to wakunode2, filter_api and waku_filter

Update waku v2 test scripts referencing Content Topics

Update ContentFilter in chat example

Remove unneccesary loops from filter api
closes #496

Apply keep-alive for chat2 (#525)

Makes the arguments of the store jsonrpc api optional (#526)

* makes pubsubTopic filter optional

* makes contentFilter optional

* append Option

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Update Changelog with changes to ContentFilter

Fix indentation and code semantics

Enables perssist-message flag in the store protocol for wakunode2 (#519)

* enables perssistmessage flag

* disables in memory storage when persist-messages is false

* adds the persistMessages input to the mountStore

* defaults the store flag to true

* adds the missing argument

* persists messages in memory conditioned to the persistMessages flag

* adds persistmessages flag to the config_bridge

* defaults persistmessages to true

* defaults the store flag to true and persist-messages to false

* updates store.md

* updates chat2 instructions about --store flag

* removes --store flag from chat2 command execution

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Fix: light-mode relay for all light protocols (#529)

* Fix: light-mode relay for all light protocols

* Clear up confusing use of overloaded concepts

Fix ContentFilter Schema in wakunode test script

Enables perssist-message flag in the store protocol for wakunode2 (#519)

* enables perssistmessage flag

* disables in memory storage when persist-messages is false

* adds the persistMessages input to the mountStore

* defaults the store flag to true

* adds the missing argument

* persists messages in memory conditioned to the persistMessages flag

* adds persistmessages flag to the config_bridge

* defaults persistmessages to true

* defaults the store flag to true and persist-messages to false

* updates store.md

* updates chat2 instructions about --store flag

* removes --store flag from chat2 command execution

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Fix: light-mode relay for all light protocols (#529)

* Fix: light-mode relay for all light protocols

* Clear up confusing use of overloaded concepts

Fix resulting issues after merge
2021-05-05 09:34:40 +01:00
Hanno Cornelius 36ff537612
Fix: light-mode relay for all light protocols (#529)
* Fix: light-mode relay for all light protocols

* Clear up confusing use of overloaded concepts
2021-05-04 14:11:41 +02:00
Sanaz Taheri Boshrooyeh c0d858af1d
Enables perssist-message flag in the store protocol for wakunode2 (#519)
* enables perssistmessage flag

* disables in memory storage when persist-messages is false

* adds the persistMessages input to the mountStore

* defaults the store flag to true

* adds the missing argument

* persists messages in memory conditioned to the persistMessages flag

* adds persistmessages flag to the config_bridge

* defaults persistmessages to true

* defaults the store flag to true and persist-messages to false

* updates store.md

* updates chat2 instructions about --store flag

* removes --store flag from chat2 command execution

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-05-03 14:30:52 -07:00
Oskar Thorén c22c1e05bc
Windows CI debug (#509)
* Make CI happy

* Disabling lightpush node test

Seems to make Windows CI happy for some reason?
2021-04-28 13:13:57 +08:00
Oskar Thorén 13ac035e5a
Lightpush protocol cont (#506)
* lightpush conf and mount with relay from node

* mount lightpush after relay

* lightpush relay integration wip

* lightpush node integrate and test node

* clean
2021-04-24 12:56:37 +08:00
Hanno Cornelius f1a6027158
Fix filter notifications (#501) 2021-04-22 15:45:13 +02:00
Sanaz Taheri Boshrooyeh fb2ea06a4f
Update store protocol interface: add history content filter (#488)
* replaces topics with seq of ContentFilters

* update topics to contentFilter

* updates the contentFilter  structure

 one content  topic per content filter instead of a sequence of topics

* updates store json rpc api

* renames ContentFilter to HistoryContentFilter

* unit test for a query with several content filters

* makes shortcut for store api

* updates chat2

* clean up

* renames topic to contentTopic

* clarifies the use of content topic in store api

* clarifies the use of contentTopic in the init method of HistoryContentFilter
2021-04-19 10:38:30 -07:00
Hanno Cornelius 17cfc32c4f
Add default handler for all topics (#483)
Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-13 07:56:49 +02:00
Hanno Cornelius 5747ff5be0
Change contentTopic to string (#463)
* Change contentTopic to string

* Missed a spot

* Try to fix Windows CI

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-08 17:55:19 +08:00
Hanno Cornelius 683577f045
Start relay protocol on wakunode2 (#446)
* Start relay protocol on wakunode2
2021-04-01 11:37:14 +02:00
Kim De Mey d1c1a0ca13
Bump chronos and fix exception tracking issues (#436)
* Bump nim-chronos and fix exception tracking issues

* Bump other Nim submodules to latest

* Fix repeatMessage properly through proc type fix in nim-eth

Also add and use unittest2 through testutils to avoid extra
annotations.
2021-03-26 10:52:04 +01:00
Sanaz Taheri Boshrooyeh d64aadc8ad
Basic proof gen and vrfy (#421)
* adds the proof field to the WakuMessage

* adds a basic zkProof api

* adds proof gen to the publish proc

* relocates the proofGen and proofVrfy to rln_relay_utils

* wip: test of proof gen

* adds a procedure to pass rln-relay message validator, adds the proof gen

* tests the proof gen and verify

* relocates zkp API

* adds documentation and todos

* adds todos and documentations

* removes an unnecessary comment

* adds todos
2021-03-16 11:18:40 -07:00
Sanaz Taheri Boshrooyeh b3208b343f
Filtering messages in the relay protocol using libp2p topic validators (#414)
* adds a test for topic validators

* edits the import

* edits the comments

* further edits on the comments

* mode comments on the test

* checks the pubsub topic in the validator

* adds a fix

* changes the sleep time to 2 secs
2021-03-11 15:42:26 -08:00
Hanno Cornelius 41fe4395a8
Bump submodules (#363)
* Bump submodules

* GossipSub refactor: Rem async from sub/unsub
2021-02-02 13:33:59 +02:00
Hanno Cornelius ed979a7d60
Fix network simulation and remote peer ID CLI argument parsing (#351)
* Fix network simulation and remote peer ID CLI argument parsing
2021-01-25 13:03:52 +02:00
Oskar Thorén 37a3a4fadd
Refactor: Put waku_type content where it belongs (#333)
- Separate out types and methods into appropriate module
- Add util folder for small utils used in multiple independent places
- Update paths
2021-01-06 17:35:05 +08:00
Oskar Thorén f860c0dda4
Refactor: waku_types -> waku_filter (#331)
Also create a folder with README for filter protocol.
2021-01-05 12:52:10 +08:00
Hanno Cornelius 8a1ca1ff8f
Await WakuRelay publish on node (#313)
* Await WakuRelay publish on node

* Reflect 'await' change on CHANGELOG
2020-12-02 10:40:53 +02:00
Oskar Thorén 44e9d4d86b
Refactor types store (#291)
* Refactor: Move waku_store into its own folder

* Refactor: Move waku store types to new home (WIP)

* Refactor: Fix errors and recursive imports

* Fix rebase errors

* Refactor: More rebase import fixes
2020-11-24 12:34:32 +08:00
Dean Eigenmann 1193b69d6c
enhancement/cleanup-warnings (#290) 2020-11-20 14:59:29 +01:00
Oskar Thorén 9518322198
Folder restructure (#274)
* Change folder structure to {v1,v2,common}/...

Addresses https://github.com/status-im/nim-waku/issues/261

* Update waku.nimble paths

* Flatten paths

* Fix import paths

* Pull out utils folder for nat

* Pull out waku_types to top level for v2

* Fix test import paths

* Remove old READMEs and replace with one liner

* Update README and split v1 and v2

* Skeleton READMEs

* Update README.md

Co-authored-by: Kim De Mey <kim.demey@gmail.com>

* Update README.md

Co-authored-by: Kim De Mey <kim.demey@gmail.com>

Co-authored-by: Kim De Mey <kim.demey@gmail.com>
2020-11-17 17:34:53 +08:00
Hanno Cornelius 24f4a087de
Feature/unsubscribe from content filter (#255)
Unsubscribe from a content topic

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2020-11-10 09:13:16 +02:00
Oskar Thoren 4b21bb6b9b
Default topic: waku -> /waku/2/default-waku/proto 2020-10-28 13:19:43 +08:00
Dean Eigenmann bfb29338a1
fixes/wait-for-start (#228)
* fixes subscribe to wait for start

* moved around

* added test

* fix

* eol

* rm

* removed as waku is default

* updated docs

* fix

* Update node.md

* Update node.md

* Update wakunode2.nim

* Update wakunode2.nim

* fix test

* rm

* fix

* fixes

* fixes

* fix

* fix

* using connect

* fix

* fix

* Update wakubridge.nim

* Update wakunode2.nim
2020-10-22 13:12:00 +02:00
Dean Eigenmann 5ddd8701c9
fix/content-type-int (#235)
* fixes

* fixes

* using a better topic

* fix

* fixed
2020-10-21 16:55:06 +08:00
Dean Eigenmann b2edfe8dda
feature/start-protocol-command (#209)
* started working

* fixes

* rename start -> mount

* started logging

* mounting relay

* Update wakunode2.nim

* start of fixe

* fix

* fix

* fixes

* Update waku/node/v2/wakunode2.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* removed comment

* fix

* update

* changed the default

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2020-10-20 10:36:27 +08:00
Dean Eigenmann a975bf6d70
feature/mount-filter-node-api (#181)
* started working on filter mount

* fixes

* fix indentation

* fixes

* save

* change

* rm

* Update waku_types.nim

* Update waku_filter.nim

* fixes

* fix

* using new filter

* did some testing stuff

* rm

* fix

* updated

* update doc

* fix

* fix

* unasynced

* fix

* w -> node

* rename

* move to content filter

* fix

* method

* rename

* renamed contentFilter -> contentFilters

* readded

* moved

* first test fix

* readded

* no more need

* fixes

* fix

* fix

* fixes

* todo

* removes mother fuck

* Update waku_types.nim
2020-10-02 14:48:56 +02:00
Dean Eigenmann 25b48bb99e
enhancement/store-message-handler (#185)
* started working on wrapper

* fixes

* changes

* fixes

* Update waku_store.nim
2020-09-25 16:02:13 +02:00
Dean Eigenmann 1f68e63185
feature/store-query-node-api (#174)
* changes

* fix

* changes

* made the query function work

* added rpc

* whoops

* Update wakurpc.nim

* minor

* fixes

* update docs

* fix

* error handling

* end-to-end waku node test

* Update test_wakunode.nim

* Update waku/node/v2/rpc/wakurpc.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* Update test_wakunode.nim

* Update wakunode2.nim

* fix

* fix

* shorter

* rm echo

* fix

* added history peer

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2020-09-24 10:16:25 +08:00
Oskar Thorén a8dbf8a7b6
Bump submodules (#157)
* Upgrade all submodules

* Use stock standard_setup and remove our version

Switch no longer relies on Pubsub argument

* Fix peerId

* Add reference to WakuRelay in WakuNode

* Use WakuRelay ref directly instead of via switch

* Tweak standard switch sig

* Fix start_network peerid

* Import nim-libp2p utils test

* Use WakuRelay in test utils

* Fix utils imports

* Tweak

* Fix trigger self test

* Disable broken filter test

* Fix and amend logscope topics

* Make subscribe calls async to use await

* Add debug stuff to nimble file

* Await for subscribe content

* Sleeping in tests

* Local checkout fixes

* XXX: Try to use .PubSub on WakuRelay

* Revert "XXX: Try to use .PubSub on WakuRelay"

This reverts commit 3a3139e4cf.

* Only using gossip seems to work

Subscribe for floodsub broken

* Fix await in examples

* Get rid of double publish, still need sleep
2020-09-16 12:23:10 +08:00
Kim De Mey 2717209f05
Move filter.notify to WakuRelay handler and add test (#128) 2020-09-02 11:15:25 +08:00
Oskar Thorén f828736e1c
Add support for WakuMessage and merge publish function (#122)
* Add WakuMessage type

* Add WakuMessage encoding and decoding

Also clean up waku_types module and imports

* Clean up waku_relay module

Imports, remove old text test, make fields and functions public, format.

* Publish WakuMessage

Also fix type mismatch in RPC

* Make publish work in examples, node and protocol

* Parse protobuf content in examples

* Update docs

* Update waku/node/v2/waku_types.nim

* Fix compilation error and disable out of date waku test

Co-authored-by: Kim De Mey <kim.demey@gmail.com>
2020-09-01 17:20:38 +02:00
Kim De Mey ea0d62993d
WakuNode init without WakuNodeConf (#117)
* WakuNode init without WakuNodeConf

* WakuNode start without WakuNodeConf

* setupNat adjustments for making common version

* Move setupNat to common.nim to be used for v1 and v2
2020-09-01 10:09:54 +08:00
Kim De Mey ddcfc3fdd6
V2cleanup (#115)
* Cleanup of warnings + imports, etc.

* Remove connected_peers metric as there is already libp2p_pubsub_peers
2020-08-31 11:32:41 +08:00
Oskar Thorén 705b4b32a1
Waku v2: Simplify Waku Relay references (#112)
- Remove wakuRelayProto as this is initiated as part of switch PubSub
- Move WakuNode type into waku_types to avoid circular references
- Make necessary adjustments to get tests and sim to work
2020-08-28 17:08:28 +08:00
Dean Eigenmann c3f57c79f5
feature/historic-api (#95)
* started working on historic api

* very rough code

* moved

* also publishing

* todos

* added tests

* added

* fix
2020-07-29 15:24:01 +02:00