Commit Graph

404 Commits

Author SHA1 Message Date
Hanno Cornelius 01d6396385
General chat2bridge improvements (#536) 2021-05-07 10:05:11 +02:00
Hanno Cornelius c5e7580149
Add chat2-matterbridge (#532)
* Add chat2-matterbridge

* Update waku/common/README.md

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-05-06 15:43:43 +02:00
Sanaz Taheri Boshrooyeh 7c5df3379b
Storing waku message version inside the store db (#530)
* WIP

* stores version in db, edits error messages

* fixes a bug

* reverts edit messages

* converts uint32 storage to int64

* unit tests for various version numbers

* minor comments edit

* removes debugging messages

* updates changelog
2021-05-05 15:51:36 -07:00
Ebube Sered Ud a6bb3f65e1
Fix redundant use of content topics (#528)
made resulting changes to waku_filter

Made changes to wakunode2, filter_api and waku_filter

Update waku v2 test scripts referencing Content Topics

Update ContentFilter in chat example

Remove unneccesary loops from filter api
closes #496

Apply keep-alive for chat2 (#525)

Makes the arguments of the store jsonrpc api optional (#526)

* makes pubsubTopic filter optional

* makes contentFilter optional

* append Option

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Update Changelog with changes to ContentFilter

Fix indentation and code semantics

Enables perssist-message flag in the store protocol for wakunode2 (#519)

* enables perssistmessage flag

* disables in memory storage when persist-messages is false

* adds the persistMessages input to the mountStore

* defaults the store flag to true

* adds the missing argument

* persists messages in memory conditioned to the persistMessages flag

* adds persistmessages flag to the config_bridge

* defaults persistmessages to true

* defaults the store flag to true and persist-messages to false

* updates store.md

* updates chat2 instructions about --store flag

* removes --store flag from chat2 command execution

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Fix: light-mode relay for all light protocols (#529)

* Fix: light-mode relay for all light protocols

* Clear up confusing use of overloaded concepts

Fix ContentFilter Schema in wakunode test script

Enables perssist-message flag in the store protocol for wakunode2 (#519)

* enables perssistmessage flag

* disables in memory storage when persist-messages is false

* adds the persistMessages input to the mountStore

* defaults the store flag to true

* adds the missing argument

* persists messages in memory conditioned to the persistMessages flag

* adds persistmessages flag to the config_bridge

* defaults persistmessages to true

* defaults the store flag to true and persist-messages to false

* updates store.md

* updates chat2 instructions about --store flag

* removes --store flag from chat2 command execution

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Fix: light-mode relay for all light protocols (#529)

* Fix: light-mode relay for all light protocols

* Clear up confusing use of overloaded concepts

Fix resulting issues after merge
2021-05-05 09:34:40 +01:00
Hanno Cornelius 36ff537612
Fix: light-mode relay for all light protocols (#529)
* Fix: light-mode relay for all light protocols

* Clear up confusing use of overloaded concepts
2021-05-04 14:11:41 +02:00
Sanaz Taheri Boshrooyeh c0d858af1d
Enables perssist-message flag in the store protocol for wakunode2 (#519)
* enables perssistmessage flag

* disables in memory storage when persist-messages is false

* adds the persistMessages input to the mountStore

* defaults the store flag to true

* adds the missing argument

* persists messages in memory conditioned to the persistMessages flag

* adds persistmessages flag to the config_bridge

* defaults persistmessages to true

* defaults the store flag to true and persist-messages to false

* updates store.md

* updates chat2 instructions about --store flag

* removes --store flag from chat2 command execution

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-05-03 14:30:52 -07:00
Sanaz Taheri Boshrooyeh 7b6d93d13b
Makes the arguments of the store jsonrpc api optional (#526)
* makes pubsubTopic filter optional

* makes contentFilter optional

* append Option

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-05-03 12:25:28 -07:00
Hanno Cornelius 466f1ecfce
Apply keep-alive for chat2 (#525) 2021-05-03 14:48:55 +08:00
Hanno Cornelius f457db9e0a
Add optional keepAlive for relay peers (#523) 2021-04-30 14:07:46 +02:00
Franck Royer a4db76f1ad
Rust is needed to compile the project (#520) 2021-04-29 16:04:50 +10:00
Hanno Cornelius 967db6d610
Allow relay=false for light filter nodes (#518) 2021-04-29 12:54:31 +08:00
Sanaz Taheri Boshrooyeh 7e9dac67c8
Integrates pubsub topic filter in the store json rpc api (#514)
* replaces topics with seq of ContentFilters

* update topics to contentFilter

* updates the contentFilter  structure

 one content  topic per content filter instead of a sequence of topics

* updates store json rpc api

* renames ContentFilter to HistoryContentFilter

* unit test for a query with several content filters

* makes shortcut for store api

* updates chat2

* clean up

* renames topic to contentTopic

* adds pubsub topic to the history query

updates message store interface to return the pubsub topic
updates waku message store implementation
updates database schema to hold pubsub topi per waku message

* clarifies the use of content topic in store api

* clarifies the use of contentTopic in the init method of HistoryContentFilter

* simplifies the test and add comments

* lowers the field number of pubsub topic in historyQuery  protobuf

* captures an empty contentFilter case

* test pubsub topic filter for the entire history and no message match

* demoves duplicates

* adds TODO

* fix a broken comment line

* updates waku store codec

* swaps the order of pubsub topic and content topic in protobuf

* Update waku/v2/protocol/waku_store/waku_store_types.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* updates the pubsub topic to the default value

* bumps protocol id

* moves the comment close to IndexedWakuMessage

* adds checks to the store put method

* makes table title a constant variable and retitles the table to Message

* updates the changelog

* minor update

* minor

* adds pubsubtopic to the store json rpc api

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-28 19:14:03 +08:00
Oskar Thorén 0d5e14f83c
Libwaku spike start (#510)
* libwaku init e2e

* basic cstring check

* wip
2021-04-28 19:13:34 +08:00
Franck Royer e2678d55ff
Fix name in licence (#517) 2021-04-28 20:00:45 +10:00
Oskar Thorén c22c1e05bc
Windows CI debug (#509)
* Make CI happy

* Disabling lightpush node test

Seems to make Windows CI happy for some reason?
2021-04-28 13:13:57 +08:00
Sanaz Taheri Boshrooyeh 50a54cb4ee
Enabling pubsub topic filter in history queries (#492)
* replaces topics with seq of ContentFilters

* update topics to contentFilter

* updates the contentFilter  structure

 one content  topic per content filter instead of a sequence of topics

* updates store json rpc api

* renames ContentFilter to HistoryContentFilter

* unit test for a query with several content filters

* makes shortcut for store api

* updates chat2

* clean up

* renames topic to contentTopic

* adds pubsub topic to the history query

updates message store interface to return the pubsub topic
updates waku message store implementation
updates database schema to hold pubsub topi per waku message

* clarifies the use of content topic in store api

* clarifies the use of contentTopic in the init method of HistoryContentFilter

* simplifies the test and add comments

* lowers the field number of pubsub topic in historyQuery  protobuf

* captures an empty contentFilter case

* test pubsub topic filter for the entire history and no message match

* demoves duplicates

* adds TODO

* fix a broken comment line

* updates waku store codec

* swaps the order of pubsub topic and content topic in protobuf

* Update waku/v2/protocol/waku_store/waku_store_types.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* updates the pubsub topic to the default value

* bumps protocol id

* moves the comment close to IndexedWakuMessage

* adds checks to the store put method

* makes table title a constant variable and retitles the table to Message

* updates the changelog

* minor update

* minor

* beta2 to beta3

* minor

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-27 16:52:24 -07:00
Sanaz Taheri Boshrooyeh 7bb58d089d
Fix store protocol identifier (#507)
* beta1 to beta2

* removing Lightpush message return success test to spot windows err

* bring back the Lightpush test
2021-04-27 15:37:50 -07:00
Oskar Thorén 13ac035e5a
Lightpush protocol cont (#506)
* lightpush conf and mount with relay from node

* mount lightpush after relay

* lightpush relay integration wip

* lightpush node integrate and test node

* clean
2021-04-24 12:56:37 +08:00
Sanaz Taheri Boshrooyeh a04643e9a2
Updates store json rpc api (#499)
* updates the store jsonrpc api and its usage across the codebase

* deletes unused codes

* deletes commented codes

* minor

* spacing issue
2021-04-23 14:14:40 -07:00
Oskar Thorén d7de633793
Light push protocol initial cut (#503)
* Init lightpush

* Lightpush types

* light push encode and decode

* lightpush protocol more

* light push test draft

* lightpush compiles

* lightpush fix tests etc

* cleanup

* pubsubTopic -> pubSubTopic
2021-04-23 14:56:04 +08:00
Hanno Cornelius f1a6027158
Fix filter notifications (#501) 2021-04-22 15:45:13 +02:00
Hanno Cornelius f0eadfec13
Add persistent backoff for peers (#497) 2021-04-21 11:36:56 +02:00
Sanaz c997860397
undo the last unwanted commit 2021-04-20 17:06:26 -07:00
Sanaz f7fbc28780
updates protocol identifier 2021-04-20 16:37:11 -07:00
Oskar Thorén 5f1bab146d
Log more in store protocol (#493)
Makes it easier to see what is going on
2021-04-20 09:59:14 +08:00
Sanaz Taheri Boshrooyeh fb2ea06a4f
Update store protocol interface: add history content filter (#488)
* replaces topics with seq of ContentFilters

* update topics to contentFilter

* updates the contentFilter  structure

 one content  topic per content filter instead of a sequence of topics

* updates store json rpc api

* renames ContentFilter to HistoryContentFilter

* unit test for a query with several content filters

* makes shortcut for store api

* updates chat2

* clean up

* renames topic to contentTopic

* clarifies the use of content topic in store api

* clarifies the use of contentTopic in the init method of HistoryContentFilter
2021-04-19 10:38:30 -07:00
Hanno Cornelius 192d4dd66b
Add backoff period before subscribing to persistent peers (#491) 2021-04-16 11:57:45 +02:00
Hanno Cornelius 900d53f9df
Implement graceful shutdown for `wakunode2` and `chat2` (#490)
* Implement graceful shutdown

* Consistent use of QuitSuccess
2021-04-15 10:18:14 +02:00
Hanno Cornelius 17cfc32c4f
Add default handler for all topics (#483)
Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-13 07:56:49 +02:00
Franck Royer d78f3c5669
Ignore .update.timestamp file (#486)
Generated as part of the build/upgrade process.
2021-04-13 12:29:12 +08:00
Sanaz Taheri Boshrooyeh 286482ea32
Handling initial paging requests over message history of size one (#479)
* handles initial paging query for history of size one

* adds two more tests to fetch the entire history in the initial paging request

* increases the pageSize
2021-04-12 10:29:09 -07:00
Sanaz Taheri Boshrooyeh 38b7a257c5
Enabling time window history query (#454)
* adds time based query

* handling temporal history queries

* more tests for corner cases

* edits tests comments

* further comments

* updates the time query logic

queries with zero-size window will result in an empty response

* returns no messages for invalid time queries

* comment update

* converts contentTopics from int to string

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-09 16:40:34 -07:00
Hanno Cornelius 58dd431779
Fix Windows CI (#467)
* Force chronological message history

* Fix start/end index computation

* Sleep before RPC call

* Lower default max cache sizes on APIs

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-09 13:47:24 -07:00
Sanaz Taheri Boshrooyeh 6a4fcaa671
Fix the issue in the order of fields of HistoryQuery (#470)
Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-09 09:50:51 -07:00
Sanaz Taheri Boshrooyeh 5f0e6201ed
Change page size type to uint64 (#469)
* changes pageSize to uint64

* WIP

* reworks tests and indexing proc to match uint64 pageSize

* minor simplification

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-09 18:04:21 +08:00
RichΛrd b5f1becace
fix: direction encoding (#464)
* fix: direction encoding

* fix: decoding pagingInfo direction
2021-04-09 11:09:20 +08:00
Hanno Cornelius 5747ff5be0
Change contentTopic to string (#463)
* Change contentTopic to string

* Missed a spot

* Try to fix Windows CI

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-08 17:55:19 +08:00
Oskar Thorén 139f148037
Make wakunode2 and wakunode2 default to DEBUG log (#462)
Logs too noisy by default, full of gossipsub TRC making it hard to see what is going on on a local node
2021-04-08 13:40:49 +08:00
Sanaz Taheri Boshrooyeh d49cb53b56
Updates the field number of proof and timestamp (#457)
* updates the fields numbers of proof and timestamp

* clarifies that proof is not part of waku message specs

* Update waku/v2/protocol/waku_message.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* fixes indentation

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-07 11:08:46 -07:00
Sanaz Taheri Boshrooyeh a0dba2fea4
adds startTime and endTime to the HistoryQuery (#451) 2021-04-02 15:53:28 -07:00
Sanaz Taheri Boshrooyeh 192c2c7ea0
Add timestamp field to the waku message (#450)
* adds timestamp field

* unit test for timestamp

* adds comments

* updates test title
2021-04-02 14:28:46 -07:00
Hanno Cornelius 683577f045
Start relay protocol on wakunode2 (#446)
* Start relay protocol on wakunode2
2021-04-01 11:37:14 +02:00
Sanaz Taheri Boshrooyeh e7c21c2f74
Rln-relay zkp module Nim bindings (#427)
* entirely replaces the prior rln header, the var variables are changed to ptr

* updates the unittest of key_gen

* adds test for update_next_member

* updates membershipKeyGen internals and prototype

* adds createRLNInstance

* adds helpers methods

* adds generateKeyPairBuffer

* cleans up the test and adds comments

* renames  merkleTreeDepth to d

* fixes a buf re decoding the keys into sk and pk

* adds getSKPK proc

* unifies key gen helper procs, adds todos

* comments out the createRLNInstance

* refactors the code based on the updated createRLNInstance interface

* adds the test for the verify proc

* fixes a variable name and replaces random key gen with the real key gen

* tests a simple hash

* adds get_root method

* fixes the data pointer issue and adds the proof breakdown

* adds rln

* adds unit tests for Merkle tree

* adds a sample hash test

* fixes the hash bug and comments out unused part of proof gen test

* cleans up the proof gent test

* replaces unsafeAddr with addr

* fixes an issue in key gen

* updates rln submodule

* fixes the verification problem

* adds a failed test

* replaces an old test scenario with a new one

* handles createRLNInstance output

* working createRLNInstance2

* refactors the code by replacing the old createRLNInstance

* renames createRLNInstance2

* adds documentation and reorganizes rln.nim

* replace echo with debug, renames vars, adds a bad proof test

* minor

* minor

* edits var names

* adds one more check

* adds one more test to the hash

* enforcing exception handling

* adds pacman -Sy

* removes update:true

* activates update
2021-03-31 17:39:27 -07:00
Hanno Cornelius 27844c9a20
Specify chat2 message format (#440)
* Specify chat2 message format

* Clarify payload is UTF8 encoded
2021-03-31 17:06:20 +02:00
Franck Royer e4fc68b6ea
Update kibana link to only filter on fleet (#444)
Also, return last 7 days of logs instead of
fixed date in the past.
2021-03-31 14:32:18 +08:00
Hanno Cornelius aa9a74c60b
Fix nil error when starting wakunode2 with store/filter node (#441) 2021-03-30 15:33:19 +02:00
Kim De Mey d1c1a0ca13
Bump chronos and fix exception tracking issues (#436)
* Bump nim-chronos and fix exception tracking issues

* Bump other Nim submodules to latest

* Fix repeatMessage properly through proc type fix in nim-eth

Also add and use unittest2 through testutils to avoid extra
annotations.
2021-03-26 10:52:04 +01:00
Hanno Cornelius 5102576234
Integrate persistent peer storage (#437)
* Integrate persistent peer storage
2021-03-26 10:49:51 +02:00
Hanno Cornelius 7a732e7cc6
Added persistent storage for peers (#435) 2021-03-25 10:37:11 +02:00
Sanaz Taheri Boshrooyeh 29d69b98cb
Adds Merkle tree Nim bindings (#430)
* entirely replaces the prior rln header, the var variables are changed to ptr

* updates the unittest of key_gen

* adds test for update_next_member

* updates membershipKeyGen internals and prototype

* adds createRLNInstance

* adds helpers methods

* adds generateKeyPairBuffer

* cleans up the test and adds comments

* renames  merkleTreeDepth to d

* fixes a buf re decoding the keys into sk and pk

* adds getSKPK proc

* unifies key gen helper procs, adds todos

* comments out the createRLNInstance

* refactors the code based on the updated createRLNInstance interface

* adds the test for the verify proc

* fixes a variable name and replaces random key gen with the real key gen

* tests a simple hash

* adds get_root method

* fixes the data pointer issue and adds the proof breakdown

* adds rln

* adds unit tests for Merkle tree

* deletes unnecessary comments

* updates createRLNInstance to return bool indicating the success of call

* updates create RLN Instance interface

* minor

* clean up

* removes unused imports

* adds documentation

* adds comments

* adds byteutils

* removes extra spaces

* updates rln submodule

* deletes genSKPK

* fixes a bug in membershipKeyGen

* unsafeAddr to addr

* Update waku/v2/protocol/waku_rln_relay/rln.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* clean up

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-03-24 10:26:56 -07:00