Commit Graph

340 Commits

Author SHA1 Message Date
Oskar Thorén c22c1e05bc
Windows CI debug (#509)
* Make CI happy

* Disabling lightpush node test

Seems to make Windows CI happy for some reason?
2021-04-28 13:13:57 +08:00
Sanaz Taheri Boshrooyeh 50a54cb4ee
Enabling pubsub topic filter in history queries (#492)
* replaces topics with seq of ContentFilters

* update topics to contentFilter

* updates the contentFilter  structure

 one content  topic per content filter instead of a sequence of topics

* updates store json rpc api

* renames ContentFilter to HistoryContentFilter

* unit test for a query with several content filters

* makes shortcut for store api

* updates chat2

* clean up

* renames topic to contentTopic

* adds pubsub topic to the history query

updates message store interface to return the pubsub topic
updates waku message store implementation
updates database schema to hold pubsub topi per waku message

* clarifies the use of content topic in store api

* clarifies the use of contentTopic in the init method of HistoryContentFilter

* simplifies the test and add comments

* lowers the field number of pubsub topic in historyQuery  protobuf

* captures an empty contentFilter case

* test pubsub topic filter for the entire history and no message match

* demoves duplicates

* adds TODO

* fix a broken comment line

* updates waku store codec

* swaps the order of pubsub topic and content topic in protobuf

* Update waku/v2/protocol/waku_store/waku_store_types.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* updates the pubsub topic to the default value

* bumps protocol id

* moves the comment close to IndexedWakuMessage

* adds checks to the store put method

* makes table title a constant variable and retitles the table to Message

* updates the changelog

* minor update

* minor

* beta2 to beta3

* minor

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-27 16:52:24 -07:00
Sanaz Taheri Boshrooyeh 7bb58d089d
Fix store protocol identifier (#507)
* beta1 to beta2

* removing Lightpush message return success test to spot windows err

* bring back the Lightpush test
2021-04-27 15:37:50 -07:00
Oskar Thorén 13ac035e5a
Lightpush protocol cont (#506)
* lightpush conf and mount with relay from node

* mount lightpush after relay

* lightpush relay integration wip

* lightpush node integrate and test node

* clean
2021-04-24 12:56:37 +08:00
Sanaz Taheri Boshrooyeh a04643e9a2
Updates store json rpc api (#499)
* updates the store jsonrpc api and its usage across the codebase

* deletes unused codes

* deletes commented codes

* minor

* spacing issue
2021-04-23 14:14:40 -07:00
Oskar Thorén d7de633793
Light push protocol initial cut (#503)
* Init lightpush

* Lightpush types

* light push encode and decode

* lightpush protocol more

* light push test draft

* lightpush compiles

* lightpush fix tests etc

* cleanup

* pubsubTopic -> pubSubTopic
2021-04-23 14:56:04 +08:00
Hanno Cornelius f1a6027158
Fix filter notifications (#501) 2021-04-22 15:45:13 +02:00
Hanno Cornelius f0eadfec13
Add persistent backoff for peers (#497) 2021-04-21 11:36:56 +02:00
Sanaz c997860397
undo the last unwanted commit 2021-04-20 17:06:26 -07:00
Sanaz f7fbc28780
updates protocol identifier 2021-04-20 16:37:11 -07:00
Oskar Thorén 5f1bab146d
Log more in store protocol (#493)
Makes it easier to see what is going on
2021-04-20 09:59:14 +08:00
Sanaz Taheri Boshrooyeh fb2ea06a4f
Update store protocol interface: add history content filter (#488)
* replaces topics with seq of ContentFilters

* update topics to contentFilter

* updates the contentFilter  structure

 one content  topic per content filter instead of a sequence of topics

* updates store json rpc api

* renames ContentFilter to HistoryContentFilter

* unit test for a query with several content filters

* makes shortcut for store api

* updates chat2

* clean up

* renames topic to contentTopic

* clarifies the use of content topic in store api

* clarifies the use of contentTopic in the init method of HistoryContentFilter
2021-04-19 10:38:30 -07:00
Hanno Cornelius 192d4dd66b
Add backoff period before subscribing to persistent peers (#491) 2021-04-16 11:57:45 +02:00
Hanno Cornelius 900d53f9df
Implement graceful shutdown for `wakunode2` and `chat2` (#490)
* Implement graceful shutdown

* Consistent use of QuitSuccess
2021-04-15 10:18:14 +02:00
Hanno Cornelius 17cfc32c4f
Add default handler for all topics (#483)
Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-13 07:56:49 +02:00
Franck Royer d78f3c5669
Ignore .update.timestamp file (#486)
Generated as part of the build/upgrade process.
2021-04-13 12:29:12 +08:00
Sanaz Taheri Boshrooyeh 286482ea32
Handling initial paging requests over message history of size one (#479)
* handles initial paging query for history of size one

* adds two more tests to fetch the entire history in the initial paging request

* increases the pageSize
2021-04-12 10:29:09 -07:00
Sanaz Taheri Boshrooyeh 38b7a257c5
Enabling time window history query (#454)
* adds time based query

* handling temporal history queries

* more tests for corner cases

* edits tests comments

* further comments

* updates the time query logic

queries with zero-size window will result in an empty response

* returns no messages for invalid time queries

* comment update

* converts contentTopics from int to string

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-09 16:40:34 -07:00
Hanno Cornelius 58dd431779
Fix Windows CI (#467)
* Force chronological message history

* Fix start/end index computation

* Sleep before RPC call

* Lower default max cache sizes on APIs

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-09 13:47:24 -07:00
Sanaz Taheri Boshrooyeh 6a4fcaa671
Fix the issue in the order of fields of HistoryQuery (#470)
Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-09 09:50:51 -07:00
Sanaz Taheri Boshrooyeh 5f0e6201ed
Change page size type to uint64 (#469)
* changes pageSize to uint64

* WIP

* reworks tests and indexing proc to match uint64 pageSize

* minor simplification

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-09 18:04:21 +08:00
RichΛrd b5f1becace
fix: direction encoding (#464)
* fix: direction encoding

* fix: decoding pagingInfo direction
2021-04-09 11:09:20 +08:00
Hanno Cornelius 5747ff5be0
Change contentTopic to string (#463)
* Change contentTopic to string

* Missed a spot

* Try to fix Windows CI

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-08 17:55:19 +08:00
Oskar Thorén 139f148037
Make wakunode2 and wakunode2 default to DEBUG log (#462)
Logs too noisy by default, full of gossipsub TRC making it hard to see what is going on on a local node
2021-04-08 13:40:49 +08:00
Sanaz Taheri Boshrooyeh d49cb53b56
Updates the field number of proof and timestamp (#457)
* updates the fields numbers of proof and timestamp

* clarifies that proof is not part of waku message specs

* Update waku/v2/protocol/waku_message.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* fixes indentation

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-07 11:08:46 -07:00
Sanaz Taheri Boshrooyeh a0dba2fea4
adds startTime and endTime to the HistoryQuery (#451) 2021-04-02 15:53:28 -07:00
Sanaz Taheri Boshrooyeh 192c2c7ea0
Add timestamp field to the waku message (#450)
* adds timestamp field

* unit test for timestamp

* adds comments

* updates test title
2021-04-02 14:28:46 -07:00
Hanno Cornelius 683577f045
Start relay protocol on wakunode2 (#446)
* Start relay protocol on wakunode2
2021-04-01 11:37:14 +02:00
Sanaz Taheri Boshrooyeh e7c21c2f74
Rln-relay zkp module Nim bindings (#427)
* entirely replaces the prior rln header, the var variables are changed to ptr

* updates the unittest of key_gen

* adds test for update_next_member

* updates membershipKeyGen internals and prototype

* adds createRLNInstance

* adds helpers methods

* adds generateKeyPairBuffer

* cleans up the test and adds comments

* renames  merkleTreeDepth to d

* fixes a buf re decoding the keys into sk and pk

* adds getSKPK proc

* unifies key gen helper procs, adds todos

* comments out the createRLNInstance

* refactors the code based on the updated createRLNInstance interface

* adds the test for the verify proc

* fixes a variable name and replaces random key gen with the real key gen

* tests a simple hash

* adds get_root method

* fixes the data pointer issue and adds the proof breakdown

* adds rln

* adds unit tests for Merkle tree

* adds a sample hash test

* fixes the hash bug and comments out unused part of proof gen test

* cleans up the proof gent test

* replaces unsafeAddr with addr

* fixes an issue in key gen

* updates rln submodule

* fixes the verification problem

* adds a failed test

* replaces an old test scenario with a new one

* handles createRLNInstance output

* working createRLNInstance2

* refactors the code by replacing the old createRLNInstance

* renames createRLNInstance2

* adds documentation and reorganizes rln.nim

* replace echo with debug, renames vars, adds a bad proof test

* minor

* minor

* edits var names

* adds one more check

* adds one more test to the hash

* enforcing exception handling

* adds pacman -Sy

* removes update:true

* activates update
2021-03-31 17:39:27 -07:00
Hanno Cornelius 27844c9a20
Specify chat2 message format (#440)
* Specify chat2 message format

* Clarify payload is UTF8 encoded
2021-03-31 17:06:20 +02:00
Franck Royer e4fc68b6ea
Update kibana link to only filter on fleet (#444)
Also, return last 7 days of logs instead of
fixed date in the past.
2021-03-31 14:32:18 +08:00
Hanno Cornelius aa9a74c60b
Fix nil error when starting wakunode2 with store/filter node (#441) 2021-03-30 15:33:19 +02:00
Kim De Mey d1c1a0ca13
Bump chronos and fix exception tracking issues (#436)
* Bump nim-chronos and fix exception tracking issues

* Bump other Nim submodules to latest

* Fix repeatMessage properly through proc type fix in nim-eth

Also add and use unittest2 through testutils to avoid extra
annotations.
2021-03-26 10:52:04 +01:00
Hanno Cornelius 5102576234
Integrate persistent peer storage (#437)
* Integrate persistent peer storage
2021-03-26 10:49:51 +02:00
Hanno Cornelius 7a732e7cc6
Added persistent storage for peers (#435) 2021-03-25 10:37:11 +02:00
Sanaz Taheri Boshrooyeh 29d69b98cb
Adds Merkle tree Nim bindings (#430)
* entirely replaces the prior rln header, the var variables are changed to ptr

* updates the unittest of key_gen

* adds test for update_next_member

* updates membershipKeyGen internals and prototype

* adds createRLNInstance

* adds helpers methods

* adds generateKeyPairBuffer

* cleans up the test and adds comments

* renames  merkleTreeDepth to d

* fixes a buf re decoding the keys into sk and pk

* adds getSKPK proc

* unifies key gen helper procs, adds todos

* comments out the createRLNInstance

* refactors the code based on the updated createRLNInstance interface

* adds the test for the verify proc

* fixes a variable name and replaces random key gen with the real key gen

* tests a simple hash

* adds get_root method

* fixes the data pointer issue and adds the proof breakdown

* adds rln

* adds unit tests for Merkle tree

* deletes unnecessary comments

* updates createRLNInstance to return bool indicating the success of call

* updates create RLN Instance interface

* minor

* clean up

* removes unused imports

* adds documentation

* adds comments

* adds byteutils

* removes extra spaces

* updates rln submodule

* deletes genSKPK

* fixes a bug in membershipKeyGen

* unsafeAddr to addr

* Update waku/v2/protocol/waku_rln_relay/rln.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* clean up

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-03-24 10:26:56 -07:00
Hanno Cornelius a30d6a8aae
Improvement/bridge improvements (#429)
* General test, import and log improvements

* Bridge improvements
2021-03-23 10:04:51 +02:00
Hanno Cornelius 2fe6935623
Fix marshalling of message payloads (#428)
Fix marshalling of message payloads
2021-03-22 17:13:56 +02:00
Hanno Cornelius 62b824c387
Add WakuBridge and test (#426)
* Add WakuBridge and test

* Fix test
2021-03-22 11:44:45 +02:00
Hanno Cornelius 4ae1cd4737
Add message passing to waku bridge script (#425) 2021-03-18 12:01:27 +02:00
Hanno Cornelius 0050f9da59
Add hook on newly added envelope (#423) 2021-03-17 11:49:06 +02:00
Hanno Cornelius 714bbebbc1
Update submodules (#424)
Update submodules and fix issues
2021-03-17 10:52:27 +02:00
Sanaz Taheri Boshrooyeh d64aadc8ad
Basic proof gen and vrfy (#421)
* adds the proof field to the WakuMessage

* adds a basic zkProof api

* adds proof gen to the publish proc

* relocates the proofGen and proofVrfy to rln_relay_utils

* wip: test of proof gen

* adds a procedure to pass rln-relay message validator, adds the proof gen

* tests the proof gen and verify

* relocates zkp API

* adds documentation and todos

* adds todos and documentations

* removes an unnecessary comment

* adds todos
2021-03-16 11:18:40 -07:00
Sanaz Taheri Boshrooyeh b3208b343f
Filtering messages in the relay protocol using libp2p topic validators (#414)
* adds a test for topic validators

* edits the import

* edits the comments

* further edits on the comments

* mode comments on the test

* checks the pubsub topic in the validator

* adds a fix

* changes the sleep time to 2 secs
2021-03-11 15:42:26 -08:00
Hanno Cornelius a575c44934
Fix Relay API: handle messages on previously subscribed topics (#413) 2021-03-11 09:48:59 +02:00
Hanno Cornelius e6b26cc059
Add post method to Admin API for ad-hoc peer dialing (#412)
* Add post method to Admin API

* Updated CHANGELOG
2021-03-11 09:05:39 +02:00
Hanno Cornelius 55096cbd1d
Improved description for config items (#410) 2021-03-10 09:03:22 +02:00
Hanno Cornelius b0abbcf597
Improved chat2 message encoding (#409) 2021-03-09 09:23:53 +02:00
Hanno Cornelius 398b701f4f
Minor fixes for chat2 and store (#407) 2021-03-08 09:45:10 +02:00
Hanno Cornelius 307a49bb78
Fix Docker image - missing librln.so file (#406) 2021-03-04 17:55:27 +02:00