Commit Graph

263 Commits

Author SHA1 Message Date
Sanaz Taheri Boshrooyeh 50a54cb4ee
Enabling pubsub topic filter in history queries (#492)
* replaces topics with seq of ContentFilters

* update topics to contentFilter

* updates the contentFilter  structure

 one content  topic per content filter instead of a sequence of topics

* updates store json rpc api

* renames ContentFilter to HistoryContentFilter

* unit test for a query with several content filters

* makes shortcut for store api

* updates chat2

* clean up

* renames topic to contentTopic

* adds pubsub topic to the history query

updates message store interface to return the pubsub topic
updates waku message store implementation
updates database schema to hold pubsub topi per waku message

* clarifies the use of content topic in store api

* clarifies the use of contentTopic in the init method of HistoryContentFilter

* simplifies the test and add comments

* lowers the field number of pubsub topic in historyQuery  protobuf

* captures an empty contentFilter case

* test pubsub topic filter for the entire history and no message match

* demoves duplicates

* adds TODO

* fix a broken comment line

* updates waku store codec

* swaps the order of pubsub topic and content topic in protobuf

* Update waku/v2/protocol/waku_store/waku_store_types.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* updates the pubsub topic to the default value

* bumps protocol id

* moves the comment close to IndexedWakuMessage

* adds checks to the store put method

* makes table title a constant variable and retitles the table to Message

* updates the changelog

* minor update

* minor

* beta2 to beta3

* minor

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-27 16:52:24 -07:00
Oskar Thorén 13ac035e5a
Lightpush protocol cont (#506)
* lightpush conf and mount with relay from node

* mount lightpush after relay

* lightpush relay integration wip

* lightpush node integrate and test node

* clean
2021-04-24 12:56:37 +08:00
Sanaz Taheri Boshrooyeh a04643e9a2
Updates store json rpc api (#499)
* updates the store jsonrpc api and its usage across the codebase

* deletes unused codes

* deletes commented codes

* minor

* spacing issue
2021-04-23 14:14:40 -07:00
Hanno Cornelius f1a6027158
Fix filter notifications (#501) 2021-04-22 15:45:13 +02:00
Hanno Cornelius f0eadfec13
Add persistent backoff for peers (#497) 2021-04-21 11:36:56 +02:00
Sanaz Taheri Boshrooyeh fb2ea06a4f
Update store protocol interface: add history content filter (#488)
* replaces topics with seq of ContentFilters

* update topics to contentFilter

* updates the contentFilter  structure

 one content  topic per content filter instead of a sequence of topics

* updates store json rpc api

* renames ContentFilter to HistoryContentFilter

* unit test for a query with several content filters

* makes shortcut for store api

* updates chat2

* clean up

* renames topic to contentTopic

* clarifies the use of content topic in store api

* clarifies the use of contentTopic in the init method of HistoryContentFilter
2021-04-19 10:38:30 -07:00
Hanno Cornelius 192d4dd66b
Add backoff period before subscribing to persistent peers (#491) 2021-04-16 11:57:45 +02:00
Hanno Cornelius 900d53f9df
Implement graceful shutdown for `wakunode2` and `chat2` (#490)
* Implement graceful shutdown

* Consistent use of QuitSuccess
2021-04-15 10:18:14 +02:00
Hanno Cornelius 17cfc32c4f
Add default handler for all topics (#483)
Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-13 07:56:49 +02:00
Hanno Cornelius 58dd431779
Fix Windows CI (#467)
* Force chronological message history

* Fix start/end index computation

* Sleep before RPC call

* Lower default max cache sizes on APIs

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-09 13:47:24 -07:00
Hanno Cornelius 5747ff5be0
Change contentTopic to string (#463)
* Change contentTopic to string

* Missed a spot

* Try to fix Windows CI

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-04-08 17:55:19 +08:00
Hanno Cornelius 683577f045
Start relay protocol on wakunode2 (#446)
* Start relay protocol on wakunode2
2021-04-01 11:37:14 +02:00
Sanaz Taheri Boshrooyeh e7c21c2f74
Rln-relay zkp module Nim bindings (#427)
* entirely replaces the prior rln header, the var variables are changed to ptr

* updates the unittest of key_gen

* adds test for update_next_member

* updates membershipKeyGen internals and prototype

* adds createRLNInstance

* adds helpers methods

* adds generateKeyPairBuffer

* cleans up the test and adds comments

* renames  merkleTreeDepth to d

* fixes a buf re decoding the keys into sk and pk

* adds getSKPK proc

* unifies key gen helper procs, adds todos

* comments out the createRLNInstance

* refactors the code based on the updated createRLNInstance interface

* adds the test for the verify proc

* fixes a variable name and replaces random key gen with the real key gen

* tests a simple hash

* adds get_root method

* fixes the data pointer issue and adds the proof breakdown

* adds rln

* adds unit tests for Merkle tree

* adds a sample hash test

* fixes the hash bug and comments out unused part of proof gen test

* cleans up the proof gent test

* replaces unsafeAddr with addr

* fixes an issue in key gen

* updates rln submodule

* fixes the verification problem

* adds a failed test

* replaces an old test scenario with a new one

* handles createRLNInstance output

* working createRLNInstance2

* refactors the code by replacing the old createRLNInstance

* renames createRLNInstance2

* adds documentation and reorganizes rln.nim

* replace echo with debug, renames vars, adds a bad proof test

* minor

* minor

* edits var names

* adds one more check

* adds one more test to the hash

* enforcing exception handling

* adds pacman -Sy

* removes update:true

* activates update
2021-03-31 17:39:27 -07:00
Hanno Cornelius aa9a74c60b
Fix nil error when starting wakunode2 with store/filter node (#441) 2021-03-30 15:33:19 +02:00
Kim De Mey d1c1a0ca13
Bump chronos and fix exception tracking issues (#436)
* Bump nim-chronos and fix exception tracking issues

* Bump other Nim submodules to latest

* Fix repeatMessage properly through proc type fix in nim-eth

Also add and use unittest2 through testutils to avoid extra
annotations.
2021-03-26 10:52:04 +01:00
Hanno Cornelius 5102576234
Integrate persistent peer storage (#437)
* Integrate persistent peer storage
2021-03-26 10:49:51 +02:00
Hanno Cornelius 7a732e7cc6
Added persistent storage for peers (#435) 2021-03-25 10:37:11 +02:00
Sanaz Taheri Boshrooyeh 29d69b98cb
Adds Merkle tree Nim bindings (#430)
* entirely replaces the prior rln header, the var variables are changed to ptr

* updates the unittest of key_gen

* adds test for update_next_member

* updates membershipKeyGen internals and prototype

* adds createRLNInstance

* adds helpers methods

* adds generateKeyPairBuffer

* cleans up the test and adds comments

* renames  merkleTreeDepth to d

* fixes a buf re decoding the keys into sk and pk

* adds getSKPK proc

* unifies key gen helper procs, adds todos

* comments out the createRLNInstance

* refactors the code based on the updated createRLNInstance interface

* adds the test for the verify proc

* fixes a variable name and replaces random key gen with the real key gen

* tests a simple hash

* adds get_root method

* fixes the data pointer issue and adds the proof breakdown

* adds rln

* adds unit tests for Merkle tree

* deletes unnecessary comments

* updates createRLNInstance to return bool indicating the success of call

* updates create RLN Instance interface

* minor

* clean up

* removes unused imports

* adds documentation

* adds comments

* adds byteutils

* removes extra spaces

* updates rln submodule

* deletes genSKPK

* fixes a bug in membershipKeyGen

* unsafeAddr to addr

* Update waku/v2/protocol/waku_rln_relay/rln.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* clean up

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-03-24 10:26:56 -07:00
Hanno Cornelius a30d6a8aae
Improvement/bridge improvements (#429)
* General test, import and log improvements

* Bridge improvements
2021-03-23 10:04:51 +02:00
Hanno Cornelius 2fe6935623
Fix marshalling of message payloads (#428)
Fix marshalling of message payloads
2021-03-22 17:13:56 +02:00
Sanaz Taheri Boshrooyeh d64aadc8ad
Basic proof gen and vrfy (#421)
* adds the proof field to the WakuMessage

* adds a basic zkProof api

* adds proof gen to the publish proc

* relocates the proofGen and proofVrfy to rln_relay_utils

* wip: test of proof gen

* adds a procedure to pass rln-relay message validator, adds the proof gen

* tests the proof gen and verify

* relocates zkp API

* adds documentation and todos

* adds todos and documentations

* removes an unnecessary comment

* adds todos
2021-03-16 11:18:40 -07:00
Hanno Cornelius a575c44934
Fix Relay API: handle messages on previously subscribed topics (#413) 2021-03-11 09:48:59 +02:00
Hanno Cornelius e6b26cc059
Add post method to Admin API for ad-hoc peer dialing (#412)
* Add post method to Admin API

* Updated CHANGELOG
2021-03-11 09:05:39 +02:00
Hanno Cornelius 55096cbd1d
Improved description for config items (#410) 2021-03-10 09:03:22 +02:00
Hanno Cornelius 398b701f4f
Minor fixes for chat2 and store (#407) 2021-03-08 09:45:10 +02:00
Sanaz Taheri Boshrooyeh e152ed349f
Integrate registration into wakunode and Enable signed registration transactions (#385)
* cleans up imported modules

* adds uploadContract proc and consolidates rln-relay test files

* deletes test_rln_relay_wrappers

* deletes wrappers tests

* adds waku_rln_relay_utils

* adds the unit test for the membership key generation

* adds the key generation procedure

* adds unit test of key gen proc

* adds RLNRelayPeer data type

* adds the register proc

* adds the register proc unit test

* minor

* edits registration test

* adds comments to the registration unit test and relocates some constants

* defines constants variables for membership contract inputs and adds todos

* fixes a typo

* enables signed transaction

* adds registration into waku rln relay

* adds a TODO

* adds debug notes

* WIP add registration test

* integrates rln relay inputs into the mountRelay proc

* minor

* adds mountRelay

* renames RlnRelayPeer to WakuRlnRelay

* adds debugging notes

* changes the test title

* adds rln to gitignore

* cleans up tests, fixes a bug

* exposes mountRlnrelay, adds some comments

* fixes a bug

* deletes async pragma from mountRelay

* exposes the wakuRlnRelay field

* adds bash command to delete any existing rln directory before cloning the rln repo

* removes -v verbose flag from rln removal

* replaces await with WaitFor

* embed mountRlnRelay inside mountRelay
2021-02-22 09:40:02 -08:00
Hanno Cornelius 3b6db72287
Feat/pm connection tracking (#377)
* Track connectedness state in peer manager
2021-02-12 10:53:52 +02:00
Hanno Cornelius bf0eab4a48
Removed local peer sets for filter, swap and store (#375) 2021-02-11 10:58:25 +02:00
Hanno Cornelius 1b96ee0adb
Integrate peer manager with Swap and Store (#370) 2021-02-09 10:31:38 +02:00
Hanno Cornelius 1d9e3afaa4
Peer manager: improvements; waku_filter integration (#368)
* Integrate peer manager with waku_filter

* Changelog and misc PR suggestions
2021-02-08 11:17:20 +02:00
Hanno Cornelius 1f5c3cc621
Feat/peer manager improvements (#367)
* Add managed peers to Admin API result

* Deal with dial failures
2021-02-05 12:49:11 +02:00
Hanno Cornelius 080d82dd9a
Added basic peer manager (#364)
* Added basic peer manager
2021-02-04 12:32:58 +02:00
Hanno Cornelius 9faff49e5b
Remove stale rpc code & repair scripts (#365)
* Remove stale rpc code & repair scripts
2021-02-04 10:39:55 +02:00
Hanno Cornelius 41fe4395a8
Bump submodules (#363)
* Bump submodules

* GossipSub refactor: Rem async from sub/unsub
2021-02-02 13:33:59 +02:00
Hanno Cornelius 886b458ff5
Added waku metrics (#356) 2021-01-29 10:42:41 +02:00
Hanno Cornelius ed979a7d60
Fix network simulation and remote peer ID CLI argument parsing (#351)
* Fix network simulation and remote peer ID CLI argument parsing
2021-01-25 13:03:52 +02:00
Hanno Cornelius e8dd014079
Extract MessageStore interface (#350) 2021-01-22 11:39:16 +02:00
Hanno Cornelius 76c0d2b839
Prefer announced addrs in listenStr (#348)
* Prefer announced addrs in listenStr
2021-01-15 09:37:20 +02:00
Sanaz Taheri Boshrooyeh cc1fc15d38
Adds rln-relay flag (#347)
* adds rlnrelay flag

* minor edit

* adds the input name
2021-01-08 09:56:58 -08:00
Sanaz Taheri Boshrooyeh 96f95ed0b6
Test for the Rln relay/membership management contract deployment (#312)
* enhancement/cleanup-warnings (#290)


Accounting: Move flag to config and add basic test for accounting state (#284)

* Accounting: Add config flag (default to false)

Also fix bug where query function is called twice when flag is set.

* Accounting: test state update after store req
Refactor: waku_types separation of concern (trial with waku_swap) (#282)

* waku_types refactor: Accounting types to waku_swap

* waku_types refactor: Separate waku_swap_types file

* Fix import path for waku_swap
adds the test file for waku rln relay


WIP: adds some initial code


commented out the rln-relay test code


refactor/db-message-store-split (#280)

* fixes

* fic

* fix

* fix

* fix

* fixed

* fix

* fixes

* fixes, using sqlite lib fully

* fix

* Update sqlite.nim
Waku v2 JSON-RPC REST API: Store protocol proof of concept (#263)

* Waku V2 history query POC

* Fix folder structure

* Improve test clarity

* Improve imports, returns and some naming

* Changed naming conventions. Refactor & improve.

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Refactor types store (#291)

* Refactor: Move waku_store into its own folder

* Refactor: Move waku store types to new home (WIP)

* Refactor: Fix errors and recursive imports

* Fix rebase errors

* Refactor: More rebase import fixes
Accounting credit receiving node (#292)

* Accounting WIP: Swap access through Store

* Fix order bug and comment scenario + typo

* WIP

* Accounting: Account for receiving store node

- Turn accountFor function into credit and debit
- Misc formatting

* Accounting: Fix bugs related to mount and test

* Accounting: Simplify query signature

We already have a ref to wakuSwap through wakuStore now.

* Resolve rebase issues
enhancement/remove-flood (#295)

* removes flood

* rm

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Updated submodules (#303)


add Jenkinsfile and some Makefile targets for CI (#302)

Signed-off-by: Jakub Sokołowski <jakub@status.im>

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Accounting: End to end with Cheque PoC (#304)


Added some basic debug and relay json-rpc calls (#305)


Uncomment and call empty rln test

adds the membership contract to the test file


adds a test utils for contract deployment


increases the gas limit for group management contract

* removes some files

* WIP: adds the interface of membership contract

* uncomments the interface of getNumber contract

* adds tests for interaction with the membership contract

* send is replaced with call

* replaces call with send

* enhancement/cleanup-warnings (#290)


Accounting: Move flag to config and add basic test for accounting state (#284)

* Accounting: Add config flag (default to false)

Also fix bug where query function is called twice when flag is set.

* Accounting: test state update after store req
Refactor: waku_types separation of concern (trial with waku_swap) (#282)

* waku_types refactor: Accounting types to waku_swap

* waku_types refactor: Separate waku_swap_types file

* Fix import path for waku_swap
adds the test file for waku rln relay


WIP: adds some initial code


commented out the rln-relay test code


refactor/db-message-store-split (#280)

* fixes

* fic

* fix

* fix

* fix

* fixed

* fix

* fixes

* fixes, using sqlite lib fully

* fix

* Update sqlite.nim
Waku v2 JSON-RPC REST API: Store protocol proof of concept (#263)

* Waku V2 history query POC

* Fix folder structure

* Improve test clarity

* Improve imports, returns and some naming

* Changed naming conventions. Refactor & improve.

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Refactor types store (#291)

* Refactor: Move waku_store into its own folder

* Refactor: Move waku store types to new home (WIP)

* Refactor: Fix errors and recursive imports

* Fix rebase errors

* Refactor: More rebase import fixes
Accounting credit receiving node (#292)

* Accounting WIP: Swap access through Store

* Fix order bug and comment scenario + typo

* WIP

* Accounting: Account for receiving store node

- Turn accountFor function into credit and debit
- Misc formatting

* Accounting: Fix bugs related to mount and test

* Accounting: Simplify query signature

We already have a ref to wakuSwap through wakuStore now.

* Resolve rebase issues
enhancement/remove-flood (#295)

* removes flood

* rm

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Updated submodules (#303)


add Jenkinsfile and some Makefile targets for CI (#302)

Signed-off-by: Jakub Sokołowski <jakub@status.im>

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Accounting: End to end with Cheque PoC (#304)


Added some basic debug and relay json-rpc calls (#305)


Uncomment and call empty rln test

adds the membership contract to the test file


adds a test utils for contract deployment


increases the gas limit for group management contract

* removes some files

* WIP: adds the interface of membership contract

* uncomments the interface of getNumber contract

* adds tests for interaction with the membership contract

* send is replaced with call

* replaces call with send

* adds a todo

* removes empty lines

* adds some comments

* adds a todo

* adds the solidity code of the NumberContract

* fixes a bug: replaces call with send

* removes store test comments

* removes excess spaces
2021-01-07 12:34:24 -08:00
Oskar Thorén 37a3a4fadd
Refactor: Put waku_type content where it belongs (#333)
- Separate out types and methods into appropriate module
- Add util folder for small utils used in multiple independent places
- Update paths
2021-01-06 17:35:05 +08:00
Oskar Thorén f860c0dda4
Refactor: waku_types -> waku_filter (#331)
Also create a folder with README for filter protocol.
2021-01-05 12:52:10 +08:00
Hanno Cornelius 8203f30099
Integrate API and add tutorial (#330) 2020-12-24 10:02:30 +02:00
Hanno Cornelius 97dc627bbe
Feature/add private api (#321)
* Added private API
2020-12-23 11:33:28 +02:00
decanus 67f852776b
fix 2020-12-22 16:53:01 +01:00
Hanno Cornelius f0636bcd9d
enhancement/unused-import-cleanup (#322) 2020-12-21 10:14:51 +01:00
Hanno Cornelius d6bafcf697
Feature/add admin api (#319)
* Add Admin API
2020-12-15 17:48:00 +02:00
Hanno Cornelius fa4d873ee3
Limit cache size on Filter API and Relay API (#317)
Minor improvements to Filter API and Relay API
2020-12-07 15:26:58 +02:00
Hanno Cornelius 57f69201d6
Added content topic polling on FilterAPI (#314) 2020-12-03 18:26:34 +02:00
Hanno Cornelius 8a1ca1ff8f
Await WakuRelay publish on node (#313)
* Await WakuRelay publish on node

* Reflect 'await' change on CHANGELOG
2020-12-02 10:40:53 +02:00
Hanno Cornelius d3c5840a79
Feature/jsonrpc basic impl (#311)
* Added some basic debug and relay json-rpc calls

* Basic relay polling
2020-12-01 11:57:54 +02:00
Hanno Cornelius 51b35c59c6
Added some basic debug and relay json-rpc calls (#309) 2020-11-30 11:44:57 +02:00
Hanno Cornelius dc3b3b87be
Added some basic debug and relay json-rpc calls (#305) 2020-11-27 09:18:48 +02:00
Oskar Thorén 35f9e52d49
Accounting: End to end with Cheque PoC (#304) 2020-11-26 18:02:10 +08:00
Oskar Thorén 63b0e8af76
Accounting credit receiving node (#292)
* Accounting WIP: Swap access through Store

* Fix order bug and comment scenario + typo

* WIP

* Accounting: Account for receiving store node

- Turn accountFor function into credit and debit
- Misc formatting

* Accounting: Fix bugs related to mount and test

* Accounting: Simplify query signature

We already have a ref to wakuSwap through wakuStore now.

* Resolve rebase issues
2020-11-24 12:53:42 +08:00
Oskar Thorén 44e9d4d86b
Refactor types store (#291)
* Refactor: Move waku_store into its own folder

* Refactor: Move waku store types to new home (WIP)

* Refactor: Fix errors and recursive imports

* Fix rebase errors

* Refactor: More rebase import fixes
2020-11-24 12:34:32 +08:00
Hanno Cornelius 135eaae9fb
Waku v2 JSON-RPC REST API: Store protocol proof of concept (#263)
* Waku V2 history query POC

* Fix folder structure

* Improve test clarity

* Improve imports, returns and some naming

* Changed naming conventions. Refactor & improve.

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2020-11-24 11:44:37 +08:00
Dean Eigenmann e875027be4
refactor/db-message-store-split (#280)
* fixes

* fic

* fix

* fix

* fix

* fixed

* fix

* fixes

* fixes, using sqlite lib fully

* fix

* Update sqlite.nim
2020-11-24 10:50:59 +08:00
Oskar Thorén 54ac399f5f
Refactor: waku_types separation of concern (trial with waku_swap) (#282)
* waku_types refactor: Accounting types to waku_swap

* waku_types refactor: Separate waku_swap_types file

* Fix import path for waku_swap
2020-11-23 10:27:45 +08:00
Oskar Thorén f8aff5756c
Accounting: Move flag to config and add basic test for accounting state (#284)
* Accounting: Add config flag (default to false)

Also fix bug where query function is called twice when flag is set.

* Accounting: test state update after store req
2020-11-21 13:31:48 +08:00
Dean Eigenmann 1193b69d6c
enhancement/cleanup-warnings (#290) 2020-11-20 14:59:29 +01:00
Oskar Thorén 010408aadb
Minimal accounting state PoC (#278)
* Add basic WakuSwap type and init

* Mount swap protocol and keep accounting state

* Flags off by default
2020-11-18 20:45:51 +08:00
Oskar Thorén 9518322198
Folder restructure (#274)
* Change folder structure to {v1,v2,common}/...

Addresses https://github.com/status-im/nim-waku/issues/261

* Update waku.nimble paths

* Flatten paths

* Fix import paths

* Pull out utils folder for nat

* Pull out waku_types to top level for v2

* Fix test import paths

* Remove old READMEs and replace with one liner

* Update README and split v1 and v2

* Skeleton READMEs

* Update README.md

Co-authored-by: Kim De Mey <kim.demey@gmail.com>

* Update README.md

Co-authored-by: Kim De Mey <kim.demey@gmail.com>

Co-authored-by: Kim De Mey <kim.demey@gmail.com>
2020-11-17 17:34:53 +08:00