Commit Graph

168 Commits

Author SHA1 Message Date
Hanno Cornelius 135eaae9fb
Waku v2 JSON-RPC REST API: Store protocol proof of concept (#263)
* Waku V2 history query POC

* Fix folder structure

* Improve test clarity

* Improve imports, returns and some naming

* Changed naming conventions. Refactor & improve.

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2020-11-24 11:44:37 +08:00
Dean Eigenmann e875027be4
refactor/db-message-store-split (#280)
* fixes

* fic

* fix

* fix

* fix

* fixed

* fix

* fixes

* fixes, using sqlite lib fully

* fix

* Update sqlite.nim
2020-11-24 10:50:59 +08:00
Oskar Thorén 54ac399f5f
Refactor: waku_types separation of concern (trial with waku_swap) (#282)
* waku_types refactor: Accounting types to waku_swap

* waku_types refactor: Separate waku_swap_types file

* Fix import path for waku_swap
2020-11-23 10:27:45 +08:00
Oskar Thorén f8aff5756c
Accounting: Move flag to config and add basic test for accounting state (#284)
* Accounting: Add config flag (default to false)

Also fix bug where query function is called twice when flag is set.

* Accounting: test state update after store req
2020-11-21 13:31:48 +08:00
Dean Eigenmann 1193b69d6c
enhancement/cleanup-warnings (#290) 2020-11-20 14:59:29 +01:00
Oskar Thorén 9518322198
Folder restructure (#274)
* Change folder structure to {v1,v2,common}/...

Addresses https://github.com/status-im/nim-waku/issues/261

* Update waku.nimble paths

* Flatten paths

* Fix import paths

* Pull out utils folder for nat

* Pull out waku_types to top level for v2

* Fix test import paths

* Remove old READMEs and replace with one liner

* Update README and split v1 and v2

* Skeleton READMEs

* Update README.md

Co-authored-by: Kim De Mey <kim.demey@gmail.com>

* Update README.md

Co-authored-by: Kim De Mey <kim.demey@gmail.com>

Co-authored-by: Kim De Mey <kim.demey@gmail.com>
2020-11-17 17:34:53 +08:00
Dean Eigenmann 1568fa6e6f
feature/persistence-simple (#268)
* implements

* add

* fix

* eol

* rebase

* added tests

* minor cleanup

* errors

* Update store.md

* fix
2020-11-16 09:38:52 +01:00
Oskar Thorén a1bb2c4954
Accounting SWAP protocol basic data types and test (#262)
* Start of SWAP

* SWAP basic data types (untested)

* SWAP Add handshake test and fix imports

* SWAP Add cheque encode/decode test and fix bug
2020-11-13 14:02:17 +08:00
Hanno Cornelius 24f4a087de
Feature/unsubscribe from content filter (#255)
Unsubscribe from a content topic

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2020-11-10 09:13:16 +02:00
Sanaz Taheri Boshrooyeh ea5f9993a7
Pagination feature/indexing waku messages (#233)
* changes the digest type to MDigest[256] and modifies the computeIndex

* fixes formatting issue

* adds the pagination with its tests

stores and retrieves IndexedWakuMessage


adds the paginate proc


adds the paginate function


fixes some formatting issues


minor


edits indentation and fixes a bug


removes unused imports


minor


fixes indentations and adds a new testcase


adds indexedWakuMessageComparison


adds `==` proc for IndexedWakuMessage


separates the comparison of index and indexed waku messages


adds testcases for the Index comparison and IndexedWakuMessage comparison


WIP


WIP: adds an decoder for Index


removes an unnecessary imports


WIP


adds findIndex() proc


removes the equality check '==' for IndexedWakuMessages


edits the code format and adds the pagination test


edits paginate() proc to work on a copy of the input list


deletes unnecessary echo


adds the boundary calculations for forward and backward pagination


adds test cases for the page boundaries


tests corner cases for the queried cursor and pagesize


minor


adds some comments


adds a proc to extract WakuMessages from a list of IndexedWakuMessages


integrates pagination into the findMessages proc


adds some comments


changes paginate to paginateWithIndex


removes some echos


modifies paginateWithIndex to handle invalid cursors


adds test case for an invalid cursor


WIP: adds a `$` proc for IndexedWakuMessages


adds some debugging message prints 


adds an integration test for handling query with pagination

* fixes a type mismatch issue in the min proc

* replaces boolean direction with their enums and updates contentTopics

* adds the unit test for the sorting of the indexed waku messages

* fixes a flaky test

* fixes a flaky test

* removes index equality check proc

* handles an initial query with an empty cursor

* adds test for the initial query

* adds integration test for pagination

* adds a test for empty message list

* adds comments and fixes an issue

* adds comments

* code cleanup

* removes the content topic validation check

* resolves the errors related to the windows CI tests

* Update waku/protocol/v2/waku_store.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* Update waku/protocol/v2/waku_store.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* Update tests/v2/test_waku_pagination.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* Update waku/protocol/v2/waku_store.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* Update waku/protocol/v2/waku_store.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* Update waku/protocol/v2/waku_store.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* changes the output type of findIndex to Option

* Update tests/v2/test_waku_pagination.nim

Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com>

* Update tests/v2/test_waku_pagination.nim

Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com>

* Update tests/v2/test_waku_pagination.nim

Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com>

* adds some comments

* fixes an indentation issue

* some code modification for array initialization

* Apply suggestions from code review

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com>

* does some code reorganizations and clean up

* CreateSampleList to createSampleList

* replaces a byte array literal initialization with a for loop

* relocates indexedWakuMessageComparison and indexComparison

* minor

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
Co-authored-by: Dean Eigenmann <7621705+decanus@users.noreply.github.com>
2020-11-09 12:48:09 +08:00
Kim De Mey 0b70fe61a4
Add encoding/decoding of payload for Waku v2 (#251)
* Add encoding/decoding of payload for Waku v2

* Allow for Waku v1 payload in chat through compile time flag
2020-11-03 21:20:40 +01:00
Sanaz Taheri Boshrooyeh 7de91599d4
fixes a bug related to contentTopic type mismatch (#252)
* fixes a bug related to contentTopic type mismatch

* adds used pragma to get rid of unused imports warning
2020-10-30 20:00:14 +01:00
Sanaz Taheri Boshrooyeh 0cb2e125b6
adds protobuf methods for pagination (#248) 2020-10-30 09:55:31 +08:00
Oskar Thoren 4b21bb6b9b
Default topic: waku -> /waku/2/default-waku/proto 2020-10-28 13:19:43 +08:00
Dean Eigenmann bfb29338a1
fixes/wait-for-start (#228)
* fixes subscribe to wait for start

* moved around

* added test

* fix

* eol

* rm

* removed as waku is default

* updated docs

* fix

* Update node.md

* Update node.md

* Update wakunode2.nim

* Update wakunode2.nim

* fix test

* rm

* fix

* fixes

* fixes

* fix

* fix

* using connect

* fix

* fix

* Update wakubridge.nim

* Update wakunode2.nim
2020-10-22 13:12:00 +02:00
Dean Eigenmann 5ddd8701c9
fix/content-type-int (#235)
* fixes

* fixes

* using a better topic

* fix

* fixed
2020-10-21 16:55:06 +08:00
Dean Eigenmann b2edfe8dda
feature/start-protocol-command (#209)
* started working

* fixes

* rename start -> mount

* started logging

* mounting relay

* Update wakunode2.nim

* start of fixe

* fix

* fix

* fixes

* Update waku/node/v2/wakunode2.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* removed comment

* fix

* update

* changed the default

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2020-10-20 10:36:27 +08:00
Sanaz Taheri Boshrooyeh d6c48da071
Pagination: Pagination feature/compute index (#225)
* adds the index object

* adds the indexedWakuMessage

* adds the PagingInfo

* Adds PagingInfo to HistoryResponse and HistoryQuery

* adds the computeIndex procedure

* Update waku/node/v2/waku_types.nim

Fixes spacing issues

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* Changes timestamp to receivedTime and checks for the  empty contentTopic

* adds a test file for pagination with test scenarios for computeIndex

* changes receivedTimestamp to the unix timestamp

* updates a test case

* replaces std/sha1 with  nimcrypto/sha2

* changes the tests titles

* minor comments

* Some clean up

* fixes some formatting issue

* edits a test-case

* adds comments

* changes the digest type to MDigest[256] and modifies the computeIndex

* fixes formatting issue

* edits indentations and fixes a bug

* minor edits

* changes suite to procSuite and adds a new text case

* fixes all the indentations

* cleanup of the imports

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2020-10-20 10:20:44 +08:00
Kim De Mey daeb736335
Cleanup warnings in v1 and split test target in Makefile (#211)
* Cleanup warnings in v1 and split test target in Makefile

* Remove protocol2 target and get chat2 build in CI
2020-10-08 11:10:45 +02:00
Kim De Mey 093efbf736
Begin of waku v2 rpc tests (#210) 2020-10-06 20:36:44 +02:00
Dean Eigenmann a975bf6d70
feature/mount-filter-node-api (#181)
* started working on filter mount

* fixes

* fix indentation

* fixes

* save

* change

* rm

* Update waku_types.nim

* Update waku_filter.nim

* fixes

* fix

* using new filter

* did some testing stuff

* rm

* fix

* updated

* update doc

* fix

* fix

* unasynced

* fix

* w -> node

* rename

* move to content filter

* fix

* method

* rename

* renamed contentFilter -> contentFilters

* readded

* moved

* first test fix

* readded

* no more need

* fixes

* fix

* fix

* fixes

* todo

* removes mother fuck

* Update waku_types.nim
2020-10-02 14:48:56 +02:00
Dean Eigenmann 3eb015ee7b
feature/use-crypto-rng (#190)
* fixes

* fix

* Update waku_types.nim

* fixes

* fix

* Improve the generateRequestId call

Co-authored-by: kdeme <kim.demey@gmail.com>
2020-09-28 23:44:14 +02:00
Dean Eigenmann 25b48bb99e
enhancement/store-message-handler (#185)
* started working on wrapper

* fixes

* changes

* fixes

* Update waku_store.nim
2020-09-25 16:02:13 +02:00
Dean Eigenmann 1f68e63185
feature/store-query-node-api (#174)
* changes

* fix

* changes

* made the query function work

* added rpc

* whoops

* Update wakurpc.nim

* minor

* fixes

* update docs

* fix

* error handling

* end-to-end waku node test

* Update test_wakunode.nim

* Update waku/node/v2/rpc/wakurpc.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* Update test_wakunode.nim

* Update wakunode2.nim

* fix

* fix

* shorter

* rm echo

* fix

* added history peer

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2020-09-24 10:16:25 +08:00
Dean Eigenmann e1414ac922
fix/make-filter-work (#182)
* starts to actually get the filter protocol working

* made it work

* Update all_tests_v2.nim

* Update test_waku_filter.nim
2020-09-23 06:19:37 +02:00
Dean Eigenmann 6b866490ad
fix/message-notifier-async (#169)
* made async

* fixes

* unused

* eol
2020-09-17 22:10:41 +02:00
Oskar Thorén a8dbf8a7b6
Bump submodules (#157)
* Upgrade all submodules

* Use stock standard_setup and remove our version

Switch no longer relies on Pubsub argument

* Fix peerId

* Add reference to WakuRelay in WakuNode

* Use WakuRelay ref directly instead of via switch

* Tweak standard switch sig

* Fix start_network peerid

* Import nim-libp2p utils test

* Use WakuRelay in test utils

* Fix utils imports

* Tweak

* Fix trigger self test

* Disable broken filter test

* Fix and amend logscope topics

* Make subscribe calls async to use await

* Add debug stuff to nimble file

* Await for subscribe content

* Sleeping in tests

* Local checkout fixes

* XXX: Try to use .PubSub on WakuRelay

* Revert "XXX: Try to use .PubSub on WakuRelay"

This reverts commit 3a3139e4cf.

* Only using gossip seems to work

Subscribe for floodsub broken

* Fix await in examples

* Get rid of double publish, still need sleep
2020-09-16 12:23:10 +08:00
Dean Eigenmann dfee0359af
fix/message-notifier-message-to-waku-message (#159)
* update message type

* cleanup
2020-09-16 10:59:10 +08:00
Dean Eigenmann 4a5acd224a
spec/alpha-v4 (#151) 2020-09-10 13:16:31 +02:00
Dean Eigenmann 76e0b49e2b
feature/mount-filter-protocol (#125) 2020-09-07 14:22:46 +02:00
Dean Eigenmann a4c6caa62f
refactor/rename-filter-subscription (#138)
* fixes

* added

* added comment
2020-09-07 13:26:32 +02:00
Dean Eigenmann 5b4e429a19
enhancement/push-based-filter (#136)
* updated protobuf

* updated test
2020-09-07 11:12:27 +02:00
Dean Eigenmann 196c54d6ab
enhancement/filter-rpc-tests (#130)
* added a test for rpc filter

* rm

* fix
2020-09-03 16:42:56 +08:00
Kim De Mey 2717209f05
Move filter.notify to WakuRelay handler and add test (#128) 2020-09-02 11:15:25 +08:00
Oskar Thorén f828736e1c
Add support for WakuMessage and merge publish function (#122)
* Add WakuMessage type

* Add WakuMessage encoding and decoding

Also clean up waku_types module and imports

* Clean up waku_relay module

Imports, remove old text test, make fields and functions public, format.

* Publish WakuMessage

Also fix type mismatch in RPC

* Make publish work in examples, node and protocol

* Parse protobuf content in examples

* Update docs

* Update waku/node/v2/waku_types.nim

* Fix compilation error and disable out of date waku test

Co-authored-by: Kim De Mey <kim.demey@gmail.com>
2020-09-01 17:20:38 +02:00
Kim De Mey fd2bb36b15
Fix compile failure + CI endless example run (#126) 2020-09-01 15:43:06 +02:00
Oskar Thorén 67a964be28
Partial filter (#120)
* start work on filter protocol

* init functions for protobufs

* comment

* added init

* playing around with subscribers

* Update waku_filter.nim

* added test for protobuf

* fixes

* fix

* Update test_waku_filter.nim

* Update waku_filter.nim

* Add note on status

Co-authored-by: decanus <7621705+decanus@users.noreply.github.com>
2020-09-01 11:03:17 +08:00
Kim De Mey ea0d62993d
WakuNode init without WakuNodeConf (#117)
* WakuNode init without WakuNodeConf

* WakuNode start without WakuNodeConf

* setupNat adjustments for making common version

* Move setupNat to common.nim to be used for v1 and v2
2020-09-01 10:09:54 +08:00
Kim De Mey ddcfc3fdd6
V2cleanup (#115)
* Cleanup of warnings + imports, etc.

* Remove connected_peers metric as there is already libp2p_pubsub_peers
2020-08-31 11:32:41 +08:00
Dean Eigenmann 1924bcc96c
enhancement/request-id-to-uuid (#113) 2020-08-28 13:07:52 +02:00
Oskar Thorén 705b4b32a1
Waku v2: Simplify Waku Relay references (#112)
- Remove wakuRelayProto as this is initiated as part of switch PubSub
- Move WakuNode type into waku_types to avoid circular references
- Make necessary adjustments to get tests and sim to work
2020-08-28 17:08:28 +08:00
Dean Eigenmann 0e62e8ffa4
feature/store-protocol (#102)
* started working on store protocol

* eol

* update

* fix

* encoding

* started

* started working on decoding

* fixes

* started fleshing out tests

* testing

* encode / decode test

* eol

* fix test

* fmt

* errors

* testing entire rpc now

* fix

* format

* added comment

* removed StoreRPC

* added comment

* readded the store rpc

* updated

* fix tests
2020-08-27 10:44:09 +08:00
Oskar Thorén 893134b536
Refactor: Waku protocol -> Waku relay protocol (#104)
* Refactor: Waku protocol -> Waku relay protocol

Including updating protocol string.

Addresses https://github.com/status-im/nim-waku/issues/101

* Refactor: WakuSub -> WakuRelay
2020-08-26 19:28:24 +08:00
Dean Eigenmann c3f57c79f5
feature/historic-api (#95)
* started working on historic api

* very rough code

* moved

* also publishing

* todos

* added tests

* added

* fix
2020-07-29 15:24:01 +02:00
Dean Eigenmann f0d40f1acc
enhancement/move-standard-setup (#94)
* moves out

* oops

* setup

* fix
2020-07-28 13:00:34 +02:00
Oskar Thorén bede5a9358
Node: Separate run into create Wakunode and start (#59) 2020-07-24 09:39:58 +08:00
Dean Eigenmann fb45502ba7
enhancement/protobuf (#47)
* changed to data

* fix

* updated

* checkpoint

* sending protobuf

* working on tests

* bump

* fix

* fix

* fix

* testing

* Update test_waku.nim
2020-07-23 10:53:29 +08:00
Dean Eigenmann d37928bb28
bump/libp2p (#62) 2020-07-21 14:19:31 +02:00
Kim De Mey 95438b9389
Bump vendor, fix builds and add v2 to CI (#45)
* Fix v2 builds, break v1 builds

* Fix build after nim-eth bump

* Add waku v2 build and tests to CI

* Fix parallel build issues by renaming
2020-07-13 12:08:03 +02:00
Kim De Mey 181889dcd8
Cleanup metrics + accounting flag + bump deps (#37)
* Cleaner metrics logging + accounting behind flag + some cleanup

* Bump vendor
2020-07-02 23:52:54 +02:00
Kim De Mey 481379815f
Refactor to abstract away the p2p request handler from mailserver (#34) 2020-06-30 15:29:02 +02:00
Oskar Thorén a8a1e39052
Upgrade submodules (#30)
* Update submodules

* Fix secureManager signature as part of libp2p upgrade
2020-06-15 11:05:37 +08:00
Kim De Mey 37d8720d52
Use metrics with labels, fix #24 (#27) 2020-06-09 13:44:39 +02:00
Oskar Thoren 407ec3534f
parameterize floodsub/gossipsub 2020-06-02 19:27:53 +08:00
Oskar Thoren 1400575067
Update nim-libp2p and other submodules
Need to use latest libp2p for GossipSub bug fix, safer to update all

Use MultiAddr.tryGet() per resultification upstream
2020-06-01 11:40:55 +08:00
Oskar Thoren 971c7a262b
Remove unused imports 2020-05-27 12:09:05 +08:00
Oskar Thorén b5006e482e
Upgrade vendor nim-libp2p and related packages (#13)
* Upgrade nim-libp2p to latest

* Upgrade nim-chronos to latest (needed for nim-libp2p)

* Fix conn.readLp varint

* Fix standard switch API change

* WIP: Crypto resultification on v2

* Fix Result usage for SkPrivateKey

* Result fixes

Co-authored-by: kdeme <kim.demey@gmail.com>
2020-05-26 11:55:53 +08:00
Oskar Thoren c67e36fc13
Use waku protocol; switch to p2p in multiaddress; trace 2020-05-18 14:03:33 +08:00
Oskar Thoren 7d51244ffe
Clean up logging and imports 2020-05-15 11:35:32 +08:00
Oskar Thoren 2070f65828
Fix SIGSERV error by initializing child methods 2020-05-15 11:29:01 +08:00
Oskar Thoren 550556bb97
Fails with SIGSERV 2020-05-14 11:18:24 +08:00
Oskar Thoren c6bd02ba38
SIGSERV failing isolateed a bit more 2020-05-14 10:42:04 +08:00
Oskar Thoren 2e9c2ed949
Move waku test to test folder; reset with floodsub test 2020-05-14 10:28:34 +08:00
Kim De Mey a254ec03da
Bump nim-eth and move tests under v1 subfolder (#3) 2020-05-08 09:36:28 +02:00
kdeme 1f90bae10e
Add check for node in peerpool in P2P post test 2020-05-01 21:13:42 +02:00
kdeme c1fb7aeacd
Add to tests to test_waku_connect and add test_waku_config 2020-05-01 15:36:20 +02:00
kdeme 5c701d35cc
Move waku_protocol + tests from nim-eth 2020-05-01 14:43:25 +02:00
kdeme 87e4e5282a Move Nimbus Waku RPC code + add converted whisper rpc test
- Move Waku RPC code from Nimbus + necessary code around it
- Add Waku RPC test which is a copy of the Whisper RPC test
- Some renaming
- Remove nimbus submodule
2020-05-01 11:29:16 +02:00