Commit Graph

356 Commits

Author SHA1 Message Date
Lorenzo Delgado 76e9a98841 feat(rest): Add HTTP REST API (#727). Debug API POC 2022-06-14 18:20:15 +02:00
Daniel Kaiser 5445303a23
feat(store): sqlite only store (#982)
* feat(sqlite): parameterized prep statement exec
  - from nim-eth
* feat(store): sql-only-store
  - add getPage(...) to sql-message-store
  - acts as separate store impl that can be activated with
    `--sqlite-store=true`
* test(store): new test-suite for sql message store
* test(store): waku_store test SQL-store compatible
* perf(store): avoid builing an additional B-tree in sql query
  - use `receiverTime` as the last ORDER BY column to fully utilize the
    primary key sorting
feat(store): retention time
  - retention time in seconds can be specified via `--sqlite-retention-time=`
2022-06-13 19:59:53 +02:00
Hanno Cornelius 6a09ed2c88
feat(enr): add discv5 UDP port to node's main discoverable ENR (#996) 2022-06-09 17:46:21 +02:00
Daniel Kaiser 271a2c750b
chore: adjust dyn bootstrap error message (#993) 2022-06-08 22:23:13 +02:00
Hanno Cornelius 0847eec6e0
chore: reduce unnecessary logging (#991) 2022-06-08 11:20:18 +02:00
Hanno Cornelius 1b29dcda7b
chore: disable swap by default (#989) 2022-06-08 09:16:45 +02:00
Hanno Cornelius 4e9215a920
chore: more regular keepalive pings (#990) 2022-06-07 22:44:58 +02:00
G c259f4cbaa
feat(noise): add Noise Handshake processing and after-handshake encryption (#934)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* feat(noise): add Noise Handshake State Machine and primitives

* feat(noise): add Handshake processing and after handshake encryption

* feat(noise): code updated according to #933

* fix(noise): export HandshakeResult

* feat(noise): add pre message validation utility

* chore(noise): improve documentation

* fix(noise): address reviewers comments
2022-06-03 21:12:41 +02:00
Hanno Cornelius d5a04a2719
chore: update submodules (#987) 2022-06-01 11:49:41 +02:00
Hanno Cornelius 37325c8f48
fix(sqlite): memory leak due to undisposed prepared statements (#981) 2022-05-31 12:47:38 +02:00
Sanaz Taheri Boshrooyeh 1ac029025a
Feat (Rln relay): adds utility procs to listen to the registration events emitted by the rln membership contract (#976)
* adds the contract handler file

* adds integration test for the group listening

* adds groupManagement proc

* deletes rln relay contract handler file

* brings back all the tests

* replaces toUINT256 with getIdCommitment proc

* replaces individual futures with an array of futures

* adds code documentation

* asyncSpawn instead of await

* adds untitest for toIDCommitment and toUInt256

* reorganizes the test and add rlnInstance

* mounts handleGroupUpdates on the rln peer

* asyncSpawn to await

* implements toIDCommitment

* updates the unittest

* improves the code documentation

* removes unused tests

* removes registration of the dynamic group management handler

* adds a comment

* adds a comment

* adds a TODO

* removes getIdCommitment
2022-05-30 12:14:07 -07:00
Lorenzo Delgado 32d230b474 feat(store): ability to decouple store from relay (#937) 2022-05-23 20:42:48 +02:00
G 710ce1315a
feat(noise): add Noise Handshake State Machine and primitives (#933)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* feat(noise): add Noise Handshake State Machine and primitives

* Fixed header

* refactor(noise): add comments, restyle code

* refactor(noise): address reviewer's comments

* feat(noise): add Noise Handshake State Machine and primitives

* refactor(noise): rebase

* refactor(noise): refactor and add documentation

* fix(noise): fixed trace parameter

* update submodules

* fix(noise): remove echo, add stdout.write in print handshake

* refactor(noise): add Noise state machine overview

* Revert "update submodules"

This reverts commit 9fc162dff8.

* update submodules

* fix Jenkins caching

* Revert "update submodules"

This reverts commit 1927d9dd26.

* revert nim-eth submodule update

* fix(noise): missing camelCase

* feat(noise): adding unit tests for Noise State machine primitives

* fix(noise): address reviewers' comments

* fix(noise): use expect for error handling in unit tests

* fix(noise): revert to implicit nonce declaration in CipherState initialization
2022-05-20 15:51:36 +02:00
Hanno Cornelius 984f3179a7
chore: update submodules (#974) 2022-05-20 12:57:19 +02:00
Daniel Kaiser 6a60458135
chore(migration): avoid unnecessarily calling migration (#967) 2022-05-18 21:44:53 +02:00
Hanno Cornelius 185485c336
docs: generate and configure nodekey (#957) 2022-05-18 15:00:30 +02:00
Daniel Kaiser 59b242fcad
fix(dns4name): dns4name + unset port resolved (#965) 2022-05-18 10:15:03 +02:00
Daniel Kaiser 41cdb8ba26
feat(info): add --version cmd flag (#970) 2022-05-17 22:11:07 +02:00
Daniel Kaiser 51cbc39d49
feat(config): add config file support (#953)
including

* bump vendor/nim-confutils
* add nim-toml-serialization submodule
* feat(config): add example config file
* add new feature to CHANGELOG
2022-05-17 17:48:08 +02:00
Hanno Cornelius bd70a1b7bb
Disable GossipSub PX by default (#958) 2022-05-13 18:18:09 +02:00
Sanaz Taheri Boshrooyeh 47a9fcfd77
chore|feat (waku-rln-relay): modules reorganization|Initial test for capturing events using nim-web3 (#941)
* first edition

* adds the full test scenario

* fixes typos

* fixes a bug in the supplied command

* further edits the description

* displays the chat prompt after spam detection

* updates changelog

* minor wording fix

* adds a new test file for onchain rln relay

* adds the Event proc

* adds one working example of event subscription

* defines a new unitt test for event subscription

* adds the new test file

* cleans up the code

* adds a working event subscription for faucet contract

* wip

* makes faucet test conditional

* updates contract byte codes

* adds a working test for event subscription and cleans up the tests

* fixes case

* adss toUInt256 unit function

* enables the tests

* fixes a bug

* undo commented tests

* cleans up the test

* logs the pk

* removes excess entry in the changelog

* fixes spacing

* comments

* removes unused test codes

* adds the conditional compilation for onchain tests

* uncomments offchain tests

* removes onchain tests

* reorganizes the code and moves the rln contract data into a separate module

* deletes txt files

* beautifies the code

* beautifies the code

* removes an excess line

* more formatting fixes

* minor fix

* updates the case of membership fee const

* renames compare to diff

* renames time to e

* edits the number of arguments of the send proc

* fixes a comment alignment

* fixes indentation

* fixed id style

* splits check from condition

* fixes a naming mismatch
2022-05-10 14:09:18 -07:00
Franck R b2eb992bd3
Fix typo (#960) 2022-05-10 12:49:12 +02:00
Daniel Kaiser 5e85eecd14
perf(store): add index on receiverTimestamp (#947) 2022-04-23 10:49:11 +02:00
G cb11c192d6
feat(noise): add support to Waku Payload V2 (#932)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* Fixed header

* refactor(noise): add comments, restyle code

* refactor(noise): address reviewer's comments

* Update nim-eth

* fix(noise): moved strutils in std
2022-04-12 13:50:51 +02:00
Hanno Cornelius 450e0494f1
fix(store): timestamp validity checks (#876) 2022-04-11 14:58:18 +02:00
G d2fccb5220
feat(noise): add support to encrypted/unencrypted public keys (#931)
* feat(noise): adding ChaChaPoly encryption

* test(noise): add test for ChaChaPoly Encryption/decryption

* style(noise): converted variables to camelCase

* feat(noise): add support to Noise public keys

* Fixed header

* refactor(noise): address reviewer's comments

* refactor(noise): small fixes/comments

* refactor(noise): add empty plaintext/ciphertext handling + string encryption test

* refactor(noise): address reviewer's comments
2022-04-06 14:37:02 +02:00
G 45b1e10647
feat(noise): add support to ChaChaPoly encryption (#930)
* feat(noise): adding ChaChaPoly encryption

* fix(noise): restored waku2 tests

* fix(noise): missing comma

* Fixed header

* fix(noise): made comment about empty ChaChaPolyKey clearer

* refactor(noise): address reviewer's comments

* test(noise): add test for ChaChaPoly Encryption/decryption

* refactor(noise): removed libp2p specific debug tools usage

* style(noise): converted variables to camelCase
2022-04-04 17:46:51 +02:00
Daniel Kaiser ecdde23add
feat(store): info message after delete (#929) 2022-03-30 22:55:39 +02:00
Daniel Kaiser a9c31b4b53
feat(store): vacuum after delete (#928) 2022-03-30 22:54:37 +02:00
Daniel Kaiser 2f13aff663
feat(jsonrpc): add call returning git version (#919) 2022-03-30 17:17:21 +02:00
Daniel Kaiser ec0a46b65b
feat(jsonrpc): add ENR to debug_v1_info call (#918) 2022-03-30 15:10:15 +02:00
Hanno Cornelius 52de0e434f
feat(relay): add support for gossipsub peer exchange (#911) 2022-03-29 10:09:48 +02:00
Daniel Kaiser d0cf3ed1f9
fix(store): capacity limit for persistent message store (#900)
* fix(store): limit capacity of persistent message store

fixes performance issue of loading a large message table by deleting
old messages in case of an overflow

* test(store): message capacity limit
2022-03-28 20:27:14 +02:00
Daniel Kaiser be8860c2fc
fix(discv5): info print ENR containing discv5 udp port (#913) 2022-03-28 10:29:19 +02:00
Sanaz Taheri Boshrooyeh c9e82873f1
Increase rln-relay logging (#908)
* updates default rln-relay content topic

* adds further debug and trace level logs

* converts a debug level log to the trace level
2022-03-24 10:47:40 -07:00
Hanno Cornelius 06f20af7d4
chore: update submodules (#905) 2022-03-23 15:20:07 +02:00
Hanno Cornelius 994ac49472
Fix: timestamp type could overflow before conversion (#892) 2022-03-18 12:04:11 +02:00
Daniel Kaiser e1146b2df8
discv5 dns bootstrap (#888) 2022-03-17 17:33:17 +01:00
Hanno Cornelius 266d6a7e80
fix: prevent empty store response due to blocked walker (#897) 2022-03-17 17:18:59 +02:00
Sanaz Taheri Boshrooyeh 2d6a2bec47
Updates the type of the `proof` field of wakumessage to `RateLimitProof` (#893)
* permanently changes the wakumessage proof field to RateLimitProof

* updates an old comment
2022-03-16 13:51:18 -07:00
Daniel Kaiser e6fdbdb42a
add RAM requirement to README (#889) 2022-03-16 17:13:23 +01:00
Hanno Cornelius eb02e4411b
Remove content topic dimension from message metrics (#891) 2022-03-15 14:39:32 +02:00
Hanno Cornelius ea72a65ef8
Increase store and store queue logging (#887) 2022-03-11 07:26:15 +02:00
Sanaz Taheri Boshrooyeh 2f73667a96
Fixes getNanosecondTime() time conversion (#878)
* fixes conversion from sec to nanosec

* uses snake case for power of 10 for readability
2022-03-07 16:56:39 +01:00
Hanno Cornelius 06fbb875c8
Update submodules (#873)
* Update submodules

* Fix MsgIDProvider
2022-03-02 13:24:48 +01:00
Daniel Kaiser 2972a50035
discv5 stage1 testnet (#863) 2022-03-01 17:12:23 +01:00
Daniel Kaiser dbe76d29ce
discv5 stage1 beta test (#862)
* update vendor/nim-eth submodule

point to the selectable `protocol-id` feature branch

* use d5waku as protocol-id

Use compiletime flag to change the discv5 protocol-id to "d5waku" allowing
nim-waku to span a separate discv5 network

* waku discv5 first beta stage

  - start discv5 during node start
  - added config options for discv5

* fix: pass to tcp port also when specifying ext ip

* bump nim-eth

Co-authored-by: ksr <kaiserd@users.noreply.github.com>
2022-03-01 15:11:56 +01:00
Hanno Cornelius 6be0fb233a
Add bridge docs and minor improvements (#867)
* Add bridge docs and minor improvements

* Clarify that bridging is bidirectional
2022-02-28 18:16:50 +01:00
Hanno Cornelius 20efdd7255
Store data model updates and fixes (#864) 2022-02-28 17:29:01 +01:00
G c27086e186
Moved PB Timestamp index to 10 (#861) 2022-02-23 08:19:52 +01:00
Hanno Cornelius 0b7b033550
Add docs on configuring domain and dns disc (#856) 2022-02-21 16:23:55 +01:00
Hanno Cornelius 2fd5b6c4ea
Domain name config for `nwaku` node (#852) 2022-02-18 12:10:38 +01:00
G dd2ed886a3
Update WakuStoreCodec string to beta4 (#855)
* Fix waku store version to beta4

* Updated changelog
2022-02-18 07:23:10 +01:00
G 21cac6d491
Refactoring timestamps (#842)
* Refactor timestamps type from float64 to int64 (milliseconds resolution)

* Revert epochs to float64

* Update 00002_addSenderTimeStamp.up.sql

* Update quicksim2.nim

* Add files via upload

* Delete 00003_convertTimestampsToInts.up.sql

* Add files via upload

* Rename 00003_convertTimestampsToInts.up.sql to 00003_addTimestampsToInts.up.sql

* Delete 00003_addTimestampsToInts.up.sql

* Rln-relay integration into chat2 (#835)

* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* removes duplicate epoch time calculation

* updates log level to trace

* updates default rln-relay content topic

* adds support for empty content topics

* updates changelog

* changelog updates

* removes a commented code block

* updates addRLNRelayValidator string doc

* Squashed commit of the following:

commit bc36c99ab2
Merge: dc2b2946 5a77d6e2
Author: G <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 01:10:06 2022 +0100

    Merge branch 'master' into int64-timestamps-ns

commit dc2b294667
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:24:45 2022 +0100

    Fix

commit f97b95a036
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:13:18 2022 +0100

    Missing import

commit 060c4f8d64
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:10:36 2022 +0100

    Fixed typo

commit 08ca99b6f6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:59:20 2022 +0100

    Time util file

commit 2b5c360746
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:33:20 2022 +0100

    Moved time utility functions to utils/time

commit fdaf121f08
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:10:25 2022 +0100

    Fix comment

commit c7e06ab4e7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:04:13 2022 +0100

    Restore previous migration script

commit 80282db1d7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:54:15 2022 +0100

    Typo

commit b9d67f89b0
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:49:29 2022 +0100

    Added utilities to get int64 nanosecond, microsecond, millisecond time resolution from float

commit 0130d496e6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:36:35 2022 +0100

    Switched to nanoseconds support.

* Update CHANGELOG.md

* Create 00003_convertTimestampsToInt64.up.sql

Migration script

* Moved migration script to right location

* Update waku_rln_relay_utils.nim

* Update waku_rln_relay_utils.nim

* Addressed reviewers' comments

* Update default fleet metrics dashboard (#844)

* Fix

* No need for float

* Aligning master to changes in PR

* Further fixes

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2022-02-17 16:00:15 +01:00
Hanno Cornelius bb3e59454e
Store performance improvements (#849) 2022-02-17 11:00:45 +01:00
Sanaz Taheri Boshrooyeh d851d48424
Updates rln-chat2 interface (#846)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* WIP

* removes duplicate epoch time calculation

* converts echo to log

* invokes handler

* bug fix

* prints calculated epoch

* changes the format of printed epoch

* updates log levels

* logs the input buffer supplied to the generate_proof

* replaces echos with logs

* changes log level to trace

* resets the log level of chat2 to INFO

* upgrades log level to debug

* exports toRLNSignal and adds a doc string

* updates log level

* enables all test2

* removes an echo statement

* modifies a comment

* further updates on the log level

* a minor update

* invokes the spam handler when provided

* checks for payload version

* deletes a redundant check

* deletes a rendant check

* updates default rln-relay cht2 content topic

* adds a todo and log

* changes the case of testnet content topic

* removes a flaky check
2022-02-16 14:52:21 -08:00
Hanno Cornelius e1254c20ae
Support for dns `multiaddrs` (#851)
* Support for dns addrs

* Tests and minor improvements
2022-02-16 17:12:09 +01:00
Franck R 5a890b5f90
Update websocket documentation to use domain certificates (#848) 2022-02-15 17:07:55 +11:00
Sanaz Taheri Boshrooyeh 5a77d6e2a6
Rln-relay integration into chat2 (#835)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* removes duplicate epoch time calculation

* updates log level to trace

* updates default rln-relay content topic

* adds support for empty content topics

* updates changelog

* changelog updates

* removes a commented code block

* updates addRLNRelayValidator string doc
2022-02-04 15:58:27 -08:00
Sanaz Taheri Boshrooyeh a5f8b90c23
Enables RLN compilation under CI env var (#786)
* compiles RLN if CI is defined

* minor ineffective change

* minor

* removes excess space

* resolves namespace conflicts

* moves pubsub import under compiler flag

* disables rln on-chain tests

* brings back the on-chain tests and fixes a name mismatch

* adds a debug log message

* minor formating

* kills ganache-cli procces on windows

* wip: adds a line to kill ganache-cli on windows

* disables rln-relay tests that depend on ganache-cli

* disables ganache-cli installation

* introduces onchain rln flag

* fixes a flaky test

* enables onchain test in non-windows CIs

* moves variables around

it seems detected_os is undefined, hence the compiler flag onchain_rln, which is conditioned to the os, does not get added

* limits onchain rln to macOS
2022-01-28 13:57:17 -08:00
Hanno Cornelius 4421b8de00
Improve node metrics (#831)
* Improve node metrics

* Add support for empty content topics as a separate label
2022-01-26 12:02:57 +01:00
Sanaz Taheri Boshrooyeh dbccecc562
Resolves naming conflict (#821)
* resolves namespace conflicts

* moves pubsub import under compiler flag
2022-01-21 10:55:36 -08:00
Daniel Kaiser b3a7722297
result aggregation in resume API (#816) 2022-01-18 23:05:41 +01:00
Hanno Cornelius 95d2e8bf71
Round receiver time (#815) 2022-01-17 19:37:06 +01:00
Hanno Cornelius 3e9ca25abc
Update submodules (#814) 2022-01-14 10:25:01 +01:00
Hanno Cornelius c422866a49
Improved `store` query performance (#812) 2022-01-11 14:32:09 +01:00
Hanno Cornelius f3ecdb80ea
Remove node peerInfo copy (#801) 2022-01-10 16:07:35 +01:00
Hanno Cornelius 62dbb3d0f3
Add MessageIdProvider for `WakuRelay` (#803)
* Add MessageIdProvider

* Fix unit test. Changelog.
2022-01-10 15:07:01 +01:00
Hanno Cornelius 685d43b2b3
Ensure store response never exceeds `MaxPageSize` (#811)
* Never exceed maxPageSize, even with no specified pagination

* fix test

* Changelog

* Forgot to stage everything...

* Account for timing difference in Windows
2022-01-07 15:01:23 +01:00
Hanno Cornelius 4e6960ad26
Fix incorrect store read buffer limit (#800)
* Fix store read buffer limit

* Changelog
2022-01-06 13:42:37 +01:00
Hanno Cornelius eb5dcadd45
Added protocol message counters (#804)
* Added protocol message counters

* Fix how stored messages are counted
2022-01-06 12:23:25 +01:00
rshiv b2273fff9a
Handles stale clients in FILTER protocol (#782)
* Handles stale clients

Signed-off-by: rshiv <reeshav96@gmail.com>

* adds test

Signed-off-by: rshiv <reeshav96@gmail.com>

* removes failed client from subscriber list

Signed-off-by: rshiv <reeshav96@gmail.com>

* adds filter timeout config

Signed-off-by: rshiv <reeshav96@gmail.com>

* reverts peer removal logic

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolve ci issues

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comment

Signed-off-by: rshiv <reeshav96@gmail.com>

* solves review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* ChangeLog update

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comment

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update CHANGELOG.md

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* fixes review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* handles CI issues

Signed-off-by: rshiv <reeshav96@gmail.com>

* tries to solve test CI

Signed-off-by: rshiv <reeshav96@gmail.com>

* solves CI issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves ci issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves CI issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2021-12-08 18:35:47 +00:00
rshiv ce0607b2eb
resolves readme (#796)
Signed-off-by: rshiv <reeshav96@gmail.com>
2021-12-08 16:04:23 +00:00
Hanno Cornelius 9eb532171e
Fix empty scripts condition. Logging. (#795) 2021-12-07 13:06:59 +01:00
Hanno Cornelius dd256e3bd2
Add support for RFC31 ENR (#789) 2021-12-06 20:51:37 +01:00
Richard Ramos 1217e25245 fix: display success message only if messages are loaded from the db 2021-12-03 09:35:40 -04:00
Richard Ramos 2b6922cdaa fix: return db error message 2021-12-03 09:35:40 -04:00
Hanno Cornelius 7ef51edfa2
Update announced addrs in switch (#784) 2021-11-30 10:12:09 +01:00
rshiv d49c95f6a1
publish addresses from announced address sequence. (#775)
* publish multiple addresses

Signed-off-by: rshiv <reeshav96@gmail.com>

* publish from announced address

Signed-off-by: rshiv <reeshav96@gmail.com>

* change in json

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes format

Signed-off-by: rshiv <reeshav96@gmail.com>

* makes listenstr a sequence

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes the wakuinfo Field

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes the test

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes the tests variable

Signed-off-by: rshiv <reeshav96@gmail.com>

* changes test

Signed-off-by: rshiv <reeshav96@gmail.com>
2021-11-24 15:10:29 +00:00
Sanaz Taheri Boshrooyeh fda5128cc1
WIP: Waku-rln-relay: Message validation and double signaling detection (#769)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds trace log for the valid messages

* brings the log trace one line up
2021-11-23 14:48:40 -08:00
rshiv 53b21be2b5
Resolve jenkins windows error (#777)
* try different key

Signed-off-by: rshiv <reeshav96@gmail.com>

* key change

Signed-off-by: rshiv <reeshav96@gmail.com>

* cert change

Signed-off-by: rshiv <reeshav96@gmail.com>

* add newline to key

Signed-off-by: rshiv <reeshav96@gmail.com>

* exception change

Signed-off-by: rshiv <reeshav96@gmail.com>

* exception log

Signed-off-by: rshiv <reeshav96@gmail.com>

* add debug log

Signed-off-by: rshiv <reeshav96@gmail.com>
2021-11-23 11:40:43 +00:00
rshiv ac163932ad
Adds readme for websockets (#768)
* test_fleet

Signed-off-by: rshiv <reeshav96@gmail.com>

* add readme for websockets

Signed-off-by: rshiv <reeshav96@gmail.com>

* add readme

Signed-off-by: rshiv <reeshav96@gmail.com>

* add readme

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update waku/v2/README.md

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/README.md

Co-authored-by: Jakub <i+github@always.fail>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: Jakub <i+github@always.fail>
2021-11-16 10:51:05 +00:00
rshiv 6cad482ca6
Publish multiple address when websocket enabled. (#766)
* publish multiple nodes

Signed-off-by: rshiv <reeshav96@gmail.com>

* Print multiple addresses if wsenabled

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* review changes

Signed-off-by: rshiv <reeshav96@gmail.com>

* review changes

Signed-off-by: rshiv <reeshav96@gmail.com>

* code review changes

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* review fixes

Signed-off-by: rshiv <reeshav96@gmail.com>

* websocket address fix

Signed-off-by: rshiv <reeshav96@gmail.com>
2021-11-15 13:29:18 +00:00
Hanno Cornelius 067478d725
Proposal: ENR filtering on Waku-field in `discv5` (#763) 2021-11-12 15:10:54 +01:00
rshiv d2f7076b53
Secure websocket (#759)
* add config

Signed-off-by: rshiv <reeshav96@gmail.com>

* secure websocket integration

Signed-off-by: rshiv <reeshav96@gmail.com>

* secure websocket debug

Signed-off-by: rshiv <reeshav96@gmail.com>

* config change

Signed-off-by: rshiv <reeshav96@gmail.com>

* working secure websocket changes

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update submodules (#761)

* Release v0.6 (#760)

* Read from path

Signed-off-by: rshiv <reeshav96@gmail.com>

* Tests for secure websockets

Signed-off-by: rshiv <reeshav96@gmail.com>

* CI failure fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* path resolver CI

Signed-off-by: rshiv <reeshav96@gmail.com>

* self review fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update examples/v2/config_chat2.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update waku/v2/utils/peers.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Update waku/v2/node/wakunode2.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update tests/v2/test_wakunode.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/utils/wakuswitch.nim

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

* handle review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
2021-11-10 12:05:36 +00:00
Hanno Cornelius 6d81f1ee55
Update submodules (#761) 2021-11-04 15:46:38 +01:00
Hanno Cornelius ee8ff014f4
Fix: store dimensioning issues (#758) 2021-11-03 11:59:51 +01:00
rshiv d1e06fa17a
Waku v2 Websocket support (#732)
* add config for ws support

* code clean up

* code clean up

* Integrate both ws and tcp transport

* change ws default port

* resolve review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* Unit test debug

Signed-off-by: rshiv <reeshav96@gmail.com>

* Websocket tests addition

Signed-off-by: rshiv <reeshav96@gmail.com>

* websocket failure unittest addition

Signed-off-by: rshiv <reeshav96@gmail.com>

* Commented testcase related to incorrect helper function

Signed-off-by: rshiv <reeshav96@gmail.com>

* Add wireAddr validation

Signed-off-by: rshiv <reeshav96@gmail.com>

* CLoses issue 756

Signed-off-by: rshiv <reeshav96@gmail.com>

* Websocket tests addition

Signed-off-by: rshiv <reeshav96@gmail.com>

* Minor Code clean up

* review comment fixes

Signed-off-by: rshiv <reeshav96@gmail.com>

* Review comment fix

* Update tests/v2/test_wakunode.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* Update tests/v2/test_wakunode.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* review comment fix

Signed-off-by: rshiv <reeshav96@gmail.com>

* Changelog update

Signed-off-by: rshiv <reeshav96@gmail.com>

* chat2 var change

Signed-off-by: rshiv <reeshav96@gmail.com>

* test flag changes

Signed-off-by: rshiv <reeshav96@gmail.com>

Co-authored-by: oskarth <ot@oskarthoren.com>
2021-11-02 10:29:11 +00:00
Hanno Cornelius f85434e072
Discv5 POC integration (#748) 2021-11-01 19:02:39 +01:00
Sanaz Taheri Boshrooyeh dbbc0f750b
RLN-Relay: Adds fix to the hash, proofGen and proofVerify procs (to support arbitrary messages) (#753)
* wip: updates the hash interface

* updates hash unittests

* exposes appendLength

* updates proof gen interface and the unittests

* enables test, fixes proofGen and verify utils and unittest, beautifies the code

* removes int size from mem index type

* enables all the rln tests

* adds documentation of two private procs

* documentation for the appendLength

* minor clean up

* adds clarificaltion on the endianness of input length

* Delete .DS_Store
2021-10-26 14:42:24 -07:00
Sanaz Taheri Boshrooyeh 6efba0dc56
Integrates proof generation and verification into wakunode2 (#735)
* WIP

* WIP: fixes a bug

* adds test for static group formation

* adds static group creation when rln-relay is enabled

* adds createStatic group

* wip: adds group formation to mount rlnrelay

* adds createMembershipList utility function

* adds doc strings and todos

* cleans up the code and add comments

* defaults createRLNInstance depth argument to 32

* renames Depth

* distinguishes between onchain and offchain modes

* updates index boundaries

* updates log levels

* updates docstring

* updates log level of displayed membership keys

* relocates a todo

* activates all the tests

* fixes some comments and todos

* extracts some utils procs for better debugging

* adds todo

* moves calculateMerkleRoot and toMembersipKeyPairs to the rln utils

* makes calls to the utils functions

* adds unit test for createMembershipList

* adds unittest for toMembershipKeyPairs and calcMerkleRoot

* cleans up the code and fixes tree root value

* reverts an unwanted change

* minor

* adds comments and cleans up the code

* updates config message

* adds more comments

* fixes a minor value mismatch

* edits the size of group

* minor rewording

* defines a const var for the group keys

* replaces the sequence literal with the StaticGroupKeys const

* adds a rudimentary unittest

* adds todos

* adds more comment

* replaces uint with MembeshipIndex type

* fixes rln relay mem index config message

* adds rln relay setup proc

* decouples relay and rln-relay

* uses MemIndexType instead of uint

* brings back the rlnRelayEnabled flag to mountRlnRelay

* deletes commented codes

* adds rln relay topic validator inside updates rln relay mounting procedure

* adds rln-relay-pubsub-topic cli option

* adds a static rln-relay topic

* deletes rlnrelayEnabled argument

* adds pubsub topic for rln-relay

* deletes static pubsub topic

* mounts relay before rlnrelay in the tests

* logs rln relay pubsub topic

* cleans up the code

* edits rlnrelay setup

* uninitializes the input parameter of rlnrelay setup

* adds comments

* removes unused comments

* compiles addRLNRelayValidtor when RLN compilation flag is set

* adds comment about topic validator

* minor

* mode modifications on the description of add validator

* adds pubsubtopic field to wakuRlnRelay type

* WIP: shaping the test

* Checks whether rln relay pubsub topic is within the supported topics of relay protocol

* minor

* WIP: unit test for actual proof

* fixes a bug

* removes a redundant proc

* refines the test for actual proof

* breaks lines to 80 chars

* defines NonSpamProof type

* adds a return

* defines Epoch type

* WIP: proof gen

* implements actual proof gen

* adds proto enc and init

* adds notes about proof structure

* adds NonSpamProof to wakumessage

* adds proof gen

* WIP: non working tests for protobuf

* fixes the protobuf encoding issue

* discards the output of copyFrom

* WIP: hash unittest and proofVrfy and ProofGen

* integrates proofVrfy

* uses toBuffer inside the hash proc

* adds comment

* fixes a bug

* removes proof field initialization

* cleans up the test

* generalizes input from byte seq to byte openArray

* adds toBuffer

* adds a bad test

* cleans up unused tests

* adds integration test

* adds comments

* cleans up

* adds description to the integration test

* adds test for unhappy path

* tides up the tests

* tides up hash unit test

* renames a few var

* uses a const for wku rln relay pubsub topic

* minor refinement

* deletes an obsolete comment

* comment revision

* adds comments

* cleans up and adds docstrings

* profGen returns proofRes instead of proof

* removes extra sleepAsync

* fixes two bugs

* returns reject when proof is not verified\

* addresses comments

* adds comments

* links to rln doc

* more comments

* fixes space format

* uncomments v2 tests

* dnsclient branch update

* undo branch update

* minor spacing fix

* makes proof field conditional
2021-10-19 17:37:29 -07:00
Hanno Cornelius 165e235158
Make maximum connections configurable (#731) 2021-10-12 14:48:48 +02:00
Pascal Precht 83f71ae905
Remove `udpPort` CLI option from waku v1 and v2 nodes (#728) 2021-10-12 13:43:01 +02:00
rshiv 5f76d90b7f Revert "wss multiaddress switch support"
This reverts commit ac76e2f189.
2021-10-06 18:03:00 +01:00
rshiv ac76e2f189 wss multiaddress switch support 2021-10-06 17:30:55 +01:00
Hanno Cornelius 50189a21a9
Update submodules: differentiate between local and remote `PeerInfo` (#730)
* Update submodules

* Remove PeerInfo where necessary
2021-10-06 14:29:08 +02:00
Sanaz Taheri Boshrooyeh 696f41e12a
Rln-Relay: Change tree depth to 20 (#726)
* adds instructions on how to generate parameters for a different tree size

* updates the depth of tree

* modifies the Merkle root

* updates parameters for the new tree size

* fixes failing tests

* updates the comment
2021-09-28 12:03:53 -07:00
Sanaz Taheri Boshrooyeh 645832e9fd
Restructuring Rln-Relay mounting process (#722)
* replaces uint with MembeshipIndex type

* fixes rln relay mem index config message

* adds rln relay setup proc

* decouples relay and rln-relay

* uses MemIndexType instead of uint

* brings back the rlnRelayEnabled flag to mountRlnRelay

* deletes commented codes

* adds rln relay topic validator inside updates rln relay mounting procedure

* adds rln-relay-pubsub-topic cli option

* adds a static rln-relay topic

* deletes rlnrelayEnabled argument

* adds pubsub topic for rln-relay

* deletes static pubsub topic

* mounts relay before rlnrelay in the tests

* logs rln relay pubsub topic

* cleans up the code

* edits rlnrelay setup

* uninitializes the input parameter of rlnrelay setup

* adds comments

* removes unused comments

* compiles addRLNRelayValidtor when RLN compilation flag is set

* adds comment about topic validator

* minor

* mode modifications on the description of add validator

* Checks whether rln relay pubsub topic is within the supported topics of relay protocol

* minor

* addresses comments

membeshipindex to membershipindex,
adds default value for rln topic
adds missing return
2021-09-24 10:32:45 -07:00
Sanaz Taheri Boshrooyeh 4895be61ec
Off-chain group construction and management (#718)
* WIP

* WIP: fixes a bug

* adds test for static group formation

* adds static group creation when rln-relay is enabled

* adds createStatic group

* wip: adds group formation to mount rlnrelay

* adds createMembershipList utility function

* adds doc strings and todos

* cleans up the code and add comments

* defaults createRLNInstance depth argument to 32

* renames Depth

* distinguishes between onchain and offchain modes

* updates index boundaries

* updates log levels

* updates docstring

* updates log level of displayed membership keys

* relocates a todo

* activates all the tests

* fixes some comments and todos

* extracts some utils procs for better debugging

* adds todo

* moves calculateMerkleRoot and toMembersipKeyPairs to the rln utils

* makes calls to the utils functions

* adds unit test for createMembershipList

* adds unittest for toMembershipKeyPairs and calcMerkleRoot

* cleans up the code and fixes tree root value

* reverts an unwanted change

* minor

* adds comments and cleans up the code

* updates config message

* adds more comments

* fixes a minor value mismatch

* edits the size of group

* minor rewording

* defines a const var for the group keys

* replaces the sequence literal with the StaticGroupKeys const

* converts var to let when applicable

* replaces hardcoded value with well-defined constants

* moves createMembershipList to the rln relay utils module

* renames HashSize to HashHexSize

* minor updates on the comments

* reorganizes the consts

* indicates that rlnRelayMemIndex is an experimental option

* fixes a type conversion bug

* revises the unittest of "mount waku rln-relay off-chain"

* clarifies the use of index

* updates a docstring

* removes redundant constants and capitalize all of them

* deletes the ETH_CLIENT const from the test file

* renames a few vars for the sake of clarity

* reorganizes unittest into blocks of execution, debug messages, and checks

* adds more comments

* more comments and clarifications

* cleans up the tests

* minor

* adds a minor fix

* replaces a var usage with let

* fixes a bug
2021-09-17 10:31:25 -07:00
Hanno Cornelius a335a40a21
Fix network simulations (#719)
* Fix/tweak network simulations

* fix terminology
2021-09-15 10:01:09 +02:00
Sanaz Taheri Boshrooyeh b7998de09d
RLN-Relay static group construction (#708)
* tests rln instance as pointer

* test insertion

* tests deletion

* tests proof generation and verification

* updates rln instance type and the rln api

* deletes old API

* removes temporary tests

* deletes unused codes

* Delete settings.json

* reverts the changes in tests v2

* removes an old comment

* adds member insertion and deletion

* adds getMerkleRoot and unit tests

* makes insertMember argument  a value type

* adds static group creation

* adds rln field to WakuRLNRelay type, and enables static group construction inside mountRlnRelay

* renames self to membershipKeyPair and removes key generation from mountRlnRelay

* renames sk,pk tp idKey and idCommitment

* updates mountRlnRelay arguments name

* logs created keys

* uncomments the key generation and adds explainer about it

* enables all the  tests

* adds comments to the rln relay types

* logs error message for the arguments that are not set
2021-08-26 16:14:51 -07:00
Hanno Cornelius 0db4107ae2
Integrate DNS resolution (#709)
* Update submodule

* Add DNS Client library

* Fork and update dnsclient submodule

* Integrate DNS resolution
2021-08-25 13:57:35 +02:00
Sanaz Taheri Boshrooyeh c222d83bcc
Enables passing RLN object across different function calls (#705)
* tests rln instance as pointer

* test insertion

* tests deletion

* tests proof generation and verification

* updates rln instance type and the rln api

* deletes old API

* removes temporary tests

* deletes unused codes

* Delete settings.json

* reverts the changes in tests v2

* removes an old comment
2021-08-24 12:25:29 -07:00
Hanno Cornelius 4aae5c55ac
Remove unnecessary ENR caching (#700) 2021-08-17 10:30:17 +02:00
Hanno Cornelius 36b9176569
Waku v2 Node Discovery via DNS: Integration (#690) 2021-08-12 10:51:38 +02:00
Hanno Cornelius 1ebcb266cb
Revert PRUNE backoff period to default (#694) 2021-08-12 09:57:19 +02:00
Sanaz Taheri Boshrooyeh 027cd7f95d
Simplify pagination (#689)
* initial simplification

* simplify forward pagination

* simplifies the backward pagination part

* removes old paginate proc

* revises comments, cleans up the code

* comments on initial paging query

* defines temporary variable for the paginate output
2021-08-04 15:24:36 -07:00
Hanno Cornelius 6b1e0c3079
Patch v0.5.1 - fix multiple protocol IDs in persistent storage (#687) 2021-07-27 08:48:56 +02:00
Hanno Cornelius 87b98a2460
Refactor wakunode2 setup (#675)
* Refactor wakunode2 setup
2021-07-22 11:46:54 +02:00
Hanno Cornelius 7be949ed67
Update submodules (#682) 2021-07-22 10:43:41 +02:00
Sanaz Taheri Boshrooyeh 71a532a414
Adds timestamp to waku relay message (#681)
* adds timestamp to WakuRelayMessage

* converts float to float64

* generates timestamp using epochTime()

* tides up imports and generates timestamps using epochTime()

* updates changelog

* adds timestamp to the post_waku_v2_relay_v1_message query parameters
2021-07-21 11:05:10 -07:00
Hanno Cornelius d44eab7cd7
Fix compilation with `rln` defined (#680)
* Fix compilation with rln defined
2021-07-21 11:37:10 +02:00
Sanaz Taheri Boshrooyeh 52b99f37af
Fixes the order of fields of History Response protobuf (#676)
* shifts history response field numbers by 1

* adds changelog

* minor rewording

* minor
2021-07-20 11:51:32 -07:00
oskarth 75ee287ca7
Adds `{.push raises: [Defect].}` to waku_swap (#677)
Doesn't try to deal with errors, just being explicit about them.
2021-07-19 16:49:27 +08:00
oskarth d3368324fe
Move Whisper types and protocol from nim-eth to this repo (#663)
* Move Whisper type and protocol from nim-eth

- Import whisper_protocol and whisper_types
- Change import locations as appropriate

* Explicit errors in wakunode1
2021-07-19 16:14:37 +08:00
Sanaz Taheri Boshrooyeh 32a8ffb867
Cleaning imports of the waku store module (#673)
* cleans up store modules

* removes endians2 as an unused imports

* reorganizes imports

* fixes an ambiguous import

* adds comments re imports order

* adds message_store
2021-07-16 15:28:35 -07:00
Hanno Cornelius 13cf7380bb
General refactoring: `nim-waku` (#671)
* General Track 1 refactoring
2021-07-16 17:13:36 +02:00
Sanaz Taheri Boshrooyeh a41b9ac80b
Refactor waku protocols and utils (#667)
* refactors waku_filter

* refactors waku_lightpush

* refactors waku_swap

* refactor namespacing.nim

* refactor peers

* refactors requests

* adds top level {.push raises: [Defect].}

* log scope for rln relay

* cleans up comments

* removes comments

* comments out raise Defect

* defines temp var then pass to constructor

* Update waku/v2/protocol/waku_rln_relay/rln.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* Update waku/v2/protocol/waku_swap/waku_swap.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* explains the potential exception in waku_swap

* creates temp var before return

* adjusts spaces

* adds line breaks, temp vars and fixes format

* removes type declaration

* fixes and indentation issue

* adjusts spacing

* adjusts line <80ch

* formating improvement

Co-authored-by: oskarth <ot@oskarthoren.com>
2021-07-15 11:25:52 -07:00
Hanno Cornelius 92f2d5a7f5
Refactor wakunode2.nim (#664)
* Refactor wakunode2.nim

* Remove empty raises

* Some formatting improvements
2021-07-14 19:58:46 +02:00
Sanaz Taheri Boshrooyeh c107072d9a
Waku store: explicit return instead of result (#665)
* replaces result with returns

* fixes a missing proc call

* removes unused comments
2021-07-14 08:59:42 -07:00
Sanaz Taheri Boshrooyeh 07586153f9
Adding error message to the History Responses (#660)
* adds error field to history response

* unittest for protobuf enc/dec

* renames HistoryResponseError to Store Error and simplifies pagination logic

* reverts the naming of StoreError to HistoryResponseError

* fixes a unittest for invalid cursor

* renames OK to NONE
2021-07-13 12:01:21 -07:00
Hanno Cornelius 2869e71865
Simplify wakunode2 subscriptions model (#662) 2021-07-13 09:18:51 +02:00
Sanaz Taheri Boshrooyeh cf7b8faf27
History response pagination using sender-generated timestamp (#657)
* enables pagination based on sender timestamp

* uncomments a test

* bumps up version number

* updates migration script

* clean up

* unpdates changelog

* undo removal of receiver timestamp

* updates message_storage

* uses epochTime()

* minor

* removes a comment

* removes receiver timestamp deletion migration script

* fixes formatting issues

* fixes a bad field name

* fixes field issue

* adjusts spacing
2021-07-07 16:56:20 -07:00
Sanaz Taheri Boshrooyeh cd703c6c52
Integrating pagination into the resume proc (#653)
* adds queryFromWithPaging

* adds unit test for paged queryFrom

* adds a new resume with paging capability

* unittest for resumePaging

* fixes the bug

* implementes a second version of query with paging

* integrates the second version of resume

* clean up

* adds the error message

* fixes a format issue

* renames variables

* defines DefaultPageSize

* adds a todo

* updates a docstring

* gets the pagesize as proc input

* updates unittest description

* removes unused gcsafe pragma

* better var naming

* updates changelog

* updates changelog

* updates the TODO

* more debug logs
2021-07-02 11:37:58 -07:00
Hanno Cornelius 734fe0795f
Add support for multiple protocol IDs (#652) 2021-07-02 10:49:41 +02:00
Hanno Cornelius 52b741ac11
Support for stable version of relay protocol (#651) 2021-06-29 16:29:04 +02:00
Sanaz Taheri Boshrooyeh 81b207624b
Add db migration support for peer store (#648)
* moves migration paths to the migration_types module

* moves USER_VERSION to the migration_types module

* updates migration execution condition to cover the peer store

* adds a message to the store if it is inserted to the db

* adds debug messages and checks migration path recursively

* deletes unused debug messages

* deletes commented codes

* clean up

* fixes a bug

* renames the first migration file of the message store
2021-06-25 14:06:56 -07:00
Ebube Sered Ud 6079ae8525
Fix issues with Peer reference in Cheque Functions (#635)
* Add function for getting peer Id

* Modify Handle Cheque and Send Cheque function

* Include PeerInfo in debit and credit functions

* Minor changes to queryWithAccounting function
2021-06-22 21:55:01 +01:00
Sanaz Taheri Boshrooyeh 30a7a7a4fe
Filter out duplicate messages in the resume procedure (#600)
* adds timestamp to waku message store impl

* stores timestamp as int64

* adds untitest

* stores timestamp as seq of bytes

* minor

* re-orders unittest

* changes receiver timestamp to float64

* unit test for receiver timestamps

* adds comments

* reorder a few lines

* updates changelog

* more updates on changelog

* WIP: deduplicate

* add debug messages

* more debugging logs

* add timestamp to the message store

* converts resume result to uint64, removes some logs

* docstring for isDuplicate

* increments `added` before insertion into the db

* unit test for deduplication

* removes float to/from Bytes procs

* deletes extra spaces

* returns failed instead of the error

* minor

* minor

* adds changelog
2021-06-22 11:30:12 +08:00
Michael Bradley, Jr c5da7c8fcd Remove unused import of ./storage/sqlite from wakunode2
In the `when isMainModule:` section of wakunode2.nim, the module
`./storage/message/waku_message_store` is imported, which in turn imports
waku's sqlite module and also the sqlite3_abi module from `vendor/`.

Outside of the `when isMainModule:` section, imports from waku's sqlite module
are not used in wakunode2. However, importing it at the top-level results in
the C code underlying sqlite3_abi to be compiled into whatever executable one
is building, and this is a problem if nim-waku is being consumed together with
another library, e.g. nim-sqlcipher, which links against a different build of
sqlite (sqlcipher being a superset of sqlite). The linker step fails with
reports of duplicate symbols for sqlite.
2021-06-18 13:56:37 -05:00
Ebube Sered Ud c1763362d7
Add Issuer to Cheque Object (#627)
* Add Issuer to Cheque Object closes  #626

* Minor changes

* Fix typo in comment

* Improve docs

* Fix Typo In Comment

* Minor changes

* Minor changes
2021-06-18 16:53:51 +01:00
Hanno Cornelius 2f53c2c60b
Feat/bridge topic mapping (#633)
* Use namespaced content topics for bridge
2021-06-18 12:30:24 +02:00
Ebube Sered Ud eb49d3148c
Improve Swap Account Balance Metrics to make use of a Histogram (#628)
Modify metrics description

Add more buckets to the swap account metrics
2021-06-17 16:36:44 +01:00
Sanaz Taheri Boshrooyeh ddf93814fe
Persisting Waku message timestamp & implementing DB migration & convert receiver timestamp data type to float64 (#607)
* adds timestamp to waku message store impl

* stores timestamp as int64

* adds untitest

* stores timestamp as seq of bytes

* minor

* re-orders unittest

* changes receiver timestamp to float64

* unit test for receiver timestamps

* adds comments

* reorder a few lines

* updates changelog

* more updates on changelog

* WIP

* WIP

* adds migration

* more debug messages

* passes the path to the migration scripts from message store module

* adds migration result type

* replaces migrationScripts with migrationScriptsResult

* adds path calculation to the message store

* removes some tests binary file

* removes redundant imports

* comments out user_version assignment in sqlite  init

* more descriptive err messages

* clean up test file

* more info logs

* minor code format

* removes a todo

* minor updates

* remove a binary file

* unit tests for migration utils

* adds split script

* integrates split query to handle scripts with multiple commands

* updates migration script for v1

* updates the v1 migration script

* update user version

* updates script

* fixes a few bugs on the splitScript

* more debug logs

* adds float64 parameter support to sqlite3

* change in timestamp type in the script

* deletes float64 toBytes utils

* enables storage of timestamp as a real number in the sqlite db

* bump up script index

* comment edits

* removes migrate unit test

* adds todo and docstring

* updates changelog

* removes an unused item in .gitignore

* minor

* updates changelog

* organizes imports

* cleans up imports

* WIP

* updates script


fixes a few bugs on the splitScript


more debug logs


adds float64 parameter support to sqlite3


change in timestamp type in the script


deletes float64 toBytes utils

* edits migration util test

* remove an empty test file

* includes migration utils tests in

* deletes unused codes

* tides up imports

* adds range based filter to the filterMigrationScripts

* renames procs: removes Migration

* tides up imports

* edits docstring

* edits docstring

* edits docstring

* removes unused imports

* more clean up

* groups std imports

* updates changelog

* adds docstring for setUserVersion

* adds unittest for the migrate

* Update waku/v2/node/storage/message/waku_message_store.nim

Co-authored-by: RichΛrd <info@richardramos.me>

* Update waku/v2/node/storage/sqlite.nim

Co-authored-by: RichΛrd <info@richardramos.me>

* Update waku/v2/node/storage/sqlite.nim

Co-authored-by: RichΛrd <info@richardramos.me>

* removes split scripts

* fixes a naming issue

* fixes a bug

* fixes a typo

* adds a log re updated user_version

* fixes a proc naming mismatch

* fixes naming mismatch

* more descriptive var names

* adds migration script of the first user version

* moves migration to after persistMessages flag is checked

* deletes unused comment

* fixes a bug

* brings back split script

* adds unit tests for split scripts

* runs scripts one command at a time

* deletes a commented line

* relocates the migrate proc to sqlite.nim

* adds unit test for filter scripts

* adds filterScripts unittest testing varying zero-prefixed user versions

* minor

Co-authored-by: RichΛrd <info@richardramos.me>
2021-06-16 13:23:55 -07:00
Hanno Cornelius a044c6a82c
Add ping protocol support and use for keep-alive (#621)
* Add ping protocol support and use for keep-alive

* Separate WakuPeerStore from libp2p PeerStore

* Revert silly test timeout
2021-06-15 10:55:47 +02:00
Kim De Mey e2ea94dc71
Remove the need for pcre (#623) 2021-06-15 09:51:09 +02:00
Ebube Sered Ud 76d4b89dbb
Add mode to waku swap (#601)
* Add mode to waku swap

* Modify swap test  and policy function

* Make rln optional (dependencies and compilation) (#602)

* makes rlnlib target optional

* makes the rln-related targets conditioned to the RLN command line flag

* adds the rln compiler flag condition

* adds the rln compiler flag condition to the rln module

* wakunode2 rln conditional compilation

* updates wakunode2 test

* updates waku rln relay tests

* removing the rln test from the default imports of the v2 tests

* imports rln module conditionally

* removes the rln flag condition from the rln module

* separates rln data types from its procs

* adds the import statement

* brings back the contract def to the rln utils

* adds rln module import to the rln unit tests

* clean up and reorganization

* adds a todo

* minor edits on a comment

* Fix indentation and typo

* Add Swap Config Object and set default config value on mount

* Minor changes

* Fix issues with grammar in code comment

* Create init procedure for setting default values in SwapConfig

* Minor changes to mountSwap call in wakunode2

* Improve Docs on Swap Mode

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2021-06-15 10:06:36 +08:00
Oskar Thorén be717ce5f3
Improve libwaku and C wrappers (#613)
- Move stuff to Makefile
- Move libwaku to root
- Make examples compile for C and Go
- Update README
2021-06-13 20:50:10 +08:00
Hanno Cornelius a8da4dfd6b
Fix: metrics-server use after submodule update (#610)
* Fix: metrics-server use after submodule update

* Remove insecure check
2021-06-10 16:18:41 +02:00
Hanno Cornelius 9e0b73bd18
Update submodules (#594) 2021-06-09 16:37:08 +02:00
Sanaz Taheri Boshrooyeh 2fe53c334c
Make rln optional (dependencies and compilation) (#602)
* makes rlnlib target optional

* makes the rln-related targets conditioned to the RLN command line flag

* adds the rln compiler flag condition

* adds the rln compiler flag condition to the rln module

* wakunode2 rln conditional compilation

* updates wakunode2 test

* updates waku rln relay tests

* removing the rln test from the default imports of the v2 tests

* imports rln module conditionally

* removes the rln flag condition from the rln module

* separates rln data types from its procs

* adds the import statement

* brings back the contract def to the rln utils

* adds rln module import to the rln unit tests

* clean up and reorganization

* adds a todo

* minor edits on a comment
2021-06-08 11:56:32 -07:00
Ebube Sered Ud e2185caf27
Add Swap account Metrics to Grafana (#585)
* Fix gcsafe issues with log account metrics function

Signed-off-by: kripsonud <kripsonud@gmail.com>

* Change Line ending to LF

* Minor changes to waku_swap

* Mount keepalive on chat2bridge and bridge (#591)

* Light push beta bump (#590)

Closes https://github.com/status-im/nim-waku/issues/565

* Filter and lightpush showcase (#593)

* Filter and lightpush showcase

* Fix log error

* Add swap account panel to grafana dashboard template

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-06-06 16:15:18 +01:00
Hanno Cornelius b615c6aa8b
Filter and lightpush showcase (#593)
* Filter and lightpush showcase

* Fix log error
2021-06-04 12:02:47 +02:00
Oskar Thorén 2568af3b98
Light push beta bump (#590)
Closes https://github.com/status-im/nim-waku/issues/565
2021-06-03 13:29:04 +08:00
Hanno Cornelius 83f546ede4
Mount keepalive on chat2bridge and bridge (#591) 2021-06-02 16:54:38 +02:00
Hanno Cornelius 2b571e205b
Fix keepalive for connected peers (#588)
* Fix keepalive for connected peers

* Remove comment
2021-06-02 09:53:34 +02:00
Hanno Cornelius ee757308c6
Fix: No keep-alive for backoff peers (#584) 2021-06-01 09:55:46 +02:00
Sanaz Taheri Boshrooyeh 8f741de483
A quick fix to the master issue (#579)
* moves resume to after the relay is mounted

* Update waku/v2/node/wakunode2.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2021-05-28 11:41:29 -07:00
Ebube Sered Ud cc5f55dbbb
Bump swap protocol to beta1 (#578)
* Bump swap protocol to beta1
2021-05-28 17:13:17 +01:00
Sanaz Taheri Boshrooyeh f75f7c4751
Add resume to waku node api (#564)
* activates resume

* adds unit test

* add store condition

* mode debug info

* updates wakunode resume api and its unittest

* adds todo

* adds documentation

* edits resume documentation

* further edits on the docs

* removes a todo

* fixes a bug

* add resume to waku node api

* further updates on the node.md

* updates the changelog

* minor

* removes return type

* adds a brief desc of the resume to node.md
2021-05-26 12:33:22 -07:00