Commit Graph

400 Commits

Author SHA1 Message Date
Daniel Kaiser 5e85eecd14
perf(store): add index on receiverTimestamp (#947) 2022-04-23 10:49:11 +02:00
G cb11c192d6
feat(noise): add support to Waku Payload V2 (#932)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* Fixed header

* refactor(noise): add comments, restyle code

* refactor(noise): address reviewer's comments

* Update nim-eth

* fix(noise): moved strutils in std
2022-04-12 13:50:51 +02:00
Hanno Cornelius 450e0494f1
fix(store): timestamp validity checks (#876) 2022-04-11 14:58:18 +02:00
G d2fccb5220
feat(noise): add support to encrypted/unencrypted public keys (#931)
* feat(noise): adding ChaChaPoly encryption

* test(noise): add test for ChaChaPoly Encryption/decryption

* style(noise): converted variables to camelCase

* feat(noise): add support to Noise public keys

* Fixed header

* refactor(noise): address reviewer's comments

* refactor(noise): small fixes/comments

* refactor(noise): add empty plaintext/ciphertext handling + string encryption test

* refactor(noise): address reviewer's comments
2022-04-06 14:37:02 +02:00
G 45b1e10647
feat(noise): add support to ChaChaPoly encryption (#930)
* feat(noise): adding ChaChaPoly encryption

* fix(noise): restored waku2 tests

* fix(noise): missing comma

* Fixed header

* fix(noise): made comment about empty ChaChaPolyKey clearer

* refactor(noise): address reviewer's comments

* test(noise): add test for ChaChaPoly Encryption/decryption

* refactor(noise): removed libp2p specific debug tools usage

* style(noise): converted variables to camelCase
2022-04-04 17:46:51 +02:00
Daniel Kaiser ecdde23add
feat(store): info message after delete (#929) 2022-03-30 22:55:39 +02:00
Daniel Kaiser a9c31b4b53
feat(store): vacuum after delete (#928) 2022-03-30 22:54:37 +02:00
Daniel Kaiser 2f13aff663
feat(jsonrpc): add call returning git version (#919) 2022-03-30 17:17:21 +02:00
Daniel Kaiser ec0a46b65b
feat(jsonrpc): add ENR to debug_v1_info call (#918) 2022-03-30 15:10:15 +02:00
Hanno Cornelius 52de0e434f
feat(relay): add support for gossipsub peer exchange (#911) 2022-03-29 10:09:48 +02:00
Daniel Kaiser d0cf3ed1f9
fix(store): capacity limit for persistent message store (#900)
* fix(store): limit capacity of persistent message store

fixes performance issue of loading a large message table by deleting
old messages in case of an overflow

* test(store): message capacity limit
2022-03-28 20:27:14 +02:00
Daniel Kaiser be8860c2fc
fix(discv5): info print ENR containing discv5 udp port (#913) 2022-03-28 10:29:19 +02:00
Sanaz Taheri Boshrooyeh c9e82873f1
Increase rln-relay logging (#908)
* updates default rln-relay content topic

* adds further debug and trace level logs

* converts a debug level log to the trace level
2022-03-24 10:47:40 -07:00
Hanno Cornelius 06f20af7d4
chore: update submodules (#905) 2022-03-23 15:20:07 +02:00
Hanno Cornelius 994ac49472
Fix: timestamp type could overflow before conversion (#892) 2022-03-18 12:04:11 +02:00
Daniel Kaiser e1146b2df8
discv5 dns bootstrap (#888) 2022-03-17 17:33:17 +01:00
Hanno Cornelius 266d6a7e80
fix: prevent empty store response due to blocked walker (#897) 2022-03-17 17:18:59 +02:00
Sanaz Taheri Boshrooyeh 2d6a2bec47
Updates the type of the `proof` field of wakumessage to `RateLimitProof` (#893)
* permanently changes the wakumessage proof field to RateLimitProof

* updates an old comment
2022-03-16 13:51:18 -07:00
Daniel Kaiser e6fdbdb42a
add RAM requirement to README (#889) 2022-03-16 17:13:23 +01:00
Hanno Cornelius eb02e4411b
Remove content topic dimension from message metrics (#891) 2022-03-15 14:39:32 +02:00
Hanno Cornelius ea72a65ef8
Increase store and store queue logging (#887) 2022-03-11 07:26:15 +02:00
Hanno Cornelius 9165720f63
Bridge improvements (#875) 2022-03-08 11:48:17 +01:00
Sanaz Taheri Boshrooyeh 2f73667a96
Fixes getNanosecondTime() time conversion (#878)
* fixes conversion from sec to nanosec

* uses snake case for power of 10 for readability
2022-03-07 16:56:39 +01:00
Hanno Cornelius 06fbb875c8
Update submodules (#873)
* Update submodules

* Fix MsgIDProvider
2022-03-02 13:24:48 +01:00
Daniel Kaiser 2972a50035
discv5 stage1 testnet (#863) 2022-03-01 17:12:23 +01:00
Daniel Kaiser dbe76d29ce
discv5 stage1 beta test (#862)
* update vendor/nim-eth submodule

point to the selectable `protocol-id` feature branch

* use d5waku as protocol-id

Use compiletime flag to change the discv5 protocol-id to "d5waku" allowing
nim-waku to span a separate discv5 network

* waku discv5 first beta stage

  - start discv5 during node start
  - added config options for discv5

* fix: pass to tcp port also when specifying ext ip

* bump nim-eth

Co-authored-by: ksr <kaiserd@users.noreply.github.com>
2022-03-01 15:11:56 +01:00
Hanno Cornelius 6be0fb233a
Add bridge docs and minor improvements (#867)
* Add bridge docs and minor improvements

* Clarify that bridging is bidirectional
2022-02-28 18:16:50 +01:00
Hanno Cornelius 20efdd7255
Store data model updates and fixes (#864) 2022-02-28 17:29:01 +01:00
G c27086e186
Moved PB Timestamp index to 10 (#861) 2022-02-23 08:19:52 +01:00
Hanno Cornelius 0b7b033550
Add docs on configuring domain and dns disc (#856) 2022-02-21 16:23:55 +01:00
Hanno Cornelius 2fd5b6c4ea
Domain name config for `nwaku` node (#852) 2022-02-18 12:10:38 +01:00
G dd2ed886a3
Update WakuStoreCodec string to beta4 (#855)
* Fix waku store version to beta4

* Updated changelog
2022-02-18 07:23:10 +01:00
G 21cac6d491
Refactoring timestamps (#842)
* Refactor timestamps type from float64 to int64 (milliseconds resolution)

* Revert epochs to float64

* Update 00002_addSenderTimeStamp.up.sql

* Update quicksim2.nim

* Add files via upload

* Delete 00003_convertTimestampsToInts.up.sql

* Add files via upload

* Rename 00003_convertTimestampsToInts.up.sql to 00003_addTimestampsToInts.up.sql

* Delete 00003_addTimestampsToInts.up.sql

* Rln-relay integration into chat2 (#835)

* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* removes duplicate epoch time calculation

* updates log level to trace

* updates default rln-relay content topic

* adds support for empty content topics

* updates changelog

* changelog updates

* removes a commented code block

* updates addRLNRelayValidator string doc

* Squashed commit of the following:

commit bc36c99ab2
Merge: dc2b2946 5a77d6e2
Author: G <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 01:10:06 2022 +0100

    Merge branch 'master' into int64-timestamps-ns

commit dc2b294667
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:24:45 2022 +0100

    Fix

commit f97b95a036
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:13:18 2022 +0100

    Missing import

commit 060c4f8d64
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:10:36 2022 +0100

    Fixed typo

commit 08ca99b6f6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:59:20 2022 +0100

    Time util file

commit 2b5c360746
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:33:20 2022 +0100

    Moved time utility functions to utils/time

commit fdaf121f08
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:10:25 2022 +0100

    Fix comment

commit c7e06ab4e7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:04:13 2022 +0100

    Restore previous migration script

commit 80282db1d7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:54:15 2022 +0100

    Typo

commit b9d67f89b0
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:49:29 2022 +0100

    Added utilities to get int64 nanosecond, microsecond, millisecond time resolution from float

commit 0130d496e6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:36:35 2022 +0100

    Switched to nanoseconds support.

* Update CHANGELOG.md

* Create 00003_convertTimestampsToInt64.up.sql

Migration script

* Moved migration script to right location

* Update waku_rln_relay_utils.nim

* Update waku_rln_relay_utils.nim

* Addressed reviewers' comments

* Update default fleet metrics dashboard (#844)

* Fix

* No need for float

* Aligning master to changes in PR

* Further fixes

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2022-02-17 16:00:15 +01:00
Hanno Cornelius bb3e59454e
Store performance improvements (#849) 2022-02-17 11:00:45 +01:00
Sanaz Taheri Boshrooyeh d851d48424
Updates rln-chat2 interface (#846)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* WIP

* removes duplicate epoch time calculation

* converts echo to log

* invokes handler

* bug fix

* prints calculated epoch

* changes the format of printed epoch

* updates log levels

* logs the input buffer supplied to the generate_proof

* replaces echos with logs

* changes log level to trace

* resets the log level of chat2 to INFO

* upgrades log level to debug

* exports toRLNSignal and adds a doc string

* updates log level

* enables all test2

* removes an echo statement

* modifies a comment

* further updates on the log level

* a minor update

* invokes the spam handler when provided

* checks for payload version

* deletes a redundant check

* deletes a rendant check

* updates default rln-relay cht2 content topic

* adds a todo and log

* changes the case of testnet content topic

* removes a flaky check
2022-02-16 14:52:21 -08:00
Hanno Cornelius e1254c20ae
Support for dns `multiaddrs` (#851)
* Support for dns addrs

* Tests and minor improvements
2022-02-16 17:12:09 +01:00
Franck R 5a890b5f90
Update websocket documentation to use domain certificates (#848) 2022-02-15 17:07:55 +11:00
Sanaz Taheri Boshrooyeh 5a77d6e2a6
Rln-relay integration into chat2 (#835)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* removes duplicate epoch time calculation

* updates log level to trace

* updates default rln-relay content topic

* adds support for empty content topics

* updates changelog

* changelog updates

* removes a commented code block

* updates addRLNRelayValidator string doc
2022-02-04 15:58:27 -08:00
Sanaz Taheri Boshrooyeh a5f8b90c23
Enables RLN compilation under CI env var (#786)
* compiles RLN if CI is defined

* minor ineffective change

* minor

* removes excess space

* resolves namespace conflicts

* moves pubsub import under compiler flag

* disables rln on-chain tests

* brings back the on-chain tests and fixes a name mismatch

* adds a debug log message

* minor formating

* kills ganache-cli procces on windows

* wip: adds a line to kill ganache-cli on windows

* disables rln-relay tests that depend on ganache-cli

* disables ganache-cli installation

* introduces onchain rln flag

* fixes a flaky test

* enables onchain test in non-windows CIs

* moves variables around

it seems detected_os is undefined, hence the compiler flag onchain_rln, which is conditioned to the os, does not get added

* limits onchain rln to macOS
2022-01-28 13:57:17 -08:00
Hanno Cornelius 4421b8de00
Improve node metrics (#831)
* Improve node metrics

* Add support for empty content topics as a separate label
2022-01-26 12:02:57 +01:00
Sanaz Taheri Boshrooyeh dbccecc562
Resolves naming conflict (#821)
* resolves namespace conflicts

* moves pubsub import under compiler flag
2022-01-21 10:55:36 -08:00
Daniel Kaiser b3a7722297
result aggregation in resume API (#816) 2022-01-18 23:05:41 +01:00
Hanno Cornelius 95d2e8bf71
Round receiver time (#815) 2022-01-17 19:37:06 +01:00
Hanno Cornelius 3e9ca25abc
Update submodules (#814) 2022-01-14 10:25:01 +01:00
Hanno Cornelius c422866a49
Improved `store` query performance (#812) 2022-01-11 14:32:09 +01:00
Hanno Cornelius f3ecdb80ea
Remove node peerInfo copy (#801) 2022-01-10 16:07:35 +01:00
Hanno Cornelius 62dbb3d0f3
Add MessageIdProvider for `WakuRelay` (#803)
* Add MessageIdProvider

* Fix unit test. Changelog.
2022-01-10 15:07:01 +01:00
Hanno Cornelius 685d43b2b3
Ensure store response never exceeds `MaxPageSize` (#811)
* Never exceed maxPageSize, even with no specified pagination

* fix test

* Changelog

* Forgot to stage everything...

* Account for timing difference in Windows
2022-01-07 15:01:23 +01:00
Hanno Cornelius 4e6960ad26
Fix incorrect store read buffer limit (#800)
* Fix store read buffer limit

* Changelog
2022-01-06 13:42:37 +01:00
Hanno Cornelius eb5dcadd45
Added protocol message counters (#804)
* Added protocol message counters

* Fix how stored messages are counted
2022-01-06 12:23:25 +01:00