* adds the contract handler file
* adds integration test for the group listening
* adds groupManagement proc
* deletes rln relay contract handler file
* brings back all the tests
* replaces toUINT256 with getIdCommitment proc
* replaces individual futures with an array of futures
* adds code documentation
* asyncSpawn instead of await
* adds untitest for toIDCommitment and toUInt256
* reorganizes the test and add rlnInstance
* mounts handleGroupUpdates on the rln peer
* asyncSpawn to await
* implements toIDCommitment
* updates the unittest
* improves the code documentation
* removes unused tests
* WIP
* uncomments the tests
* defines a new mountRlnRelayStatic proc
* splits mountRlnRelay into two procs for dynamic and static group management
* adds a config for off-chain and on-chain rln-relay
* runs dynamic or static mode of rln-relay based on the input config
* adds Eth private key and account configs
* reads Eth private key and account the configs
* comments put the second register proc
* add proper call to the rlnrelay dynamic mode
* adds todo
* adds new rln relay configs
* splits register into two procs
* makes eth account private key non-optional
* removes getIdCommitment and edits the register proc
* removes getIdCommitment calls
* uncomments the commented tests
* fixes a bug
* removes contract deployment for the offchain test
* fixes a bug, edits comments
* removes custom types without proper parsing and serialization routines from the configs
* fixes a bug
* switches to stew byte utils
* removes log decoding
* WIP
* updates register proc
* edits test titles
* removes eth private key config
* changes the output of register proc to return the registered index
* integrates the registration process into mountRlnRelayDynamic
* integration test for the register proc
* brings back the onchain tests
* updates comments
* cleans up
* disambiguates the Address type namespace
* fixes type ambiguities
* adds default values for rln key and index
* updates config descriptions
* adds type conversion from hex to MembershipKeyPair
* adds more code documentation
* passed the group value instead of option to the mount proc
* fix a bug
* a minor input type fix for rln chat2
* groups let declarations
* adds default values for addresses
* logs registered keys
* feat(sqlite): parameterized prep statement exec
- from nim-eth
* feat(store): sql-only-store
- add getPage(...) to sql-message-store
- acts as separate store impl that can be activated with
`--sqlite-store=true`
* test(store): new test-suite for sql message store
* test(store): waku_store test SQL-store compatible
* perf(store): avoid builing an additional B-tree in sql query
- use `receiverTime` as the last ORDER BY column to fully utilize the
primary key sorting
feat(store): retention time
- retention time in seconds can be specified via `--sqlite-retention-time=`
* adds the contract handler file
* adds integration test for the group listening
* adds groupManagement proc
* deletes rln relay contract handler file
* brings back all the tests
* replaces toUINT256 with getIdCommitment proc
* replaces individual futures with an array of futures
* adds code documentation
* asyncSpawn instead of await
* adds untitest for toIDCommitment and toUInt256
* reorganizes the test and add rlnInstance
* mounts handleGroupUpdates on the rln peer
* asyncSpawn to await
* implements toIDCommitment
* updates the unittest
* improves the code documentation
* removes unused tests
* removes registration of the dynamic group management handler
* adds a comment
* adds a comment
* adds a TODO
* removes getIdCommitment
* fix(store): limit capacity of persistent message store
fixes performance issue of loading a large message table by deleting
old messages in case of an overflow
* test(store): message capacity limit
* update vendor/nim-eth submodule
point to the selectable `protocol-id` feature branch
* use d5waku as protocol-id
Use compiletime flag to change the discv5 protocol-id to "d5waku" allowing
nim-waku to span a separate discv5 network
* waku discv5 first beta stage
- start discv5 during node start
- added config options for discv5
* fix: pass to tcp port also when specifying ext ip
* bump nim-eth
Co-authored-by: ksr <kaiserd@users.noreply.github.com>
* Refactor timestamps type from float64 to int64 (milliseconds resolution)
* Revert epochs to float64
* Update 00002_addSenderTimeStamp.up.sql
* Update quicksim2.nim
* Add files via upload
* Delete 00003_convertTimestampsToInts.up.sql
* Add files via upload
* Rename 00003_convertTimestampsToInts.up.sql to 00003_addTimestampsToInts.up.sql
* Delete 00003_addTimestampsToInts.up.sql
* Rln-relay integration into chat2 (#835)
* adds ProofMetadata
* adds EPOCH_INTERVAL
* adds messageLog field
* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo
* adds unit test for toEpoch and fromEpoch
* adds unit test for Epoch comparison
* adds result codes for updateLog
* adds unit test for update log
* renames epoch related consts
* modifies updateLog with new return type and new logic of spam detection
* adds unit text for the modified updateLog
* changes max epoch gap type size
* splits updateLog into two procs isSpam and updateLog
* updates unittests
* fixes a bug, returns false when the message is not spam
* renames messageLog to nullifierLog
* renames isSpam to hasDuplicate
* updates the rln validator, adds comments
* adds appendRLNProof proc plus some code beatification
* unit test for validate message
* adds unhappy test to validateMessage unit test
* renames EPOCH_UNIT_SECONDS
* renames MAX_CLOCK_GAP_SECONDS
* WIP: integration test
* fixes compile errors
* sets a real epoch value
* updates on old unittests
* adds comments to the rln relay tests
* adds more comments
* makes rln import conditional
* adds todos
* adds more todos
* adds rln-relay mount process into chat2
* further todos
* logs contentTopic
* introduces rln relay configs
* changes default pubsub topic
* adds contentTopic config
* imports rln relay dependencies
* consolidates imports
* removes module identifier from ContentTopic
* adds contentTopic field
* adds contentTopic argument to mountRlnRelay calls
* appends rln proof to chat2 messages
* changes the default chat2 contentTopic
* adds missing content topic fields
* fixes a bug
* adds a new logic about empty content topics
* appends proof only when rln flag is active
* removes unnecessary todos
* fixes an indentation issue
* adds log messages
* verifies the proof against the concatenation of msg payload and content topic
* a bug fix
* removes duplicate epoch time calculation
* updates log level to trace
* updates default rln-relay content topic
* adds support for empty content topics
* updates changelog
* changelog updates
* removes a commented code block
* updates addRLNRelayValidator string doc
* Squashed commit of the following:
commit bc36c99ab2
Merge: dc2b29465a77d6e2
Author: G <28568419+s1fr0@users.noreply.github.com>
Date: Sat Feb 5 01:10:06 2022 +0100
Merge branch 'master' into int64-timestamps-ns
commit dc2b294667
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date: Sat Feb 5 00:24:45 2022 +0100
Fix
commit f97b95a036
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date: Sat Feb 5 00:13:18 2022 +0100
Missing import
commit 060c4f8d64
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date: Sat Feb 5 00:10:36 2022 +0100
Fixed typo
commit 08ca99b6f6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date: Fri Feb 4 23:59:20 2022 +0100
Time util file
commit 2b5c360746
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date: Fri Feb 4 23:33:20 2022 +0100
Moved time utility functions to utils/time
commit fdaf121f08
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date: Fri Feb 4 23:10:25 2022 +0100
Fix comment
commit c7e06ab4e7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date: Fri Feb 4 23:04:13 2022 +0100
Restore previous migration script
commit 80282db1d7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date: Fri Feb 4 22:54:15 2022 +0100
Typo
commit b9d67f89b0
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date: Fri Feb 4 22:49:29 2022 +0100
Added utilities to get int64 nanosecond, microsecond, millisecond time resolution from float
commit 0130d496e6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date: Fri Feb 4 22:36:35 2022 +0100
Switched to nanoseconds support.
* Update CHANGELOG.md
* Create 00003_convertTimestampsToInt64.up.sql
Migration script
* Moved migration script to right location
* Update waku_rln_relay_utils.nim
* Update waku_rln_relay_utils.nim
* Addressed reviewers' comments
* Update default fleet metrics dashboard (#844)
* Fix
* No need for float
* Aligning master to changes in PR
* Further fixes
Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
* adds ProofMetadata
* adds EPOCH_INTERVAL
* adds messageLog field
* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo
* adds unit test for toEpoch and fromEpoch
* adds unit test for Epoch comparison
* adds result codes for updateLog
* adds unit test for update log
* renames epoch related consts
* modifies updateLog with new return type and new logic of spam detection
* adds unit text for the modified updateLog
* changes max epoch gap type size
* splits updateLog into two procs isSpam and updateLog
* updates unittests
* fixes a bug, returns false when the message is not spam
* renames messageLog to nullifierLog
* renames isSpam to hasDuplicate
* updates the rln validator, adds comments
* adds appendRLNProof proc plus some code beatification
* unit test for validate message
* adds unhappy test to validateMessage unit test
* renames EPOCH_UNIT_SECONDS
* renames MAX_CLOCK_GAP_SECONDS
* WIP: integration test
* fixes compile errors
* sets a real epoch value
* updates on old unittests
* adds comments to the rln relay tests
* adds more comments
* makes rln import conditional
* adds todos
* adds more todos
* adds rln-relay mount process into chat2
* further todos
* logs contentTopic
* introduces rln relay configs
* changes default pubsub topic
* adds contentTopic config
* imports rln relay dependencies
* consolidates imports
* removes module identifier from ContentTopic
* adds contentTopic field
* adds contentTopic argument to mountRlnRelay calls
* appends rln proof to chat2 messages
* changes the default chat2 contentTopic
* adds missing content topic fields
* fixes a bug
* adds a new logic about empty content topics
* appends proof only when rln flag is active
* removes unnecessary todos
* fixes an indentation issue
* adds log messages
* verifies the proof against the concatenation of msg payload and content topic
* a bug fix
* WIP
* removes duplicate epoch time calculation
* converts echo to log
* invokes handler
* bug fix
* prints calculated epoch
* changes the format of printed epoch
* updates log levels
* logs the input buffer supplied to the generate_proof
* replaces echos with logs
* changes log level to trace
* resets the log level of chat2 to INFO
* upgrades log level to debug
* exports toRLNSignal and adds a doc string
* updates log level
* enables all test2
* removes an echo statement
* modifies a comment
* further updates on the log level
* a minor update
* invokes the spam handler when provided
* checks for payload version
* deletes a redundant check
* deletes a rendant check
* updates default rln-relay cht2 content topic
* adds a todo and log
* changes the case of testnet content topic
* removes a flaky check
* adds ProofMetadata
* adds EPOCH_INTERVAL
* adds messageLog field
* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo
* adds unit test for toEpoch and fromEpoch
* adds unit test for Epoch comparison
* adds result codes for updateLog
* adds unit test for update log
* renames epoch related consts
* modifies updateLog with new return type and new logic of spam detection
* adds unit text for the modified updateLog
* changes max epoch gap type size
* splits updateLog into two procs isSpam and updateLog
* updates unittests
* fixes a bug, returns false when the message is not spam
* renames messageLog to nullifierLog
* renames isSpam to hasDuplicate
* updates the rln validator, adds comments
* adds appendRLNProof proc plus some code beatification
* unit test for validate message
* adds unhappy test to validateMessage unit test
* renames EPOCH_UNIT_SECONDS
* renames MAX_CLOCK_GAP_SECONDS
* WIP: integration test
* fixes compile errors
* sets a real epoch value
* updates on old unittests
* adds comments to the rln relay tests
* adds more comments
* makes rln import conditional
* adds todos
* adds more todos
* adds rln-relay mount process into chat2
* further todos
* logs contentTopic
* introduces rln relay configs
* changes default pubsub topic
* adds contentTopic config
* imports rln relay dependencies
* consolidates imports
* removes module identifier from ContentTopic
* adds contentTopic field
* adds contentTopic argument to mountRlnRelay calls
* appends rln proof to chat2 messages
* changes the default chat2 contentTopic
* adds missing content topic fields
* fixes a bug
* adds a new logic about empty content topics
* appends proof only when rln flag is active
* removes unnecessary todos
* fixes an indentation issue
* adds log messages
* verifies the proof against the concatenation of msg payload and content topic
* a bug fix
* removes duplicate epoch time calculation
* updates log level to trace
* updates default rln-relay content topic
* adds support for empty content topics
* updates changelog
* changelog updates
* removes a commented code block
* updates addRLNRelayValidator string doc
* adds ProofMetadata
* adds EPOCH_INTERVAL
* adds messageLog field
* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo
* adds unit test for toEpoch and fromEpoch
* adds unit test for Epoch comparison
* adds result codes for updateLog
* adds unit test for update log
* renames epoch related consts
* modifies updateLog with new return type and new logic of spam detection
* adds unit text for the modified updateLog
* changes max epoch gap type size
* splits updateLog into two procs isSpam and updateLog
* updates unittests
* fixes a bug, returns false when the message is not spam
* renames messageLog to nullifierLog
* renames isSpam to hasDuplicate
* updates the rln validator, adds comments
* adds appendRLNProof proc plus some code beatification
* unit test for validate message
* adds unhappy test to validateMessage unit test
* renames EPOCH_UNIT_SECONDS
* renames MAX_CLOCK_GAP_SECONDS
* WIP: integration test
* fixes compile errors
* sets a real epoch value
* updates on old unittests
* adds comments to the rln relay tests
* adds more comments
* makes rln import conditional
* adds trace log for the valid messages
* brings the log trace one line up
* WIP
* WIP: fixes a bug
* adds test for static group formation
* adds static group creation when rln-relay is enabled
* adds createStatic group
* wip: adds group formation to mount rlnrelay
* adds createMembershipList utility function
* adds doc strings and todos
* cleans up the code and add comments
* defaults createRLNInstance depth argument to 32
* renames Depth
* distinguishes between onchain and offchain modes
* updates index boundaries
* updates log levels
* updates docstring
* updates log level of displayed membership keys
* relocates a todo
* activates all the tests
* fixes some comments and todos
* extracts some utils procs for better debugging
* adds todo
* moves calculateMerkleRoot and toMembersipKeyPairs to the rln utils
* makes calls to the utils functions
* adds unit test for createMembershipList
* adds unittest for toMembershipKeyPairs and calcMerkleRoot
* cleans up the code and fixes tree root value
* reverts an unwanted change
* minor
* adds comments and cleans up the code
* updates config message
* adds more comments
* fixes a minor value mismatch
* edits the size of group
* minor rewording
* defines a const var for the group keys
* replaces the sequence literal with the StaticGroupKeys const
* adds a rudimentary unittest
* adds todos
* adds more comment
* replaces uint with MembeshipIndex type
* fixes rln relay mem index config message
* adds rln relay setup proc
* decouples relay and rln-relay
* uses MemIndexType instead of uint
* brings back the rlnRelayEnabled flag to mountRlnRelay
* deletes commented codes
* adds rln relay topic validator inside updates rln relay mounting procedure
* adds rln-relay-pubsub-topic cli option
* adds a static rln-relay topic
* deletes rlnrelayEnabled argument
* adds pubsub topic for rln-relay
* deletes static pubsub topic
* mounts relay before rlnrelay in the tests
* logs rln relay pubsub topic
* cleans up the code
* edits rlnrelay setup
* uninitializes the input parameter of rlnrelay setup
* adds comments
* removes unused comments
* compiles addRLNRelayValidtor when RLN compilation flag is set
* adds comment about topic validator
* minor
* mode modifications on the description of add validator
* adds pubsubtopic field to wakuRlnRelay type
* WIP: shaping the test
* Checks whether rln relay pubsub topic is within the supported topics of relay protocol
* minor
* WIP: unit test for actual proof
* fixes a bug
* removes a redundant proc
* refines the test for actual proof
* breaks lines to 80 chars
* defines NonSpamProof type
* adds a return
* defines Epoch type
* WIP: proof gen
* implements actual proof gen
* adds proto enc and init
* adds notes about proof structure
* adds NonSpamProof to wakumessage
* adds proof gen
* WIP: non working tests for protobuf
* fixes the protobuf encoding issue
* discards the output of copyFrom
* WIP: hash unittest and proofVrfy and ProofGen
* integrates proofVrfy
* uses toBuffer inside the hash proc
* adds comment
* fixes a bug
* removes proof field initialization
* cleans up the test
* generalizes input from byte seq to byte openArray
* adds toBuffer
* adds a bad test
* cleans up unused tests
* adds integration test
* adds comments
* cleans up
* adds description to the integration test
* adds test for unhappy path
* tides up the tests
* tides up hash unit test
* renames a few var
* uses a const for wku rln relay pubsub topic
* minor refinement
* deletes an obsolete comment
* comment revision
* adds comments
* cleans up and adds docstrings
* profGen returns proofRes instead of proof
* removes extra sleepAsync
* fixes two bugs
* returns reject when proof is not verified\
* addresses comments
* adds comments
* links to rln doc
* more comments
* fixes space format
* uncomments v2 tests
* dnsclient branch update
* undo branch update
* minor spacing fix
* makes proof field conditional
* replaces uint with MembeshipIndex type
* fixes rln relay mem index config message
* adds rln relay setup proc
* decouples relay and rln-relay
* uses MemIndexType instead of uint
* brings back the rlnRelayEnabled flag to mountRlnRelay
* deletes commented codes
* adds rln relay topic validator inside updates rln relay mounting procedure
* adds rln-relay-pubsub-topic cli option
* adds a static rln-relay topic
* deletes rlnrelayEnabled argument
* adds pubsub topic for rln-relay
* deletes static pubsub topic
* mounts relay before rlnrelay in the tests
* logs rln relay pubsub topic
* cleans up the code
* edits rlnrelay setup
* uninitializes the input parameter of rlnrelay setup
* adds comments
* removes unused comments
* compiles addRLNRelayValidtor when RLN compilation flag is set
* adds comment about topic validator
* minor
* mode modifications on the description of add validator
* Checks whether rln relay pubsub topic is within the supported topics of relay protocol
* minor
* addresses comments
membeshipindex to membershipindex,
adds default value for rln topic
adds missing return
* WIP
* WIP: fixes a bug
* adds test for static group formation
* adds static group creation when rln-relay is enabled
* adds createStatic group
* wip: adds group formation to mount rlnrelay
* adds createMembershipList utility function
* adds doc strings and todos
* cleans up the code and add comments
* defaults createRLNInstance depth argument to 32
* renames Depth
* distinguishes between onchain and offchain modes
* updates index boundaries
* updates log levels
* updates docstring
* updates log level of displayed membership keys
* relocates a todo
* activates all the tests
* fixes some comments and todos
* extracts some utils procs for better debugging
* adds todo
* moves calculateMerkleRoot and toMembersipKeyPairs to the rln utils
* makes calls to the utils functions
* adds unit test for createMembershipList
* adds unittest for toMembershipKeyPairs and calcMerkleRoot
* cleans up the code and fixes tree root value
* reverts an unwanted change
* minor
* adds comments and cleans up the code
* updates config message
* adds more comments
* fixes a minor value mismatch
* edits the size of group
* minor rewording
* defines a const var for the group keys
* replaces the sequence literal with the StaticGroupKeys const
* converts var to let when applicable
* replaces hardcoded value with well-defined constants
* moves createMembershipList to the rln relay utils module
* renames HashSize to HashHexSize
* minor updates on the comments
* reorganizes the consts
* indicates that rlnRelayMemIndex is an experimental option
* fixes a type conversion bug
* revises the unittest of "mount waku rln-relay off-chain"
* clarifies the use of index
* updates a docstring
* removes redundant constants and capitalize all of them
* deletes the ETH_CLIENT const from the test file
* renames a few vars for the sake of clarity
* reorganizes unittest into blocks of execution, debug messages, and checks
* adds more comments
* more comments and clarifications
* cleans up the tests
* minor
* adds a minor fix
* replaces a var usage with let
* fixes a bug
* tests rln instance as pointer
* test insertion
* tests deletion
* tests proof generation and verification
* updates rln instance type and the rln api
* deletes old API
* removes temporary tests
* deletes unused codes
* Delete settings.json
* reverts the changes in tests v2
* removes an old comment
* adds member insertion and deletion
* adds getMerkleRoot and unit tests
* makes insertMember argument a value type
* adds static group creation
* adds rln field to WakuRLNRelay type, and enables static group construction inside mountRlnRelay
* renames self to membershipKeyPair and removes key generation from mountRlnRelay
* renames sk,pk tp idKey and idCommitment
* updates mountRlnRelay arguments name
* logs created keys
* uncomments the key generation and adds explainer about it
* enables all the tests
* adds comments to the rln relay types
* logs error message for the arguments that are not set
* tests rln instance as pointer
* test insertion
* tests deletion
* tests proof generation and verification
* updates rln instance type and the rln api
* deletes old API
* removes temporary tests
* deletes unused codes
* Delete settings.json
* reverts the changes in tests v2
* removes an old comment
* adds timestamp to WakuRelayMessage
* converts float to float64
* generates timestamp using epochTime()
* tides up imports and generates timestamps using epochTime()
* updates changelog
* adds timestamp to the post_waku_v2_relay_v1_message query parameters
* Move Whisper type and protocol from nim-eth
- Import whisper_protocol and whisper_types
- Change import locations as appropriate
* Explicit errors in wakunode1
* cleans up store modules
* removes endians2 as an unused imports
* reorganizes imports
* fixes an ambiguous import
* adds comments re imports order
* adds message_store
* enables pagination based on sender timestamp
* uncomments a test
* bumps up version number
* updates migration script
* clean up
* unpdates changelog
* undo removal of receiver timestamp
* updates message_storage
* uses epochTime()
* minor
* removes a comment
* removes receiver timestamp deletion migration script
* fixes formatting issues
* fixes a bad field name
* fixes field issue
* adjusts spacing
* moves migration paths to the migration_types module
* moves USER_VERSION to the migration_types module
* updates migration execution condition to cover the peer store
* adds a message to the store if it is inserted to the db
* adds debug messages and checks migration path recursively
* deletes unused debug messages
* deletes commented codes
* clean up
* fixes a bug
* renames the first migration file of the message store
* adds timestamp to waku message store impl
* stores timestamp as int64
* adds untitest
* stores timestamp as seq of bytes
* minor
* re-orders unittest
* changes receiver timestamp to float64
* unit test for receiver timestamps
* adds comments
* reorder a few lines
* updates changelog
* more updates on changelog
* WIP: deduplicate
* add debug messages
* more debugging logs
* add timestamp to the message store
* converts resume result to uint64, removes some logs
* docstring for isDuplicate
* increments `added` before insertion into the db
* unit test for deduplication
* removes float to/from Bytes procs
* deletes extra spaces
* returns failed instead of the error
* minor
* minor
* adds changelog
In the `when isMainModule:` section of wakunode2.nim, the module
`./storage/message/waku_message_store` is imported, which in turn imports
waku's sqlite module and also the sqlite3_abi module from `vendor/`.
Outside of the `when isMainModule:` section, imports from waku's sqlite module
are not used in wakunode2. However, importing it at the top-level results in
the C code underlying sqlite3_abi to be compiled into whatever executable one
is building, and this is a problem if nim-waku is being consumed together with
another library, e.g. nim-sqlcipher, which links against a different build of
sqlite (sqlcipher being a superset of sqlite). The linker step fails with
reports of duplicate symbols for sqlite.
Lorenzo Delgado
Sanaz Taheri Boshrooyeh
Daniel Kaiser
Hanno Cornelius
G
rshiv
Richard Ramos
Pascal Precht
oskarth
Michael Bradley, Jr