Commit Graph

142 Commits

Author SHA1 Message Date
oskarth d3368324fe
Move Whisper types and protocol from nim-eth to this repo (#663)
* Move Whisper type and protocol from nim-eth

- Import whisper_protocol and whisper_types
- Change import locations as appropriate

* Explicit errors in wakunode1
2021-07-19 16:14:37 +08:00
Sanaz Taheri Boshrooyeh 32a8ffb867
Cleaning imports of the waku store module (#673)
* cleans up store modules

* removes endians2 as an unused imports

* reorganizes imports

* fixes an ambiguous import

* adds comments re imports order

* adds message_store
2021-07-16 15:28:35 -07:00
Hanno Cornelius 13cf7380bb
General refactoring: `nim-waku` (#671)
* General Track 1 refactoring
2021-07-16 17:13:36 +02:00
Sanaz Taheri Boshrooyeh a41b9ac80b
Refactor waku protocols and utils (#667)
* refactors waku_filter

* refactors waku_lightpush

* refactors waku_swap

* refactor namespacing.nim

* refactor peers

* refactors requests

* adds top level {.push raises: [Defect].}

* log scope for rln relay

* cleans up comments

* removes comments

* comments out raise Defect

* defines temp var then pass to constructor

* Update waku/v2/protocol/waku_rln_relay/rln.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* Update waku/v2/protocol/waku_swap/waku_swap.nim

Co-authored-by: oskarth <ot@oskarthoren.com>

* explains the potential exception in waku_swap

* creates temp var before return

* adjusts spaces

* adds line breaks, temp vars and fixes format

* removes type declaration

* fixes and indentation issue

* adjusts spacing

* adjusts line <80ch

* formating improvement

Co-authored-by: oskarth <ot@oskarthoren.com>
2021-07-15 11:25:52 -07:00
Hanno Cornelius 92f2d5a7f5
Refactor wakunode2.nim (#664)
* Refactor wakunode2.nim

* Remove empty raises

* Some formatting improvements
2021-07-14 19:58:46 +02:00
Sanaz Taheri Boshrooyeh c107072d9a
Waku store: explicit return instead of result (#665)
* replaces result with returns

* fixes a missing proc call

* removes unused comments
2021-07-14 08:59:42 -07:00
Sanaz Taheri Boshrooyeh 07586153f9
Adding error message to the History Responses (#660)
* adds error field to history response

* unittest for protobuf enc/dec

* renames HistoryResponseError to Store Error and simplifies pagination logic

* reverts the naming of StoreError to HistoryResponseError

* fixes a unittest for invalid cursor

* renames OK to NONE
2021-07-13 12:01:21 -07:00
Hanno Cornelius 2869e71865
Simplify wakunode2 subscriptions model (#662) 2021-07-13 09:18:51 +02:00
Sanaz Taheri Boshrooyeh cf7b8faf27
History response pagination using sender-generated timestamp (#657)
* enables pagination based on sender timestamp

* uncomments a test

* bumps up version number

* updates migration script

* clean up

* unpdates changelog

* undo removal of receiver timestamp

* updates message_storage

* uses epochTime()

* minor

* removes a comment

* removes receiver timestamp deletion migration script

* fixes formatting issues

* fixes a bad field name

* fixes field issue

* adjusts spacing
2021-07-07 16:56:20 -07:00
Sanaz Taheri Boshrooyeh cd703c6c52
Integrating pagination into the resume proc (#653)
* adds queryFromWithPaging

* adds unit test for paged queryFrom

* adds a new resume with paging capability

* unittest for resumePaging

* fixes the bug

* implementes a second version of query with paging

* integrates the second version of resume

* clean up

* adds the error message

* fixes a format issue

* renames variables

* defines DefaultPageSize

* adds a todo

* updates a docstring

* gets the pagesize as proc input

* updates unittest description

* removes unused gcsafe pragma

* better var naming

* updates changelog

* updates changelog

* updates the TODO

* more debug logs
2021-07-02 11:37:58 -07:00
Hanno Cornelius 734fe0795f
Add support for multiple protocol IDs (#652) 2021-07-02 10:49:41 +02:00
Hanno Cornelius 52b741ac11
Support for stable version of relay protocol (#651) 2021-06-29 16:29:04 +02:00
Sanaz Taheri Boshrooyeh 81b207624b
Add db migration support for peer store (#648)
* moves migration paths to the migration_types module

* moves USER_VERSION to the migration_types module

* updates migration execution condition to cover the peer store

* adds a message to the store if it is inserted to the db

* adds debug messages and checks migration path recursively

* deletes unused debug messages

* deletes commented codes

* clean up

* fixes a bug

* renames the first migration file of the message store
2021-06-25 14:06:56 -07:00
Ebube Sered Ud 6079ae8525
Fix issues with Peer reference in Cheque Functions (#635)
* Add function for getting peer Id

* Modify Handle Cheque and Send Cheque function

* Include PeerInfo in debit and credit functions

* Minor changes to queryWithAccounting function
2021-06-22 21:55:01 +01:00
Sanaz Taheri Boshrooyeh 30a7a7a4fe
Filter out duplicate messages in the resume procedure (#600)
* adds timestamp to waku message store impl

* stores timestamp as int64

* adds untitest

* stores timestamp as seq of bytes

* minor

* re-orders unittest

* changes receiver timestamp to float64

* unit test for receiver timestamps

* adds comments

* reorder a few lines

* updates changelog

* more updates on changelog

* WIP: deduplicate

* add debug messages

* more debugging logs

* add timestamp to the message store

* converts resume result to uint64, removes some logs

* docstring for isDuplicate

* increments `added` before insertion into the db

* unit test for deduplication

* removes float to/from Bytes procs

* deletes extra spaces

* returns failed instead of the error

* minor

* minor

* adds changelog
2021-06-22 11:30:12 +08:00
Michael Bradley, Jr c5da7c8fcd Remove unused import of ./storage/sqlite from wakunode2
In the `when isMainModule:` section of wakunode2.nim, the module
`./storage/message/waku_message_store` is imported, which in turn imports
waku's sqlite module and also the sqlite3_abi module from `vendor/`.

Outside of the `when isMainModule:` section, imports from waku's sqlite module
are not used in wakunode2. However, importing it at the top-level results in
the C code underlying sqlite3_abi to be compiled into whatever executable one
is building, and this is a problem if nim-waku is being consumed together with
another library, e.g. nim-sqlcipher, which links against a different build of
sqlite (sqlcipher being a superset of sqlite). The linker step fails with
reports of duplicate symbols for sqlite.
2021-06-18 13:56:37 -05:00
Ebube Sered Ud c1763362d7
Add Issuer to Cheque Object (#627)
* Add Issuer to Cheque Object closes  #626

* Minor changes

* Fix typo in comment

* Improve docs

* Fix Typo In Comment

* Minor changes

* Minor changes
2021-06-18 16:53:51 +01:00
Hanno Cornelius 2f53c2c60b
Feat/bridge topic mapping (#633)
* Use namespaced content topics for bridge
2021-06-18 12:30:24 +02:00
Ebube Sered Ud eb49d3148c
Improve Swap Account Balance Metrics to make use of a Histogram (#628)
Modify metrics description

Add more buckets to the swap account metrics
2021-06-17 16:36:44 +01:00
Sanaz Taheri Boshrooyeh ddf93814fe
Persisting Waku message timestamp & implementing DB migration & convert receiver timestamp data type to float64 (#607)
* adds timestamp to waku message store impl

* stores timestamp as int64

* adds untitest

* stores timestamp as seq of bytes

* minor

* re-orders unittest

* changes receiver timestamp to float64

* unit test for receiver timestamps

* adds comments

* reorder a few lines

* updates changelog

* more updates on changelog

* WIP

* WIP

* adds migration

* more debug messages

* passes the path to the migration scripts from message store module

* adds migration result type

* replaces migrationScripts with migrationScriptsResult

* adds path calculation to the message store

* removes some tests binary file

* removes redundant imports

* comments out user_version assignment in sqlite  init

* more descriptive err messages

* clean up test file

* more info logs

* minor code format

* removes a todo

* minor updates

* remove a binary file

* unit tests for migration utils

* adds split script

* integrates split query to handle scripts with multiple commands

* updates migration script for v1

* updates the v1 migration script

* update user version

* updates script

* fixes a few bugs on the splitScript

* more debug logs

* adds float64 parameter support to sqlite3

* change in timestamp type in the script

* deletes float64 toBytes utils

* enables storage of timestamp as a real number in the sqlite db

* bump up script index

* comment edits

* removes migrate unit test

* adds todo and docstring

* updates changelog

* removes an unused item in .gitignore

* minor

* updates changelog

* organizes imports

* cleans up imports

* WIP

* updates script


fixes a few bugs on the splitScript


more debug logs


adds float64 parameter support to sqlite3


change in timestamp type in the script


deletes float64 toBytes utils

* edits migration util test

* remove an empty test file

* includes migration utils tests in

* deletes unused codes

* tides up imports

* adds range based filter to the filterMigrationScripts

* renames procs: removes Migration

* tides up imports

* edits docstring

* edits docstring

* edits docstring

* removes unused imports

* more clean up

* groups std imports

* updates changelog

* adds docstring for setUserVersion

* adds unittest for the migrate

* Update waku/v2/node/storage/message/waku_message_store.nim

Co-authored-by: RichΛrd <info@richardramos.me>

* Update waku/v2/node/storage/sqlite.nim

Co-authored-by: RichΛrd <info@richardramos.me>

* Update waku/v2/node/storage/sqlite.nim

Co-authored-by: RichΛrd <info@richardramos.me>

* removes split scripts

* fixes a naming issue

* fixes a bug

* fixes a typo

* adds a log re updated user_version

* fixes a proc naming mismatch

* fixes naming mismatch

* more descriptive var names

* adds migration script of the first user version

* moves migration to after persistMessages flag is checked

* deletes unused comment

* fixes a bug

* brings back split script

* adds unit tests for split scripts

* runs scripts one command at a time

* deletes a commented line

* relocates the migrate proc to sqlite.nim

* adds unit test for filter scripts

* adds filterScripts unittest testing varying zero-prefixed user versions

* minor

Co-authored-by: RichΛrd <info@richardramos.me>
2021-06-16 13:23:55 -07:00
Hanno Cornelius a044c6a82c
Add ping protocol support and use for keep-alive (#621)
* Add ping protocol support and use for keep-alive

* Separate WakuPeerStore from libp2p PeerStore

* Revert silly test timeout
2021-06-15 10:55:47 +02:00
Kim De Mey e2ea94dc71
Remove the need for pcre (#623) 2021-06-15 09:51:09 +02:00
Ebube Sered Ud 76d4b89dbb
Add mode to waku swap (#601)
* Add mode to waku swap

* Modify swap test  and policy function

* Make rln optional (dependencies and compilation) (#602)

* makes rlnlib target optional

* makes the rln-related targets conditioned to the RLN command line flag

* adds the rln compiler flag condition

* adds the rln compiler flag condition to the rln module

* wakunode2 rln conditional compilation

* updates wakunode2 test

* updates waku rln relay tests

* removing the rln test from the default imports of the v2 tests

* imports rln module conditionally

* removes the rln flag condition from the rln module

* separates rln data types from its procs

* adds the import statement

* brings back the contract def to the rln utils

* adds rln module import to the rln unit tests

* clean up and reorganization

* adds a todo

* minor edits on a comment

* Fix indentation and typo

* Add Swap Config Object and set default config value on mount

* Minor changes

* Fix issues with grammar in code comment

* Create init procedure for setting default values in SwapConfig

* Minor changes to mountSwap call in wakunode2

* Improve Docs on Swap Mode

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2021-06-15 10:06:36 +08:00
Oskar Thorén be717ce5f3
Improve libwaku and C wrappers (#613)
- Move stuff to Makefile
- Move libwaku to root
- Make examples compile for C and Go
- Update README
2021-06-13 20:50:10 +08:00
Hanno Cornelius a8da4dfd6b
Fix: metrics-server use after submodule update (#610)
* Fix: metrics-server use after submodule update

* Remove insecure check
2021-06-10 16:18:41 +02:00
Hanno Cornelius 9e0b73bd18
Update submodules (#594) 2021-06-09 16:37:08 +02:00
Sanaz Taheri Boshrooyeh 2fe53c334c
Make rln optional (dependencies and compilation) (#602)
* makes rlnlib target optional

* makes the rln-related targets conditioned to the RLN command line flag

* adds the rln compiler flag condition

* adds the rln compiler flag condition to the rln module

* wakunode2 rln conditional compilation

* updates wakunode2 test

* updates waku rln relay tests

* removing the rln test from the default imports of the v2 tests

* imports rln module conditionally

* removes the rln flag condition from the rln module

* separates rln data types from its procs

* adds the import statement

* brings back the contract def to the rln utils

* adds rln module import to the rln unit tests

* clean up and reorganization

* adds a todo

* minor edits on a comment
2021-06-08 11:56:32 -07:00
Ebube Sered Ud e2185caf27
Add Swap account Metrics to Grafana (#585)
* Fix gcsafe issues with log account metrics function

Signed-off-by: kripsonud <kripsonud@gmail.com>

* Change Line ending to LF

* Minor changes to waku_swap

* Mount keepalive on chat2bridge and bridge (#591)

* Light push beta bump (#590)

Closes https://github.com/status-im/nim-waku/issues/565

* Filter and lightpush showcase (#593)

* Filter and lightpush showcase

* Fix log error

* Add swap account panel to grafana dashboard template

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-06-06 16:15:18 +01:00
Hanno Cornelius b615c6aa8b
Filter and lightpush showcase (#593)
* Filter and lightpush showcase

* Fix log error
2021-06-04 12:02:47 +02:00
Oskar Thorén 2568af3b98
Light push beta bump (#590)
Closes https://github.com/status-im/nim-waku/issues/565
2021-06-03 13:29:04 +08:00
Hanno Cornelius 83f546ede4
Mount keepalive on chat2bridge and bridge (#591) 2021-06-02 16:54:38 +02:00
Hanno Cornelius 2b571e205b
Fix keepalive for connected peers (#588)
* Fix keepalive for connected peers

* Remove comment
2021-06-02 09:53:34 +02:00
Hanno Cornelius ee757308c6
Fix: No keep-alive for backoff peers (#584) 2021-06-01 09:55:46 +02:00
Sanaz Taheri Boshrooyeh 8f741de483
A quick fix to the master issue (#579)
* moves resume to after the relay is mounted

* Update waku/v2/node/wakunode2.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2021-05-28 11:41:29 -07:00
Ebube Sered Ud cc5f55dbbb
Bump swap protocol to beta1 (#578)
* Bump swap protocol to beta1
2021-05-28 17:13:17 +01:00
Sanaz Taheri Boshrooyeh f75f7c4751
Add resume to waku node api (#564)
* activates resume

* adds unit test

* add store condition

* mode debug info

* updates wakunode resume api and its unittest

* adds todo

* adds documentation

* edits resume documentation

* further edits on the docs

* removes a todo

* fixes a bug

* add resume to waku node api

* further updates on the node.md

* updates the changelog

* minor

* removes return type

* adds a brief desc of the resume to node.md
2021-05-26 12:33:22 -07:00
Ebube Sered Ud e6d7e3a2b4
Improve SWAP logging and enable by default (#549)
* Enable Swap by default
* Add PeerId and amount to Logs
* Add Policy function in swap protocol
* Modified Changelog to reflect changes to swap config
* Commented out test for updating account state after a cheque has been sent
2021-05-26 11:05:56 +01:00
Hanno Cornelius e0d8d8f468
Minor bridge improvements (#563) 2021-05-24 13:19:33 +02:00
Sanaz Taheri Boshrooyeh c9e94cba41
Enable time-based history query in the store rpc api (#560)
* adds time based query

* updates the changelog
2021-05-21 10:36:24 -07:00
Sanaz Taheri Boshrooyeh 9b38e5c893
Ft-store: Peer management (#548)
* adds an optional list of peers to the resume proc

WIP working


adds unit tests

* cleans up and adds comments

* adds unittest for queryFrom

* converts queryLoop to a private func

* elaborates on the peer selection of the resume proc

* minor format correction

* returns the status of the resume call as a Result object

* updates unittest based on the new return type

* defines QueryResult type
2021-05-19 12:28:09 -07:00
Sanaz Taheri Boshrooyeh 89909d281f
Fault tolerant store protocol: resume message history (#539)
* adds resume function

* unittest for findLastSeen

* fixes a bug in find last seen unit test

* argument type change to indexed waku message

* relocates findLastSeen

* adds unit test for resume

* adds comments

* adds a TODO

* adds offset

* cleans up

* modifies some of tests title and deletes a duplicate unit test

* more condition on running resume()

* adds more detailed docstring and simplifies if statement

* Update waku/v2/protocol/waku_store/waku_store.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* new doc string

* Update waku/v2/protocol/waku_store/waku_store.nim

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

* fixes a bug

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-05-13 14:21:46 -07:00
Hanno Cornelius 1ae767206d
Remove deprecated config items (#545) 2021-05-12 12:55:09 +02:00
Hanno Cornelius 50f2235bfc
Fix nim-waku config inconsistencies (#543)
* Fix nim-waku config inconsistencies

* Revert staticnode to singular form

* Fix reference to deprecated config item

* Revert nodekey config item to be consistent with Waku v1
2021-05-11 17:07:26 +02:00
Hanno Cornelius c5e7580149
Add chat2-matterbridge (#532)
* Add chat2-matterbridge

* Update waku/common/README.md

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-05-06 15:43:43 +02:00
Sanaz Taheri Boshrooyeh 7c5df3379b
Storing waku message version inside the store db (#530)
* WIP

* stores version in db, edits error messages

* fixes a bug

* reverts edit messages

* converts uint32 storage to int64

* unit tests for various version numbers

* minor comments edit

* removes debugging messages

* updates changelog
2021-05-05 15:51:36 -07:00
Ebube Sered Ud a6bb3f65e1
Fix redundant use of content topics (#528)
made resulting changes to waku_filter

Made changes to wakunode2, filter_api and waku_filter

Update waku v2 test scripts referencing Content Topics

Update ContentFilter in chat example

Remove unneccesary loops from filter api
closes #496

Apply keep-alive for chat2 (#525)

Makes the arguments of the store jsonrpc api optional (#526)

* makes pubsubTopic filter optional

* makes contentFilter optional

* append Option

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Update Changelog with changes to ContentFilter

Fix indentation and code semantics

Enables perssist-message flag in the store protocol for wakunode2 (#519)

* enables perssistmessage flag

* disables in memory storage when persist-messages is false

* adds the persistMessages input to the mountStore

* defaults the store flag to true

* adds the missing argument

* persists messages in memory conditioned to the persistMessages flag

* adds persistmessages flag to the config_bridge

* defaults persistmessages to true

* defaults the store flag to true and persist-messages to false

* updates store.md

* updates chat2 instructions about --store flag

* removes --store flag from chat2 command execution

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Fix: light-mode relay for all light protocols (#529)

* Fix: light-mode relay for all light protocols

* Clear up confusing use of overloaded concepts

Fix ContentFilter Schema in wakunode test script

Enables perssist-message flag in the store protocol for wakunode2 (#519)

* enables perssistmessage flag

* disables in memory storage when persist-messages is false

* adds the persistMessages input to the mountStore

* defaults the store flag to true

* adds the missing argument

* persists messages in memory conditioned to the persistMessages flag

* adds persistmessages flag to the config_bridge

* defaults persistmessages to true

* defaults the store flag to true and persist-messages to false

* updates store.md

* updates chat2 instructions about --store flag

* removes --store flag from chat2 command execution

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>

Fix: light-mode relay for all light protocols (#529)

* Fix: light-mode relay for all light protocols

* Clear up confusing use of overloaded concepts

Fix resulting issues after merge
2021-05-05 09:34:40 +01:00
Hanno Cornelius 36ff537612
Fix: light-mode relay for all light protocols (#529)
* Fix: light-mode relay for all light protocols

* Clear up confusing use of overloaded concepts
2021-05-04 14:11:41 +02:00
Sanaz Taheri Boshrooyeh c0d858af1d
Enables perssist-message flag in the store protocol for wakunode2 (#519)
* enables perssistmessage flag

* disables in memory storage when persist-messages is false

* adds the persistMessages input to the mountStore

* defaults the store flag to true

* adds the missing argument

* persists messages in memory conditioned to the persistMessages flag

* adds persistmessages flag to the config_bridge

* defaults persistmessages to true

* defaults the store flag to true and persist-messages to false

* updates store.md

* updates chat2 instructions about --store flag

* removes --store flag from chat2 command execution

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-05-03 14:30:52 -07:00
Sanaz Taheri Boshrooyeh 7b6d93d13b
Makes the arguments of the store jsonrpc api optional (#526)
* makes pubsubTopic filter optional

* makes contentFilter optional

* append Option

Co-authored-by: Oskar Thorén <ot@oskarthoren.com>
2021-05-03 12:25:28 -07:00
Hanno Cornelius f457db9e0a
Add optional keepAlive for relay peers (#523) 2021-04-30 14:07:46 +02:00