Commit Graph

1197 Commits

Author SHA1 Message Date
Aaryamann Challani debc5f19d9
feat(relay): ordered validator execution (#1966)
* feat(relay): ordered validator execution

* fix: make more readable

* test: ignore accepts only

* fix: idempotent .subscribe

* fix(rln-relay): make validators private

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* fix: include comments, unsubscribe behaviour

* fix: compilation

---------

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
2023-09-05 14:35:07 +05:30
Aaryamann Challani 97a7c9d04d
chore(rln-relay): log levels for certain logs (#1986) 2023-09-04 18:43:59 +05:30
Aaryamann Challani a14c32614a
chore(rln-relay): use the only key from keystore if only 1 exists (#1984)
* chore(rln-relay): use the only key from keystore if only 1 exists

* fix: convert iterator to seq and then index into it
2023-09-04 15:46:44 +05:30
Aaryamann Challani be48891f77
chore(rln-relay): confirm that the provided credential is correct using onchain query (#1980) 2023-09-04 11:22:31 +05:30
Alvaro Revuelta 05c988645d
chore(api): validate rln message before sending (rest + rpc) (#1968) 2023-09-01 15:03:59 +02:00
Simon-Pierre Vivier 037b166263
fix: --topic should be ignore when using --pubsub-topic or --content-topic (#1977) 2023-08-31 16:13:45 -04:00
Aaryamann Challani d262837ea5
fix(rln-relay): deserialization of valid merkle roots (#1973) 2023-08-31 13:19:43 +05:30
richΛrd e272bec9c5
chore: add debug log indicating succesful message pushes and also log the message hash (#1965) 2023-08-30 18:10:51 -04:00
Aaryamann Challani e7b2b88f5b
fix(rln-relay): modify keystore credentials logic (#1956)
* fix(rln-relay): modify keystore credentials logic

fix: bump version

* Update waku/waku_rln_relay/group_manager/on_chain/group_manager.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* Update tests/waku_rln_relay/test_waku_rln_relay.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* Update waku/waku_keystore/protocol_types.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* fix: greatly improve error handling

* fix: display proc and appropriate assert

---------

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
2023-08-29 17:46:21 +05:30
Aaryamann Challani bd3be21929
fix(rln-relay): sync from deployed block number (#1955)
* fix(rln-relay): sync from deployed block number

* fix(rln-relay): remove option usage for ints

* fix: unnecessary decl
2023-08-29 13:12:13 +05:30
Aaryamann Challani 01634f57f0
fix(rln-relay): window of acceptable roots synced to rln metadata (#1953)
* fix(rln-relay): window of acceptable roots synced to rln metadata

* fix(rln-relay): s/var/let, use for loop
2023-08-25 23:29:17 +05:30
Aaryamann Challani cc9f8d4254
chore(rln-relay): integrate waku rln registry (#1943) 2023-08-25 22:48:52 +05:30
Simon-Pierre Vivier 4539dfc761
feat(discv5): topic subscriptions update discv5 filter predicate (#1918) 2023-08-23 11:50:59 -04:00
Simon-Pierre Vivier c369b3294e
feat: topic subscriptions updates discv5 ENR (#1875) 2023-08-23 09:53:17 -04:00
Aaryamann Challani c07d63db46
fix(rln-relay): flush_interval incorrectly set (#1933) 2023-08-23 18:23:44 +05:30
Aaryamann Challani 1ae5b5a951
fix(rln-relay): RLN DB should be aware of chain and contract address (#1932) 2023-08-23 18:23:30 +05:30
Aaryamann Challani 8239b45582
chore(rln-relay): rename keystore application to waku-rln-relay (#1924) 2023-08-23 12:02:03 +05:30
Alvaro Revuelta 56c228f815
chore(rln): remove old and add new rln metric (#1926) 2023-08-22 16:30:33 +02:00
Aaryamann Challani 6c6302f9e1
fix(rln-relay): waitFor startup, otherwise valid proofs will be marked invalid (#1920) 2023-08-22 17:47:06 +05:30
Aaryamann Challani f08315cdde
fix(rln-relay): remove registration capability (#1916) 2023-08-22 11:49:18 +05:30
Alvaro Revuelta af95b5713f
chore(rln): run rln in all relay pubsubtopics + remove cli flags (#1917) 2023-08-21 08:55:34 +02:00
Aaryamann Challani b3bb7a1113
fix(rln-relay): invalid start index being set results in invalid proofs (#1915)
* fix(rln-relay): invalid proof usage

* fix(rln-relay): use startIndex from first event in block

* fix: latestIndex set after registerBatch
2023-08-18 16:38:24 +05:30
Simon-Pierre Vivier cf3013962c
feat: update various protocols to autoshard (#1857)
* feat: update FILTER & LIGHTPUSH to autoshard
2023-08-17 08:11:18 -04:00
Aaryamann Challani 8c568cabbe
fix(rln-relay): should error out on rln-relay mount failure (#1904) 2023-08-16 19:00:10 +05:30
Aaryamann Challani 8bcb0acf1a
test(rln-relay): rpc handler to support waku rln relay (#1852) 2023-08-16 10:32:22 +02:00
Hanno Cornelius b9d5d28af5
chore: remove references to v2 (#1898)
* chore: remove references to v2

* fix: lingering rln-relay import path
2023-08-09 18:11:50 +01:00
Alvaro Revuelta 03363f1bd8
fix: fixes out of bounds crash when waku2 is not set (#1895) 2023-08-08 16:22:53 +02:00
Hanno Cornelius ab344a9dec
chore: remove Waku v1 and wakubridge code (#1874)
* chore: remove v1 code

* chore: deprecate support for v1 compatibility from JSON-RPC API

* chore: remove v1 tests from JSON-RPC suite

* chore: remove wakubridge code

* chore: remove v1 compatibility functions

* chore: remove support for v1 payloads from chat2 app

* chore: remove v1 from CI processes

* fix: lingering references to private API functions

* fix: remove v1 compat from chat2

* fix: some more lingering references in tests
2023-08-07 15:11:46 +01:00
NagyZoltanPeter 08ff667227
feat: Rest API interface for legacy (v1) filter service. (#1851)
* Added Rest API interface for legacy (v1) filter service with tests.
2023-08-04 11:34:22 +02:00
Vaclav Pavlin 2fc48842b7
fix: check nil before calling clearTimer (#1869) 2023-08-02 13:31:14 +02:00
Aaryamann Challani 4756ccc179
fix(rln-relay): mark duplicated messages as spam (#1867)
* fix(rln-relay): mark duplicated messages as spam

* chore: fix fn desc
2023-08-02 10:40:18 +05:30
Simon-Pierre Vivier bbff1ac138
feat: autosharding core algorithm (#1854)
- basic rendezvous hashing
- content topic parsing
- sharding config
- tests
2023-08-01 09:05:16 -04:00
Aaryamann Challani 76c73b62cd
feat(rln-relay): close db connection appropriately (#1858) 2023-07-27 17:21:21 +05:30
Tanguy 08f3bba334
feat: enable TcpNoDelay (#1470) 2023-07-15 16:41:26 +02:00
Ivan Folgueira Bande 1d3410c7bd
fix(libp2p): Updating nim-libp2p to fix the `wss` connectivity issue (#1848)
* Updating nim-libp2p to fix the `wss` connectivity issue

* The `nim-libp2p` adds many changes although we are actually interested
in the latest commit:

> wstransport.nim: avoid re-raising 'TransportOsError' to avoid stopping
`switch.accept` (#929)

* The `nim-stew` bump is needed so that the `nim-libp2p` can compile.

* The changes in `waku_node.nim` are needed due to the changes in
`nim-stew`.

* waku_node.nim: returning a "completed" future within 'statusAndConfidenceHandler'

  This is aimed to avoid the next exception happening in the wakunode:

  ```
  Unhandled defect: Async procedure (service.nim(74)    callHandler)
  yielded `nil`, are you await'ing a `nil` Future? [AssertionDefect]
  ```

  * Adding 'async' to 'statusAndConfidenceHandler' so that it properly returns a Future[void]
2023-07-13 18:00:55 +02:00
Aaryamann Challani 3fe4522a7e
chore(rln-relay): verify proofs based on bandwidth usage (#1844)
* chore(rln-relay): Verify proofs based on bandwidth usage

* fix(rln-relay): make default threshold 0, for backwards compat

* fix(rln-relay): add unit test for bandwidth cutoff

* fix(rln-relay): rlnRelayBandwidthTreshold option
2023-07-07 17:28:37 +05:30
Alvaro Revuelta 17b24cded6
feat: limit relay connections below max conns (#1813) 2023-07-04 13:31:18 +02:00
Ivan Folgueira Bande 97d3b9f79f
chore(refactor): Move record creation & fix libwaku compilation (#1833)
* Move record creation & fix libwaku

* app,external_config,internal_config,libwaku,sharding: refactorgin to make it compile

---------

Co-authored-by: SionoiS <simvivier@status.im>
2023-06-29 21:59:53 +02:00
Ivan Folgueira Bande 88b7481f29
feat(postgres): integration of postgres in wakunode2 (#1808)
* Making the wakunode2 to support postgres driver

* driver/builder.nim: controling possible errors when creating the messages table

* postgres_driver.nim: adding protection in getInt and fixing typo
2023-06-28 18:47:28 +02:00
Simon-Pierre Vivier cf46fb7cf6
chore: discv5 re-org clean-up (#1823)
- Remove discv5 from node and it's builder
- Require a record to init a node (was previously using discv5 record).
- Fix various tests that assumed nodes with discv5.
- Fix ENR creation
- Fix wakubridge
- Fix networkmonitor
2023-06-28 08:57:10 -04:00
Aaryamann Challani e4d9ee1f7a
fix(rln-relay): chunk event fetching (#1830) 2023-06-28 18:00:45 +05:30
Alvaro Revuelta e12c979c4e
chore(networking): disconnect due to colocation ip in conn handler (#1821) 2023-06-28 09:14:11 +02:00
Ivan Folgueira Bande 3aefade647
fix(discv5): Fixing issue that prevented the wakunode2 from starting (#1829)
* Fixing issue that prevented the wakunode2 from starting

The issue was introduced in PR#1818.

Before this commit, the `wakunode2` app crashed with the next error:

ERR 2023-06-27 15:57:27.268+00:00 5/7 Starting node and protocols failed
topics="wakunode main" tid=1 file=wakunode2.nim:92 error="failed to
start waku discovery v5: "

* fix tests accordingly

* publisher.nim, subscriber.nim: fix
2023-06-27 15:16:59 -04:00
Simon-Pierre Vivier 62d3653022
chore: move discv5 out of node. (#1818)
- Refactor discv5 start, stop & loop.
- Fix tests.
2023-06-27 09:50:11 -04:00
Ivan Folgueira Bande 52894a82d0
refactor(archive): Moving waku archive logic from app.nim to the archive module (#1817)
* Refactoring the Waku Archive. Simplifying the app.nim

This change is needed to accommodate the further PRs where we will integrate Postgres in `wakunode2`.
2023-06-27 13:24:31 +02:00
Alvaro Revuelta 71c4ac1641
chore: add peer manager config to builder (#1816) 2023-06-23 15:30:28 +02:00
Simon-Pierre Vivier 44f9d8dc0e
chore: discv5 re-org setup (#1815)
Key, netconfig, enr are created at App initialization and config files has been renamed.
2023-06-22 16:58:14 -04:00
Ivan Folgueira Bande a44d4bfbcd
refactor(databases): Creation of the databases folder to keep the logic for sqlite and postgres (#1811)
* Refactoring in sqlite and postgres. Creation of the databases folder.
2023-06-22 11:27:40 +02:00
Simon-Pierre Vivier a4da87bb8c
feat: discovery peer filtering for relay shard (#1804)
Add discv6 predicate that filter peer by static shard.

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2023-06-20 12:08:10 -04:00
Simon-Pierre Vivier 50412d1880
sharded topic validation & ENR update (#1795)
Add sharded topic validation and update the node ENR accordingly.
2023-06-19 18:16:25 -04:00
Aaryamann Challani bbded9eea7
feat(rln-relay): resume onchain sync from persisted tree db (#1805)
* feat(rln-relay): resume onchain sync from persisted tree db

* chore(rln-relay): bump zerokit
2023-06-19 15:21:18 +05:30
Aaryamann Challani 045f07c616
feat(rln-relay): metadata ffi api (#1803)
* feat(rln-relay): metadata ffi api

* chore(rln-relay): bump to latest master after merge

* fix(rln-relay): naming, visibility, tests
2023-06-16 11:33:41 +05:30
Tanguy f6e89c311d
chore(deps): bump libp2p & websock (#1800) 2023-06-15 13:17:19 +02:00
Aaryamann Challani 940206b439
feat(rln-relay): integrate get_leaf ffi api (#1790) 2023-06-12 15:30:07 +05:30
Ivan Folgueira Bande 5fc5770da9
feat(postgres): complete implementation of driver and apply more tests (#1785) 2023-06-09 12:42:33 +02:00
Tanguy 613974c1b6
Revert "chore(libp2p): update nim-libp2p (#1786)" (#1788)
This reverts commit 93e09b8b77.
2023-06-09 10:10:47 +02:00
Aaryamann Challani dba84248f1
chore(rln-relay): pass in the path to the tree db (#1782)
* chore(rln-relay): pass in the path to the tree db

* fix(rln-relay): address visibility

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* fix(rln-relay): make db used more explicit

* fix(rln-relay): reduce visibility

---------

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
2023-06-08 17:22:21 +05:30
Vaclav Pavlin 93e09b8b77
chore(libp2p): update nim-libp2p (#1786) 2023-06-08 12:54:28 +02:00
Aaryamann Challani ba8ec7048e
chore(rln-relay): update tree_config (#1781) 2023-06-07 13:39:20 +05:30
Ivan Folgueira Bande cb2e3d86a6
feat(postgres): Adding a postgres async pool to make the db interactions asynchronous (#1779)
* feat(postgres): added postgres async pool wrapper

---------

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2023-06-07 10:08:43 +02:00
Aaryamann Challani a00aa8cc59
feat(rln-relay): pass in index to keystore credentials (#1777)
* feat(rln-relay): pass in index to keystore credentials

* fix(rln-relay): s/MembershipIndex/uint
2023-06-06 23:13:05 +05:30
Alvaro Revuelta 34a9263191
feat(networking): integrate gossipsub scoring (#1769) 2023-06-06 19:28:47 +02:00
Hanno Cornelius 21737c7c1b
feat(discv5): added find random nodes with predicate (#1762) (#1763)
* feat(discv5): added find random nodes with predicate (#1762)

Co-authored-by: Lorenzo Delgado <lnsdev@proton.me>

* chore: play around with sleep times on tests

* chore: simplify flaky tests

* chore: update waku/v2/node/waku_node.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* chore: update tests/v2/test_waku_discv5.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* chore: update examples/v2/subscriber.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

---------

Co-authored-by: Lorenzo Delgado <lnsdev@proton.me>
Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
2023-06-06 16:36:20 +02:00
Vaclav Pavlin 835a409d10
feat(wakunode2): enable libp2p rendezvous protocol by default (#1770) 2023-06-01 21:43:10 +02:00
Ivan Folgueira Bande 7df6f4c851
feat(postgresql): align Andrea's PR#1590 changes into master (#1764)
* Add postgres basic support
* test_driver_postgres.nim: adding simple test to validate asynchronous-ness
* Aligning the changes made by Andrea to the current master state
* test_driver_postgres.nim: new unit test (checking duplicate insertion)

---------

Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2023-05-31 10:28:48 +02:00
Alvaro Revuelta 047d1cf095
feat(networking): prune peers from same ip beyond colocation limit (#1765) 2023-05-31 09:47:56 +02:00
Ivan Folgueira Bande 59ca03a875
feat(postgresql): 1st commit to async sql (waku_archive/driver...) (#1755) 2023-05-25 17:34:34 +02:00
Alvaro Revuelta 3c2d2891e5
chore(px): close px streams after resp is sent (#1746) 2023-05-25 16:42:48 +02:00
Aaryamann Challani 6257874633
chore(rln-relay): updated metrics for testnet 3 (#1744)
* fix(rln-relay): trace log

* chore(rln-relay): updated metrics for testnet 3

fix(rln-relay): group manager metric import
2023-05-19 16:24:35 +05:30
Aaryamann Challani 5eae60e817
fix(rln-relay): trace log (#1743) 2023-05-19 12:52:46 +05:30
Alvaro Revuelta 87f694a8b6
chore(networking): set and use target outbound connections + prune (#1739) 2023-05-18 09:40:14 +02:00
Aaryamann Challani 611e9539a6
feat(rln-relay): use new atomic_operation ffi api (#1733)
* chore(rln-relay): bump zerokit

* feat(rln-relay): use new atomic_operations ffi api

* fix(rln-relay): static gm
2023-05-18 10:42:08 +05:30
Ivan Folgueira Bande 665484c17b
refactor: proper use of setupNat (#1740)
Notice that I had to adapt to use 'rlpx_connected_peers' instead
of 'connected_peers' in 'wakunode1.nim' because due to the update
of the 'vendor/nim-eth', which adds the dependency-break with
'confutils' but also includes another changes.

Aside note: we cannot have 'confutils' dependency in 'nim-eth' because
that will prevent the generation of any waku dynamic library.
2023-05-17 18:32:53 +02:00
Ivan Folgueira Bande 2defbd2301
feat(cbindings): first commit - waku relay (#1632) (#1714)
* feat(cbindings): first commit - waku relay (#1632)

* test_app.nim: fix compilation issue. App.init(..) -> App.new(..)

* Simplifying library name (libwaku) and standardizing function names (waku_*)

* Proper wrapper of the waku_node API and creation of the libwaku.a

* Rolling back changes that are not needed

* Rolling back changes that are out of the scope of this task

* wakunode.nim: Removing unnecessary import

* Aplying PR suggestions

* Renaming 'waku.h' -> 'libwaku.h'

* Use of 'isNil' instead of '== nil'

* libwaku.nim: explicitly setting waku_poll() as gcsafe
2023-05-12 18:08:41 +02:00
Aaryamann Challani b277ce1013
chore(rln-relay): update args to contract (#1724)
* chore(rln-relay): update args to contract

* fix(rln-relay): update membership contract code

* fix: bytecode
2023-05-11 19:30:26 +05:30
Alvaro Revuelta b9e0763e6b
fix: streams was used instead of connections (#1722) 2023-05-10 15:28:00 +02:00
Aaryamann Challani 44c543129e
chore(rln-relay): use new config for ffi (#1718)
* chore(rln-relay): use new config for ffi

* chore: update zerokit submodule

* fix: missing comma
2023-05-10 12:54:58 +05:30
Hanno Cornelius 7c39be9a2e
fix: change filter request default behaviour to ping (#1721) 2023-05-09 09:18:14 +02:00
Hanno Cornelius 68a39c65bd
fix: fix filter v2 proto fields (#1716) 2023-05-05 10:29:53 +02:00
Alvaro Revuelta 3e0a693d2e
chore: add timestamp and ephemeral for opt-in dos validator (#1713) 2023-05-05 10:12:49 +02:00
Alvaro Revuelta 3e14686950
chore: add validator for dos protec metrics and move to app (#1704) 2023-05-02 16:20:38 +02:00
fryorcraken.eth ba12d1c1e4
Fix links and add notes on Waku v1 deprecation (#1707) 2023-05-02 09:56:25 +02:00
Lorenzo Delgado ce92fc1aed
refactor(wakunode2): flatten and simplify app setup (#1705) 2023-04-26 19:25:18 +02:00
Vaclav Pavlin b397ed6cbd
refactor:optimize getting number of connections and streams (#1673) 2023-04-26 10:47:46 +02:00
Alvaro Revuelta c6d291d3f3
chore: add payload bytes to trace log (#1703) 2023-04-25 09:50:43 +02:00
Lorenzo Delgado e041e04342
chore(core): move peers utils module to waku_core 2023-04-24 16:37:54 +02:00
Alvaro Revuelta 67e96ba894
feat: add metrics with msg size histogram (#1697) 2023-04-20 13:45:29 +02:00
Alvaro Revuelta 7d12adf636
fix: import nimchronos instead heartbeat (#1695) 2023-04-19 16:51:15 +02:00
Lorenzo Delgado 93b0c0713e
chore(core): move utils time module to waku_core 2023-04-19 16:39:52 +02:00
Alvaro Revuelta 14305c610a
feat: curate peers shared over px protocol (#1671) 2023-04-19 16:12:00 +02:00
Lorenzo Delgado d5ef9331e7
fix(rest): change rest server result error type to string 2023-04-19 14:55:39 +02:00
Lorenzo Delgado e8dceb2aa6
chore: add deprecation notice to utils module. move heartbeat to common 2023-04-19 14:27:16 +02:00
Ivan Folgueira Bande 81d03558b1
Adding more detail in an error produced by 'parsePeerInfo' (#1687)
* Adding more detail in an error produced by 'parsePeerInfo'

* utils/peers.nim: casting to 'string' in a more idiomatic/convenient way
2023-04-19 14:02:54 +02:00
Lorenzo Delgado c9b6b2308c
chore(core): rename waku_message module to waku_core 2023-04-19 13:29:23 +02:00
Ivan Folgueira Bande edd8419a1b
peers.nim: adding more detail in case the p2pPart doesn't contain '/' (#1684) 2023-04-19 10:24:06 +02:00
Lorenzo Delgado d7b72ac736
chore: flatten waku v2 protocols folder 2023-04-18 15:22:10 +02:00
Aaryamann Challani b62193e59b
fix(rln-relay): scope of getEvents (#1672) 2023-04-18 16:55:10 +05:30
Lorenzo Delgado 9616253611
feat(enr): added support for relay shards field 2023-04-17 15:21:20 +02:00
Alvaro Revuelta 1a885b963b
fix(logs): fix log reporting wrong ok connected peers (#1675) 2023-04-17 11:46:15 +02:00
Vaclav Pavlin d5c2770c54
bug: move canBeConnected to PeerManager and check for potential overflow (#1670) 2023-04-14 15:12:22 +02:00
Alvaro Revuelta 73cbafa658
chore(networking): get relay number of connections from protocol conns/streams (#1609) 2023-04-12 13:05:34 +02:00
Ivan Folgueira Bande b2dcb07751
Adding parsePeerInfo and deprecating 'parseRemotePeerInfo' (#1658) 2023-04-12 11:29:11 +02:00
Lorenzo Delgado 9e1432c9ef
fix: wrap untracked protocol handler exceptions 2023-04-11 13:12:45 +02:00
Hanno Cornelius 418efca27f
feat: integrate new filter protocol, other improvements (#1637) 2023-04-11 10:12:54 +02:00
Lorenzo Delgado 1cfb251b65
fix(wakunode2): made setup nat return errors 2023-04-06 13:41:05 +02:00
Ivansete-status b2acb54d6a
feat(rest-api-store): new rest api to retrieve store waku messages (#1611) (#1630)
* feat: new rest api based on the current store json-rpc api and
following the same structure as the current relay rest api.

* feat: the store api attend GET requests to retrieve historical messages

* feat: unit tests.

* feat: allow return message to rest-client in case error (4XX or 5XX)

* chore: always allow to call the store api endpoints (only rest) without explicit storenode (#1575)

* feat: always mounting the current node as storenode client
2023-04-06 11:43:19 +02:00
Lorenzo Delgado 9dadc1f5a0
refactor(node): remove deprecated constructor and extend testlib with builder 2023-04-05 16:01:51 +02:00
Alvaro Revuelta 5d853b8630
chore: do not mount relay more than once (#1650) 2023-04-05 15:12:49 +02:00
Lorenzo Delgado e931fa5dc8
feat(node): added waku node builder type 2023-04-05 14:27:11 +02:00
Lorenzo Delgado e8448dfdbe
chore: pointed all waku node imports to the barrel import 2023-04-05 11:58:59 +02:00
Lorenzo Delgado 139428883f
refactor(node): added waku_node barrel import and split config module 2023-04-05 09:46:13 +02:00
Lorenzo Delgado 0627b4f8f2
chore: remove deprecated enr record init method 2023-04-04 15:35:15 +02:00
Lorenzo Delgado caf78249b2
fix: fixed multiple bare except warnings 2023-04-04 15:34:53 +02:00
Alvaro Revuelta c26dcb2b39
feat: dos protected topic relay msgs based on meta field (#1614) 2023-04-04 10:58:45 +02:00
Lorenzo Delgado 2b5fd2a21f
chore: remove waku swap protocol 2023-03-31 17:21:40 +02:00
Aaryamann Challani 00a3812b91
chore(rln-relay): gracefully handle chain forks (#1623)
* chore(rln-relay): gracefully handle chain forks

* fix(rln-relay): better root windowing

* fix(rln-relay): better future generation for test

* fix(rln-relay): reduced width

* fix: better naming of futs, collision free
2023-03-31 19:15:04 +05:30
Lorenzo Delgado 048ca45dc9
build: added testcommon target to makefile 2023-03-31 15:24:04 +02:00
Alvaro Revuelta 1f79375679
chore: increase meta size to 64 bytes + tests (#1629) 2023-03-30 10:38:45 +02:00
Lorenzo Delgado 2ffd2f8010
refactor(enr): move waku enr multiaddr to typedrecod and builder extensions 2023-03-30 09:35:13 +02:00
Aaryamann Challani b1bafda2ee
fix(rln-relay): on chain registration (#1627)
* fix(rln-relay): on chain registration

* fix(rln-relay): bump gas lim
2023-03-29 14:22:57 +05:30
Alvaro Revuelta 85f33a8efd
bug: connect instead dial relay peers (#1622) 2023-03-28 13:29:48 +02:00
Alvaro Revuelta c42ac16fe6
bug: fix hash size greater than 32 (#1621) 2023-03-24 13:06:27 +01:00
Lorenzo Delgado 157724d9b9
refactor(enr): added waku2 capabilities accessor 2023-03-22 16:01:02 +01:00
Hanno Cornelius d920b973b1
feat: further filter improvements (#1617) 2023-03-22 10:32:53 +02:00
Lorenzo Delgado ac56e1dcdd
feat(common): added extensible implementation of the enr typed record 2023-03-21 17:27:51 +01:00
Hanno Cornelius be446b9892
feat(filter-v2): new filter protocol increment - message handling and clients (#1600)
* feat: further filter v2 progress

* feat: filter client implementation

* chore: rename test modules

* feat: extend tests and minor improvements
2023-03-20 13:19:53 +02:00
Aaryamann Challani 2f3ba3d6d1
chore(rln-relay): reduce exports (#1615) 2023-03-20 16:21:35 +05:30
Aaryamann Challani 08b2180441
fix(rln-relay): msg validation according to new circuit (#1594) 2023-03-13 20:09:33 +05:30
Aaryamann Challani 6584b58593
feat(rln-relay): use precompiled resources w/ rln (#1578)
* feat(rln-relay): use precompiled resources w/ rln

* chore: remove resource cp stage from Dockerfile

* chore(makefile): remove DOCKER_IMAGE_STAGE from makefile

* chore: bump submodule, rev branch
2023-03-13 12:03:21 +05:30
Lorenzo Delgado 075815b072
feat(enr): added enr builder waku capabilities extension 2023-03-10 11:49:41 +01:00
Lorenzo Delgado 1995bbec6c
feat(common): added the enr builder 2023-03-10 01:33:52 +01:00
Alvaro Revuelta 622ec27fcd
refactor(networking): unify peer data models, remove StoredInfo (#1597) 2023-03-09 11:05:50 -07:00
Lorenzo Delgado 7639d8d273
refactor(enr): node capabilities code clean up and reorganization 2023-03-08 15:44:10 +01:00
Lorenzo Delgado 67db35e29d
feat(message): differentiate content and pubsub topic namespacing 2023-03-07 11:10:36 +01:00
Lorenzo Delgado d5f93e385d
chore(enr): move waku_enr to protocols module 2023-03-07 10:52:12 +01:00
Lorenzo Delgado 0bf8d573a7
fix: clean and normalize module imports 2023-03-06 17:19:06 +01:00
Lorenzo Delgado 412ac20606
chore: move discovery protocols under protocol module 2023-03-06 17:18:41 +01:00
Lorenzo Delgado 1c39de7fbc
refactor(message): move namespacing utils to waku message module 2023-03-03 10:07:25 +01:00
Hanno Cornelius ef7ce679df
feat: new filter protocol increment - subscribe request handling (#1584)
* feat: new filter protocol service node

* feat: add test and fix compilation errors

* feat: more tests and minor fixes

* chore: update waku/v2/protocol/waku_filter_v2/protocol.nim

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>

* chore: add some convenience functions

---------

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2023-03-02 17:23:30 +02:00
Lorenzo Delgado a57bea299b
feat(message): added waku message deterministic hashing 2023-03-02 11:41:41 +01:00
Aaryamann Challani 92546bd997
chore(rln-relay): address laundry list of improvements (#1585) 2023-03-01 17:29:13 +05:30
Lorenzo Delgado 4865fd4b7b
feat(message): added meta attribute to waku message 2023-03-01 10:11:29 +01:00
RichΛrd 43b3f1cef4
chore: use base64 url safe encoding for noise (#1569)
Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2023-02-28 14:13:27 -04:00
Aaryamann Challani 8f66e61ac1
feat(rln-relay): group manager integration (#1496)
* feat(rln-relay): init group manager integration

fix(rln-relay): integrate group manager. todo spam and reg handlers

fix(rln-relay): decouple waku-relay and waku-rln-relay

fix(rln-relay): compiles now

fix(chat2): compilation

fix(rln-relay): wip segfault

fix(rln-relay): segfault

fix(chat2|wakunode2): use optional field

fix(rln-relay): wakunode test

fix(rln-relay): uncomment fields in proto decode

fix(rln-relay): used pragma on tests

fix(rln-relay): include cred processing

fix(rln-relay): add reg callback

fix(rln-relay): args to mount

fix(rln-relay): add timeout to waitForExit

fix(rln-relay): use osproc term instead of posix kill

fix(rln-relay): use poParentStream to prevent deadlock

fix(rln-relay): remove poParentStream, remove ganache log output

* fix(rln-relay): abstract tuple into own type
2023-02-28 19:08:30 +05:30
Alvaro Revuelta 700dbbb7fd
refactor(networking): peermanager refactor and cleanups (#1539)
* refactor(networking): use addServicePeer where needed + add metrics
2023-02-27 10:24:31 -07:00
Hanno Cornelius c254115578
chore: log node peer ID with relay trace logging (#1574)
* chore: log node peer ID with relay trace logging

* feat: add peerId accessor
2023-02-27 17:36:24 +02:00
Lorenzo Delgado 25660b4adf
refactor(relay): split the message id provider into a module 2023-02-27 15:38:24 +01:00
Lorenzo Delgado 8258415804
fix(jsonrpc): use padded-base64 for payload encoding 2023-02-23 09:31:06 +01:00
Aaryamann Challani b6a011733d
chore(rln-relay): bump zerokit and update ffi (#1571)
fix(chat2): compilation
2023-02-22 19:47:12 +05:30
Hanno Cornelius 59203c7453
fix: allow higher resolution timestamps (#1570)
* fix: allow higher resolution timestamps

* fix: higher precision requires coarser test
2023-02-21 14:52:30 +02:00
Lorenzo Delgado f7584dfc49
feat(protobuf): added error wrappers for invalid length validation 2023-02-20 15:03:32 +01:00
Alvaro Revuelta edce01cd6a
fix(networking): fix wrong peer connected state (#1560) 2023-02-14 15:38:32 +01:00
Hanno Cornelius 740e4f2bd6
chore: lower per-message log level to trace (#1561) 2023-02-14 15:07:26 +02:00
Lorenzo Delgado 9258914057
fix(jsonrpc): encode waku messages payload in base64 (#1139) 2023-02-14 09:19:06 +01:00
Aaryamann Challani 416bc80092
fix(discv5): add multiaddrs only if len > 0 (#1559) 2023-02-14 13:41:48 +05:30
Alvaro Revuelta 28e5bcf8b5
fix(networking): fix flaky peerstore test (#1554) 2023-02-13 18:10:20 +01:00
Hanno Cornelius dd5243a41c
fix: change message ID provider to match go-waku (#1556) 2023-02-13 18:22:34 +02:00
Lorenzo Delgado 0a15ce48d8
refactor(rest): deep code and tests reorganization 2023-02-13 15:22:24 +01:00
Lorenzo Delgado 9a40ad6400
fix: remove unused imports 2023-02-13 15:02:34 +01:00
Lorenzo Delgado a7f0a7f704
fix(node): remove subscription to waku default network topic (#1545) 2023-02-10 17:55:47 +01:00
Lorenzo Delgado 274101af43
refactor(relay): improve waku_relay apis and add tests 2023-02-10 15:17:50 +01:00
Lorenzo Delgado 2f390ce884
refactor(jsonrpc): deep code and tests reorganization 2023-02-10 10:43:16 +01:00
Alvaro Revuelta 9c3e5f5bc5
refactor(px): refactor peer exchange + tests (#1527) 2023-02-09 16:59:29 +01:00
G cdc09aeeb4
feat(credentials): store and processing generic app credentials (#1466)
* feat(credentials): store and processing generic app credentials

* feat(credentials): separate module; minimal tests

* more work

* feat(credentials): check presence of idCredential in keystore and add only new membership groups

* feat(credential): refactor, new data structure, dynamic add credential, filter

* feat(credential): add filter, get credentials

* feat(credential): encode/decode utility

* feat(credential): sort groups, test credential retrieval/group merging

* fix(credential): remove unnecessary order in sort

* fix(credentials): fix vendor commits

* fix(credential/rln): embed credential module in rln relay

* feat(credentials/rln): use credentials API in rln-relay to store/read credentials

* refactor(credentials): implement hasKeys for JsonNode

* fix(credentials): restore connectToNodes call

* refactor(credentials): remove unnecessary imports

* refactor(credentials): add Res suffix to results

* refactor(credential): moved save json to separate proc; added comments

* feat(credentials): use appInfo

* refactor(keystore): refactor code in a more structured module; address reviewers

* fix(keystore): fix indentation
2023-02-08 16:26:23 +01:00
Lorenzo Delgado 1a9f633311
refactor(message): split waku message module into rpc and codec 2023-02-08 16:16:21 +01:00
Lorenzo Delgado 00f4ce4cbc
refactor(relay): improve wakuy relay api 2023-02-08 16:09:59 +01:00
Alvaro Revuelta 55bac8dedf
feat: add WakuMessage validation in gossipsub (#1537) 2023-02-07 17:42:17 +01:00
Aaryamann Challani 9ddf0fe1e2
feat(discv5): advertise custom multiaddresses (#1512)
* feat(discv5): allow custom multiaddr advertisement in discv5

feat(discv5): allow custom multiaddr advertisement in discv5

feat(discv5): move discv5 setup from wakunode2 to waku_node

fix(waku_node): def param

test(discv5): test for ext multiaddr

fix(discv5): address comments

fix(discv5): address comments

fix(wakunode2): discoveryconfig in var before init

fix(discv5): pass multiaddr to discv5 directly

fix(discv5): make multiaddrs optional

fix(test): discv5 init

fix(discv5): split discv5 mounting from waku_node

chore(discv5): s/WakuAddressMetadata/WakuNodeAddrMeta/g

* fix(waku_node): 1.25 max conns

* fix(discv5): s/WakuNodeAddrMeta/NetConfig/g

* fix(discv5): address reviews

* fix(discv5): smaller try-catches

* fix(discv5): missing arg

* fix: compile error
2023-02-07 18:36:50 +05:30
Lorenzo Delgado 7f2ea1caeb
refactor(utils): moved noise/compat payload encryption code to utils 2023-02-07 10:45:25 +01:00
Aaryamann Challani c9bc774895
fix(rng): create rng only within isMainModule blocks (#1523)
* fix(rng): create rng only within isMainModule blocks

* fix(rng): wakucanary rng

* fix(rng): address comments
2023-02-06 17:23:05 +05:30
Lorenzo Delgado 4ea6039323
chore(discv5): waku discv5 module code cleanup 2023-02-06 11:36:37 +01:00
Lorenzo Delgado 8ba36e2dfa
chore(peer_manager): added peer_manager barrel module 2023-02-06 10:03:30 +01:00
Lorenzo Delgado e56dbd06d8
chore(v1): fix deprecation warnings 2023-02-06 10:00:57 +01:00
Hanno Cornelius 528cc5ad34
chore: add better e2 trace logging for relay (#1526) 2023-02-03 10:06:21 +02:00
Alvaro Revuelta 94459b4886
fix: add safe default values for peer-store-capacity (#1525) 2023-01-31 17:26:22 +01:00
Alvaro Revuelta da7592bcc7
feat(networking): prune peers from peerstore exceeding capacity (#1513)
* feat(networking): prune peers from peerstore

* chore: add comments

* feat(networking): fix comments

* Add tests
2023-01-31 13:24:49 +01:00
Lorenzo Delgado 77c64043f9
fix(store): set store rpc codec pagesize type to uint64 2023-01-31 11:48:44 +01:00
Alvaro Revuelta 93a07babf2
fix(rpc): fix out of bound when empty multiadd (#1507) 2023-01-26 12:05:25 +01:00
Aaryamann Challani d09ec815ce
feat(wakunode): advertise custom multiaddresses (#1509)
* feat(wakunode2): ability to advertise custom multiaddresses

* test(wakunode): test the feature

* fix(wakunode): remove rln diff

* revert(rln-relay): change that creeped into the diff

* fix(wakunode): move extMultiaddrs closer to nat

* fix(waku_node): idiomatic default arg

* fix(config): shortened validation

* fix(wakunode): discoverable via discv5 and dnsdisc
2023-01-26 15:48:30 +05:30
Alvaro Revuelta ea4703e9a2
feat(networking): add service slots to peer manager (#1473) 2023-01-26 10:20:20 +01:00
Lorenzo Delgado 95d31b3ed3
fix(archive): reverse the db query results in the waku archive front-end 2023-01-26 10:19:58 +01:00
Lorenzo Delgado 8c7a931f65
refactor(archive): sqlite query optimization. replace or-ed equal conditions list with in-list 2023-01-25 16:52:13 +01:00
Lorenzo Delgado b84c54bbcb
refactor(relay): simplify relay publish jsonrpc api 2023-01-25 14:35:38 +01:00
Alvaro Revuelta 028efc8547
feat(networking): add backoff period after failed dial (#1462)
* feat(networking): add exponential backoff when dialing relay peers

* feat(networking): fix tests

* revert withTimeout

* feat(networking): refactor tests

* feat(networking): improve logs + ping using switch

* feat(networking): fix backoff bug + fix tests

* feat(networking): fix comments
2023-01-23 21:24:46 +01:00
Aaryamann Challani 9a12872465
fix(rln-relay): use new credentials for compatibility (#1497) 2023-01-19 14:20:08 +05:30
Alvaro Revuelta fd1ec4a74a
feat(networking): add relay connectivity loop (#1482)
* feat(networking): add relay connectivity loop

* Add unit tests

* feat(networking): fix comments

* Fix lnsd comments
2023-01-18 15:17:56 +01:00
Aaryamann Challani 605cf1c38c
feat(rln-relay): abstract group management into its own api (#1465)
* feat(rln-relay): group manager api

* fix(rln-relay): static gm, method async works now

* fix(rln-relay): cb naming, ensure merkle root changed

* chore(rln-relay): static group manager fully tested

* chore(rln-relay): split ffi into its own module, split conversion_utils

* chore(rln-relay): refactor onchain group

* fix(rln-relay): throw error if privatekey is malformed

* chore(rln-relay): convert methods to procs, remove warnings, formatting

* chore(rln-relay): remove comment

* style(rln-relay): fmt test

* feat(rln-relay): complete onchain group manager. pending tests

* fix(rln-relay): onchain implementation works now

* fix(rln-relay): reg index

* fix(rln): imports

* fix(rln-relay): revert method to proc conv

* fix(rln-relay): s/ffi/rln

* fix(rln-relay): remove rln/ from gitignore

* fix(rln-relay): s/ffi/rln
2023-01-16 18:26:18 +05:30
Lorenzo Delgado 3ee87d1848
feat(archive): respond with error on more than 10 content topics query 2023-01-11 12:19:59 +01:00
Alvaro Revuelta 982cd28265
feat(networking): use autonatservice and log if node is reachable (#1472) 2023-01-11 10:57:49 +01:00
Alvaro Revuelta 53bf00de69
refactor: remove unnecessary peermanager dependency (#1474) 2023-01-11 09:10:46 +01:00
Alvaro Revuelta 4eb9375b62
refactor(networking): wait for all futures together in connectToNodes (#1471)
* refactor(networking): wait for all futures together in connectToNodes

* refactor(networking): await fix style
2023-01-09 21:45:50 +01:00
Alvaro Revuelta 333ce0a2e1
fix(rpc): return error if peer cant be connected (#1476) 2023-01-09 14:15:03 +01:00
Alvaro Revuelta 5042b06215
chore(p2p): unify dialpeer functions (#1458) 2022-12-14 16:04:11 +01:00
G 226b44c86d
refactor(rln): use zerokit extended_key_gen in place of key_gen (#1459)
* chore(rln-relay): refactor mounting procedure

* fix(rln-relay): tests

* fix(rln-relay|chat2): update mounting proc in chat2, fix args to mountRlnRelay

* refactor(rln): use zerokit extended_key_gen in place of key_gen

refactor(rln): rename membershipKeyPair to identityCredential

update zerokit submodule

refactor(rln): use extended_key_gen; rebrand keypairs to idCredentials

refactor(rln): rename mountRlnRelay memKeyPair argument

* fix(rln): fix wrong field

* fix(rln): add () to toHex

Co-authored-by: Aaryamann Challani <43716372+rymnc@users.noreply.github.com>

Co-authored-by: rymnc <43716372+rymnc@users.noreply.github.com>
2022-12-14 12:28:09 +01:00
Aaryamann Challani 33e9d8b9b2
chore(rln-relay): refactor mounting procedure (#1457)
* chore(rln-relay): refactor mounting procedure

* fix(rln-relay): tests

* fix(rln-relay|chat2): update mounting proc in chat2, fix args to mountRlnRelay

* fix(rln-relay): remove stale todo
2022-12-13 14:56:24 +05:30
Lorenzo Delgado e4c10ed661
feat(wakunode2): support log format format selection 2022-12-07 12:30:32 +01:00
G 002316cceb
feat(rln): use zerokit as static lib (#1385) 2022-12-06 00:33:03 +01:00
Aaryamann Challani 2e0db18cde
chore(rln-relay): re-enable root validation (#1445)
* fix(rln-relay): enable root validation

* fix(rln-relay): unused import
2022-12-05 23:52:54 +05:30
Lorenzo Delgado 3a4c4c660b
chore(build): simplify project makefile and update dockerfile 2022-11-30 21:02:39 +01:00
Tanguy e487c7d79e
Decouple waku v2 from waku v1 (#1421)
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2022-11-29 20:15:07 +01:00
Alvaro Revuelta 8cce06b8a2
feat(peerstore): store peer direction (#1424)
* feat(peerstore): store peer direction

* feat(peerstore): add getPeersByDirection function + tests

* feat(peerstore): set out own MaxConnectionsPerPeer to 1

* feat(peermanager): add metric for inbound/outbound peers
2022-11-29 17:35:25 +01:00
G bcab36902d
fix(noise): T_ss is ss not se (#1432) 2022-11-29 06:22:18 +01:00
Hanno Cornelius 8d69846aa4
feat: enable AutoNAT and libp2p circuit relay (#1425)
* feat: enable libp2p circuit relay

* feat: enable libp2p autonat
2022-11-28 16:56:01 +02:00
Lorenzo Delgado 3474bef8e3
fix(archive): restore query and insert metrics duration precision 2022-11-25 23:20:24 +01:00
Lorenzo Delgado a4970d2625
fix(archive): fix message validator error metric labels 2022-11-25 02:06:42 +01:00
Lorenzo Delgado 59a204c32c
fix(archive): fix message validator error metric labels 2022-11-25 01:49:43 +01:00
Lorenzo Delgado 0050deed05
chore(store): remove obsolete metrics and error labels 2022-11-24 19:42:19 +01:00
Lorenzo Delgado 41b2f87f68
fix(archive): patch the sqlite driver to solve results inconsistencies 2022-11-24 18:59:25 +01:00
Alvaro Revuelta 43fd11b4dc
refactor: reuse nim-libp2p peerstore + move peermanager logic (#1383)
* refactor: reuse nim-libp2p peerstore + move peermanager logic

* refactor: fix comments

* refactor: modify reconnectPeers and unittest

* feat(apps): new flag for peerStoreCapacity

* fix(examples): fix example2 target

* refactor: fix comments
2022-11-24 14:11:23 +01:00
Lorenzo Delgado 6811af328c
feat(archive): add waku archive implementation 2022-11-23 10:08:00 +01:00
Lorenzo Delgado 0188ec1a9e
feat(archive): add waku archive implementation 2022-11-22 19:40:24 +01:00
Lorenzo Delgado 299ee3eb5d
refactor(rln): decouple rln types from waku message type 2022-11-22 18:29:43 +01:00
Alvaro Revuelta 1e8e60caed
fix(tests): add missing import for tests (#1375) 2022-11-22 08:13:51 +01:00
Aaryamann Challani 853995e17f
chore(rln-relay): barrel imports (#1386)
Co-authored-by: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2022-11-21 15:15:03 +01:00
Lorenzo Delgado 59c18ab5c4
feat(store): add support for history query handler 2022-11-21 11:16:57 +01:00
Lorenzo Delgado e85b5cbae1
chore(store): remove waku swap integration from store protocol 2022-11-21 09:36:41 +01:00
Lorenzo Delgado 9396f64489
fix(lightpush): waku lightpush rpc codec support optional fields 2022-11-18 20:01:01 +01:00
Lorenzo Delgado af4fb5f5a7
fix(filter): waku filter rpc codec support optional fields 2022-11-18 18:37:08 +01:00
Lorenzo Delgado 1d72ee31b0
refactor(relay): collocate relay protocol initialization logic 2022-11-18 14:50:56 +01:00
alrevuelta bd5046f466
chore: add version to logs at startup 2022-11-18 11:59:35 +01:00
Lorenzo Delgado f89e6869cb
fix(store): waku store rpc codec support optional fields 2022-11-17 20:40:08 +01:00
Lorenzo Delgado 39bf289f43
refactor(protocols): decouple protocol rpc types from api types 2022-11-16 20:02:38 +01:00
Lorenzo Delgado e2a2ea6477
chore: move protobuf module from utils to common 2022-11-16 20:02:14 +01:00
Aaryamann Challani 2336522d7f
fix(rln-relay): feature guard (#1373)
* fix(rln-relay): feature guard

* Revert "fix(rln-relay): feature guard"

This reverts commit 9344f41a34.

* fix(rln-relay): surgically fix imports

* fix(rln-relay): indents

* fix(rln-relay): more usages of rln types
2022-11-14 19:29:34 +05:30
Aaryamann Challani 07833ce313
feat(rln-relay): process blocks atomically (#1349)
* test(rln-relay): atomic block processing

* fix(rln-relay): use correct starting index

* fix(rln-relay): args

* fix(rln-relay): append length

* fix(rln-relay): tests, remove insertMember

* fix(rln-relay): camelCase, cleanup

* fix(rln-relay): actually process per block

* fix(rln-relay): clean up

* chore(gitignore): Update .gitignore

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Sanaz Taheri <35961250+staheri14@users.noreply.github.com>

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Sanaz Taheri <35961250+staheri14@users.noreply.github.com>

* fix(rln-relay): args

* fix(rln-relay): add prefix def

* fix(rln-relay): make test cleaner

* chore(rln-relay): apply suggestions

Co-authored-by: G. <28568419+s1fr0@users.noreply.github.com>
Co-authored-by: Lorenzo Delgado <lorenzo@status.im>

* chore(rln-relay): add member order check

* test(rln-relay): batch insert in tests

* fix(rln-relay): test batching

* fix(rln-relay): toSeq the HSlice

* fix(rln-relay): naming

* fix(rln-relay): add insertMember back

* fix(rln-relay): serialize util, address review

* fix(rln-relay): add atomicity desc

* fix(rln-relay): inHex

* fix(rln-relay): explicit proc def

* fix(rln-relay): indexGap condition

* fix(rln-relay): func sig

* fix(rln-relay): onchain test

* fix(rln-relay): use asyncSpawn vs asyncCheck

* fix(rln-relay): do not explicitly insert into the index

* fix(rln-relay): condition, semantics

* fix(rln-relay): index must be 1

* chore(rln-relay): line br

* fix(rln-relay): missing return ok(true)

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
Co-authored-by: Sanaz Taheri <35961250+staheri14@users.noreply.github.com>
Co-authored-by: G. <28568419+s1fr0@users.noreply.github.com>
2022-11-10 22:28:31 +05:30
Alvaro Revuelta 7917e05d9d
chore(networkmonitor): tool to discover and provide metrics on peers (#1290)
* chore(networkmonitor): initial prototype

* chore(networkmonitor): add cli, metrics and PoC

* feat(utils): add supportsCapability function + tests

* feat(utils): add supportedCapabilites function

* chore(networkmonitor): add metrics with enr/ip/capabilities

* chore(networkmonitor): refactor + tests

* chore(networkmonitor): add discovered timestamp

* chore(networkmonitor): add metrics on connected nodes

* chore(networkmonitor): new flags + utils file + readme

* chore(networkmonitor): add user-agent metrics

* chore(networkmonitor): connect only to randomly discovered peers

* chore(networkmonitor): get location of peer using ip

* chore(networkmonitor): expose peer metrics with simple rest server

* chore(networkmonitor): update README

* chore(networkmonitor): fix wakunode2 to waku_node

* chore(networkmonitor): fix import order

* chore(networkmonitor): fix comments + refactor + pushraises

* chore(networkmonitor): refactor + handle exceptions

* chore(networkmonitor): fix makefile after rebase

* chore(networkmonitor): address review comments 1

* chore(networkmonitor): add nim.cfg
2022-11-10 10:29:34 +01:00
G 63137f3e2a
refactor(rln): remove kilic lib (#1365)
* refactor(rln): remove kilic lib

* chore(rln): address reviewerS
2022-11-09 19:45:04 +01:00
Lorenzo Delgado b07cdb1841
refactor(store): decouple waku store public api types from rpc types 2022-11-09 18:50:18 +01:00
Alvaro Revuelta 054dc61763
refactor: continue pubsub/content types started in #1352 (#1362)
* refactor: continue gossip/content topic refactor started in #1352

* refactor: enforce using pubsubTopic instead of topic
2022-11-09 15:00:11 +01:00
Lorenzo Delgado 4acc611e47
refactor: unify pubsub topic type and default value 2022-11-09 09:55:47 +01:00
Lorenzo Delgado c4f9813ab3
refactor(queue_store): rename queue_store module and simplify api 2022-11-08 16:10:23 +01:00
Lorenzo Delgado b4bda3c10b
fix(rln-relay): remove dependency on applications' configuration 2022-11-08 12:53:47 +01:00
Lorenzo Delgado bcc6c32287
chore: rename protocols rpc_codec procs from init to decode 2022-11-07 16:24:16 +01:00
Lorenzo Delgado 4509f4f361
refactor(message_store): move message_store to node module 2022-11-04 19:48:22 +01:00
Lorenzo Delgado 7ef4eaf46d
chore: update push raises pragma according to nim 1.6 2022-11-04 10:52:27 +01:00
Lorenzo Delgado cb1f4d46f4
chore: remove unused imports 2022-11-04 10:52:08 +01:00
Lorenzo Delgado 0569beadbe
refactor(peer_manager): move peer_store under peer_manager module 2022-11-04 09:40:13 +01:00
Aaryamann Challani 53e8979aa9
chore(rln-relay): remove all raises and replace with Result types (#1321)
* chore(rln-relay): remove all raises and replace with Result types

* chore(rln-relay): s/var/let

* chore(rln-relay): s/isOk/isOk()

* fix(rln-relay): proc def comments about the result type

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2022-11-04 08:30:42 +05:30
Lorenzo Delgado d1df046c87
feat(wakunode2): support configuration via environment variables 2022-11-03 17:58:48 +01:00
Lorenzo Delgado 27a7040f8e
chore(migrations): move migration scripts to project root 2022-11-03 17:58:05 +01:00
Lorenzo Delgado 5c4f1ce43f
chore: make log topics consistent with nim-chronicles style 2022-11-03 16:36:24 +01:00
Lorenzo Delgado 1d3943febb
refactor(sqlite): move migrations logic to sqlite common module 2022-11-02 16:36:33 +01:00
Lorenzo Delgado 83ae6987b8
fix: replace TaintedString type deprecated in nim 1.5 2022-11-02 14:55:48 +01:00
Lorenzo Delgado 8fee1b9bed
feat(filter): split waku filter protocol into server and client 2022-11-02 11:59:58 +01:00
Lorenzo Delgado 1c46b61402
refactor(peer-exchange): move peer management to waku_node module 2022-11-02 09:45:21 +01:00
Aaryamann Challani cd73029a0c
feat(rln-relay): track last seen event (#1296)
* feat(rln-relay): track last seen event

* fix(rln-relay): clean up subscribeToMemberRegistrations proc

* fix(rln-relay): tests

* fix(rln-relay): unnecessary try-except

* fix(rln-relay): proc descriptions, logging

Co-authored-by: G <28568419+s1fr0@users.noreply.github.com>
2022-11-01 08:15:34 +05:30
Aaryamann Challani 24d288ccb4
fix(rln-relay): RangeDefect test (#1318)
* fix(rln-relay): RangeDefect test

* fix(rln-relay): any overflow

* fix(rln-relay): dont use int64 when its absolute value is being used anyway

* chore(rln): rename proc to be more meaningful about operation

* fix(rln): remove unused imports
2022-11-01 05:55:39 +05:30
Lorenzo Delgado 0de5a122ef
chore(common): move sqlite module to common module 2022-10-28 20:13:41 +02:00
Lorenzo Delgado ee86d190d6
refactor(node): use store client in waku_node 2022-10-28 20:11:28 +02:00
Lorenzo Delgado 2c1ad9f760
refactor(node): use lightpush client in waku_node 2022-10-28 16:30:01 +02:00
Alvaro Revuelta d479aacec5
feat(wakunode2): add nwaku agent-string to switch (#1302) 2022-10-28 15:12:06 +02:00
Hanno Cornelius 16c85db43c
chore: update submodules and bump Nim to 1.6.6 (#1307)
* chore: update submodules

* fix: libp2p now provides callback to update peer addrs

* fix: breaking change in EthereumNode constructor

* fix: contentType type has changed (again)

* fix: explicit future type

* fix: nim 1.6.6 error handling requirements

* fix: missed a spot - peer info addrs in sim2

* fix: help compiler a bit here
2022-10-28 12:51:46 +03:00
G ea8d72188e
feat: add keyfile support for RLN credentials secure storage (#1285)
* feat(utils): add keyfile + test

* feat(keyfile): make id and version field optional

* feat(rln): enable input password for keyfile encryption

* refactor(keyfile): store multiple keyfiles in single file

* refactor(rln): write and read rln credentials proc using keyfiles + test update

* feat(keyfile): allow skip successful decryption to iterate among multiple keyfiles + test

* fix(rln): add rln-relay-cred-password in wakunode2 config

* chore(keyfile): remove unnencessary echo

* refactor(rln/kesyore): address reviewers' comments

* refactor(rln/keyfile): use defer instead of try/finally

* feat(rln): address reviewers comments; return all successful decryptions

* chore(keyfile): add comments; changed name to loadKeyfiles

* feat(keyfile): chmod 600 for keyfiles

* fix(rln): fix errors raised by readRlnCredentials

* feat(keyfile): add nim-eth tests; address reviewers comments

* fix(keyfile/rln): better error handling

* refactor(keyfile): refactoring nim-eth tests

* fix(keyfile): fix wrong index

* fix(keyfile): fix wrong password generation in test

* feat(keyfile): add wrong mac test

Co-authored-by: Aaryamann Challani <43716372+rymnc@users.noreply.github.com>
2022-10-28 11:13:05 +02:00
Lorenzo Delgado 62328f0abd
feat(wakunode2): simplify wakunode2 config and decouple peer persistence 2022-10-28 00:05:02 +02:00
Lorenzo Delgado 0725da0b66
chore(node): waku node code reorganization 2022-10-27 17:29:09 +02:00
Lorenzo Delgado 7e7bba4a98
feat(lightpush): add waku lightpush protocol client 2022-10-25 14:55:31 +02:00
Aaryamann Challani bdb120d842
feat(chat2): metrics server + metrics logging (#1279)
* feat(rln-relay): periodically log metrics

* fix(chat2): enable metrics by default in chat2 config

* test(chat2): metrics, gc compilation error

* chore(metrics): make metrics a util, and import into wakunode2 apps dir

* fix(metrics): raise error

* fix(metrics): gc error

* fix(wakunode2): remove setup_metrics

* chore(metrics): waku utils metrics

* fix(metrics): create waku_metrics in node dir

* fix(metrics): log scope

* fix(chat2): disable metrics server by default

* fix(utils): collectorAsF64 proc def

* fix(metrics): store metrics path
2022-10-21 14:03:36 +05:30
Lorenzo Delgado e5c3aa560e
feat(store): add waku store client module 2022-10-20 18:09:40 +02:00
Hanno Cornelius a12a359b79
fix: store cursors should be exclusive and match a DB item (#1263)
* fix(store): use exclusive cursor

* fix(store): read correct column length

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2022-10-20 13:24:40 +03:00
Lorenzo Delgado 757804c9ee
chore(wakunode2): move wakunode2 to apps directory 2022-10-18 12:35:26 -05:00
Lorenzo Delgado 5046a4b3da
refactor(node): split wakunode into waku_node and wakunode2 2022-10-18 09:05:53 -05:00
Lorenzo Delgado b139692b05
refactor(wakunode2): move config random private key generation to a procedure (#1273) 2022-10-18 07:06:54 -05:00
Aaryamann Challani 9a6a7334ef
feat(rln-relay): periodically log metrics (#1262)
* feat(rln-relay): periodically log metrics

* chore(rln-relay): log rln metrics immediately

* fix(metrics): calculation etc

* chore(metrics): parseAndAccumulate into collector utils

* fix(metrics): use standard logging interval
2022-10-18 17:07:44 +05:30
Lorenzo Delgado 5fbc537cbf
chore: move apps and tools to their own directory 2022-10-12 14:41:25 -05:00
G a2a5a6ef01
refactor(rln): display byte arrays in hex as uints and fix Kilic's proofVerify proc (#1259)
* refactor(rln): convert 32 bytes arrays to hex as uints

* fix(rln): fix Kilic's proofVerify proc to take stub validRoots

* refactor(rln): use input's length when convert to/from hex

* chore(rln): print more commitments inHex in debug logs
2022-10-12 10:38:48 +02:00
Aaryamann Challani 04154ab90f
chore(rln-relay): make account address optional (#1258)
* chore(rln-relay): make account address optional

* fix(rln-relay): onchain test

* chore(chat2): update docs, and handle error case in chat2

* fix(rln-relay): handle registration better
2022-10-12 07:48:11 +05:30
G f282621a1c
fix(rln): set maxEpochGap as maximum epoch gap (#1257) 2022-10-10 21:50:16 +02:00
G 586d549113
fix(rln): fix encoding to LE for idcommitments (#1256) 2022-10-10 19:35:43 +02:00
Aaryamann Challani 3a80fae634
test(rln-relay): setup heartbeat ws subscription and log when connection drops (#1253)
* test(rln-relay): setup heartbeat ws subscription and log when connection drops

* fix(rln-relay): quantity

* fix(rln-relay): allow fromBlock to be passed to sub proc

* chore(rln-relay): cleaner logs

* fix(rln-relay): typo
2022-10-10 17:55:12 +05:30
Aaryamann Challani f8c4d9aebc
chore(rln-relay): additional logging (#1243)
* chore(rln-relay): additional logging

* fix: whitespace
2022-10-08 12:14:52 +05:30
G 3e1e7cc74f
feat(RLN): Add verify against multiple roots (#1250)
* update zerokit submodule

* fix(rln): allow Buffer creation on empty arrays

* feat(rln): use verify_with_root

* fix(rln): update test to work with verify_with_root
2022-10-07 21:24:54 +02:00
Aaryamann Challani 8a8c745e30
chore(rln-relay): disable merkle root validation (#1246)
* chore(rln-relay): disable merkle root validation

* brings back validation of the root but without invalidating the message

* adds the tracking issue

Co-authored-by: sanaz <staheri14@ku.edu.tr>
2022-10-06 13:17:01 -07:00
Lorenzo Delgado 1cb3f4b7a4
fix(store): fix store validity timestamp variance constant (#1244) 2022-10-06 16:07:12 +01:00
Hanno Cornelius 8a184e208c
chore: publish nwaku version to metric (#1234) 2022-10-06 13:06:54 +01:00
Lorenzo Delgado 2c975597e1
fix(store): fix waku store resume tests 2022-10-05 17:58:24 +02:00
Lorenzo Delgado 14abdef796
chore(store): improve database migrations logging 2022-10-05 11:45:44 +02:00
G f9c05aae52
chore(rln): Update zerokit submodule and RLN APIs (#1209)
* chore(rln): update zerokit submodule and APIs

* feat(rln): add new_circuit_from_data zerokit API

* chore(rln): update zerokit module
2022-10-05 00:06:33 +02:00
Sanaz Taheri Boshrooyeh fe3a3136a3
feat(rln-relay): adds rln identifier to the RateLimitProof (#1219)
* adds rln identifier to the RateLimitProof

* enables rln-relay compilation for wakunode2

* unifies the two RateLimitProof types

* a minor update

Co-authored-by: G <28568419+s1fr0@users.noreply.github.com>
2022-10-04 13:20:44 -07:00
Hanno Cornelius ad628b5b52
fix: store config without db-path leads SIGSEGV (#1218) 2022-10-04 14:34:05 +01:00
Aaryamann Challani 787464487b
chore(rln-relay): namespaced flags (#1204)
* chore(rln-relay): namespaced flags

* chore(rln-relay): shorten flag size

* fix(rln-relay): consistent config naming

* chore(rln-relay): consistency with address configs

* fix(rln-relay): config
2022-10-03 13:25:56 -07:00
Lorenzo Delgado fa6a44eb3c
fix(store): Move page info logic to waku store protocol 2022-10-03 17:36:17 +02:00
Lorenzo Delgado adf1dab34f
fix(store): update sqlite db table primary key and decouple queries module from pagination types 2022-10-03 15:23:39 +02:00
Richard Ramos 8af3438814
fix: add lastEpoch when using zerokit, and missing return (#1210) 2022-10-03 11:48:01 +02:00
Lorenzo Delgado c61e88b352
refactor(store): simplify store queue query implementation 2022-10-03 08:40:01 +02:00
Aaryamann Challani d614e4504b
feat(rln-relay): metrics (#1185)
* feat(rln-relay): metrics

* fix(rln-relay): invalid token

* fix(rln-relay): return outside time

* style(rln-relay): multiline proc def

* fix(rln-relay): oserror

* fix(rln-relay): Exception should not be raised

* Revert "fix(rln-relay): oserror"

This reverts commit edcf39c0088ca526ca50256d856eee20d87a3c37.

* Revert "fix(rln-relay): Exception should not be raised"

This reverts commit c8d09b055314a83bb4a94900a47e899773cb47df.

* fix(rln-relay): revert exception removal

* feat(rln-relay): granular tracking

* fix(rln-relay): observe only on histogram, convert mount to hist

* chore(rln-relay): add additional metrics

* fix(rln-relay): missing import

* fix(rln-relay): template should expand only when times is defined

* fix(rln-relay): follow prometheus naming style

* fix(rln-relay): explicitly assert that the root window size should be less than 10

* fix(rln-relay): outstanding issues with infinite labels, minor refactor

* fix(rln-relay): typo

* fix(rln-relay): screaming case to pascal case

* fix(rln-relay): typos

* fix(rln-relay): imports

* fix(rln-relay): remove dependence on times, make durations as gauges

* fix(rln-relay): import order
2022-09-30 18:13:42 +05:30
Lorenzo Delgado 612023fab0
chore(store): collocate pagination and message store types 2022-09-28 13:36:05 +02:00
Lorenzo Delgado 5ee4a00765
refactor(store): decouple message store queue from pagination index type (#1187) 2022-09-27 21:10:11 +02:00
Lorenzo Delgado d7d51dc4c1
fix(store): rename store timestamp sqlite column 2022-09-27 11:47:25 +02:00
Aaryamann Challani 37cd8b593b
feat(rln-relay): multiple acceptable roots (#1177)
* feat(rln-relay): multiple acceptable roots

* fix(rln-relay): make sure onchain handler uses correct proc

* fix(rln-relay): typo

* style(rln-relay): convert const to pascalcase

* chore(rln-relay): address pr review

* fix(rln-relay): add require to test

* fix(rln-relay): add proc desc

* fix(rln-relay): ensure that group id commitments were added correctly

* fix(rln-relay): potential mem leak

* style(rln-relay): comments

* fix(rln-relay): magic number ambiguity

* chore(rln-relay): comment

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
2022-09-27 10:10:04 +05:30
Lorenzo Delgado 1b4a9e5dfd
fix(store): move insert criteria logic to waku store protocol module 2022-09-26 11:50:15 +02:00
Sanaz Taheri Boshrooyeh 489ca1b01e
chore(rln-relay): specifies the address type for the eth-client-address config parameter (#1176) 2022-09-23 21:57:28 -07:00
Hanno Cornelius 6b86c4a22a
fix: fail to start switch with ext ip (#1181) 2022-09-23 15:27:15 +01:00
Hanno Cornelius ca1718c504
fix: signing key for wakuv2 DNS lists has changed (#1174) 2022-09-22 14:51:17 +02:00
Lorenzo Delgado aebda46a53
chore(store): move constants to retention policies modules 2022-09-22 11:17:38 +02:00
Lorenzo Delgado 5e3a75c56e
refactor(wakunode): remove deprecated non-async methods 2022-09-21 18:27:40 +02:00
Lorenzo Delgado 11832411a0
feat(store): execute retention policies periodically (#1155) 2022-09-21 11:32:59 +02:00
Hanno Cornelius efe82eade2
fix: no need to start protocols twice (#1162)
* fix: no need to start protocols twice

* fix: relay should still be explicitly started

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2022-09-20 21:02:20 +01:00
Aaryamann Challani e8d77306c8
feat(rln-relay): validate the merkle root in the RateLimitProof (#1158)
* feat(rln-relay): validate the merkle root in the RateLimitProof

* style|chore: replace snake case usage, convert proofVerify to Result[bool,string]

* chore(rln-relay): remove unhandled usage of result returned

* fix(rln-relay): typos

* chore(rln-relay): validate root before proof verification

* fix(rln-relay): cmp

* fix(rln-relay): generic rln relay result

* style(rln-relay): readability

* fix(rln-relay): use patterns for testing, requires instead of checks

* fix(rln-relay): move root validation to message validation

* fix(rln-relay): typo

* fix(rln-relay): typo

* chore(rln-relay): address pr review
2022-09-20 18:38:05 +05:30
kaiserd 371016b04f
feat: waku peer exchange (RFC34) (#1152) 2022-09-20 13:03:34 +02:00
Lorenzo Delgado 4a9f6f3c5e
refactor(store): protocol code refactoring and api reorganization 2022-09-20 11:39:52 +02:00
Hanno Cornelius 9be3d296be
fix: nodes in AddressBook are not necessarily connected (#1161) 2022-09-20 10:00:40 +01:00
Lorenzo Delgado f73477be6e
refactor(store): invert retention policies dependency on message store interface 2022-09-16 12:55:22 +02:00
Lorenzo Delgado 706c355b18
refactor(store): extend message store interface with store manipulation methods 2022-09-15 21:12:13 +02:00
Lorenzo Delgado 70fdd85066
refactor(store): don't cache the message count in sqlite message store 2022-09-15 18:13:30 +02:00
Lorenzo Delgado 87479a603b feat(store): run sqlite database vacuum at node start 2022-09-14 21:40:17 +02:00
Lorenzo Delgado 89069aa34f
fix(store): follow good practices in store insert and query metrics (#1150) 2022-09-14 13:40:11 +01:00
Lorenzo Delgado f7b4e8f57d refactor(wakunode): remove waku rln relay export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado ddbc57bc88 refactor(wakunode): remove waku message export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado c2c2d30cdf refactor(wakunode): remove libp2p builders export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado 2912a45433 refactor(wakunode): remove waku relay export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado fdd57c7bce refactor(wakunode): remove waku swap export from wakunode 2022-09-14 12:25:05 +02:00
Lorenzo Delgado a4860d92a0 feat(store): add query and insertion time metrics 2022-09-13 19:36:11 +02:00
Lorenzo Delgado 728e298dd7 fix(store): support stored messages metric for persistent store 2022-09-13 19:36:11 +02:00
Lorenzo Delgado 9561f12172 feat(store): add support for getting stored messages count 2022-09-13 19:36:11 +02:00
Lorenzo Delgado 83cffd50ed refactor(store): reorganise waku store protocol is sqqlite-only logic 2022-09-13 19:36:11 +02:00
Lorenzo Delgado d49cd1348d
refactor(store): remove deprecated waku message store type 2022-09-13 13:36:04 +02:00
Aaryamann Challani fdcc9824a2
feat(store): Allow messages to be marked as "don't store" (#1119)
* feat(store): init allow messages to be marked as ephemeral

* feat(store): replace ephemeral with ttl model

* Revert "feat(store): replace ephemeral with ttl model"

This reverts commit 4398c61c919dda8f014b4fcc0679c573c74854e4.

* fix(store): flags and test

* chore(store): remove storeTTL

* chore(store): remove unused utils

* fix: conflict

* fix(store): make test more brief
2022-09-13 16:07:06 +05:30
Aaryamann Challani 4745c7872c
fix(ci): macos builds (#1145)
* chore: revert json rpc

* chore(ci): cleanup

* chore(ci): remove windows from build matrix

* Revert "chore(ci): remove windows from build matrix"

This reverts commit 1f1d33b5f4.
2022-09-13 15:13:56 +05:30
Lorenzo Delgado 051f5db9af
refactor(store): rename to sqlite store and introduce retention policies interface 2022-09-12 16:49:01 +02:00
Lorenzo Delgado e7ebd190a3
refactor(utils): split message digest calculation form index calculation 2022-09-12 15:51:06 +02:00
Lorenzo Delgado ff619de037
chore(utils): remove unused types and methods 2022-09-12 15:29:52 +02:00
G 01cd201f58
PoC implementation of Waku Pairing and Secure Transfer (#1117)
* feat(noise): add PoC implementation for WakuPairing and Secure Transfer
2022-09-12 02:23:14 +02:00
Lorenzo Delgado 30f0e19d79
fix(store): add a table index to improve message insertion time 2022-09-09 15:04:48 +02:00
Hanno Cornelius f4e9d39524
chore: always include node ENR in response to info requests (#1128) 2022-09-08 14:25:02 +01:00
Lorenzo Delgado d857c4a421
chore(wakunode): collocate wakuswitch file to the depending modules 2022-09-08 11:02:50 +02:00
Hanno Cornelius 7f23bdf29f
chore: update nwaku submodules (#1123)
* chore: update submodules

* fix: SIGTERM ambiguity

* fix: ambiguous RNGs

* fix: ContentType is no longer a string

* fix: more fixes related to ambiguous RNGs

* fix: start all protocols

* chore: also update nim-eth

* chore: important new fixes in nim-libp2p

* fix: more changes related to RNG. Some reversion to reflect nim-eth update

* fix: breaking changes in nim-eth submodule

* fix: start protocols in tests

* fix: chat2bridge protocols async mounting

* fix: v1 test compilation

* fix: rln test compilation

* fix: remove confusing keys qualifier for the same HmacDrbgContext
2022-09-07 16:31:27 +01:00
Lorenzo Delgado 3f7aa8c6e4
fix(waku-store): added an index to improve messages query time 2022-09-05 15:18:47 +02:00
Lorenzo Delgado 21e9f25276
refactor(wakunode2): deprecate non-async protocol api methods 2022-09-02 10:17:14 +02:00
Lorenzo Delgado 4f3a3da084
chore(waku-store): improve logging and query traceability 2022-09-02 10:14:58 +02:00
Lorenzo Delgado 982fb08c77
refactor(wakunode2): add convenience methods for adding protocol peers 2022-09-01 17:02:58 +02:00
Aaryamann Challani ea62e2b7ce
chore(noise): minor refactor of default checks of keypairs (#1104)
* chore(noise): readable default checks for keypairs

* chore(noise): fix naming of arg in exists proc

* chore(noise): fix naming to make it clearer

* chore(noise): inline and remove result usage

* chore(noise): move the isDefault function up, replace defaultyVar with value

* chore(noise): inline style

* chore(noise): static the default check, remove inline pragma
2022-09-01 14:31:48 +05:30
Bulat-Ziganshin 80aeebc61c
Fixed handling of bindParam(uint32) in sqlite.nim (#1014) 2022-08-31 14:50:05 +02:00
Aaryamann Challani ad7f9ed5bf
chore(lightpush): fix minor typo (#1100) 2022-08-31 17:00:10 +05:30
Lorenzo Delgado 6d0168bff2
feat(rest): add waku filter openapi rest api specification file 2022-08-31 13:24:35 +02:00
Aaryamann Challani ed53bcdec6
chore(logging): Periodic logging for additional metrics (#1094)
* chore: enable metrics logging by default

* feat(logging): add additional logging metrics, change period to 30 seconds

* chore(logging): s/var/let

* chore(logging): import filter directly

* chore(logging): add store/lightpush peer count, fix style
2022-08-31 15:23:28 +05:30
Sanaz Taheri Boshrooyeh 0deedf0d7a
fix|feat(rln-relay|rln-chat2): Handling on-chain transactions errors and informing chat2 users (#1093)
If any error occurs during the on-chain registration, it will be handled; the chat2 users will get informed about the cause and the installation of rln-relay gets discarded and the chat program continues with no crashing.
2022-08-30 10:59:02 -07:00
Lorenzo Delgado ecf4ba1167
feat(message-cache): make the topic cache generic 2022-08-30 15:57:45 +02:00
Lorenzo Delgado 791ce6d222 fix(rest): encode waku message payload in base64 2022-08-30 14:53:04 +02:00
Lorenzo Delgado 842cfb3476 refactor(waku-filter): remove wakunode2 waku_filter exports 2022-08-29 10:14:35 +02:00
Lorenzo Delgado c964aea885 refactor(waku-filter): waku filter protocol code reorganisation 2022-08-26 22:17:56 +02:00
Hanno Cornelius caa47896c1
fix: peer manager dial logic when dialing an existing peer (#1090) 2022-08-26 11:20:55 +02:00
Sanaz Taheri Boshrooyeh bc29619679
feat(Rln relay): Improves rln-chat2 interface, displays links to the registration tx, warns about spam messages (#1082)
* warns  about publishing spam messages, display tx hash on Goerli

* passes registration handler
2022-08-24 15:47:06 -07:00
Hanno Cornelius 11158c3ea2
fix(setup): do not attempt connecting to empy dynamic bootstrap node list (#1088) 2022-08-24 17:44:43 +02:00
Sanaz Taheri Boshrooyeh 8a6b1cfba6
feat(Rln-Relay): Consolidates the input paths of rln credential, chat2 reads rln credential from file only (#1080)
* Consolidates the input paths of rln credential, accepts credential file only

* fixes a typo
2022-08-18 10:35:02 -07:00
Lorenzo Delgado 73df74f8c6 refactor(waku-store): remove wakunode2 waku_store exports 2022-08-18 17:32:47 +02:00
Sanaz Taheri Boshrooyeh f373f8bc56
chore(RLN-Relay): onchain rln-relay chat2 testnet editorial feadbacks
Signed-off-by: sanaz <staheri14@ku.edu.tr>

adds a newline


fixe a minor spacing issue
2022-08-17 16:55:49 -07:00
Lorenzo Delgado 7a30e485ec refactor(waku-store): deprecate waku store callback methods 2022-08-17 17:22:56 +02:00
G 2c4730c850
fix(rln): add zerokit rln to chat2 + fix static group members (#1077) 2022-08-17 12:11:46 +02:00
Lorenzo Delgado 2cb7123df8 fix(waku-store): integrate the predicate into the WHERE clause instead of calling it on each result line (#1026) 2022-08-12 12:48:45 +02:00
Lorenzo Delgado 4f93510fc9 refactor(waku-store): inherit from message_store in store_queue 2022-08-11 10:15:20 +02:00
Lorenzo Delgado 613812023e refactor(waku-lightpush): remove wakunode2 waku_lightpush exports 2022-08-10 19:44:40 +02:00
Lorenzo Delgado d201a37330 refactor(waku-lightpush): waku lightpush protocol code reorganization 2022-08-10 12:30:34 +02:00
G 2aeaba48e4
feat(noise): add padding to transport messages (#1056)
* feat(noise): add padding to transport messages (handshakes+write+read)

* fix(noise): address reviewers' comments

* fix(noise): update padding block size const name
2022-08-07 16:20:34 +02:00
G 8a14b5b243
Zerokit RLN integration in nwaku: alternative approach with inline compilation flag (#1060)
* refactor(rln): add inline compilation flags for RLN zerokit

* update submodule

update submodule

* fix(rln): fix wrong merge conflict resolution

* fix(rln): align hardcoded values in light of vacp2p/zerokit/issues/28
2022-08-05 22:58:19 +02:00
Keshav Gupta 115982f5c3
Persisting rln credentials (#1037)
* Persisting Credentials implemented by writing json of keypair and rlnindex to files

* Removing testing files and ignores

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Daniel Kaiser <git@kais3r.de>

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Daniel Kaiser <git@kais3r.de>

* Comments

* Comments

* Security warning in comments

* Redundant echos. Omitting unused variables.

* Update waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* Limit line lengths using line breaks and indents

* Membership file paths declared as const

* Const fix

* Rln Credentials Merged.
Reading credentials from file abstracted away.

* Spaces

* Spaces

* Dangling constants removed. Comments position.

* Import sequence.

* Path as argument to readPersistentKeys.
Spaces in comments

* readPersistentKeys test

* Debug and info

* Index check in test

* Update tests/v2/test_waku_rln_relay.nim

Co-authored-by: G. <28568419+s1fr0@users.noreply.github.com>

* Abstracted writeRlnCredentials.
Fix var name in test.

* Usage of writeRlnCredentials in test

* Dnsclient?

* Test reverted to direct call to writeFile.
Abstrated writePersistentRlnCredentials usage causing error,
with readPersistentRlnCredentials

* Indentation

* Revert "Dnsclient?"

This reverts commit 3f4dba1a0b.

* Reverting abstraction of writing..
..persiting credential

Co-authored-by: Daniel Kaiser <git@kais3r.de>
Co-authored-by: Keshav Gupta <keshav.pg@hotmail.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
Co-authored-by: G. <28568419+s1fr0@users.noreply.github.com>
2022-08-05 12:48:01 +02:00
G 32f91cb26b
Noise: split Noise submodule in smaller submodules (#979)
* refactor(noise): split Noise submodule in smaller submodules
2022-08-04 10:47:00 +02:00
Lorenzo Delgado 888f7cb312 refactor(waku-store): major code reorganization, move StoreQueue to message_store folder 2022-08-01 13:02:40 +02:00
Sanaz Taheri Boshrooyeh 79459d29ce
Feat(Rln relay): enables relay-mode only (#1051)
* makes eth private key optional

* makes eth account private key field optional

* clarifies a comment
2022-07-28 11:45:29 -07:00
Lorenzo Delgado e7ba81ff88 refactor(wakunode2): Rename waku_setup.nim to wakunode2_setup.nim 2022-07-19 15:09:23 +02:00
Lorenzo Delgado 10c237dc5f refactor(wakunode2): Split wankunode2 SQL migrations setup to its own file 2022-07-19 15:09:23 +02:00
Lorenzo Delgado e16747137e refactor(wakunode2): Split wankunode2 JSON-RPC setup to its own file 2022-07-19 15:09:23 +02:00
Lorenzo Delgado 538ee12a25 refactor(wakunode2): Split wankunode2 metrics setup to its own file 2022-07-19 15:09:23 +02:00
Lorenzo Delgado 732f853f55 feat(rest): Add HTTP REST API (#727). Integrate REST API with the Waku V2 node 2022-07-19 11:02:43 +02:00
Sanaz Taheri Boshrooyeh daaca4a4a3
Feat(Rln relay): Enables and integrates onchain rln into chat2 (#1029)
* adds wakunode2_types

* removes unused imports

* adds dynamic rln relay config options

* deletes unused imports

* replaces static rln-relay with both modes

* introduces spamHandler parameter to mountRlnRelay

* some debugging info

* introduces the config option

* adds the eth priv key to the register proc

* refactors the code by adding eth private key to the relevant procs

* revises config descriptions

* minor wording fix

* make a minor edit

* further edits

* adds Eth private key to the config options

* calls spam handler when to rln credential is given

* defaults to none for the spam handler

* updates the membership fee to match the contract

* fixes an issue for event subscription

* adds eth private key to the transactions made in the tests

* adds eth private key for the test of register process

* adds comments

* deletes lingering echo

* displays registration info

* removes an echo

* removes Goerli testnet from the config descriptions

* removes an excess space
2022-07-14 12:23:52 +02:00
Sanaz Taheri Boshrooyeh a40686e26e
Feat (Rln relay): enabling signed tx (#1023) 2022-06-30 18:05:38 -07:00
Lorenzo Delgado 993781a137 chore: remove unused imports 2022-06-30 00:35:18 +02:00
Sanaz Taheri Boshrooyeh 5c7baf87e5
Chore(rln-relay): adds more documentation for rln nim bindings (#1016)
* annotates rln nim bindings for more clarity

* adds pointer to the rln construct

* revises further

* addresses comments

* revises proc description of verify

* removes AuthObj
2022-06-29 13:45:56 -07:00
Daniel Kaiser 3b60ea70c0
perf(store): remove vacuum (#1025) 2022-06-29 12:06:42 +02:00
Sanaz Taheri Boshrooyeh 3e20127933
Chore(rln-relay): decouples rln-relay and wakunode2 modules (#1019)
* adds wakunode2_types

* removes unused imports
2022-06-27 12:35:26 -07:00
Hanno Cornelius 9be64a55b6
fix: bridge loses connection to v1 (#1020)
* fix: bridge loses connection to v1

* Remove magic numbers in checks
2022-06-27 14:16:15 +02:00
Daniel Kaiser a2749c4a52
fix(store): limit dbPageSize (#1018) 2022-06-23 17:05:48 +02:00
Lorenzo Delgado cf1a7c3074 feat(rest): Add HTTP REST API (#727). Add Relay REST API 2022-06-23 10:12:38 +02:00
Lorenzo Delgado d2feb7c763 feat(rest): Add HTTP REST API (#727). Amend Debug REST API client 2022-06-22 20:30:44 +02:00
Richard Ramos 17ebb32331 fix: encode protobuffers using proto3 version 2022-06-21 17:21:03 -04:00
Sanaz Taheri Boshrooyeh 6c8ab0ab0f
Feat(rln-relay): enables two modes of onchain and offchain (#992)
* adds the contract handler file

* adds integration test for the group listening

* adds groupManagement proc

* deletes rln relay contract handler file

* brings back all the tests

* replaces toUINT256 with getIdCommitment proc

* replaces individual futures with an array of futures

* adds code documentation

* asyncSpawn instead of await

* adds untitest for toIDCommitment and toUInt256

* reorganizes the test and add rlnInstance

* mounts handleGroupUpdates on the rln peer

* asyncSpawn to await

* implements toIDCommitment

* updates the unittest

* improves the code documentation

* removes unused tests

* WIP

* uncomments the tests

* defines a new mountRlnRelayStatic proc

* splits mountRlnRelay into two procs for dynamic and static group management

* adds a config for off-chain and on-chain rln-relay

* runs dynamic or static mode of rln-relay based on the input config

* adds Eth private key and account configs

* reads Eth private key and account the configs

* comments put the second register proc

* add proper call to the rlnrelay dynamic mode

* adds todo

* adds new rln relay configs

* splits register into two procs

* makes eth account private key non-optional

* removes getIdCommitment and edits the register proc

* removes getIdCommitment calls

* uncomments the commented tests

* fixes a bug

* removes contract deployment for the offchain test

* fixes a bug, edits comments

* removes custom types  without proper parsing and serialization routines from the configs

* fixes a bug

* switches to stew byte utils

* removes log decoding

* WIP

* updates register proc

* edits test titles

* removes eth private key config

* changes the output of register proc to return the registered index

* integrates the registration process into mountRlnRelayDynamic

* integration test for the register proc

* brings back the onchain tests

* updates comments

* cleans up

* disambiguates the Address type namespace

* fixes type ambiguities

* adds default values for rln key and index

* updates config descriptions

* adds type conversion from hex to MembershipKeyPair

* adds more code documentation

* passed the group value instead of option  to the mount proc

* fix a bug

* a minor input type fix for rln chat2

* groups let declarations

* adds default values for addresses

* logs registered keys
2022-06-17 15:00:19 -07:00
Lorenzo Delgado 76e9a98841 feat(rest): Add HTTP REST API (#727). Debug API POC 2022-06-14 18:20:15 +02:00
Daniel Kaiser 5445303a23
feat(store): sqlite only store (#982)
* feat(sqlite): parameterized prep statement exec
  - from nim-eth
* feat(store): sql-only-store
  - add getPage(...) to sql-message-store
  - acts as separate store impl that can be activated with
    `--sqlite-store=true`
* test(store): new test-suite for sql message store
* test(store): waku_store test SQL-store compatible
* perf(store): avoid builing an additional B-tree in sql query
  - use `receiverTime` as the last ORDER BY column to fully utilize the
    primary key sorting
feat(store): retention time
  - retention time in seconds can be specified via `--sqlite-retention-time=`
2022-06-13 19:59:53 +02:00
Hanno Cornelius 6a09ed2c88
feat(enr): add discv5 UDP port to node's main discoverable ENR (#996) 2022-06-09 17:46:21 +02:00
Daniel Kaiser 271a2c750b
chore: adjust dyn bootstrap error message (#993) 2022-06-08 22:23:13 +02:00
Hanno Cornelius 0847eec6e0
chore: reduce unnecessary logging (#991) 2022-06-08 11:20:18 +02:00
Hanno Cornelius 1b29dcda7b
chore: disable swap by default (#989) 2022-06-08 09:16:45 +02:00
Hanno Cornelius 4e9215a920
chore: more regular keepalive pings (#990) 2022-06-07 22:44:58 +02:00
G c259f4cbaa
feat(noise): add Noise Handshake processing and after-handshake encryption (#934)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* feat(noise): add Noise Handshake State Machine and primitives

* feat(noise): add Handshake processing and after handshake encryption

* feat(noise): code updated according to #933

* fix(noise): export HandshakeResult

* feat(noise): add pre message validation utility

* chore(noise): improve documentation

* fix(noise): address reviewers comments
2022-06-03 21:12:41 +02:00
Hanno Cornelius d5a04a2719
chore: update submodules (#987) 2022-06-01 11:49:41 +02:00
Hanno Cornelius 37325c8f48
fix(sqlite): memory leak due to undisposed prepared statements (#981) 2022-05-31 12:47:38 +02:00
Sanaz Taheri Boshrooyeh 1ac029025a
Feat (Rln relay): adds utility procs to listen to the registration events emitted by the rln membership contract (#976)
* adds the contract handler file

* adds integration test for the group listening

* adds groupManagement proc

* deletes rln relay contract handler file

* brings back all the tests

* replaces toUINT256 with getIdCommitment proc

* replaces individual futures with an array of futures

* adds code documentation

* asyncSpawn instead of await

* adds untitest for toIDCommitment and toUInt256

* reorganizes the test and add rlnInstance

* mounts handleGroupUpdates on the rln peer

* asyncSpawn to await

* implements toIDCommitment

* updates the unittest

* improves the code documentation

* removes unused tests

* removes registration of the dynamic group management handler

* adds a comment

* adds a comment

* adds a TODO

* removes getIdCommitment
2022-05-30 12:14:07 -07:00
Lorenzo Delgado 32d230b474 feat(store): ability to decouple store from relay (#937) 2022-05-23 20:42:48 +02:00
G 710ce1315a
feat(noise): add Noise Handshake State Machine and primitives (#933)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* feat(noise): add Noise Handshake State Machine and primitives

* Fixed header

* refactor(noise): add comments, restyle code

* refactor(noise): address reviewer's comments

* feat(noise): add Noise Handshake State Machine and primitives

* refactor(noise): rebase

* refactor(noise): refactor and add documentation

* fix(noise): fixed trace parameter

* update submodules

* fix(noise): remove echo, add stdout.write in print handshake

* refactor(noise): add Noise state machine overview

* Revert "update submodules"

This reverts commit 9fc162dff8.

* update submodules

* fix Jenkins caching

* Revert "update submodules"

This reverts commit 1927d9dd26.

* revert nim-eth submodule update

* fix(noise): missing camelCase

* feat(noise): adding unit tests for Noise State machine primitives

* fix(noise): address reviewers' comments

* fix(noise): use expect for error handling in unit tests

* fix(noise): revert to implicit nonce declaration in CipherState initialization
2022-05-20 15:51:36 +02:00
Hanno Cornelius 984f3179a7
chore: update submodules (#974) 2022-05-20 12:57:19 +02:00
Daniel Kaiser 6a60458135
chore(migration): avoid unnecessarily calling migration (#967) 2022-05-18 21:44:53 +02:00
Hanno Cornelius 185485c336
docs: generate and configure nodekey (#957) 2022-05-18 15:00:30 +02:00
Daniel Kaiser 59b242fcad
fix(dns4name): dns4name + unset port resolved (#965) 2022-05-18 10:15:03 +02:00
Daniel Kaiser 41cdb8ba26
feat(info): add --version cmd flag (#970) 2022-05-17 22:11:07 +02:00
Daniel Kaiser 51cbc39d49
feat(config): add config file support (#953)
including

* bump vendor/nim-confutils
* add nim-toml-serialization submodule
* feat(config): add example config file
* add new feature to CHANGELOG
2022-05-17 17:48:08 +02:00
Hanno Cornelius bd70a1b7bb
Disable GossipSub PX by default (#958) 2022-05-13 18:18:09 +02:00
Sanaz Taheri Boshrooyeh 47a9fcfd77
chore|feat (waku-rln-relay): modules reorganization|Initial test for capturing events using nim-web3 (#941)
* first edition

* adds the full test scenario

* fixes typos

* fixes a bug in the supplied command

* further edits the description

* displays the chat prompt after spam detection

* updates changelog

* minor wording fix

* adds a new test file for onchain rln relay

* adds the Event proc

* adds one working example of event subscription

* defines a new unitt test for event subscription

* adds the new test file

* cleans up the code

* adds a working event subscription for faucet contract

* wip

* makes faucet test conditional

* updates contract byte codes

* adds a working test for event subscription and cleans up the tests

* fixes case

* adss toUInt256 unit function

* enables the tests

* fixes a bug

* undo commented tests

* cleans up the test

* logs the pk

* removes excess entry in the changelog

* fixes spacing

* comments

* removes unused test codes

* adds the conditional compilation for onchain tests

* uncomments offchain tests

* removes onchain tests

* reorganizes the code and moves the rln contract data into a separate module

* deletes txt files

* beautifies the code

* beautifies the code

* removes an excess line

* more formatting fixes

* minor fix

* updates the case of membership fee const

* renames compare to diff

* renames time to e

* edits the number of arguments of the send proc

* fixes a comment alignment

* fixes indentation

* fixed id style

* splits check from condition

* fixes a naming mismatch
2022-05-10 14:09:18 -07:00
Franck R b2eb992bd3
Fix typo (#960) 2022-05-10 12:49:12 +02:00
Daniel Kaiser 5e85eecd14
perf(store): add index on receiverTimestamp (#947) 2022-04-23 10:49:11 +02:00
G cb11c192d6
feat(noise): add support to Waku Payload V2 (#932)
* feat(noise): adding ChaChaPoly encryption

* feat(noise): add support to Noise public keys

* feat(noise): added support to Waku Payload V2

* fix(noise): missing comma

* Fixed header

* refactor(noise): add comments, restyle code

* refactor(noise): address reviewer's comments

* Update nim-eth

* fix(noise): moved strutils in std
2022-04-12 13:50:51 +02:00
Hanno Cornelius 450e0494f1
fix(store): timestamp validity checks (#876) 2022-04-11 14:58:18 +02:00
G d2fccb5220
feat(noise): add support to encrypted/unencrypted public keys (#931)
* feat(noise): adding ChaChaPoly encryption

* test(noise): add test for ChaChaPoly Encryption/decryption

* style(noise): converted variables to camelCase

* feat(noise): add support to Noise public keys

* Fixed header

* refactor(noise): address reviewer's comments

* refactor(noise): small fixes/comments

* refactor(noise): add empty plaintext/ciphertext handling + string encryption test

* refactor(noise): address reviewer's comments
2022-04-06 14:37:02 +02:00
G 45b1e10647
feat(noise): add support to ChaChaPoly encryption (#930)
* feat(noise): adding ChaChaPoly encryption

* fix(noise): restored waku2 tests

* fix(noise): missing comma

* Fixed header

* fix(noise): made comment about empty ChaChaPolyKey clearer

* refactor(noise): address reviewer's comments

* test(noise): add test for ChaChaPoly Encryption/decryption

* refactor(noise): removed libp2p specific debug tools usage

* style(noise): converted variables to camelCase
2022-04-04 17:46:51 +02:00
Daniel Kaiser ecdde23add
feat(store): info message after delete (#929) 2022-03-30 22:55:39 +02:00
Daniel Kaiser a9c31b4b53
feat(store): vacuum after delete (#928) 2022-03-30 22:54:37 +02:00
Daniel Kaiser 2f13aff663
feat(jsonrpc): add call returning git version (#919) 2022-03-30 17:17:21 +02:00
Daniel Kaiser ec0a46b65b
feat(jsonrpc): add ENR to debug_v1_info call (#918) 2022-03-30 15:10:15 +02:00
Hanno Cornelius 52de0e434f
feat(relay): add support for gossipsub peer exchange (#911) 2022-03-29 10:09:48 +02:00
Daniel Kaiser d0cf3ed1f9
fix(store): capacity limit for persistent message store (#900)
* fix(store): limit capacity of persistent message store

fixes performance issue of loading a large message table by deleting
old messages in case of an overflow

* test(store): message capacity limit
2022-03-28 20:27:14 +02:00
Daniel Kaiser be8860c2fc
fix(discv5): info print ENR containing discv5 udp port (#913) 2022-03-28 10:29:19 +02:00
Sanaz Taheri Boshrooyeh c9e82873f1
Increase rln-relay logging (#908)
* updates default rln-relay content topic

* adds further debug and trace level logs

* converts a debug level log to the trace level
2022-03-24 10:47:40 -07:00
Hanno Cornelius 06f20af7d4
chore: update submodules (#905) 2022-03-23 15:20:07 +02:00
Hanno Cornelius 994ac49472
Fix: timestamp type could overflow before conversion (#892) 2022-03-18 12:04:11 +02:00
Daniel Kaiser e1146b2df8
discv5 dns bootstrap (#888) 2022-03-17 17:33:17 +01:00
Hanno Cornelius 266d6a7e80
fix: prevent empty store response due to blocked walker (#897) 2022-03-17 17:18:59 +02:00
Sanaz Taheri Boshrooyeh 2d6a2bec47
Updates the type of the `proof` field of wakumessage to `RateLimitProof` (#893)
* permanently changes the wakumessage proof field to RateLimitProof

* updates an old comment
2022-03-16 13:51:18 -07:00
Daniel Kaiser e6fdbdb42a
add RAM requirement to README (#889) 2022-03-16 17:13:23 +01:00
Hanno Cornelius eb02e4411b
Remove content topic dimension from message metrics (#891) 2022-03-15 14:39:32 +02:00
Hanno Cornelius ea72a65ef8
Increase store and store queue logging (#887) 2022-03-11 07:26:15 +02:00
Hanno Cornelius 9165720f63
Bridge improvements (#875) 2022-03-08 11:48:17 +01:00
Sanaz Taheri Boshrooyeh 2f73667a96
Fixes getNanosecondTime() time conversion (#878)
* fixes conversion from sec to nanosec

* uses snake case for power of 10 for readability
2022-03-07 16:56:39 +01:00
Hanno Cornelius 06fbb875c8
Update submodules (#873)
* Update submodules

* Fix MsgIDProvider
2022-03-02 13:24:48 +01:00
Daniel Kaiser 2972a50035
discv5 stage1 testnet (#863) 2022-03-01 17:12:23 +01:00
Daniel Kaiser dbe76d29ce
discv5 stage1 beta test (#862)
* update vendor/nim-eth submodule

point to the selectable `protocol-id` feature branch

* use d5waku as protocol-id

Use compiletime flag to change the discv5 protocol-id to "d5waku" allowing
nim-waku to span a separate discv5 network

* waku discv5 first beta stage

  - start discv5 during node start
  - added config options for discv5

* fix: pass to tcp port also when specifying ext ip

* bump nim-eth

Co-authored-by: ksr <kaiserd@users.noreply.github.com>
2022-03-01 15:11:56 +01:00
Hanno Cornelius 6be0fb233a
Add bridge docs and minor improvements (#867)
* Add bridge docs and minor improvements

* Clarify that bridging is bidirectional
2022-02-28 18:16:50 +01:00
Hanno Cornelius 20efdd7255
Store data model updates and fixes (#864) 2022-02-28 17:29:01 +01:00
G c27086e186
Moved PB Timestamp index to 10 (#861) 2022-02-23 08:19:52 +01:00
Hanno Cornelius 0b7b033550
Add docs on configuring domain and dns disc (#856) 2022-02-21 16:23:55 +01:00
Hanno Cornelius 2fd5b6c4ea
Domain name config for `nwaku` node (#852) 2022-02-18 12:10:38 +01:00
G dd2ed886a3
Update WakuStoreCodec string to beta4 (#855)
* Fix waku store version to beta4

* Updated changelog
2022-02-18 07:23:10 +01:00
G 21cac6d491
Refactoring timestamps (#842)
* Refactor timestamps type from float64 to int64 (milliseconds resolution)

* Revert epochs to float64

* Update 00002_addSenderTimeStamp.up.sql

* Update quicksim2.nim

* Add files via upload

* Delete 00003_convertTimestampsToInts.up.sql

* Add files via upload

* Rename 00003_convertTimestampsToInts.up.sql to 00003_addTimestampsToInts.up.sql

* Delete 00003_addTimestampsToInts.up.sql

* Rln-relay integration into chat2 (#835)

* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* removes duplicate epoch time calculation

* updates log level to trace

* updates default rln-relay content topic

* adds support for empty content topics

* updates changelog

* changelog updates

* removes a commented code block

* updates addRLNRelayValidator string doc

* Squashed commit of the following:

commit bc36c99ab2
Merge: dc2b2946 5a77d6e2
Author: G <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 01:10:06 2022 +0100

    Merge branch 'master' into int64-timestamps-ns

commit dc2b294667
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:24:45 2022 +0100

    Fix

commit f97b95a036
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:13:18 2022 +0100

    Missing import

commit 060c4f8d64
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Sat Feb 5 00:10:36 2022 +0100

    Fixed typo

commit 08ca99b6f6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:59:20 2022 +0100

    Time util file

commit 2b5c360746
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:33:20 2022 +0100

    Moved time utility functions to utils/time

commit fdaf121f08
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:10:25 2022 +0100

    Fix comment

commit c7e06ab4e7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 23:04:13 2022 +0100

    Restore previous migration script

commit 80282db1d7
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:54:15 2022 +0100

    Typo

commit b9d67f89b0
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:49:29 2022 +0100

    Added utilities to get int64 nanosecond, microsecond, millisecond time resolution from float

commit 0130d496e6
Author: s1fr0 <28568419+s1fr0@users.noreply.github.com>
Date:   Fri Feb 4 22:36:35 2022 +0100

    Switched to nanoseconds support.

* Update CHANGELOG.md

* Create 00003_convertTimestampsToInt64.up.sql

Migration script

* Moved migration script to right location

* Update waku_rln_relay_utils.nim

* Update waku_rln_relay_utils.nim

* Addressed reviewers' comments

* Update default fleet metrics dashboard (#844)

* Fix

* No need for float

* Aligning master to changes in PR

* Further fixes

Co-authored-by: Sanaz Taheri Boshrooyeh <35961250+staheri14@users.noreply.github.com>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2022-02-17 16:00:15 +01:00
Hanno Cornelius bb3e59454e
Store performance improvements (#849) 2022-02-17 11:00:45 +01:00
Sanaz Taheri Boshrooyeh d851d48424
Updates rln-chat2 interface (#846)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* WIP

* removes duplicate epoch time calculation

* converts echo to log

* invokes handler

* bug fix

* prints calculated epoch

* changes the format of printed epoch

* updates log levels

* logs the input buffer supplied to the generate_proof

* replaces echos with logs

* changes log level to trace

* resets the log level of chat2 to INFO

* upgrades log level to debug

* exports toRLNSignal and adds a doc string

* updates log level

* enables all test2

* removes an echo statement

* modifies a comment

* further updates on the log level

* a minor update

* invokes the spam handler when provided

* checks for payload version

* deletes a redundant check

* deletes a rendant check

* updates default rln-relay cht2 content topic

* adds a todo and log

* changes the case of testnet content topic

* removes a flaky check
2022-02-16 14:52:21 -08:00
Hanno Cornelius e1254c20ae
Support for dns `multiaddrs` (#851)
* Support for dns addrs

* Tests and minor improvements
2022-02-16 17:12:09 +01:00
Franck R 5a890b5f90
Update websocket documentation to use domain certificates (#848) 2022-02-15 17:07:55 +11:00
Sanaz Taheri Boshrooyeh 5a77d6e2a6
Rln-relay integration into chat2 (#835)
* adds ProofMetadata

* adds EPOCH_INTERVAL

* adds messageLog field

* adds updateLog, toEpoch, fromEpoch, getEpoch, compareTo

* adds unit test for toEpoch and fromEpoch

* adds unit test for Epoch comparison

* adds result codes for updateLog

* adds unit test for update log

* renames epoch related consts

* modifies updateLog with new return type and new logic of spam detection

* adds unit text for the modified updateLog

* changes max epoch gap type size

* splits updateLog into two procs isSpam and updateLog

* updates unittests

* fixes a bug, returns false when the message is not spam

* renames messageLog to nullifierLog

* renames isSpam to hasDuplicate

* updates the rln validator, adds comments

* adds appendRLNProof proc plus some code beatification

* unit test for validate message

* adds unhappy test to validateMessage unit test

* renames EPOCH_UNIT_SECONDS

* renames MAX_CLOCK_GAP_SECONDS

* WIP: integration test

* fixes compile errors

* sets a real epoch value

* updates on old unittests

* adds comments to the rln relay tests

* adds more comments

* makes rln import conditional

* adds todos

* adds more todos

* adds rln-relay mount process into chat2

* further todos

* logs contentTopic

* introduces rln relay configs

* changes default pubsub topic

* adds contentTopic config

* imports rln relay dependencies

* consolidates imports

* removes module identifier from ContentTopic

* adds contentTopic field

* adds contentTopic argument to mountRlnRelay calls

* appends rln proof to chat2 messages

* changes the default chat2 contentTopic

* adds missing content topic fields

* fixes a bug

* adds a new logic about empty content topics

* appends proof only when rln flag is active

* removes unnecessary todos

* fixes an indentation issue

* adds log messages

* verifies the proof against the concatenation of msg payload and content topic

* a bug fix

* removes duplicate epoch time calculation

* updates log level to trace

* updates default rln-relay content topic

* adds support for empty content topics

* updates changelog

* changelog updates

* removes a commented code block

* updates addRLNRelayValidator string doc
2022-02-04 15:58:27 -08:00
Sanaz Taheri Boshrooyeh a5f8b90c23
Enables RLN compilation under CI env var (#786)
* compiles RLN if CI is defined

* minor ineffective change

* minor

* removes excess space

* resolves namespace conflicts

* moves pubsub import under compiler flag

* disables rln on-chain tests

* brings back the on-chain tests and fixes a name mismatch

* adds a debug log message

* minor formating

* kills ganache-cli procces on windows

* wip: adds a line to kill ganache-cli on windows

* disables rln-relay tests that depend on ganache-cli

* disables ganache-cli installation

* introduces onchain rln flag

* fixes a flaky test

* enables onchain test in non-windows CIs

* moves variables around

it seems detected_os is undefined, hence the compiler flag onchain_rln, which is conditioned to the os, does not get added

* limits onchain rln to macOS
2022-01-28 13:57:17 -08:00
Hanno Cornelius 4421b8de00
Improve node metrics (#831)
* Improve node metrics

* Add support for empty content topics as a separate label
2022-01-26 12:02:57 +01:00
Sanaz Taheri Boshrooyeh dbccecc562
Resolves naming conflict (#821)
* resolves namespace conflicts

* moves pubsub import under compiler flag
2022-01-21 10:55:36 -08:00
Daniel Kaiser b3a7722297
result aggregation in resume API (#816) 2022-01-18 23:05:41 +01:00
Hanno Cornelius 95d2e8bf71
Round receiver time (#815) 2022-01-17 19:37:06 +01:00
Hanno Cornelius 3e9ca25abc
Update submodules (#814) 2022-01-14 10:25:01 +01:00
Hanno Cornelius c422866a49
Improved `store` query performance (#812) 2022-01-11 14:32:09 +01:00
Hanno Cornelius f3ecdb80ea
Remove node peerInfo copy (#801) 2022-01-10 16:07:35 +01:00
Hanno Cornelius 62dbb3d0f3
Add MessageIdProvider for `WakuRelay` (#803)
* Add MessageIdProvider

* Fix unit test. Changelog.
2022-01-10 15:07:01 +01:00
Hanno Cornelius 685d43b2b3
Ensure store response never exceeds `MaxPageSize` (#811)
* Never exceed maxPageSize, even with no specified pagination

* fix test

* Changelog

* Forgot to stage everything...

* Account for timing difference in Windows
2022-01-07 15:01:23 +01:00
Hanno Cornelius 4e6960ad26
Fix incorrect store read buffer limit (#800)
* Fix store read buffer limit

* Changelog
2022-01-06 13:42:37 +01:00
Hanno Cornelius eb5dcadd45
Added protocol message counters (#804)
* Added protocol message counters

* Fix how stored messages are counted
2022-01-06 12:23:25 +01:00
rshiv b2273fff9a
Handles stale clients in FILTER protocol (#782)
* Handles stale clients

Signed-off-by: rshiv <reeshav96@gmail.com>

* adds test

Signed-off-by: rshiv <reeshav96@gmail.com>

* removes failed client from subscriber list

Signed-off-by: rshiv <reeshav96@gmail.com>

* adds filter timeout config

Signed-off-by: rshiv <reeshav96@gmail.com>

* reverts peer removal logic

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolve ci issues

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comment

Signed-off-by: rshiv <reeshav96@gmail.com>

* solves review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* ChangeLog update

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comment

Signed-off-by: rshiv <reeshav96@gmail.com>

* Update CHANGELOG.md

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>

* fixes review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

* handles CI issues

Signed-off-by: rshiv <reeshav96@gmail.com>

* tries to solve test CI

Signed-off-by: rshiv <reeshav96@gmail.com>

* solves CI issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves ci issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves CI issue

Signed-off-by: rshiv <reeshav96@gmail.com>

* resolves review comments

Signed-off-by: rshiv <reeshav96@gmail.com>

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2021-12-08 18:35:47 +00:00
rshiv ce0607b2eb
resolves readme (#796)
Signed-off-by: rshiv <reeshav96@gmail.com>
2021-12-08 16:04:23 +00:00
Hanno Cornelius 9eb532171e
Fix empty scripts condition. Logging. (#795) 2021-12-07 13:06:59 +01:00