Commit Graph

855 Commits

Author SHA1 Message Date
gabrielmer 47ae19c104
fix: prevent IP 0.0.0.0 from being published and update peers with empty ENR data (#1982) 2023-09-11 11:30:12 +03:00
Aaryamann Challani 645b034367
feat(rln-relay): removed rln from experimental 🚀 (#2001)
* feat(rln-relay): removed rln from experimental 🚀

* fix(waku_node): if rln-relay is nil, return true

* chore(tests): mark rest health as skipped
2023-09-11 12:02:31 +05:30
Aaryamann Challani 21604e6bfd
fix(rln-relay): missed roots during sync (#2015) 2023-09-08 18:24:27 +05:30
Alvaro Revuelta 7d9d8a3fb4
fix(p2p): fix possible connectivity issue (#1996) 2023-09-08 13:36:26 +02:00
NagyZoltanPeter fc6194bb6b
feat: Rest endoint /health for rln (#2011)
* Rest endoint /health added

* Remove dev-debug echo

* Changed not ready message

* Update waku/node/rest/health/handlers.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* Various fixes on PR foundings, added comments

---------

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
2023-09-08 11:19:47 +02:00
Ivan Folgueira Bande 1fb13b0967
Adding healtcheck and reconnection mechanism to the postgres archive driver (#1997)
It starts an asynchronous infinite task that checks the connectivity
with the database. In case of error, the postgres_healthcheck task
tries to reconnect for a while, and if it determines that the connection
cannot be resumed, then it invokes a callback indicating that
situation. For the case of the `wakunode2` app, this callback
quits the application itself and adds a log trace indicating
the connectivity issue with the database.
2023-09-06 19:16:37 +02:00
Aaryamann Challani 5638bd06bb
chore(rln-relay): add isReady check (#1989)
* chore(rln-relay): add isReady check

* fix(rln-relay): multiple parameters for checking if node is in sync

* fix: set latesthead in newHeadCallback

* fix: explicit rpc call

* fix: unhandled exception
2023-09-06 14:16:19 +05:30
Aaryamann Challani 483f40c8f7
chore(rln-relay): clean up nullifier table every MaxEpochGap (#1994) 2023-09-06 13:48:02 +05:30
Aaryamann Challani debc5f19d9
feat(relay): ordered validator execution (#1966)
* feat(relay): ordered validator execution

* fix: make more readable

* test: ignore accepts only

* fix: idempotent .subscribe

* fix(rln-relay): make validators private

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* fix: include comments, unsubscribe behaviour

* fix: compilation

---------

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
2023-09-05 14:35:07 +05:30
Aaryamann Challani 97a7c9d04d
chore(rln-relay): log levels for certain logs (#1986) 2023-09-04 18:43:59 +05:30
Aaryamann Challani a14c32614a
chore(rln-relay): use the only key from keystore if only 1 exists (#1984)
* chore(rln-relay): use the only key from keystore if only 1 exists

* fix: convert iterator to seq and then index into it
2023-09-04 15:46:44 +05:30
Aaryamann Challani be48891f77
chore(rln-relay): confirm that the provided credential is correct using onchain query (#1980) 2023-09-04 11:22:31 +05:30
Alvaro Revuelta 05c988645d
chore(api): validate rln message before sending (rest + rpc) (#1968) 2023-09-01 15:03:59 +02:00
Simon-Pierre Vivier 037b166263
fix: --topic should be ignore when using --pubsub-topic or --content-topic (#1977) 2023-08-31 16:13:45 -04:00
Aaryamann Challani d262837ea5
fix(rln-relay): deserialization of valid merkle roots (#1973) 2023-08-31 13:19:43 +05:30
richΛrd e272bec9c5
chore: add debug log indicating succesful message pushes and also log the message hash (#1965) 2023-08-30 18:10:51 -04:00
Aaryamann Challani e7b2b88f5b
fix(rln-relay): modify keystore credentials logic (#1956)
* fix(rln-relay): modify keystore credentials logic

fix: bump version

* Update waku/waku_rln_relay/group_manager/on_chain/group_manager.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* Update tests/waku_rln_relay/test_waku_rln_relay.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* Update waku/waku_keystore/protocol_types.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* fix: greatly improve error handling

* fix: display proc and appropriate assert

---------

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
2023-08-29 17:46:21 +05:30
Aaryamann Challani bd3be21929
fix(rln-relay): sync from deployed block number (#1955)
* fix(rln-relay): sync from deployed block number

* fix(rln-relay): remove option usage for ints

* fix: unnecessary decl
2023-08-29 13:12:13 +05:30
Aaryamann Challani 01634f57f0
fix(rln-relay): window of acceptable roots synced to rln metadata (#1953)
* fix(rln-relay): window of acceptable roots synced to rln metadata

* fix(rln-relay): s/var/let, use for loop
2023-08-25 23:29:17 +05:30
Aaryamann Challani cc9f8d4254
chore(rln-relay): integrate waku rln registry (#1943) 2023-08-25 22:48:52 +05:30
Simon-Pierre Vivier 4539dfc761
feat(discv5): topic subscriptions update discv5 filter predicate (#1918) 2023-08-23 11:50:59 -04:00
Simon-Pierre Vivier c369b3294e
feat: topic subscriptions updates discv5 ENR (#1875) 2023-08-23 09:53:17 -04:00
Aaryamann Challani c07d63db46
fix(rln-relay): flush_interval incorrectly set (#1933) 2023-08-23 18:23:44 +05:30
Aaryamann Challani 1ae5b5a951
fix(rln-relay): RLN DB should be aware of chain and contract address (#1932) 2023-08-23 18:23:30 +05:30
Aaryamann Challani 8239b45582
chore(rln-relay): rename keystore application to waku-rln-relay (#1924) 2023-08-23 12:02:03 +05:30
Alvaro Revuelta 56c228f815
chore(rln): remove old and add new rln metric (#1926) 2023-08-22 16:30:33 +02:00
Aaryamann Challani 6c6302f9e1
fix(rln-relay): waitFor startup, otherwise valid proofs will be marked invalid (#1920) 2023-08-22 17:47:06 +05:30
Aaryamann Challani f08315cdde
fix(rln-relay): remove registration capability (#1916) 2023-08-22 11:49:18 +05:30
Alvaro Revuelta af95b5713f
chore(rln): run rln in all relay pubsubtopics + remove cli flags (#1917) 2023-08-21 08:55:34 +02:00
Aaryamann Challani b3bb7a1113
fix(rln-relay): invalid start index being set results in invalid proofs (#1915)
* fix(rln-relay): invalid proof usage

* fix(rln-relay): use startIndex from first event in block

* fix: latestIndex set after registerBatch
2023-08-18 16:38:24 +05:30
Simon-Pierre Vivier cf3013962c
feat: update various protocols to autoshard (#1857)
* feat: update FILTER & LIGHTPUSH to autoshard
2023-08-17 08:11:18 -04:00
Aaryamann Challani 8c568cabbe
fix(rln-relay): should error out on rln-relay mount failure (#1904) 2023-08-16 19:00:10 +05:30
Aaryamann Challani 8bcb0acf1a
test(rln-relay): rpc handler to support waku rln relay (#1852) 2023-08-16 10:32:22 +02:00
Hanno Cornelius b9d5d28af5
chore: remove references to v2 (#1898)
* chore: remove references to v2

* fix: lingering rln-relay import path
2023-08-09 18:11:50 +01:00
Alvaro Revuelta 03363f1bd8
fix: fixes out of bounds crash when waku2 is not set (#1895) 2023-08-08 16:22:53 +02:00
Hanno Cornelius ab344a9dec
chore: remove Waku v1 and wakubridge code (#1874)
* chore: remove v1 code

* chore: deprecate support for v1 compatibility from JSON-RPC API

* chore: remove v1 tests from JSON-RPC suite

* chore: remove wakubridge code

* chore: remove v1 compatibility functions

* chore: remove support for v1 payloads from chat2 app

* chore: remove v1 from CI processes

* fix: lingering references to private API functions

* fix: remove v1 compat from chat2

* fix: some more lingering references in tests
2023-08-07 15:11:46 +01:00
NagyZoltanPeter 08ff667227
feat: Rest API interface for legacy (v1) filter service. (#1851)
* Added Rest API interface for legacy (v1) filter service with tests.
2023-08-04 11:34:22 +02:00
Vaclav Pavlin 2fc48842b7
fix: check nil before calling clearTimer (#1869) 2023-08-02 13:31:14 +02:00
Aaryamann Challani 4756ccc179
fix(rln-relay): mark duplicated messages as spam (#1867)
* fix(rln-relay): mark duplicated messages as spam

* chore: fix fn desc
2023-08-02 10:40:18 +05:30
Simon-Pierre Vivier bbff1ac138
feat: autosharding core algorithm (#1854)
- basic rendezvous hashing
- content topic parsing
- sharding config
- tests
2023-08-01 09:05:16 -04:00
Aaryamann Challani 76c73b62cd
feat(rln-relay): close db connection appropriately (#1858) 2023-07-27 17:21:21 +05:30
Tanguy 08f3bba334
feat: enable TcpNoDelay (#1470) 2023-07-15 16:41:26 +02:00
Ivan Folgueira Bande 1d3410c7bd
fix(libp2p): Updating nim-libp2p to fix the `wss` connectivity issue (#1848)
* Updating nim-libp2p to fix the `wss` connectivity issue

* The `nim-libp2p` adds many changes although we are actually interested
in the latest commit:

> wstransport.nim: avoid re-raising 'TransportOsError' to avoid stopping
`switch.accept` (#929)

* The `nim-stew` bump is needed so that the `nim-libp2p` can compile.

* The changes in `waku_node.nim` are needed due to the changes in
`nim-stew`.

* waku_node.nim: returning a "completed" future within 'statusAndConfidenceHandler'

  This is aimed to avoid the next exception happening in the wakunode:

  ```
  Unhandled defect: Async procedure (service.nim(74)    callHandler)
  yielded `nil`, are you await'ing a `nil` Future? [AssertionDefect]
  ```

  * Adding 'async' to 'statusAndConfidenceHandler' so that it properly returns a Future[void]
2023-07-13 18:00:55 +02:00
Aaryamann Challani 3fe4522a7e
chore(rln-relay): verify proofs based on bandwidth usage (#1844)
* chore(rln-relay): Verify proofs based on bandwidth usage

* fix(rln-relay): make default threshold 0, for backwards compat

* fix(rln-relay): add unit test for bandwidth cutoff

* fix(rln-relay): rlnRelayBandwidthTreshold option
2023-07-07 17:28:37 +05:30
Alvaro Revuelta 17b24cded6
feat: limit relay connections below max conns (#1813) 2023-07-04 13:31:18 +02:00
Ivan Folgueira Bande 97d3b9f79f
chore(refactor): Move record creation & fix libwaku compilation (#1833)
* Move record creation & fix libwaku

* app,external_config,internal_config,libwaku,sharding: refactorgin to make it compile

---------

Co-authored-by: SionoiS <simvivier@status.im>
2023-06-29 21:59:53 +02:00
Ivan Folgueira Bande 88b7481f29
feat(postgres): integration of postgres in wakunode2 (#1808)
* Making the wakunode2 to support postgres driver

* driver/builder.nim: controling possible errors when creating the messages table

* postgres_driver.nim: adding protection in getInt and fixing typo
2023-06-28 18:47:28 +02:00
Simon-Pierre Vivier cf46fb7cf6
chore: discv5 re-org clean-up (#1823)
- Remove discv5 from node and it's builder
- Require a record to init a node (was previously using discv5 record).
- Fix various tests that assumed nodes with discv5.
- Fix ENR creation
- Fix wakubridge
- Fix networkmonitor
2023-06-28 08:57:10 -04:00
Aaryamann Challani e4d9ee1f7a
fix(rln-relay): chunk event fetching (#1830) 2023-06-28 18:00:45 +05:30
Alvaro Revuelta e12c979c4e
chore(networking): disconnect due to colocation ip in conn handler (#1821) 2023-06-28 09:14:11 +02:00
Ivan Folgueira Bande 3aefade647
fix(discv5): Fixing issue that prevented the wakunode2 from starting (#1829)
* Fixing issue that prevented the wakunode2 from starting

The issue was introduced in PR#1818.

Before this commit, the `wakunode2` app crashed with the next error:

ERR 2023-06-27 15:57:27.268+00:00 5/7 Starting node and protocols failed
topics="wakunode main" tid=1 file=wakunode2.nim:92 error="failed to
start waku discovery v5: "

* fix tests accordingly

* publisher.nim, subscriber.nim: fix
2023-06-27 15:16:59 -04:00
Simon-Pierre Vivier 62d3653022
chore: move discv5 out of node. (#1818)
- Refactor discv5 start, stop & loop.
- Fix tests.
2023-06-27 09:50:11 -04:00
Ivan Folgueira Bande 52894a82d0
refactor(archive): Moving waku archive logic from app.nim to the archive module (#1817)
* Refactoring the Waku Archive. Simplifying the app.nim

This change is needed to accommodate the further PRs where we will integrate Postgres in `wakunode2`.
2023-06-27 13:24:31 +02:00
Alvaro Revuelta 71c4ac1641
chore: add peer manager config to builder (#1816) 2023-06-23 15:30:28 +02:00
Simon-Pierre Vivier 44f9d8dc0e
chore: discv5 re-org setup (#1815)
Key, netconfig, enr are created at App initialization and config files has been renamed.
2023-06-22 16:58:14 -04:00
Ivan Folgueira Bande a44d4bfbcd
refactor(databases): Creation of the databases folder to keep the logic for sqlite and postgres (#1811)
* Refactoring in sqlite and postgres. Creation of the databases folder.
2023-06-22 11:27:40 +02:00
Simon-Pierre Vivier a4da87bb8c
feat: discovery peer filtering for relay shard (#1804)
Add discv6 predicate that filter peer by static shard.

Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2023-06-20 12:08:10 -04:00
Simon-Pierre Vivier 50412d1880
sharded topic validation & ENR update (#1795)
Add sharded topic validation and update the node ENR accordingly.
2023-06-19 18:16:25 -04:00
Aaryamann Challani bbded9eea7
feat(rln-relay): resume onchain sync from persisted tree db (#1805)
* feat(rln-relay): resume onchain sync from persisted tree db

* chore(rln-relay): bump zerokit
2023-06-19 15:21:18 +05:30
Aaryamann Challani 045f07c616
feat(rln-relay): metadata ffi api (#1803)
* feat(rln-relay): metadata ffi api

* chore(rln-relay): bump to latest master after merge

* fix(rln-relay): naming, visibility, tests
2023-06-16 11:33:41 +05:30
Tanguy f6e89c311d
chore(deps): bump libp2p & websock (#1800) 2023-06-15 13:17:19 +02:00
Aaryamann Challani 940206b439
feat(rln-relay): integrate get_leaf ffi api (#1790) 2023-06-12 15:30:07 +05:30
Ivan Folgueira Bande 5fc5770da9
feat(postgres): complete implementation of driver and apply more tests (#1785) 2023-06-09 12:42:33 +02:00
Tanguy 613974c1b6
Revert "chore(libp2p): update nim-libp2p (#1786)" (#1788)
This reverts commit 93e09b8b77.
2023-06-09 10:10:47 +02:00
Aaryamann Challani dba84248f1
chore(rln-relay): pass in the path to the tree db (#1782)
* chore(rln-relay): pass in the path to the tree db

* fix(rln-relay): address visibility

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* fix(rln-relay): make db used more explicit

* fix(rln-relay): reduce visibility

---------

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
2023-06-08 17:22:21 +05:30
Vaclav Pavlin 93e09b8b77
chore(libp2p): update nim-libp2p (#1786) 2023-06-08 12:54:28 +02:00
Aaryamann Challani ba8ec7048e
chore(rln-relay): update tree_config (#1781) 2023-06-07 13:39:20 +05:30
Ivan Folgueira Bande cb2e3d86a6
feat(postgres): Adding a postgres async pool to make the db interactions asynchronous (#1779)
* feat(postgres): added postgres async pool wrapper

---------

Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
2023-06-07 10:08:43 +02:00
Aaryamann Challani a00aa8cc59
feat(rln-relay): pass in index to keystore credentials (#1777)
* feat(rln-relay): pass in index to keystore credentials

* fix(rln-relay): s/MembershipIndex/uint
2023-06-06 23:13:05 +05:30
Alvaro Revuelta 34a9263191
feat(networking): integrate gossipsub scoring (#1769) 2023-06-06 19:28:47 +02:00
Hanno Cornelius 21737c7c1b
feat(discv5): added find random nodes with predicate (#1762) (#1763)
* feat(discv5): added find random nodes with predicate (#1762)

Co-authored-by: Lorenzo Delgado <lnsdev@proton.me>

* chore: play around with sleep times on tests

* chore: simplify flaky tests

* chore: update waku/v2/node/waku_node.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* chore: update tests/v2/test_waku_discv5.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

* chore: update examples/v2/subscriber.nim

Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>

---------

Co-authored-by: Lorenzo Delgado <lnsdev@proton.me>
Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
2023-06-06 16:36:20 +02:00
Vaclav Pavlin 835a409d10
feat(wakunode2): enable libp2p rendezvous protocol by default (#1770) 2023-06-01 21:43:10 +02:00
Ivan Folgueira Bande 7df6f4c851
feat(postgresql): align Andrea's PR#1590 changes into master (#1764)
* Add postgres basic support
* test_driver_postgres.nim: adding simple test to validate asynchronous-ness
* Aligning the changes made by Andrea to the current master state
* test_driver_postgres.nim: new unit test (checking duplicate insertion)

---------

Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2023-05-31 10:28:48 +02:00
Alvaro Revuelta 047d1cf095
feat(networking): prune peers from same ip beyond colocation limit (#1765) 2023-05-31 09:47:56 +02:00
Ivan Folgueira Bande 59ca03a875
feat(postgresql): 1st commit to async sql (waku_archive/driver...) (#1755) 2023-05-25 17:34:34 +02:00
Alvaro Revuelta 3c2d2891e5
chore(px): close px streams after resp is sent (#1746) 2023-05-25 16:42:48 +02:00
Aaryamann Challani 6257874633
chore(rln-relay): updated metrics for testnet 3 (#1744)
* fix(rln-relay): trace log

* chore(rln-relay): updated metrics for testnet 3

fix(rln-relay): group manager metric import
2023-05-19 16:24:35 +05:30
Aaryamann Challani 5eae60e817
fix(rln-relay): trace log (#1743) 2023-05-19 12:52:46 +05:30
Alvaro Revuelta 87f694a8b6
chore(networking): set and use target outbound connections + prune (#1739) 2023-05-18 09:40:14 +02:00
Aaryamann Challani 611e9539a6
feat(rln-relay): use new atomic_operation ffi api (#1733)
* chore(rln-relay): bump zerokit

* feat(rln-relay): use new atomic_operations ffi api

* fix(rln-relay): static gm
2023-05-18 10:42:08 +05:30
Ivan Folgueira Bande 665484c17b
refactor: proper use of setupNat (#1740)
Notice that I had to adapt to use 'rlpx_connected_peers' instead
of 'connected_peers' in 'wakunode1.nim' because due to the update
of the 'vendor/nim-eth', which adds the dependency-break with
'confutils' but also includes another changes.

Aside note: we cannot have 'confutils' dependency in 'nim-eth' because
that will prevent the generation of any waku dynamic library.
2023-05-17 18:32:53 +02:00
Ivan Folgueira Bande 2defbd2301
feat(cbindings): first commit - waku relay (#1632) (#1714)
* feat(cbindings): first commit - waku relay (#1632)

* test_app.nim: fix compilation issue. App.init(..) -> App.new(..)

* Simplifying library name (libwaku) and standardizing function names (waku_*)

* Proper wrapper of the waku_node API and creation of the libwaku.a

* Rolling back changes that are not needed

* Rolling back changes that are out of the scope of this task

* wakunode.nim: Removing unnecessary import

* Aplying PR suggestions

* Renaming 'waku.h' -> 'libwaku.h'

* Use of 'isNil' instead of '== nil'

* libwaku.nim: explicitly setting waku_poll() as gcsafe
2023-05-12 18:08:41 +02:00
Aaryamann Challani b277ce1013
chore(rln-relay): update args to contract (#1724)
* chore(rln-relay): update args to contract

* fix(rln-relay): update membership contract code

* fix: bytecode
2023-05-11 19:30:26 +05:30
Alvaro Revuelta b9e0763e6b
fix: streams was used instead of connections (#1722) 2023-05-10 15:28:00 +02:00
Aaryamann Challani 44c543129e
chore(rln-relay): use new config for ffi (#1718)
* chore(rln-relay): use new config for ffi

* chore: update zerokit submodule

* fix: missing comma
2023-05-10 12:54:58 +05:30
Hanno Cornelius 7c39be9a2e
fix: change filter request default behaviour to ping (#1721) 2023-05-09 09:18:14 +02:00
Hanno Cornelius 68a39c65bd
fix: fix filter v2 proto fields (#1716) 2023-05-05 10:29:53 +02:00
Alvaro Revuelta 3e0a693d2e
chore: add timestamp and ephemeral for opt-in dos validator (#1713) 2023-05-05 10:12:49 +02:00
Alvaro Revuelta 3e14686950
chore: add validator for dos protec metrics and move to app (#1704) 2023-05-02 16:20:38 +02:00
fryorcraken.eth ba12d1c1e4
Fix links and add notes on Waku v1 deprecation (#1707) 2023-05-02 09:56:25 +02:00
Lorenzo Delgado ce92fc1aed
refactor(wakunode2): flatten and simplify app setup (#1705) 2023-04-26 19:25:18 +02:00
Vaclav Pavlin b397ed6cbd
refactor:optimize getting number of connections and streams (#1673) 2023-04-26 10:47:46 +02:00
Alvaro Revuelta c6d291d3f3
chore: add payload bytes to trace log (#1703) 2023-04-25 09:50:43 +02:00
Lorenzo Delgado e041e04342
chore(core): move peers utils module to waku_core 2023-04-24 16:37:54 +02:00
Alvaro Revuelta 67e96ba894
feat: add metrics with msg size histogram (#1697) 2023-04-20 13:45:29 +02:00
Alvaro Revuelta 7d12adf636
fix: import nimchronos instead heartbeat (#1695) 2023-04-19 16:51:15 +02:00
Lorenzo Delgado 93b0c0713e
chore(core): move utils time module to waku_core 2023-04-19 16:39:52 +02:00
Alvaro Revuelta 14305c610a
feat: curate peers shared over px protocol (#1671) 2023-04-19 16:12:00 +02:00
Lorenzo Delgado d5ef9331e7
fix(rest): change rest server result error type to string 2023-04-19 14:55:39 +02:00
Lorenzo Delgado e8dceb2aa6
chore: add deprecation notice to utils module. move heartbeat to common 2023-04-19 14:27:16 +02:00