* db_postgres, postgres_driver: better performance by using callback.
There were a bunch of milliseconds being lost due to multiple-row
processing. This commit aims to have the minimum possible row
process time.
* pgasyncpool: clarifying logic around pool conn management.
* db_postgres: removing duplicate code and more searchable proc names.
* postgres_healthcheck: validate once per minute instead of 30 sec
* postgres_driver.nim: change MaxNumCons from 5 to 50
* postgres_driver.nim: split connPool into writeConPool and readConPool
This aims to avoid clashes in insert and select queries
because the inserts and selects can happen concurrently
in relay and store events, respectively.
/admin rest api implementation and tests
* open api doc
* Add rest admin test to all tests
* Enrich /admin get peers interface, group protocols by peers in response
* This reverts former change on lighpush error handling, now zero peer publish still succeed. This will allow js-waku use case to succeed.
* Adjust lightpush rest-api test
* chore: add retention policy with GB or MB limitation #1885
* chore: add retention policy with GB or MB limitation
* chore: updated code post review- retention policy
* ci: extract discordNotify to separate file
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* ci: push images to new wakuorg/nwaku repo
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* ci: enforce default Docker image tags strictly
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* ci: push GIT_REF if it looks like a version
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* fix: update wakuv2 fleet DNS discovery enrtree
https://github.com/status-im/infra-misc/issues/171
* chore: resolving DNS IP and publishing it when no extIp is provided (#2030)
* feat(coverage): Add simple coverage (#2067)
* Add test aggregator to all directories.
* Implement coverage script.
* fix(ci): fix name of discord notify method
Also use absolute path to load Groovy script.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* chore(networkmonitor): refactor setConnectedPeersMetrics, make it partially concurrent, add version (#2080)
* chore(networkmonitor): refactor setConnectedPeersMetrics, make it partially concurrent, add version
* add more metrics, refactor how most metrics are calculated
* rework metrics table fillup
* reset connErr to make sure we honour successful reconnection
* chore(cbindings): Adding cpp example that integrates the 'libwaku' (#2079)
* Adding cpp example that integrates the `libwaku`
---------
Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
* fix(ci): update the dependency list in pre-release WF (#2088)
* chore: adding NetConfig test suite (#2091)
---------
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Co-authored-by: Jakub Sokołowski <jakub@status.im>
Co-authored-by: Anton Iakimov <yakimant@gmail.com>
Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Álex Cabeza Romero <alex93cabeza@gmail.com>
Co-authored-by: Vaclav Pavlin <vaclav@status.im>
Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
* dburl.nim: simpler regex that can support db_urls with . and - in hostname
* dbrul.nim: accepting any non-empty sequence for user and password
* dburl.nim: skipping validation for 'sqlite' db paths
RestApi Lightpush endpoint implemented
* Openapi definition for lightpush rest api
* Update waku/node/rest/lightpush/handlers.nim
* Fix install handler naming, added negative test cases and fixes for restapi lightpush
* Fix error handling in lightpush rest handler
* Fix main success case - relay message that comes with pushRequest
* Fix rest relay serdes test with RelayWakuMessage validation changes
* Checking response message tests
* When adding the dependency with 'librln' to 'libwaku', it is required
to remove the dependency with 'confutils' because the 'nim-confutils'
module prevents the creation of a dynamic library (libwaku.so.)
* waku_example.c: less code is needed because the Waku Thread attends
any Waku event whereas the main thread can have a blocking scanf
to retrieve user inputs.
Filter v2 rest api support implemented
Filter rest api documentation updated with v1 and v2 interface support.
Separated legacy filter rest interface
Fix code and tests of v2 Filter rest api
Filter v2 message push test added
Applied autoshard to Filter V2
Redesigned FilterPushHandling, code style, catch up apps and tests with filter v2 interface changes
Rename of FilterV1SubscriptionsRequest to FilterLegacySubscribeRequest, fix broken chat2 app, fix tests
Changed Filter v2 push handler subscription to simple register
Separate node's filterUnsubscribe and filterUnsubscribeAll
* removing implicit dependency with libpq if postgres is not being used.
* We only run the postgres tests when explicitly willing to, i.e. make
POSTGRES=1 test. The reason is that the postgres tests expect a
database instance to be running locally.
It starts an asynchronous infinite task that checks the connectivity
with the database. In case of error, the postgres_healthcheck task
tries to reconnect for a while, and if it determines that the connection
cannot be resumed, then it invokes a callback indicating that
situation. For the case of the `wakunode2` app, this callback
quits the application itself and adds a log trace indicating
the connectivity issue with the database.
* chore: remove v1 code
* chore: deprecate support for v1 compatibility from JSON-RPC API
* chore: remove v1 tests from JSON-RPC suite
* chore: remove wakubridge code
* chore: remove v1 compatibility functions
* chore: remove support for v1 payloads from chat2 app
* chore: remove v1 from CI processes
* fix: lingering references to private API functions
* fix: remove v1 compat from chat2
* fix: some more lingering references in tests
* Updating nim-libp2p to fix the `wss` connectivity issue
* The `nim-libp2p` adds many changes although we are actually interested
in the latest commit:
> wstransport.nim: avoid re-raising 'TransportOsError' to avoid stopping
`switch.accept` (#929)
* The `nim-stew` bump is needed so that the `nim-libp2p` can compile.
* The changes in `waku_node.nim` are needed due to the changes in
`nim-stew`.
* waku_node.nim: returning a "completed" future within 'statusAndConfidenceHandler'
This is aimed to avoid the next exception happening in the wakunode:
```
Unhandled defect: Async procedure (service.nim(74) callHandler)
yielded `nil`, are you await'ing a `nil` Future? [AssertionDefect]
```
* Adding 'async' to 'statusAndConfidenceHandler' so that it properly returns a Future[void]
* chore(rln-relay): Verify proofs based on bandwidth usage
* fix(rln-relay): make default threshold 0, for backwards compat
* fix(rln-relay): add unit test for bandwidth cutoff
* fix(rln-relay): rlnRelayBandwidthTreshold option
* Move record creation & fix libwaku
* app,external_config,internal_config,libwaku,sharding: refactorgin to make it compile
---------
Co-authored-by: SionoiS <simvivier@status.im>
* Making the wakunode2 to support postgres driver
* driver/builder.nim: controling possible errors when creating the messages table
* postgres_driver.nim: adding protection in getInt and fixing typo
- Remove discv5 from node and it's builder
- Require a record to init a node (was previously using discv5 record).
- Fix various tests that assumed nodes with discv5.
- Fix ENR creation
- Fix wakubridge
- Fix networkmonitor
* Fixing issue that prevented the wakunode2 from starting
The issue was introduced in PR#1818.
Before this commit, the `wakunode2` app crashed with the next error:
ERR 2023-06-27 15:57:27.268+00:00 5/7 Starting node and protocols failed
topics="wakunode main" tid=1 file=wakunode2.nim:92 error="failed to
start waku discovery v5: "
* fix tests accordingly
* publisher.nim, subscriber.nim: fix
* Refactoring the Waku Archive. Simplifying the app.nim
This change is needed to accommodate the further PRs where we will integrate Postgres in `wakunode2`.
* chore(rln-relay): pass in the path to the tree db
* fix(rln-relay): address visibility
Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
* fix(rln-relay): make db used more explicit
* fix(rln-relay): reduce visibility
---------
Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
* feat(discv5): added find random nodes with predicate (#1762)
Co-authored-by: Lorenzo Delgado <lnsdev@proton.me>
* chore: play around with sleep times on tests
* chore: simplify flaky tests
* chore: update waku/v2/node/waku_node.nim
Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
* chore: update tests/v2/test_waku_discv5.nim
Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
* chore: update examples/v2/subscriber.nim
Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
---------
Co-authored-by: Lorenzo Delgado <lnsdev@proton.me>
Co-authored-by: Ivan Folgueira Bande <128452529+Ivansete-status@users.noreply.github.com>
* Add postgres basic support
* test_driver_postgres.nim: adding simple test to validate asynchronous-ness
* Aligning the changes made by Andrea to the current master state
* test_driver_postgres.nim: new unit test (checking duplicate insertion)
---------
Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
Notice that I had to adapt to use 'rlpx_connected_peers' instead
of 'connected_peers' in 'wakunode1.nim' because due to the update
of the 'vendor/nim-eth', which adds the dependency-break with
'confutils' but also includes another changes.
Aside note: we cannot have 'confutils' dependency in 'nim-eth' because
that will prevent the generation of any waku dynamic library.
* feat(cbindings): first commit - waku relay (#1632)
* test_app.nim: fix compilation issue. App.init(..) -> App.new(..)
* Simplifying library name (libwaku) and standardizing function names (waku_*)
* Proper wrapper of the waku_node API and creation of the libwaku.a
* Rolling back changes that are not needed
* Rolling back changes that are out of the scope of this task
* wakunode.nim: Removing unnecessary import
* Aplying PR suggestions
* Renaming 'waku.h' -> 'libwaku.h'
* Use of 'isNil' instead of '== nil'
* libwaku.nim: explicitly setting waku_poll() as gcsafe
* feat: new rest api based on the current store json-rpc api and
following the same structure as the current relay rest api.
* feat: the store api attend GET requests to retrieve historical messages
* feat: unit tests.
* feat: allow return message to rest-client in case error (4XX or 5XX)
* chore: always allow to call the store api endpoints (only rest) without explicit storenode (#1575)
* feat: always mounting the current node as storenode client
* feat: new filter protocol service node
* feat: add test and fix compilation errors
* feat: more tests and minor fixes
* chore: update waku/v2/protocol/waku_filter_v2/protocol.nim
Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
* chore: add some convenience functions
---------
Co-authored-by: Lorenzo Delgado <lorenzo@status.im>
* feat(rln-relay): init group manager integration
fix(rln-relay): integrate group manager. todo spam and reg handlers
fix(rln-relay): decouple waku-relay and waku-rln-relay
fix(rln-relay): compiles now
fix(chat2): compilation
fix(rln-relay): wip segfault
fix(rln-relay): segfault
fix(chat2|wakunode2): use optional field
fix(rln-relay): wakunode test
fix(rln-relay): uncomment fields in proto decode
fix(rln-relay): used pragma on tests
fix(rln-relay): include cred processing
fix(rln-relay): add reg callback
fix(rln-relay): args to mount
fix(rln-relay): add timeout to waitForExit
fix(rln-relay): use osproc term instead of posix kill
fix(rln-relay): use poParentStream to prevent deadlock
fix(rln-relay): remove poParentStream, remove ganache log output
* fix(rln-relay): abstract tuple into own type
* feat(credentials): store and processing generic app credentials
* feat(credentials): separate module; minimal tests
* more work
* feat(credentials): check presence of idCredential in keystore and add only new membership groups
* feat(credential): refactor, new data structure, dynamic add credential, filter
* feat(credential): add filter, get credentials
* feat(credential): encode/decode utility
* feat(credential): sort groups, test credential retrieval/group merging
* fix(credential): remove unnecessary order in sort
* fix(credentials): fix vendor commits
* fix(credential/rln): embed credential module in rln relay
* feat(credentials/rln): use credentials API in rln-relay to store/read credentials
* refactor(credentials): implement hasKeys for JsonNode
* fix(credentials): restore connectToNodes call
* refactor(credentials): remove unnecessary imports
* refactor(credentials): add Res suffix to results
* refactor(credential): moved save json to separate proc; added comments
* feat(credentials): use appInfo
* refactor(keystore): refactor code in a more structured module; address reviewers
* fix(keystore): fix indentation
gabrielmer
Ivan FB
Simon-Pierre Vivier
Alvaro Revuelta
NagyZoltanPeter
Abhimanyu
Álex Cabeza Romero
Sergei Tikhomirov
Hanno Cornelius
Anton Iakimov
Aaryamann Challani
richΛrd
Vaclav Pavlin
Tanguy
Tanguy
fryorcraken.eth
Lorenzo Delgado
Vaclav Pavlin
G