5989de88a4
fix: multi nat initialization causing dead lock in waku tests + serialize test runs to avoid timing and port occupied issues ( #2799 )
...
* Prevent multiple nat module initialization that cause dead lock in nat refresh thread tear down during tests.
* NPROC to 1 to avoid parallel test runs can lead to timing and port allocation issues
Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
2024-06-12 07:49:55 +02:00
25a8b98f12
Remove filterTimeout configuration option as it remained after removing legacy filter where it belong to. ( #2797 )
2024-06-11 14:07:46 +02:00
f5d87c5b23
chore: set msg_hash logs to notice level ( #2737 )
2024-06-10 15:56:55 +02:00
55a87d21d6
fix: more detailed logs to differentiate shards with peers ( #2794 )
2024-06-10 13:40:18 +05:30
31bd6d71c0
chore: Minor enhancements ( #2789 )
...
* archive.nim: reduce the database report interval from 1 to 30 min
This counts the number of rows with "select count(1) from messages"
which is quite intense and we shouldn't run it every minute
* aside cleanup
2024-06-09 23:09:23 +02:00
c5d19c4491
chore: postgres_driver - acquire/release advisory lock when creating partitions ( #2784 )
2024-06-07 17:54:26 +02:00
77adfccd93
chore: simple link refactor ( #2781 )
2024-06-07 13:07:15 +05:30
dbe79250da
postgres_driver: simple reformat with nph ( #2785 )
2024-06-06 12:04:40 +02:00
3a93437300
add new index to optimize the order by storedAt ( #2778 )
2024-06-06 11:38:58 +02:00
24661ea969
postgres partitions: ensure the partition ranges are o'clock ( #2776 )
...
Also, skip the error "partition already exists" because that happens
when multiple nodes interact with the same database.
2024-06-05 17:45:38 +02:00
21a4b5bd4a
unifying clusterId to be uint16 ( #2777 )
2024-06-05 15:32:35 +02:00
dfc979a8ef
chore: extract common prefixes into a constant for multiple query ( #2747 )
...
* chore: extract select to constant for multiple prefix
* chore: Add a space at the end of selectClause to facilitate better string merging.
Signed-off-by: DarshanBPatel <darshan@status.im>
---------
Signed-off-by: DarshanBPatel <darshan@status.im>
2024-06-03 21:52:53 +05:30
88b8e1867a
fix(waku_archive): only allow a single instance to execute migrations ( #2736 )
2024-05-31 12:08:16 -04:00
8b42f199ba
postgres_driver.nim: add missing meta field to select queries ( #2741 )
2024-05-29 22:13:16 +02:00
a65b13fcb8
fix: invalid cursor returning messages ( #2724 )
...
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-05-27 10:54:10 -04:00
40296f9dbc
fix: do not print the db url on error ( #2725 )
2024-05-23 18:37:04 -04:00
4a6ec468db
fix: store v3 bug fixes ( #2718 )
2024-05-23 08:01:52 -04:00
1e2aa57a26
chore: android support ( #2554 )
2024-05-21 21:00:22 -04:00
7464684842
chore: Discovery in libwaku ( #2711 )
...
* cwaku_example: add discoveryv5-discovery bool option
* libwaku: implement discovery capabilities
* node_lifecycle_request.nim: better control of possible errors when parsing config
2024-05-21 18:37:50 +02:00
176c98c589
standardize store types by using camel case instead of snake case ( #2709 )
2024-05-17 16:56:54 +02:00
3faffdbcaa
chore: libwaku - allow to properly set the log level in libwaku and unify a little ( #2708 )
...
* waku.nimble: set properly chronicles compilation flags for static libwaku
* adapt examples to new log setup
2024-05-17 16:28:54 +02:00
401630ee24
chore: waku_discv5, peer_manager - add more logs help debug discovery issues ( #2705 )
2024-05-16 22:30:51 +02:00
78132dc12e
chore: generic change to reduce the number of compilation warnings ( #2696 )
2024-05-16 22:29:11 +02:00
9dfdfa2774
feat: Added message size check before relay for lightpush ( #2695 )
2024-05-15 14:13:13 +04:00
3a2caaec6f
refactor shard parsing ( #2699 )
2024-05-14 20:17:17 +02:00
840e012294
chore: move code from wakunode2 to a more generic place, waku ( #2670 )
...
* testlib/wakunode.nim: not use cluster-id == 1 to avoid test rln by default
2024-05-13 17:45:48 +02:00
49dd6c1989
test(peer-and-connection-management): Implement tests ( #2566 )
...
* Implement peer and connection management tests.
* Fix multiple peers added on initialisation.
* Remove clusterId parameter from newTestWakuNode.
2024-05-13 17:25:44 +02:00
7d4857ea6c
chore: closing ping streams ( #2692 )
2024-05-13 12:07:57 +02:00
8451cf8e1b
chore: Postgres enhance get oldest timestamp ( #2687 )
...
* postgres: consider also the existing paritions when getting oldest timestamp
* test_driver_postgres_query: adapt test to oldest timestamp
2024-05-10 18:31:01 +02:00
be5471c6f3
feat: adding json string support to bindings config ( #2685 )
2024-05-10 10:56:17 +02:00
026d804a0d
feat: Added flexible rate limit checks for store, legacy store and lightpush ( #2668 )
...
* Added flexible rate limit checks for store, legacy store and lightpush. Also added rate and traffic metrics.
* Fix chat2 after WakuLegacyStoreCodec rename
* Update waku/common/ratelimit.nim
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
* Update waku/common/ratelimit.nim
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
* Update waku/waku_store_legacy/protocol.nim
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
* Fix review findings, added limit to debug logs
---------
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-05-09 20:07:49 +02:00
fa26d05f8e
fix: message cache removal crash ( #2682 )
2024-05-09 10:38:55 -04:00
d700006a6a
feat: store v3 return pubsub topics ( #2676 )
2024-05-08 15:35:56 -04:00
6a1af92276
chore(rln-relay): health check should account for window of roots ( #2664 )
...
* test(rln-relay): health check should account for window of roots
* fix: some type-fu
* fix: widen the type vs narrowing
* fix: add extra parens
2024-05-08 17:48:44 +05:30
b46226fca0
postgres_driver: delete partitions in time retention policy ( #2679 )
2024-05-07 23:42:01 +02:00
82f95999cd
fix: add `meta` to sqlite migration scripts ( #2675 )
2024-05-07 09:39:06 -04:00
22f64bbd44
chore: updating TWN bootstrap fleet to waku.sandbox ( #2638 )
2024-05-07 13:37:17 +02:00
a46d4451eb
feat: supporting meta field in store ( #2609 )
2024-05-06 10:20:21 +02:00
f8184a6de8
postgres_driver.nim: debug -> trace put PotsgresDriver ( #2667 )
2024-05-03 17:41:14 +02:00
cee020f26d
fix(filter): log is too large ( #2665 )
2024-05-03 19:05:24 +05:30
2463527b24
refactor: big refactor to add waku component in libwaku instead of onlu waku node ( #2658 )
2024-05-03 14:07:15 +02:00
853ec1869e
fix: issue #2644 properly ( #2663 )
2024-05-03 13:40:20 +05:30
404810aa8d
refactor: simplify app.nim and move discovery items to appropriate modules ( #2657 )
2024-05-01 21:13:08 +02:00
e03d1165e6
fix: store v3 validate cursor & remove messages ( #2636 )
2024-05-01 14:47:06 -04:00
5dd645cfc3
fix(waku_keystore): sigsegv on different appInfo ( #2654 )
...
* fix(waku_keystore): sigsegv on different appInfo
* fix: field specific errors
* fix: more verbose error logs
2024-05-01 23:05:22 +05:30
d5e0e4a9b1
chore: log enhancement for message reliability analysis ( #2640 )
...
* log enhancement for message reliability analysis
The next modules are touched:
- waku_node.nim
- archive.nim
- waku_filter_v2/protocol.nim
- waku_relay/protocol.nim
Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
2024-05-01 10:25:33 +02:00
a9e19efd7a
fix(rln-relay): persist metadata every batch during initial sync ( #2649 )
...
* fix(rln-relay): persist metadata every batch during initial sync
* fix: test
* Apply suggestions from code review
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
* patch: isOkOr template
---------
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
2024-04-30 18:52:47 +05:30
4a110f65ff
refactor: metrics server. Simplify app.nim module ( #2650 )
2024-04-30 15:07:17 +02:00
6382dedb42
waku_node: first of all stop the waku-switch when stopping the waku-node ( #2651 )
...
This is aimed to avoid having flaky tests
2024-04-30 12:52:11 +02:00
069b23f67e
waku_node.nim: simplify stop proc ( #2645 )
...
There is no need to explicitly stop mounted libp2p protocols
because they are already being stopped after the switch.stop()
is being called
2024-04-29 17:47:18 +02:00
28389cf043
rest/store/types: contentTopic -> content_topic ( #2646 )
2024-04-29 16:19:07 +02:00
8d1b0834db
fix: handle named sharding in enr ( #2647 )
2024-04-29 18:53:49 +05:30
6e6cb298a3
fix: parse shards properly in enr config for non twn ( #2633 )
2024-04-26 17:51:52 +05:30
2a4c0f1543
chore: Enabling to use a full node for lightpush via rest api without lightpush client configured ( #2626 )
...
* Enabling to use a full node for lightpush via rest api without light push client configured
2024-04-26 12:42:47 +02:00
1d7ff2881b
chore(rln-relay): resultify rln-relay 1/n ( #2607 )
...
* chore(rln-relay): resultify rln-relay 1/n
* fix: v2 too
* fix: for static group manager
* fix: cleanup, make PR digestable
* fix: remove resultified retry wrapper
* fix: cleanup
* fix: cleanup
2024-04-26 11:53:58 +02:00
843fe217fa
fix: proto field numbers & status desc ( #2632 )
2024-04-25 15:43:21 -04:00
5f65565c85
fix: missing rate limit setting for legacy store protocol ( #2631 )
2024-04-25 17:51:34 +02:00
0b0fbfad5c
feat: store v3 ( #2431 )
2024-04-25 09:09:52 -04:00
9c9883a630
fix(rln-relay): enforce error callback to remove exception raised from retryWrapper ( #2622 )
2024-04-24 17:11:22 +02:00
b28207ab07
refactor: addition of waku_api/rest/builder.nim and reduce app.nim ( #2623 )
2024-04-24 15:59:50 +02:00
6d135b0d1b
chore: Separation of node health and initialization state from rln_relay ( #2612 )
...
* Separation of node health and initialization state from rln_relay status. Make (only) health endpoint avail early and install others in the last stage of node setup.
* Proper json report from /health, adjusted and fixed test, added convenient script for checking node health
* Stop wakunode2 if configured rest server cannot be started
* Fix wakuRlnRelay protocol existence check
* Fix typo
* Removed unused imports from touched files.
* Added missing /health test for all
2024-04-23 18:53:18 +02:00
1a23700d54
fix(rln-relay): increase retries for 1 minute recovery time ( #2614 )
2024-04-23 15:11:14 +02:00
6bc79bc739
chore: enabling rest api as default ( #2600 )
2024-04-23 10:23:13 +03:00
e61e4ff90a
fix: don't use WakuMessageSize in req/resp protocols ( #2601 )
...
* fix: don't use WakuMessageSize in req/resp protocols
2024-04-20 09:10:52 +05:30
1ba9df4be0
refactor: move app.nim and networks_config.nim to waku/factory ( #2608 )
2024-04-19 20:03:36 +02:00
768c61b114
fix: create options api for cors preflight request ( #2598 )
2024-04-18 18:29:50 +08:00
4a8e62ac5e
fix: node restart test issue ( #2576 )
...
* test_protocol.nim: enhance test reboot and connect
- Is not necessary to start the node if the switch object has been
already started
- Enable an existing "Relay can receive messages after reboot and
reconnect" test
- Explicit reconnect to peer in "Relay can receive messages after reboot
and reconnect" test
* tests/waku_relay/utils: avoid starting the proto again in newTestSwitch proc
With that, we avoid double start of the protocol.
* bump nim-libp2p
2024-04-18 11:20:39 +02:00
828583adc2
refactor: start moving discovery modules to waku/discovery ( #2587 )
2024-04-17 21:48:20 +02:00
a00f350cd1
feat: Added simple, configurable rate limit for lightpush and store-query ( #2390 )
...
* feat: Added simple, configurable rate limit for lightpush and store-query
Adjust lightpush rest response to rate limit, added tests ann some fixes
Add rest store query test for rate limit checks and proper error response
Update apps/wakunode2/external_config.nim
Move chronos/tokenbucket to nwaku codebasee with limited and fixed feature set
Add meterics counter to lightpush rate limits
Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
2024-04-15 15:28:35 +02:00
4117fe65b7
chore: adding migration script adding i_query index ( #2578 )
2024-04-15 12:57:35 +03:00
a04e0d9971
chore: bumping chronicles version ( #2583 )
2024-04-15 10:59:37 +03:00
480a62facd
fix(rln-relay): reduce sync time ( #2577 )
...
* fix(rln-relay): reduce sync time
* fix: add batch handling of futures to prevent over utilization of cpu
* fix: need to handle the futures on last iteration when it isnt full
2024-04-12 19:02:48 +03:00
d257855353
fix: rest store: content_topic -> contentTopic in the response ( #2584 )
2024-04-12 17:47:32 +02:00
f68ac79248
chore: bumping dependencies for 0.27.0 ( #2572 )
2024-04-09 11:17:46 +03:00
c87545d53d
fix: regex for rpc endpoint ( #2563 )
2024-04-02 15:14:55 +02:00
2aa835e3bf
feat(c-bindings): rln relay ( #2544 )
2024-03-27 10:08:53 -04:00
b5e4795f55
fix(rln): set a minimum epoch gap ( #2555 )
2024-03-27 11:36:14 +01:00
47ad0fb001
fix: fix regresion + remove deprecated flag ( #2556 )
2024-03-26 19:44:55 +01:00
65530264d5
feat(incentivization): add codec for eligibility proof and status ( #2419 )
...
* incentivization: add codec for eligibility proofs
* add codec for eligibility proof and eligibility status
* address minor comments
* make status code mandatory in eligibility status
2024-03-26 18:25:42 +01:00
e861317209
chore: remove deprecated legacy filter protocol ( #2507 )
...
* chore: remove deprecated legacy filter protocol
* fix: do not use legacy import in test
* fix: remove legacy test references
* fix: more test fixes, starting filter client
* fix: sigh. more references to remove.
* fix: fix dereferencing error
* fix: fix merge mess up
* fix: sigh. merge tool used tabs.
* fix: more peer manager tests needed fixing
---------
Co-authored-by: Hanno Cornelius <hanno@status.im>
Co-authored-by: Hanno Cornelius <68783915+jm-clius@users.noreply.github.com>
2024-03-25 18:07:56 +00:00
b5e2edb724
postgres_driver.nim: make sure the partition list ist loaded in correct order ( #2548 )
2024-03-25 18:02:07 +01:00
94ff5eab68
fix: remove subscription queue limit ( #2551 )
2024-03-25 10:33:01 -04:00
18a053591d
chore: switch wakuv2 to waku fleet ( #2519 )
...
See https://github.com/status-im/infra-nim-waku/issues/91
2024-03-20 16:28:00 +01:00
2173fe2243
fix: peer_manager - extend the number of connection requests to known peers ( #2534 )
...
* peer_manager: limit the max num out conns from within the conn loop
2024-03-19 19:07:03 +01:00
33774fada0
fix(2491): Fix metadata protocol disconnecting light nodes ( #2533 )
...
* Fix metadata protocol disconnecting light nodes.
* Implement test cases.
2024-03-19 16:18:52 +01:00
7aea2d4fc4
fix(rest): filter/v2/subscriptions response ( #2529 )
2024-03-18 18:21:06 -04:00
693a1778d1
chore: Better postgres duplicate insert ( #2535 )
...
* postgres_driver: add ON CONFLICT DO NOTHING in the insert statement
* test_driver_postgres: adapt test to ON CONFLICT DO NOTHING
The insert does not fail when duplicate, it returns a positive response
when doing 'put' of a duplicated row. The test is adapted so that
we just check that the number of messages doesn't grow after
trying to add a duplicated row.
2024-03-18 15:59:45 +01:00
3ba4378cbe
Generic re-style with nph 0.5.1 ( #2396 )
2024-03-16 00:08:47 +01:00
23a291b372
fix(store): retention policy regex ( #2532 )
2024-03-15 09:46:35 -04:00
0894f0cfea
test(discv5): Implement discv5 tests ( #2487 )
...
* Implement discv5 tests.
2024-03-14 19:01:13 +01:00
f436240d53
test(peer-exchange): Implement peer exchange tests ( #2464 )
...
* Implement peer exchange tests.
* Refactor, and remove duplicated tests.
* feat(wakunode): Resultify fetch peer exchange peers (#2486 )
2024-03-14 17:48:09 +01:00
2c1391d304
chore: add 150 kB to msg size histogram metric ( #2430 )
2024-03-14 12:38:02 +01:00
c734f60d06
feat(rest): add support to ephemeral field ( #2525 )
2024-03-13 08:49:21 -04:00
5a225809cd
fix: enable autosharding in any cluster ( #2505 )
2024-03-13 10:58:13 +01:00
8d7eb3a636
chore: factory cleanup ( #2523 )
2024-03-12 07:44:54 -06:00
505479b870
feat: archive update for store v3 ( #2451 )
2024-03-12 07:51:03 -04:00
059cb97518
chore(rln-relay-v2): wakunode testing + improvements ( #2501 )
...
* chore(rln-relay-v2): additional testing
* fix: bump librln to v0.4.2 for v2
* fix: catch possible error from the copyFrom
* ci: rename step title for rln-version
2024-03-12 16:20:30 +05:30
f9f92b7d2f
fix: introduce new field for shards in metadata protocol ( #2511 )
...
* fix: repeated fields are packed in proto3
* fix: add new field for shards in metadata protobuffers to avoid breaking change and deprecate original field
2024-03-11 10:08:46 -04:00
dcc88ee0b2
chore: migrating logic from wakunode2.nim to node_factory.nim ( #2504 )
2024-03-08 16:46:42 -06:00
b890f9c663
rest: rm openapi defs. they are in https://github.com/waku-org/waku-rest-api ( #2520 )
2024-03-08 19:43:55 +01:00
NagyZoltanPeter
Ivan FB
Darshan K
gabrielmer
richΛrd
Simon-Pierre Vivier
Akhil
Álex Cabeza Romero
Aaryamann Challani
Prem Chaitanya Prathi
kaichao
Alvaro Revuelta
Sergei Tikhomirov
Anton Iakimov
Vaclav Pavlin