1405 Commits

Author SHA1 Message Date
gabrielmer
a5d74095c4 chore: changing digest and hash log format from bytes to hex (#2363) 2024-01-25 16:03:48 +01:00
Prem Chaitanya Prathi
bff4942361 chore: log messageHash for lightpush request that helps in debugging (#2366) 2024-01-24 19:20:21 +05:30
Aaryamann Challani
2065f3db3c chore(rln-relay): remove websocket from OnchainGroupManager (#2364)
* chore(rln-relay): remove websocket from OnchainGroupManager

* fix: swap ws for http
2024-01-23 23:22:45 +05:30
Hanno Cornelius
105b3c2089 fix: Fix test for filter client receiving messages after restart (#2360) 2024-01-19 13:05:06 +02:00
gabrielmer
c573fd0538 chore: improve POST /relay/v1/auto/messages/{topic} error handling (#2339) 2024-01-18 13:49:13 +01:00
gabrielmer
e1f97fff82 fix: making filter admin data test order independent (#2355) 2024-01-17 09:42:22 +01:00
NagyZoltanPeter
f048babdc4 chore: Refactor of FilterV2 subscription management with Time-to-live maintenance (#2341)
* Refactor of FilterV2 subscription handling and maintenance with addition subscription time-to-live support.
Fixed all tests and reworked where subscription handling changes needed it.
Adapted REST API /admin filter subscription retrieve to new filter subscription structure.

* Fix tests and PR comments

* Added filter v2 subscription timeout tests and fixed

* Fix review comments and suggestions. No functional change.

* Remove leftover echoes from test_rest_admin

* Fix failed legacy filter tests due to separation of mounting the filters.

* Small fixes, fix naming typo, removed duplicated checks in test
2024-01-16 17:27:40 +01:00
gabrielmer
ce32156a8b chore: Bump nim-dnsdisc (#2354)
* chore: bump `vendor/nim-dnsdisc`

* chore: Update import path of dnsdisc

---------

Co-authored-by: Emil Ivanichkov <emil.ivanichkov@gmail.com>
2024-01-15 16:54:02 +02:00
Ivan FB
b623ace3e5 docs: postgres-adoption.md add metadata title, description, and better first-readable-title (#2346) 2024-01-12 16:37:32 +01:00
Simon-Pierre Vivier
2381de7c90 chore: update CHANGELOG.md for v0.24.0 (#2347) 2024-01-12 07:30:29 -05:00
vuittont60
cc8045d860 docs: fix typo (#2348) 2024-01-11 15:38:56 +01:00
Ivan FB
b3ec2e9c00 chore: Update CHANGELOG.md to reflect bug fix for issue #2317 (#2340) in v0.23.1 2024-01-10 09:45:26 +01:00
gabrielmer
2b2eec9535 feat: adding filter data admin endpoint (REST) (#2314) 2024-01-09 11:42:29 +01:00
NagyZoltanPeter
25cd5e2ff7 test(peer-connection-managenent): Functional Tests (#2321) @ b16e20e48 introduced build error through ambigous function call, testwaku build failed on master (#2337) 2024-01-08 16:45:03 +01:00
Simon-Pierre Vivier
f97ad7a12c docs: update after release steps (#2336) 2024-01-08 08:11:06 -05:00
joao
9d2971c379 docs: Fix Typos Across Various Documentation Files (#2310) 2024-01-08 13:13:34 +01:00
Álex Cabeza Romero
b16e20e48e test(peer-connection-managenent): Functional Tests (#2321)
* Add simple mock mechanism.
* Implement migrations tests.
* Implement peer storage tests.
* Add simple protobuf serialisation testcase.
2024-01-05 14:49:04 +01:00
Emil Ivanichkov
f2a2f960ae fix: Set record to the Waku Node Builder in the examples as it is required (#2328) 2024-01-05 10:00:41 +01:00
Simon-Pierre Vivier
b5f9987efa fix(discv5): add bootnode filter exception (#2267) 2024-01-04 16:39:03 -05:00
Ivan FB
1900118f3b bump vendors for 0.24.0 (#2333)
The following vendors have changes but are not being updated for
the reason explained.

nim-web3: not updated because unit tests started to fail and no
straightforward solution found.

nim-toml-serialization: not updated because it introduced a breaking
change on how the --config-file attribute is parsed. The array
attributes now need a comma. For example, the following attribute
from within the config file:

pubsub-topic = [ "/waku/2/default-waku/proto" "/waku/2/testing-store" ]

... should be converted to:

pubsub-topic = [ "/waku/2/default-waku/proto", "/waku/2/testing-store" ]

and we cannot accept that breaking change
2024-01-04 17:35:00 +01:00
Álex Cabeza Romero
09129f56ab test(autosharding): Functional Tests (#2318)
* Implement autosharding tests.
2024-01-04 16:26:27 +01:00
Ivan FB
7708082809 docs: add benchmar around postgres adoption (#2316) 2024-01-03 13:13:47 +01:00
Ivan FB
c738841d43 chore: message.nim - set max message size to 150KiB according to spec (#2298)
* message.nim: set max message size to 150KiB according to spec

Using KiB instead of KB because that seems more aligned with
the actual default defined in nim-libp2p (1024 * 1024)

Spec details: https://rfc.vac.dev/spec/64/#message-size

* test_protocol.nim: align test to current WakuMessage limit
* test_waku_client.nim: adapt test to MaxWakuMessageSize change
* make maxMessageSize configurable for wakunode2
* wakunode2 app now accepts max-num-bytes-msg-size with KiB, KB, or B units
* testlib/wakunode.nim: set maxMessageSize: "1024 KiB"
* test_waku_client.nim: remove duplicate check in "Valid Payload Sizes"
* set DefaultMaxWakuMessageSizeStr as the only source of truth
* external_config.nim: rename max-num-bytes-msg-size -> max-msg-size
2024-01-03 13:11:50 +01:00
Ivan FB
ce567acb62 ip colocation is parameterizable. If set to 0, it is disabled (#2323)
The "ip colocation" concept refers to the maximum allowed peers
from the same IP address. For example, we allow disabling this limit when the
node works behind a reverse proxy.
2024-01-02 14:01:18 +01:00
Ivan FB
0999fa44fe CHANGELOG.md for 0.23.0 (#2309) 2023-12-20 15:48:59 +01:00
Ivan FB
cb59623466 fix: Revert "feat: shard aware peer management (#2151)" (#2312)
This reverts commit dc1d6ce4bf7390e23b73d96634ff87ca9341e129.

We need to revert this commit because
the waku-simulator stopped working. i.e. the nodes couldn't establish
connections among them: 054ba9e33f

Also, the following js-waku test fails due to this commit:
"same cluster, different shard: nodes connect"

* waku_lightpush/protocol.nim: minor changes to make it compile after revert
2023-12-20 15:23:41 +01:00
gabrielmer
29c182195d fix: setting connectivity loop interval to 15 seconds (#2307) 2023-12-20 09:38:14 +01:00
Álex Cabeza Romero
91c402b1cc test(store): Implement store tests (#2235, 2240)
* Implement store tests
2023-12-19 15:38:43 +01:00
Álex Cabeza Romero
b8b32eb857 refactor(store): HistoryQuery.direction (#2263)
* Fix issue with default history query ascending value in serde operations: Should use the same value.
* Update direction types to PagingDirection.
2023-12-19 15:10:27 +01:00
Ivan FB
88f2a9f89b test_driver_postgres: enhance test coverage, multiple and single topic (#2301)
Co-authored-by: Abhimanyu <ABresting@users.noreply.github.com>
2023-12-19 10:41:50 +01:00
Ivan FB
51199214f9 chore: examples/nodejs - adapt code to latest callback and ctx/userData definitions (#2281) 2023-12-18 23:16:54 +01:00
Ivan FB
b34f5d3117 chore: archive - move error to trace level when insert row fails (#2283)
* archive: move error to trace level when insert row fails

That is helpful to prevent the node to spam the logs when it shares
connection to the same Postgres database with other nodes, in
which case the following log appears too much:

topics="waku archive" tid=1 file=archive.nim:113 err="error in
runStmt: error in dbConnQueryPrepared calling waitQueryToFinish: error
in query: ERROR: duplicate key value violates unique constraint
"messageindex" DETAIL: Key
(messagehash)=(88f4ee115eef6f233a7dceaf975f03946e18666adda877e38d61be98add934e8)
already exists. "
2023-12-15 18:58:35 +01:00
gabrielmer
e19996951b chore: including content topics on FilterSubscribeRequest logs (#2295) 2023-12-15 14:18:12 +01:00
Ivan FB
45bc8add32 relay: add trace logs in case of msg validation rejection (#2285) 2023-12-15 13:34:30 +01:00
Ivan FB
d7e8477b3f libwaku: avoid using waku_init. Only use waku_new to create node and context (#2282) 2023-12-15 13:32:12 +01:00
Alvaro Revuelta
43ffd1371b fix: make rln rate limit spec compliant (#2294) 2023-12-15 10:26:17 +01:00
Ivan FB
2b434d7e8d bug fix: update num-msgs archive metrics every minute and not only at the beginning (#2287) 2023-12-14 17:00:13 +01:00
Ivan FB
f191b79f28 postgres_driver.nim: restrict getMessages prepared stmt to query with 1 content topic (#2296)
Before this commit, the following execution of a prepared statement
returned nothing even though the database had 2 rows to be returned:

nwaku-db-1  | 2023-12-14 12:55:17.575 UTC [73] LOG:  execute SelectWithoutCursorAsc: SELECT storedAt, contentTopic, payload, pubsubTopic, version, timestamp, id FROM messages
nwaku-db-1  | 	    WHERE contentTopic IN ($1) AND
nwaku-db-1  | 	          pubsubTopic = $2 AND
nwaku-db-1  | 	          storedAt >= $3 AND
nwaku-db-1  | 	          storedAt <= $4
nwaku-db-1  | 	    ORDER BY storedAt ASC LIMIT $5;
nwaku-db-1  | 2023-12-14 12:55:17.575 UTC [73] DETAIL:  parameters: $1 =
'my/ctopic/1,my/ctopic/2', $2 = '/waku/2/default-waku/proto', $3 = '1702552968570786800', $4 = '1702552968585347557', $5 = '101'

The reason why it is not returning anything is that the 'IN' statement doesn't work when using prepared statements with multiple items. It only works when the 'IN' content, i.e. $1, contains one single item.
2023-12-14 15:37:12 +01:00
Ivan FB
7c692cc313 chore: vendor bump for 0.23.0 (#2274)
* on_chain/group_manager: use .async: (raises:[Exception]).
* bump nim-dnsdisc
* update nim-chronos to the latest state
* chat2.nim: catch any possible exception when stopping
* chat2bridge.nim: make it to compile after vendor bump
* ValidIpAddress (deprecated) -> IpAddress
* vendor/nim-libp2p additional bump
* libwaku: adapt to vendor bump
* testlib/wakunode.nim: adapt to vendor bump (ValidIpAddress -> IpAddress)
* waku_node: avoid throwing any exception from stop*(node: WakuNode)
* test_confutils_envvar.nim: ValidIpAddress -> IpAddress
* test_jsonrpc_store: capture exception
* test_rln*: handling exceptions
* adaptation to make test_rln_* to work properly
* signature enhancement of group_manager methods
2023-12-14 07:16:39 +01:00
Ivan FB
ac3a3737de chore: peer_manager.nim - reduce logs from debug to trace (#2279) 2023-12-12 16:00:18 +01:00
Aaryamann Challani
f9bd39ea66 fix(rln-relay): graceful retries on rpc calls (#2250)
* fix(rln-relay): graceful retries on rpc calls

* fix: missing file
2023-12-11 14:59:16 +05:30
Ivan FB
e6fe6df52e archive: simplify and enhance async retention policy application (#2278)
* Avoid using timer and just use an infinite async loop that can be
cancelled at any time.
2023-12-11 08:50:40 +01:00
Ivan FB
de11e19a9f waku_thread_response.nim: use correct alloc() proc to allocate response correctly (#2277) 2023-12-11 08:50:03 +01:00
Ivan FB
2a03416f47 chore: Cbindings allow mounting the Store protocol from libwaku (#2276)
* libwaku: add changes to mount store in self-node
* libwaku: remove unnecessary code for store
2023-12-11 08:49:13 +01:00
Simon-Pierre Vivier
c2ded25e11 added sharded peer store pruning (#2167) 2023-12-07 07:21:18 -05:00
Simon-Pierre Vivier
dc1d6ce4bf feat: shard aware peer management (#2151) 2023-12-07 06:48:28 -05:00
Ivan FB
32d5c24d29 fix: add protection in rest service to always publish with timestamp if user doesn't provide it (#2261) 2023-12-06 14:02:21 +01:00
Sasha
0f3cf94ac2 fix: remove trailing commas from keystore json (#2200)
* fix: remove trailing commas from keystore json

* keyfile.nim: try a different Json formatting approach

* build keystore

* address comment

---------

Co-authored-by: Ivan Folgueira Bande <ivansete@status.im>
2023-12-01 12:57:19 +01:00
Vaclav Pavlin
a0eb05ea19 fix(dockerfile): update dockerignore and base image (#2262) 2023-12-01 11:35:50 +01:00
Ivan FB
75296f055c waku_store/common.nim: correct ret code in PEER_DIAL_FAILURE (#2260) 2023-11-30 16:42:58 +01:00