* chore(rln-relay-v2): additional testing
* fix: bump librln to v0.4.2 for v2
* fix: catch possible error from the copyFrom
* ci: rename step title for rln-version
* fix: repeated fields are packed in proto3
* fix: add new field for shards in metadata protobuffers to avoid breaking change and deprecate original field
* postgres: first step to implement partition management
* postgres_driver: use of times.now().toTime().toUnix() instead of Moment.now()
* postgres migrations: set new version to 2
* test_driver_postgres: use of assert instead of require and avoid using times.now()
* postgres_driver: better implementation of the reset method with partitions
* Remove createMessageTable, init, and deleteMessageTable procs
* postgres: ensure we use the version 15.4 in tests
* postgres_driver.nim: enhance debug logs partition addition
* ci.yml: ensure logs are printed without colors
* postgres_driver: starting the loop factory in an asynchronous task
* postgres_driver: log partition name and size when removing a partition
* Add postgres_driver/migrations.nim
* Postgres and archive logic adaptation to the migration implementation
* libwaku: adapt node_lifecycle_request.nim to migration refactoring
* test_app.nim: add more detail for test that only fails in CI
* postgres migrations: store the migration scripts inside the resulting wakunode binary instead of external .sql files.
* Add allowOrigin configuration for wakunode and WakuRestServer
Update nim-presto to the latest master that contains middleware support
Rework Rest Server in waku to utilize chronos' and presto's new middleware design and added proper CORS handling.
Added cors tests and fixes
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
* new proc to foster different size retention policy implementations
The new proc, decreaseDatabaseSize, will have different implementations
per each driver. For example, in future commits we will implement a size
retention policy thanks to partitions management, in Postgres.
* RetentionPolicy: use of new instead of init for ref object types
* waku_archive: fix signatures in decreaseDatabaseSize methods
* retention_policy_size: minor cleanup of comments and imports
* chore(rln-relay-v2): add tests for serde
* fix: call isOk fn instead of prop access
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
* fix: make cast more explicit
---------
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
* Add ligthpush payload tests.
* Add end to end lightpush tests.
* updating vendor/nim-unittest2 to protect against core dump issue
* Enable "Valid Payload Sizes" test again
---------
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
A node that handles REST-Store requests normally acts as a
Store-client and therefore it retrieved the messages from another
Store-node.
With these changes, we allow a node with Store mounted, to retrieve
its messages. In other words, the node can act as a Store-server of
its messages.
* test_rest_store.nim: add a new test to validate that the self-node can
retrieve its messages to the REST client.
* rest/store/client.nim: add new proc to allow making a GET store
request without peerAddr.
* rest/store/handle.nim: add logic to handle requests that don't
provide peerAddr but the self/local node has Store mounted. In this case,
the self/local node will retrieve its locally stored messages.
* waku_store/self_req_handler.nim: logic to handle "store" requests
allowing the REST-store node to act as a Store-server node. The
'self_req_handler.nim' helps to bypass the store protocol and directly
retrieve the messages from the local/self node. I added this logic in
a separate file from 'protocol.nim' because it doesn't participate in
any libp2p communication.
* waku_store/protocol.nim: make 'queryHandler' attribute public so that
it can be used from the 'self_req_handler.nim' module.
* Refactor of FilterV2 subscription handling and maintenance with addition subscription time-to-live support.
Fixed all tests and reworked where subscription handling changes needed it.
Adapted REST API /admin filter subscription retrieve to new filter subscription structure.
* Fix tests and PR comments
* Added filter v2 subscription timeout tests and fixed
* Fix review comments and suggestions. No functional change.
* Remove leftover echoes from test_rest_admin
* Fix failed legacy filter tests due to separation of mounting the filters.
* Small fixes, fix naming typo, removed duplicated checks in test
The following vendors have changes but are not being updated for
the reason explained.
nim-web3: not updated because unit tests started to fail and no
straightforward solution found.
nim-toml-serialization: not updated because it introduced a breaking
change on how the --config-file attribute is parsed. The array
attributes now need a comma. For example, the following attribute
from within the config file:
pubsub-topic = [ "/waku/2/default-waku/proto" "/waku/2/testing-store" ]
... should be converted to:
pubsub-topic = [ "/waku/2/default-waku/proto", "/waku/2/testing-store" ]
and we cannot accept that breaking change
* message.nim: set max message size to 150KiB according to spec
Using KiB instead of KB because that seems more aligned with
the actual default defined in nim-libp2p (1024 * 1024)
Spec details: https://rfc.vac.dev/spec/64/#message-size
* test_protocol.nim: align test to current WakuMessage limit
* test_waku_client.nim: adapt test to MaxWakuMessageSize change
* make maxMessageSize configurable for wakunode2
* wakunode2 app now accepts max-num-bytes-msg-size with KiB, KB, or B units
* testlib/wakunode.nim: set maxMessageSize: "1024 KiB"
* test_waku_client.nim: remove duplicate check in "Valid Payload Sizes"
* set DefaultMaxWakuMessageSizeStr as the only source of truth
* external_config.nim: rename max-num-bytes-msg-size -> max-msg-size
The "ip colocation" concept refers to the maximum allowed peers
from the same IP address. For example, we allow disabling this limit when the
node works behind a reverse proxy.
This reverts commit dba9820c1f.
We need to revert this commit because
the waku-simulator stopped working. i.e. the nodes couldn't establish
connections among them: 054ba9e33f
Also, the following js-waku test fails due to this commit:
"same cluster, different shard: nodes connect"
* waku_lightpush/protocol.nim: minor changes to make it compile after revert
* archive: move error to trace level when insert row fails
That is helpful to prevent the node to spam the logs when it shares
connection to the same Postgres database with other nodes, in
which case the following log appears too much:
topics="waku archive" tid=1 file=archive.nim:113 err="error in
runStmt: error in dbConnQueryPrepared calling waitQueryToFinish: error
in query: ERROR: duplicate key value violates unique constraint
"messageindex" DETAIL: Key
(messagehash)=(88f4ee115eef6f233a7dceaf975f03946e18666adda877e38d61be98add934e8)
already exists. "
Before this commit, the following execution of a prepared statement
returned nothing even though the database had 2 rows to be returned:
nwaku-db-1 | 2023-12-14 12:55:17.575 UTC [73] LOG: execute SelectWithoutCursorAsc: SELECT storedAt, contentTopic, payload, pubsubTopic, version, timestamp, id FROM messages
nwaku-db-1 | WHERE contentTopic IN ($1) AND
nwaku-db-1 | pubsubTopic = $2 AND
nwaku-db-1 | storedAt >= $3 AND
nwaku-db-1 | storedAt <= $4
nwaku-db-1 | ORDER BY storedAt ASC LIMIT $5;
nwaku-db-1 | 2023-12-14 12:55:17.575 UTC [73] DETAIL: parameters: $1 =
'my/ctopic/1,my/ctopic/2', $2 = '/waku/2/default-waku/proto', $3 = '1702552968570786800', $4 = '1702552968585347557', $5 = '101'
The reason why it is not returning anything is that the 'IN' statement doesn't work when using prepared statements with multiple items. It only works when the 'IN' content, i.e. $1, contains one single item.
* on_chain/group_manager: use .async: (raises:[Exception]).
* bump nim-dnsdisc
* update nim-chronos to the latest state
* chat2.nim: catch any possible exception when stopping
* chat2bridge.nim: make it to compile after vendor bump
* ValidIpAddress (deprecated) -> IpAddress
* vendor/nim-libp2p additional bump
* libwaku: adapt to vendor bump
* testlib/wakunode.nim: adapt to vendor bump (ValidIpAddress -> IpAddress)
* waku_node: avoid throwing any exception from stop*(node: WakuNode)
* test_confutils_envvar.nim: ValidIpAddress -> IpAddress
* test_jsonrpc_store: capture exception
* test_rln*: handling exceptions
* adaptation to make test_rln_* to work properly
* signature enhancement of group_manager methods
Álex Cabeza Romero
Vaclav Pavlin
richΛrd
Alvaro Revuelta
gabrielmer
Simon-Pierre Vivier
Aaryamann Challani
Ivan FB
NagyZoltanPeter
Benjamin Arntzen
Hanno Cornelius
Prem Chaitanya Prathi
joao
Sasha
omahs
Abhimanyu