* update RLN contract abi functions and procs
* Clean up debugging lines
* Use more descriptive object field names for MembershipInfo
* Fix formatting
* fix group_manager after rebase to use new contract method sig
* Fix linting for group_manager.nim
* Test idcommitment to BE and debug logs
* Improve IdCommitment logging
* Update all keystore credentials to use BE format
* Add workaround for groupmanager web3 eth_call
* Add await to sendEthCallWithChainID
* Add error handling for failed eth_call
* Improve error handling for eth_call workaround
* Revert keystore credentials back to using LE
* Update toRateCommitment proc to use LE instead of BE
* Add IdCommitment to calldata as BE
* feat: Update rln contract deployment and tests (#3408)
* update RLN contract abi functions and procs
* update waku-rlnv2-contract submodule commit to latest
* Add RlnV2 contract deployment using forge scripts
* Clean up output of forge script command, debug logs to trace, warn to error
* Move TestToken deployment to own proc
* first implementation of token minting and approval
* Update rln tests with usermessagelimit new minimum
* Clean up code and error handling
* Rework RLN tests WIP
* Fix RLN test for new contract
* RLN Tests updated
* Fix formatting
* Improve error logs
* Fix error message formatting
* Fix linting
* Add pnpm dependency installation for rln tests
* Update test dependencies in makefile
* Minor updates, error messages etc
* Code cleanup and change some debug logging to trace
* Improve handling of Result return value
* Use absolute path for waku-rlnv2-contract
* Simplify token approval and balance check
* Remove unused Anvil options
* Add additional checks for stopAnvil process
* Fix anvil process call to null
* Add lock to tests for rln_group_manager_onchain
* Debug for forge command
* Verify paths
* Install pnpm as global
* Cleanup anvil running procs
* Add check before installing anvil
* CLean up onchain group_manager
* Add proc to setup environment for contract deployer
* Refactoring and improved error handling
* Fix anvil install directory string
* Fix linting in test_range_split
* Add const for the contract address length
* Add separate checks for why Approval transaction fails
* Update RLN contract address and chainID for TWN
* Upgrade lpt to new config methods
* Make choice of legacy and v3 lightpush configurable on cli
* Adjust runner script to allow easy lightpush version selection
* Prepare selectable lightpush for infra env runs
* Fix misused result vs return
* Fixes and more explanatory comments added
* Fix ~pure virtual~ notice to =discard
* waku_relay protocol fix unsubscribe and remove topic validator
* simplify subscription and avoid unneeded code
* tests adaptations
* call wakuRelay.subscribe only in one place within waku_node
* properly pass userMessageLimit to OnchainGroupManager
* waku.nimble 2.2.4 Nim compiler
* rm stew/shims/net import
* change ValidIpAddress.init with parseIpAddress
* fix serialize for zerokit
* group_manager: separate if statements
* protocol_types: add encode UInt32 with zeros up to 32 bytes
* windows build: skip libunwind build and rm libunwind.a inlcusion step
* bump nph to overcome the compilation issues with 2.2.x
* bump nim-libp2p to v1.10.1
- Add more possible protocols to monitor
- Simplify protocol validation simple algorithm
- Properly pass the shard CLI parameter to the ENR info
- Mount metadata protocol
- Properly use of quit(QuitFailure)
* use of multiple Eth clients instead of just one
* config_chat2 enhance param comment
* group_manager: raise exception if could not connect to any of the eth clients
Split `WakuNodeConfig` object for better separation of concerns and to introduce a tree-like structure to configuration.
* fix: ensure twn cluster conf is still applied when clusterId=1
* test: remove usage of `WakuNodeConf`
* Remove macro, split builder files, remove wakunodeconf from tests
* rm network_conf_builder module as it is not used
---------
Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
Co-authored-by: Ivan Folgueira Bande <ivansete@status.im>
* add new unit test to validate that any peer can be retrieved
* add new discv5 test and better peer store management
* wakuPeerStore -> switch.peerStore
* simplify waku_peer_store, better logs and peer_manager enhancements
* Separate new lightpush protocol
New RPC defined
Rename al occurence of old lightpush to legacy lightpush, fix rest tests of lightpush
New lightpush protocol added back
Setup new lightpush protocol, mounting and rest api for it
modified: apps/chat2/chat2.nim
modified: tests/node/test_wakunode_lightpush.nim
modified: tests/node/test_wakunode_sharding.nim
modified: tests/test_peer_manager.nim
modified: tests/test_wakunode_lightpush.nim
renamed: tests/waku_lightpush/lightpush_utils.nim -> tests/waku_lightpush_legacy/lightpush_utils.nim
renamed: tests/waku_lightpush/test_all.nim -> tests/waku_lightpush_legacy/test_all.nim
renamed: tests/waku_lightpush/test_client.nim -> tests/waku_lightpush_legacy/test_client.nim
renamed: tests/waku_lightpush/test_ratelimit.nim -> tests/waku_lightpush_legacy/test_ratelimit.nim
modified: tests/wakunode_rest/test_all.nim
renamed: tests/wakunode_rest/test_rest_lightpush.nim -> tests/wakunode_rest/test_rest_lightpush_legacy.nim
modified: waku/factory/node_factory.nim
modified: waku/node/waku_node.nim
modified: waku/waku_api/rest/admin/handlers.nim
modified: waku/waku_api/rest/builder.nim
new file: waku/waku_api/rest/legacy_lightpush/client.nim
new file: waku/waku_api/rest/legacy_lightpush/handlers.nim
new file: waku/waku_api/rest/legacy_lightpush/types.nim
modified: waku/waku_api/rest/lightpush/client.nim
modified: waku/waku_api/rest/lightpush/handlers.nim
modified: waku/waku_api/rest/lightpush/types.nim
modified: waku/waku_core/codecs.nim
modified: waku/waku_lightpush.nim
modified: waku/waku_lightpush/callbacks.nim
modified: waku/waku_lightpush/client.nim
modified: waku/waku_lightpush/common.nim
modified: waku/waku_lightpush/protocol.nim
modified: waku/waku_lightpush/rpc.nim
modified: waku/waku_lightpush/rpc_codec.nim
modified: waku/waku_lightpush/self_req_handler.nim
new file: waku/waku_lightpush_legacy.nim
renamed: waku/waku_lightpush/README.md -> waku/waku_lightpush_legacy/README.md
new file: waku/waku_lightpush_legacy/callbacks.nim
new file: waku/waku_lightpush_legacy/client.nim
new file: waku/waku_lightpush_legacy/common.nim
new file: waku/waku_lightpush_legacy/protocol.nim
new file: waku/waku_lightpush_legacy/protocol_metrics.nim
new file: waku/waku_lightpush_legacy/rpc.nim
new file: waku/waku_lightpush_legacy/rpc_codec.nim
new file: waku/waku_lightpush_legacy/self_req_handler.nim
Adapt to non-invasive libp2p observers
cherry pick latest lightpush (v1) changes into legacy lightpush code after rebase to latest master
Fix vendor dependencies from origin/master after failed rebase of them
Adjust examples, test to new lightpush - keep using of legacy
Fixup error code mappings
Fix REST admin interface with distinct legacy and new lightpush
Fix lightpush v2 tests
* Utilize new publishEx interface of pubsub libp2p
* Adapt to latest libp2p pubslih design changes. publish returns an outcome as Result error.
* Fix review findings
* Fix tests, re-added lost one
* Fix rebase
* Apply suggestions from code review
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
* Addressing review comments
* Fix incentivization tests
* Fix build failed on libwaku
* Change new lightpush endpoint version to 3 instead of 2. Noticed that old and new lightpush metrics can cause trouble in monitoring dashboards so decided to give new name as v3 for the new lightpush metrics and change legacy ones back - temporarly till old lightpush will be decommissioned
* Fixing flaky test with rate limit timing
* Fixing logscope of lightpush and legacy lightpush
---------
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
* Fix receiver exit criteria, not let it wait forever in some cases, added a timely check from the last arrived message
* Extend dial and service usage failure metrics with agent string to reveal service nodes origins
* Adjusted infra testing content topic to be unique in the system
* Extend error logs with peer's agent string, fix exit criteria
* Add informative log for not waiting for more messages
* Add unknown as default for empty agent identifier
* better explain exit logic of receiver
* Address review comment - checking for last message arrival return Optional Moment instead of result - better explains what is happening.
* Adding lpt-runner script and assemble into liteprotocoltester image - to ease infra deployment
* Add supervisor that can run lpt continously in infra environment, infra.env defines defaults for run, in case image tag of lpt docker image is deploy it will build a specific image for infra deployment.
* Added message latency metrics
* DELAY_MESSAGES to MESSAGE_INTERVAL renaming
* Adjust name of START_PUBLISHING_AFTER
* Extend lpt readme with how to use make to build dockerized image and notice about infra deployment
* As fixed in discussion, we will control infra testing by built in predefined test setup
* Prevent peer switch in case using fixed service peers
* undo apt install libpcre (not circuit-relay related.)
* nat.nim: protect against possible exceptions when calling getExternalIP
* new external CLI argument, isRelayClient
* waku factory change to mount circuit hop proto by default
* waku_node: move autonat_service to a separate module
* Added bootstrap peer exchange discovery option for easy setup ltp
* Extended with PX discovery, auto-dial of PX cap peers, added switching service peers if failed with original
* Added peer-exchange, found capable peers test, metrics on peer stability and availability, dashboard adjustments
* Updated and actualized README.md for liteprotocoltester
* Created jenkinsfile for liteprotocoltester deployment
* Fixed dial exception during lightpublish
* Add configuration for requesting and testing peer exchange peers
* Extended examples added to Readme
* Added metrics port configurability
---------
Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
* Switch libnegentropy library build from shared to static linkage
* Update negentropy with -fPIC compile option that is necessary for libwaku build
* Bump waku-org/negentropy to the latest on master to incorporate merged static build of libnegentropy
* Added metrics to tests
Fix liteprotocoltester docker files with libnegentropy COPY
docker compose with waku-sim simulation now having test performance dashboard and localhost:3033
Mention dashboard in Readme
* Update apps/liteprotocoltester/statistics.nim
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
* indent fix, more stable finding of service/bootstrap nodes, pre-set for TWN
---------
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
* queue driver refactor (#2753)
* chore(archive): archive refactor (#2752)
* chore(archive): sqlite driver refactor (#2754)
* chore(archive): postgres driver refactor (#2755)
* chore(archive): renaming & copies (#2751)
* posgres legacy: stop using the storedAt field
* migration script 6: we still need the id column
The id column is needed because it contains the message digest
which is used in store v2, and we need to keep support to
store v2 for a while
* legacy archive: set target migration version to 6
* waku_node: try to use wakuLegacyArchive if wakuArchive is nil
* node_factory, waku_node: mount legacy and future store simultaneously
We want the nwaku node to simultaneously support store-v2
requests and store-v3 requests.
Only the legacy archive is in charge of archiving messages, and the
archived information is suitable to fulfill both store-v2 and
store-v3 needs.
* postgres_driver: adding temporary code until store-v2 is removed
---------
Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com>
Co-authored-by: gabrielmer <101006718+gabrielmer@users.noreply.github.com>
Co-authored-by: Ivan Folgueira Bande <ivansete@status.im>
* bump_dependencies.md: add nim-results dependency
* change imports stew/results to results
* switching to Nim 2.0.8
* waku.nimble: reflect the requirement nim 1.6.0 to 2.0.8
Adding --mm:refc as nim 2.0 enables a new garbage collector that we're
not yet ready to support
* adapt waku code to Nim 2.0
* gcsafe adaptations because Nim 2.0 is more strict
