diff --git a/docs/operators/README.md b/docs/operators/README.md index 05535d222..af42d02a8 100644 --- a/docs/operators/README.md +++ b/docs/operators/README.md @@ -1,6 +1,6 @@ # The nwaku guide for operators -*If you're eager to get started, check out our [quickstart guide](./quickstart.md).* +*If you're eager to get started, check out our [quickstart guide](./quickstart.md) for typical configurations or [step-by-step overview](./overview.md) for newcomers.* Nwaku is a client implementation in Nim of the [Waku v2 family of protocols](https://rfc.vac.dev/spec/10/) for peer-to-peer communication. The protocols are designed to be secure, privacy-preserving, censorship-resistant and able to run in resource restricted environments. diff --git a/docs/operators/docker-quickstart.md b/docs/operators/docker-quickstart.md new file mode 100644 index 000000000..b10cf36f7 --- /dev/null +++ b/docs/operators/docker-quickstart.md @@ -0,0 +1,79 @@ +# Quickstart: running nwaku in a Docker container + +This guide explains how to run a nwaku node in a Docker container. + +## Prerequisites + +Make sure you have Docker installed. +Installation instructions for different platforms can be found in the [Docker docs](https://docs.docker.com/engine/install/). + +For example, to use Docker's convenience script for installation: + +```bash +curl -fsSL https://get.docker.com -o get-docker.sh +sudo sh get-docker.sh +``` + +## Step 1: Get Docker image + +Nwaku Docker images are published to the Docker Hub public registry under [`statusteam/nim-waku`](https://hub.docker.com/r/statusteam/nim-waku). +For specific releases the published images are tagged with the release version, e.g. [`statusteam/nim-waku:v0.12.0`](https://hub.docker.com/layers/statusteam/nim-waku/v0.12.0/images/sha256-7b5bcd3f8084218b411e627ab7864ce999d7b49e37ad1d0b1089b18cbf4b79d4?context=explore). +Images are also published for each commit to the `master` branch in the [nwaku repo](https://github.com/status-im/nwaku/commits/master) +and tagged with the corresponding commit hash. +See [`statusteam/nim-waku`](https://hub.docker.com/r/statusteam/nim-waku/tags) on Docker Hub for a full list of available tags. + +To pull the image of your choice, use + +```bash +docker pull statusteam/nim-waku:v0.12.0 # or, whichever tag you prefer in the format statusteam/nim-waku:[tag] +``` + +You can also build the Docker image locally using + +```bash +git clone --recurse-submodules https://github.com/status-im/nwaku +cd nwaku +docker build -t statusteam/nim-waku:latest . +``` + +## Step 2: Run + +To run nwaku in a new Docker container, +use the following command: + +```bash +docker run [OPTIONS] IMAGE [ARG...] +``` + +where `OPTIONS` are your selected Docker options, +`IMAGE` the image and tag you pulled from the registry or built in Step 1 +and `ARG...` the list of nwaku arguments for your [chosen nwaku configuration](./how-to/configure.md). + +For Docker options we recommend explicit port mappings (`-p`) at least +for your exposed libp2p listening ports +and any discovery ports (e.g. the Waku discv5 port) that must be reachable from outside the host. + +As an example, consider the following command to run nwaku in a Docker container with the most typical configuration: + +```bash +docker run -i -t -p 60000:60000 -p 9000:9000/udp statusteam/nim-waku:v0.12.0 \ + --dns-discovery:true \ + --dns-discovery-url:enrtree://AOGECG2SPND25EEFMAJ5WF3KSGJNSGV356DSTL2YVLLZWIV6SAYBM@prod.waku.nodes.status.im \ + --discv5-discovery \ + --nat:extip:[yourpublicip] # or, if you are behind a nat: --nat=any +``` + +This runs nwaku in a new container from the `statusteam/nim-waku:v0.12.0` image, +connects to `wakuv2.prod` as bootstrap fleet and +enables [Waku Discovery v5](https://rfc.vac.dev/spec/33/) for ambient peer discovery, +while mapping the default libp2p listening port (`60000`) +and default discv5 UDP port (`9000`) to the host. + +> **Tip:** The `docker run` command will pull the specified image from Docker Hub if it's not yet available locally, +so it's possible to skip Step 1 and pull the image from your configured registry automatically when running. + +If you've used the `-i` and `-t` Docker options when running the new container, +the `run` command would have allocated an interactive terminal +where you'll see the `stdout` logs from the running nwaku process. +To detach gracefully from the running container, +use `Ctrl-P` followed by `Ctrl-Q`. diff --git a/docs/operators/overview.md b/docs/operators/overview.md new file mode 100644 index 000000000..9038264a7 --- /dev/null +++ b/docs/operators/overview.md @@ -0,0 +1,47 @@ +# Overview: running a nwaku node + +This guide provides on overview for newcomers +on how to build and run a nwaku node +for the most common use cases. +For a more advanced configuration see our [configuration guides](./how-to/configure.md) + +To set up a nwaku node on a DigitalOcean droplet, +refer to our [quickstart guide for droplets](./droplet-quickstart.md). +If you prefer running nwaku in Docker container, +see our [Docker guide](./docker-quickstart.md). + +## 1. Build + +[Build the nwaku node](./how-to/build.md) +or download a precompiled binary from our [releases page](https://github.com/status-im/nwaku/releases). +Docker images are published to [statusteam/nim-waku](https://hub.docker.com/r/statusteam/nim-waku/tags) on Docker Hub. +See our [Docker quickstart guide](./docker-quickstart.md) to run nwaku in a Docker container. + +## 2. Run + +[Run the nwaku node](./how-to/run.md) using a default or common configuration +or [configure](./how-to/configure.md) the node for more advanced use cases. + +[Connect](./how-to/connect.md) the nwaku node to other peers to start communicating. + +## 3. Interact + +A running nwaku node can be interacted with using the [Waku v2 JSON RPC API](https://rfc.vac.dev/spec/16/). + +> **Note:** Private and Admin API functionality are disabled by default. +To configure a nwaku node with these enabled, +use the `--rpc-admin:true` and `--rpc-private:true` CLI options. + +```bash +curl -d '{"jsonrpc":"2.0","method":"get_waku_v2_debug_v1_info","params":[],"id":1}' -H 'Content-Type: application/json' localhost:8546 -s | jq +``` + + +Or using the [Waku v2 HTTP REST API](../api/v2/rest-api.md): + +> **Note:** REST API functionality is in ALPHA and therefore it is disabled by default. To configure a nwaku node with this enabled, use the `--rest:true` CLI option. + + +```bash +curl http://localhost:8546/debug/v1/info -s | jq +``` diff --git a/docs/operators/quickstart.md b/docs/operators/quickstart.md index 118917e84..360b63af9 100644 --- a/docs/operators/quickstart.md +++ b/docs/operators/quickstart.md @@ -1,44 +1,46 @@ # Quickstart: running a nwaku node -This guide explains how to build and run a nwaku node -for the most common use cases. -For a more advanced configuration see our [configuration guides](./how-to/configure.md) +This guide helps you run a nwaku node with typical configuration. +It connects your node to the `wakuv2.prod` fleet for bootstrapping +and enables discovery v5 for continuous peer discovery. +Only [`relay`](https://rfc.vac.dev/spec/11/) protocol is enabled. +For a more comprehensive overview, +see our [step-by-step guide](./overview.md). -To quickly set up a nwaku node on DigitalOcean, refer to this [guide](./droplet-quickstart.md) +## Option 1: run nwaku binary -## 1. Build - -[Build the nwaku node](./how-to/build.md) -or download a precompiled binary from our [releases page](https://github.com/status-im/nwaku/releases). -Docker images are published to [statusteam/nim-waku](https://hub.docker.com/r/statusteam/nim-waku/tags) on DockerHub. - - - -## 2. Run - -[Run the nwaku node](./how-to/run.md) using a default or common configuration -or [configure](./how-to/configure.md) the node for more advanced use cases. - -[Connect](./how-to/connect.md) the nwaku node to other peers to start communicating. - -## 3. Interact - -A running nwaku node can be interacted with using the [Waku v2 JSON RPC API](https://rfc.vac.dev/spec/16/). - -> **Note:** Private and Admin API functionality are disabled by default. -To configure a nwaku node with these enabled, -use the `--rpc-admin:true` and `--rpc-private:true` CLI options. +*Prerequisites are the usual developer tools, +such as a C compiler, Make, Bash and Git.* ```bash -curl -d '{"jsonrpc":"2.0","method":"get_waku_v2_debug_v1_info","params":[],"id":1}' -H 'Content-Type: application/json' localhost:8546 -s | jq +git clone --recurse-submodules https://github.com/status-im/nwaku +cd nwaku +make wakunode2 +./build/wakunode2 \ + --dns-discovery:true \ + --dns-discovery-url:enrtree://AOGECG2SPND25EEFMAJ5WF3KSGJNSGV356DSTL2YVLLZWIV6SAYBM@prod.waku.nodes.status.im \ + --discv5-discovery \ + --nat=extip:[yourpublicip] # or, if you are behind a nat: --nat=any ``` +## Option 2: run nwaku in a Docker container -Or using the [Waku v2 HTTP REST API](../api/v2/rest-api.md): - -> **Note:** REST API functionality is in ALPHA and therefore it is disabled by default. To configure a nwaku node with this enabled, use the `--rest:true` CLI option. - +*Prerequisite is a [Docker installation](./docker-quickstart.md#prerequisites).* ```bash -curl http://localhost:8546/debug/v1/info -s | jq +docker run -i -t -p 60000:60000 -p 9000:9000/udp \ + statusteam/nim-waku:v0.12.0 \ # or, the image:tag of your choice + --dns-discovery:true \ + --dns-discovery-url:enrtree://AOGECG2SPND25EEFMAJ5WF3KSGJNSGV356DSTL2YVLLZWIV6SAYBM@prod.waku.nodes.status.im \ + --discv5-discovery \ + --nat:extip:[yourpublicip] # or, if you are behind a nat: --nat=any +``` + +## Tips and tricks + +To find the public IP of your host, +you can use + +```bash +dig TXT +short o-o.myaddr.l.google.com @ns1.google.com | awk -F'"' '{ print $2}' ``` diff --git a/tests/v2/test_message_cache.nim b/tests/v2/test_message_cache.nim index d69f8dbf4..fecbcba08 100644 --- a/tests/v2/test_message_cache.nim +++ b/tests/v2/test_message_cache.nim @@ -6,16 +6,10 @@ import chronicles import ../../waku/v2/protocol/waku_message, - ../../waku/v2/node/message_cache + ../../waku/v2/node/message_cache, + ./testlib/common -proc fakeWakuMessage(payload = toBytes("TEST"), contentTopic = "test"): WakuMessage = - WakuMessage( - payload: payload, - contentTopic: contentTopic, - version: 1, - timestamp: 2022 - ) type PubsubTopicString = string diff --git a/tests/v2/test_message_store_queue_index.nim b/tests/v2/test_message_store_queue_index.nim index c5568d1cf..3d2142efa 100644 --- a/tests/v2/test_message_store_queue_index.nim +++ b/tests/v2/test_message_store_queue_index.nim @@ -8,12 +8,8 @@ import import ../../waku/v2/protocol/waku_message, ../../waku/v2/utils/time, - ../../waku/v2/node/storage/message/queue_store/index - - -const - DefaultPubsubTopic = "/waku/2/default-waku/proto" - DefaultContentTopic = ContentTopic("/waku/2/default-content/proto") + ../../waku/v2/node/storage/message/queue_store/index, + ./testlib/common ## Helpers diff --git a/tests/v2/test_message_store_queue_pagination.nim b/tests/v2/test_message_store_queue_pagination.nim index 3343f8973..c10561d79 100644 --- a/tests/v2/test_message_store_queue_pagination.nim +++ b/tests/v2/test_message_store_queue_pagination.nim @@ -1,7 +1,7 @@ {.used.} import - std/[options, sequtils, times, algorithm], + std/[options, sequtils, algorithm], testutils/unittests, nimcrypto/sha2, libp2p/protobuf/minprotobuf @@ -9,12 +9,8 @@ import ../../waku/v2/node/storage/message/waku_store_queue, ../../waku/v2/protocol/waku_store, ../../waku/v2/protocol/waku_message, - ../../waku/v2/utils/time - - -const - DefaultPubsubTopic = "/waku/2/default-waku/proto" - DefaultContentTopic = ContentTopic("/waku/2/default-content/proto") + ../../waku/v2/utils/time, + ./testlib/common proc getTestStoreQueue(numMessages: int): StoreQueueRef = @@ -36,12 +32,6 @@ proc getTestStoreQueue(numMessages: int): StoreQueueRef = return testStoreQueue -proc now(): Timestamp = - getNanosecondTime(getTime().toUnixFloat()) - -proc ts(offset=0, origin=now()): Timestamp = - origin + getNanosecondTime(offset) - suite "Queue store - pagination": test "Forward pagination test": diff --git a/tests/v2/test_message_store_sqlite.nim b/tests/v2/test_message_store_sqlite.nim index f47688661..ab3f2c28f 100644 --- a/tests/v2/test_message_store_sqlite.nim +++ b/tests/v2/test_message_store_sqlite.nim @@ -13,35 +13,13 @@ import ../../waku/v2/protocol/waku_message, ../../waku/v2/protocol/waku_store/pagination, ../../waku/v2/utils/time, - ./utils - - -const - DefaultPubsubTopic = "/waku/2/default-waku/proto" - DefaultContentTopic = ContentTopic("/waku/2/default-content/proto") + ./utils, + ./testlib/common proc newTestDatabase(): SqliteDatabase = SqliteDatabase.init("", inMemory = true).tryGet() -proc now(): Timestamp = - getNanosecondTime(getTime().toUnixFloat()) - -proc ts(offset=0, origin=now()): Timestamp = - origin + getNanosecondTime(offset) - -proc fakeWakuMessage( - payload = "TEST-PAYLOAD", - contentTopic = DefaultContentTopic, - ts = now() -): WakuMessage = - WakuMessage( - payload: toBytes(payload), - contentTopic: contentTopic, - version: 1, - timestamp: ts - ) - suite "SQLite message store - init store": test "init store": diff --git a/tests/v2/test_message_store_sqlite_query.nim b/tests/v2/test_message_store_sqlite_query.nim index 7f20d6610..a68f33248 100644 --- a/tests/v2/test_message_store_sqlite_query.nim +++ b/tests/v2/test_message_store_sqlite_query.nim @@ -1,45 +1,23 @@ {.used.} import - std/[options, tables, sets, times, strutils, sequtils, algorithm], - stew/byteutils, + std/[options, tables, sets, strutils, sequtils, algorithm], unittest2, chronos, - chronicles, + chronicles +import ../../waku/v2/node/storage/message/sqlite_store, ../../waku/v2/node/storage/sqlite, ../../waku/v2/protocol/waku_message, ../../waku/v2/protocol/waku_store/pagination, ../../waku/v2/utils/time, - ./utils + ./utils, + ./testlib/common -const - DefaultPubsubTopic = "/waku/2/default-waku/proto" - DefaultContentTopic = ContentTopic("/waku/2/default-content/proto") - - -proc now(): Timestamp = - getNanosecondTime(getTime().toUnixFloat()) - -proc ts(offset=0, origin=now()): Timestamp = - origin + getNanosecondTime(offset) - proc newTestDatabase(): SqliteDatabase = SqliteDatabase.init("", inMemory = true).tryGet() -proc fakeWakuMessage( - payload = "TEST-PAYLOAD", - contentTopic = DefaultContentTopic, - ts = now() -): WakuMessage = - WakuMessage( - payload: toBytes(payload), - contentTopic: contentTopic, - version: 1, - timestamp: ts - ) - suite "message store - history query": @@ -477,7 +455,7 @@ suite "message store - history query": test "single content topic and valid time range": ## Given const contentTopic = "test-content-topic" - let timeOrigin = getNanosecondTime(epochTime()) + let timeOrigin = now() let database = newTestDatabase() @@ -522,7 +500,7 @@ suite "message store - history query": test "single content topic and invalid time range - no results": ## Given const contentTopic = "test-content-topic" - let timeOrigin = getNanosecondTime(epochTime()) + let timeOrigin = now() let database = newTestDatabase() @@ -561,7 +539,7 @@ suite "message store - history query": test "single content topic and only time range start": ## Given const contentTopic = "test-content-topic" - let timeOrigin = getNanosecondTime(epochTime()) + let timeOrigin = now() let database = newTestDatabase() @@ -602,7 +580,7 @@ suite "message store - history query": test "single content topic, cursor and only time range start": ## Given const contentTopic = "test-content-topic" - let timeOrigin = getNanosecondTime(epochTime()) + let timeOrigin = now() let database = newTestDatabase() diff --git a/tests/v2/test_rest_relay_api.nim b/tests/v2/test_rest_relay_api.nim index ff921b0d8..07c3138e6 100644 --- a/tests/v2/test_rest_relay_api.nim +++ b/tests/v2/test_rest_relay_api.nim @@ -13,7 +13,9 @@ import ../../waku/v2/protocol/waku_message, ../../waku/v2/node/waku_node, ../../waku/v2/node/rest/[server, client, base64, utils], - ../../waku/v2/node/rest/relay/[api_types, relay_api, topic_cache] + ../../waku/v2/node/rest/relay/[api_types, relay_api, topic_cache], + ../../waku/v2/utils/time, + ./testlib/common proc testWakuNode(): WakuNode = @@ -26,14 +28,6 @@ proc testWakuNode(): WakuNode = WakuNode.new(privkey, bindIp, port, some(extIp), some(port)) -proc fakeWakuMessage(payload = toBytes("TEST"), contentTopic = "test"): WakuMessage = - WakuMessage( - payload: payload, - contentTopic: contentTopic, - version: 1, - timestamp: 2022 - ) - suite "REST API - Relay": asyncTest "Subscribe a node to an array of topics - POST /relay/v1/subscriptions": @@ -167,8 +161,8 @@ suite "REST API - Relay": response.data.all do (msg: RelayWakuMessage) -> bool: msg.payload == Base64String.encode("TEST-1") and msg.contentTopic.get().string == "content-topic-x" and - msg.version.get() == Natural(1) and - msg.timestamp.get() == int64(2022) + msg.version.get() == 2 and + msg.timestamp.get() != Timestamp(0) check: diff --git a/tests/v2/test_waku_filter.nim b/tests/v2/test_waku_filter.nim index d407cf7c7..935071879 100644 --- a/tests/v2/test_waku_filter.nim +++ b/tests/v2/test_waku_filter.nim @@ -5,7 +5,6 @@ import testutils/unittests, chronos, chronicles, - libp2p/switch, libp2p/crypto/crypto, libp2p/multistream import @@ -13,20 +12,13 @@ import ../../waku/v2/protocol/waku_message, ../../waku/v2/protocol/waku_filter, ../test_helpers, - ./utils + ./utils, + ./testlib/common, + ./testlib/switch -const - DefaultPubsubTopic = "/waku/2/default-waku/proto" - DefaultContentTopic = ContentTopic("/waku/2/default-content/proto") - const dummyHandler = proc(requestId: string, msg: MessagePush) {.async, gcsafe, closure.} = discard -proc newTestSwitch(key=none(PrivateKey), address=none(MultiAddress)): Switch = - let peerKey = key.get(PrivateKey.random(ECDSA, rng[]).get()) - let peerAddr = address.get(MultiAddress.init("/ip4/127.0.0.1/tcp/0").get()) - return newStandardSwitch(some(peerKey), addrs=peerAddr) - # TODO: Extend test coverage procSuite "Waku Filter": diff --git a/tests/v2/test_waku_lightpush.nim b/tests/v2/test_waku_lightpush.nim index 358fd75d9..4ace16011 100644 --- a/tests/v2/test_waku_lightpush.nim +++ b/tests/v2/test_waku_lightpush.nim @@ -11,12 +11,8 @@ import ../../waku/v2/node/peer_manager/peer_manager, ../../waku/v2/protocol/waku_message, ../../waku/v2/protocol/waku_lightpush, - ../test_helpers - - -const - DefaultPubsubTopic = "/waku/2/default-waku/proto" - DefaultContentTopic = ContentTopic("/waku/2/default-content/proto") + ../test_helpers, + ./testlib/common # TODO: Extend lightpush protocol test coverage diff --git a/tests/v2/test_waku_rln_relay.nim b/tests/v2/test_waku_rln_relay.nim index 80be99552..418df7a57 100644 --- a/tests/v2/test_waku_rln_relay.nim +++ b/tests/v2/test_waku_rln_relay.nim @@ -6,7 +6,8 @@ import testutils/unittests, chronos, chronicles, stint, stew/byteutils, stew/shims/net as stewNet, libp2p/crypto/crypto, - json, + json +import ../../waku/v2/protocol/waku_message, ../../waku/v2/protocol/waku_rln_relay/[rln, waku_rln_relay_utils, diff --git a/tests/v2/test_waku_store.nim b/tests/v2/test_waku_store.nim index 3816872c1..b76248aca 100644 --- a/tests/v2/test_waku_store.nim +++ b/tests/v2/test_waku_store.nim @@ -1,12 +1,11 @@ {.used.} import - std/[options, tables, sets, sequtils, times], + std/[options, tables, sets, sequtils], stew/byteutils, testutils/unittests, chronos, chronicles, - libp2p/switch, libp2p/crypto/crypto import ../../waku/v2/protocol/waku_message, @@ -17,39 +16,13 @@ import ../../waku/v2/node/storage/message/sqlite_store, ../../waku/v2/node/peer_manager/peer_manager, ../../waku/v2/utils/time, - ../test_helpers + ./testlib/common, + ./testlib/switch -const - DefaultPubsubTopic = "/waku/2/default-waku/proto" - DefaultContentTopic = ContentTopic("/waku/2/default-content/proto") - - -proc now(): Timestamp = - getNanosecondTime(getTime().toUnixFloat()) - proc newTestDatabase(): SqliteDatabase = SqliteDatabase.init("", inMemory = true).tryGet() -proc fakeWakuMessage( - payload = "TEST-PAYLOAD", - contentTopic = DefaultContentTopic, - ts = getNanosecondTime(epochTime()), - ephemeral = false, -): WakuMessage = - WakuMessage( - payload: toBytes(payload), - contentTopic: contentTopic, - version: 1, - timestamp: ts, - ephemeral: ephemeral, - ) - -proc newTestSwitch(key=none(PrivateKey), address=none(MultiAddress)): Switch = - let peerKey = key.get(PrivateKey.random(ECDSA, rng[]).get()) - let peerAddr = address.get(MultiAddress.init("/ip4/127.0.0.1/tcp/0").get()) - return newStandardSwitch(some(peerKey), addrs=peerAddr) - proc newTestMessageStore(): MessageStore = let database = newTestDatabase() SqliteStore.init(database).tryGet() @@ -340,7 +313,7 @@ procSuite "Waku Store - history query": client.setPeer(serverSwitch.peerInfo.toRemotePeerInfo()) ## Given - let currentTime = getNanosecondTime(getTime().toUnixFloat()) + let currentTime = now() let msgList = @[ WakuMessage(payload: @[byte 0], contentTopic: ContentTopic("2"), timestamp: currentTime - 9), WakuMessage(payload: @[byte 1], contentTopic: DefaultContentTopic, timestamp: currentTime - 8), @@ -409,7 +382,7 @@ procSuite "Waku Store - history query": client.setPeer(serverSwitch.peerInfo.toRemotePeerInfo()) ## Given - let currentTime = getNanosecondTime(getTime().toUnixFloat()) + let currentTime = now() let msgList = @[ WakuMessage(payload: @[byte 0], contentTopic: ContentTopic("2"), timestamp: currentTime - 9), WakuMessage(payload: @[byte 1], contentTopic: DefaultContentTopic, timestamp: currentTime - 8), @@ -695,7 +668,7 @@ suite "Waku Store - message handling": ## Given let - now = getNanoSecondTime(getTime().toUnixFloat()) + now = now() invalidSenderTime = now + MaxMessageTimestampVariance + 1 let message = fakeWakuMessage(ts=invalidSenderTime) @@ -718,7 +691,7 @@ suite "Waku Store - message handling": ## Given let - now = getNanoSecondTime(getTime().toUnixFloat()) + now = now() invalidSenderTime = now - MaxMessageTimestampVariance - 1 let message = fakeWakuMessage(ts=invalidSenderTime) diff --git a/tests/v2/test_waku_store_client.nim b/tests/v2/test_waku_store_client.nim index 07b667292..955ba7bcb 100644 --- a/tests/v2/test_waku_store_client.nim +++ b/tests/v2/test_waku_store_client.nim @@ -1,12 +1,10 @@ {.used.} import - std/[options, tables, sets, times], - stew/byteutils, + std/[options, tables, sets], testutils/unittests, chronos, chronicles, - libp2p/switch, libp2p/crypto/crypto import ../../waku/v2/protocol/waku_message, @@ -16,51 +14,24 @@ import ../../waku/v2/node/storage/sqlite, ../../waku/v2/node/storage/message/sqlite_store, ../../waku/v2/node/peer_manager/peer_manager, - ../../waku/v2/utils/time, - ../test_helpers + ./testlib/common, + ./testlib/switch -const - DefaultPubsubTopic = "/waku/2/default-waku/proto" - DefaultContentTopic = ContentTopic("/waku/2/default-content/proto") - - -proc now(): Timestamp = - getNanosecondTime(getTime().toUnixFloat()) - proc newTestDatabase(): SqliteDatabase = SqliteDatabase.init("", inMemory = true).tryGet() -proc fakeWakuMessage( - payload = toBytes("TEST-PAYLOAD"), - contentTopic = DefaultContentTopic, - ts = now(), - ephemeral = false, -): WakuMessage = - WakuMessage( - payload: payload, - contentTopic: contentTopic, - version: 1, - timestamp: ts, - ephemeral: ephemeral, - ) - -proc newTestSwitch(key=none(PrivateKey), address=none(MultiAddress)): Switch = - let peerKey = key.get(PrivateKey.random(ECDSA, rng[]).get()) - let peerAddr = address.get(MultiAddress.init("/ip4/127.0.0.1/tcp/0").get()) - return newStandardSwitch(some(peerKey), addrs=peerAddr) - proc newTestStore(): MessageStore = let database = newTestDatabase() SqliteStore.init(database).tryGet() -proc newTestWakuStore(switch: Switch, store=newTestStore()): WakuStore = +proc newTestWakuStore(switch: Switch, store=newTestStore()): Future[WakuStore] {.async.} = let peerManager = PeerManager.new(switch) rng = crypto.newRng() proto = WakuStore.init(peerManager, rng, store) - waitFor proto.start() + await proto.start() switch.mount(proto) return proto @@ -107,7 +78,7 @@ procSuite "Waku Store Client": await allFutures(serverSwitch.start(), clientSwitch.start()) let - server = newTestWakuStore(serverSwitch, store=testStore) + server = await newTestWakuStore(serverSwitch, store=testStore) client = newTestWakuStoreClient(clientSwitch) ## Given @@ -143,7 +114,7 @@ procSuite "Waku Store Client": await allFutures(serverSwitch.start(), clientSwitch.start()) let - server = newTestWakuStore(serverSwitch, store=testStore) + server = await newTestWakuStore(serverSwitch, store=testStore) client = newTestWakuStoreClient(clientSwitch) ## Given @@ -177,8 +148,8 @@ procSuite "Waku Store Client": await allFutures(serverSwitchA.start(), serverSwitchB.start(), clientSwitch.start()) let - serverA = newTestWakuStore(serverSwitchA, store=testStore) - serverB = newTestWakuStore(serverSwitchB, store=testStore) + serverA = await newTestWakuStore(serverSwitchA, store=testStore) + serverB = await newTestWakuStore(serverSwitchB, store=testStore) client = newTestWakuStoreClient(clientSwitch) ## Given @@ -214,7 +185,7 @@ procSuite "Waku Store Client": await allFutures(serverSwitch.start(), clientSwitch.start()) let - server = newTestWakuStore(serverSwitch, store=testStore) + server = await newTestWakuStore(serverSwitch, store=testStore) client = newTestWakuStoreClient(clientSwitch) ## Given @@ -243,7 +214,7 @@ procSuite "Waku Store Client": await allFutures(serverSwitch.start(), clientSwitch.start()) let - server = newTestWakuStore(serverSwitch, store=testStore) + server = await newTestWakuStore(serverSwitch, store=testStore) client = newTestWakuStoreClient(clientSwitch) ## Given diff --git a/tests/v2/test_waku_store_resume.nim b/tests/v2/test_waku_store_resume.nim index 8a5f7b39e..164ddfb7e 100644 --- a/tests/v2/test_waku_store_resume.nim +++ b/tests/v2/test_waku_store_resume.nim @@ -1,12 +1,10 @@ {.used.} import - std/[options, tables, sets, times], - stew/byteutils, + std/[options, tables, sets], testutils/unittests, chronos, chronicles, - libp2p/switch, libp2p/crypto/crypto import ../../waku/v2/protocol/waku_message, @@ -14,54 +12,24 @@ import ../../waku/v2/node/storage/sqlite, ../../waku/v2/node/storage/message/sqlite_store, ../../waku/v2/node/peer_manager/peer_manager, - ../../waku/v2/utils/time, - ../test_helpers + ./testlib/common, + ./testlib/switch -const - DefaultPubsubTopic = "/waku/2/default-waku/proto" - DefaultContentTopic = ContentTopic("/waku/2/default-content/proto") - - -proc now(): Timestamp = - getNanosecondTime(getTime().toUnixFloat()) - -proc ts(offset=0, origin=now()): Timestamp = - origin + getNanosecondTime(offset) - proc newTestDatabase(): SqliteDatabase = SqliteDatabase.init("", inMemory = true).tryGet() -proc fakeWakuMessage( - payload = toBytes("TEST-PAYLOAD"), - contentTopic = DefaultContentTopic, - ts = now(), - ephemeral = false, -): WakuMessage = - WakuMessage( - payload: payload, - contentTopic: contentTopic, - version: 1, - timestamp: ts, - ephemeral: ephemeral, - ) - -proc newTestSwitch(key=none(PrivateKey), address=none(MultiAddress)): Switch = - let peerKey = key.get(PrivateKey.random(ECDSA, rng[]).get()) - let peerAddr = address.get(MultiAddress.init("/ip4/127.0.0.1/tcp/0").get()) - return newStandardSwitch(some(peerKey), addrs=peerAddr) - proc newTestMessageStore(): MessageStore = let database = newTestDatabase() SqliteStore.init(database).tryGet() -proc newTestWakuStore(switch: Switch, store=newTestMessageStore()): WakuStore = +proc newTestWakuStore(switch: Switch, store=newTestMessageStore()): Future[WakuStore] {.async.} = let peerManager = PeerManager.new(switch) rng = crypto.newRng() proto = WakuStore.init(peerManager, rng, store) - waitFor proto.start() + await proto.start() switch.mount(proto) return proto @@ -71,7 +39,6 @@ procSuite "Waku Store - resume store": ## Fixtures let storeA = block: let store = newTestMessageStore() - let msgList = @[ fakeWakuMessage(payload= @[byte 0], contentTopic=ContentTopic("2"), ts=ts(0)), fakeWakuMessage(payload= @[byte 1], contentTopic=ContentTopic("1"), ts=ts(1)), @@ -118,8 +85,8 @@ procSuite "Waku Store - resume store": await allFutures(serverSwitch.start(), clientSwitch.start()) let - _ = newTestWakuStore(serverSwitch, store=storeA) - client = newTestWakuStore(clientSwitch) + server = await newTestWakuStore(serverSwitch, store=storeA) + client = await newTestWakuStore(clientSwitch) client.setPeer(serverSwitch.peerInfo.toRemotePeerInfo()) @@ -146,7 +113,7 @@ procSuite "Waku Store - resume store": await clientSwitch.start() - let client = newTestWakuStore(clientSwitch) + let client = await newTestWakuStore(clientSwitch) ## Given let peers = @[offlineSwitch.peerInfo.toRemotePeerInfo()] @@ -171,9 +138,9 @@ procSuite "Waku Store - resume store": await allFutures(serverASwitch.start(), serverBSwitch.start(), clientSwitch.start()) let - serverA = newTestWakuStore(serverASwitch, store=storeA) - serverB = newTestWakuStore(serverBSwitch, store=storeB) - client = newTestWakuStore(clientSwitch) + serverA = await newTestWakuStore(serverASwitch, store=storeA) + serverB = await newTestWakuStore(serverBSwitch, store=storeB) + client = await newTestWakuStore(clientSwitch) ## Given let peers = @[ diff --git a/tests/v2/test_waku_store_rpc_codec.nim b/tests/v2/test_waku_store_rpc_codec.nim index 41e1a3efa..a6e189ebb 100644 --- a/tests/v2/test_waku_store_rpc_codec.nim +++ b/tests/v2/test_waku_store_rpc_codec.nim @@ -9,24 +9,8 @@ import import ../../waku/v2/protocol/waku_message, ../../waku/v2/protocol/waku_store, - ../../waku/v2/utils/time - -const - DefaultPubsubTopic = "/waku/2/default-waku/proto" - DefaultContentTopic = ContentTopic("/waku/2/default-content/proto") - - -proc fakeWakuMessage( - payload = "TEST-PAYLOAD", - contentTopic = DefaultContentTopic, - ts = getNanosecondTime(epochTime()) -): WakuMessage = - WakuMessage( - payload: toBytes(payload), - contentTopic: contentTopic, - version: 1, - timestamp: ts - ) + ../../waku/v2/utils/time, + ./testlib/common procSuite "Waku Store - RPC codec": diff --git a/tests/v2/test_waku_swap.nim b/tests/v2/test_waku_swap.nim index 022afcff2..0055fb6a3 100644 --- a/tests/v2/test_waku_swap.nim +++ b/tests/v2/test_waku_swap.nim @@ -22,28 +22,8 @@ import ../../waku/v2/utils/peers, ../../waku/v2/utils/time, ../test_helpers, - ./utils - - -const - DefaultPubsubTopic = "/waku/2/default-waku/proto" - DefaultContentTopic = ContentTopic("/waku/2/default-content/proto") - - -proc now(): Timestamp = - getNanosecondTime(getTime().toUnixFloat()) - -proc fakeWakuMessage( - payload = "TEST-PAYLOAD", - contentTopic = DefaultContentTopic, - ts = now() -): WakuMessage = - WakuMessage( - payload: toBytes(payload), - contentTopic: contentTopic, - version: 1, - timestamp: ts - ) + ./utils, + ./testlib/common procSuite "Waku SWAP Accounting": diff --git a/tests/v2/test_wakunode_lightpush.nim b/tests/v2/test_wakunode_lightpush.nim index 469fcc0bc..5b7e7f882 100644 --- a/tests/v2/test_wakunode_lightpush.nim +++ b/tests/v2/test_wakunode_lightpush.nim @@ -7,35 +7,14 @@ import chronicles, chronos, libp2p/crypto/crypto, - libp2p/switch, + libp2p/switch +import ../../waku/v2/protocol/waku_message, ../../waku/v2/protocol/waku_lightpush, ../../waku/v2/node/peer_manager/peer_manager, ../../waku/v2/utils/peers, - ../../waku/v2/utils/time, - ../../waku/v2/node/waku_node - -from std/times import getTime, toUnixFloat - - -const - DefaultPubsubTopic = "/waku/2/default-waku/proto" - DefaultContentTopic = ContentTopic("/waku/2/default-content/proto") - -proc now(): Timestamp = - getNanosecondTime(getTime().toUnixFloat()) - -proc fakeWakuMessage( - payload = "TEST-PAYLOAD", - contentTopic = DefaultContentTopic, - ts = now() -): WakuMessage = - WakuMessage( - payload: toBytes(payload), - contentTopic: contentTopic, - version: 1, - timestamp: ts - ) + ../../waku/v2/node/waku_node, + ./testlib/common procSuite "WakuNode - Lightpush": diff --git a/tests/v2/test_wakunode_relay.nim b/tests/v2/test_wakunode_relay.nim index cad16061f..eb1a6ce7b 100644 --- a/tests/v2/test_wakunode_relay.nim +++ b/tests/v2/test_wakunode_relay.nim @@ -16,7 +16,7 @@ import libp2p/protocols/pubsub/pubsub, libp2p/protocols/pubsub/gossipsub import - ../../waku/v2/protocol/[waku_relay, waku_message], + ../../waku/v2/protocol/waku_message, ../../waku/v2/node/peer_manager/peer_manager, ../../waku/v2/utils/peers, ../../waku/v2/node/waku_node diff --git a/tests/v2/test_wakunode_store.nim b/tests/v2/test_wakunode_store.nim index 342ab9911..b30152275 100644 --- a/tests/v2/test_wakunode_store.nim +++ b/tests/v2/test_wakunode_store.nim @@ -23,34 +23,14 @@ import ../../waku/v2/node/peer_manager/peer_manager, ../../waku/v2/utils/peers, ../../waku/v2/utils/time, - ../../waku/v2/node/waku_node + ../../waku/v2/node/waku_node, + ./testlib/common -from std/times import getTime, toUnixFloat - - -const - DefaultPubsubTopic = "/waku/2/default-waku/proto" - DefaultContentTopic = ContentTopic("/waku/2/default-content/proto") - -proc now(): Timestamp = - getNanosecondTime(getTime().toUnixFloat()) proc newTestMessageStore(): MessageStore = let database = SqliteDatabase.init("", inMemory = true)[] SqliteStore.init(database).tryGet() -proc fakeWakuMessage( - payload = "TEST-PAYLOAD", - contentTopic = DefaultContentTopic, - ts = now() -): WakuMessage = - WakuMessage( - payload: toBytes(payload), - contentTopic: contentTopic, - version: 1, - timestamp: ts - ) - procSuite "WakuNode - Store": let rng = crypto.newRng() @@ -199,7 +179,7 @@ procSuite "WakuNode - Store": # Insert the same message in both node's store let - receivedTime3 = getNanosecondTime(getTime().toUnixFloat() + 10.float) + receivedTime3 = now() + getNanosecondTime(10) digest3 = computeDigest(msg3) require server.wakuStore.store.put(DefaultTopic, msg3, digest3, receivedTime3).isOk() require client.wakuStore.store.put(DefaultTopic, msg3, digest3, receivedTime3).isOk() diff --git a/tests/v2/testlib/common.nim b/tests/v2/testlib/common.nim new file mode 100644 index 000000000..6551fe1ae --- /dev/null +++ b/tests/v2/testlib/common.nim @@ -0,0 +1,38 @@ +import + std/times +import + ../../../waku/v2/protocol/waku_message, + ../../../waku/v2/utils/time + +const + DefaultPubsubTopic* = "/waku/2/default-waku/proto" + DefaultContentTopic* = ContentTopic("/waku/2/default-content/proto") + + +proc now*(): Timestamp = + getNanosecondTime(getTime().toUnixFloat()) + +proc ts*(offset=0, origin=now()): Timestamp = + origin + getNanosecondTime(offset) + + +proc fakeWakuMessage*( + payload: string|seq[byte] = "TEST-PAYLOAD", + contentTopic = DefaultContentTopic, + ts = now(), + ephemeral = false +): WakuMessage = + var payloadBytes: seq[byte] + when payload is string: + payloadBytes = toBytes(payload) + else: + payloadBytes = payload + + WakuMessage( + payload: payloadBytes, + contentTopic: contentTopic, + version: 2, + timestamp: ts, + ephemeral: ephemeral + ) + \ No newline at end of file diff --git a/tests/v2/testlib/switch.nim b/tests/v2/testlib/switch.nim new file mode 100644 index 000000000..f7742200f --- /dev/null +++ b/tests/v2/testlib/switch.nim @@ -0,0 +1,11 @@ +import + std/options, + libp2p/switch, + libp2p/builders +import + ../../test_helpers + +proc newTestSwitch*(key=none(PrivateKey), address=none(MultiAddress)): Switch = + let peerKey = key.get(PrivateKey.random(ECDSA, rng[]).get()) + let peerAddr = address.get(MultiAddress.init("/ip4/127.0.0.1/tcp/0").get()) + return newStandardSwitch(some(peerKey), addrs=peerAddr) \ No newline at end of file