status-go/waku

waku

Table of contents

• What is Waku?
• Waku versioning
• What does this package do?
• Waku package files

What is Waku?

Waku is a communication protocol for sending messages between Dapps. Waku is a fork of the Ethereum Whisper subprotocol, although not directly compatible with Whisper, both Waku and Whisper subprotocols can communicate via bridging.

Waku was created to solve scaling issues with Whisper and currently diverges from Whisper in the following ways:

• RLPx subprotocol is changed from shh/6 to waku/1.
• Light node capability is added.*
• Optional rate limiting is added.
• Status packet has following additional parameters: light-node, confirmations-enabled and rate-limits
• Mail Server and Mail Client functionality is now part of the specification.
• P2P Message packet contains a list of envelopes instead of a single envelope.

*As per vacp2p/specs#117 Waku de jure introduced light nodes as far as updates to the written Whisper specifications. Though the de facto case is that the go-ethereum Whisper implementation had already implemented light nodes and weren't a new feature in code.

Although status-go's Waku light node functionality is a direct fork of go-ethereum's Whisper light node functionality, technically, as far as specifications are concerned, light nodes are considered a new feature introduced in Waku.

Waku versioning

This package follows a versioning pattern that makes clean separation between breaking versions. As detailed in the PR that introduced this strategy to the package.

... the way we will move across versions is to maintain completely separate codebases and eventually remove those that are not supported anymore.

This has the drawback of some code duplication, but the advantage is that is more explicit what each version requires, and changes in one version will not impact the other, so we won't pile up backward compatible code. This is the same strategy used by whisper in go ethereum and is influenced by https://www.youtube.com/watch?v=oyLBGkS5ICk.

Familiarise yourself with the Spec-ulation Keynote by Rich Hickey, if you wish to more deeply understand the rationale for this versioning implementation.

This means that breaking changes will necessitate a new protocol version and a new version sub-package. The packages follow the naming convention of v* where * represents the major / breaking version number of the protocol.

Currently the package has the following version sub-packages:

• version 0 - v0
• version 1 - v1

What does this package do?

The basic function of this package is to implement the waku specifications, and provide the status-go binary with the ability to send and receive messages via Waku.

Waku package files

• waku.go
• api.go
• config.go
• mailserver.go
• common
  • bloomfilter.go
  • const.go
  • envelope.go
  • errors.go
  • events.go
  • filter.go
  • helpers.go
  • message.go
  • metrics.go
  • protocol.go
  • rate_limiter.go
  • topic.go
• Versioned
  • const.go
  • init.go
  • message.go
  • peer.go
  • status_options.go

Root

waku.go

waku.go serves as the main entry point for the package and where the main Waku{} struct lives.

---

api.go

api.go is home to the PublicWakuAPI{} struct which provides the waku RPC service that can be used publicly without security implications.

PublicWakuAPI{} wraps the main Waku{}, making the Waku{} functionality suitable for external consumption.

Consumption

PublicWakuAPI{} is wrapped by eth-node\bridge\geth.gethPublicWakuAPIWrapper{}, which is initialised via eth-node\bridge\geth.NewGethPublicWakuAPIWrapper() and exposed via gethWakuWrapper.PublicWakuAPI() and is finally consumed by wider parts of the application.

Notes

It is worth noting that each function of PublicWakuAPI{} received an unused context.Context parameter. This is originally passed in way higher up the food-chain and without significant refactoring is not a simple thing to remove / change. Mobile bindings depend on the ability to pass in a context.

---

config.go

config.go is home to the Config{} struct and the declaration of DefaultConfig.

Config{} is used to initialise the settings of an instantiated Waku{}. waku.New() creates a new instance of a Waku{} and takes a Config{} as a parameter, if nil is passed instead of an instance of Config{}, DefaultConfig is used.

Configuration values

Name	Type	Description
MaxMessageSize	uint32	Sets the maximum size of a waku message in bytes
MinimumAcceptedPoW	float64	Sets the minimum amount of work a message needs to have to be accepted by the waku node
BloomFilterMode	bool	When true, the waku node only matches against bloom filter
LightClient	bool	When true, the waku node does not forward messages
FullNode	bool	When true, the waku node forwards all messages
RestrictLightClientsConn	bool	When true, the waku node does not accept light clients as peers if it is a light client itself
EnableConfirmations	bool	When true, sends message confirmations

Default

The default configuration for a status-go Waku node is:

MaxMessageSize           : 1Mb
MinimumAcceptedPoW       : 0.2
RestrictLightClientsConn : true

---

mailserver.go

mailserver.go is home to MailServer interface, which is implemented by mailserver.WakuMailServer{} found in the package file mailserver/mailserver.go. MailServer represents a mail server, capable of receiving and archiving messages for subsequent delivery to the peers.

Additionally this package is home to MailServerResponse{} which represents the response payload sent by the mail-server. MailServerResponse{} is ultimately initialised by CreateMailServerEvent(), which is tied to the main Waku{} via the Waku.OnP2PRequestCompleted() function. This is ultimately accessed via the Peer.Run() function and is made available outside of the package with the waku.HandlePeer() function via Waku.protocol.Run := waku.HandlePeer.

---

Common

bloomfilter.go

bloomfilter.go holds a few bloomfilter specific functions.

---

const.go

const.go, originally a hangover from the go-ethereum whisperv6/doc.go package file later refactored, is home to the common Waku constants.

Notes

Versions also have version specific const.go files.

---

envelope.go

envelope.go is home to the Evelope{} and EnvelopeError{} structs. Envelope{} is used as the data packet in which message data is sent through the Waku network.

Envelope{} is accessed via the initialisation function NewEnvelope(), which is exclusively consumed by Message.Wrap() that prepares a message to be sent via Waku.

---

errors.go

errors.go holds generic package errors.

---

events.go

events.go handles data related to Waku events. This file contains string type consts that identify known Waku events.

Additionally, the file contains EnvelopeEvent{}, which serves as a representation of events created by envelopes. EnvelopeEvent{}s are initialised exclusively within the waku package.

---

filter.go

filter.go is home to Filter{} which represents a waku filter.

Usage

A status-go node will install / register filters through RPC calls from a client (eg status-mobile). The basic implementation of a filter requires at least 2 things:

1. An encryption key, example "superSafeEncryptionKey"
2. A 4 byte topic (TopicType), example "0x1234"

The node will install the filter [0x1234][{"superSafeEncryptionKey"}] on an instance of Filters{} and will notify its peers of this event

When a node receives an envelope it will attempt to match the topics against the installed filters, and then try to decrypt the envelope if the topic matches.

For example, if a node receives an envelope with topic 0x1234, the node will try to use the installed filter key superSafeEncryptionKey to decrypt the message. On success the node passes the decrypted message to the client.

In addition to the basic example above Filter{} allows for richer filtering:

Field Name	Type	Description
Src	*ecdsa.PublicKey	Sender of the message. Currently not used
KeyAsym	*ecdsa.PrivateKey	Private Key of recipient
KeySym	[]byte	Key associated with the Topic
Topics	[][]byte	Topics to filter messages with
PoW	float64	Proof of work as described in the Waku specs . Note: In status-mobile each client listens to the topic hash(pk), if a client wants to send a message to hash(pk1) they will also need to listen the hash(pk1) topic. However if the client doesn't want to receive envelopes for topic hash(pk1), the client may set the PoW to 1 so that all envelopes for topic hash(pk1) are discarded.
AllowP2P	bool	Indicates whether this filter is interested in direct peer-to-peer messages. Note: Typically set to true, we always want to receive P2P envelopes on a filter from trusted peers
SymKeyHash	common.Hash	The Keccak256Hash of the symmetric key, needed for optimization

Waku / Whisper divergence

Whisper, will process all the installed filters that the node has, and build a BloomFilter from all the topics of each installed filter (i.e. func ToBloomFilter(topics []TopicType) []byte { ... }). When a peer receives this BloomFilter, it will match the topic on each envelope that they receive against the BloomFilter, if it matches, it will forward this to the peer.

Waku, by default, does not send a BloomFilter, instead sends the topic in a clear array of []TopicType. This is an improvement on Whisper's usage as a BloomFilter may include false positives, which increase bandwidth usage. In contrast, clear topics are matched exactly and therefore don't create redundant bandwidth usage.

---

helpers.go

helpers.go holds the package's generic functions.

---

message.go

message.go is home to all message related functionality and contains a number of structs:

Name	Description
MessageParams{}	Specifies the exact way a message should be wrapped into an Envelope
sentMessage{}	Represents an end-user data packet to transmit through the Waku protocol. These are wrapped into Envelopes that need not be understood by intermediate nodes, just forwarded.
ReceivedMessage{}	Represents a data packet to be received through the Waku protocol and successfully decrypted.
MessagesRequest{}	Contains details of a request for historic messages.
MessagesResponse{}	Represents a request response sent after processing batch of envelopes.
MessageStore interface	Implemented by MemoryMessageStore{}
MemoryMessageStore{}	Represents messages stored in a memory hash table.

---

metrics.go

metrics.go is home to Prometheus metric hooks, for counting a range of Waku related metrics.

---

protocol.go

protocol.go houses the Peer and WakuHost interfaces.

Peer represents a remote Waku client with which the local host waku instance exchanges data / messages.

WakuHost is the local instance of waku, which both interacts with remote clients (peers) and local clients (like status-mobile, via a RPC API).

---

rate_limiter.go

rate_limiter.go was introduced as an improvement to Whisper allowing Waku nodes to limit the rate at which data is transferred. These limits are defined by the RateLimits{} which allows nodes to limit the following:

Limit Type	Description
IPLimits	Messages per second from a single IP (default 0, no limits)
PeerIDLimits	Messages per second from a single peer ID (default 0, no limits)
TopicLimits	Messages per second from a single topic (default 0, no limits)

In addition to the RateLimits{} this file also contains the following interfaces and structs.

Name	Description
RateLimiterPeer interface	Represents a Peer{} that is capable of being rate limited
RateLimiterHandler interface	Represents handler functionality for a Rate Limiter in the cases of exceeding a peer limit and exceeding an IP limit
MetricsRateLimiterHandler{}	Implements RateLimiterHandler, represents a handler for reporting rate limit Exceed data to the metrics collection service (currently prometheus)
DropPeerRateLimiterHandler{}	Implements RateLimiterHandler, represents a handler that introduces Tolerance to the number of Peer connections before Limit Exceeded errors are returned.
RateLimits{}	Represents rate limit settings exchanged using rateLimitingCode packet or in the handshake.
PeerRateLimiterConfig{}	Represents configurations for initialising a PeerRateLimiter
PeerRateLimiter{}	Represents a rate limiter that limits communication between Peers

The default PeerRateLimiterConfig is:

LimitPerSecIP:      10,
LimitPerSecPeerID:  5,
WhitelistedIPs:     nil,
WhitelistedPeerIDs: nil,

---

topic.go

topic.go houses the TopicType type.

TopicType represents a cryptographically secure, probabilistic partial classification of a message, determined as the first (leftmost) 4 bytes of the SHA3 hash of some arbitrary data given by the original author of a message.

Topics are used to filter incoming messages that the host's user has registered interest in. For further details on filtering see filter.go.

---

Versioned

For details about the divergence between versions please consult the READMEs of each version package.

• version 0
• version 1

---

Version const.go

const.go is home to the version sub-package's consts. These constants are version dependant and, as expected, may change from version to version.

---

Version init.go

init.go is home to the version sub-package's initialisation, and is used to initialise struct based variables at runtime.

---

Version message.go

message.go is home to the MultiVersionResponse{} and Version1MessageResponse{} structs, both of which are exclusively consumed by the version subpackage's Peer{}.

Both of these structs are used for handling Waku message responses, also known as message confirmations.

Version1MessageResponse{} is used for sending message responses. MultiVersionResponse{} is used for handling incoming message responses.

Usage

Message confirmations are used to inform a user, via the UI, that a message has been sent. Initially the message is marked as "Pending" and eventually as "Sent".

In order to trigger the message state transition from "pending" to "sent", Waku uses MessageResponse{}. Each peer on receiving a message will send back a MessageResponse, see the NewMessagesResponse() function.

The Waku host checks that the peer is a mailserver and if this mailserver was selected by the user, if so the message will be marked as "Sent" in the UI.

For further details read the Waku specification section on the subject.

Notes

Versioning at the MessageResponse level (see the struct Version field) should be phased out and defer to the subpackage's version number. Consider removal once we decide to move to a new major Waku version (i.e. waku/2).

---

Version peer.go

peer.go holds the version's sub-package Peer{} implementation of the common.Peer interface.

---

Version status_options.go

status_options.go holds the version's sub-package StatusOptions{} which implements the ethereum/go-ethereum/rlp Decoder and Encoder interfaces.

StatusOptions defines additional information shared between peers during the handshake. There might be more options provided than fields in StatusOptions, and as per the specs, should be ignored during deserialisation to stay forward compatible. In the case of RLP, options should be serialised to an array of tuples where the first item is a field name and the second is a RLP-serialised value.

For further details on RLP see:

• https://github.com/ethereum/wiki/wiki/RLP
• https://specs.vac.dev/specs/waku/waku.html#use-of-rlpx-transport-protocol