From 8d7b6efd433dd843fa354a74788138b01f4440c7 Mon Sep 17 00:00:00 2001 From: Sergei Tikhomirov Date: Wed, 1 Nov 2023 18:21:25 +0100 Subject: [PATCH] incentivization outline majorly restructured --- incentivization.md | 319 ++++++++++++++++++++++++++------------------- 1 file changed, 183 insertions(+), 136 deletions(-) diff --git a/incentivization.md b/incentivization.md index 3cff8be..2b4799b 100644 --- a/incentivization.md +++ b/incentivization.md @@ -1,27 +1,84 @@ -Our goal is to add an incentivization scheme to Waku to make it (more) incentive compatible. -In what follows, we abbreviate incentivization as i13n. +Waku is a family of decentralized communication protocols. +The Waku network consists of independent nodes running the corresponding protocols. +Waku needs incentivization (i13n) to ensure proper node behavior in the absence of any centralized coordinator. -We aim to answer the following questions: +In this document, we overview the problem of i13n in decentralized systems. +We classify the possible methods of i13n and give example used in prior successful P2P networks. +We then briefly introduce Waku and outline the unique i13n challenges it presents. -1. what is the structure of the protocols in question? -2. what is the desired behavior of protocol participants? -3. what deviations from the desired behavior occur without incentivization? -4. what incentivization tools do we have? -5. what tools are appropriate for our purpose? -6. what parameters can we chose? what are our restrictions? -7. suggest a concrete i13n architecture. -8. how do we check if we've solved the problem? +We then go into more detail into one of the Waku's protocols, Store, responsible for archival storage. +We propose an i13n scheme for Store and implement an MVP solution. +We discuss the choices we have made for the MVP version, and what design options may be considered in the future. -# Overview +# Classification of i13n tools -Waku implements a modular decentralized censorship resistant P2P communications protocol. -Waku consists of multiple protocols (see [architecture](https://waku.org/about/architect)). -We focus on the main four are Relay (a P2P protocol), and three light protocols: Filter, Store, and Lightpush, which have a client-server architecture (aka request-response). +We can think of incentivization tools as a two-by-two matrix: +- rewards vs punishment; +- monetary vs reputation. -A Waku node is a node that runs at least one of the Waku protocols. -A full Waku node is a node that runs Relay. -A light Waku node as a node that only runs client-side of one of the light protocols. -See also: https://github.com/waku-org/research/issues/28 +In other words, there are four quadrants: +- monetary reward: the client pays the server; +- monetary punishment: the server makes a deposit in advance and gets slashed in case of misbehavior; +- reputation reward: the server's reputation increases if it behaves well; +- reputation punishment: the server's reputation decreases if it behaves badly. + +Reputation can only work if there are tangible benefits of having a high reputation and drawbacks of having a low reputation. +For example: +- clients are more likely to connect to servers with high reputation; +- clients disconnect from servers with low reputation. + +In the presence of monetary rewards, low-reputation servers miss out on potential revenue or lose their deposit. +Without the monetary aspects, low-reputation nodes can't get as much benefit from the network. +Reputation either assumes a repeated interaction (i.e., local reputation), or some amount of trust (centrally managed rankings). + +Ideally, monetary motivation should be atomically linked with performance. +A node should be rewarded if and only if it performed the desired action. +Analogously, it should be punished if and only it it misbehaved. +In other words, if the client pays first, the server cannot deny service, +and if the client pays after the fact, it's impossible to default on the obligation. + +In blockchain networks, the desired behavior of miners or validators can be automatically verified and rewarded with native tokens (or punished by slashing). +Enforcing atomicity in decentralized data-focused networks can be challenging: +it is non-trivial to prove that a certain piece of data was sent or received. +Therefore, such cases may warrant a combination of monetary and reputation-based approaches. + +# Related work + +There have been many example of incentivized decentralized systems. + +## Early P2P file-sharing + +Early P2P file-sharing networks employed reputation-based approaches and stickly defaults. +For instance, in BitTorrent, a peer by default shares pieces of a file before having received it in whole. +At the same time, the bandwidth that a peer can use depends on how much is has shared previously. +This policy rewards nodes who share by allowing them to download file faster. +While this reward is not monetary, it has proven to be sufficient in practice. + +## Blockchains + +The key innovation of Bitcoin, inherited and built upon by later blockchain networks, is the introduction of native monetary i13 mechanism. +In the case of Bitcoin, miners create new blocks and are automatically rewarded with newly mined coins, as prescribed by the protocol. +An invalid block will be rejected by other nodes and not rewarded, which incentivizes good behavior. +There are no intrinsic monetary punishments in Bitcoin, only rewards. +However, mining nodes are required to expend physical resources for block generation. + +Proof-of-stake consensus algorithms introduced intrinsic monetary punishments in the blockchain context. +A validator locks up (stakes) native tokens and gets rewarded for validating new blocks. +In case of misbehavior, the deposit is automatically taken away (i.e., the bad actor is slashed). + +## Decentralized storage + +Multiple decentralized storage networks have appeared in recent years, including Codex, Storj, Sia, Filecoin, IPFS. +They combine the techniques from early P2P file-sharing and blockchain-inspired reward mechanisms. + +# Waku + +Waku is a family of protocols (see [architecture](https://waku.org/about/architect)) for a modular decentralized censorship-resistant P2P communications network. +The backbone of Waku is the Relay protocol ([RLN-Relay](https://rfc.vac.dev/spec/17/) is an spam-protected version of the protocol). +Additionally, there are three light (client-server, request-response) protocols: Filter, Store, and Lightpush. + +There is no strict definition of a full node vs a light node in Waku (see https://github.com/waku-org/research/issues/28). +In this document, we may refer to a node that is running Relay and Store (server-side) as a full node, and to a node that is running a client-side of any of the light protocols as a light node. In light protocols, a client sends a request to a server. A server (a Relay node) performs some actions and returns a response, in particular: @@ -29,14 +86,21 @@ A server (a Relay node) performs some actions and returns a response, in particu - [[Store]]: the server responds with messages broadcast within the specified time frame; - [[Lightpush]]: the server publishes the client's message to the Relay network. -Waku aims to function on widely available hardware. -Hardware requirements for light nodes are lower than for full nodes. -In particular, bandwidth consumption should be limited (estimated at 10 Mbps). -See also: https://github.com/waku-org/research/issues/31 +## Waku i13n challenges -# Store protocol +As a communication protocol, Waku lacks consensus or a native token. +These properties bring Waku closer to purely reputation-incentivized file-sharing systems. +Our goal nevertheless is to combine monetary and reputation-based incentives in Waku. +The rationale for that is that monetary incentives have demonstrated their robustness in blockchain networks, +and are well-suited for a network designed to scale well beyond the initial phase when it's mainly maintained by enthusiasts for altruistic reasons. -We first focus on i13n of the Store protocol. +In our i13n framework, currently Waku only operates under reputation-based rewards and punishments. +While [RLN-Relay](https://rfc.vac.dev/spec/17/) adds monetary punishments for spammers, slashing is yet to be activated. + + +# Waku Store + +In this section, we design a monetary-based i13n scheme for Waku Store. Similar techniques may be later applied to other protocols. Store is a client-server protocol. @@ -44,14 +108,6 @@ A client asks the node to respond with relevant messages previously relayed thro A relevant message is a message that has been broadcast via Relay within the specified time frame. The response may be split into multiple parts, as specified by pagination parameters. -TODO: Strictly speaking, the definition of relevant is inconsistent because there is no consensus over messages. A message may be broadcast but not received by some nodes. Does this happen often? Can and should we do something about it? - -## Desired behavior - -The desired behavior of a Store-server node is to store all non-ephemeral messages forever. - -TODO: address the obvious concern that storing everything forever is unsustainable. Should there be some cut-off time after which old messages are no longer stored? - Let's say, a client issues a request to the server. We want the following to happen: @@ -59,89 +115,44 @@ We want the following to happen: - all the messages in the response are relevant; - the response contains only relevant messages. -TODO: is this the full definition of the desired behavior? +From a security standpoint, each Store node should enforce limits on requests as to not be DoS-ed. -### RLN as a proxy metric of message relevance +As Waku doesn't intent to establish consensus over past messages, +we can only rely on heuristics to determine whether a message had been relayed earlier. +To decrease the chance of missing some messages, a client may query multiple servers and combine their replies (union of all messages; messages reported by some majority of servers, etc). -RLN (rate limiting nullifiers) is a method of spam prevention in Relay. -The message sender generates a proof of enrollment in some membership set. -Multiple proofs generated within one epoch lead to punishment. -This technique limits the message rate from each node to at most one message per epoch. -See also: https://rfc.vac.dev/spec/17/ +# Store i13n MVP -In the i13n context, we can't prove whether a message has indeed been broadcast in the past. -Instead, we use RLN proofs as a proxy metric. -A valid RLN proof signifies that the message has been generated by a node with an active membership during a particular epoch. -TODO: make sure the above is correct: what exactly does RLN prove? +We propose Store-i13n-MVP - the simplest version of i13n in Store. -## Deviations from the desired behavior +In broad strokes: +- client: I want this piece of history +- server (after internal calculations): here is the price +- client: pays (if price is ok; otherwise conversation ends) +- server: responds with data +- client: checks the data: if data is irrelevant - decreases server's reputation +- client (optionally): queries another server; compares responses; maybe decreases reputation of both (?) if responses diverge. Or queries 3 servers and assumes that messages returned by 2/3 or 3/3 are "real" ("Never Take Two Chronometers to Sea"). -There are multiple ways for a node to deviate from the desired behavior. -We look at potential misbehavior from the server side and from the client side separately. -### Server: Slow response -The server takes too long to respond. -Possible reasons: -- the server is offline accidentally; -- the request describes too many relevant messages (the server is overwhelmed); -- the server is malicious and deliberately delays the response; -- the server doesn't have some of the relevant messages and tries to request them from other nodes. +# Evaluation -### Server: Incomplete response -A relevant message is missing from the response. -Possible explanations: -- the server didn't receive the message when it was broadcast; -- the server deliberately withholds the message. +We measure the performance of our i13-ed Store protocol by the following metrics... -Contrary to blockchains, Relay doesn't have consensus over relayed messages. -Therefore, it's impossible to distinguish between the two scenarios above. -TODO: given this fact, what's the best we can aim for? +TODO: how do we check if we've solved the problem? -### Server: Irrelevant response -The response contains a message that is not relevant. -There are two scenarios here depending on whether RLN proofs are enforced. -If RLN is not enforced, a server may insert any number or irrelevant messages into the response. -If RLN is enforced, a server can only do so as long as it has a valid membership to generate the respective proofs. -This doesn't eliminate the attack but limits its consequences. +# Future work -TODO: what are the powers of a malicious server when it comes to generating proofs for irrelevant messages? Can the server generate proofs for past epochs? +Store-i13n-MVP is the simplest protocol we can start with. +Let us now outline the design choices to be made if we were to go beyond MVP. -### Client: Too many requests -The client sends many request to the server within a short period of time. -This may be seen as a DoS attack. - -### Client: Request is too large -The client sends a response that incurs excessive expenses on the server. -For example, the request covers a very long period in history, or, more generically, -a period that contains many messages. -This may also be seen as a DoS attack. - -## Privacy considerations - -Light protocols, in general, have weaker privacy properties than P2P protocols. -In a client-server exchange, a client wants to selectively interact with the network. -By doing so, it often reveals what it is interested in (e.g., subscribes to particular topics). - -A malicious Store server can spy on a client in the following ways: -- track what time frames a client is interested in; -- analyze the timing of requests; -- link requests done by the same client. - -TODO: expand in the context of an incentivized protocol. - - -# Cost-benefit analysis -The goal of i13n is to make nodes more likely to exhibit the desired behavior. -An incentive scheme links the payoffs to whether nodes follow the protocol or not. -Good behavior should be rewarded, bad behavior punished. +## Price discovery An incentive scheme should balance the costs and benefits for a node. Rewards should compensate the cost of good behavior. Punishments should offset the benefits that bad behavior may bring. -Let us analyze the costs and benefit of a server that are specific to the Store protocol: -- storage; -- bandwidth; -- computation. +In the MVP i13n protocol, clients and servers establish a free market by negotiating prices. +A server should understand its true costs to negotiate effectively. +The costs of a Store server are storage, bandwidth, and computation. Let us assume a constant flow of messages per epoch and a constant flow of requests for older messages. There are two processes: storing incoming messages, and serving old messages to clients. @@ -163,8 +174,6 @@ The cost of storing messages to clients, per epoch, is composed of: - download: proportional to the number of requests; - computational cost of handling requests. -TODO: write this down mathematically. - Storage is likely the dominating cost. Storage costs is proportional to the amount of information stored and the time it is stored for. A cumulative cost of storing a single message grows linearly with time. @@ -177,40 +186,41 @@ Computation: the server spends computing cycles while handling requests. This costs likely depends not only on the computation itself, but also at the database structure. For example, retrieving old or rarely requested messages from the local database may be more expensive than fresh or popular ones due to caching. -TODO: In file storage, I store a file and I pay for the ability to query it later. In Store, Alice relays a message, a server stores is, and later Bob queries it (and pays for it under an i13n scheme). Is there a mismatch between who incurs costs and who pays for it? Shall we think of ways to make Alice incur some costs too? See: https://github.com/waku-org/research/issues/32 +## RLN as a proxy for message relevance -# Incentivization tools +RLN (rate limiting nullifiers) is a method of spam prevention in Relay ([RLN-Relay](https://rfc.vac.dev/spec/17/)). +The message sender generates a proof of enrollment in some membership set. +Multiple proofs generated within one epoch lead to punishment. +This technique limits the message rate from each node to at most one message per epoch. -We can think of incentivization tools as a two-by-two matrix: -- rewards vs punishment; -- monetary vs reputation. +In the i13n context, we can't prove whether a message has indeed been broadcast in the past. +Instead, we use RLN proofs as a proxy metric. +A valid RLN proof signifies that the message has been generated by a node with an active membership during a particular epoch. +Note that a malicious node with a valid membership can generate messages but not broadcast them. +Such messages would not be "relevant" (i.e., other nodes would be unaware of them), but they would satisfy the RLN-based heuristic. -In other words, there are four quadrants: -- monetary reward: the client pays the server; -- monetary punishment: the server makes a deposit in advance and gets slashed in case of misbehavior; -- reputation reward: the server's reputation increases if it behaves well; -- reputation punishment: the server's reputation decreases if it behaves badly. +Ideally, we would like to punish a server that omits relevant messages. +But as this can't be proven, we resort to reputation in this case. +In other words: if a client is dissatisfied with the response, it simply won't query this server anymore. +A way for the client to know (with some certainty) whether relevant messages have been omitted is to query another server. -Reputation can only work if there are tangible benefits of having a high reputation and drawbacks of having a low reputation. -For example: -- clients are more likely to connect to servers with high reputation; -- clients disconnect from servers with low reputation. -Assuming there is a monetary aspect too, low-reputation servers miss out on potential revenue or lose their deposit. -Reputation, however, assumes ether a repeated interaction (i.e., local reputation), or some amount of trust / centralization (centrally managed rankings). +## Privacy considerations -Monetary i13n tools, in turn, pose a key question: how to ensure atomicity between performance and reward or punishment? -In other words, if the client pays first, the server may take the money and not provide the servers. -Analogously, if the payment is due after the fact, the client can refuse to pay. -Linking payments with behavior involves a certain amount of trust as well. +Light protocols, in general, have weaker privacy properties than P2P protocols. +In a client-server exchange, a client wants to selectively interact with the network. +By doing so, it often reveals what it is interested in (e.g., subscribes to particular topics). -This issue is somewhat linked to the problem of Lightning watchtower incentivization (see https://www.talaia.watch/). - -A general observation: if monetary flows are dependent on events in the past, and there is no consensus on what exactly happened in the past, the scheme can be exploited. -TODO: can we use some on-chain component here as a semi-trusted arbiter? +A malicious Store server can spy on a client in the following ways: +- track what time frames a client is interested in; +- analyze the timing of requests; +- link requests done by the same client. ## Payment methods -What we want from a payment method (order of priority to be discussed): +The MVP protocol is agnostic to payment methods. +However, some payment methods may be more suitable than others. + +What we want from a payment method: - wide distribution (many people already have it); - high liquidity (i.e., easy to buy or sell at a reasonable exchange rate); - low latency; @@ -226,15 +236,52 @@ Let's list all (decentralized) payment options that we have: - a token on an EVM-based rollup - a token on a non-EVM blockchain (BTC / Lightning?) -# Related work +Note also that there may be different market models. +One model is that each client pays for its requests. +Another model assumes that (centralized) applications built on top of Waku buy "credits" in bulk for their users, for whom using the application (which may involve querying Store servers under the hood) is free of charge. -Decentralized storage is not a new idea. What is relevant for us? +## Incentive compatibility -1. Federated real-time messaging (IRC, mailing lists). There is no "sync" in IRC; there are simply logs of prior conversations optionally hosted wherever. -2. Centralized file storage (FTP, later Dropbox). Requires trust in availability, but not necessarily confidentiality: content can be encrypted (modulo metadata). -3. P2P file-sharing: Napster, BitTorrent, eDonkey. The power of defaults, local reputation. -4. Decentralized storage in the blockchain age: Storj, Sia, Filecoin, IPFS, Codex... +In file storage, I store a file and I pay for the ability to query it later. In Store, Alice relays a message, a server stores is, and later Bob queries it (and pays for it under an i13n scheme). Is there a mismatch between who incurs costs and who pays for it? Shall we think of ways to make Alice incur some costs too? See: https://github.com/waku-org/research/issues/32 -# Future work +## Generalization for other Waku protocols -How to generalize i13n for Store to other Waku protocols? +We plan to generalize i13n for Store to other Waku protocols, in particular, to light protocols (Lightpush and Filter). + +# Appendix: Deviations from the desired behavior + +There are multiple ways for a node to deviate from the desired behavior. +We look at potential misbehavior from the server side and from the client side separately. +### Server: Slow response +The server takes too long to respond. +Possible reasons: +- the server is offline accidentally; +- the request describes too many relevant messages (the server is overwhelmed); +- the server is malicious and deliberately delays the response; +- the server doesn't have some of the relevant messages and tries to request them from other nodes. + +### Server: Incomplete response +A relevant message is missing from the response. +Possible explanations: +- the server didn't receive the message when it was broadcast; +- the server deliberately withholds the message. + +Contrary to blockchains, Relay doesn't have consensus over relayed messages. +Therefore, it's impossible to distinguish between the two scenarios above. + +### Server: Irrelevant response +The response contains a message that is not relevant. +There are two scenarios here depending on whether RLN proofs are enforced. +If RLN is not enforced, a server may insert any number or irrelevant messages into the response. +If RLN is enforced, a server can only do so as long as it has a valid membership to generate the respective proofs. +This doesn't eliminate the attack but limits its consequences. + +### Client: Too many requests +The client sends many request to the server within a short period of time. +This may be seen as a DoS attack. + +### Client: Request is too large +The client sends a response that incurs excessive expenses on the server. +For example, the request covers a very long period in history, or, more generically, +a period that contains many messages. +This may also be seen as a DoS attack. \ No newline at end of file