vacp2p
/
research
mirror of https://github.com/vacp2p/research.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
research/whisper_scalability/report.html

166 lines
9.3 KiB
HTML
Raw Permalink Normal View History

add html report
2019-10-16 08:24:34 +00:00
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<!-- This file was created with the aha Ansi HTML Adapter. https://github.com/theZiz/aha -->
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="application/xml+xhtml; charset=UTF-8" />
<title>stdin</title>
</head>
<body style="color:white; background-color:black">
<pre>
tweaks
2019-10-17 04:45:21 +00:00
<span style="filter: contrast(70%) brightness(190%);color:fuchsia;">Whisper theoretical model. Attempts to encode characteristics of it.</span>
add html report
2019-10-16 08:24:34 +00:00
Goals:
1. Ensure network scales by being user or usage bound, as opposed to bandwidth growing in proportion to network size.
2. Staying with in a reasonable bandwidth limit for limited data plans.
3. Do the above without materially impacting existing nodes.
tweaks
2019-10-17 04:45:21 +00:00
add naive case disclaimer
2019-10-17 04:54:56 +00:00
<span style="filter: contrast(70%) brightness(190%);color:fuchsia;">Case 1. Only receiving messages meant for you [naive case]</span>
add html report
2019-10-16 08:24:34 +00:00
Assumptions:
- A1. Envelope size (static): 1024kb
- A2. Envelopes / message (static): 10
- A3. Received messages / day (static): 100
- A4. Only receiving messages meant for you.
<span style="filter: contrast(70%) brightness(190%);color:#3333FF;">For 100 users, receiving bandwidth is 1000.0KB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:#3333FF;">For 10k users, receiving bandwidth is 1000.0KB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:#3333FF;">For 1m users, receiving bandwidth is 1000.0KB/day</span>
------------------------------------------------------------
add naive case disclaimer
2019-10-17 04:54:56 +00:00
<span style="filter: contrast(70%) brightness(190%);color:fuchsia;">Case 2. Receiving messages for everyone [naive case]</span>
add html report
2019-10-16 08:24:34 +00:00
Assumptions:
- A1. Envelope size (static): 1024kb
- A2. Envelopes / message (static): 10
- A3. Received messages / day (static): 100
- A5. Received messages for everyone.
<span style="filter: contrast(70%) brightness(190%);color:red;">For 100 users, receiving bandwidth is 97.7MB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:red;">For 10k users, receiving bandwidth is 9.5GB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:red;">For 1m users, receiving bandwidth is 953.7GB/day</span>
------------------------------------------------------------
<span style="filter: contrast(70%) brightness(190%);color:fuchsia;">Case 3. All private messages go over one discovery topic</span>
Assumptions:
- A1. Envelope size (static): 1024kb
- A2. Envelopes / message (static): 10
- A3. Received messages / day (static): 100
- A6. Proportion of private messages (static): 0.5
- A7. Public messages only received by relevant recipients (static).
- A8. All private messages are received by everyone (same topic) (static).
<span style="filter: contrast(70%) brightness(190%);color:yellow;">For 100 users, receiving bandwidth is 49.3MB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:red;">For 10k users, receiving bandwidth is 4.8GB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:red;">For 1m users, receiving bandwidth is 476.8GB/day</span>
------------------------------------------------------------
naive case explicit
2019-10-17 07:35:20 +00:00
<span style="filter: contrast(70%) brightness(190%);color:fuchsia;">Case 4. All private messages are partitioned into shards [naive case]</span>
add html report
2019-10-16 08:24:34 +00:00
Assumptions:
- A1. Envelope size (static): 1024kb
- A2. Envelopes / message (static): 10
- A3. Received messages / day (static): 100
- A6. Proportion of private messages (static): 0.5
- A7. Public messages only received by relevant recipients (static).
- A9. Private messages are partitioned evenly across partition shards (static), n=5000
<span style="filter: contrast(70%) brightness(190%);color:#3333FF;">For 100 users, receiving bandwidth is 1000.0KB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:#3333FF;">For 10k users, receiving bandwidth is 1.5MB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:red;">For 1m users, receiving bandwidth is 98.1MB/day</span>
------------------------------------------------------------
<span style="filter: contrast(70%) brightness(190%);color:fuchsia;">Case 5. Case 4 + All messages are passed through bloom filter with false positive rate</span>
Assumptions:
- A1. Envelope size (static): 1024kb
- A2. Envelopes / message (static): 10
- A3. Received messages / day (static): 100
- A6. Proportion of private messages (static): 0.5
- A7. Public messages only received by relevant recipients (static).
- A9. Private messages are partitioned evenly across partition shards (static), n=5000
- A10. Bloom filter size (m) (static): 512
- A11. Bloom filter hash functions (k) (static): 3
- A12. Bloom filter elements, i.e. topics, (n) (static): 100
- A13. Bloom filter assuming optimal k choice (sensitive to m, n).
- A14. Bloom filter false positive proportion of full traffic, p=0.1
<span style="filter: contrast(70%) brightness(190%);color:lime;">For 100 users, receiving bandwidth is 10.7MB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:red;">For 10k users, receiving bandwidth is 978.0MB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:red;">For 1m users, receiving bandwidth is 95.5GB/day</span>
Add report
2019-10-17 04:20:13 +00:00
NOTE: Traffic extremely sensitive to bloom false positives
This completely dominates network traffic at scale.
With p=1% we get 10k users ~100MB/day and 1m users ~10gb/day)
add html report
2019-10-16 08:24:34 +00:00
------------------------------------------------------------
<span style="filter: contrast(70%) brightness(190%);color:fuchsia;">Case 6. Case 5 + Benign duplicate receives</span>
Assumptions:
- A1. Envelope size (static): 1024kb
- A2. Envelopes / message (static): 10
- A3. Received messages / day (static): 100
- A6. Proportion of private messages (static): 0.5
- A7. Public messages only received by relevant recipients (static).
- A9. Private messages are partitioned evenly across partition shards (static), n=5000
- A10. Bloom filter size (m) (static): 512
- A11. Bloom filter hash functions (k) (static): 3
- A12. Bloom filter elements, i.e. topics, (n) (static): 100
- A13. Bloom filter assuming optimal k choice (sensitive to m, n).
- A14. Bloom filter false positive proportion of full traffic, p=0.1
- A15. Benign duplicate receives factor (static): 2
Add report
2019-10-17 04:40:15 +00:00
- A16. No bad envelopes, bad PoW, expired, etc (static).
add html report
2019-10-16 08:24:34 +00:00
<span style="filter: contrast(70%) brightness(190%);color:lime;">For 100 users, receiving bandwidth is 21.5MB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:red;">For 10k users, receiving bandwidth is 1.9GB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:red;">For 1m users, receiving bandwidth is 190.9GB/day</span>
------------------------------------------------------------
mailserver bloom filter bug
2019-10-17 05:42:26 +00:00
<span style="filter: contrast(70%) brightness(190%);color:fuchsia;">Case 7. Case 6 + Mailserver case under good conditions with smaller bloom false positive and mostly offline</span>
add html report
2019-10-16 08:24:34 +00:00
Add report
2019-10-17 04:20:13 +00:00
Assumptions:
- A1. Envelope size (static): 1024kb
- A2. Envelopes / message (static): 10
- A3. Received messages / day (static): 100
- A6. Proportion of private messages (static): 0.5
- A7. Public messages only received by relevant recipients (static).
- A9. Private messages are partitioned evenly across partition shards (static), n=5000
- A10. Bloom filter size (m) (static): 512
- A11. Bloom filter hash functions (k) (static): 3
- A12. Bloom filter elements, i.e. topics, (n) (static): 100
- A13. Bloom filter assuming optimal k choice (sensitive to m, n).
- A14. Bloom filter false positive proportion of full traffic, p=0.1
- A15. Benign duplicate receives factor (static): 2
Add report
2019-10-17 04:40:15 +00:00
- A16. No bad envelopes, bad PoW, expired, etc (static).
- A17. User is offline p% of the time (static) p=0.9
- A18. No bad request, duplicate messages for mailservers, and overlap/retires are perfect (static).
mailserver bloom filter bug
2019-10-17 05:42:26 +00:00
- A19. Mailserver requests can change false positive rate to be p=0.01
Add report
2019-10-17 04:40:15 +00:00
mailserver bloom filter bug
2019-10-17 05:42:26 +00:00
<span style="filter: contrast(70%) brightness(190%);color:#3333FF;">For 100 users, receiving bandwidth is 3.9MB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:red;">For 10k users, receiving bandwidth is 284.8MB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:red;">For 1m users, receiving bandwidth is 27.8GB/day</span>
add html report
2019-10-16 08:24:34 +00:00
Add report
2019-10-17 04:20:13 +00:00
------------------------------------------------------------
<span style="filter: contrast(70%) brightness(190%);color:fuchsia;">Case 8. Waka mode - no metadata protection with bloom filter and one node connected; still static shard</span>
add html report
2019-10-16 08:24:34 +00:00
Next step up is to either only use contact code, or shard more aggressively.
Note that this requires change of other nodes behavior, not just local node.
Assumptions:
- A1. Envelope size (static): 1024kb
- A2. Envelopes / message (static): 10
- A3. Received messages / day (static): 100
- A6. Proportion of private messages (static): 0.5
- A7. Public messages only received by relevant recipients (static).
- A9. Private messages are partitioned evenly across partition shards (static), n=5000
<span style="filter: contrast(70%) brightness(190%);color:#3333FF;">For 100 users, receiving bandwidth is 1000.0KB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:#3333FF;">For 10k users, receiving bandwidth is 1.5MB/day</span>
<span style="filter: contrast(70%) brightness(190%);color:red;">For 1m users, receiving bandwidth is 98.1MB/day</span>
------------------------------------------------------------
</pre>
</body>
</html>