logos-co
/
roadmap
mirror of https://github.com/logos-co/roadmap.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Waku weekly draft

This commit is contained in:
fryorcraken.eth 2023-11-06 19:34:12 +11:00
parent d1d83ffcaa
commit 3307762cbe
No known key found for this signature in database
GPG Key ID: A82ED75A8DFC50A4
1 changed files with 23 additions and 27 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
content/waku/updates/2023-11-06.md
View File

@ -6,8 +6,6 @@ date: 2023-11-06
draft: true
---
Internal dogfooding of Waku Network.
Might also want to add under achievements that we have in fact launched the proto-network and that it is already possible to run a node in this network
# 2023-11-06 Waku weekly
@ -15,19 +13,19 @@ Might also want to add under achievements that we have in fact launched the prot
- _achieved:_
- further PostgreSQL optimisations nearing conclusion
- implemented bridge to allow Status Community to move to static sharding with backwards compatibility towards default pubsub topic
- solution for shared bootstrap nodes being filtered out in discv5 as more static shards are activated
- ensured no unknown blockers from Waku's side to start dogfooding in conversation with Status Communities
- further PostgreSQL optimisations nearing conclusion
- implemented bridge to allow Status Community to move to static sharding with backwards compatibility towards default pubsub topic
- solution for shared bootstrap nodes being filtered out in discv5 as more static shards are activated
- ensured no unknown blockers from Waku's side to start dogfooding in conversation with Status Communities
- _next:_
- continue integration of static sharding in status-go.
- deploy bridge for backwards compatibility
- dogfooding of Status Desktop with Status staging fleet. Will aim to create a small internal Waku community
- continue integration of static sharding in status-go.
- deploy bridge for backwards compatibility
- dogfooding of Status Desktop with Status staging fleet. Will aim to create a small internal Waku community
- _risks:_
- Dependency on Vac/DST to conclude ~1k nodes simulations.
- Implementation of static sharding in Status Communities and design decisions mostly driven by go-waku developer, with minimal input from Status dev ([1](https://github.com/status-im/status-go/pull/4161), [2](https://github.com/status-im/status-go/pull/4094), [3](https://github.com/status-im/status-go/pull/4093)). See [status-go#4057](https://github.com/status-im/status-go/issues/4057) for remaining work. Mitigation by on-boarding Chat SDK lead on 6 Nov to drive effort.
- lack of confidence in simulation results: results so far exhibits various artifacts and anomalies seemingly related to tooling limitations. It is therefore difficult to draw certain conclusions re Waku scalability.
- lack of clarity in terms of Status fleet ownership, monitoring and maintenance, which is an integral part of the solution.
- Dependency on Vac/DST to conclude ~1k nodes simulations.
- Implementation of static sharding in Status Communities and design decisions mostly driven by go-waku developer, with minimal input from Status dev ([1](https://github.com/status-im/status-go/pull/4161), [2](https://github.com/status-im/status-go/pull/4094), [3](https://github.com/status-im/status-go/pull/4093)). See [status-go#4057](https://github.com/status-im/status-go/issues/4057) for remaining work. Mitigation by on-boarding Chat SDK lead on 6 Nov to drive effort.
- lack of confidence in simulation results: results so far exhibits various artifacts and anomalies seemingly related to tooling limitations. It is therefore difficult to draw certain conclusions re Waku scalability.
- lack of clarity in terms of Status fleet ownership, monitoring and maintenance, which is an integral part of the solution.
### [Targeted dogfooding for Status Communities](https://github.com/waku-org/pm/issues/97)
@ -42,11 +40,11 @@ Might also want to add under achievements that we have in fact launched the prot
- See 10k milestone update for PostgreSQL status.
- First version of the 10k-tool by DST is ready and is being tested with simulation running a small nim-libp2p/gossipsub binary.
- _risks_:
- Dependency on Vac/DST to run 10k nodes simulations. Tracked under
[`vac:dst:eng-10ktool`](https://roadmap.logos.co/tags/vac-updates).
- Wakutorsis tool is being dropped, meaning new tooling needs to be developed for 10k nodes simulations. It is currently uncertain whether such tool can be developed.
- Large scale simulations done by Vac/DST only covered nwaku relay. go-waku, status-go simulations are not planned short term (theoretical review of Status Communities messages is), nor are simulations including request-response protocols such as store and filter.
- lack of real world feedback/dogfooding: the complete static sharding solution involves significant changes to the Waku protocol and tech stack. Although each element is unit tested, dogfooding may hit corner cases in the integrated solution that cannot be foreseen/recreated in lab conditions.
- Dependency on Vac/DST to run 10k nodes simulations. Tracked under
[`vac:dst:eng-10ktool`](https://roadmap.logos.co/tags/vac-updates).
- Wakutorsis tool is being dropped, meaning new tooling needs to be developed for 10k nodes simulations. It is currently uncertain whether such tool can be developed.
- Large scale simulations done by Vac/DST only covered nwaku relay. go-waku, status-go simulations are not planned short term (theoretical review of Status Communities messages is), nor are simulations including request-response protocols such as store and filter.
- lack of real world feedback/dogfooding: the complete static sharding solution involves significant changes to the Waku protocol and tech stack. Although each element is unit tested, dogfooding may hit corner cases in the integrated solution that cannot be foreseen/recreated in lab conditions.
### [PostgreSQL in service node: Further optimisations](https://github.com/waku-org/pm/issues/84)
@ -58,18 +56,16 @@ Might also want to add under achievements that we have in fact launched the prot
**[nwaku] [PostgreSQL](https://github.com/waku-org/nwaku/issues/1888)**
- _achieved_:
- Time processing enhancement when performing _SELECT_ operations. There was an overhead caused by looping too many times over the returned rows, in order to convert the row types. By applying a "rowCallback" approach we can reduce by 30ms the time spent on the query under analysis.
- Time processing enhancement when performing _SELECT_ operations. There was an overhead caused by looping too many times over the returned rows, in order to convert the row types. By applying a "rowCallback" approach we can reduce by 30ms the time spent on the query under analysis.
- _next_:
- The queries used in the comparison analysis still perform much better in _SQLite_ (< ~5ms) than in _Postgres_ (< ~15ms.) Therefore we need to push the investigation further to enhance that.
( Edited: notice that the timings indicated above are for tests using consecutive queries. If the queries are performed concurrently, then the timings are worse. I will elaborate more in a report shortly .)
- The queries used in the comparison analysis still perform much better in _SQLite_ (< ~5ms) than in _Postgres_ (< ~15ms.) Therefore we need to push the investigation further to enhance that.
( Edited: notice that the timings indicated above are for tests using consecutive queries. If the queries are performed concurrently, then the timings are worse. I will elaborate more in a report shortly .)
## [Waku Network Gen 0](https://github.com/waku-org/pm/issues/50) - 2023-12-01
- _achieved_:
- Further simulation done, with a continued focus on message propagation time and possible improvements.
- Progress across all client son sharded peer management discovery
- First PRs merged towards basic distributed store services
- Starting internal dogfooding of Waku Network: We have launched the proto-network and it is already possible to run a node in said network.
- _risks_:
- Usage of RLN in js-waku and dependency on a (centralized?) Web3Provider remains unclear as one needs to know the merkle tree state (on chain) to generate proofs.
- We are progressively moving a nwaku engineer to a solution engineer role we need to backfill the role.
@ -107,10 +103,10 @@ Might also want to add under achievements that we have in fact launched the prot
**[nwaku] [PostgreSQL](https://github.com/waku-org/nwaku/issues/1888)**
- _achieved_:
- Time processing enhancement when performing _SELECT_ operations. There was an overhead caused by looping too many times over the returned rows, in order to convert the row types. By applying a "rowCallback" approach we can reduce by 30ms the time spent on the query under analysis.
- Time processing enhancement when performing _SELECT_ operations. There was an overhead caused by looping too many times over the returned rows, in order to convert the row types. By applying a "rowCallback" approach we can reduce by 30ms the time spent on the query under analysis.
- _next_:
- The queries used in the comparison analysis still perform much better in _SQLite_ (< ~5ms) than in _Postgres_ (< ~15ms.) Therefore we need to push the investigation further to enhance that.
( Edited: notice that the timings indicated above are for tests using consecutive queries. If the queries are performed concurrently, then the timings are worse. I will elaborate more in a report shortly .)
- The queries used in the comparison analysis still perform much better in _SQLite_ (< ~5ms) than in _Postgres_ (< ~15ms.) Therefore we need to push the investigation further to enhance that.
( Edited: notice that the timings indicated above are for tests using consecutive queries. If the queries are performed concurrently, then the timings are worse. I will elaborate more in a report shortly .)
## [Quality Assurance processes are in place](https://github.com/waku-org/pm/issues/73) - 2024-03-31