Vac weekly update

Signed-off-by: ksr <kaiserd@users.noreply.github.com>
This commit is contained in:
ksr 2023-11-13 20:18:11 +01:00
parent c545664e97
commit 18b9031f6f
2 changed files with 108 additions and 103 deletions

View File

@ -48,7 +48,7 @@ draft: false
- `eng-10ktool:vac:bandwidth-test`:
- Machines are no longer blocked
- Added Kubernetes network policies to void having machines blocked.
- 600 node simulations with Kubernetes to try to replicate 0 rate anomaly (https://discord.com/channels/864066763682218004/1163414326934257684/1170039531500740718)
- 600 node simulations with Kubernetes to try to replicate 0 rate anomaly
- Started an aproximation of waku-simulator with Kurtosis
- Meeting with Slava to investigate prometheus dropping container labeling information
- `software-testing:waku:test-automation-js-waku`

View File

@ -8,140 +8,145 @@ draft: false
---
### vac:p2p:
- `nimlibp2p:vac:webrtc-transport`
- cont. fixed segfault.
- `nimlibp2p:vac:maintenance`
- Revert https://github.com/status-im/nim-libp2p/issues/868
- for Nimbus. A new libp2p branch was create for that https://github.com/status-im/nim-libp2p/tree/b2eac7e-and-revert-c6aa085 - https://github.com/status-im/nimbus-eth2/pull/5549
- new Issue #975: CI workflow is failing frequently, https://github.com/status-im/nim-libp2p/issues/975, https://github.com/status-im/nim-libp2p/tree/fix/ci-workflow-stability
- in progress: PR #968: fix: move workflows for Nim Devel and legacy i386 from "Daily"
- new workflows "Daily - Devel" and "Daily - Legacy Platforms" https://github.com/status-im/nim-libp2p/pull/968
- Issue #972: Daily workflow could fail randomly with [OSError] https://github.com/status-im/nim-libp2p/issues/972
- Dive into the Yamux/Relayv2 problem - https://github.com/status-im/nim-libp2p/pull/979
- `nimlibp2p:vac:maintenance`
- Add Hole Punching to libp2p test-plans https://github.com/status-im/nim-libp2p/issues/966 and https://github.com/libp2p/test-plans/pull/322
- Yamux doesn't work in a Relayv2 connection https://github.com/status-im/nim-libp2p/pull/
- Single-board computer (SBC) support https://github.com/status-im/nim-libp2p/issues/978
- ARM64/aarch64 support https://github.com/status-im/nim-libp2p/issues/980
- Test Plans repo fork updated and resources to run tests requested https://github.com/status-im/libp2p-test-plans
- Implementing Nimble lock file functionality https://github.com/status-im/nim-libp2p/issues/975 https://github.com/status-im/nim-libp2p/tree/fix/ci-workflow-stability
- fix: move workflows for Nim Devel and legacy i386 from "Daily" -> workflows renamed to "Nim Devel" and "Legacy Platforms" https://github.com/status-im/nim-libp2p/pull/968
- Daily workflow could fail randomly with [OSError] https://github.com/status-im/nim-libp2p/issues/972
- `nimlibp2p:vac:webrtc-transport`
- Fix read/write on the DTLS
- Retrieve the remote certificate
- Start creating a way to log packets behind the DTLS encryption into pcap file in order to make it readable with wireshark
- WebRTC is done. WebRTC for libp2p isn't:
- chrome://webrtc-internals/ show no errors, neither wireshark.
- But libp2p spec requires a weird handshake that I haven't finished yet
### vac:tke:
- admin/misc: (7 CC conference days)
- `vac:tke::status:SNT-staking`
- Finalizing the setup & shape of Snapshot space (@Martin)
- `vac:tke::nomos:economic-analysis`
- Researching properties of rewards functions (@Frederico)
- `vac:tke::waku:economic-analysis`
- Preparing an overview of possible revenue models (@Martin)
- `vac:tke::codex:economic-analysis`
- Meeting with Codex on token allocation (@Matty)
- Review Codex modeling and litepaper with Codex (all)
- One-pager draft requested by Matt Nemer for fundraising purposes (@Matty)
- `vac:tke::status:SNT-staking`
- Managing snaphot for Status go-live this week (@Martin)
- Reviewing and updating Cyprien's governance proposal draft (all)
- Status growth modeling followup discussion (@Matty)
- `vac:tke::nomos:economic-analysis`
- Continuing research of PoS economics and token distributions (@Frederico)
- `vac:tke::waku:economic-analysis`
- Monitoring Sergei's research (@Martin)
- Waku growth monitoring (@Martin)
### vac:dst:
- `wakurtosis:vac:retrospective-rlog`
- Reviewed comments; soon to publish
- `wakurtosis:vac:rlog`
- Analysis of new Wakurtosis simulations regarding the 600 nodes anomaly
- Analysis of K8 simulations regarding the 600 nodes anomaly
- `wakurtosis:waku:gossipsub-topology-analysis`
- Generated shadow simulation topology slices
- `analysis-shadow:vac:shadow-gossipsub-analysis`
- worked on Topology slices
- (added more RAM to the server)
- `analysis-shadow:waku:shadow-waku-relay-analysis`
- Run 600 nodes NWaku Shadow simulations with and without load
- Run 35K nodes simulation in shadow with low traffic
- Implemented constant traffic in the node
- `analysis:nomos:simulation-analysis`
- The network delay/bandwidth tuning, readjusting the probabilities, none of them helped. The bug(s) cannot be side-steped in any meaningful way.
- New issue: for > 10 views, the disk usage blows up. *1.7 TERABYTES*; and the output is just text files! This was quite unexpected; we now have yet another scalability issue with the nomos sim.
- spent couple of days on the Rust code and worked on adjustments. None of them helped with the bug.
- `analysis-gsub-model:vac:refactoring`
- Tuned/cleanedup to the control messages code
- sync with Moh and Nomos
- Nomos sim team now has eveything to reproduce and fix the bug: exact configs and access to full runs
- Suggested improvements to the data output that will reduce both the memory and disk overload of nomos simulation by orders of magnitude. Moh and Gusto agree that this will work.
- Wrapped up the nomos analysis for now: waiting for the sims team to finish fixing the bugs
- `analysis-shadow:vac:shadow-gossipsub-analysis`
- Wrote an analysis script that can read graphs generated by Shadow runs
- `analysis-gsub-model:status:control-messages`
- started write up on old and new Waku-models
- `eng-10ktool:vac:bandwidth-test`:
- Machines are no longer blocked
- Added Kubernetes network policies to void having machines blocked.
- 600 node simulations with Kubernetes to try to replicate 0 rate anomaly (https://discord.com/channels/864066763682218004/1163414326934257684/1170039531500740718)
- Started an aproximation of waku-simulator with Kurtosis
- Meeting with Slava to investigate prometheus dropping container labeling information
- `software-testing:waku:test-automation-js-waku`
- Helped Danish with implementing the testing part of a Static Sharding [PR](https://github.com/waku-org/js-waku/pull/1697/commits/762bc5ad6ecad1d41a4aa6ca9bd6e2832a4779ee)
- Add publishing waku messages with Kubernetes
- Tried to fix Prometheus labeling
- Keep investigating 0 rate anomaly with Kubernetes
- Meet with Florin to talk about tool repositories
- Ran 4.5k waku nodes with no traffic
- `vac:dst:software-testing:waku:test-automation-interop-testing`
- First [PR](https://github.com/waku-org/waku-interop-tests/pull/1#pullrequestreview-1714207334):
- start/stop waku docker nodes and connect them in a network
- send RPC or REST API calls and validate that messages are reaching the peers
- setup ci runs (on pr, on demand and nightly) via github actions
- allure [reports](https://waku-org.github.io/waku-interop-tests/4/) via github pages that contain test and docker log attachments for failing tests
- automated linting and code formatting
- 2 basic tests for now but will extend after the initial set of reviews
- Merged 1st [PR](https://github.com/waku-org/waku-interop-tests/pull/1)
- Draft 2nd [PR](https://github.com/waku-org/waku-interop-tests/pull/2):
- add more tests (17 ATOW)
- framework improvements and adjustments as the number of tests increase
- gowaku issues found:
- failures with relay [get-messages](https://github.com/waku-org/go-waku/pull/878) API
- response message contains extra [redundant fields](https://github.com/waku-org/go-waku/pull/888) compared with what is published
- REST API HTTP 500 [Internal Server Error](https://github.com/waku-org/go-waku/pull/880) when publishing messages
- Docker DEBUG logs [floaded](https://github.com/waku-org/go-waku/pull/881) in certain conditions
- nwaku issue found : container [crashes](https://github.com/waku-org/nwaku/issues/2198) when a message is published with malformed timestamp
- rest-api-specs issue found: [missing fields](https://github.com/waku-org/waku-rest-api/pull/2) in the REST API schema
- `software-testing:waku:test-automation-nwaku`
- Moved most of the PRs, missing one.
- Implement some store tests.
- Found (and fixed) issue with default values encoding/decoding for HistoryQuery.
- bug: `assert false` SEGFAULT.
- It only triggers on some files, and imports don't seem to be related.
- bug: Stopped filter node can receive messages
- It's actually expected behaviour.
- [Issue](https://github.com/waku-org/nwaku/issues/2178)
- bug: Filter doesn't receive messages after subscribing and restarting
- [Issue](https://github.com/waku-org/nwaku/issues/2175)
- Updated last PR, missing reviewer responses.
- Investigating `assert false`: Ivan took charge of that task; found it doesn't happen anymore (will write notes on the issue to resume investigation later)
- Picking up pace with store tests.
- Investigate a `PEER_DIAL_FAILURE` error in `store`; Happens when `archive` isn't mounted (unexpected); Not yet reported.
- `software-testing:waku:test-automation-go-waku`
- Wrote 4 tests related to filter unsubscribe and closed the PR https://github.com/waku-org/go-waku/pull/855
- Wrote 2 tests related to filter unsubscribe all https://github.com/waku-org/go-waku/pull/875
- Wrote string generator functions for tests with variable data https://github.com/waku-org/go-waku/pull/879
### vac:acz:
- `rlnp2p:waku:rln-doc-and-outreach`
- Make changes as per review on rlog
- `admin/misc`
- Study the research paper on the Reinforced Concrete hash function.
- Implemented Reinforced concrete in huff - https://github.com/rymnc/reinforced-concrete-huff
- tl;dr: lesser gas consumed than poseidon (2 inputs)
- RC hash writeup on vacp2p/research - https://github.com/vacp2p/research/pull/196
- @ devconnect, **participated in zk-hack, submitted https://devfolio.co/projects/reinforced-concrete-implementations-e82e, won a bounty from polygon**
- updated rln to use RC, significantly lower constraints, we can potentially bring down proof generation ti
- `rlnp2p:waku:rln-doc-and-outreach`
- **vac blog post: https://vac.dev/rlog/rln-anonymous-dos-prevention/**
- `secure-channels:waku:ethereum-chat`
- Keep working on the comments from the team and finish the raw RFC.
- https://github.com/vacp2p/rfc/pull/626
- Improving the raw RFC by writing it in terms of Noise, including: X3DH. XEdDSA. Double Ratchet. ADKG. https://github.com/vacp2p/rfc/tree/ethsecpm_improvements
- `rlnp2p:waku:multi-epoch-constraints`
- Keep working on the multi-constrained epoch project.
- `zerokit:vac:maintenance`
- merged PR 220
### vac:sc::
- `status:community-curation-contracts`
- Adjusted import remappings
- https://github.com/status-im/community-dapp/pull/95
- Added Goerli OP deployment config
- https://github.com/status-im/community-dapp/pull/96
- Deployed contracts on Goerli OP
- Verified contracts on OP Mainnet
- `vac:maintainance/misc`
- Deployed OP SNT on Goerli OP
- Created a few screen casts on deployment
- https://www.notion.so/f24bc8154bfd4757989216dde0f50af0?v=eb8f6f301de94f4889ee6179d16eaf47
- Implemented SNT V2 which will be used to make SNT available on Sepolia
- https://github.com/status-im/status-network-token-v2/pull/1
- `codex:review-codex-contracts`
- finished watching the call recording
- reviewed the code again based on the knoledge from call
- sent PRs based on our review
- https://github.com/codex-storage/codex-contracts-eth/pull/73
- https://github.com/codex-storage/codex-contracts-eth/pull/74
- Recorded new screen casts
- Started working on presentation for Research Call
- mostly working in the Discover bug
- contacted dapp deployer
- started a postmortem that I'm updating
- just exported a full list of dapps and contact
- `status:governance-contract-mvp`
- reseach and development; proposal types
- implemented delegation
### vac:nescience:
- `state-separation:vac:state-separation-doc`
- Keep researching techniques for harmonizing UTXO and based-account model for state separation -> Model to model adapter (Goal 1)
- Privacy-enhancing: Prepare document comparing Dory and IPA polynomial commitment schemes.
- Research ring signatures that use Dory and IPA.
- Still researching techniques for harmonizing UTXO and based-account model for state separation (delays due to Istanbul trip)
- Research for privacy enhancing (from state separation document): ring signatures (Dory, IPA) assuming Nova.
- For Flexibility Operation: researching Verkle trees implementation in other blockchains.
- Drafting document for privacy enhancing.
- `proofsystems:vac:research-existing-proof-systems`
- Researching techniques for proof creation and verification for Nova. (Goal 3)
- More readings on zkVM and how to build from scratch
- Preparing for Zk hack
- Done slides for ProgCrypto
- Preparing a summary of a [zk-Benchmark paper](https://eprint.iacr.org/2023/1503.pdf)
- Preparing for ProgCrypto
- `proofsystems:vac:benchmarks`
- Reviewed Starky implementation
- Reviewed Nova implementation
- Merged the Nova-Bellman PR (https://github.com/vacp2p/zk-explorations/pull/14)
- Merged the posiedon-starky PR (https://github.com/vacp2p/zk-explorations/pull/16)
- Reduced the number of columns in the halo2 circut
- Successfully ran shplonk implementation of poseidon halo2 circuit
- Wrote a GWC implementation of poseidon circuit for halo2
- Successfully ran GWC implementation of poseidon halo2 circuit
- Researched GPU halo2 enhancement for our own possible use (https://github.com/kroma-network/tachyon/tree/main/vendors/halo2)
### vac:dr:
- `valpriv:vac:tor-push-poc`
- Investigated, drove measurements from other fleet nodes for latency
- Got testbed results with 10 validators, comparing and adding
- Got separate measurements for aggregate vs attestation, Pushed all duties' broadcasts on tor.
- Managed to get libp2p logs from other nimbus fleet machine. Large files, need to process them
- `valpriv:vac:tor-push-paper`
- Adding scaled up execution results, revise discussion
- revised [presentation](https://docs.google.com/presentation/d/10icRYX3qh_B7B1bOzDSxX6QVOLru7QsCAO8K3RXLpyg/edit?usp=sharing)
- `gsub-scaling:vac:gossipsub-simulation`
- Completed small-scale simulations for large message handling
- Created an updated PR for shadow simulation scripts https://github.com/vacp2p/dst-gossipsub-test-node/pull/3
- Ready to share the paper. Finishing adding new results
- Updating presentations, simplifying the slides.
- `gsub-scaling:vac:unstructured-p2p-improvements-survey`
- Compiled things, revisited documents, and worked on presentation for logos research call on GossipSub Improvements
- revisited documents, and worked on presentation for logos research call on GossipSub Improvements.
- Based on the feedback from the logos research call, revisited nim-libp2p documentation, codebase etc.
- `gsub-scaling:vac:gossipsub-improvements-paper`
- Started revisiting the GossipSub improvement paper to reflect current work and finalize writeup (Still work in progress. will need 1-2 more days to reflect in overleaf document).
- Requested the DST team for initial simulation. I intend to use the outcomes to finalize test patterns.
- **published Vac blog post: https://vac.dev/rlog/GossipSub%20Improvements/**
- `zk:codex:storage-proofs-open-problems-review`
- Review [Groth16](https://github.com/codex-storage/zk-research-artifacts/blob/master/notes/groth16/groth16-notes.pdf) notes for Codex
### vac:rfc:
- `status:port-status-specs`
- Added the pull request for 71/STATUS-Push-Notification-Server https://github.com/vacp2p/rfc/pull/629/files (still WiP)
- Removed mailserver from 71/STATUS-PUSH-NOTIFICATION RFC https://github.com/vacp2p/rfc/pull/629
- Added references to 71/STATUS-PUSH-NOTIFICATION RFC https://github.com/vacp2p/rfc/pull/629
- Did first read of 10/WAKU-USAGE looking for improvements
- `waku:waku-keystore`
- Created outline
- Created draft pull request - https://github.com/vacp2p/rfc/pull/631