roadmap/codex/updates/2024-02-05.html

167 lines
118 KiB
HTML
Raw Normal View History

2024-02-06 16:26:39 +00:00
<!DOCTYPE html>
2024-04-03 11:13:02 +00:00
<html><head><title>2024-02-05 Codex Weekly</title><meta charSet="utf-8"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><meta property="og:title" content="2024-02-05 Codex Weekly"/><meta property="og:description" content="Weekly update of Codex."/><meta property="og:image" content="https://roadmap.logos.co/static/og-image.png"/><meta property="og:width" content="1200"/><meta property="og:height" content="675"/><link rel="icon" href="../../static/icon.png"/><meta name="description" content="Weekly update of Codex."/><meta name="generator" content="Quartz"/><link rel="preconnect" href="https://fonts.googleapis.com"/><link rel="preconnect" href="https://fonts.gstatic.com"/><link href="../../index.css" rel="stylesheet" type="text/css" spa-preserve/><link href="https://cdn.jsdelivr.net/npm/katex@0.16.0/dist/katex.min.css" rel="stylesheet" type="text/css" spa-preserve/><link href="https://fonts.googleapis.com/css2?family=IBM Plex Mono&amp;family=sans-serif:wght@400;700&amp;family=sans-serif:ital,wght@0,400;0,600;1,400;1,600&amp;display=swap" rel="stylesheet" type="text/css" spa-preserve/><script src="../../prescript.js" type="application/javascript" spa-preserve></script><script type="application/javascript" spa-preserve>const fetchData = fetch(`../../static/contentIndex.json`).then(data => data.json())</script></head><body data-slug="codex/updates/2024-02-05"><div id="quartz-root" class="page"><div id="quartz-body"><div class="left sidebar"><h1 class="page-title "><a href="../..">Logos Collective Project Roadmaps</a></h1><div class="spacer mobile-only"></div><div class="search "><div id="search-icon"><p>Search</p><div></div><svg tabIndex="0" aria-labelledby="title desc" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 19.9 19.7"><title id="title">Search</title><desc id="desc">Search</desc><g class="search-path" fill="none"><path stroke-linecap="square" d="M18.5 18.3l-5.4-5.4"></path><circle cx="8" cy="8" r="7"></circle></g></svg></div><div id="search-container"><div id="search-space"><input autocomplete="off" id="search-bar" name="search" type="text" aria-label="Search for something" placeholder="Search for something"/><div id="results-container"></div></div></div></div><div class="darkmode "><input class="toggle" id="darkmode-toggle" type="checkbox" tabIndex="-1"/><label id="toggle-label-light" for="darkmode-toggle" tabIndex="-1"><svg xmlns="http://www.w3.org/2000/svg" xmlnsXlink="http://www.w3.org/1999/xlink" version="1.1" id="dayIcon" x="0px" y="0px" viewBox="0 0 35 35" style="enable-background:new 0 0 35 35;" xmlSpace="preserve"><title>Light mode</title><path d="M6,17.5C6,16.672,5.328,16,4.5,16h-3C0.672,16,0,16.672,0,17.5 S0.672,19,1.5,19h3C5.328,19,6,18.328,6,17.5z M7.5,26c-0.414,0-0.789,0.168-1.061,0.439l-2,2C4.168,28.711,4,29.086,4,29.5 C4,30.328,4.671,31,5.5,31c0.414,0,0.789-0.168,1.06-0.44l2-2C8.832,28.289,9,27.914,9,27.5C9,26.672,8.329,26,7.5,26z M17.5,6 C18.329,6,19,5.328,19,4.5v-3C19,0.672,18.329,0,17.5,0S16,0.672,16,1.5v3C16,5.328,16.671,6,17.5,6z M27.5,9 c0.414,0,0.789-0.168,1.06-0.439l2-2C30.832,6.289,31,5.914,31,5.5C31,4.672,30.329,4,29.5,4c-0.414,0-0.789,0.168-1.061,0.44 l-2,2C26.168,6.711,26,7.086,26,7.5C26,8.328,26.671,9,27.5,9z M6.439,8.561C6.711,8.832,7.086,9,7.5,9C8.328,9,9,8.328,9,7.5 c0-0.414-0.168-0.789-0.439-1.061l-2-2C6.289,4.168,5.914,4,5.5,4C4.672,4,4,4.672,4,5.5c0,0.414,0.168,0.789,0.439,1.06 L6.439,8.561z M33.5,16h-3c-0.828,0-1.5,0.672-1.5,1.5s0.672,1.5,1.5,1.5h3c0.828,0,1.5-0.672,1.5-1.5S34.328,16,33.5,16z M28.561,26.439C28.289,26.168,27.914,26,27.5,26c-0.828,0-1.5,0.672-1.5,1.5c0,0.414,0.168,0.789,0.439,1.06l2,2 C28.711,30.832,29.086,31,29.5,31c0.828,0,1.5-0.672,1.5-1.5c0-0.414-0.168-0.789-0.439-1.061L28.561,26.439z M17.5,29 c-0.829,0-1.5,0.672-1.5,1.5v3c0,0.828,0.671,1.5,1.5,1.5s1.5-0.672,1.5-1.5v-3C19,29.672,18.329,29,17.5,29z M17.5,7 C11.71,7,7,11.71,7,17.5S11.71,28,17.5,28S28,23.29,28,17.5S23.29,7,17.5,7z M17.5,25c-4.136,0-7.5-3.364-7.5-7.5 c0-4.136,3.364-7.5,7.5-7.5c4.136,0,7.5,3.364,7.5,7.5C25,21.636,21.636,25,17.5,2
2024-02-06 16:26:39 +00:00
<blockquote>
<p>The Codex team continues to make progress with various initiatives to wrap up the demo for the Q1/Q2 public testnet release. An internal testnet has been running for the past few weeks and has been used to test the latest version of Codex and can be accessed using the <a href="https://github.com/codex-storage/codex-testnet-starter" class="external">Codex Testnet Starter</a> documentation.</p>
<p>Ongoing and new lines of research and development will soon begin in preparation for the next version of Codex that will be used for the mainnet release.. Here are the updates from different team members and their ongoing work.
Development is currently broken into three distinct teams:</p>
</blockquote>
<ol>
<li>Client, Testing, and Infrastructure</li>
<li>Marketplace</li>
<li>Research</li>
</ol>
<p>The different teams have actively moving on various fronts. The following are their team updates to various ongoing Epics.</p>
<h2 id="client">Client<a aria-hidden="true" tabindex="-1" href="#client" class="internal"> §</a></h2>
<h3 id="epic-nim-improvements">Epic: Nim Improvements<a aria-hidden="true" tabindex="-1" href="#epic-nim-improvements" class="internal"> §</a></h3>
<p>Completed:</p>
<ul>
<li><a href="https://github.com/nim-lang/langserver/issues/154" class="external">Filed issue for adding Atlas / non-Nimble support for packages</a></li>
<li><a href="https://github.com/nim-lang/atlas/pull/108" class="external">Start working on Atlas command changes</a></li>
</ul>
<p>Ongoing:</p>
<ul>
<li><a href="https://github.com/elcritch/apatheia" class="external">Create a repo as a place to start implementing some core async-threading tools for Chronos like worker pool &amp; disk io on top of the ThreadSignalPtr primitive</a>
<ul>
<li>plans to support refc &amp; orc</li>
</ul>
</li>
</ul>
<h3 id="epic-wiring-the-proving-system">Epic: Wiring the Proving System<a aria-hidden="true" tabindex="-1" href="#epic-wiring-the-proving-system" class="internal"> §</a></h3>
<p>Completed:</p>
<ul>
<li>Wrapped <a href="https://github.com/arkworks-rs/circom-compat" class="external">ark-circom</a> in a C FFI via:
<ul>
<li><a href="https://github.com/codex-storage/nim-circom-compat" class="external">nim-circom-compat</a> and</li>
<li><a href="https://github.com/codex-storage/circom-compat-ffi" class="external">circom-compat-ffi</a></li>
</ul>
</li>
</ul>
<p>Ongoing:</p>
<ul>
<li><a href="https://github.com/codex-storage/codex-storage-proofs-circuits" class="external">Integration of codex-storage-proofs-circuits</a> with <a href="https://github.com/codex-storage/nim-codex/pull/686" class="external">a PR in nim-codex</a></li>
</ul>
<h3 id="epic-improve-client-stability">Epic: Improve Client Stability<a aria-hidden="true" tabindex="-1" href="#epic-improve-client-stability" class="internal"> §</a></h3>
<p>Completed:</p>
<ul>
<li><a href="https://github.com/codex-storage/nim-codex/pull/691" class="external">Updated profiler branch for debugging</a></li>
<li><a href="https://github.com/codex-storage/nim-chronos/pull/7" class="external">Ported the profiler to Chronos V4</a></li>
<li><a href="https://github.com/gmega/alternate-test-runner/" class="external">Wrote separate test runner for two client test to try to figure out the origin of a file size bug which magically disappeared</a></li>
</ul>
<p>Ongoing:</p>
<ul>
<li><a href="https://github.com/codex-storage/nim-codex/pull/678" class="external">Finish work to take down draft flag from PR Expiry per dataset</a></li>
<li><a href="https://github.com/codex-storage/nim-codex/pull/631" class="external">Write tests for PR Safe block deletion (with ref count)</a></li>
<li><a href="https://github.com/codex-storage/nim-codex/pull/680" class="external">Look into the CI/docker packaging/local build tooling for Waku and Nimbus as part of build improvements PR</a></li>
<li><a href="https://github.com/codex-storage/nim-codex/pull/673" class="external">Chronos V4 branch</a></li>
<li>Pinned versions for Chronos and json-rpc</li>
</ul>
<h2 id="marketplace">Marketplace<a aria-hidden="true" tabindex="-1" href="#marketplace" class="internal"> §</a></h2>
<h3 id="epic-end-to-end-testing">Epic: End-to-end Testing<a aria-hidden="true" tabindex="-1" href="#epic-end-to-end-testing" class="internal"> §</a></h3>
<p>Completed:</p>
<ul>
<li><a href="https://github.com/codex-storage/nim-codex/pull/662" class="external">Rebased multinode integration test refactor which had two failing tests due to the erasure coding changes</a></li>
<li>Rebased Marketplace integration test suite</li>
<li>Added support for Result types using formatIt for logging proxy</li>
<li><a href="https://github.com/codex-storage/codex-contracts-eth/pull/79" class="external">Finished the verifier contract</a></li>
<li><a href="https://github.com/codex-storage/codex-contracts-eth/pull/82" class="external">Deployed a dummy verifier on local networks for testing</a></li>
<li>Finished updates to nim-ethers, all tests passing, including in Nim v2</li>
<li>Fixed an issue in the nim-ethers json-rpc update
<ul>
<li>Derived Signers could not access the derived getAddress and sendTransaction when their async raises were updated with SignerError</li>
</ul>
</li>
</ul>
<p>Ongoing:</p>
<ul>
<li>Continue work on updating nim-ethers to support json-rpc breaking changes</li>
<li>Continue work on supporting json-rpc breaking changes and pulling out utils/json to its own lib</li>
<li><a href="https://github.com/codex-storage/nim-codex/pull/683" class="external">Integrate contract changes into nim-codex</a></li>
<li>Look into removing waitFor in integration tests</li>
<li>Review and clean up nim-ethers changes
<ul>
<li>Try to figure out a cleaner way to handle exceptions instead of catching all CatchableErrors</li>
</ul>
</li>
<li>Start tweaking the nim-json api to normalize both serialize and deserialize pragmas, with modes: OptOut, OptIn, and Strict</li>
<li>WIP on adding PATCH call for Availabilities</li>
</ul>
<h2 id="research">Research<a aria-hidden="true" tabindex="-1" href="#research" class="internal"> §</a></h2>
<pre><code>2024 R&amp;D Goals
1. Proving system and aggregation improvements (folding or lookups)
2. Aggregator/validator design
3. DHT improvements
4. Tokenomics and incentive design
5. Bandwidth incentives
6. Dynamic data (appendable data)
</code></pre>
<p>Completed:</p>
<ul>
<li>Frobenius endomorphism &amp; pairing implementation</li>
<li>Review the Solidity Groth16 verifier</li>
</ul>
<p>Ongoing:</p>
<ul>
<li>DAS simulator improvements to cover more diffusion models</li>
<li>Start DAS sample query mechanism design</li>
<li>Proof recursion ideation</li>
</ul></article></div><div class="right sidebar"><div class="graph "><h3>Graph View</h3><div class="graph-outer"><div id="graph-container" data-cfg="{&quot;drag&quot;:true,&quot;zoom&quot;:true,&quot;depth&quot;:1,&quot;scale&quot;:1.1,&quot;repelForce&quot;:0.5,&quot;centerForce&quot;:0.3,&quot;linkDistance&quot;:30,&quot;fontSize&quot;:0.6,&quot;opacityScale&quot;:1,&quot;showTags&quot;:true,&quot;removeTags&quot;:[]}"></div><svg version="1.1" id="global-graph-icon" xmlns="http://www.w3.org/2000/svg" xmlnsXlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 55 55" fill="currentColor" xmlSpace="preserve"><path d="M49,0c-3.309,0-6,2.691-6,6c0,1.035,0.263,2.009,0.726,2.86l-9.829,9.829C32.542,17.634,30.846,17,29,17
s-3.542,0.634-4.898,1.688l-7.669-7.669C16.785,10.424,17,9.74,17,9c0-2.206-1.794-4-4-4S9,6.794,9,9s1.794,4,4,4
c0.74,0,1.424-0.215,2.019-0.567l7.669,7.669C21.634,21.458,21,23.154,21,25s0.634,3.542,1.688,4.897L10.024,42.562
C8.958,41.595,7.549,41,6,41c-3.309,0-6,2.691-6,6s2.691,6,6,6s6-2.691,6-6c0-1.035-0.263-2.009-0.726-2.86l12.829-12.829
c1.106,0.86,2.44,1.436,3.898,1.619v10.16c-2.833,0.478-5,2.942-5,5.91c0,3.309,2.691,6,6,6s6-2.691,6-6c0-2.967-2.167-5.431-5-5.91
v-10.16c1.458-0.183,2.792-0.759,3.898-1.619l7.669,7.669C41.215,39.576,41,40.26,41,41c0,2.206,1.794,4,4,4s4-1.794,4-4
s-1.794-4-4-4c-0.74,0-1.424,0.215-2.019,0.567l-7.669-7.669C36.366,28.542,37,26.846,37,25s-0.634-3.542-1.688-4.897l9.665-9.665
C46.042,11.405,47.451,12,49,12c3.309,0,6-2.691,6-6S52.309,0,49,0z M11,9c0-1.103,0.897-2,2-2s2,0.897,2,2s-0.897,2-2,2
S11,10.103,11,9z M6,51c-2.206,0-4-1.794-4-4s1.794-4,4-4s4,1.794,4,4S8.206,51,6,51z M33,49c0,2.206-1.794,4-4,4s-4-1.794-4-4
s1.794-4,4-4S33,46.794,33,49z M29,31c-3.309,0-6-2.691-6-6s2.691-6,6-6s6,2.691,6,6S32.309,31,29,31z M47,41c0,1.103-0.897,2-2,2
s-2-0.897-2-2s0.897-2,2-2S47,39.897,47,41z M49,10c-2.206,0-4-1.794-4-4s1.794-4,4-4s4,1.794,4,4S51.206,10,49,10z"></path></svg></div><div id="global-graph-outer"><div id="global-graph-container" data-cfg="{&quot;drag&quot;:true,&quot;zoom&quot;:true,&quot;depth&quot;:-1,&quot;scale&quot;:0.9,&quot;repelForce&quot;:0.5,&quot;centerForce&quot;:0.3,&quot;linkDistance&quot;:30,&quot;fontSize&quot;:0.6,&quot;opacityScale&quot;:1,&quot;showTags&quot;:true,&quot;removeTags&quot;:[]}"></div></div></div><div class="toc desktop-only"><button type="button" id="toc"><h3>Table of Contents</h3><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="fold"><polyline points="6 9 12 15 18 9"></polyline></svg></button><div id="toc-content"><ul class="overflow"><li class="depth-0"><a href="#codex-update-jan-29nd---feb-5th" data-for="codex-update-jan-29nd---feb-5th">Codex Update Jan 29nd - Feb 5th</a></li><li class="depth-1"><a href="#client" data-for="client">Client</a></li><li class="depth-2"><a href="#epic-nim-improvements" data-for="epic-nim-improvements">Epic: Nim Improvements</a></li><li class="depth-2"><a href="#epic-wiring-the-proving-system" data-for="epic-wiring-the-proving-system">Epic: Wiring the Proving System</a></li><li class="depth-2"><a href="#epic-improve-client-stability" data-for="epic-improve-client-stability">Epic: Improve Client Stability</a></li><li class="depth-1"><a href="#marketplace" data-for="marketplace">Marketplace</a></li><li class="depth-2"><a href="#epic-end-to-end-testing" data-for="epic-end-to-end-testing">Epic: End-to-end Testing</a></li><li class="depth-1"><a href="#research" data-for="research">Research</a></li></ul></div></div><div class="backlinks "><h3>Backlinks</h3><ul class="overflow"><li>No backlinks found</li></ul></div></div></div><footer class><hr/><p>Created with <a href="https://quartz.jzhao.xyz/">Quartz v4.1.0</a>, © 2024</p><ul><li><a href="https://github.com/logos-co/roadmap">GitHub</a></li><li><a href="https://discord.com/invite/logos-state">Discord Community</a></li></ul></footer></div></body><script type="application/javascript">// quartz/components/scripts/quartz/components/scripts/callout.inline.ts
function toggleCallout() {
const outerBlock = this.parentElement;
outerBlock.classList.toggle(`is-collapsed`);
const collapsed = outerBlock.classList.contains(`is-collapsed`);
const height = collapsed ? this.scrollHeight : outerBlock.scrollHeight;
outerBlock.style.maxHeight = height + `px`;
let current = outerBlock;
let parent = outerBlock.parentElement;
while (parent) {
if (!parent.classList.contains(`callout`)) {
return;
}
const collapsed2 = parent.classList.contains(`is-collapsed`);
const height2 = collapsed2 ? parent.scrollHeight : parent.scrollHeight + current.scrollHeight;
parent.style.maxHeight = height2 + `px`;
current = parent;
parent = parent.parentElement;
}
}
function setupCallout() {
const collapsible = document.getElementsByClassName(
`callout is-collapsible`
);
for (const div of collapsible) {
const title = div.firstElementChild;
if (title) {
title.removeEventListener(`click`, toggleCallout);
title.addEventListener(`click`, toggleCallout);
const collapsed = div.classList.contains(`is-collapsed`);
const height = collapsed ? title.scrollHeight : div.scrollHeight;
div.style.maxHeight = height + `px`;
}
}
}
document.addEventListener(`nav`, setupCallout);
window.addEventListener(`resize`, setupCallout);
</script><script type="module">
import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.esm.min.mjs';
const darkMode = document.documentElement.getAttribute('saved-theme') === 'dark'
mermaid.initialize({
startOnLoad: false,
securityLevel: 'loose',
theme: darkMode ? 'dark' : 'default'
});
document.addEventListener('nav', async () => {
await mermaid.run({
querySelector: '.mermaid'
})
});
</script><script src="https://cdn.jsdelivr.net/npm/katex@0.16.7/dist/contrib/copy-tex.min.js" type="application/javascript"></script><script src="../../postscript.js" type="module"></script></html>