<titledata-rh="true">PostgreSQL | Waku Documentation</title><metadata-rh="true"name="viewport"content="width=device-width,initial-scale=1"><metadata-rh="true"name="twitter:card"content="summary_large_image"><metadata-rh="true"property="og:url"content="https://docs.waku.org/research/benchmarks/postgres-adoption"><metadata-rh="true"name="docusaurus_locale"content="en-GB"><metadata-rh="true"name="docsearch:language"content="en-GB"><metadata-rh="true"name="keywords"content="waku, web3"><metadata-rh="true"name="image"content="theme/image/preview-image.png"><metadata-rh="true"name="docusaurus_version"content="current"><metadata-rh="true"name="docusaurus_tag"content="docs-default-current"><metadata-rh="true"name="docsearch:version"content="current"><metadata-rh="true"name="docsearch:docusaurus_tag"content="docs-default-current"><metadata-rh="true"property="og:title"content="PostgreSQL | Waku Documentation"><metadata-rh="true"name="description"content="Document that describes why Nim-Waku started to use Postgres and shows some benchmark and comparison results."><metadata-rh="true"property="og:description"content="Document that describes why Nim-Waku started to use Postgres and shows some benchmark and comparison results."><linkdata-rh="true"rel="icon"href="/theme/image/favicon.ico"><linkdata-rh="true"rel="canonical"href="https://docs.waku.org/research/benchmarks/postgres-adoption"><linkdata-rh="true"rel="alternate"href="https://docs.waku.org/research/benchmarks/postgres-adoption"hreflang="en-GB"><linkdata-rh="true"rel="alternate"href="https://docs.waku.org/research/benchmarks/postgres-adoption"hreflang="x-default"><linkrel="alternate icon"type="image/png"href="/theme/image/favicon.png">
The following diagram shows the scenery used for such analysis.</p><p></p><divclass="wrapper_SWrM active_qZD5"><imgloading="lazy"alt="digram_multiple_nodes_one_database"src="/assets/images/digram_multiple_nodes_one_database-f503210c733fbeb8a572f9f6ccd81adf.png"width="1581"height="867"class="img_ev3q"><buttonclass="fullscreenButton_Bocn lsd-icon-button lsd-icon-button--medium lsd-icon-button--outlined"><divclass="icon_S7Kx m_thRi"><svgwidth="14"height="14"viewBox="0 0 14 14"fill="none"xmlns="http://www.w3.org/2000/svg"><pathd="M1.75 2.917V5.25h1.167V2.917H5.25V1.75H2.917A1.17 1.17 0 0 0 1.75 2.917ZM2.917 8.75H1.75v2.333a1.17 1.17 0 0 0 1.167 1.167H5.25v-1.167H2.917V8.75Zm8.166 2.333H8.75v1.167h2.333a1.17 1.17 0 0 0 1.167-1.167V8.75h-1.167v2.333Zm0-9.333H8.75v1.167h2.333V5.25h1.167V2.917a1.17 1.17 0 0 0-1.167-1.167Z"fill="#fff"></path></svg></div></button></div><p></p><p>There are three nim-waku nodes that are connected to the same database and all of them are trying to write messages to the same <em>PostgreSQL</em> instance. With that, it is very common to see errors like:</p><divclass="codeBlockContainer_EB2s codeBlockContainer_Ckt0 theme-code-block"style="--prism-color:#F8F8F2;--prism-background-color:rgba(var(--lsd-surface-secondary), 0.08)"><divclass="codeBlockContent_ugSV"><pretabindex="0"class="prism-code language-text codeBlock_TWhw thin-scrollbar"><codeclass="codeBlockLines_LDrR"><spanclass="token-line"style="color:#F8F8F2"><spanclass="token plain">ERR 2023-11-27 13:18:07.575+00:00 failed to insert message topics="waku archive" tid=2921 file=archive.nim:111 err="error in runStmt: error in dbConnQueryPrepared calling waitQueryToFinish: error in query: ERROR: duplicate key value violates unique constraint \"messageindex\"\nDETAIL: Key (storedat, id, pubsubtopic)=(1701091087417938405, 479c95bbf74222417abf76c7f9c480a6790e454374dc4f59bbb15ca183ce1abd, /waku/2/default-waku/proto) already exists.\n</span><br></span></code></pre><divclass="buttonGroup_Qu4e"><buttontype="button"aria-label="Copy code to clipboard"title="Copy"class="clean-btn"><spanclass="copyButtonIcons_an20"aria-hidden="true"><divclass="icon_S7Kx m_thRi copyButtonIcon_ZL7v"><svgwidth="14"height="14"viewBox="0 0 14 14"fill="none"xmlns="http://www.w3.org/2000/svg"><pathd="M2.917 12.833c-.321 0-.596-.114-.825-.343a1.121 1.121 0 0 1-.342-.823V3.5h1.167v8.167h6.416v1.166H2.917ZM5.25 10.5c-.32 0-.596-.114-.824-.343a1.121 1.121 0 0 1-.343-.824v-7c0-.32.115-.595.343-.824.229-.229.504-.343.824-.342h5.25c.32 0 .596.114.824.343.229.228.343.503.343.823v7c0 .321-.115.596-.343.825a1.121 1.121 0 0 1-.824.342H5.25Zm0-1.167h5.25v-7H5.25v7Z"fill="#fff"></path></svg></div></span></button></div></div></div><p>The <code>db-postgres-hammer</code> is aimed to stress the database from the <code>select</code> point of view. It performs <code>N</code> concurrent <code>select</code> queries with a certain rate.</p><h4class="anchor anchorWithHideOnScrollNavbar_WYt5"id="results">Results<ahref="#results"class="hash-link"aria-label="Direct link to Results"title="Direct link to Results"></a></h4><p>The following results were obtained by using the sandbox machine (metal-01.he-eu-hel1.wakudev.misc) and running nim-waku nodes from <ahref="https://github.com/waku-org/nwaku/tree/b452ed865466a33b7f5b87fa937a8471b28e466e"target="_blank"rel="noopener noreferrer">https://github.com/waku-org/nwaku/tree/b452ed865466a33b7f5b87fa937a8471b28e466e</a> and using the <code>test-waku-query</code> project from <ahref="https://github.com/waku-org/test-waku-query/tree/fef29cea182cc744c7940abc6c96d38a68739356"target="_blank"rel="noopener noreferrer">https://github.com/waku-org/test-waku-query/tree/fef29cea182cc744c7940abc6c96d38a68739356</a></p><p>The following shows the results</p><ol><li>Two <code>nwaku-postgres-additional</code> inserting messages plus 50 <code>db-postgres-hammer</code> making 10 <code>selects</code> per second.</li></ol><p></p><divclass="wrapper_SWrM active_qZD5"><imgloading="lazy"alt="
<divclass="wrapper_SWrM active_qZD5"><imgloading="lazy"alt="Insert time distribution Postgres"src="/assets/images/insert-time-dist-postgres-2-78f1bb4fc86aba4604ceda390a37dc96.png"width="913"height="307"class="img_ev3q"><buttonclass="fullscreenButton_Bocn lsd-icon-button lsd-icon-button--medium lsd-icon-button--outlined"><divclass="icon_S7Kx m_thRi"><svgwidth="14"height="14"viewBox="0 0 14 14"fill="none"xmlns="http://www.w3.org/2000/svg"><pathd="M1.75 2.917V5.25h1.167V2.917H5.25V1.75H2.917A1.17 1.17 0 0 0 1.75 2.917ZM2.917 8.75H1.75v2.333a1.17 1.17 0 0 0 1.167 1.167H5.25v-1.167H2.917V8.75Zm8.166 2.333H8.75v1.167h2.333a1.17 1.17 0 0 0 1.167-1.167V8.75h-1.167v2.333Zm0-9.333H8.75v1.167h2.333V5.25h1.167V2.917a1.17 1.17 0 0 0-1.167-1.167Z"fill="#fff"></path></svg></div></button></div><divclass="wrapper_SWrM active_qZD5"><imgloading="lazy"alt="Query time distribution Postgres"src="/assets/images/query-time-dist-postgres-2-347ba236b22fccce5efa56b29dae2021.png"width="913"height="307"class="img_ev3q"><buttonclass="fullscreenButton_Bocn lsd-icon-button lsd-icon-button--medium lsd-icon-button--outlined"><divclass="icon_S7Kx m_thRi"><svgwidth="14"height="14"viewBox="0 0 14 14"fill="none"xmlns="http://www.w3.org/2000/svg"><pathd="M1.75 2.917V5.25h1.167V2.917H5.25V1.75H2.917A1.17 1.17 0 0 0 1.75 2.917ZM2.917 8.75H1.75v2.333a1.17 1.17 0 0 0 1.167 1.167H5.25v-1.167H2.917V8.75Zm8.166 2.333H8.75v1.167h2.333a1.17 1.17 0 0 0 1.167-1.167V8.75h-1.167v2.333Zm0-9.333H8.75v1.167h2.333V5.25h1.167V2.917a1.17 1.17 0 0 0-1.167-1.167Z"fill="#fff"></path></svg></div></button></div></li></ol><p>In this case, the insert time gets more spread because the insert operations are shared amongst five more nodes. The <em>Store</em> query time remains the same on average.</p><olstart="3"><li>Five <code>nwaku-postgres-additional</code> inserting messages plus 100 <code>db-postgres-hammer</code> making 10 <code>selects</code> per second.