Commit Graph

51 Commits

Author SHA1 Message Date
Jaremy Creechley 36a9766c40
work around windows 2024-05-20 20:54:52 +03:00
Jaremy Creechley f5725a8505
work around windows 2024-05-20 19:51:11 +03:00
Jaremy Creechley 9986d22c71
fix nim c -r /Users/elcritch/projs/status/nim-codex/tests/codex/slots/testprover.nim 2024-05-20 19:23:53 +03:00
Jaremy Creechley 695686ad2c
fixup prover test setup 2024-05-20 19:16:33 +03:00
Jaremy Creechley a414e29a99
revert back erasure code 2024-05-20 19:04:58 +03:00
Jaremy Creechley eb22397ca7
implement async verify 2024-05-20 18:58:29 +03:00
Jaremy Creechley 77dfa466a2
implement async verify 2024-05-20 18:57:42 +03:00
Jaremy Creechley 2beb5ab2aa
try queue setup 2024-05-20 18:32:14 +03:00
Jaremy Creechley f90c4cdb66
try queue setup 2024-05-20 18:13:34 +03:00
Jaremy Creechley 9e2820944d
try queue setup 2024-05-20 18:04:05 +03:00
Jaremy Creechley 899a0f6f95
try queue setup 2024-05-20 17:54:06 +03:00
Jaremy Creechley 683d2186b0
try queue setup 2024-05-20 17:45:13 +03:00
Jaremy Creechley 5fbf2abcb0
try queue setup 2024-05-20 17:43:27 +03:00
Jaremy Creechley f603b642a2
try queue setup 2024-05-20 17:33:41 +03:00
Jaremy Creechley 13c59ca575
try queue setup 2024-05-20 17:31:34 +03:00
Jaremy Creechley 98abf47a23
try queue setup 2024-05-20 17:15:51 +03:00
Jaremy Creechley 30e087efb2
cleanup 2024-05-20 15:51:21 +03:00
Jaremy Creechley a96ea80148
fix awaitThreadResult 2024-05-18 18:12:50 +03:00
Jaremy Creechley 54edcb1ce7
fix spwan 2024-05-17 01:41:19 +03:00
Jaremy Creechley 0bbd558ee7
fix results stuff 2024-05-17 01:36:04 +03:00
Jaremy Creechley ebc284ae6a
fix results stuff 2024-05-17 01:31:53 +03:00
Jaremy Creechley 96862372e5
wire in prove 2024-05-17 01:03:19 +03:00
Jaremy Creechley 1e86b5c051
wire in prove 2024-05-17 00:59:00 +03:00
Jaremy Creechley 0dfa2cc488
wire in prove 2024-05-17 00:51:57 +03:00
Jaremy Creechley 38ac70809e
add asynthreads utils 2024-05-17 00:39:07 +03:00
Jaremy Creechley 542ba13222
add asynccircom backend 2024-05-17 00:30:11 +03:00
Jaremy Creechley c09c27ad2c
add asynccircom backend 2024-05-17 00:14:37 +03:00
Jaremy Creechley 2af5ed1490
add asynccircom backend 2024-05-17 00:11:52 +03:00
Jaremy Creechley 41167c6a74
add asynccircom backend 2024-05-16 21:49:31 +03:00
Jaremy Creechley 2e26a77e90
add asynccircom backend 2024-05-16 21:46:34 +03:00
Jaremy Creechley 5e39fe8380
add asynccircom backend 2024-05-16 21:43:35 +03:00
Jaremy Creechley 0ac9750308
add asynccircom backend 2024-05-16 21:41:24 +03:00
Jaremy Creechley aa9f2d2a1f
modifying circomcompat for use with asyncthreading 2024-05-16 21:36:02 +03:00
Jaremy Creechley a4010d1844
modifying circomcompat for use with asyncthreading 2024-05-16 21:35:24 +03:00
Jaremy Creechley eb75887631
modifying circomcompat for use with asyncthreading 2024-05-16 21:27:28 +03:00
Jaremy Creechley 7949a253ce
modifying circomcompat for use with asyncthreading 2024-05-16 21:26:18 +03:00
Jaremy Creechley 8f8d51faa4
modifying circomcompat for use with asyncthreading 2024-05-16 21:12:38 +03:00
Eric f567f4ec15
refactor: use serde for json de/serialization instead of utils/json (#704)
* json > nim-serde bump

Should wait until serde is integrated into nim-ethers before making these changes as there will be less import exceptions required.

* bump nim-serde

* change func to proc due to chronicles side effects

* import serde into utils/json, use as proxy

import nim-serde into utils/json and use utils/json as a proxy for serde functions, including overloading `%` and `fromJson` for application types.

* update tests to use serde

* bump serde to latest

* remove testjson -- no longer needed

* bump serde in nimble

* updates to reconcile rebase with master
2024-03-19 03:25:13 +00:00
markspanbroek 8589e63d34
Update contract (#734)
* Update codex-contracts-eth

* contracts: update G2Point definition

* integration: disable automatic advancing of time

reason: makes reasoning about timing in tests harder,
because the period is set to 60 seconds in the
marketplace configuration, but this code switches to
a new period every 500 milliseconds

* integration: fix parameters of marketplace payouts test

* integration: update test settings

* integration: fix typo

* integration: workaround for hardhat issue

Subscriptions expire after 5 minutes when using
websockets. Use http and polling instead.

* integration: remove origDatasetSizeInBlocks

* integration: fix proof parameters for test

* integration: do not log output by default

* integration: fix failure rate in test

* integration: fix warning

* integration: include clock in logs

* integration: allow for more periods

5 periods was cutting it close, if we get too much
pointer downtime, then the test would fail
2024-03-12 08:18:25 +00:00
Dmitriy Ryajov 9e13d2251a
Prover works with zkey (#711)
* rework backend to instantiate key at initialization

* add groth16 convertes for solidity

* prover taks num samples on construction

* add zkey file

* rework helpers

* rename types

* update tests

* reworked test helpers

* rename types

* rework test

* test all slots artifacts

* bump to latest version
2024-02-19 18:58:39 +00:00
Dmitriy Ryajov 0497114e44
Use correct blocks count in builder (#709) 2024-02-19 02:19:59 +00:00
Dmitriy Ryajov 6fab252bf1
safer backend release (#701) 2024-02-10 23:17:11 +00:00
Dmitriy Ryajov e23159b065
wire in circom backend (#698)
* wire in circom backend

* should contain leafs

* adding circom compad and circuits deps

* update windows build

* fix windows build

* improve test names

* move proving defaults to codextypes

* remove unnedded inmports and move defaults to codextypes

* capture error code on backend failure
2024-02-09 21:40:30 +00:00
Dmitriy Ryajov 825766eea0
Rework builder & sampler (#697)
* add proof initializer for poseidon2 and `$`

* fix padding and block selection

* fix sample selection and input construction

* fix sample selection & more descriptive names

* add concrete types for sampler & builder

* add missing digest calls

* use concrete types

* add sample test fixtures

* use concrete types and don't fetch dummy blocks
2024-02-08 02:27:11 +00:00
Dmitriy Ryajov 2fc538337b
rework indexing strategy to use an enum (#694) 2024-02-07 11:13:27 -08:00
Ben Bierens 0b8811c98d
Debug/sampling (#681)
* Extra logging in sampler

* wip: Fixing sampling issue in padded slot cells

* Cleanup

* Restores tests
2024-01-25 15:34:38 -08:00
Eric de88fd2c53
feat: create logging proxy (#663)
* implement a logging proxy

The logging proxy:
- prevents the need to import chronicles (as well as export except toJson),
- prevents the need to override `writeValue` or use or import nim-json-seralization elsewhere in the codebase, allowing for sole use of utils/json for de/serialization,
- and handles json formatting correctly in chronicles json sinks

* Rename logging -> logutils to avoid ambiguity with common names

* clean up

* add setProperty for JsonRecord, remove nim-json-serialization conflict

* Allow specifying textlines and json format separately

Not specifying a LogFormat will apply the formatting to both textlines and json sinks.

Specifying a LogFormat will apply the formatting to only that sink.

* remove unneeded usages of std/json

We only need to import utils/json instead of std/json

* move serialization from rest/json to utils/json so it can be shared

* fix NoColors ambiguity

Was causing unit tests to fail on Windows.

* Remove nre usage to fix Windows error

Windows was erroring with `could not load: pcre64.dll`. Instead of fixing that error, remove the pcre usage :)

* Add logutils module doc

* Shorten logutils.formatIt for `NBytes`

Both json and textlines formatIt were not needed, and could be combined into one formatIt

* remove debug integration test config

debug output and logformat of json for integration test logs

* Use ## module doc to support docgen

* bump nim-poseidon2 to export fromBytes

Before the changes in this branch, fromBytes was likely being resolved by nim-stew, or other dependency. With the changes in this branch, that dependency was removed and fromBytes could no longer be resolved. By exporting fromBytes from nim-poseidon, the correct resolution is now happening.

* fixes to get compiling after rebasing master

* Add support for Result types being logged using formatIt
2024-01-22 23:35:03 -08:00
Dmitriy Ryajov 72da534856
Wire sampler (#676)
* Setting up testfixture for proof datasampler

* Sets up calculating number of cells in a slot

* Sets up tests for bitwise modulo

* Implements cell index collection

* setting up slot blocks module

* Implements getting treeCID from slot

* implements getting slot blocks by index

* Implements out-of-range check for slot index

* cleanup

* Sets up getting sample from block

* Implements selecting a cell sample from a block

* Implements building a minitree for block cells

* Adds method to get dataset block index from slot block index

* It's running

* splits up indexing

* almost there

* Fixes test. Implementation is now functional

* Refactoring to object-oriented

* Cleanup

* Lining up output type with updated reference code.

* setting up

* Updates expected samples

* Updates proof checking test to match new format

* move builder to own dir

* move sampler to own dir

* fix paths

* various changes to add support for the sampler

* wip sampler implementation

* don't use upraises

* wip sampler integration

* misc

* move tests around

* Various fixes to select correct slot and block index

* removing old tests

* cleanup

* misc

fix tests that work with correct cell indices

* remove unused file

* fixup logging

* add logscope

* truncate entropy to 31 bytes, otherwise it might be > than mod

* forwar getCidAndProof to local store

* misc

* Adds missing test for initial-proving state

* reverting back to correct slot/block indexing

* fix tests for revert

* misc

* misc

---------

Co-authored-by: benbierens <thatbenbierens@gmail.com>
2024-01-17 11:24:34 -08:00
Dmitriy Ryajov 2fc7c75fd2
Contracts handler (#672)
* get rid of unneeded files

* don't reuse batch callback

* move out storage contract handlers

* wip

* add storage handler, to handle storage contracts interactions

* split out node tests

* a bit more cleanup

* remove storage handler, move back into node

* add missing raises:

* wip: add support for rebuilding slot

* split out tests

* wip

* rework indexing strategy to return an iterator

* convert to seq

* minor api change (meh, might revert)

* steping strategy should not die

* allow fetching batches for manifests and indicies

* restored expiry update

* restored expiry update functionality

* avoid closing datastores twice

* correct wrong rename

* fixes sigsegv
2024-01-15 08:45:04 -08:00
Dmitriy Ryajov 8b12934fe2
Build slots (#668)
Wiring in slots builder functionality into `requestStorage`
2024-01-11 08:45:23 -08:00