Commit Graph

20 Commits

Author SHA1 Message Date
Ben Bierens 4e8630791a
Rework circuit downloader (#882)
* Introduces a start method to prover

* Moves backend creation into start method

* sets up three paths for backend initialization

* Extracts backend initialization to backend-factory

* Implements loading backend from cli files or previously downloaded local files

* Wires up downloading and unzipping

* functional implementation

* Fixes testprover.nim

* Sets up tests for backendfactory

* includes libzip-dev

* pulls in updated contracts

* removes integration cli tests for r1cs, wasm, and zkey file arguments.

* Fixes issue where inner-scope values are lost before returning

* sets local proof verification for dist-test images

* Adds two traces and bumps nim-ethers

* Adds separate path for circuit files

* Create circuit dir if not exists

* fix: make sure requestStorage is mined

* fix: correct place to plug confirm

* test: fixing contracts tests

* Restores gitmodules

* restores nim-datastore reference

* Sets up downloader exe

* sets up tool skeleton

* implements getting of circuit hash

* Implements downloader tool

* sets up test skeleton

* Implements test for cirdl

* includes testTools in testAll

* Cleanup building.md

* cleans up previous downloader implementation

* cleans up testbackendfactory

* moves start of prover into node.nim

* Fills in arguments in example command

* Initializes backend in prover constructor

* Restores tests

* Restores tests for cli instructions

* Review comments by Dmitriy, part 1

* Quotes path in download instruction.

* replaces curl with chronos http session

* Moves cirdl build output to 'build' folder.

* Fixes chronicles log output

* Add cirdl support to the codex Dockerfile

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Add cirdl support to the docker entrypoint

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Add cirdl support to the release workflow

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Disable verify_circuit flag for releases

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Removes backendFactory placeholder type

* wip

* Replaces zip library with status-im/zippy library (which supports zip and tar)

* Updates cirdl to not change circuitdir folder

* Switches from zip to tar.gz

* Review comments by Dmitriy

* updates codex-contracts-eth

* Adds testTools to CI

* Adds check for access to config.circuitdir

* Update fixture circuit zkey

* Update matrix to run tools tests on Windows

* Adds 'deps' dependency for cirdl

* Adjust docker-entrypoint.sh to use CODEX_CIRCUIT_DIR env var

* Review comments by Giuliano

---------

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
Co-authored-by: Adam Uhlíř <adam@uhlir.dev>
Co-authored-by: Veaceslav Doina <20563034+veaceslavdoina@users.noreply.github.com>
2024-09-23 14:37:17 +00:00
Ben Bierens 3246c43174
Updates rust in docker. Pins circom-compat to pinned FFI branch (#818)
* Updates rust in docker. Pins circom-compat to pinned FFI branch

* Extracts builder image from codex.Dockerfile

* Updates readme

* Use single Dockerfile

* Revert readme back

---------

Co-authored-by: Veaceslav Doina <20563034+veaceslavdoina@users.noreply.github.com>
2024-05-31 08:23:11 +00:00
Ben Bierens 4038eed3c9
Copies openapi yaml into image (#751) 2024-03-25 15:23:30 +00:00
Slava 6ee1162989
Update Docker files and building Codex docs (#722)
* Make Build Codex copy/paste friendly

* Add auto-export for env var from files

* Use latest Ubuntu LTS image and install rust via package manager
2024-03-01 12:23:05 +00:00
Slava c99085c396
Set environment variables from files in Docker (#717)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-02-21 20:50:57 +00:00
Slava 79af5c0e64
Fix Docker entrypoint NAT helper variables (#706)
Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-02-15 13:57:50 +00:00
Slava 64e6a602fd
Install Rust during Docker builds (#703)
* Install Rust during Docker builds

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

* Stop CI matrix on error

---------

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2024-02-10 17:48:40 +00:00
Ben Bierens 750fe2392f
Adds option to automatically announce codex by its public IP (#621)
* Adds option to automatically announce codex by its public IP

* Review comments by Slava

* Make NAT_PUBLIC_IP_AUTO more verbose and error resistant

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>

---------

Signed-off-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
Co-authored-by: Slava <20563034+veaceslavdoina@users.noreply.github.com>
2023-11-17 13:48:25 +00:00
Ben Bierens 13de6dc500
Adds script to inject private key for the purpose of testing. (#553) 2023-09-19 09:57:39 +02:00
Vaclav Pavlin d399290ba6
chore: move the docker-compose volume (#507) 2023-09-05 16:50:47 -07:00
Slava 6dd7e55719
Enable codex_enable_api_debug_peers (#471) (#474)
- Enable codex_enable_api_debug_peers

 - Add NAT_IP_AUTO for Dist-Tests
2023-07-10 16:15:06 +03:00
Slava f053135f68
feat: add docker auto-builds (#461)
https://github.com/codex-storage/nim-codex/issues/412
2023-07-04 14:46:49 +03:00
Ben Bierens 219c7704b9
Dist-test failure due to argument name change (#451) 2023-06-19 15:35:13 +02:00
Ben Bierens db7c4b5811
Fixes docker image. (#448) 2023-06-19 11:00:21 +02:00
Ben Bierens 8c232b6759
Docker multiarch debug (#447)
* Uses correct string for marketplace address

* first steps towards support for arm64

* Applies multiarch ubuntu dockerfile as codex-dockerfile.

* Add `--simulate-proof-failures` env variable, update docker-compose to point to slimmed image

* Add image to CI, and update startCodex.sh

* Sets up separate docker build for arm

* Update arm64 arm of docker ci

* [docker] modify startCodex.sh

- include overridden node name in log output if specified in test
- quote `—log-level` value so that multiple log levels can be specified
- ensure any CLI parameter env vars are passed through to the codex binary, instead of conditionally including them
- add `—persistence`
- add `—validator`

* fixes load and push for amd docker build

---------

Co-authored-by: Eric Mastro <github@egonat.me>
2023-06-19 08:28:27 +02:00
Ben Bierens db9d90b465
Dht debug (#446)
* Support for building docker images with local modifications for the purpose of testing and debugging

* exposes peer information via debug/info

* api-names slightly kinder to json serializers

* Moves debug image-building

* fixes misalignment of debug peer info array

* Changes switchPeers source from KeyBook to AddressBook (filed ticket in libp2p, discussed with Tanguy)

* Limited success with dist-test peer discovery tests

* Removes unnecessary random-timer

* bumps dht repo. Adds peerId to formatNode

* Removes unused prints

* bumps libp2p-dht

* Exposes node address on debug api

* Adds traces

* review comments by me

* Hides debug/peers api behind compile flag

* Waiting for nim-libp2p-dht PR merge

* bumps libp2p-dht back to main after PRs were merged there.

* Cleanup
2023-06-19 08:21:03 +02:00
Ben Bierens 80106cd3d2
Creates marketplace_address.json during docker start. (#397)
* Creates marketplace_address.json during docker start.

Signed-off-by: Ben Bierens <39762930+benbierens@users.noreply.github.com>

* Enables manual start for docker workflow

---------

Signed-off-by: Ben Bierens <39762930+benbierens@users.noreply.github.com>
2023-04-18 14:17:24 +02:00
Ben Bierens b2048377e8
Makes required arguments overrideable from environment variables (#377) 2023-03-20 15:07:07 +01:00
Dmitriy Ryajov 98f6bb4ed9
fix docker to use new make command (#364) 2023-03-14 16:28:17 -06:00
Ben Bierens d263ca0735
Docker build (#354)
* Sets up working dockerized build and codex docker image creation

* Making codex configurable from the docker environment

* Sets up two networks with three codex nodes

* enables and exposes metrics endpoint for first node

* Manually performed two-client test scenario with docker containers

* Sets up docker-ignore and docker github workflow

* Wires up all codex CLI arguments to docker env vars

* Makes API_PORT variable optional as well

* Removes duplicate docker-login step

* Fixes path to docker file

* Switches target dockerhub for debugging

* Adds git tag info to --version output

* Exposes version information via debug endpoint

* Debugging docker image

* specifies target platforms for docker build

* specifies platform for QEMU and buildx steps

* Attempt to debug line endings

* Disables march-native in config.nims as test

* Applies make argument to disable architecture optimization during docker build

* Removes subset version tags from docker build

* Restore multi-arch build

* Removes docker-build test branch from CI branches
2023-03-08 12:45:55 +01:00