nim-dagger/config.nims

128 lines
5.1 KiB
Plaintext
Raw Permalink Normal View History

include "build.nims"
import std/os
const currentDir = currentSourcePath()[0 .. ^(len("config.nims") + 1)]
when getEnv("NIMBUS_BUILD_SYSTEM") == "yes" and
# BEWARE
# In Nim 1.6, config files are evaluated with a working directory
# matching where the Nim command was invocated. This means that we
# must do all file existence checks with full absolute paths:
system.fileExists(currentDir & "nimbus-build-system.paths"):
include "nimbus-build-system.paths"
when defined(release):
switch("nimcache", joinPath(currentSourcePath.parentDir, "nimcache/release/$projectName"))
2021-11-23 16:55:16 +00:00
else:
switch("nimcache", joinPath(currentSourcePath.parentDir, "nimcache/debug/$projectName"))
2021-11-23 16:55:16 +00:00
when defined(limitStackUsage):
ci: update GitHub Actions CI workflow to use msys2/setup-msys2@v2 Main goal is to update the nim-codex CI workflow to use the [`msys2/setup-msys2@v2`][setmsys2] GitHub Action, as used by the [nimbus-eth2 workflow][ne2w] per changes made to it several months ago. The `msys2/setup-msys2@v2` action has been used by other Status-org projects prior to this commit, e.g. nim-leopard, nim-datastore, nim-status. A fix is included for the failing macOS builds, related to [actions/virtual-environments#5819][ave5819]. See [L151][L151]. All builds [succeed][succeed] (with build arch-targets verified as far as possible via `ldd`, `otool`, `ntldd`), including `linux-i386` and `windows-i386`, though the `i386` builds are presently disabled (commented out). The `i386` builds can be enabled simply by uncommenting: ``` # - os: linux # cpu: i386 ... # - os: windows # cpu: i386 ``` The resulting `.github/workflows/ci.yml` is a "remix" of the current workflows for nimbus-eth2 and nim-codex (i.e. prior to this commit) along with techniques learned from developing workflows for other Status-org repos. Some comments and code-reorg help to clarify/explain what's done in the `Derive environment variables` step. `-d:limitStackUsage` has been adopted for `linux-amd64` builds from [nimbus-eth2's workflow][ne2wL155] and [related code][ne2config] has been copied into `config.nims` `-d:limitStackUsage` can easily be dropped if it's not desirable for Codex. Build targets use `-latest` for `runs-on`, i.e. `macos-latest`, `ubuntu-latest`, `windows-latest`. Through a combination of local testing and iterative pushes to GitHub, the workflow's embedded Bash scripts have been revised to include only the necessary steps for all builds to succeed, including `linux-i386` and `windows-i386`. The GitHub Actions workflow `.github/workflows/codecov.yml` has been removed, while coverage data generation/upload steps have been added to `.github/workflows/ci.yml` as the final steps conditional on `if: runner.os == 'Linux' && matrix.target.cpu == 'amd64' && matrix.nim_branch == matrix.cov_branch`. A redundant `--passC:'-m32 -mno-adx'` is used for `linux-i386` builds; the redundant flags do not affect the build, but can be helpful when eyeballing GitHub Actions builds with increased compile-time verbosity. Some variable expansions used in `github/workflows/ci.yml` could result in compilation failures if related paths include whitespace. It's not a problem for this commit but could be a problem for a user copy-pasting from the workflow; solving that problem is left as an exercise for the reader. [setmsys2]: https://github.com/msys2/setup-msys2#readme [ne2w]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml [ave5819]: https://github.com/actions/virtual-environments/pull/5819 [L151]: https://github.com/status-im/nim-codex/blob/ci/msys2/.github/workflows/ci.yml#L151 [succeed]: https://github.com/status-im/nim-codex/actions/runs/2606854455 [ne2wL155]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml#L155-L159 [ne2config]: https://github.com/status-im/nimbus-eth2/blob/stable/config.nims#L43-L49
2022-07-04 02:37:34 +00:00
# This limits stack usage of each individual function to 1MB - the option is
# available on some GCC versions but not all - run with `-d:limitStackUsage`
# and look for .su files in "./build/", "./nimcache/" or $TMPDIR that list the
# stack size of each function.
switch("passC", "-fstack-usage -Werror=stack-usage=1048576")
switch("passL", "-fstack-usage -Werror=stack-usage=1048576")
when defined(windows):
# https://github.com/nim-lang/Nim/pull/19891
switch("define", "nimRawSetjmp")
2021-11-23 16:55:16 +00:00
# disable timestamps in Windows PE headers - https://wiki.debian.org/ReproducibleBuilds/TimestampsInPEBinaries
switch("passL", "-Wl,--no-insert-timestamp")
# increase stack size
switch("passL", "-Wl,--stack,8388608")
# https://github.com/nim-lang/Nim/issues/4057
--tlsEmulation:off
if defined(i386):
# set the IMAGE_FILE_LARGE_ADDRESS_AWARE flag so we can use PAE, if enabled, and access more than 2 GiB of RAM
switch("passL", "-Wl,--large-address-aware")
# The dynamic Chronicles output currently prevents us from using colors on Windows
# because these require direct manipulations of the stdout File object.
switch("define", "chronicles_colors=off")
# This helps especially for 32-bit x86, which sans SSE2 and newer instructions
# requires quite roundabout code generation for cryptography, and other 64-bit
# and larger arithmetic use cases, along with register starvation issues. When
# engineering a more portable binary release, this should be tweaked but still
# use at least -msse2 or -msse3.
when defined(disableMarchNative):
when defined(i386) or defined(amd64):
switch("passC", "-mssse3")
elif defined(macosx) and defined(arm64):
# Apple's Clang can't handle "-march=native" on M1: https://github.com/status-im/nimbus-eth2/issues/2758
switch("passC", "-mcpu=apple-a14")
# TODO: newer Clang >=15.0 can: https://github.com/llvm/llvm-project/commit/fcca10c69aaab539962d10fcc59a5f074b73b0de
2021-11-23 16:55:16 +00:00
else:
switch("passC", "-march=native")
if defined(windows):
# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65782
# ("-fno-asynchronous-unwind-tables" breaks Nim's exception raising, sometimes)
switch("passC", "-mno-avx512vl")
Node setup (#32) * settup basic nim node * adding http utils * adding confutils * rough rest api proto * adding missing deps * turn tls emulation off * adding toml serialization * wip * adding missing deps * make sure to clean old state in teardown * adding file upload rest endpoint * renaming blockexchange to networkstore * updating nim-presto * updating libp2p * wip adding streaming upload * reworked chunking * bump to latest unstable * adding asyncfutures stream * make streamable * deleting unused files * reworking stores api * use new stores api * rework blockset and remove blockstream * don't return option from constructor * rework chunker * wip implement upload * fix tests * move unrelated logic to engine * don't print entire message * logging * basic encode/decode to/from dag-pb * add basic upload/download support * fix tests * renaming blockset to manifest * don't pass config to node * remove config and use new manifest * wip: make endpoints more reliable * wip: adding node tests * include correct manifest test * removing asyncfutures * proper chunking of files * simplify stream reading * test with encoding/decoding with many blocks * add block storing tests * adding retrieval test * add logging * tidy up chunker * tidy up manifest and node * use default chunk size * fix tests * fix tests * make sure Eof is set properly * wip * minor cleanup * add file utils * cleanup config * splitout DaggerServer and "main" * remove events since they are not used * add broadcast method to network peer * add and wire localstore * use localstore in the node * wip * logging * move file utils * use the constant * updating deps * fix memstore * use latest libp2p unstable * fix tests * rework block streaming * don't fail storing if the block already exists * add helper info endpoint * correct comment * rename localstore to fsstore * fix tests * remove unused tests * add test to retrieve one block * move some test files around * consolidate setup * Update dagger/blockexchange/engine.nim Co-authored-by: Tanguy <tanguy@status.im> * typo * better block path handling * don't inherit rootobj * remove useless template * Update tests/dagger/blockexc/testblockexc.nim Co-authored-by: markspanbroek <mark@spanbroek.net> * use isMainModule * use proper flag for starter/stoped * cleanup optional use * wrap in isMainModule * use `cancelAndAwait` * remove unused imports * wip * don't use optional * use functional error api * rework store tests and add fs tests * Block.new() to Block.init() * don't use optional for engine blocks * use result instead of optional for getBlock * remove unused imports * move stopping servers to `shutdown` * use result instead of optional * rework with results * fix tests * use waitFor in signal handlers * error helper * use `?` and mapFailure where possible * remove unnecesary `=?` * improve empty cid digest initialization Co-authored-by: Tanguy <tanguy@status.im> Co-authored-by: markspanbroek <mark@spanbroek.net>
2022-01-10 15:32:56 +00:00
--tlsEmulation:off
2021-11-23 16:55:16 +00:00
--threads:on
--opt:speed
--excessiveStackTrace:on
# enable metric collection
--define:metrics
# for heap-usage-by-instance-type metrics and object base-type strings
--define:nimTypeNames
--styleCheck:usages
--styleCheck:error
--maxLoopIterationsVM:1000000000
--fieldChecks:on
--warningAsError:"ProveField:on"
2021-11-23 16:55:16 +00:00
when (NimMajor, NimMinor) >= (1, 4):
--warning:"ObservableStores:off"
--warning:"LockLevel:off"
--hint:"XCannotRaiseY:off"
when (NimMajor, NimMinor) >= (1, 6):
--warning:"DotLikeOps:off"
when (NimMajor, NimMinor, NimPatch) >= (1, 6, 11):
--warning:"BareExcept:off"
2021-11-23 16:55:16 +00:00
switch("define", "withoutPCRE")
# the default open files limit is too low on macOS (512), breaking the
# "--debugger:native" build. It can be increased with `ulimit -n 1024`.
if not defined(macosx):
# add debugging symbols and original files and line numbers
--debugger:native
if not (defined(windows) and defined(i386)) and not defined(disable_libbacktrace):
# light-weight stack traces using libbacktrace and libunwind
--define:nimStackTraceOverride
switch("import", "libbacktrace")
# `switch("warning[CaseTransition]", "off")` fails with "Error: invalid command line option: '--warning[CaseTransition]'"
switch("warning", "CaseTransition:off")
# The compiler doth protest too much, methinks, about all these cases where it can't
# do its (N)RVO pass: https://github.com/nim-lang/RFCs/issues/230
switch("warning", "ObservableStores:off")
# Too many false positives for "Warning: method has lock level <unknown>, but another method has 0 [LockLevel]"
switch("warning", "LockLevel:off")
switch("define", "libp2p_pki_schemes=secp256k1")
#TODO this infects everything in this folder, ideally it would only
2022-05-19 19:56:03 +00:00
# apply to codex.nim, but since codex.nims is used for other purpose
# we can't use it. And codex.cfg doesn't work
switch("define", "chronicles_sinks=textlines[dynamic],json[dynamic],textlines[dynamic]")
2022-04-12 00:42:18 +00:00
# Workaround for assembler incompatibility between constantine and secp256k1
switch("define", "use_asm_syntax_intel=false")
switch("define", "ctt_asm=false")
Prover CLI updates (#735) * rework cli to accept circuit params * check circom files extension * adding new required cli changes * don't use ufcs * persistence is a command now * use `nimOldCaseObjects` switch for nim confutils compat * misc * Update cli integration tests * Fix: simulateProofFailures option is not for validator * moving circom params under `prover` command * update tests * Use circuit assets from codex-contract-eth in tests * Add "prover" cli command to tests * use correct stores * make `verifier` a cmd option * update circuit artifacts path * fix cli tests * Update integration tests to use cli commands Integration tests have been updated to use the new cli commands. The api for usage in the integration tests has also changed a bit. Proofs tests have been updated to use 5 nodes and 8 blocks of data. The remaining integration tests also need to be updated. * remove parsedCli from CodexConfig Instead, parse the cli args on the fly when needed * remove unneeded gcsafes * graceful shutdowns Where possible, do not raise assert, as other nodes in the test may already be running. Instead, raise exceptions, catch them in multinodes.nim, and attempt to do a teardown before failing the test. `abortOnError` is set to true so that `fail()` will quit immediately, after teardown has been run. * update testmarketplace to new api, with valid EC params --------- Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com> Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>
2024-03-12 09:57:13 +00:00
# Allow the use of old-style case objects for nim config compatibility
switch("define", "nimOldCaseObjects")
2022-04-12 00:42:18 +00:00
# begin Nimble config (version 1)
when system.fileExists("nimble.paths"):
include "nimble.paths"
# end Nimble config