nim-codex/.github/workflows/ci.yml

107 lines
2.5 KiB
YAML
Raw Normal View History

name: CI
on:
push:
branches:
- main
pull_request:
workflow_dispatch:
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
env:
cache_nonce: 0 # Allows for easily busting actions/cache caches
Poc 2 (#7) * moving protobuf into bitswap * adding block type * reworking bitswap * adding chunker * adding license header * use 1.2.6 * adding fixed size chunker * add blockstore * add iterator to chunker * more bitswap changes * rename ipfs to dagger * rename to dagger * blockstore inherits from BlockProvider * wip - add core block handling logic * smal changes * use proper block store methods * adding asynq heapqueue * wip prepare for bitswap task runner * adding `$` * adding memory store and tests * fixed chunking * extracted desicion engine from bitswap * added helper random funcs * adding testing helpers * only handle seqs * add peer events * cleanup pending blocks on blockstore event * allow nil handlers * move protobuf type helpers * allow initializing block from Cid * testing and fixes * small fixes * expose `<` * spelling * default value * spelling * pending blocks manager * adding stores manager * more tests a wip around bitswap * small changes * merge bitswap and engine for now * for now run only the new poc's tests * add a more complete ci setup * use template in map * remove p2pd * remove go * dont use asyncCheck * few small changes * adding ability to update items * adding multiple task runners * handle cancelation properly * use Result instead of throwing * wip bitswap tests * moving things around * split out engine again * add request and handlers interface * fix tests * wip - engine tests * remove unused imports * fix tests * cleanup block requesting logic * add block request tests * more block requests * add support for max heap * don't use result * use max heap & send block presence in task handler * add task handler tests * rename store to localStore * cleanup & logging * cancel task on stop * don't depend on local store for events * dont use heap queue for wants * add chronicles * fix issue with peer wants * add test for delayed block sends * remove obsolete tests * wip chunker * run all tests * add todo * misc * remove irrelevant files * removing more files * adding helpers for bitswap tests * moved bitswap file * misc * make blocks timeout longer * adjust block timeout * speedup test * compile with threads * import missing crypto * misc * disable threads for now * fix 32 bit platforms * re-enable threads support in tests
2021-02-26 00:23:22 +00:00
jobs:
build:
strategy:
matrix:
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
os: [linux, macos, windows]
Poc 2 (#7) * moving protobuf into bitswap * adding block type * reworking bitswap * adding chunker * adding license header * use 1.2.6 * adding fixed size chunker * add blockstore * add iterator to chunker * more bitswap changes * rename ipfs to dagger * rename to dagger * blockstore inherits from BlockProvider * wip - add core block handling logic * smal changes * use proper block store methods * adding asynq heapqueue * wip prepare for bitswap task runner * adding `$` * adding memory store and tests * fixed chunking * extracted desicion engine from bitswap * added helper random funcs * adding testing helpers * only handle seqs * add peer events * cleanup pending blocks on blockstore event * allow nil handlers * move protobuf type helpers * allow initializing block from Cid * testing and fixes * small fixes * expose `<` * spelling * default value * spelling * pending blocks manager * adding stores manager * more tests a wip around bitswap * small changes * merge bitswap and engine for now * for now run only the new poc's tests * add a more complete ci setup * use template in map * remove p2pd * remove go * dont use asyncCheck * few small changes * adding ability to update items * adding multiple task runners * handle cancelation properly * use Result instead of throwing * wip bitswap tests * moving things around * split out engine again * add request and handlers interface * fix tests * wip - engine tests * remove unused imports * fix tests * cleanup block requesting logic * add block request tests * more block requests * add support for max heap * don't use result * use max heap & send block presence in task handler * add task handler tests * rename store to localStore * cleanup & logging * cancel task on stop * don't depend on local store for events * dont use heap queue for wants * add chronicles * fix issue with peer wants * add test for delayed block sends * remove obsolete tests * wip chunker * run all tests * add todo * misc * remove irrelevant files * removing more files * adding helpers for bitswap tests * moved bitswap file * misc * make blocks timeout longer * adjust block timeout * speedup test * compile with threads * import missing crypto * misc * disable threads for now * fix 32 bit platforms * re-enable threads support in tests
2021-02-26 00:23:22 +00:00
include:
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
- os: linux
ci: update GitHub Actions CI workflow to use msys2/setup-msys2@v2 Main goal is to update the nim-codex CI workflow to use the [`msys2/setup-msys2@v2`][setmsys2] GitHub Action, as used by the [nimbus-eth2 workflow][ne2w] per changes made to it several months ago. The `msys2/setup-msys2@v2` action has been used by other Status-org projects prior to this commit, e.g. nim-leopard, nim-datastore, nim-status. A fix is included for the failing macOS builds, related to [actions/virtual-environments#5819][ave5819]. See [L151][L151]. All builds [succeed][succeed] (with build arch-targets verified as far as possible via `ldd`, `otool`, `ntldd`), including `linux-i386` and `windows-i386`, though the `i386` builds are presently disabled (commented out). The `i386` builds can be enabled simply by uncommenting: ``` # - os: linux # cpu: i386 ... # - os: windows # cpu: i386 ``` The resulting `.github/workflows/ci.yml` is a "remix" of the current workflows for nimbus-eth2 and nim-codex (i.e. prior to this commit) along with techniques learned from developing workflows for other Status-org repos. Some comments and code-reorg help to clarify/explain what's done in the `Derive environment variables` step. `-d:limitStackUsage` has been adopted for `linux-amd64` builds from [nimbus-eth2's workflow][ne2wL155] and [related code][ne2config] has been copied into `config.nims` `-d:limitStackUsage` can easily be dropped if it's not desirable for Codex. Build targets use `-latest` for `runs-on`, i.e. `macos-latest`, `ubuntu-latest`, `windows-latest`. Through a combination of local testing and iterative pushes to GitHub, the workflow's embedded Bash scripts have been revised to include only the necessary steps for all builds to succeed, including `linux-i386` and `windows-i386`. The GitHub Actions workflow `.github/workflows/codecov.yml` has been removed, while coverage data generation/upload steps have been added to `.github/workflows/ci.yml` as the final steps conditional on `if: runner.os == 'Linux' && matrix.target.cpu == 'amd64' && matrix.nim_branch == matrix.cov_branch`. A redundant `--passC:'-m32 -mno-adx'` is used for `linux-i386` builds; the redundant flags do not affect the build, but can be helpful when eyeballing GitHub Actions builds with increased compile-time verbosity. Some variable expansions used in `github/workflows/ci.yml` could result in compilation failures if related paths include whitespace. It's not a problem for this commit but could be a problem for a user copy-pasting from the workflow; solving that problem is left as an exercise for the reader. [setmsys2]: https://github.com/msys2/setup-msys2#readme [ne2w]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml [ave5819]: https://github.com/actions/virtual-environments/pull/5819 [L151]: https://github.com/status-im/nim-codex/blob/ci/msys2/.github/workflows/ci.yml#L151 [succeed]: https://github.com/status-im/nim-codex/actions/runs/2606854455 [ne2wL155]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml#L155-L159 [ne2config]: https://github.com/status-im/nimbus-eth2/blob/stable/config.nims#L43-L49
2022-07-04 02:37:34 +00:00
builder: ubuntu-latest
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
shell: bash --noprofile --norc -e -o pipefail
- os: macos
ci: update GitHub Actions CI workflow to use msys2/setup-msys2@v2 Main goal is to update the nim-codex CI workflow to use the [`msys2/setup-msys2@v2`][setmsys2] GitHub Action, as used by the [nimbus-eth2 workflow][ne2w] per changes made to it several months ago. The `msys2/setup-msys2@v2` action has been used by other Status-org projects prior to this commit, e.g. nim-leopard, nim-datastore, nim-status. A fix is included for the failing macOS builds, related to [actions/virtual-environments#5819][ave5819]. See [L151][L151]. All builds [succeed][succeed] (with build arch-targets verified as far as possible via `ldd`, `otool`, `ntldd`), including `linux-i386` and `windows-i386`, though the `i386` builds are presently disabled (commented out). The `i386` builds can be enabled simply by uncommenting: ``` # - os: linux # cpu: i386 ... # - os: windows # cpu: i386 ``` The resulting `.github/workflows/ci.yml` is a "remix" of the current workflows for nimbus-eth2 and nim-codex (i.e. prior to this commit) along with techniques learned from developing workflows for other Status-org repos. Some comments and code-reorg help to clarify/explain what's done in the `Derive environment variables` step. `-d:limitStackUsage` has been adopted for `linux-amd64` builds from [nimbus-eth2's workflow][ne2wL155] and [related code][ne2config] has been copied into `config.nims` `-d:limitStackUsage` can easily be dropped if it's not desirable for Codex. Build targets use `-latest` for `runs-on`, i.e. `macos-latest`, `ubuntu-latest`, `windows-latest`. Through a combination of local testing and iterative pushes to GitHub, the workflow's embedded Bash scripts have been revised to include only the necessary steps for all builds to succeed, including `linux-i386` and `windows-i386`. The GitHub Actions workflow `.github/workflows/codecov.yml` has been removed, while coverage data generation/upload steps have been added to `.github/workflows/ci.yml` as the final steps conditional on `if: runner.os == 'Linux' && matrix.target.cpu == 'amd64' && matrix.nim_branch == matrix.cov_branch`. A redundant `--passC:'-m32 -mno-adx'` is used for `linux-i386` builds; the redundant flags do not affect the build, but can be helpful when eyeballing GitHub Actions builds with increased compile-time verbosity. Some variable expansions used in `github/workflows/ci.yml` could result in compilation failures if related paths include whitespace. It's not a problem for this commit but could be a problem for a user copy-pasting from the workflow; solving that problem is left as an exercise for the reader. [setmsys2]: https://github.com/msys2/setup-msys2#readme [ne2w]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml [ave5819]: https://github.com/actions/virtual-environments/pull/5819 [L151]: https://github.com/status-im/nim-codex/blob/ci/msys2/.github/workflows/ci.yml#L151 [succeed]: https://github.com/status-im/nim-codex/actions/runs/2606854455 [ne2wL155]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml#L155-L159 [ne2config]: https://github.com/status-im/nimbus-eth2/blob/stable/config.nims#L43-L49
2022-07-04 02:37:34 +00:00
builder: macos-latest
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
shell: bash --noprofile --norc -e -o pipefail
- os: windows
ci: update GitHub Actions CI workflow to use msys2/setup-msys2@v2 Main goal is to update the nim-codex CI workflow to use the [`msys2/setup-msys2@v2`][setmsys2] GitHub Action, as used by the [nimbus-eth2 workflow][ne2w] per changes made to it several months ago. The `msys2/setup-msys2@v2` action has been used by other Status-org projects prior to this commit, e.g. nim-leopard, nim-datastore, nim-status. A fix is included for the failing macOS builds, related to [actions/virtual-environments#5819][ave5819]. See [L151][L151]. All builds [succeed][succeed] (with build arch-targets verified as far as possible via `ldd`, `otool`, `ntldd`), including `linux-i386` and `windows-i386`, though the `i386` builds are presently disabled (commented out). The `i386` builds can be enabled simply by uncommenting: ``` # - os: linux # cpu: i386 ... # - os: windows # cpu: i386 ``` The resulting `.github/workflows/ci.yml` is a "remix" of the current workflows for nimbus-eth2 and nim-codex (i.e. prior to this commit) along with techniques learned from developing workflows for other Status-org repos. Some comments and code-reorg help to clarify/explain what's done in the `Derive environment variables` step. `-d:limitStackUsage` has been adopted for `linux-amd64` builds from [nimbus-eth2's workflow][ne2wL155] and [related code][ne2config] has been copied into `config.nims` `-d:limitStackUsage` can easily be dropped if it's not desirable for Codex. Build targets use `-latest` for `runs-on`, i.e. `macos-latest`, `ubuntu-latest`, `windows-latest`. Through a combination of local testing and iterative pushes to GitHub, the workflow's embedded Bash scripts have been revised to include only the necessary steps for all builds to succeed, including `linux-i386` and `windows-i386`. The GitHub Actions workflow `.github/workflows/codecov.yml` has been removed, while coverage data generation/upload steps have been added to `.github/workflows/ci.yml` as the final steps conditional on `if: runner.os == 'Linux' && matrix.target.cpu == 'amd64' && matrix.nim_branch == matrix.cov_branch`. A redundant `--passC:'-m32 -mno-adx'` is used for `linux-i386` builds; the redundant flags do not affect the build, but can be helpful when eyeballing GitHub Actions builds with increased compile-time verbosity. Some variable expansions used in `github/workflows/ci.yml` could result in compilation failures if related paths include whitespace. It's not a problem for this commit but could be a problem for a user copy-pasting from the workflow; solving that problem is left as an exercise for the reader. [setmsys2]: https://github.com/msys2/setup-msys2#readme [ne2w]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml [ave5819]: https://github.com/actions/virtual-environments/pull/5819 [L151]: https://github.com/status-im/nim-codex/blob/ci/msys2/.github/workflows/ci.yml#L151 [succeed]: https://github.com/status-im/nim-codex/actions/runs/2606854455 [ne2wL155]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml#L155-L159 [ne2config]: https://github.com/status-im/nimbus-eth2/blob/stable/config.nims#L43-L49
2022-07-04 02:37:34 +00:00
builder: windows-latest
shell: msys2
defaults:
run:
shell: ${{ matrix.shell }} {0}
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
name: '${{ matrix.os }}'
Poc 2 (#7) * moving protobuf into bitswap * adding block type * reworking bitswap * adding chunker * adding license header * use 1.2.6 * adding fixed size chunker * add blockstore * add iterator to chunker * more bitswap changes * rename ipfs to dagger * rename to dagger * blockstore inherits from BlockProvider * wip - add core block handling logic * smal changes * use proper block store methods * adding asynq heapqueue * wip prepare for bitswap task runner * adding `$` * adding memory store and tests * fixed chunking * extracted desicion engine from bitswap * added helper random funcs * adding testing helpers * only handle seqs * add peer events * cleanup pending blocks on blockstore event * allow nil handlers * move protobuf type helpers * allow initializing block from Cid * testing and fixes * small fixes * expose `<` * spelling * default value * spelling * pending blocks manager * adding stores manager * more tests a wip around bitswap * small changes * merge bitswap and engine for now * for now run only the new poc's tests * add a more complete ci setup * use template in map * remove p2pd * remove go * dont use asyncCheck * few small changes * adding ability to update items * adding multiple task runners * handle cancelation properly * use Result instead of throwing * wip bitswap tests * moving things around * split out engine again * add request and handlers interface * fix tests * wip - engine tests * remove unused imports * fix tests * cleanup block requesting logic * add block request tests * more block requests * add support for max heap * don't use result * use max heap & send block presence in task handler * add task handler tests * rename store to localStore * cleanup & logging * cancel task on stop * don't depend on local store for events * dont use heap queue for wants * add chronicles * fix issue with peer wants * add test for delayed block sends * remove obsolete tests * wip chunker * run all tests * add todo * misc * remove irrelevant files * removing more files * adding helpers for bitswap tests * moved bitswap file * misc * make blocks timeout longer * adjust block timeout * speedup test * compile with threads * import missing crypto * misc * disable threads for now * fix 32 bit platforms * re-enable threads support in tests
2021-02-26 00:23:22 +00:00
runs-on: ${{ matrix.builder }}
ci: update GitHub Actions CI workflow to use msys2/setup-msys2@v2 Main goal is to update the nim-codex CI workflow to use the [`msys2/setup-msys2@v2`][setmsys2] GitHub Action, as used by the [nimbus-eth2 workflow][ne2w] per changes made to it several months ago. The `msys2/setup-msys2@v2` action has been used by other Status-org projects prior to this commit, e.g. nim-leopard, nim-datastore, nim-status. A fix is included for the failing macOS builds, related to [actions/virtual-environments#5819][ave5819]. See [L151][L151]. All builds [succeed][succeed] (with build arch-targets verified as far as possible via `ldd`, `otool`, `ntldd`), including `linux-i386` and `windows-i386`, though the `i386` builds are presently disabled (commented out). The `i386` builds can be enabled simply by uncommenting: ``` # - os: linux # cpu: i386 ... # - os: windows # cpu: i386 ``` The resulting `.github/workflows/ci.yml` is a "remix" of the current workflows for nimbus-eth2 and nim-codex (i.e. prior to this commit) along with techniques learned from developing workflows for other Status-org repos. Some comments and code-reorg help to clarify/explain what's done in the `Derive environment variables` step. `-d:limitStackUsage` has been adopted for `linux-amd64` builds from [nimbus-eth2's workflow][ne2wL155] and [related code][ne2config] has been copied into `config.nims` `-d:limitStackUsage` can easily be dropped if it's not desirable for Codex. Build targets use `-latest` for `runs-on`, i.e. `macos-latest`, `ubuntu-latest`, `windows-latest`. Through a combination of local testing and iterative pushes to GitHub, the workflow's embedded Bash scripts have been revised to include only the necessary steps for all builds to succeed, including `linux-i386` and `windows-i386`. The GitHub Actions workflow `.github/workflows/codecov.yml` has been removed, while coverage data generation/upload steps have been added to `.github/workflows/ci.yml` as the final steps conditional on `if: runner.os == 'Linux' && matrix.target.cpu == 'amd64' && matrix.nim_branch == matrix.cov_branch`. A redundant `--passC:'-m32 -mno-adx'` is used for `linux-i386` builds; the redundant flags do not affect the build, but can be helpful when eyeballing GitHub Actions builds with increased compile-time verbosity. Some variable expansions used in `github/workflows/ci.yml` could result in compilation failures if related paths include whitespace. It's not a problem for this commit but could be a problem for a user copy-pasting from the workflow; solving that problem is left as an exercise for the reader. [setmsys2]: https://github.com/msys2/setup-msys2#readme [ne2w]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml [ave5819]: https://github.com/actions/virtual-environments/pull/5819 [L151]: https://github.com/status-im/nim-codex/blob/ci/msys2/.github/workflows/ci.yml#L151 [succeed]: https://github.com/status-im/nim-codex/actions/runs/2606854455 [ne2wL155]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml#L155-L159 [ne2config]: https://github.com/status-im/nimbus-eth2/blob/stable/config.nims#L43-L49
2022-07-04 02:37:34 +00:00
timeout-minutes: 80
Poc 2 (#7) * moving protobuf into bitswap * adding block type * reworking bitswap * adding chunker * adding license header * use 1.2.6 * adding fixed size chunker * add blockstore * add iterator to chunker * more bitswap changes * rename ipfs to dagger * rename to dagger * blockstore inherits from BlockProvider * wip - add core block handling logic * smal changes * use proper block store methods * adding asynq heapqueue * wip prepare for bitswap task runner * adding `$` * adding memory store and tests * fixed chunking * extracted desicion engine from bitswap * added helper random funcs * adding testing helpers * only handle seqs * add peer events * cleanup pending blocks on blockstore event * allow nil handlers * move protobuf type helpers * allow initializing block from Cid * testing and fixes * small fixes * expose `<` * spelling * default value * spelling * pending blocks manager * adding stores manager * more tests a wip around bitswap * small changes * merge bitswap and engine for now * for now run only the new poc's tests * add a more complete ci setup * use template in map * remove p2pd * remove go * dont use asyncCheck * few small changes * adding ability to update items * adding multiple task runners * handle cancelation properly * use Result instead of throwing * wip bitswap tests * moving things around * split out engine again * add request and handlers interface * fix tests * wip - engine tests * remove unused imports * fix tests * cleanup block requesting logic * add block request tests * more block requests * add support for max heap * don't use result * use max heap & send block presence in task handler * add task handler tests * rename store to localStore * cleanup & logging * cancel task on stop * don't depend on local store for events * dont use heap queue for wants * add chronicles * fix issue with peer wants * add test for delayed block sends * remove obsolete tests * wip chunker * run all tests * add todo * misc * remove irrelevant files * removing more files * adding helpers for bitswap tests * moved bitswap file * misc * make blocks timeout longer * adjust block timeout * speedup test * compile with threads * import missing crypto * misc * disable threads for now * fix 32 bit platforms * re-enable threads support in tests
2021-02-26 00:23:22 +00:00
steps:
ci: update GitHub Actions CI workflow to use msys2/setup-msys2@v2 Main goal is to update the nim-codex CI workflow to use the [`msys2/setup-msys2@v2`][setmsys2] GitHub Action, as used by the [nimbus-eth2 workflow][ne2w] per changes made to it several months ago. The `msys2/setup-msys2@v2` action has been used by other Status-org projects prior to this commit, e.g. nim-leopard, nim-datastore, nim-status. A fix is included for the failing macOS builds, related to [actions/virtual-environments#5819][ave5819]. See [L151][L151]. All builds [succeed][succeed] (with build arch-targets verified as far as possible via `ldd`, `otool`, `ntldd`), including `linux-i386` and `windows-i386`, though the `i386` builds are presently disabled (commented out). The `i386` builds can be enabled simply by uncommenting: ``` # - os: linux # cpu: i386 ... # - os: windows # cpu: i386 ``` The resulting `.github/workflows/ci.yml` is a "remix" of the current workflows for nimbus-eth2 and nim-codex (i.e. prior to this commit) along with techniques learned from developing workflows for other Status-org repos. Some comments and code-reorg help to clarify/explain what's done in the `Derive environment variables` step. `-d:limitStackUsage` has been adopted for `linux-amd64` builds from [nimbus-eth2's workflow][ne2wL155] and [related code][ne2config] has been copied into `config.nims` `-d:limitStackUsage` can easily be dropped if it's not desirable for Codex. Build targets use `-latest` for `runs-on`, i.e. `macos-latest`, `ubuntu-latest`, `windows-latest`. Through a combination of local testing and iterative pushes to GitHub, the workflow's embedded Bash scripts have been revised to include only the necessary steps for all builds to succeed, including `linux-i386` and `windows-i386`. The GitHub Actions workflow `.github/workflows/codecov.yml` has been removed, while coverage data generation/upload steps have been added to `.github/workflows/ci.yml` as the final steps conditional on `if: runner.os == 'Linux' && matrix.target.cpu == 'amd64' && matrix.nim_branch == matrix.cov_branch`. A redundant `--passC:'-m32 -mno-adx'` is used for `linux-i386` builds; the redundant flags do not affect the build, but can be helpful when eyeballing GitHub Actions builds with increased compile-time verbosity. Some variable expansions used in `github/workflows/ci.yml` could result in compilation failures if related paths include whitespace. It's not a problem for this commit but could be a problem for a user copy-pasting from the workflow; solving that problem is left as an exercise for the reader. [setmsys2]: https://github.com/msys2/setup-msys2#readme [ne2w]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml [ave5819]: https://github.com/actions/virtual-environments/pull/5819 [L151]: https://github.com/status-im/nim-codex/blob/ci/msys2/.github/workflows/ci.yml#L151 [succeed]: https://github.com/status-im/nim-codex/actions/runs/2606854455 [ne2wL155]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml#L155-L159 [ne2config]: https://github.com/status-im/nimbus-eth2/blob/stable/config.nims#L43-L49
2022-07-04 02:37:34 +00:00
- name: Checkout sources
uses: actions/checkout@v3
with:
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
submodules: recursive
ci: update GitHub Actions CI workflow to use msys2/setup-msys2@v2 Main goal is to update the nim-codex CI workflow to use the [`msys2/setup-msys2@v2`][setmsys2] GitHub Action, as used by the [nimbus-eth2 workflow][ne2w] per changes made to it several months ago. The `msys2/setup-msys2@v2` action has been used by other Status-org projects prior to this commit, e.g. nim-leopard, nim-datastore, nim-status. A fix is included for the failing macOS builds, related to [actions/virtual-environments#5819][ave5819]. See [L151][L151]. All builds [succeed][succeed] (with build arch-targets verified as far as possible via `ldd`, `otool`, `ntldd`), including `linux-i386` and `windows-i386`, though the `i386` builds are presently disabled (commented out). The `i386` builds can be enabled simply by uncommenting: ``` # - os: linux # cpu: i386 ... # - os: windows # cpu: i386 ``` The resulting `.github/workflows/ci.yml` is a "remix" of the current workflows for nimbus-eth2 and nim-codex (i.e. prior to this commit) along with techniques learned from developing workflows for other Status-org repos. Some comments and code-reorg help to clarify/explain what's done in the `Derive environment variables` step. `-d:limitStackUsage` has been adopted for `linux-amd64` builds from [nimbus-eth2's workflow][ne2wL155] and [related code][ne2config] has been copied into `config.nims` `-d:limitStackUsage` can easily be dropped if it's not desirable for Codex. Build targets use `-latest` for `runs-on`, i.e. `macos-latest`, `ubuntu-latest`, `windows-latest`. Through a combination of local testing and iterative pushes to GitHub, the workflow's embedded Bash scripts have been revised to include only the necessary steps for all builds to succeed, including `linux-i386` and `windows-i386`. The GitHub Actions workflow `.github/workflows/codecov.yml` has been removed, while coverage data generation/upload steps have been added to `.github/workflows/ci.yml` as the final steps conditional on `if: runner.os == 'Linux' && matrix.target.cpu == 'amd64' && matrix.nim_branch == matrix.cov_branch`. A redundant `--passC:'-m32 -mno-adx'` is used for `linux-i386` builds; the redundant flags do not affect the build, but can be helpful when eyeballing GitHub Actions builds with increased compile-time verbosity. Some variable expansions used in `github/workflows/ci.yml` could result in compilation failures if related paths include whitespace. It's not a problem for this commit but could be a problem for a user copy-pasting from the workflow; solving that problem is left as an exercise for the reader. [setmsys2]: https://github.com/msys2/setup-msys2#readme [ne2w]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml [ave5819]: https://github.com/actions/virtual-environments/pull/5819 [L151]: https://github.com/status-im/nim-codex/blob/ci/msys2/.github/workflows/ci.yml#L151 [succeed]: https://github.com/status-im/nim-codex/actions/runs/2606854455 [ne2wL155]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml#L155-L159 [ne2config]: https://github.com/status-im/nimbus-eth2/blob/stable/config.nims#L43-L49
2022-07-04 02:37:34 +00:00
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
- name: Setup Nimbus Build System
uses: ./.github/actions/nimbus-build-system
with:
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
os: ${{ matrix.os }}
shell: ${{ matrix.shell }}
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
- name: Unit tests
run: make -j${ncpu} test
Poc 2 (#7) * moving protobuf into bitswap * adding block type * reworking bitswap * adding chunker * adding license header * use 1.2.6 * adding fixed size chunker * add blockstore * add iterator to chunker * more bitswap changes * rename ipfs to dagger * rename to dagger * blockstore inherits from BlockProvider * wip - add core block handling logic * smal changes * use proper block store methods * adding asynq heapqueue * wip prepare for bitswap task runner * adding `$` * adding memory store and tests * fixed chunking * extracted desicion engine from bitswap * added helper random funcs * adding testing helpers * only handle seqs * add peer events * cleanup pending blocks on blockstore event * allow nil handlers * move protobuf type helpers * allow initializing block from Cid * testing and fixes * small fixes * expose `<` * spelling * default value * spelling * pending blocks manager * adding stores manager * more tests a wip around bitswap * small changes * merge bitswap and engine for now * for now run only the new poc's tests * add a more complete ci setup * use template in map * remove p2pd * remove go * dont use asyncCheck * few small changes * adding ability to update items * adding multiple task runners * handle cancelation properly * use Result instead of throwing * wip bitswap tests * moving things around * split out engine again * add request and handlers interface * fix tests * wip - engine tests * remove unused imports * fix tests * cleanup block requesting logic * add block request tests * more block requests * add support for max heap * don't use result * use max heap & send block presence in task handler * add task handler tests * rename store to localStore * cleanup & logging * cancel task on stop * don't depend on local store for events * dont use heap queue for wants * add chronicles * fix issue with peer wants * add test for delayed block sends * remove obsolete tests * wip chunker * run all tests * add todo * misc * remove irrelevant files * removing more files * adding helpers for bitswap tests * moved bitswap file * misc * make blocks timeout longer * adjust block timeout * speedup test * compile with threads * import missing crypto * misc * disable threads for now * fix 32 bit platforms * re-enable threads support in tests
2021-02-26 00:23:22 +00:00
ci: update GitHub Actions CI workflow to use msys2/setup-msys2@v2 Main goal is to update the nim-codex CI workflow to use the [`msys2/setup-msys2@v2`][setmsys2] GitHub Action, as used by the [nimbus-eth2 workflow][ne2w] per changes made to it several months ago. The `msys2/setup-msys2@v2` action has been used by other Status-org projects prior to this commit, e.g. nim-leopard, nim-datastore, nim-status. A fix is included for the failing macOS builds, related to [actions/virtual-environments#5819][ave5819]. See [L151][L151]. All builds [succeed][succeed] (with build arch-targets verified as far as possible via `ldd`, `otool`, `ntldd`), including `linux-i386` and `windows-i386`, though the `i386` builds are presently disabled (commented out). The `i386` builds can be enabled simply by uncommenting: ``` # - os: linux # cpu: i386 ... # - os: windows # cpu: i386 ``` The resulting `.github/workflows/ci.yml` is a "remix" of the current workflows for nimbus-eth2 and nim-codex (i.e. prior to this commit) along with techniques learned from developing workflows for other Status-org repos. Some comments and code-reorg help to clarify/explain what's done in the `Derive environment variables` step. `-d:limitStackUsage` has been adopted for `linux-amd64` builds from [nimbus-eth2's workflow][ne2wL155] and [related code][ne2config] has been copied into `config.nims` `-d:limitStackUsage` can easily be dropped if it's not desirable for Codex. Build targets use `-latest` for `runs-on`, i.e. `macos-latest`, `ubuntu-latest`, `windows-latest`. Through a combination of local testing and iterative pushes to GitHub, the workflow's embedded Bash scripts have been revised to include only the necessary steps for all builds to succeed, including `linux-i386` and `windows-i386`. The GitHub Actions workflow `.github/workflows/codecov.yml` has been removed, while coverage data generation/upload steps have been added to `.github/workflows/ci.yml` as the final steps conditional on `if: runner.os == 'Linux' && matrix.target.cpu == 'amd64' && matrix.nim_branch == matrix.cov_branch`. A redundant `--passC:'-m32 -mno-adx'` is used for `linux-i386` builds; the redundant flags do not affect the build, but can be helpful when eyeballing GitHub Actions builds with increased compile-time verbosity. Some variable expansions used in `github/workflows/ci.yml` could result in compilation failures if related paths include whitespace. It's not a problem for this commit but could be a problem for a user copy-pasting from the workflow; solving that problem is left as an exercise for the reader. [setmsys2]: https://github.com/msys2/setup-msys2#readme [ne2w]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml [ave5819]: https://github.com/actions/virtual-environments/pull/5819 [L151]: https://github.com/status-im/nim-codex/blob/ci/msys2/.github/workflows/ci.yml#L151 [succeed]: https://github.com/status-im/nim-codex/actions/runs/2606854455 [ne2wL155]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml#L155-L159 [ne2config]: https://github.com/status-im/nimbus-eth2/blob/stable/config.nims#L43-L49
2022-07-04 02:37:34 +00:00
- name: Start Ethereum node with Codex contracts
working-directory: vendor/codex-contracts-eth
run: |
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
if [[ '${{ matrix.os }}' == 'windows' ]]; then
export PATH="${PATH}:/c/program files/nodejs"
ci: update GitHub Actions CI workflow to use msys2/setup-msys2@v2 Main goal is to update the nim-codex CI workflow to use the [`msys2/setup-msys2@v2`][setmsys2] GitHub Action, as used by the [nimbus-eth2 workflow][ne2w] per changes made to it several months ago. The `msys2/setup-msys2@v2` action has been used by other Status-org projects prior to this commit, e.g. nim-leopard, nim-datastore, nim-status. A fix is included for the failing macOS builds, related to [actions/virtual-environments#5819][ave5819]. See [L151][L151]. All builds [succeed][succeed] (with build arch-targets verified as far as possible via `ldd`, `otool`, `ntldd`), including `linux-i386` and `windows-i386`, though the `i386` builds are presently disabled (commented out). The `i386` builds can be enabled simply by uncommenting: ``` # - os: linux # cpu: i386 ... # - os: windows # cpu: i386 ``` The resulting `.github/workflows/ci.yml` is a "remix" of the current workflows for nimbus-eth2 and nim-codex (i.e. prior to this commit) along with techniques learned from developing workflows for other Status-org repos. Some comments and code-reorg help to clarify/explain what's done in the `Derive environment variables` step. `-d:limitStackUsage` has been adopted for `linux-amd64` builds from [nimbus-eth2's workflow][ne2wL155] and [related code][ne2config] has been copied into `config.nims` `-d:limitStackUsage` can easily be dropped if it's not desirable for Codex. Build targets use `-latest` for `runs-on`, i.e. `macos-latest`, `ubuntu-latest`, `windows-latest`. Through a combination of local testing and iterative pushes to GitHub, the workflow's embedded Bash scripts have been revised to include only the necessary steps for all builds to succeed, including `linux-i386` and `windows-i386`. The GitHub Actions workflow `.github/workflows/codecov.yml` has been removed, while coverage data generation/upload steps have been added to `.github/workflows/ci.yml` as the final steps conditional on `if: runner.os == 'Linux' && matrix.target.cpu == 'amd64' && matrix.nim_branch == matrix.cov_branch`. A redundant `--passC:'-m32 -mno-adx'` is used for `linux-i386` builds; the redundant flags do not affect the build, but can be helpful when eyeballing GitHub Actions builds with increased compile-time verbosity. Some variable expansions used in `github/workflows/ci.yml` could result in compilation failures if related paths include whitespace. It's not a problem for this commit but could be a problem for a user copy-pasting from the workflow; solving that problem is left as an exercise for the reader. [setmsys2]: https://github.com/msys2/setup-msys2#readme [ne2w]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml [ave5819]: https://github.com/actions/virtual-environments/pull/5819 [L151]: https://github.com/status-im/nim-codex/blob/ci/msys2/.github/workflows/ci.yml#L151 [succeed]: https://github.com/status-im/nim-codex/actions/runs/2606854455 [ne2wL155]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml#L155-L159 [ne2config]: https://github.com/status-im/nimbus-eth2/blob/stable/config.nims#L43-L49
2022-07-04 02:37:34 +00:00
fi
npm install
npm start &
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
- name: Contract tests
run: make -j${ncpu} testContracts
ci: update GitHub Actions CI workflow to use msys2/setup-msys2@v2 Main goal is to update the nim-codex CI workflow to use the [`msys2/setup-msys2@v2`][setmsys2] GitHub Action, as used by the [nimbus-eth2 workflow][ne2w] per changes made to it several months ago. The `msys2/setup-msys2@v2` action has been used by other Status-org projects prior to this commit, e.g. nim-leopard, nim-datastore, nim-status. A fix is included for the failing macOS builds, related to [actions/virtual-environments#5819][ave5819]. See [L151][L151]. All builds [succeed][succeed] (with build arch-targets verified as far as possible via `ldd`, `otool`, `ntldd`), including `linux-i386` and `windows-i386`, though the `i386` builds are presently disabled (commented out). The `i386` builds can be enabled simply by uncommenting: ``` # - os: linux # cpu: i386 ... # - os: windows # cpu: i386 ``` The resulting `.github/workflows/ci.yml` is a "remix" of the current workflows for nimbus-eth2 and nim-codex (i.e. prior to this commit) along with techniques learned from developing workflows for other Status-org repos. Some comments and code-reorg help to clarify/explain what's done in the `Derive environment variables` step. `-d:limitStackUsage` has been adopted for `linux-amd64` builds from [nimbus-eth2's workflow][ne2wL155] and [related code][ne2config] has been copied into `config.nims` `-d:limitStackUsage` can easily be dropped if it's not desirable for Codex. Build targets use `-latest` for `runs-on`, i.e. `macos-latest`, `ubuntu-latest`, `windows-latest`. Through a combination of local testing and iterative pushes to GitHub, the workflow's embedded Bash scripts have been revised to include only the necessary steps for all builds to succeed, including `linux-i386` and `windows-i386`. The GitHub Actions workflow `.github/workflows/codecov.yml` has been removed, while coverage data generation/upload steps have been added to `.github/workflows/ci.yml` as the final steps conditional on `if: runner.os == 'Linux' && matrix.target.cpu == 'amd64' && matrix.nim_branch == matrix.cov_branch`. A redundant `--passC:'-m32 -mno-adx'` is used for `linux-i386` builds; the redundant flags do not affect the build, but can be helpful when eyeballing GitHub Actions builds with increased compile-time verbosity. Some variable expansions used in `github/workflows/ci.yml` could result in compilation failures if related paths include whitespace. It's not a problem for this commit but could be a problem for a user copy-pasting from the workflow; solving that problem is left as an exercise for the reader. [setmsys2]: https://github.com/msys2/setup-msys2#readme [ne2w]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml [ave5819]: https://github.com/actions/virtual-environments/pull/5819 [L151]: https://github.com/status-im/nim-codex/blob/ci/msys2/.github/workflows/ci.yml#L151 [succeed]: https://github.com/status-im/nim-codex/actions/runs/2606854455 [ne2wL155]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml#L155-L159 [ne2config]: https://github.com/status-im/nimbus-eth2/blob/stable/config.nims#L43-L49
2022-07-04 02:37:34 +00:00
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
- name: Integration tests
run: make -j${ncpu} testIntegration
ci: update GitHub Actions CI workflow to use msys2/setup-msys2@v2 Main goal is to update the nim-codex CI workflow to use the [`msys2/setup-msys2@v2`][setmsys2] GitHub Action, as used by the [nimbus-eth2 workflow][ne2w] per changes made to it several months ago. The `msys2/setup-msys2@v2` action has been used by other Status-org projects prior to this commit, e.g. nim-leopard, nim-datastore, nim-status. A fix is included for the failing macOS builds, related to [actions/virtual-environments#5819][ave5819]. See [L151][L151]. All builds [succeed][succeed] (with build arch-targets verified as far as possible via `ldd`, `otool`, `ntldd`), including `linux-i386` and `windows-i386`, though the `i386` builds are presently disabled (commented out). The `i386` builds can be enabled simply by uncommenting: ``` # - os: linux # cpu: i386 ... # - os: windows # cpu: i386 ``` The resulting `.github/workflows/ci.yml` is a "remix" of the current workflows for nimbus-eth2 and nim-codex (i.e. prior to this commit) along with techniques learned from developing workflows for other Status-org repos. Some comments and code-reorg help to clarify/explain what's done in the `Derive environment variables` step. `-d:limitStackUsage` has been adopted for `linux-amd64` builds from [nimbus-eth2's workflow][ne2wL155] and [related code][ne2config] has been copied into `config.nims` `-d:limitStackUsage` can easily be dropped if it's not desirable for Codex. Build targets use `-latest` for `runs-on`, i.e. `macos-latest`, `ubuntu-latest`, `windows-latest`. Through a combination of local testing and iterative pushes to GitHub, the workflow's embedded Bash scripts have been revised to include only the necessary steps for all builds to succeed, including `linux-i386` and `windows-i386`. The GitHub Actions workflow `.github/workflows/codecov.yml` has been removed, while coverage data generation/upload steps have been added to `.github/workflows/ci.yml` as the final steps conditional on `if: runner.os == 'Linux' && matrix.target.cpu == 'amd64' && matrix.nim_branch == matrix.cov_branch`. A redundant `--passC:'-m32 -mno-adx'` is used for `linux-i386` builds; the redundant flags do not affect the build, but can be helpful when eyeballing GitHub Actions builds with increased compile-time verbosity. Some variable expansions used in `github/workflows/ci.yml` could result in compilation failures if related paths include whitespace. It's not a problem for this commit but could be a problem for a user copy-pasting from the workflow; solving that problem is left as an exercise for the reader. [setmsys2]: https://github.com/msys2/setup-msys2#readme [ne2w]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml [ave5819]: https://github.com/actions/virtual-environments/pull/5819 [L151]: https://github.com/status-im/nim-codex/blob/ci/msys2/.github/workflows/ci.yml#L151 [succeed]: https://github.com/status-im/nim-codex/actions/runs/2606854455 [ne2wL155]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml#L155-L159 [ne2config]: https://github.com/status-im/nimbus-eth2/blob/stable/config.nims#L43-L49
2022-07-04 02:37:34 +00:00
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
coverage:
continue-on-error: true
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v3
with:
submodules: recursive
- name: Setup Nimbus Build System
uses: ./.github/actions/nimbus-build-system
with:
os: linux
- name: Generate coverage data
run: make -j${ncpu} coverage
shell: bash
- name: Upload coverage data to Codecov
ci: update GitHub Actions CI workflow to use msys2/setup-msys2@v2 Main goal is to update the nim-codex CI workflow to use the [`msys2/setup-msys2@v2`][setmsys2] GitHub Action, as used by the [nimbus-eth2 workflow][ne2w] per changes made to it several months ago. The `msys2/setup-msys2@v2` action has been used by other Status-org projects prior to this commit, e.g. nim-leopard, nim-datastore, nim-status. A fix is included for the failing macOS builds, related to [actions/virtual-environments#5819][ave5819]. See [L151][L151]. All builds [succeed][succeed] (with build arch-targets verified as far as possible via `ldd`, `otool`, `ntldd`), including `linux-i386` and `windows-i386`, though the `i386` builds are presently disabled (commented out). The `i386` builds can be enabled simply by uncommenting: ``` # - os: linux # cpu: i386 ... # - os: windows # cpu: i386 ``` The resulting `.github/workflows/ci.yml` is a "remix" of the current workflows for nimbus-eth2 and nim-codex (i.e. prior to this commit) along with techniques learned from developing workflows for other Status-org repos. Some comments and code-reorg help to clarify/explain what's done in the `Derive environment variables` step. `-d:limitStackUsage` has been adopted for `linux-amd64` builds from [nimbus-eth2's workflow][ne2wL155] and [related code][ne2config] has been copied into `config.nims` `-d:limitStackUsage` can easily be dropped if it's not desirable for Codex. Build targets use `-latest` for `runs-on`, i.e. `macos-latest`, `ubuntu-latest`, `windows-latest`. Through a combination of local testing and iterative pushes to GitHub, the workflow's embedded Bash scripts have been revised to include only the necessary steps for all builds to succeed, including `linux-i386` and `windows-i386`. The GitHub Actions workflow `.github/workflows/codecov.yml` has been removed, while coverage data generation/upload steps have been added to `.github/workflows/ci.yml` as the final steps conditional on `if: runner.os == 'Linux' && matrix.target.cpu == 'amd64' && matrix.nim_branch == matrix.cov_branch`. A redundant `--passC:'-m32 -mno-adx'` is used for `linux-i386` builds; the redundant flags do not affect the build, but can be helpful when eyeballing GitHub Actions builds with increased compile-time verbosity. Some variable expansions used in `github/workflows/ci.yml` could result in compilation failures if related paths include whitespace. It's not a problem for this commit but could be a problem for a user copy-pasting from the workflow; solving that problem is left as an exercise for the reader. [setmsys2]: https://github.com/msys2/setup-msys2#readme [ne2w]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml [ave5819]: https://github.com/actions/virtual-environments/pull/5819 [L151]: https://github.com/status-im/nim-codex/blob/ci/msys2/.github/workflows/ci.yml#L151 [succeed]: https://github.com/status-im/nim-codex/actions/runs/2606854455 [ne2wL155]: https://github.com/status-im/nimbus-eth2/blob/stable/.github/workflows/ci.yml#L155-L159 [ne2config]: https://github.com/status-im/nimbus-eth2/blob/stable/config.nims#L43-L49
2022-07-04 02:37:34 +00:00
uses: codecov/codecov-action@v3
with:
directory: ./coverage/
fail_ci_if_error: true
files: ./coverage/coverage.f.info
flags: unittests
name: codecov-umbrella
verbose: true
Speed up development (#303) * [ci] Extract setup of nimbus build system into separate file * [ci] Cleanup NodeJS setup * [ci] Use amd64 by default * [ci] Separate steps for unit, contract and integration tests * [ci] Separate build job for coverage that is allowed to fail * [ci] Separate build job for Nim 1.2 * [ci] Calculate coverage on unit tests only This enables faster development cycles at the expense of code coverage accuracy. * [PoR] Test proofs of retrievability with a small block size To get a reasonable test execution time. * [ci] Set NIM_COMMIT environment variable To prevent subsequent `make` command to build a different version of Nim. * [ci] Checkout submodules recursively So that nimbus-build-system doesn't have to do it afterwards. * [ci] Update caches on every run Will automatically update caches when a new minor version of Nim is released. See also: https://github.com/actions/cache/blob/main/workarounds.md#update-a-cache * [ci] Set cache_nonce for all jobs, not just those in the matrix * [build] update to latest nimbus-build-system Requires an update to latest nim-nat-traversal * [erasure] Test erasure coding with a small block size To get a reasonable test execution time. * [erasure] fix typo * [PoR] Test PoR store with a small block size To get a reasonable test execution time. * [PoR] Test PoR network with a small block size To get a reasonable test execution time. * [ci] Ensure that unit tests are not dependent on Ethereum Start Ethereum node *after* the unit tests are run. * [ci] Cancel all other jobs when one fails Allows for faster feedback loops; a next attempt to run the CI will start sooner.
2022-11-14 13:50:00 +00:00
nim_1_2:
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v3
with:
submodules: recursive
- name: Setup Nimbus Build System
uses: ./.github/actions/nimbus-build-system
with:
os: linux
nim_branch: version-1-2
- name: Unit tests
run: make -j${ncpu} test