Commit Graph

216 Commits

Author SHA1 Message Date
Jacek Sieka 1a87e60f7a
remove unnecessary binaries from builds (#3179)
* `fork_choice` and `proto_array` are already covered by better tests
* `nimbus_signing_node` is not used by local testnet
2021-12-10 11:56:51 +01:00
Jacek Sieka cb3f1fd4fc
remove nbench (#3152)
Used during initial development of the spec, `nbench` has fallen behind
and by and large been superceded by `block_sim`, `state_sim` and
`ncli_db bench`.
2021-12-03 15:46:56 +01:00
Eugene Kabanov e62c7c7c37
Remote signing client/server. (#3077) 2021-11-30 03:20:21 +02:00
Jacek Sieka 0c4bfb1911
test fixture cleanups (#2976)
* "official" -> "scenarios", like the submodule
* fewer test binaries - various compile hacks have been improved over
time, test suite should follow
* remove obsolete bls tests - there are better test vectors in
nim-blscurve
* remove obsolete mentions of `ssz_testing`
* remove obsolete comments about proc vs globals, unittest2 already uses
proc's
2021-10-12 13:36:52 +02:00
Ștefan Talpalaru 1c50cd224e
Grafana: add 3 more panels (#2946)
(and delete old tool)
2021-10-04 12:04:33 +02:00
Ștefan Talpalaru b67e43e9f1
remove unused Docker test (#2945) 2021-10-04 08:32:29 +00:00
Ștefan Talpalaru 9b2b3bf1ab
enable libp2p agent metrics (#2933) 2021-10-01 16:15:08 +02:00
Etan Kissling fa3d7fa086
fix build output color on macOS (#2857)
On macOS, `echo` does not support the `\e` extension from GNU coreutils.
Replacing with the portable `\x1B` to fix Terminal build output colors.
2021-09-09 14:14:19 +02:00
Jacek Sieka a7a65bce42
disentangle eth2 types from the ssz library (#2785)
* reorganize ssz dependencies

This PR continues the work in
https://github.com/status-im/nimbus-eth2/pull/2646,
https://github.com/status-im/nimbus-eth2/pull/2779 as well as past
issues with serialization and type, to disentangle SSZ from eth2 and at
the same time simplify imports and exports with a structured approach.

The principal idea here is that when a library wants to introduce SSZ
support, they do so via 3 files:

* `ssz_codecs` which imports and reexports `codecs` - this covers the
basic byte conversions and ensures no overloads get lost
* `xxx_merkleization` imports and exports `merkleization` to specialize
and get access to `hash_tree_root` and friends
* `xxx_ssz_serialization` imports and exports `ssz_serialization` to
specialize ssz for a specific library

Those that need to interact with SSZ always import the `xxx_` versions
of the modules and never `ssz` itself so as to keep imports simple and
safe.

This is similar to how the REST / JSON-RPC serializers are structured in
that someone wanting to serialize spec types to REST-JSON will import
`eth2_rest_serialization` and nothing else.

* split up ssz into a core library that is independendent of eth2 types
* rename `bytes_reader` to `codec` to highlight that it contains coding
and decoding of bytes and native ssz types
* remove tricky List init overload that causes compile issues
* get rid of top-level ssz import
* reenable merkleization tests
* move some "standard" json serializers to spec
* remove `ValidatorIndex` serialization for now
* remove test_ssz_merkleization
* add tests for over/underlong byte sequences
* fix broken seq[byte] test - seq[byte] is not an SSZ type

There are a few things this PR doesn't solve:

* like #2646 this PR is weak on how to handle root and other
dontSerialize fields that "sometimes" should be computed - the same
problem appears in REST / JSON-RPC etc

* Fix a build problem on macOS

* Another way to fix the macOS builds

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 20:57:58 +02:00
tersec f0321d2eb5
remove inspector (#2737) 2021-07-29 14:00:04 +00:00
Jacek Sieka 3f9c1fdf4e
More RuntimeConfig cleanup (#2716)
* remove from BeaconChainDB (doesn't depend on runtime config)
* eth2-testnets -> eth2-networks
* use `cfg` name throughout
2021-07-13 16:27:10 +02:00
tersec 6b07b79427
re-add minimal tests, now that const sanity checks gone (#2715) 2021-07-13 08:38:35 +02:00
Jacek Sieka 23eea197f6
Implement split preset/config support (#2710)
* Implement split preset/config support

This is the initial bulk refactor to introduce runtime config values in
a number of places, somewhat replacing the existing mechanism of loading
network metadata.

It still needs more work, this is the initial refactor that introduces
runtime configuration in some of the places that need it.

The PR changes the way presets and constants work, to match the spec. In
particular, a "preset" now refers to the compile-time configuration
while a "cfg" or "RuntimeConfig" is the dynamic part.

A single binary can support either mainnet or minimal, but not both.
Support for other presets has been removed completely (can be readded,
in case there's need).

There's a number of outstanding tasks:

* `SECONDS_PER_SLOT` still needs fixing
* loading custom runtime configs needs redoing
* checking constants against YAML file

* yeerongpilly support

`build/nimbus_beacon_node --network=yeerongpilly --discv5:no --log-level=DEBUG`

* load fork epoch from config

* fix fork digest sent in status
* nicer error string for request failures
* fix tools

* one more

* fixup

* fixup

* fixup

* use "standard" network definition folder in local testnet

Files are loaded from their standard locations, including genesis etc,
to conform to the format used in the `eth2-networks` repo.

* fix launch scripts, allow unknown config values

* fix base config of rest test

* cleanups

* bundle mainnet config using common loader
* fix spec links and names
* only include supported preset in binary

* drop yeerongpilly, add altair-devnet-0, support boot_enr.yaml
2021-07-12 15:01:38 +02:00
Eugene Kabanov 32702d37a3
Move REST simulation test to Jenkins. (#2688) 2021-06-30 08:57:07 +00:00
tersec ae1abf24af
add Altair support to block quarantine/clearance and block_sim (#2662)
* add Altair support to the block quarantine

* switch some spec/datatypes imports to spec/datatypes/base

* add Altair support to block_clearance

* allow runtime configuration of Altair transition slot

* enable Altair in block_sim, including in CI
2021-06-23 14:43:18 +00:00
Ștefan Talpalaru 201d9d4056
Makefile: the "+" that makes a difference 2021-06-01 15:23:45 +02:00
tersec e8c62553ec
reorganize SSZ-based tests to support phase0 and altair (#2612)
* reorganize SSZ-based tests to support phase0 and altair

* label Altair SSZ consensus object tests as such

* update test summary
2021-05-28 20:32:26 +00:00
Zahary Karadjov b7aa30adfd
Merge stable into unstable 2021-05-20 13:50:40 +03:00
Jacek Sieka ce49da6c0a
Introduce unittest2 and junit reports (#2522)
* Introduce unittest2 and junit reports

* fix XML path

* don't combine multiple CI runs

* fixup

* public combined report also

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 18:41:02 +02:00
Eugene Kabanov 0c3302b826
REST API test framework and tests. (#2492)
* REST API test framework and tests.

* Fix ValidatorIndex tests to properly handle int32, but not uint32 values.

* Fix tests to follow latest REST fixes.

* refactor restapi.sh

and add it to the test suite

* Fix issues.
Add delay timeout which is required.

* Fix restapi.sh script for Windows.

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-27 23:46:24 +03:00
Ștefan Talpalaru 5b32f7ca17 Nim-1.2.12 2021-04-19 10:53:58 +03:00
Zahary Karadjov 61669f269d Clarify that the deposit cretion tools are intended for testnets 2021-04-08 19:58:22 +03:00
Ștefan Talpalaru b2c4441834
book: document "logrotate" (#2390)
* book: document "logrotate"

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

* rewording

* Update docs/the_nimbus_book/src/log-rotate.md

Co-authored-by: Sacha Saint-Leger <sacha@status.im>

Co-authored-by: Sacha Saint-Leger <sacha@status.im>
2021-03-22 16:40:26 +01:00
tersec 9ff6118c28
clear nimcache on make update (#2426)
* test clearing nimcache

* don't be verbose
2021-03-17 17:29:34 +00:00
tersec b059cb42c5
increase block proposal speed with many validators (#2423)
* increase block proposal speed with many validators

* document CookedSig rationale
2021-03-17 13:35:59 +00:00
Zahary Karadjov c519c6012e
Merge branch 'stable' into unstable 2021-03-15 21:52:12 +02:00
Zahary Karadjov f3fc551ea7
Support for the prater testnet 2021-03-15 21:51:24 +02:00
Jacek Sieka f5180e0840
Revert "Nim-1.2.10" (#2411)
This reverts commit 12e15a0b8d.
2021-03-15 16:26:41 +01:00
Ștefan Talpalaru eafe694536 dist: faster builds for ARM and ARM64
We're moving from very slow QEMU-virtualised builds to very fast ones
based on cross toolchains.

- refactor jobs; add checksums to release note
- build and publish Docker images with ARM/ARM64 binaries
2021-03-15 14:38:39 +02:00
Ștefan Talpalaru 12e15a0b8d Nim-1.2.10 2021-03-15 14:29:48 +02:00
Mamy Ratsimbazafy 2f17ac7b64
Move SSZ, deposit_contracts & eth1_monitor [reorg files 3/5] (#2371)
* move deposit_contract

* Move SSZ

* fix ssz import in tests

* move also eth1_monitor

* forgot to delete the original

* fix comma [skip ci]

* Fix "make" & tools imports

* Fix import

* Fix import again

* rename deposit_contract -> eth1

* Revert ssz move to subfolder

* path fixes [skip ci]
2021-03-03 07:23:05 +01:00
Ștefan Talpalaru 786137ffb4 reinstate LOG_LEVEL
and introduce RUNTIME_LOG_LEVEL
2021-02-19 18:11:34 +02:00
Zahary Karadjov 5f924c0038 Bump modules; Compile with DEBUG logs by default 2021-02-19 08:44:13 +02:00
Ștefan Talpalaru f398720fc4
compile_nim_program.sh: bring back Nim warnings with V=0 (#2325) 2021-02-15 18:08:18 +01:00
Ștefan Talpalaru 4bb90351f6
books: sanity checks and docs (#2320) 2021-02-15 08:33:49 +01:00
Ștefan Talpalaru e4f4cb1513
remove "beacon_chain.nimble" (#2316)
Tests have been moved to the Makefile and the test binaries are built in
parallel.

The Nim compilation scheme has been moved to a script and adapted to
work with different binaries being built in parallel from the same main
source file (think minimal/mainnet tests).
2021-02-14 20:04:54 +00:00
Ștefan Talpalaru 80c11546ff Windows binary release
CI: use both cores on GitHub Actions and set timeouts for the local testnet tests
2021-02-04 10:25:44 +02:00
Jakub Sokołowski 095b191aee
add cname file to docs to fix domain issues (#2260)
mdbook has an option called `cname`:
https://rust-lang.github.io/mdBook/format/config.html#html-renderer-options

But it appears to be only included starting from `0.4.3`:
https://github.com/rust-lang/mdBook/blob/master/CHANGELOG.md#mdbook-043

Which means this will be more robust, if uglier.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-01-22 17:57:55 +01:00
tersec 62908b0e91
remove backwards compatibility targets (#2223) 2021-01-12 18:00:30 +01:00
Ștefan Talpalaru e58a355a1c
CI release: ARM and ARM64 builds (#2213) 2021-01-07 10:19:29 +01:00
Ștefan Talpalaru e59437c5e2
new Make target: "benchmarks" (#2174)
It runs some benchmarks, collects the output and generates HTML charts.
2020-12-11 12:12:43 +01:00
Ștefan Talpalaru 92750d5313 reduce peak memory usage during build 2020-12-10 17:17:12 +02:00
Zahary Karadjov 7225a6d6ed Retry all web3 requests up to 3 times with exponential backoff 2020-12-02 01:43:54 +02:00
Ștefan Talpalaru c776d78c7d Makefile: better build completion message 2020-11-26 10:09:34 +02:00
Ștefan Talpalaru ab4880d4b4 LTO: allow GCC to use Make's jobserver 2020-11-25 18:33:58 +02:00
zah 372c9b798c
Fix the corrupted database state on Pyrmont nodes; Add mainnet genesis (#2056)
* Handle some web3 timeouts better

* Add support for developer .env files

* Eth1 improvements; Mainnet genesis state

Notable changes:

* The deposits table have been removed from the database. The client
  will no longer process all deposits on start-up.

* The network metadata now includes a "state snapshot" of the deposit
  contract. This allows the client to skip syncing deposits made prior
  to the snapshot (i.e. genesis). Suitable metadata added for Pyrmont
  and Mainnet.

* The Eth1 monitor won't be started unless there are validators attached
  to the node.

* The genesis detection code is now optional and disabled by default

* Bugfix: The client should not produce blocks that will fail validation
  when it hasn't downloaded the latest deposits yet

* Bugfix: Work around the database corruption affecting Pyrmont nodes

* Remove metadata for Toledo and Medalla
2020-11-24 22:21:47 +01:00
Ștefan Talpalaru 7b7dc6fed4
Makefile: the workaround shouldn't break after a bugfix 2020-11-24 17:27:21 +01:00
tersec 3c4cf95d0e
remove v0.12.3 and medalla support (#2038) 2020-11-20 14:49:49 +01:00
Zahary Karadjov 430bac81dc Add a development-purpose target for making deposits on Pyrmont 2020-11-15 21:40:40 +02:00
Jacek Sieka ab8f8ccaba
clean up dependencies (#2008)
* clean up dependencies

* use non-prce-mode for metrics
* clean up obsolete snappy and gflags deps from manuals

* conditional pcre
2020-11-13 16:00:45 +01:00