Commit Graph

296 Commits

Author SHA1 Message Date
Ștefan Talpalaru 555ce310cc
Jenkins: get rid of macOS warnings (#3412)
* Jenkins: get rid of macOS warnings

We're linking object files with debug info with ones without it. Perfectly justified, from our point of view, but dsymutil complains verbosely about it, with no switch to silence it.
2022-02-18 15:30:43 +01:00
tersec 8df4290dde
simple script to time module compilations (#3414) 2022-02-18 10:52:01 +00:00
zah 178431102d
[skip ci] scripts/run-geth-el.sh Allow connecting to any merge devnet (#3409) 2022-02-17 19:14:15 +02:00
tersec f64fc6dc3a
include Bellatrix for in CI finalization tests (#3407) 2022-02-17 14:02:56 +01:00
Jacek Sieka 87e98b9e54
Revert "bump submodules (#3366)" (#3406)
This reverts commit 6e1ad080e8.
2022-02-17 12:50:37 +01:00
Ștefan Talpalaru 6e1ad080e8
bump submodules (#3366)
and add Nim-1.6 compatibility
2022-02-16 13:41:50 +02:00
tersec 2160d02977
use REST for make pyrmont/prater; use --rest options for local testnet (#3380) 2022-02-12 12:27:43 +01:00
Dustin Brody 3daa52ab87
update docs for geth/kiln 2022-02-11 20:06:06 +00:00
tersec fc0ce57b68
kiln merge test vectors for EL (#3377) 2022-02-11 18:17:37 +00:00
tersec bf3ef987e4
deactivate doppelganger protection during genesis (#3362)
* deactivate Doppelganger Protection during genesis

* also don't actually flag supposed-doppelgangers (because they're before broadcastStartEpoch) on GENESIS_SLOT start
2022-02-07 07:12:36 +02:00
tersec c9aa1bee01
spec URL updates (#3342) 2022-01-31 09:56:59 +00:00
tersec 2b4a960270
rename On{Merge,Bellatrix}BlockAdded and Rollback{Merge,Bellatrix}HashedProc (#3321) 2022-01-26 13:21:29 +01:00
tersec ae61512ee9
rename upgrade_to_{merge,bellatrix}; detect unchanging spec YAMLs (#3265) 2022-01-10 09:39:43 +00:00
tersec a4667d1840
script to automate aspects of spec URL updating (#3252) 2022-01-06 12:59:36 +00:00
Emil 2a12d1c49f Support more content-types when specifying the Graffiti string 2022-01-06 10:56:59 +02:00
tersec d4680df8d2
convert between engine and consensus ExecutionPayloads (#3228)
* convert between engine and consensus ExecutionPayloads
2022-01-03 13:22:56 +01:00
Zahary Karadjov 6b4f32ae23
Replicate a recent fix from the launch_local_testnet script due to a wide-spread code duplication 2021-12-22 17:59:45 +02:00
Zahary Karadjov 5eabeef75d Allow a docker image to be used instead of building nimbus_beacon_node
This would be useful when testing nimbus-eth1 in merge-mode simulations
2021-12-16 14:10:18 +02:00
Jacek Sieka 1a87e60f7a
remove unnecessary binaries from builds (#3179)
* `fork_choice` and `proto_array` are already covered by better tests
* `nimbus_signing_node` is not used by local testnet
2021-12-10 11:56:51 +01:00
Jacek Sieka e2a2157370
RPM/DEB for systemd-based systems (#3034)
* RPM/DEB for systemd-based systems

Similar to https://github.com/status-im/nimbus-eth2/pull/2691, this PR
introduces packaging for nimbus beacon node.

Instead of relying on interactive prompts, a default configuration is
included that connects to mainnet and expects there to be an execution
client running with websockets enabled on the same host, on the standard
websocket port.

Should the user need to configure their nimbus after installation (for
example with a different web3 url), the "standard" way of doing so via
`systemctl` edit is recommended.
2021-12-07 15:23:57 +02:00
tersec e6921f808f
cleanups, partly from kintsugi branch (#3161)
* cleanups, partly from kintsugi branch

* re-export shortLog(EthBlock) and preserve exception messages in batchVerify and processBatch
2021-12-05 17:32:41 +00:00
Eugene Kabanov e62c7c7c37
Remote signing client/server. (#3077) 2021-11-30 03:20:21 +02:00
tersec 39f6a6534e
document how to run merge local testnet with Nethermind (#3110) 2021-11-17 20:45:39 +01:00
tersec c5fa7387f9
update kintsugi docs (#3104) 2021-11-15 17:01:27 +00:00
Zahary Karadjov 5e62df5ad5 /bin/bash -> /usr/bin/env bash (needed on NixOS) 2021-11-15 15:45:01 +02:00
tersec 97ad5d49b0
kintsugi merge vector tests (#3072) 2021-11-10 13:41:02 +02:00
zah 2c0be052a5
Reduce the logging-related breaking changes (#3070)
* Reduce the logging-related breaking changes

* Don't disable the stdout log when the `--log-file` option is used
* Rename `--log-stdout` to `--log-format` and hide it
* Deprecate the `--log-file` option
2021-11-10 11:02:18 +02:00
Ștefan Talpalaru b09cd71348 build: show stderr by default
Also added: `-d:cwarnings` and `-d:limitStackUsage`
2021-11-05 16:06:48 +02:00
Jacek Sieka 3bfb5ad367
logtrace needs json logs (#3047) 2021-11-03 07:24:03 +01:00
Jacek Sieka 233d756518
Logging and startup improvements (#3038)
* Logging and startup improvements

Color support for released binaries!

* startup scripts no longer log to file by default - this only affects
source builds - released binaries don't support file logging
* add --log-stdout option to control logging to stdout (colors, json)
* detect tty:s vs redirected logs and log accordingly
* add option to disable log colors at runtime
* simplify several "common" logs, showing the most important information
earlier and more clearly
* remove line numbers / file information / tid - these take up space and
are of little use to end users
  * still enabled in debug builds and tools
* remove `testnet_servers_image` compile-time option
* server images, released binaries and compile-from-source now offer
the same behaviour and features
* fixes https://github.com/status-im/nimbus-eth2/issues/2326
* fixes https://github.com/status-im/nimbus-eth2/issues/1794
* remove instanteneous block speed from sync message, keeping only
average

before:

```
INF 2021-10-28 16:45:59.000+02:00 Slot start                                 topics="beacnde" tid=386429 file=nimbus_beacon_node.nim:884 lastSlot=2384027 wallSlot=2384028 delay=461us84ns peers=0 head=75a10ee5:3348 headEpoch=104 finalized=cd6804ba:3264 finalizedEpoch=102 sync="wwwwwwwwww:0:0.0000:0.0000:00h00m (3348)"
INF 2021-10-28 16:45:59.046+02:00 Slot end                                   topics="beacnde" tid=386429 file=nimbus_beacon_node.nim:821 slot=2384028 nextSlot=2384029 head=75a10ee5:3348 headEpoch=104 finalizedHead=cd6804ba:3264 finalizedEpoch=102 nextAttestationSlot=-1 nextProposalSlot=-1 nextActionWait=n/a
```

after:

```
INF 2021-10-28 22:43:23.033+02:00 Slot start                                 topics="beacnde" slot=2385815 epoch=74556 sync="DDPDDPUDDD:10:5.2258:01h19m (2361088)" peers=37 head=eacd2dae:2361096 finalized=73782:a4751487 delay=33ms687us715ns
INF 2021-10-28 22:43:23.291+02:00 Slot end                                   topics="beacnde" slot=2385815 nextActionWait=n/a nextAttestationSlot=-1 nextProposalSlot=-1 head=eacd2dae:2361096
```

* fix comment

* documentation updates

* mention `--log-file` may be deprecated in the future
* update various docs
2021-11-02 18:06:36 +01:00
Zahary Karadjov 6b3b82d549
Enable REST and metrics in the ./run-mainnet-node.sh script 2021-10-14 14:36:08 +03:00
tersec 2eb9a608a4
add payloadId; add merge vector test script; remove consensusValidated (#2982) 2021-10-13 16:08:50 +02:00
tersec 1f51331dd3
use specified execution_payload when constructing blocks; update Geth launch script (#2975)
* use specified execution_payload when constructing blocks; update Geth launch script

* revert DepositContractState change
2021-10-12 13:37:40 +02:00
Jacek Sieka 0c4bfb1911
test fixture cleanups (#2976)
* "official" -> "scenarios", like the submodule
* fewer test binaries - various compile hacks have been improved over
time, test suite should follow
* remove obsolete bls tests - there are better test vectors in
nim-blscurve
* remove obsolete mentions of `ssz_testing`
* remove obsolete comments about proc vs globals, unittest2 already uses
proc's
2021-10-12 13:36:52 +02:00
Dustin Brody 4a26e48226
add initial merge test vector scripts 2021-09-30 03:00:25 +00:00
zah 3689c68cbf
Carry out the sync committee gossip duties
Other changes:

* Add server getBlockV2(), and produceBlockV2().
* Add getBlockV2() to REST test suite.
* Add client getBlockV2(), and produceBlockV2().
* Fix URLs in comments.
* Add some primitives and fix some issues in forks.nim.
* Switch `validator_client` to V2 calls usage.
* Bump `chronos` with imports fixes.
* Bump `nim-json-serialization` for `requireAllFields`.
2021-08-30 03:58:30 +03:00
Zachinquarantine 80f2d4d9ee
run-beacon-node.sh: update URL (#2803) 2021-08-21 22:39:01 +02:00
Jacek Sieka a67970e9a4
subscribe to all subnets in minimal local testnet (#2799) 2021-08-20 16:41:16 +02:00
tersec df48409969
enable trace-level networking logging in local simulation (#2780)
* enable trace-level networking logging in local simulation

* remove Jenkinsfile overrides
2021-08-13 14:12:40 +00:00
Jacek Sieka 9697b73e71
forkedbeaconstate_helpers -> forks (#2772)
Simpler module name for stuff that covers forks

* check that runtime config matches database state
* also include some assorted altair cleanups
* use "standard" genesis fork in local testnet to work around missing
runtime config support
2021-08-10 22:46:35 +02:00
tersec 61825f4979
add aggregated attestation tracing to logtrace and enable it in Jenkins (#2766)
* add aggregated attestation tracing to logtrace and enable it in Jenkins CI

* use a slightly less cryptic acronym than aasr

* mostly, nimbus and the eth2 spec use aggregate attestation, not aggregated attestation
2021-08-06 21:25:30 +00:00
Ștefan Talpalaru 59b1a4772c
don't try to use lsof on Windows (#2762) 2021-08-05 10:49:34 +02:00
Ștefan Talpalaru ca96a98131
launch_local_testnet.sh: start the timeout earlier (#2753) 2021-08-03 12:12:21 +02:00
Ștefan Talpalaru 17c94c4d8e
restapi.sh: prevent port clashes (#2746) 2021-07-29 18:34:01 +02:00
Ștefan Talpalaru 7fe4569586
launch_local_testnet.sh: check if lsof is installed (#2742) 2021-07-27 21:45:36 +02:00
Ștefan Talpalaru 6299e14ffb
launch_local_testnet.sh: kill old procs before start (#2739) 2021-07-27 17:57:16 +02:00
tersec 941cc125a3
update to v1.1.0-beta.2 tests, with exception for altair sync committees (#2736)
* update to v1.1.0-beta.2 tests, with exception for altair sync committees

* bootstrap node can't drop any attestations
2021-07-26 17:34:13 +00:00
zah d9f2a91374
Remove the obsolete testnet0/1 scripts (#2727)
Also fixes `make eth2_network_simulation`
2021-07-16 13:02:27 +03:00
Jacek Sieka 3f9c1fdf4e
More RuntimeConfig cleanup (#2716)
* remove from BeaconChainDB (doesn't depend on runtime config)
* eth2-testnets -> eth2-networks
* use `cfg` name throughout
2021-07-13 16:27:10 +02:00
Eugene Kabanov 3b6f4fab4a
New validator client using REST API. (#2651)
* Initial commit.

* Exporting getConfig().

* Add beacon node checking procedures.

* Post rebase fixes.

* Use runSlotLoop() from nimbus_beacon_node.
Fallback implementation.
Fixes for ETH2 REST serialization.

* Add beacon_clock.durationToNextSlot().
Move type declarations from beacon_rest_api to json_rest_serialization.
Fix seq[ValidatorIndex] serialization.
Refactor ValidatorPool and add some utility procedures.
Create separate version of validator_client.

* Post-rebase fixes.
Remove CookedPubKey from validator_pool.nim.

* Now we should be able to produce attestations and aggregate and proofs.
But its not working yet.

* Debugging attestation sending.

* Add durationToNextAttestation.
Optimize some debug logs.
Fix aggregation_bits encoding.
Bump chronos/presto.

* Its alive.

* Fixes for launch_local_testnet script.
Bump chronos.

* Switch client API to not use `/api` prefix.

* Post-rebase adjustments.

* Fix endpoint for publishBlock().

* Add CONFIG_NAME.
Add more checks to ensure that beacon_node is compatible.

* Add beacon committee subscription support to validator_client.

* Fix stacktrace should be an array of strings.
Fix committee subscriptions should not be `data` keyed.

* Log duration to next block proposal.

* Fix beacon_node_status import.

* Use jsonMsgResponse() instead of jsonError().

* Fix graffityBytes usage.
Remove unnecessary `await`.
Adjust creation of SignedBlock instance.
Remove legacy files.

* Rework durationToNextSlot() and durationToNextEpoch() to use `fromNow`.

* Fix race condition for block proposal and attestations for same slot.
Fix local_testnet script to properly kill tasks on Windows.
Bump chronos and nim-http-tools, to allow connections to infura.io (basic auth).

* Catch services errors.
Improve performance of local_testnet.sh script on Windows.
Fix race condition when attestation producing.

* Post-rebase fixes.

* Bump chronos and presto.

* Calculate block publishing delay.
Fix pkill in one more place.

* Add error handling and timeouts to firstSuccess() template.
Add onceToAll() template.
Add checkNodes() procedure.
Refactor firstSuccess() template.
Add error checking to api.nim calls.

* Deprecated usage onceToAll() for better stability.
Address comment and send attestations asap.

* Avoid unnecessary loop when calculating minimal duration.
2021-07-13 13:15:07 +02:00