Commit Graph

429 Commits

Author SHA1 Message Date
Etan Kissling fd4cf35c20
fix concurrent Jenkins stages (#3904)
The ports for the concurrently executing REST and Minimal testnet clash,
leading to some CI failures since #3827 introduced further concurrency.
Adjusting the ports to be distinct across various tests should fix this.
2022-07-23 14:28:10 +00:00
Etan Kissling a6deacd878
allow driving EL with LC (#3865)
Adds the `--web3-url` launch argument to `nimbus_light_client` to enable
driving the EL with the optimistic head obtained from LC sync protocol.
This will keep issuing `newPayload` / `forkChoiceUpdated` requests for
new blocks, marking them as optimistic. `ZERO_HASH` is reported as the
finalized block for now.
2022-07-14 04:07:40 +00:00
Etan Kissling 22e77131f8
export `DepositContractSnapshot` in eth1 monitor (#3860)
Avoid requiring explicit `beacon_chain_db` import in eth1 monitor use.
2022-07-13 01:45:48 +00:00
tersec e0e7af7aff
MEV boost script for ropsten and sepolia builder network (#3851)
* MEV boost script for ropsten and sepolia builder network

* networkInfo not a global variable
2022-07-12 20:50:12 +03:00
Etan Kissling 20491560b6
re-enable LC in local testnet (#3857)
`a48d741022f6b0da1bb679e0ede4e38c019242cf` disabled LC in local testnet
as an undocumented side effect. Re-enabling for more thorough testing,
and added handling of LC with `--eth2-docker-image`.
2022-07-12 11:38:50 +00:00
tersec 19bf7f1599
check in MEV boost test script that payload gets unblinded (#3838) 2022-07-05 04:13:07 +00:00
tersec 31772a2ac2
don't use JWT if one hasn't set up a JWT secret (#3822) 2022-06-29 07:36:22 +00:00
Jacek Sieka ff12c7f9ce
remove web3 url prompt in launcher script (#3810)
* remove web3 url prompt in launcher script

The interactive prompt for web3 has outlived its utility as we now load
url:s from command line params and config files, preventing the prompt
from correctly detecting when it's needed.

Also, after the merge, a JWT secret will (likely) be needed.

* log notice when web3 url is missing
* fix docs to not mention default that doesn't exist
* fix scripts to properly quote arguments
2022-06-28 23:49:52 +03:00
Jacek Sieka 0152f4ade5
Make deb/rpm packages when creating release (#3796)
* Make deb/rpm packages when creating release

A step towards #3782

* fix arch for arm
2022-06-28 23:34:18 +03:00
Zahary Karadjov 73227508db
Fix regressions in launch_local_simulation 2022-06-28 19:22:06 +03:00
Nikolay Mitev a48d741022 Update local-testnet-simulation with changes from kiln-dev-auth branch 2022-06-28 13:33:49 +02:00
zah b318a74d83
Add support for launching Geth and Nimbus-eth1 nodes in launch_local_testnet (#3790)
* Add support for launching Geth and Nimbus-eth1 nodes in launch_local_simulation

* Prevent undefined variables errors when Nimbus and Geth are not launched

Co-authored-by: Nikolay Mitev <github@hmel.org>
2022-06-27 11:34:12 +03:00
tersec 7e61198260
fix merge test scripts (#3781) 2022-06-20 18:03:57 +00:00
Etan Kissling 9dcbc44fc8
fix `--light-clients` option in `launch_local_testnet` (#3779)
`launch_local_testnet` was not recognizing `--light-clients` correctly.
2022-06-19 20:56:34 +03:00
tersec 8eb5d5de09
use ZERO_HASH for default(Eth2Digest)/Eth2Digest() in func calls (#3770) 2022-06-18 04:57:37 +00:00
Jacek Sieka c48ea3cdc8
book updates (#3763)
* systemd: use same service file as package
* systemd: recommend setting up a user
* clean up obsolete / ephemeral notes
* eth1 -> execution, in many places
* clean up docker instructions to match the experience of running the
runner script
* add basic instructions / links for execution clients other than geth
* clear a few more references to JSON-RPC, update to REST
* update hardware requirements with execution client extras
* start talking about merge requirements (this needs a dedicated page)
* note that binaries do exist, where relevant
* list gitcoin under donations
* update syncing docs
* `NOT` is for `NOTICE` in logs
* update rpi hardware recommendations
* prevent automatic doppelganger restarts in systemd service file
2022-06-17 18:28:13 +03:00
Etan Kissling 52ba4f7999
rename light client config parameters (#3740)
For consistency with other options, use a common prefix for light client
data configuration options.

* `--serve-light-client-data` --> `--light-client-data-serve`
* `--import-light-client-data` --> `--light-client-data-import-mode`

No deprecation of the old identifiers as they were only sparingly used
and all usage can be easily updated without interferance.
2022-06-14 12:03:39 +03:00
Etan Kissling 21a1f7eeb7
error handling for local testnet REST query (#3739)
When querying `ALTAIR_FORK_EPOCH` fails, the `launch_local_testnet`
script got stuck because comparing `CURRENT_FORK_EPOCH` against it
would keep failing with `[: : integer expression expected`.
Querying `ALTAIR_FORK_EPOCH` is now retried until a number is returned.
2022-06-14 08:38:39 +00:00
tersec 0cedb1b6a7
seq(1) can count backwards on macOS; catch undefined variable use (#3735) 2022-06-13 20:01:39 +03:00
Etan Kissling 15967c4076
keep track of latest blocks for optimistic sync (#3715)
When launched with `--light-client-enable` the latest blocks are fetched
and optimistic candidate blocks are passed to a callback (log for now).
This helps accelerate syncing in the future (optimistic sync).
2022-06-10 14:16:37 +00:00
tersec 1262e50fa5
document MEV/mergemock test script (#3694) 2022-06-06 16:55:02 +03:00
Etan Kissling 01efa93cf6
add light client (standalone) (#3653)
Introduces a new library for syncing using libp2p based light client
sync protocol, and adds a new `nimbus_light_client` executable that uses
this library for syncing. The new executable emits log messages when
new beacon block headers are received, and is integrated into testing.
2022-05-31 12:45:37 +02:00
tersec 92c1e75915
stop using --rpc in run-beacon-node and simulations (#3662) 2022-05-24 16:21:55 +00:00
Nikolay Mitev b3dc9e2c08
Update merge test vector script and documentation (#3638) 2022-05-16 16:12:32 +03:00
zah a2ba34f686
Implement all sync committee duties in the validator client (#3583)
Other changes:

* logtrace can now verify sync committee messages and contributions
* Many unnecessary use of pairs() have been removed for consistency
* Map 40x BN response codes to BeaconNodeStatus.Incompatible in the VC
2022-05-10 10:03:40 +00:00
zah 6d11ad6ce1
Support for distributed keystores with multiple remotes based on threshold signatures (#3616)
Other fixes:

* Fix bit rot in the `make prater-dev-deposit` target.
* Correct content-type in the responses of the Nimbus signing node
* Invalid JSON payload was being sent in the web3signer requests
2022-05-10 03:32:12 +03:00
Etan Kissling 89ac586bd4
serve light client data in CI / dev builds (#3487)
Adjust config for CI / dev builds to serve light client data by default:
`--serve-light-client-data=1 --import-light-client-data=only-new`
2022-03-12 22:12:18 +01:00
tersec 21b71bd29c
update URL and document Nim bug blocking further genericizing cleanups (#3483) 2022-03-11 15:03:47 +00:00
Ștefan Talpalaru 857a71be6c
launch_local_testnet.sh: Lighthouse VC nodes (#3477)
* launch_local_testnet.sh: Lighthouse VC nodes
2022-03-11 13:44:56 +01:00
tersec 35f712e378
kiln v2 documentation and testing (#3446) 2022-03-05 16:40:58 +02:00
zah cdeae90806
Add support for TOML config files (--config-file) (#3442) 2022-03-05 04:33:15 +02:00
Ștefan Talpalaru ebba093362
Nim-1.6 compatibility (#3434) 2022-02-25 10:19:12 +02:00
tersec 05bc61b712
add mev-boost RPC test, with docs (#3430)
* bump nim-web3 and add mev-boost RPC test, with docs

* remove trailing space

* use specific commithash
2022-02-24 14:38:31 +01:00
Ștefan Talpalaru 555ce310cc
Jenkins: get rid of macOS warnings (#3412)
* Jenkins: get rid of macOS warnings

We're linking object files with debug info with ones without it. Perfectly justified, from our point of view, but dsymutil complains verbosely about it, with no switch to silence it.
2022-02-18 15:30:43 +01:00
tersec 8df4290dde
simple script to time module compilations (#3414) 2022-02-18 10:52:01 +00:00
zah 178431102d
[skip ci] scripts/run-geth-el.sh Allow connecting to any merge devnet (#3409) 2022-02-17 19:14:15 +02:00
tersec f64fc6dc3a
include Bellatrix for in CI finalization tests (#3407) 2022-02-17 14:02:56 +01:00
Jacek Sieka 87e98b9e54
Revert "bump submodules (#3366)" (#3406)
This reverts commit 6e1ad080e8.
2022-02-17 12:50:37 +01:00
Ștefan Talpalaru 6e1ad080e8
bump submodules (#3366)
and add Nim-1.6 compatibility
2022-02-16 13:41:50 +02:00
tersec 2160d02977
use REST for make pyrmont/prater; use --rest options for local testnet (#3380) 2022-02-12 12:27:43 +01:00
Dustin Brody 3daa52ab87
update docs for geth/kiln 2022-02-11 20:06:06 +00:00
tersec fc0ce57b68
kiln merge test vectors for EL (#3377) 2022-02-11 18:17:37 +00:00
tersec bf3ef987e4
deactivate doppelganger protection during genesis (#3362)
* deactivate Doppelganger Protection during genesis

* also don't actually flag supposed-doppelgangers (because they're before broadcastStartEpoch) on GENESIS_SLOT start
2022-02-07 07:12:36 +02:00
tersec c9aa1bee01
spec URL updates (#3342) 2022-01-31 09:56:59 +00:00
tersec 2b4a960270
rename On{Merge,Bellatrix}BlockAdded and Rollback{Merge,Bellatrix}HashedProc (#3321) 2022-01-26 13:21:29 +01:00
tersec ae61512ee9
rename upgrade_to_{merge,bellatrix}; detect unchanging spec YAMLs (#3265) 2022-01-10 09:39:43 +00:00
tersec a4667d1840
script to automate aspects of spec URL updating (#3252) 2022-01-06 12:59:36 +00:00
Emil 2a12d1c49f Support more content-types when specifying the Graffiti string 2022-01-06 10:56:59 +02:00
tersec d4680df8d2
convert between engine and consensus ExecutionPayloads (#3228)
* convert between engine and consensus ExecutionPayloads
2022-01-03 13:22:56 +01:00
Zahary Karadjov 6b4f32ae23
Replicate a recent fix from the launch_local_testnet script due to a wide-spread code duplication 2021-12-22 17:59:45 +02:00
Zahary Karadjov 5eabeef75d Allow a docker image to be used instead of building nimbus_beacon_node
This would be useful when testing nimbus-eth1 in merge-mode simulations
2021-12-16 14:10:18 +02:00
Jacek Sieka 1a87e60f7a
remove unnecessary binaries from builds (#3179)
* `fork_choice` and `proto_array` are already covered by better tests
* `nimbus_signing_node` is not used by local testnet
2021-12-10 11:56:51 +01:00
Jacek Sieka e2a2157370
RPM/DEB for systemd-based systems (#3034)
* RPM/DEB for systemd-based systems

Similar to https://github.com/status-im/nimbus-eth2/pull/2691, this PR
introduces packaging for nimbus beacon node.

Instead of relying on interactive prompts, a default configuration is
included that connects to mainnet and expects there to be an execution
client running with websockets enabled on the same host, on the standard
websocket port.

Should the user need to configure their nimbus after installation (for
example with a different web3 url), the "standard" way of doing so via
`systemctl` edit is recommended.
2021-12-07 15:23:57 +02:00
tersec e6921f808f
cleanups, partly from kintsugi branch (#3161)
* cleanups, partly from kintsugi branch

* re-export shortLog(EthBlock) and preserve exception messages in batchVerify and processBatch
2021-12-05 17:32:41 +00:00
Eugene Kabanov e62c7c7c37
Remote signing client/server. (#3077) 2021-11-30 03:20:21 +02:00
tersec 39f6a6534e
document how to run merge local testnet with Nethermind (#3110) 2021-11-17 20:45:39 +01:00
tersec c5fa7387f9
update kintsugi docs (#3104) 2021-11-15 17:01:27 +00:00
Zahary Karadjov 5e62df5ad5 /bin/bash -> /usr/bin/env bash (needed on NixOS) 2021-11-15 15:45:01 +02:00
tersec 97ad5d49b0
kintsugi merge vector tests (#3072) 2021-11-10 13:41:02 +02:00
zah 2c0be052a5
Reduce the logging-related breaking changes (#3070)
* Reduce the logging-related breaking changes

* Don't disable the stdout log when the `--log-file` option is used
* Rename `--log-stdout` to `--log-format` and hide it
* Deprecate the `--log-file` option
2021-11-10 11:02:18 +02:00
Ștefan Talpalaru b09cd71348 build: show stderr by default
Also added: `-d:cwarnings` and `-d:limitStackUsage`
2021-11-05 16:06:48 +02:00
Jacek Sieka 3bfb5ad367
logtrace needs json logs (#3047) 2021-11-03 07:24:03 +01:00
Jacek Sieka 233d756518
Logging and startup improvements (#3038)
* Logging and startup improvements

Color support for released binaries!

* startup scripts no longer log to file by default - this only affects
source builds - released binaries don't support file logging
* add --log-stdout option to control logging to stdout (colors, json)
* detect tty:s vs redirected logs and log accordingly
* add option to disable log colors at runtime
* simplify several "common" logs, showing the most important information
earlier and more clearly
* remove line numbers / file information / tid - these take up space and
are of little use to end users
  * still enabled in debug builds and tools
* remove `testnet_servers_image` compile-time option
* server images, released binaries and compile-from-source now offer
the same behaviour and features
* fixes https://github.com/status-im/nimbus-eth2/issues/2326
* fixes https://github.com/status-im/nimbus-eth2/issues/1794
* remove instanteneous block speed from sync message, keeping only
average

before:

```
INF 2021-10-28 16:45:59.000+02:00 Slot start                                 topics="beacnde" tid=386429 file=nimbus_beacon_node.nim:884 lastSlot=2384027 wallSlot=2384028 delay=461us84ns peers=0 head=75a10ee5:3348 headEpoch=104 finalized=cd6804ba:3264 finalizedEpoch=102 sync="wwwwwwwwww:0:0.0000:0.0000:00h00m (3348)"
INF 2021-10-28 16:45:59.046+02:00 Slot end                                   topics="beacnde" tid=386429 file=nimbus_beacon_node.nim:821 slot=2384028 nextSlot=2384029 head=75a10ee5:3348 headEpoch=104 finalizedHead=cd6804ba:3264 finalizedEpoch=102 nextAttestationSlot=-1 nextProposalSlot=-1 nextActionWait=n/a
```

after:

```
INF 2021-10-28 22:43:23.033+02:00 Slot start                                 topics="beacnde" slot=2385815 epoch=74556 sync="DDPDDPUDDD:10:5.2258:01h19m (2361088)" peers=37 head=eacd2dae:2361096 finalized=73782:a4751487 delay=33ms687us715ns
INF 2021-10-28 22:43:23.291+02:00 Slot end                                   topics="beacnde" slot=2385815 nextActionWait=n/a nextAttestationSlot=-1 nextProposalSlot=-1 head=eacd2dae:2361096
```

* fix comment

* documentation updates

* mention `--log-file` may be deprecated in the future
* update various docs
2021-11-02 18:06:36 +01:00
Zahary Karadjov 6b3b82d549
Enable REST and metrics in the ./run-mainnet-node.sh script 2021-10-14 14:36:08 +03:00
tersec 2eb9a608a4
add payloadId; add merge vector test script; remove consensusValidated (#2982) 2021-10-13 16:08:50 +02:00
tersec 1f51331dd3
use specified execution_payload when constructing blocks; update Geth launch script (#2975)
* use specified execution_payload when constructing blocks; update Geth launch script

* revert DepositContractState change
2021-10-12 13:37:40 +02:00
Jacek Sieka 0c4bfb1911
test fixture cleanups (#2976)
* "official" -> "scenarios", like the submodule
* fewer test binaries - various compile hacks have been improved over
time, test suite should follow
* remove obsolete bls tests - there are better test vectors in
nim-blscurve
* remove obsolete mentions of `ssz_testing`
* remove obsolete comments about proc vs globals, unittest2 already uses
proc's
2021-10-12 13:36:52 +02:00
Dustin Brody 4a26e48226
add initial merge test vector scripts 2021-09-30 03:00:25 +00:00
zah 3689c68cbf
Carry out the sync committee gossip duties
Other changes:

* Add server getBlockV2(), and produceBlockV2().
* Add getBlockV2() to REST test suite.
* Add client getBlockV2(), and produceBlockV2().
* Fix URLs in comments.
* Add some primitives and fix some issues in forks.nim.
* Switch `validator_client` to V2 calls usage.
* Bump `chronos` with imports fixes.
* Bump `nim-json-serialization` for `requireAllFields`.
2021-08-30 03:58:30 +03:00
Zachinquarantine 80f2d4d9ee
run-beacon-node.sh: update URL (#2803) 2021-08-21 22:39:01 +02:00
Jacek Sieka a67970e9a4
subscribe to all subnets in minimal local testnet (#2799) 2021-08-20 16:41:16 +02:00
tersec df48409969
enable trace-level networking logging in local simulation (#2780)
* enable trace-level networking logging in local simulation

* remove Jenkinsfile overrides
2021-08-13 14:12:40 +00:00
Jacek Sieka 9697b73e71
forkedbeaconstate_helpers -> forks (#2772)
Simpler module name for stuff that covers forks

* check that runtime config matches database state
* also include some assorted altair cleanups
* use "standard" genesis fork in local testnet to work around missing
runtime config support
2021-08-10 22:46:35 +02:00
tersec 61825f4979
add aggregated attestation tracing to logtrace and enable it in Jenkins (#2766)
* add aggregated attestation tracing to logtrace and enable it in Jenkins CI

* use a slightly less cryptic acronym than aasr

* mostly, nimbus and the eth2 spec use aggregate attestation, not aggregated attestation
2021-08-06 21:25:30 +00:00
Ștefan Talpalaru 59b1a4772c
don't try to use lsof on Windows (#2762) 2021-08-05 10:49:34 +02:00
Ștefan Talpalaru ca96a98131
launch_local_testnet.sh: start the timeout earlier (#2753) 2021-08-03 12:12:21 +02:00
Ștefan Talpalaru 17c94c4d8e
restapi.sh: prevent port clashes (#2746) 2021-07-29 18:34:01 +02:00
Ștefan Talpalaru 7fe4569586
launch_local_testnet.sh: check if lsof is installed (#2742) 2021-07-27 21:45:36 +02:00
Ștefan Talpalaru 6299e14ffb
launch_local_testnet.sh: kill old procs before start (#2739) 2021-07-27 17:57:16 +02:00
tersec 941cc125a3
update to v1.1.0-beta.2 tests, with exception for altair sync committees (#2736)
* update to v1.1.0-beta.2 tests, with exception for altair sync committees

* bootstrap node can't drop any attestations
2021-07-26 17:34:13 +00:00
zah d9f2a91374
Remove the obsolete testnet0/1 scripts (#2727)
Also fixes `make eth2_network_simulation`
2021-07-16 13:02:27 +03:00
Jacek Sieka 3f9c1fdf4e
More RuntimeConfig cleanup (#2716)
* remove from BeaconChainDB (doesn't depend on runtime config)
* eth2-testnets -> eth2-networks
* use `cfg` name throughout
2021-07-13 16:27:10 +02:00
Eugene Kabanov 3b6f4fab4a
New validator client using REST API. (#2651)
* Initial commit.

* Exporting getConfig().

* Add beacon node checking procedures.

* Post rebase fixes.

* Use runSlotLoop() from nimbus_beacon_node.
Fallback implementation.
Fixes for ETH2 REST serialization.

* Add beacon_clock.durationToNextSlot().
Move type declarations from beacon_rest_api to json_rest_serialization.
Fix seq[ValidatorIndex] serialization.
Refactor ValidatorPool and add some utility procedures.
Create separate version of validator_client.

* Post-rebase fixes.
Remove CookedPubKey from validator_pool.nim.

* Now we should be able to produce attestations and aggregate and proofs.
But its not working yet.

* Debugging attestation sending.

* Add durationToNextAttestation.
Optimize some debug logs.
Fix aggregation_bits encoding.
Bump chronos/presto.

* Its alive.

* Fixes for launch_local_testnet script.
Bump chronos.

* Switch client API to not use `/api` prefix.

* Post-rebase adjustments.

* Fix endpoint for publishBlock().

* Add CONFIG_NAME.
Add more checks to ensure that beacon_node is compatible.

* Add beacon committee subscription support to validator_client.

* Fix stacktrace should be an array of strings.
Fix committee subscriptions should not be `data` keyed.

* Log duration to next block proposal.

* Fix beacon_node_status import.

* Use jsonMsgResponse() instead of jsonError().

* Fix graffityBytes usage.
Remove unnecessary `await`.
Adjust creation of SignedBlock instance.
Remove legacy files.

* Rework durationToNextSlot() and durationToNextEpoch() to use `fromNow`.

* Fix race condition for block proposal and attestations for same slot.
Fix local_testnet script to properly kill tasks on Windows.
Bump chronos and nim-http-tools, to allow connections to infura.io (basic auth).

* Catch services errors.
Improve performance of local_testnet.sh script on Windows.
Fix race condition when attestation producing.

* Post-rebase fixes.

* Bump chronos and presto.

* Calculate block publishing delay.
Fix pkill in one more place.

* Add error handling and timeouts to firstSuccess() template.
Add onceToAll() template.
Add checkNodes() procedure.
Refactor firstSuccess() template.
Add error checking to api.nim calls.

* Deprecated usage onceToAll() for better stability.
Address comment and send attestations asap.

* Avoid unnecessary loop when calculating minimal duration.
2021-07-13 13:15:07 +02:00
Jacek Sieka 23eea197f6
Implement split preset/config support (#2710)
* Implement split preset/config support

This is the initial bulk refactor to introduce runtime config values in
a number of places, somewhat replacing the existing mechanism of loading
network metadata.

It still needs more work, this is the initial refactor that introduces
runtime configuration in some of the places that need it.

The PR changes the way presets and constants work, to match the spec. In
particular, a "preset" now refers to the compile-time configuration
while a "cfg" or "RuntimeConfig" is the dynamic part.

A single binary can support either mainnet or minimal, but not both.
Support for other presets has been removed completely (can be readded,
in case there's need).

There's a number of outstanding tasks:

* `SECONDS_PER_SLOT` still needs fixing
* loading custom runtime configs needs redoing
* checking constants against YAML file

* yeerongpilly support

`build/nimbus_beacon_node --network=yeerongpilly --discv5:no --log-level=DEBUG`

* load fork epoch from config

* fix fork digest sent in status
* nicer error string for request failures
* fix tools

* one more

* fixup

* fixup

* fixup

* use "standard" network definition folder in local testnet

Files are loaded from their standard locations, including genesis etc,
to conform to the format used in the `eth2-networks` repo.

* fix launch scripts, allow unknown config values

* fix base config of rest test

* cleanups

* bundle mainnet config using common loader
* fix spec links and names
* only include supported preset in binary

* drop yeerongpilly, add altair-devnet-0, support boot_enr.yaml
2021-07-12 15:01:38 +02:00
Zahary Karadjov ed2f6f753d Allow the custom testnet metadata to specify a path to the genesis file 2021-06-01 15:50:50 +03:00
tersec ea9ceb693a
update ChainDAG.effective_balance() to use StateData; rm ChainDAG.getBlockByPreciseSlot() (#2622)
* update ChainDAG.effective_balance() to use StateData; rm unused ChainDAG.getBlockByPreciseSlot()

* update get_effective_balances to avoid god object; avoid most memory allocation in Altair epoch reward and penalty processing
2021-06-01 12:40:13 +00:00
Zahary Karadjov b7aa30adfd
Merge stable into unstable 2021-05-20 13:50:40 +03:00
tersec d8bb91d9a9
partially integrate eth1 merge changes (#2548)
* partially integrate eth1 merge changes

* use hexToSeqByte() and validate execution engine opaque transaction length

* remove incorrect REST serialization code
2021-05-20 10:44:13 +00:00
tersec 290b889ce6
non-intrusive, novel portions of merge (#2535) 2021-05-04 11:54:19 +00:00
Zahary Karadjov 6806ffe1c8
Remove some unused parameters (part 2) 2021-04-14 21:49:42 +03:00
Ștefan Talpalaru 37b3f8d577 run-beacon-node.sh: handle "--web3-url=..." properly
Now the script looks at those command line arguments meant for
nimbus_beacon_node to decide whether to prompt the user for a WEB3 URL
or not.
2021-04-14 16:08:54 +03:00
Zahary Karadjov 61669f269d Clarify that the deposit cretion tools are intended for testnets 2021-04-08 19:58:22 +03:00
Jacek Sieka f821bc878e
Remove `-d:insecure` compile option (#2468)
With metrics running on top of chronos, the metrics server no longer
needs to be compiled in conditionally - it remains disabled by default.
2021-04-01 14:44:11 +02:00
Jacek Sieka 714620b13f
script for finding slow logs 2021-03-30 09:39:16 +02:00
Ștefan Talpalaru eafe694536 dist: faster builds for ARM and ARM64
We're moving from very slow QEMU-virtualised builds to very fast ones
based on cross toolchains.

- refactor jobs; add checksums to release note
- build and publish Docker images with ARM/ARM64 binaries
2021-03-15 14:38:39 +02:00
tersec dfc3322fe2
last few v1.0.0 spec refs to v1.0.1 (#2404) 2021-03-13 20:51:39 +00:00
Jacek Sieka ff94e1a212
Document log rotation (#2327)
* Document log rotation

* phrasing updates

Co-authored-by: Dustin Brody <tersec@users.noreply.github.com>
2021-02-18 07:57:41 +01:00
Ștefan Talpalaru f398720fc4
compile_nim_program.sh: bring back Nim warnings with V=0 (#2325) 2021-02-15 18:08:18 +01:00
Ștefan Talpalaru e4f4cb1513
remove "beacon_chain.nimble" (#2316)
Tests have been moved to the Makefile and the test binaries are built in
parallel.

The Nim compilation scheme has been moved to a script and adapted to
work with different binaries being built in parallel from the same main
source file (think minimal/mainnet tests).
2021-02-14 20:04:54 +00:00
Ștefan Talpalaru f84a52e83e launch_local_testnet.sh: NPROC for macOS 2021-02-11 22:08:30 +02:00
Mamy Ratsimbazafy 03f47c8f2f
Slashing protection refactor - EIP 3076 (#2094)
* Create CLI tool for slashing export

* Use SQLite as a DB instead of a KV-store

* Keeps v1 and v2 DBs around

* Uses the same schema as Lighthouse v1.1.0

* Passes all interchange tests + skeleton of finalization pruning

* Removes tests that would violate v5 / minimal slashing DB and MinSlot rules

* Migration tool added using low-watermark scheme for faster migration of large number of validators
2021-02-09 17:23:06 +02:00
Ștefan Talpalaru 979bb39b41 docs: binary distribution internals 2021-02-08 14:53:12 +02:00
Ștefan Talpalaru 80c11546ff Windows binary release
CI: use both cores on GitHub Actions and set timeouts for the local testnet tests
2021-02-04 10:25:44 +02:00
Dustin Brody 67e4a045a3 simplify doppelganger detection to boolean 2021-02-03 20:55:33 +02:00
Ștefan Talpalaru cadb1aae15
bump submodules (#2279)
* bump submodules

* undo a couple of bumps (32-bit failure)

* launch_local_testnet.sh: timeout and various fixes

* undo nim-http-utils bump (breaks RPC for VC)
2021-02-02 19:02:52 +01:00
tersec 6141286547
rename doppelganger protection to doppelganger detection; switch default from warn to stop (#2281) 2021-02-01 12:18:16 +01:00
Dustin Brody 281853eee8 rename options and internal structures to doppelgangerFoo and remove probing 2021-01-30 00:17:54 +02:00
Kim De Mey 40a5d44887
Fix selection of bootstrap nodes from metadata (#2273)
Also removes again the doubling of max peers
2021-01-29 08:56:02 +01:00
Dmitriy Ryajov 1117993a24
Libp2p connection limits (#2272)
* max peers gets set as connection limits

* pass max connections
2021-01-28 07:31:29 +01:00
Ștefan Talpalaru 20435880db CI: build and publish a Docker image for end-users
Also disable the log file and log colours for distribution binaries, to
avoid duplicate logs.
2021-01-26 18:52:59 +02:00
Dustin Brody a16f5afcd5 pre-emptive duplicate validator detection heuristic 2021-01-21 16:03:02 +02:00
Kim De Mey 6fabefa76f
Fix discv5 loop in case of no nodes (#2243)
* Fix discovery loop in case of no peers in routing table

* local testnet: Stop searching when amount of peers in testnet is reached
2021-01-18 14:13:26 +01:00
asymmetric 51f967a548
scripts: make shebang more portable (#2216)
Some Linux distros (like NixOS) don't place bash in /bin/bash.
2021-01-08 11:29:43 +01:00
Ștefan Talpalaru e58a355a1c
CI release: ARM and ARM64 builds (#2213) 2021-01-07 10:19:29 +01:00
Ștefan Talpalaru 92750d5313 reduce peak memory usage during build 2020-12-10 17:17:12 +02:00
Ștefan Talpalaru ad54e9d3bf
Prometheus/Grafana changes (#2092)
- grafana/prometheus.yml.example
- drop the custom label
- raise the minimum supported Grafana version to 7.2
- increase scrape_interval
- add validator reward rate panel (Zahary's)
2020-12-02 21:47:00 +01:00
Ștefan Talpalaru 9c5cef346b set file and dir permissions 2020-11-17 23:28:26 +02:00
Zahary Karadjov 500301c6c7
Restore the NODE_ID functionality and use the old data directories 2020-11-10 10:11:13 +02:00
tersec a010ef2061
update tests from v1.0.0-rc.0 to v1.0.0 (#1976) 2020-11-09 17:44:18 +00:00
Zahary Karadjov 1774da15f1 Implement help message for the run-*-beacon-node scripts 2020-11-09 17:09:49 +02:00
Zahary Karadjov e9b9cd75ee Rename binaries; Mimic the original repo layout in the distribution 2020-11-09 11:38:52 +02:00
yslcrypto 6f9d0fa29e update beth1 client guide link 2020-11-07 15:16:53 +01:00
Zahary Karadjov 0ea68d61ed
Add another missing file 2020-11-07 13:51:14 +02:00
Ștefan Talpalaru f3254fd754
dist: add version and book (#1965) 2020-11-06 17:26:03 +01:00
Ștefan Talpalaru c7d8b81c86 dist: add Medalla binary and starting script 2020-11-06 01:29:03 +02:00
cheatfate 4caaf86332
Fix permissions on script. 2020-10-27 13:58:56 +02:00
Eugene Kabanov eee01a32d5
Regression fix of eth2_network_simulation on Windows. (#1900)
* Concentrate all sensitive writeFile/createPath calls in one place.
Fix eth2_network_simulation for Windows.

* Remove artifacts.

* fix import

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-10-27 12:04:17 +01:00
tersec c446eb7cff
update v1.0.0-rc.0 preset spec references (#1891)
* update ve1.0.0-rc.0 preset spec references

* remove runtime preset ETH1_FOLLOW_DISTANCE from preset files; remove two CI build items to try to keep Travis from timing out
2020-10-19 13:53:23 +00:00
cheatfate f632a61cb9 Fix posix version. 2020-10-15 20:18:33 +03:00
cheatfate e0fcda090f Fix executable permissions. 2020-10-15 20:18:33 +03:00
cheatfate 84fe294c00 Fix protection of data folder and security files using Windows ACL. 2020-10-15 20:18:33 +03:00
Zahary Karadjov aed291128a Add support for starting from weak subjectivity checkpoints
Also removes the `genesis.ssz` file stored in the data folder.
The `medalla-fast-sync` target has been adapted to use the new features.
2020-10-07 09:32:03 +03:00
cheatfate 20f695515a Fix localtestnet simulation. 2020-10-05 22:19:50 +03:00
cheatfate 28630a1d71 Create DATA_DIR explicitly with 0750 permissions. 2020-10-05 22:19:50 +03:00
cheatfate add22a20e1 Update local_testnet and simulation scripts to use netkey-file and insecure-netkey-password.
Add more logging
2020-10-05 22:19:50 +03:00
Zahary Karadjov f5340998b9 Add a fuzzing test for the beacon node CLI parser 2020-10-02 12:42:32 +03:00
tersec 7eaaab908c
fix output of proposer slashing test fixture (#1780)
* fix output of proposer slashing test fixture

* run make test

* a few more v0.12.3 spec refs
2020-09-30 13:12:03 +00:00
tersec 5d1ea5e440
match log filenames more precisely for logtrace analysis (#1617)
* match log filenames more precisely for logtrace analysis
2020-09-09 13:08:21 +00:00
Viktor Kirilov d9f9949ef0 use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process 2020-09-02 12:47:00 +03:00
Viktor Kirilov 65d7787b1e 50/50 bn/vc split for the validator keys ON by default for the testnet scripts 2020-09-01 16:39:07 +03:00
tersec b4f9908d9d
run logtrace in finalization CI (#1512)
* run logtrace in finalization CI

* use specified DATA_DIR rather than assuming default

* convert rest of hardcoded local_testnet_data references

* logtrace asr needs DEBUG-level logs

* clean up some shell (not) quoting issues
2020-08-17 08:36:29 +02:00
tersec bc6eefe31e
add --enable-logtrace argument to launch_local_testnet (#1502)
* add --enable-logtrace argument to launch_local_testnet

* scan for all available logfiles

* remove specific filename references

* update v0.11.3 spec ref to v0.12.2
2020-08-16 11:12:19 +02:00
Mamy Ratsimbazafy 454b9d0724
Bump nim-blscurve (#1491)
* Bump BLSCurve

* Use unified aggregation API

* use new blscurve with unified aggregate API

* bump

* fix toRaw

* replace state_sim combine with AggregateSignature

* Fix 32-bit

* Fix 32-bit for real and test deactivating ccache for fno-tree-lopp-vectorize flag

* change compilation switches to narrow down Linux issue

* Use -fno-tree-vectorize to disable both tree-loop-vectorize and tree-slp-vectorize

* blscurve now disables both Loop and SLP vectorization

* Add tests for the miracl/milagro fallback

* Travis has max log size of 4MB

* Test with Miracl in the finalization test

* fix state_sim log level

* Coment out the slow fallback tests
2020-08-15 19:33:58 +02:00
tersec af3355e0f8
create local testnet mode for eth2_network (#1494) 2020-08-12 14:16:59 +00:00
Zahary Karadjov d58668157a Use consistent naming for the synonyms 'deposits-dir' and 'validators-dir' 2020-08-02 23:00:43 +03:00
Ștefan Talpalaru 6a30c4a67d
launch_local_testnet.sh: add "--reuse-existing-data-dir" (#1396)
- also unify the indentation towards the least readable denominator, due
  to ".editorconfig" making my Vim display tabs as 2 spaces wide.
2020-07-30 02:46:10 +02:00
Ștefan Talpalaru fa9f35e148
Jenkins: run local testnet test on macOS (#1391) 2020-07-29 14:08:27 +02:00
Zahary Karadjov f4c19e303a Non-interactive generation of keystores in the local sim 2020-07-28 07:36:25 +03:00
Zahary Karadjov 40ea9e9886 Integrate the Wallet support with the Deposits creation; Produce Launchpad-compatible deposits 2020-07-28 07:36:25 +03:00
tersec 20a2525390
v0.12.2 beacon chain protocol update (#1378) 2020-07-27 12:59:57 +02:00