Commit Graph

202 Commits

Author SHA1 Message Date
Zahary Karadjov b46c8410a6 Retire Spadina and prepare for the Zinken network 2020-10-07 18:12:21 +03:00
Ștefan Talpalaru e9b2253fac
bump nim-nat-traversal and nimbus-build-system (#1824)
* bump nim-nat-traversal and nimbus-build-system

The "clean-cross" target has been moved to the latter.

* Windows fix
2020-10-07 12:01:27 +02:00
Zahary Karadjov 63d2e56255 Fix the medalla-fast-sync target 2020-10-07 09:32:03 +03:00
Zahary Karadjov aed291128a Add support for starting from weak subjectivity checkpoints
Also removes the `genesis.ssz` file stored in the data folder.
The `medalla-fast-sync` target has been adapted to use the new features.
2020-10-07 09:32:03 +03:00
Ștefan Talpalaru dc428e00db
Makefile: macOS fix 2020-10-06 17:18:02 +02:00
cheatfate 0101bca828 Fix rebase issues. 2020-10-05 22:19:50 +03:00
cheatfate 582402d1b9 Fix mkdir on Windows. 2020-10-05 22:19:50 +03:00
cheatfate c5c788a9db Secure network key file and data directory. 2020-10-05 22:19:50 +03:00
Mamy Ratsimbazafy 9f104d4344
more user friendly default log-level (#1777) 2020-09-29 17:55:58 +02:00
mratsim fe16fba07d Makefile fix for make 3.81 (macOS) 2020-09-28 19:13:36 +02:00
Ștefan Talpalaru 67b9f2b87f
Makefile: some cosmetic refactoring 2020-09-26 23:10:56 +02:00
Ștefan Talpalaru cdeab0308b
Makefile: fix typos 2020-09-26 22:55:46 +02:00
Ștefan Talpalaru 4c516a1fce
Makefile: fix clean-* targets 2020-09-25 18:15:27 +02:00
Zahary Karadjov 73724b2c4b Add attacknet-beta1-mc-0 2020-09-22 18:51:51 +03:00
Zahary Karadjov 10cfd1ec37 Adds the Spedina testnet; Upgrades the spec presets to spec v0.12.3
Other changes:

* Removed all references to Altona
* Refactored the Makefile to avoid code duplication
2020-09-22 18:51:51 +03:00
Ștefan Talpalaru 4c51eee7f9
USE_LIBBACKTRACE=0: speed up compilation by 3.31% (#1686)
* USE_LIBBACKTRACE=0: speed up compilation by 3.31%

* USE_LIBBACKTRACE=0: no line numbers in stack traces; blame Jacek
2020-09-21 08:40:01 +02:00
Ștefan Talpalaru ce85f58d29
better handling of Ctrl+C during `make update` (#1588) 2020-09-16 17:46:05 +02:00
Ștefan Talpalaru aa3298f0ea
support spaces in $(MAKE) (#1603) 2020-09-04 08:33:37 +02:00
Viktor Kirilov d9f9949ef0 use a separate process for the private keys (Off by default) - there is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process 2020-09-02 12:47:00 +03:00
Zed f2500127ed
Makefile: add clean-cross task for cross compiling (#1554) 2020-08-28 14:50:50 +02:00
Ștefan Talpalaru 504e96ca68 fix medalla-dev 2020-08-27 18:44:10 +03:00
Ștefan Talpalaru ecc70c6405 remove BUILD_LOG_LEVEL 2020-08-27 18:44:10 +03:00
Zahary Karadjov 273123d9c1 Add a CPU_LIMIT option for the Medalla and Altona targets [skip ci]
This can be used to trigger problems such as peer dropping that
manifest only on higher CPU loads.
2020-08-22 01:50:38 +03:00
Zahary Karadjov 21b5ad3259
make clearn-medalla will preserve your validator files 2020-08-21 21:50:46 +03:00
Zahary Karadjov 224ebdfd72 A simple metric for measuring the delay in the onSecond timer 2020-08-10 23:53:55 +03:00
Viktor Kirilov 5bbeb38f2d fixes the BN/VC communication - properly getting the attestation duties & also fixed start.sh 2020-08-06 15:29:05 +03:00
Zahary Karadjov c4268f954b
Another simple Makefile fix 2020-08-06 15:23:01 +03:00
Zahary Karadjov 0cdd13bf7c
Fix Prometheus-related error on 'make medalla' on fresh installations 2020-08-06 15:21:07 +03:00
Zahary Karadjov 4b8ebb5d71 Correct instructions in the README for running prometheus on Medalla 2020-08-05 19:28:35 +03:00
Ștefan Talpalaru 9522c611bc
Makefile: move `--dump` to *-dev targets (#1423) 2020-08-04 10:13:22 +02:00
D-Nice 7ab28c6a83
Fix Makefile dir typo (#1426) 2020-08-04 10:12:51 +02:00
Zahary Karadjov 8c2a9a4950
Switch back to the old undepleted Infura key; Bake in the Medalla genesis state 2020-08-03 03:59:45 +03:00
Zahary Karadjov d58668157a Use consistent naming for the synonyms 'deposits-dir' and 'validators-dir' 2020-08-02 23:00:43 +03:00
Zahary Karadjov 2a94f6eca6 medalla-deposit target capable of executing mass deposits 2020-08-02 23:00:43 +03:00
tersec af012ed4dc
combine ncli_hash_tree_root and ncli_pretty into ncli_ssz (#1393)
* combine ncli_hash_tree_root and ncli_pretty into ncli_ssz

* disable ncli_query building by default

* fold ncli_transition into ncli_ssz and rename to ncli
2020-07-29 21:19:36 +00:00
Viktor Kirilov 0ffcbd10e5 All logs should be stored in appropriate node's folder - related to #1349 2020-07-28 20:14:17 +03:00
Zahary Karadjov b801dd65b9 Makefile targets for Medalla 2020-07-28 19:27:45 +03:00
Zahary Karadjov f4c19e303a Non-interactive generation of keystores in the local sim 2020-07-28 07:36:25 +03:00
Zahary Karadjov 40ea9e9886 Integrate the Wallet support with the Deposits creation; Produce Launchpad-compatible deposits 2020-07-28 07:36:25 +03:00
Ștefan Talpalaru e9193fc9da
eth2_network_simulation: fixes (#1368)
- delete "tests/simulation/{data,validators}" by default, because old
  validator keys can lead to a crash with a cryptic error message
- actually start the Prometheus daemon on `make eth2_network_simulation`
- kill any running Prometheus daemon on exit
- fix some shell script syntax incompatible with Bash
2020-07-23 17:58:54 +02:00
Eugene Kabanov 431a399662
All logs should be stored in appropriate node's folder. (#1349) 2020-07-21 12:29:22 +02:00
Viktor Kirilov 3dfbc311ad log files also for the VC version of the altona target 2020-07-15 17:16:37 +03:00
Viktor Kirilov 4767eba465 - properly distributing the keys between BNs & VCs - fixing finalization - will switch to ON by default as soon as finalization becomes overall stable
- added a new altona-vc Makefile target which uses a separate VC process
2020-07-15 16:18:21 +03:00
zah 8970a22fdd
Add a non-optional Json file log for the beacon node as stopgap measure (#1322)
* Add a non-optional Json file log for the beacon node as stopgap measure

* Fix the build with -d:testnet_servers_image
2020-07-15 15:15:55 +02:00
Ștefan Talpalaru b01be646cf
Makefile: "publish-book" target combines both books (#1306)
Since both books share the same Git branch and the same base URL, it
makes sense to combine them when publishing.
2020-07-13 12:46:42 +02:00
Ștefan Talpalaru 61c6926263 fix docker/shared_testnet/entry_point.sh [skip ci] 2020-07-11 19:00:19 +03:00
Zahary Karadjov 87928e0069
Documentation; 'deposits create' can now ask for a private key 2020-07-10 18:18:14 +03:00
Zahary Karadjov d16676b294
Bake the metadata for testnet0 and testnet1 as well; Remove the old scripts 2020-07-10 17:39:54 +03:00
Zahary Karadjov 785d8b1950
Use the old altona data directories 2020-07-10 16:23:11 +03:00
Zahary Karadjov 6b721950a6
New Makefile targets for Altona 2020-07-10 01:41:51 +03:00
Mamy Ratsimbazafy 0f55ab13e8
The Auditors Handbook (#1280) - first pass
* initial auditor handbook commit

* Move in publishable state

* otu of-line comment to avoid mdbook bug

* Fix links

* Add threat model

* Update: move rocksdb and protobuf, placeholder for areas of focus
2020-07-07 10:44:21 +02:00
Mamy Ratsimbazafy 0b3a468883
Move the nimbus books in a subfolder to allow for more books (#1263) 2020-07-01 10:37:04 +02:00
tersec b4db2ad693
remove v0.11.3 support; add block_sim to CI (#1253)
* remove v0.11.3 support; add block_sim to CI

* rm stray PERSISTENT_COMMITTEE_PERIOD

* remove TopicFilter.InteropAttestations

* bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
Ștefan Talpalaru 20a0165809
Makefile: fix broken NIM_PARAMS substitution
it was causing local beacon_node instances to be built in debug mode
2020-06-24 13:42:25 +02:00
Ștefan Talpalaru 33ad3c05fe
connect_to_testnet.nims: add "--buildOnly" and "--runOnly" opts [skip ci]
and "--writeLogFile"
2020-06-24 12:54:17 +02:00
tersec 1b3c19dc21
default to v0.12.1 beacon chain spec, keeping witti working (#1222)
* default to v0.12.1 beacon chain spec, keeping witti working

* a couple of fixes that help for the Onyx testnet
2020-06-23 19:40:44 +00:00
Ștefan Talpalaru 62d5b5995d
Makefile: don't delete "build" in the "clean" target 2020-06-23 18:49:41 +02:00
Ștefan Talpalaru c06682f588
Makefile: move "nat-libs" to "deps" 2020-06-22 17:08:29 +02:00
Zahary Karadjov 7211a99086 Smooth out the differences between Ganache and Infura; Working local sim and Altona target 2020-06-22 17:30:04 +03:00
Zahary Karadjov 01ea66f24e Some documentation and better defaults 2020-06-22 17:30:04 +03:00
Zahary Karadjov 1def383ad1 Reword the Tmux setup to handle all simulation scenarios in more visible way 2020-06-22 17:30:04 +03:00
Zahary Karadjov 4275abbefe Makefile targets for ctail and ntu 2020-06-22 17:30:04 +03:00
Ștefan Talpalaru ed054d601f
Azure: reduce 32-bit Windows job runtime (#1168)
* Azure: lighten the load on 32-bit

* disable mainnet tests on 32-bit Windows
2020-06-14 04:25:19 +02:00
Ștefan Talpalaru f5ec1bc5bb
support multiple local Witti nodes [skip ci] 2020-06-11 18:42:58 +02:00
Ștefan Talpalaru e2025c5752
Prometheus & Grafana refactoring
- moved "process_dashboard.nim" in "tools/"
- README: made Witti the documented testnet and added instructions for
  getting metrics out of the local node
- moved Prometheus config file generation in its own script
- the static Grafana dashboard definition now covers all nodes, using
  a variable; only the remote testnet dashboards need to be dynamically
  generated
- "launch_local_testnet.sh" no longer needs a "--grafana" option
2020-06-10 19:55:26 +02:00
Jacek Sieka 061be037d1 ncli_db: database tool
includes a benchmark tool for now
2020-05-28 17:43:02 +00:00
Jacek Sieka f53b55cbe0 SSZ cleanup
* be stricter about SSZ length prefix
* compute zeroHash list at compile time
* remove SSZ schema stuff
* move SSZ navigation to ncli
* cleanup a few leftover openArray uses
2020-05-28 19:15:16 +03:00
Ștefan Talpalaru 489565b5ab
Makefile: small testnet targets refactoring (#1078) 2020-05-28 06:28:10 +02:00
kdeme a857168c38 Add witti make targets 2020-05-27 13:49:18 +03:00
Viktor Kirilov 8760494c72 first batch of work towards the VC/BN split:
- we have a new binary which connects via RPC to the respective BN and has an internal clock - waking it up on every slot
- the BN has a new option called --external-validators and currently in order to have the VC binaries to run we need to pass EXTERNAL_VALIDATORS=yes to make
- factored some code out of beacon_node.nim for easier reuse in validator_api.nim and validator_client.nim
- the VC loads its associated private keys from the datadir for its BN
- most of the validator API calls have been implemented as a stub.
- the VC polls its BN at the start of each epoch - getting a list of all active validators for the current epoch - and then continues to request blocks and sign them with its appropriate validators when necessary
2020-05-25 16:23:15 +03:00
Mamy Ratsimbazafy 1916e1d2da
Revive nbench (#1022)
* Fix nbench compilation with HashedBeaconState

* Add nbench to tooling

* use newClone - fix 265e01e404 (r425198575)

* Detail advance_slot and hashTreeRoot

* Report throughput

* Fallback for ARM

* windows does not support inline ASM
2020-05-15 00:54:10 +02:00
Ștefan Talpalaru 2d78523423
fix Grafana dashboards (#1006)
* fix Grafana dashboards

and refactor a couple of scripts

* start.sh: allow running standalone script
2020-05-11 18:35:53 +00:00
Jacek Sieka 01e9df97cb
cleanups (#962)
* remove broken serialized_sizes
* actually use sszdump module
* avoid bitops
* fix stack_sizes module name
2020-05-04 07:38:14 +02:00
Ștefan Talpalaru 3ac01140c6
Merge pull request #959 from status-im/publish
Makefile: add "publish-book" target
2020-05-02 20:57:23 +02:00
Ștefan Talpalaru 150aeaee74
make schlesi: fix NIMFLAGS and verbosity handling 2020-05-02 16:58:42 +02:00
Ștefan Talpalaru fa7d64c291
Makefile fix 2020-05-02 16:46:24 +02:00
Ștefan Talpalaru 43679bd04f
Makefile: add "publish-book" target
See https://github.com/rust-lang/mdBook/blob/master/book-example/src/continuous-integration.md#deploying-to-github-pages-manually
2020-05-02 16:30:34 +02:00
Jacek Sieka a3e098cf92
block pool simulator (#956)
* block pool simulator

like state_sim, but more
2020-05-01 17:51:24 +02:00
Zahary Karadjov fada11e1e9 Address review comments
* Create a json log named "nbc.log" only when the --dev-build option
  is specified to the connect_to_testnet script.

  This option is specified for the targets "testnet0", "testnet1"
  and "schlesi-dev"
2020-04-28 13:49:30 +03:00
Zahary Karadjov 667bdf066a Bump Chronicles to fix a minor bug; Add a placeholder for disabled topics 2020-04-28 13:49:30 +03:00
Zahary Karadjov 7237a370ba Implement https://github.com/status-im/nim-beacon-chain/issues/872
Please see the newly added 'schlesi-dev' Makefile target.
It demonstrates how the log level can be specified for individual topics.

Additionally, when connecting to testnets like 'schlesi' there will be
two additional log files produced in the working directory:

* json-log.txt
* text-log.txt (in the textblocks format)
2020-04-28 13:49:30 +03:00
Jacek Sieka 28d6cd2524
avoid memory allocations and copies when loading states (#942)
* rolls back some of the ref changes
* adds utility to calculate stack sizes
* works around bugs in nim exception handling and rvo
2020-04-28 10:08:32 +02:00
Zahary Karadjov 80b538452e Updated the custom preset loader to the latest spec; Working makefile target for connecting to the Schlesi testnet 2020-04-27 23:22:54 +03:00
Ștefan Talpalaru 39a893ea90
use the right preset in "make testnet{0,1}" [skip ci] (#932)
...and stop pretending to handle the preset in "connect_to_testnet.nims".
2020-04-24 17:28:03 +02:00
Ștefan Talpalaru 9974b4bd7b
Jenkins: frak it, we're doing it live 2020-04-19 13:36:11 +02:00
Ștefan Talpalaru 1babc00255
new initial submodule update strategy (#904)
* new initial submodule update strategy

* Azure: fix command order
2020-04-19 01:25:21 +02:00
Ștefan Talpalaru 7b8d6ea61c
bump vendor/nimbus-build-system (#894)
* bump vendor/nimbus-build-system

* change beacon_node banner

* beacon_node: update copyright years

* version.nim is imported in beacon_chain.nimble
2020-04-16 00:20:27 +02:00
Ștefan Talpalaru 3239b62904
move "--import:libbacktrace" to config.nims
This simplifies the `./env.sh nim c -r ...` scenario.
2020-03-31 14:53:41 +02:00
Zahary Karadjov d7c36a677e Remove p2pd from the build recipes 2020-03-23 14:59:33 +02:00
Mamy André-Ratsimbazafy 47ccae4ad6 Disable compilation of bench_bls_sig_agggregation 2020-03-13 12:20:41 +00:00
cheatfate abcc71daa0 Add logtrace to Makefile. 2020-03-11 17:26:30 +00:00
Ștefan Talpalaru 929fcf0036
eth2_network_simulation: propagate LOG_LEVEL [skip ci]
also add a "--log-level" option to launch_local_testnet.sh
2020-03-09 01:37:57 +01:00
Ștefan Talpalaru e533e9f344
Makefile: build libnfuzz in the default target 2020-03-05 18:56:07 +01:00
Ștefan Talpalaru 0b05fecedc
make SCRIPT_PARAMS="--skipGoerliKey" testnet1 2020-02-25 02:49:47 +01:00
Ștefan Talpalaru ad36808b7d
Make: add USE_LIBBACKTRACE flag 2020-02-20 17:41:10 +01:00
Ștefan Talpalaru 4d62f3f1e1
Merge branch 'simstart' into devel 2020-02-15 13:02:30 +01:00
Ștefan Talpalaru e72dfeed73
nim-libbacktrace: don't build the C++ wrapper 2020-02-15 12:58:58 +01:00
Ștefan Talpalaru 98160d3227
tests/simulation/start.sh: refactoring [skip ci]
to avoid NIMFLAGS duplication
2020-02-14 13:35:23 +01:00
Ștefan Talpalaru 06322385d9
lightweight stack traces
- plus some light test runner refactoring and some Makefile cosmetic changes
- compile tools with LOG_LEVEL=TRACE in CI
- bump a couple of submodules
2020-02-13 17:54:53 +01:00
Stefan Talpalaru 7feaa11372
make testnetX: propagate LOG_LEVEL to beacon_node runtime [skip ci]
This allows running `make LOG_LEVEL=TRACE testnet1` and having that log
level also enabled at runtime, not just at compile time.
2020-02-08 00:57:48 +01:00