Commit Graph

4064 Commits

Author SHA1 Message Date
sacha 624345bcee
Update logging.md
mainly style edits
2021-11-08 22:05:11 +01:00
Tanguy 8092418746
Bump libp2p (Short agent fix) (#3064)
* cherry-pick libp2p #645

* switch to master
2021-11-08 21:40:23 +01:00
tersec 5c296c08b7
17 spec URL updates to v1.1.4 (#3060) 2021-11-08 10:30:52 +00:00
tersec 2c8600e746
mass/mechanical update of 1.1.3 phase0 spec URLs to 1.1.4 in markdown (#3059) 2021-11-08 09:26:18 +00:00
tersec eb3ad25859
mass/mechanical update of 1.1.3 phase/altair spec URLs to 1.1.4 (#3058) 2021-11-08 06:18:10 +00:00
Dustin Brody 023a6a9387 use v1.1.4 test vectors 2021-11-07 21:42:11 +02:00
Zahary Karadjov 29e5700838 Bugfix: Avoid the aggregation of duplicate signatures when creating sync committee contributions 2021-11-07 21:41:10 +02:00
Ștefan Talpalaru 74f03ddadb
CI: statically check stack size on Linux (#3055) 2021-11-05 18:11:25 +01:00
Jacek Sieka ea0a191723
Better REST/RPC error messages (#3046)
* Better REST/RPC error messages
* homogenise block logging (root first)
* homegenise message verification pipeline (verify in
`gossip_verification`, act in `eth2_processor`)
* use `subcommitteeIdx` consistently
* log each sent contribution
* fix block_sim
* fix block topic
* don't recalc root on gossip block validation
* move position loop into sync pool
2021-11-05 17:39:47 +02:00
Ștefan Talpalaru b09cd71348 build: show stderr by default
Also added: `-d:cwarnings` and `-d:limitStackUsage`
2021-11-05 16:06:48 +02:00
sacha 122055b65e
cp (#3053) 2021-11-05 11:33:28 +01:00
Jacek Sieka a086cf01ac
altair fork handling cleanups (#3050)
* fix stack overflow crash in REST/debug/getStateV2
* introduce `ForkyXxx` for generic type matching of `Xxx` across
branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) -
`Some` is already used for other types of type classes
* consolidate function naming in BeaconChainDB, use some generics
* import `forks.nim` from other spec modules and move `Forked*` helpers
around to resolve circular imports
* remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout
(less data shuffling between the types)
* fix several cases of states being stored on stack in tests, causing
random failures on some platforms
* remove reading json support from ncli - this should be ported to the
rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 08:34:34 +01:00
Jacek Sieka 98668dbf16
reuse pre-computed root for sig verification (#3048)
no point re-hashing arguments
2021-11-05 05:50:01 +00:00
Jacek Sieka 3bfb5ad367
logtrace needs json logs (#3047) 2021-11-03 07:24:03 +01:00
Etan Kissling 51aef34be7
introduce strong typing for sync committee periods (#3040)
The sync committee period used to be a plain `uint64`. With the light
client sync relying more heavily on them, it makes sense to introduce
a proper type for them, similar to how they are already used for `Slot`
and `Epoch`. This introduces such a `SyncCommitteePeriod` type.
Furthermore, some usage code dealing with those periods is cleaned up.
2021-11-02 21:32:34 +01:00
Etan Kissling 1943258c66
extend REST config with more spec constants (#2962)
So far, the REST config response did not include all spec constants.
The spec for `/eth/v1/config/spec` defines that the response should
include constants for all hard forks known by the beacon node. This
patch extends the corresponding response to include more constants.
2021-11-02 19:23:31 +01:00
Jacek Sieka 233d756518
Logging and startup improvements (#3038)
* Logging and startup improvements

Color support for released binaries!

* startup scripts no longer log to file by default - this only affects
source builds - released binaries don't support file logging
* add --log-stdout option to control logging to stdout (colors, json)
* detect tty:s vs redirected logs and log accordingly
* add option to disable log colors at runtime
* simplify several "common" logs, showing the most important information
earlier and more clearly
* remove line numbers / file information / tid - these take up space and
are of little use to end users
  * still enabled in debug builds and tools
* remove `testnet_servers_image` compile-time option
* server images, released binaries and compile-from-source now offer
the same behaviour and features
* fixes https://github.com/status-im/nimbus-eth2/issues/2326
* fixes https://github.com/status-im/nimbus-eth2/issues/1794
* remove instanteneous block speed from sync message, keeping only
average

before:

```
INF 2021-10-28 16:45:59.000+02:00 Slot start                                 topics="beacnde" tid=386429 file=nimbus_beacon_node.nim:884 lastSlot=2384027 wallSlot=2384028 delay=461us84ns peers=0 head=75a10ee5:3348 headEpoch=104 finalized=cd6804ba:3264 finalizedEpoch=102 sync="wwwwwwwwww:0:0.0000:0.0000:00h00m (3348)"
INF 2021-10-28 16:45:59.046+02:00 Slot end                                   topics="beacnde" tid=386429 file=nimbus_beacon_node.nim:821 slot=2384028 nextSlot=2384029 head=75a10ee5:3348 headEpoch=104 finalizedHead=cd6804ba:3264 finalizedEpoch=102 nextAttestationSlot=-1 nextProposalSlot=-1 nextActionWait=n/a
```

after:

```
INF 2021-10-28 22:43:23.033+02:00 Slot start                                 topics="beacnde" slot=2385815 epoch=74556 sync="DDPDDPUDDD:10:5.2258:01h19m (2361088)" peers=37 head=eacd2dae:2361096 finalized=73782:a4751487 delay=33ms687us715ns
INF 2021-10-28 22:43:23.291+02:00 Slot end                                   topics="beacnde" slot=2385815 nextActionWait=n/a nextAttestationSlot=-1 nextProposalSlot=-1 head=eacd2dae:2361096
```

* fix comment

* documentation updates

* mention `--log-file` may be deprecated in the future
* update various docs
2021-11-02 18:06:36 +01:00
sacha 56c6d6b583
update troubleshooting (#3044)
* update troubleshooting

* remove pyrmont references
2021-11-01 17:46:59 +01:00
Jacek Sieka d6cd1cd46c
Remove web3-mode, always keep web3 monitor enabled when given (#3042)
* Init some components fully before BeaconNode, per component dependency
graph
* remove `--web3-mode` option
* fixes https://github.com/status-im/nimbus-eth2/issues/2685
* reshuffle some beacon node init code
2021-11-01 15:50:24 +01:00
Etan Kissling 74f2350a2c
update nim-presto to bfcbece (#3039)
Upstream changes:
- allow reserved Nim keywords as REST params
2021-10-31 19:51:05 +01:00
tersec 36e37bda40
v1.1.3 spec refs URLs (#3036) 2021-10-27 18:40:17 +00:00
Eugene Kabanov 7c9a6b7170
Improve chronos.Future tracking. (#2988)
* Add `child_id` field.

* Fix json-rpc api call and bump chronos.

* Bump chronos master and fix compilation warnings.

* One more bump of `chronos`.

* add random import

* export rest_utils a bit more

Co-authored-by: Jacek Sieka <jacek@status.im>
2021-10-27 14:01:11 +02:00
Etan Kissling 23b5a945ea
merge stable into unstable 2021-10-27 09:58:24 +02:00
Etan Kissling fdbfe19b3e
fix CI for contributions from forked repos (#3028)
This updates the publish-unit-test-result integration with support for
pull requests coming from forked repositories. The update changes the
behaviour so that (a) unit test results are displayed as a comment, and
(b) macOS and Windows builds no longer time out during the CI action.
The official guide for the publish-unit-test-result action has been
followed for this purpose. See the official readme at:
https://github.com/EnricoMi/publish-unit-test-result-action/blob/v1.23/README.md#support-fork-repositories-and-dependabot-branches
2021-10-27 00:49:35 +02:00
tersec 968a5b3bec
some spec refs to v1.1.3 (#3032) 2021-10-26 18:42:48 +00:00
tersec 8307e9c601
mechanical non-merge v1.1.2 to v1.1.3 spec URL updates (#3030) 2021-10-26 16:44:23 +00:00
Etan Kissling d32c25ea32
use `GeneralizedIndex` type for spec constants (#3026)
Spec constants currently use `uint16` instead of `GeneralizedIndex` to
define generalized indices in a merkle tree. Adjusted to use the latter
instead to avoid problems when adjusting `GeneralizedIndex`'s bit width.
2021-10-26 12:30:36 +00:00
sacha 07e4dd5f23
edit networking, remove references to pyrmont (#3027) 2021-10-25 19:40:29 +02:00
Eugene Kabanov 7c49d657a6
Fix regression in REST validators and validator_balances requests. (#3022)
* Fix state validators and validator_balances responses when validator's identifier could not be found in state.

* Add tests.
2021-10-25 15:52:00 +02:00
tersec c10f504a03
bump nim-web3 (#3024) 2021-10-22 13:55:48 +00:00
Kim De Mey f88b7f60ad
Remove unused datatypes import from ssz (#3019) 2021-10-21 19:31:43 +02:00
Ștefan Talpalaru 5339800ae4
Grafana: update dashboards (#3017) 2021-10-21 17:15:35 +02:00
Ștefan Talpalaru 99113a4b62
new metric: network_name (#3018) 2021-10-21 17:13:35 +02:00
sacha dc3c23788a
update networking, update hardware requirements, set up health.md (#3020) 2021-10-21 16:18:46 +02:00
sacha c3c8e27011
Network setup update (#2999)
* cp

* cp

* Update docs/the_nimbus_book/src/faq.md

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>

* Update docs/the_nimbus_book/src/faq.md

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>

* Update docs/the_nimbus_book/src/networking.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* Update docs/the_nimbus_book/src/networking.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* Update docs/the_nimbus_book/src/networking.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* Update docs/the_nimbus_book/src/networking.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* Update docs/the_nimbus_book/src/networking.md

Co-authored-by: Jacek Sieka <jacek@status.im>

* fix links

* new line

* extip edits

* update read logs

* cp

* cp

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
Co-authored-by: Jacek Sieka <jacek@status.im>
2021-10-21 15:34:46 +02:00
Jacek Sieka 421bf936ff
odds and ends (#3015)
* `allSyncCommittees` => `allSyncSubcommittees`
* simplify `_snappy` topic generation (avoid pointless string copies)
* simplify gossip id generator (avoid pointless string copies)
* avoid redundant syncnet ENR updates
* simplify topic validation (allow only validated topics)
2021-10-21 15:09:19 +02:00
nbc-bump-bot[bot] 29b26f3f6b
nim-libp2p auto bump (#2840)
* auto-bump nim-libp2p

* Remove peer info for other peers

Not definitive, just to test the libp2p's unstable branch

* finish up Remove peer info for other peers

* getKey -> getPublicKey

* bump libp2p

* libp2p bump

Co-authored-by: = <Menduist@users.noreply.github.com>
Co-authored-by: Tanguy <tanguy@status.im>
2021-10-21 13:01:29 +02:00
Etan Kissling 15ce2de3f0 avoid func call when merkleizing UintN arrays
This gets rid of an unnecessary function call when merkleizing `UintN`
arrays on `littleEndian` architectures.
2021-10-21 11:26:03 +03:00
Zahary Karadjov 052bec1c1f remove BETA tag on rest command line help 2021-10-21 11:24:22 +03:00
Zahary Karadjov 9e8081e405 Merge branch 'stable' into unstable 2021-10-21 04:13:23 +03:00
Zahary Karadjov 3b3d8b7431 Version 1.5.2 2021-10-21 04:11:56 +03:00
Etan Kissling f5791122f6 widen allowed specs for validator client
The validator client was only able to connect to beacon nodes exposing
the exact same set of spec constants that are locally known via their
config/spec REST API. However, that set of spec constants is dynamic.
As the validator client only requires a subset of relevant constants,
this may lead to compatible specs being rejected. This patch widens the
allowed specs by only verifying that the required set of constants are
present in the spec response, ignoring any spec constants that are not
locally known, and ignoring missing spec constants that are locally
known but not included by the remote beacon node when not relevant for
operation of the validator client.
2021-10-21 00:59:05 +03:00
Etan Kissling 9b334c31f1 fix big-endian merkleization for UintN arrays
UintN arrays were incorrectly merkleized on big-endian. This was fixed
by making sure to use the correct buffer to store the final chunk.
2021-10-21 00:57:14 +03:00
Etan Kissling 4a3e94702e allow `ElemType(XXX)` on inputs not called `T`
Currently, `ElemType` can only be called on `HashArray`, `HashList` and
`List` types when they are first assigned to a variable called `T`. The
templates have been adjusted so that different variable names may be
used. The templates can now also be applied to in-line computed types,
e.g., `ElemType(typeof(x))`.
2021-10-21 00:50:01 +03:00
Etan Kissling 6720e6e2a6 fix running local validator client simulation
Updated the local network simulation scripts to use the REST interface
for connecting the validator client to the beacon node. Otherwise, the
current scripts produce errors.
2021-10-21 00:47:33 +03:00
Ștefan Talpalaru 7a4b154705 Grafana: update metrics.status.im dashboard 2021-10-21 00:42:17 +03:00
Jacek Sieka 9cf32c3748 clean up sync subcommittee handling
* `SyncCommitteeIndex` -> `SyncSubcommitteeIndex`
* `syncCommitteePeriod` -> `sync_committee_period` (spec spelling)
* tighten period comparisons
* fix assert when validating committee message with non-altair state in
REST api
2021-10-20 22:59:13 +03:00
Jacek Sieka bf6ad41d7d add drop and sync committee metrics
* use storeBlock for processing API blocks
* avoid double block dump
* count all gossip metrics at the same spot
* simplify block broadcast
2021-10-20 18:20:12 +03:00
Etan Kissling 47343ff7c4 fix `markleizer` typo (-> `merkleizer`)
Fix for a typo in a variable name.
2021-10-20 15:19:57 +03:00
Jacek Sieka c247702ebc normalize subnet logging
* call it subnet id everywhere
* log aggregate sent from VC
* log subnet with aggregate
2021-10-20 15:06:44 +03:00