Commit Graph

3800 Commits

Author SHA1 Message Date
tersec c5fa7387f9
update kintsugi docs (#3104) 2021-11-15 17:01:27 +00:00
tersec fde73b5a70
bump nim-web3 to switch payload ID from uint64 to 8-byte buffer (#3103) 2021-11-15 14:55:03 +00:00
Zahary Karadjov 5e62df5ad5 /bin/bash -> /usr/bin/env bash (needed on NixOS) 2021-11-15 15:45:01 +02:00
tersec b4d27b36bc
rename eth1Chain to depositsChain (#3094) 2021-11-15 12:01:47 +00:00
tersec ab742468e8
rename EthAddress to ExecutionAddresss per spec (#3100) 2021-11-15 10:36:07 +00:00
tersec 6f8eff8f13
merge topic sub/unsub infrastructure (#3099) 2021-11-14 09:00:25 +01:00
tersec 8cab7a74e7
remove shared/altair-devnet-3 support (#3098) 2021-11-14 08:57:27 +01:00
tersec 00d066f7dd
add merge gossip validation, except for beaconblocks (#3095) 2021-11-13 22:26:02 +01:00
Jacek Sieka 222674b203
better logging on invalid database (#3097)
* remove redundant test report
2021-11-13 17:27:28 +01:00
Jacek Sieka f3881cbf73
Merge remote-tracking branch 'origin/stable' into unstable 2021-11-13 11:04:32 +01:00
Jacek Sieka b22d86e161
REST/JSON-RPC: speed up several requests (#3092)
REST/JSON-RPC and a few more also invalidate caches unnecessarily,
similar to https://github.com/status-im/nimbus-eth2/pull/3089

* avoid copying validator on balance request
2021-11-12 23:29:28 +01:00
Kim De Mey 2da9309d2a
Remove SSZ files forgotten in move to ssz repo (#3091) 2021-11-12 13:36:35 +01:00
Jacek Sieka 00bbc8e0fe
speed up epoch processing 6x+ (#3089)
* speed up epoch processing 6x+

This change above all helps contain long replay times on epoch change, reorg
and deep history inspection via REST/RPC

* most effective balances don't actually change due to MAX_EFFECTIVE_BALANCE
* ditto for inactivity scores
* avoid signature check for trusted sync aggregates

pre:
```
./ncli_db --db:mainnet_0/db bench --start-slot=-3200
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed

    3468.621,        0.000,     3468.621,     3468.621,            1, Initialize DB
       0.357,        0.938,        0.171,       52.752,         3155, Load block from database
   15691.471,        0.000,    15691.471,    15691.471,            1, Load state from database
       6.100,        9.469,        0.033,      526.816,         3101, Advance slot, non-epoch
     579.131,        9.523,      566.936,      610.328,          100, Advance slot, epoch
      18.551,       16.317,       12.664,      136.668,         3155, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

post:
```
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    3488.541,        0.000,     3488.541,     3488.541,            1, Initialize DB
       0.369,        1.123,        0.183,       63.208,         3155, Load block from database
   13430.642,        0.000,    13430.642,    13430.642,            1, Load state from database
       6.522,        1.721,        0.034,       36.708,         3101, Advance slot, non-epoch
      89.074,        3.162,       83.573,      101.436,          100, Advance slot, epoch
      18.325,       18.346,       13.005,      145.040,         3155, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

* Update beacon_chain/spec/state_transition_block.nim

Co-authored-by: zah <zahary@gmail.com>

* avoid copying validator data in accessor

```
    5291.227,        0.000,     5291.227,     5291.227,            1, Initialize DB
       0.436,        0.928,        0.138,       51.438,         3155, Load block from database
   11962.826,        0.000,    11962.826,    11962.826,            1, Load state from database
       6.477,        1.675,        0.037,       34.174,         3101, Advance slot, non-epoch
      76.633,        3.705,       71.106,       98.085,          100, Advance slot, epoch
      18.301,       18.593,       13.208,      149.153,         3155, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

* work around compiler bug

Co-authored-by: zah <zahary@gmail.com>
2021-11-11 20:24:29 +01:00
Mamy Ratsimbazafy 9f4a55b82b
SSZ markdown test update missing [skip ci] (#3086) 2021-11-10 14:47:02 +01:00
tersec 97ad5d49b0
kintsugi merge vector tests (#3072) 2021-11-10 13:41:02 +02:00
Jacek Sieka ec650c7fd7
Support starting from altair (#3054)
* Support starting from altair

* hide `finalized-checkpoint-` - they are incomplete and usage may cause
crashes
* remove genesis detection code (broken, obsolete)
* enable starting ChainDAG from altair checkpoints - this is a
prerequisite for checkpoint sync (TODO: backfill)
* tighten checkpoint state conditions
* show error when starting from checkpoint with existing database (not
supported)
* print rest-compatible JSON in ncli/state_sim
* altair/merge support in ncli
* more altair/merge support in ncli_db
* pre-load header to speed up loading
* fix forked block decoding
2021-11-10 13:39:08 +02:00
kdeme 03a70fbf36 Use nim-ssz-serialization module and rm local ssz code 2021-11-10 13:37:24 +02:00
Ștefan Talpalaru b1218c6cd1
Version 1.5.4 2021-11-10 11:55:57 +01:00
Ștefan Talpalaru 782318d3fc
Nim-1.2.14 (#3074)
* Nim-1.2.14

* bump submodules
2021-11-10 11:39:43 +01:00
Zahary Karadjov df31c46cb3 Version 1.5.3 2021-11-10 11:02:45 +02:00
zah 2c0be052a5
Reduce the logging-related breaking changes (#3070)
* Reduce the logging-related breaking changes

* Don't disable the stdout log when the `--log-file` option is used
* Rename `--log-stdout` to `--log-format` and hide it
* Deprecate the `--log-file` option
2021-11-10 11:02:18 +02:00
tersec 5c48982280
a dozen spec URL updates to v1.1.5 (#3078) 2021-11-10 08:12:41 +00:00
tersec 59db4b4f9c
remove intsets usage (#3075) 2021-11-10 02:18:52 +00:00
zah 3545d4d1e1
Address review comments in #3057 (#3069)
* Address review comments in #3057

* reorder imports in rest_utils

maybe this will help with the mysterious serialization issues

Co-authored-by: Jacek Sieka <jacek@status.im>
2021-11-09 20:21:36 +01:00
tersec 941eb609ba
update statediffs to work with Altair (#3061)
* update statediffs to work with Altair
2021-11-09 16:17:57 +00:00
sacha 9d82606eba
Update logging.md 2021-11-09 15:38:48 +01:00
sacha db612ae906
Update logging.md
add warning
2021-11-09 15:38:32 +01:00
sacha 87fa67045f
Update logging.md
replace `--log-stdout` with -`-log-format`. remove references to `--log-file`
2021-11-09 15:21:48 +01:00
Zahary Karadjov 66d60e47ae Bump Chronicles 2021-11-09 13:33:53 +02:00
Zahary Karadjov 200902ccb3 Don't start the Eth1Monitor when there are no validators attached
This is a temporary revert of the new logic introduced in
d6cd1cd46c, because it leads
to a memory leak in the Eth1Monitor caused by the lack of
pruning of the Eth1Chain when the validator duties are not
executed.
2021-11-09 12:23:11 +02:00
Zahary Karadjov ba42b2b316 Correct implementation of the /validator/duties/sync/{epoch} API
According to the spec, this call should return the positions of
the specified validators within the sync committee. The existing
code was instead returning the indices of the sync sub-committees
where the validator is a member.
2021-11-09 11:45:00 +02:00
tersec 48eba59971
manually-verified v1.1.5 spec URL updates (#3068) 2021-11-09 08:54:59 +00:00
tersec 2e868dc2ba
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067) 2021-11-09 07:40:41 +00:00
tersec b76326f4b8
use v1.1.5 consensus spec test vectors (#3066) 2021-11-09 06:46:24 +00:00
tersec 95b0ecc5a2
only invalidate {current,previous}_epoch_participation flag cache once (#3063) 2021-11-09 02:44:02 +00:00
tersec a0f6a19267
bump nim-web3 for Kintsugi engine API changes (#3065) 2021-11-09 02:43:51 +00:00
sacha 624345bcee
Update logging.md
mainly style edits
2021-11-08 22:05:11 +01:00
Tanguy 8092418746
Bump libp2p (Short agent fix) (#3064)
* cherry-pick libp2p #645

* switch to master
2021-11-08 21:40:23 +01:00
tersec 5c296c08b7
17 spec URL updates to v1.1.4 (#3060) 2021-11-08 10:30:52 +00:00
tersec 2c8600e746
mass/mechanical update of 1.1.3 phase0 spec URLs to 1.1.4 in markdown (#3059) 2021-11-08 09:26:18 +00:00
tersec eb3ad25859
mass/mechanical update of 1.1.3 phase/altair spec URLs to 1.1.4 (#3058) 2021-11-08 06:18:10 +00:00
Dustin Brody 023a6a9387 use v1.1.4 test vectors 2021-11-07 21:42:11 +02:00
Zahary Karadjov 29e5700838 Bugfix: Avoid the aggregation of duplicate signatures when creating sync committee contributions 2021-11-07 21:41:10 +02:00
Ștefan Talpalaru 74f03ddadb
CI: statically check stack size on Linux (#3055) 2021-11-05 18:11:25 +01:00
Jacek Sieka ea0a191723
Better REST/RPC error messages (#3046)
* Better REST/RPC error messages
* homogenise block logging (root first)
* homegenise message verification pipeline (verify in
`gossip_verification`, act in `eth2_processor`)
* use `subcommitteeIdx` consistently
* log each sent contribution
* fix block_sim
* fix block topic
* don't recalc root on gossip block validation
* move position loop into sync pool
2021-11-05 17:39:47 +02:00
Ștefan Talpalaru b09cd71348 build: show stderr by default
Also added: `-d:cwarnings` and `-d:limitStackUsage`
2021-11-05 16:06:48 +02:00
sacha 122055b65e
cp (#3053) 2021-11-05 11:33:28 +01:00
Jacek Sieka a086cf01ac
altair fork handling cleanups (#3050)
* fix stack overflow crash in REST/debug/getStateV2
* introduce `ForkyXxx` for generic type matching of `Xxx` across
branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) -
`Some` is already used for other types of type classes
* consolidate function naming in BeaconChainDB, use some generics
* import `forks.nim` from other spec modules and move `Forked*` helpers
around to resolve circular imports
* remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout
(less data shuffling between the types)
* fix several cases of states being stored on stack in tests, causing
random failures on some platforms
* remove reading json support from ncli - this should be ported to the
rest json reading instead (doesn't currently work because stack sizes)
2021-11-05 08:34:34 +01:00
Jacek Sieka 98668dbf16
reuse pre-computed root for sig verification (#3048)
no point re-hashing arguments
2021-11-05 05:50:01 +00:00
Jacek Sieka 3bfb5ad367
logtrace needs json logs (#3047) 2021-11-03 07:24:03 +01:00