Commit Graph

1335 Commits

Author SHA1 Message Date
tersec fde73b5a70
bump nim-web3 to switch payload ID from uint64 to 8-byte buffer (#3103) 2021-11-15 14:55:03 +00:00
Zahary Karadjov 5e62df5ad5 /bin/bash -> /usr/bin/env bash (needed on NixOS) 2021-11-15 15:45:01 +02:00
tersec 97ad5d49b0
kintsugi merge vector tests (#3072) 2021-11-10 13:41:02 +02:00
kdeme 03a70fbf36 Use nim-ssz-serialization module and rm local ssz code 2021-11-10 13:37:24 +02:00
Ștefan Talpalaru 782318d3fc
Nim-1.2.14 (#3074)
* Nim-1.2.14

* bump submodules
2021-11-10 11:39:43 +01:00
Zahary Karadjov 66d60e47ae Bump Chronicles 2021-11-09 13:33:53 +02:00
tersec b76326f4b8
use v1.1.5 consensus spec test vectors (#3066) 2021-11-09 06:46:24 +00:00
tersec a0f6a19267
bump nim-web3 for Kintsugi engine API changes (#3065) 2021-11-09 02:43:51 +00:00
Tanguy 8092418746
Bump libp2p (Short agent fix) (#3064)
* cherry-pick libp2p #645

* switch to master
2021-11-08 21:40:23 +01:00
Dustin Brody 023a6a9387 use v1.1.4 test vectors 2021-11-07 21:42:11 +02:00
Ștefan Talpalaru 74f03ddadb
CI: statically check stack size on Linux (#3055) 2021-11-05 18:11:25 +01:00
Ștefan Talpalaru b09cd71348 build: show stderr by default
Also added: `-d:cwarnings` and `-d:limitStackUsage`
2021-11-05 16:06:48 +02:00
Jacek Sieka 233d756518
Logging and startup improvements (#3038)
* Logging and startup improvements

Color support for released binaries!

* startup scripts no longer log to file by default - this only affects
source builds - released binaries don't support file logging
* add --log-stdout option to control logging to stdout (colors, json)
* detect tty:s vs redirected logs and log accordingly
* add option to disable log colors at runtime
* simplify several "common" logs, showing the most important information
earlier and more clearly
* remove line numbers / file information / tid - these take up space and
are of little use to end users
  * still enabled in debug builds and tools
* remove `testnet_servers_image` compile-time option
* server images, released binaries and compile-from-source now offer
the same behaviour and features
* fixes https://github.com/status-im/nimbus-eth2/issues/2326
* fixes https://github.com/status-im/nimbus-eth2/issues/1794
* remove instanteneous block speed from sync message, keeping only
average

before:

```
INF 2021-10-28 16:45:59.000+02:00 Slot start                                 topics="beacnde" tid=386429 file=nimbus_beacon_node.nim:884 lastSlot=2384027 wallSlot=2384028 delay=461us84ns peers=0 head=75a10ee5:3348 headEpoch=104 finalized=cd6804ba:3264 finalizedEpoch=102 sync="wwwwwwwwww:0:0.0000:0.0000:00h00m (3348)"
INF 2021-10-28 16:45:59.046+02:00 Slot end                                   topics="beacnde" tid=386429 file=nimbus_beacon_node.nim:821 slot=2384028 nextSlot=2384029 head=75a10ee5:3348 headEpoch=104 finalizedHead=cd6804ba:3264 finalizedEpoch=102 nextAttestationSlot=-1 nextProposalSlot=-1 nextActionWait=n/a
```

after:

```
INF 2021-10-28 22:43:23.033+02:00 Slot start                                 topics="beacnde" slot=2385815 epoch=74556 sync="DDPDDPUDDD:10:5.2258:01h19m (2361088)" peers=37 head=eacd2dae:2361096 finalized=73782:a4751487 delay=33ms687us715ns
INF 2021-10-28 22:43:23.291+02:00 Slot end                                   topics="beacnde" slot=2385815 nextActionWait=n/a nextAttestationSlot=-1 nextProposalSlot=-1 head=eacd2dae:2361096
```

* fix comment

* documentation updates

* mention `--log-file` may be deprecated in the future
* update various docs
2021-11-02 18:06:36 +01:00
Etan Kissling 74f2350a2c
update nim-presto to bfcbece (#3039)
Upstream changes:
- allow reserved Nim keywords as REST params
2021-10-31 19:51:05 +01:00
Eugene Kabanov 7c9a6b7170
Improve chronos.Future tracking. (#2988)
* Add `child_id` field.

* Fix json-rpc api call and bump chronos.

* Bump chronos master and fix compilation warnings.

* One more bump of `chronos`.

* add random import

* export rest_utils a bit more

Co-authored-by: Jacek Sieka <jacek@status.im>
2021-10-27 14:01:11 +02:00
tersec c10f504a03
bump nim-web3 (#3024) 2021-10-22 13:55:48 +00:00
nbc-bump-bot[bot] 29b26f3f6b
nim-libp2p auto bump (#2840)
* auto-bump nim-libp2p

* Remove peer info for other peers

Not definitive, just to test the libp2p's unstable branch

* finish up Remove peer info for other peers

* getKey -> getPublicKey

* bump libp2p

* libp2p bump

Co-authored-by: = <Menduist@users.noreply.github.com>
Co-authored-by: Tanguy <tanguy@status.im>
2021-10-21 13:01:29 +02:00
Kim De Mey 724a9083ec
Bump nim-eth for allowing decoding ENRs with RLP lists (#3000) 2021-10-19 18:52:49 +02:00
tersec b66e7c8573
v1.1.3 tests; re-enable minimal-preset altair transition tests (#2993) 2021-10-16 07:17:07 +00:00
Zahary Karadjov 9381ad352f
The deposit contract uses dynamic bytes for most of its event properties
https://github.com/ethereum/consensus-specs/blob/dev/solidity_deposit_contract/deposit_contract.sol#L19-L24
2021-10-14 15:33:55 +03:00
Zahary Karadjov dae7b6aeea
Bump nim-web3 2021-10-14 14:38:07 +03:00
Etan Kissling 25a22fae5f
bump `nim-stint` (#2969)
Upstream changes:
- Parsing should raise CatchableError for user inputs
- RangeError are actually defect and not catchable.
2021-10-14 14:13:51 +03:00
Zahary Karadjov fe8bbb2388
Merge stable into unstable 2021-10-14 13:32:42 +03:00
Zahary Karadjov dbbdbea7da Hotfix: Revert a nim-web3 change that interferes with Eth1 deposit syncing 2021-10-10 13:35:06 +02:00
tersec 3c14e5728d
use v1.1.2 test vectors (#2968) 2021-10-08 11:36:35 +00:00
tersec 5cc2acb44e
use v1.1.1 test vectors (#2954) 2021-10-05 02:26:56 +00:00
Kim De Mey 0b1e0772de
Bump nim-eth to latest on master (#2927) 2021-09-30 09:00:47 +02:00
Zahary Karadjov 66b9f33edd Definitions related to the new execution engine JSON-RPC API 2021-09-30 00:20:21 +03:00
kdeme 933879e298 Bump nim-eth to allow for tcp/udp ports to always be configured in ENR 2021-09-29 20:15:24 +03:00
Mamy Ratsimbazafy 4aed54200d
fix multithreading memory leak (#2920)
* fix multithreading memory leak

* remove destructors for Nim 1.2 taskpool
2021-09-29 18:09:53 +02:00
Zahary Karadjov cc5d85331b Altair config for mainnet
This still doesn't work properly because it leads to a change in
the "eth" ENR field of the client. Since the client uses the value
of this field to look for other nodes on the network, the change
effectively prevents us from finding peers.
2021-09-29 14:19:20 +03:00
tersec f2df006ee9
update to v1.1.0 (#2909) 2021-09-28 04:28:33 +00:00
tersec edad6e5e83
use v1.1.0-beta.5 test vectors (#2900) 2021-09-25 03:07:07 +00:00
Etan Kissling aaaae89445
bump nimbus-build-system (#2898)
Changes:
- QUICK_AND_DIRTY_NIMBLE
- fix build output color on macOS
2021-09-24 15:34:23 +02:00
Eugene Kabanov b566d4657f
REST /eth/v1/events API call implementation. (#2878)
* Placing callbacks into strategic places.

* Initial events call implementation.

* Post rebase fixes.

* Change addSyncContribution() implementation.

* Add `attestation-sent` event.
Remove gcsafe, raises from callbacks implementations.
Move `attestation-received` fire at the end of attestation processing.

* Address review comments.
2021-09-22 14:17:15 +02:00
Jacek Sieka 855d0257ff
update lighthouse bootnodes (#2880)
https://github.com/eth2-clients/eth2-networks/pull/60
2021-09-21 13:34:25 +02:00
Mamy Ratsimbazafy 0dad1ec091
bump taskpools: remove default CPU pinning 2eb22c61b3 2021-09-21 11:09:48 +02:00
Mamy Ratsimbazafy 77f4169ffe
bump BLST v0.3.5 (#2876) 2021-09-17 10:20:21 +02:00
Mamy Ratsimbazafy d1cb5b7220
Parallel attestation verification (#2718)
* Add parallel attestation verification

* Update tests, batchVerify doesn't use the threadpool with only single core (nim-blscurve update)

* bump nim-blscurve

* Debug info for failing eth2 test vectors

* remove submodule eth2-testnets

* verbose debugging of make failure on Windows (libbacktrace?)

* Remove CI debug mode

* initialization convention

* Fix new altair tests
2021-09-17 03:13:52 +03:00
Eugene Kabanov 7ab1856c04
SSZ encoded responses for REST API calls. (#2851) 2021-09-16 16:32:32 +03:00
Zahary Karadjov a71de3feda Address review comments 2021-09-14 12:12:49 +03:00
tersec 5ea9e0baf3
use v1.1.0-beta.4 test vectors (#2861) 2021-09-10 17:35:14 +00:00
Kim De Mey ec736bc578
Bump nim-eth vendor module (#2848) 2021-09-06 21:59:20 +02:00
Jacek Sieka 6d47d96c84
altair upgrade for prater (#2828)
and a few import fixes for free
2021-08-27 16:54:51 +00:00
Jacek Sieka 01596c45dd
cleanups and fixes (#2827)
* import cleanup
* fix json-rpc exception handlers
* avoid unnecessary presto client import
* introduce ForkedBeaconBlock, some altair logging
* url fixes
2021-08-27 11:00:06 +02:00
Ștefan Talpalaru 5f148d9a26
bump vendor/news - this time the "status" branch (#2824) 2021-08-27 08:32:51 +02:00
tersec 4492775c6b
add random tests and rename "Official" to "Ethereum Foundation" (#2826)
* add random tests and rename "Official" to "Ethereum Foundation"

* checkDir = true covers dirExists(...)

* invalidate CI EF fixtures cache

* more correct cache invalidation
2021-08-27 06:30:30 +00:00
tersec 8de2342f60
use v1.1.0-beta.3 test vectors (#2825) 2021-08-26 16:01:21 +00:00
Ștefan Talpalaru b113d69f48
bump vendor/news (#2816) 2021-08-25 14:03:42 +00:00
tersec 79b2539ef0
eth2.0-spec-tests -> consensus-spec-tests repo rename (#2807) 2021-08-23 07:13:30 +00:00
Jacek Sieka e6e69f5f54
module bumps (#2800) 2021-08-20 18:39:18 +00:00
Ștefan Talpalaru bc37a6eb63
Windows: fix build when '.' not in PATH (#2787) 2021-08-20 14:00:17 +02:00
Jacek Sieka a7a65bce42
disentangle eth2 types from the ssz library (#2785)
* reorganize ssz dependencies

This PR continues the work in
https://github.com/status-im/nimbus-eth2/pull/2646,
https://github.com/status-im/nimbus-eth2/pull/2779 as well as past
issues with serialization and type, to disentangle SSZ from eth2 and at
the same time simplify imports and exports with a structured approach.

The principal idea here is that when a library wants to introduce SSZ
support, they do so via 3 files:

* `ssz_codecs` which imports and reexports `codecs` - this covers the
basic byte conversions and ensures no overloads get lost
* `xxx_merkleization` imports and exports `merkleization` to specialize
and get access to `hash_tree_root` and friends
* `xxx_ssz_serialization` imports and exports `ssz_serialization` to
specialize ssz for a specific library

Those that need to interact with SSZ always import the `xxx_` versions
of the modules and never `ssz` itself so as to keep imports simple and
safe.

This is similar to how the REST / JSON-RPC serializers are structured in
that someone wanting to serialize spec types to REST-JSON will import
`eth2_rest_serialization` and nothing else.

* split up ssz into a core library that is independendent of eth2 types
* rename `bytes_reader` to `codec` to highlight that it contains coding
and decoding of bytes and native ssz types
* remove tricky List init overload that causes compile issues
* get rid of top-level ssz import
* reenable merkleization tests
* move some "standard" json serializers to spec
* remove `ValidatorIndex` serialization for now
* remove test_ssz_merkleization
* add tests for over/underlong byte sequences
* fix broken seq[byte] test - seq[byte] is not an SSZ type

There are a few things this PR doesn't solve:

* like #2646 this PR is weak on how to handle root and other
dontSerialize fields that "sometimes" should be computed - the same
problem appears in REST / JSON-RPC etc

* Fix a build problem on macOS

* Another way to fix the macOS builds

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 20:57:58 +02:00
Ștefan Talpalaru 9caf852dd4
cron CI: various fixes (#2594) 2021-08-18 14:40:03 +02:00
Jacek Sieka 63717531dc remove remaining traces of nim-prompt
upstream dead, as is feature in eth2
2021-08-16 21:56:50 +03:00
Jacek Sieka 584e70e1cb
bump faststreams (#2783) 2021-08-16 07:50:38 +02:00
Jacek Sieka 07728e4f7c
altair devnet 3 (#2782) 2021-08-15 19:50:52 +02:00
Jacek Sieka 92aba71604
Pyrmont altair fork (#2771)
Prater upgrading to altair at epoch 61650 -
https://github.com/eth2-clients/eth2-networks/pull/56

also includes altair-devnet-2 support in binary
2021-08-10 10:16:43 +02:00
Eugene Kabanov b5dd6ca11c
Bump `chronos` and `http-utils` to fix number of issues. (#2765) 2021-08-06 18:37:51 +00:00
Ștefan Talpalaru e576064788
bump nimbus-build-system (#2749) 2021-07-30 10:27:56 +02:00
tersec 941cc125a3
update to v1.1.0-beta.2 tests, with exception for altair sync committees (#2736)
* update to v1.1.0-beta.2 tests, with exception for altair sync committees

* bootstrap node can't drop any attestations
2021-07-26 17:34:13 +00:00
nbc-bump-bot[bot] f68061b2d8
nim-libp2p auto bump (#2647)
* auto-bump nim-libp2p

* update peerhook for libp2p

Co-authored-by: = <Menduist@users.noreply.github.com>
Co-authored-by: Tanguy Cizain <tanguycizain@gmail.com>
2021-07-26 12:36:17 +00:00
Kim De Mey 3e3e17fec3
Remove raises Exception and bump nim-eth (#2729) 2021-07-19 14:25:11 +02:00
Jacek Sieka 2d6a661ac6
Syncv2 (#2723)
* bump libp2p

* altair sync v2

Use V2 sync requests after the altair fork has happened, according to
the wall clock

* Fix the behavior of the v1 req/resp calls after Altair

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-07-15 21:01:07 +02:00
tersec 11541ff61b
bump nim-eth2-scenarios to use v1.1.0-beta.1 test vectors (#2724) 2021-07-15 09:04:51 +00:00
Jacek Sieka 3f9c1fdf4e
More RuntimeConfig cleanup (#2716)
* remove from BeaconChainDB (doesn't depend on runtime config)
* eth2-testnets -> eth2-networks
* use `cfg` name throughout
2021-07-13 16:27:10 +02:00
Eugene Kabanov 3b6f4fab4a
New validator client using REST API. (#2651)
* Initial commit.

* Exporting getConfig().

* Add beacon node checking procedures.

* Post rebase fixes.

* Use runSlotLoop() from nimbus_beacon_node.
Fallback implementation.
Fixes for ETH2 REST serialization.

* Add beacon_clock.durationToNextSlot().
Move type declarations from beacon_rest_api to json_rest_serialization.
Fix seq[ValidatorIndex] serialization.
Refactor ValidatorPool and add some utility procedures.
Create separate version of validator_client.

* Post-rebase fixes.
Remove CookedPubKey from validator_pool.nim.

* Now we should be able to produce attestations and aggregate and proofs.
But its not working yet.

* Debugging attestation sending.

* Add durationToNextAttestation.
Optimize some debug logs.
Fix aggregation_bits encoding.
Bump chronos/presto.

* Its alive.

* Fixes for launch_local_testnet script.
Bump chronos.

* Switch client API to not use `/api` prefix.

* Post-rebase adjustments.

* Fix endpoint for publishBlock().

* Add CONFIG_NAME.
Add more checks to ensure that beacon_node is compatible.

* Add beacon committee subscription support to validator_client.

* Fix stacktrace should be an array of strings.
Fix committee subscriptions should not be `data` keyed.

* Log duration to next block proposal.

* Fix beacon_node_status import.

* Use jsonMsgResponse() instead of jsonError().

* Fix graffityBytes usage.
Remove unnecessary `await`.
Adjust creation of SignedBlock instance.
Remove legacy files.

* Rework durationToNextSlot() and durationToNextEpoch() to use `fromNow`.

* Fix race condition for block proposal and attestations for same slot.
Fix local_testnet script to properly kill tasks on Windows.
Bump chronos and nim-http-tools, to allow connections to infura.io (basic auth).

* Catch services errors.
Improve performance of local_testnet.sh script on Windows.
Fix race condition when attestation producing.

* Post-rebase fixes.

* Bump chronos and presto.

* Calculate block publishing delay.
Fix pkill in one more place.

* Add error handling and timeouts to firstSuccess() template.
Add onceToAll() template.
Add checkNodes() procedure.
Refactor firstSuccess() template.
Add error checking to api.nim calls.

* Deprecated usage onceToAll() for better stability.
Address comment and send attestations asap.

* Avoid unnecessary loop when calculating minimal duration.
2021-07-13 13:15:07 +02:00
tersec 667a01ef42
bump nim-eth2-scenarios to stop downloading alpha.5 test vectors (#2714) 2021-07-12 21:56:48 +00:00
Jacek Sieka 23eea197f6
Implement split preset/config support (#2710)
* Implement split preset/config support

This is the initial bulk refactor to introduce runtime config values in
a number of places, somewhat replacing the existing mechanism of loading
network metadata.

It still needs more work, this is the initial refactor that introduces
runtime configuration in some of the places that need it.

The PR changes the way presets and constants work, to match the spec. In
particular, a "preset" now refers to the compile-time configuration
while a "cfg" or "RuntimeConfig" is the dynamic part.

A single binary can support either mainnet or minimal, but not both.
Support for other presets has been removed completely (can be readded,
in case there's need).

There's a number of outstanding tasks:

* `SECONDS_PER_SLOT` still needs fixing
* loading custom runtime configs needs redoing
* checking constants against YAML file

* yeerongpilly support

`build/nimbus_beacon_node --network=yeerongpilly --discv5:no --log-level=DEBUG`

* load fork epoch from config

* fix fork digest sent in status
* nicer error string for request failures
* fix tools

* one more

* fixup

* fixup

* fixup

* use "standard" network definition folder in local testnet

Files are loaded from their standard locations, including genesis etc,
to conform to the format used in the `eth2-networks` repo.

* fix launch scripts, allow unknown config values

* fix base config of rest test

* cleanups

* bundle mainnet config using common loader
* fix spec links and names
* only include supported preset in binary

* drop yeerongpilly, add altair-devnet-0, support boot_enr.yaml
2021-07-12 15:01:38 +02:00
Ștefan Talpalaru 17da9aaa09
fix metrics on Windows (#2707) 2021-07-07 10:19:41 +02:00
Mamy Ratsimbazafy 38a5bc8de3
bump blscurve closes #2693 and #2695 (#2700) 2021-07-06 14:31:54 +02:00
tersec df381d0cf0
update spec test fixtures to v1.1.0-alpha.8 (#2687) 2021-06-29 16:47:09 +00:00
Ștefan Talpalaru 39591ac858
bump two submodules (#2657) 2021-06-17 17:07:57 +00:00
tersec 1c3314f08b
update to Altair as of v1.1.0-alpha.7 (#2649)
* update to Altair as of v1.1.0-alpha.7

* introduce Altair types into attestation pool

* avoid allocating/copying pubkeys excessively in get_next_sync_committee()
2021-06-14 17:42:46 +00:00
Jacek Sieka d859bc12f0
write uncompressed validator keys to database (#2639)
* write uncompressed validator keys to database

Loading 150k+ validator keys on startup in compressed format takes a lot
of time - better store them in uncompressed format which makes behaviour
just after startup faster / more predictable.

* refactor cached validator key access
* fix isomorphic cast to work with non-var instances
* remove cooked pubkey cache - directly use database cache in chaindag
as well (one less cache to keep in sync)
* bump blscurve, introduce loadValid for known-to-be-valid keys
2021-06-10 10:37:02 +03:00
nbc-bump-bot[bot] 1a76007858
auto-bump nim-libp2p (#2591)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2021-06-04 12:36:03 +02:00
tersec 820a6f65d5
use v1.1.0-alpha.6 test vectors (#2618)
* bump nim-eth2-vendors and use v1.1.0-alpha6 test vectors

* continue to download alpha 5 test vectors for const sanity checks
2021-05-30 09:51:01 +00:00
Jacek Sieka d16da06c92 ncli_db: validator performance database tool
Record attestation performance per epoch in sqlite database
2021-05-27 19:14:26 +03:00
Ștefan Talpalaru 9ddf7fea23 dist: use Ubuntu-18.04 to build ARM64 binaries 2021-05-26 16:51:06 +03:00
Jacek Sieka 80ed54378c
nim-eth: fix commit 2021-05-26 09:40:08 +02:00
Jacek Sieka eebc828778
create new database in separate file (#2596)
The V1 table structure shows great improvements in performance, but if
there's an old `kvstore` without rowid:s, these benefits are nullified:
reorgs during writes and deletes remain expensive (even if the
degradation is reduced somewhat).

This PR creates the tables in a new file instead, and uses the old file
as a read-only store - this has several interesting properties:

* the old database is left completely untouched - this guarantees that
downgrades work smooth (they'll only need to resync their missing
portions)
* starting sync after this PR means only a v1 database is created
* v0 databases stick around - no migration is performed (for now)

Future PR:s can introduce migration of the data from one database to
another - a simply copy will take hours which is downtime we want to
avoid - at that point, it might make sense to migrate straight to era
files instead.
2021-05-26 09:07:18 +02:00
tersec d69e06e519
bump nim-eth2-scenarios to stop downloading v1.0.1 test vectors (#2598) 2021-05-24 19:31:48 +00:00
tersec 056ff59595
bump eth2-testnets (#2589) 2021-05-21 13:09:35 +00:00
tersec 1c05865656
bump nim-eth2-scenarios to download v1.1.0-alpha.5 test vectors (#2588) 2021-05-21 10:54:59 +00:00
Zahary Karadjov dc49a51654
Merge stable into unstable (take 2) 2021-05-20 13:52:09 +03:00
Zahary Karadjov b7aa30adfd
Merge stable into unstable 2021-05-20 13:50:40 +03:00
Eugene Kabanov cf06c4e87e
Make REST server more compatible with Lighthouse and Teku validator clients. (#2575)
* Allow REST server to parse arrays with comma delimiter.

* Fix compilation issues because of new presto framework.
2021-05-18 12:24:57 +02:00
tersec fb88309b34
bump nim-eth2-scenarios to download v1.1.0-alpha.4 test vectors (#2574) 2021-05-18 11:59:46 +02:00
Zahary Karadjov 6b14d33ad6
Bump Confutils
This fixes an issue with the --bootstrap-file parameter which was
accidentally considered mandatory due to an upstream issue.
2021-05-18 11:02:01 +03:00
Ștefan Talpalaru 1f1367d09a
macOS binary distribution (both AMD64 and ARM64) 2021-05-17 21:42:28 +03:00
Zahary Karadjov f6a1f602b4
Handle changes in the latest version of Confutils 2021-05-17 21:42:28 +03:00
Zahary Karadjov b9924214ab
Better error-handling for the slashingdb import/export feature
* Error when specifying an invalid --data-dir (or --validator-dir)
* Error when entering an invalid validator public key (e.g. invalid hex value)
* Warning when attempting to export a validator not present in the local database

Some unnecessary remains of the v1 mode has been removed as well
2021-05-17 21:42:23 +03:00
Jacek Sieka 97f4e1fffe
Db1 cont (#2573)
* Revert "Revert "Upgrade database schema" (#2570)"

This reverts commit 6057c2ffb4.

* ssz: fix loading empty lists into existing instances

Not a problem earlier because we didn't reuse instances

* bump nim-eth

* bump nim-web3
2021-05-17 18:37:26 +02:00
Zahary Karadjov 5c313b958e
Simplify the slashing db import/export CLI 2021-05-17 17:12:03 +03:00
tersec 6057c2ffb4
Revert "Upgrade database schema" (#2570)
This reverts commit 22ddf74752.
2021-05-17 06:34:44 +00:00
Jacek Sieka 2c257438a0 Upgrade database schema
This is a minimal performance hotfix for the kvstore table, such that
new databases get a kvstore table with rowid, fixing the most urgent
performance problem we have with pruning.
2021-05-14 20:08:07 +03:00
Jacek Sieka 22ddf74752 Upgrade database schema
The `kvstore` design we're using now turns out to not be the best way to
use `sqlite` - in particular, there are some significant benefits to
using rowid in certain situations and to keep data in separate tables.

With this branch, there are massive improvements in startup time
(seconds instead of minutes) and state/block storage and pruning times
(milliseconds instead of seconds) - these improvements can in particular
be seen on slow drives and translate directly into better attestation
performance.

* update kvstore to new keyspace design
* remove `DirStoreRef` and the hidden `--state-db-kind` option - this
was an experiment to store large blobs in files, but with the new
kvstore, there's no compelling reason to do so
* remove `DbMap` - unused and would need updating for new keyspace
design
* introduce separate tables for each data type (blocks, states etc)
* remove "WITHOUT ROWID" pessimization for tables with large blobs
* close DbSeq statements explicitly (and earlier)
* store beacon block summaries in separate table, without SSZ
compression and load them all with single query on startup
* stop storing backwards compat full states
* mark genesis beacon block as trusted
* avoid faststreams when loading SSZ data
* remove `DisagreementBehavior` (unused)
2021-05-14 20:05:23 +03:00
tersec f9b964ca5d
bump nim-eth2-scenarios for merge test vectors (#2553) 2021-05-10 13:56:31 +00:00
Jacek Sieka 5cd5da74c4
Gossipsub unsubscribe fixes (fixes #2540) (#2545) 2021-05-07 06:31:28 +02:00
Jacek Sieka 01b404f43d
bump chronos (#2544) 2021-05-06 18:20:37 +02:00
tersec dd43a2c3b0
bump nim-eth2-scenarios to get merge SSZ test vectors (#2541) 2021-05-05 15:35:36 +00:00
Ștefan Talpalaru 2018a55c51
bump a few submodules (#2529) 2021-05-05 08:55:39 +02:00
Jacek Sieka ce49da6c0a
Introduce unittest2 and junit reports (#2522)
* Introduce unittest2 and junit reports

* fix XML path

* don't combine multiple CI runs

* fixup

* public combined report also

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2021-04-28 18:41:02 +02:00
nbc-bump-bot[bot] d4cda02c97
auto-bump nim-libp2p (#2523)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-04-28 09:57:25 +02:00
cheatfate 18da0f8928 Bump nim-chronos to fix issues in httpserver.
Bump nim-http-utils to fix issues in httpserver.
2021-04-28 00:00:00 +03:00
nbc-bump-bot[bot] 8a3c070337
auto-bump nim-libp2p (#2517)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-04-23 12:54:06 +02:00
Ștefan Talpalaru 5b32f7ca17 Nim-1.2.12 2021-04-19 10:53:58 +03:00
Jacek Sieka 4959da78a8
mem fixes / libp2p (#2510)
* mem fixes / libp2p

* bump
2021-04-18 12:30:25 +02:00
Ștefan Talpalaru 9e439f19c8
bump nim-metrics (#2511) 2021-04-18 10:22:59 +02:00
cheatfate 14742890d6 Fix browsers unable to access HTTP endpoints (HTTP 400 error). 2021-04-14 22:05:00 +03:00
Jacek Sieka 4ed2e34a9e Revamp attestation pool
This is a revamp of the attestation pool that cleans up several aspects
of attestation processing as the network grows larger and block space
becomes more precious.

The aim is to better exploit the divide between attestation subnets and
aggregations by keeping the two kinds separate until it's time to either
produce a block or aggregate. This means we're no longer eagerly
combining single-vote attestations, but rather wait until the last
moment, and then try to add singles to all aggregates, including those
coming from the network.

Importantly, the branch improves on poor aggregate quality and poor
attestation packing in cases where block space is running out.

A basic greed scoring mechanism is used to select attestations for
blocks - attestations are added based on how much many new votes they
bring to the table.

* Collect single-vote attestations separately and store these until it's
time to make aggregates
* Create aggregates based on single-vote attestations
* Select _best_ aggregate rather than _first_ aggregate when on
aggregation duty
* Top up all aggregates with singles when it's time make the attestation
cut, thus improving the chances of grabbing the best aggregates out
there
* Improve aggregation test coverage
* Improve bitseq operations
* Simplify aggregate signature creation
* Make attestation cache temporary instead of storing it in attestation
pool - most of the time, blocks are not being produced, no need to keep
the data around
* Remove redundant aggregate storage that was used only for RPC
* Use tables to avoid some linear seeks when looking up attestation data
* Fix long cleanup on large slot jumps
* Avoid some pointers
* Speed up iterating all attestations for a slot (fixes #2490)
2021-04-13 20:24:02 +03:00
cheatfate 79d4735355 Address #2491. 2021-04-13 17:07:41 +03:00
Dmitriy Ryajov 03f478748f bump libp2p to lastest master 2021-04-12 20:28:39 +03:00
Dmitriy Ryajov 5eebf4b5f5 bump libp2p 2021-04-12 20:28:39 +03:00
Dmitriy Ryajov 4f4901b10a use master builders 2021-04-12 20:28:39 +03:00
Dmitriy Ryajov 920de91c37 use libp2p builder 2021-04-12 20:28:39 +03:00
cheatfate cc3851ec02 Bump nim-presto. 2021-04-09 21:42:13 +03:00
cheatfate 72695dd62a Rest API initial implementation. 2021-04-09 21:42:13 +03:00
cheatfate 3d701d8973 Add nim-presto submodule. 2021-04-09 21:42:13 +03:00
cheatfate c4d891f583 Fix sync_manager.nim to return proper status.
Bump REST API dependencies.
2021-04-09 21:42:13 +03:00
Zahary Karadjov 9776fbfe17
Merge branch 'version-1.1.0' into unstable 2021-04-08 20:50:06 +03:00
Zahary Karadjov ba59dd85cd Support for fallback web3 providers; Fix resource leaks on Eth1 monitor restarts 2021-04-08 12:46:42 +03:00
tersec 5f5e3e43c3
bump num-eth2-scenarios from v1.1.0-alpha.2 to v1.1.0-alpha3 (#2481) 2021-04-07 13:52:17 +00:00
Jacek Sieka adec9d878e
bump chronos (#2476) 2021-04-06 13:58:36 +02:00
tersec 49a5667288
update some v1.1.0 alpha1 to alpha2 (#2457)
* update some v1.1.0 alpha1 to alpha2

* remove unused getDepositMessage overload and move other out of datatypes/base

* bump nim-eth2-scenarios to download v1.1.0-alpha.2 test vectors

* construct object rather than result
2021-03-29 19:17:48 +00:00
Jacek Sieka 74732a23fe
json cleanups (#2456)
* move json-rpc specific marshalling to rpc
* serialize Epoch/Slot with cast to avoid Defect
* avoid a few eth1 deps
* simplify imports
2021-03-26 15:11:06 +01:00
Jacek Sieka 2695cfa864
EH cleanup (#2455)
almost 100% raises in nimbus-eth2 now!

* fix some rare exception-related crashes in json-rpc
2021-03-26 07:52:01 +01:00
Zahary Karadjov 2eacfc4685 Bump modules to take advantage of the new Json format flavors support
Since quite a lot of additional procs were now compiled as generics, this lead to compiler bugs that had
to be worked-around:

* The `Domain` type was renamed to `Eth2Domain` to avoid compilation errors
  due to conflicts with `nativesockets.Domain`.
  Similarly, `eth2_network.KeyPair` was renamed to `NetKeyPair`.

* A new more robust version of `hexToByteArray` was added to stew
2021-03-25 09:37:35 +02:00
Jacek Sieka 8b76ceed52
Fix minor exception effect issues (#2448)
Makes code compatible with
https://github.com/status-im/nim-chronos/pull/166 without requiring it.
2021-03-24 17:20:55 +01:00
Zahary Karadjov a727fc8e52
Merge stable into unstable 2021-03-24 03:24:20 +02:00
Zahary Karadjov 8391d3f83e
v1.0.12 2021-03-24 03:23:14 +02:00
Zahary Karadjov 5fd82ddaf5
Merge stable into unstable 2021-03-22 23:13:05 +02:00
Jacek Sieka d1357801e7
bumps 2021-03-22 20:53:27 +01:00
Zahary Karadjov af0d302398
Use the latest bootstrap nodes for Prater 2021-03-22 18:37:32 +02:00
Jacek Sieka 3743df6434
introduce unittest2 (#2437)
This commit introduces `unittest2`, driven by the need to fix several
long-standing issues with `unittest`, in particular exception handling,
test isolation, timing and others.

Upgrading to `unittest2` generally requires updating the unittest import
and potentially fixing the code to support the new `one-proc-per-test`
setup.

* nim-unittest2 has been moved from Stefans repository to status-im -
https://github.com/status-im/nim-unittest2/pull/2 fixes some exception
handling issues and brings it in line with latest nim devel - a notable
difference to unittest is that each test is run inside a proc which
gives it a fresh stack and removes a lot of globals, meaning tests
should run under conditions more similar to "normal" code and interfere
with each other less - this is a crucial first step to improving the
testing experience in general, that stefan implemented for the
multithreading support
* the multithreading in unittest2 is optional, and disabled by default
* nim-testutils contains utilities for fuzzing, as well as other
"large:ish" features that have dependencies -
https://github.com/status-im/nim-testutils/pull/27 pulls in unittest2 so
unittest2 can be kept "low-dependency"
* nim-chronos needs the above unittest changes to test its own new,
stricter exception handling -
https://github.com/status-im/nim-chronos/pull/166 - the strictess
introduced is expected to increase with time
2021-03-22 08:21:22 +01:00
Jacek Sieka 01fe564e46
chronos-based metrics (#2432)
This opens up the road for removing `-d:insecure` for metrics.
2021-03-19 10:36:38 +01:00
Zahary Karadjov c519c6012e
Merge branch 'stable' into unstable 2021-03-15 21:52:12 +02:00
Zahary Karadjov f3fc551ea7
Support for the prater testnet 2021-03-15 21:51:24 +02:00
nbc-bump-bot[bot] 2201480bd9
auto-bump nim-libp2p (#2409)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-03-15 17:39:10 +00:00
Jacek Sieka eb830c7968
Merge remote-tracking branch 'origin/stable' into unstable 2021-03-15 17:29:43 +01:00
Jacek Sieka f5180e0840
Revert "Nim-1.2.10" (#2411)
This reverts commit 12e15a0b8d.
2021-03-15 16:26:41 +01:00
Ștefan Talpalaru c5035c6eca dist: enable libbacktrace on ARM and ARM64
(and make sure it will work on Windows, when enabled)
2021-03-15 14:38:39 +02:00
Ștefan Talpalaru 12e15a0b8d Nim-1.2.10 2021-03-15 14:29:48 +02:00
nbc-bump-bot[bot] 4fcff4cd27
auto-bump nim-libp2p (#2406)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-03-15 07:36:36 +00:00
Zahary Karadjov 77ee210762
v1.0.10 2021-03-10 23:33:37 +02:00
zah 33c7f264bc
Bump news to fix resource leaks (#2388) 2021-03-10 12:22:41 +00:00
nbc-bump-bot[bot] a88d17dc04
auto-bump nim-libp2p (#2389)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2021-03-09 14:28:28 +01:00
Mamy André-Ratsimbazafy d425aabce2 BLST v0.3.3 2021-03-08 14:43:48 +02:00
kdeme 5f750f84b4 Use setupAddress for better IP and ports configuration 2021-03-06 00:09:09 +02:00
kdeme 769225b591 Bump nimbus-build-system to use nim-1-2-6-RC1-hotfixed
This should practically solve the segfaults we have been seeing in
issue #2134
2021-03-04 23:00:44 +02:00
nbc-bump-bot[bot] 9408a9af1c
auto-bump nim-libp2p (#2373)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2021-03-04 10:14:00 +01:00
nbc-bump-bot[bot] 965972dd0b
auto-bump nim-libp2p (#2360)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-03-01 10:26:16 +00:00
nbc-bump-bot[bot] 037a3e8946
auto-bump nim-libp2p (#2359)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-02-26 09:43:35 +00:00
Dustin Brody f14e7babb6 update eth2 specs to version v1.0.1 2021-02-25 14:21:59 +02:00
Ștefan Talpalaru 16abf2989b bump NimYAML 2021-02-25 14:20:26 +02:00
Ștefan Talpalaru 22620afa8f bump NimYAML for Nim-1.4 compatibility 2021-02-25 14:20:26 +02:00
nbc-bump-bot[bot] 45087f36bd
auto-bump nim-libp2p (#2350)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-02-25 11:25:00 +00:00
Eugene Kabanov 8478345619
Bump nim-bearssl to include latest fixes in submodule repository and cacert.pem. (#2354) 2021-02-24 19:22:40 +01:00
Zahary Karadjov 3f6834cce7
Fix a crash occurring with --slashing-db-kind=both 2021-02-22 21:05:08 +02:00
Zahary Karadjov 7e9648e52c
Use the latest Teku bootstrap nodes 2021-02-22 17:16:53 +02:00
Zahary Karadjov 5f924c0038 Bump modules; Compile with DEBUG logs by default 2021-02-19 08:44:13 +02:00
Kim De Mey aa29be288e
Bump eth2-testnets for updated LH bootstrap nodes (#2322) 2021-02-15 17:56:34 +01:00
Jacek Sieka 946608b572
bump json-rpc (#2321) 2021-02-15 13:47:57 +01:00
nbc-bump-bot[bot] ad66ba507e
auto-bump nim-libp2p (#2317)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-02-13 06:16:39 +00:00
nbc-bump-bot[bot] 8192015b86
auto-bump nim-libp2p (#2315)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-02-12 05:53:38 +00:00
nbc-bump-bot[bot] 0419a4122e
auto-bump nim-libp2p (#2309)
Co-authored-by: = <dryajov@users.noreply.github.com>
2021-02-11 02:20:39 +00:00
Jacek Sieka f012d7060b
better error message on disk / database issues (#2307)
bumps stew for better result defects as well
2021-02-10 13:21:06 +01:00
nbc-bump-bot[bot] c13205035e
auto-bump nim-libp2p (#2305)
Co-authored-by: = <dryajov@users.noreply.github.com>
2021-02-10 05:32:19 +00:00
Mamy Ratsimbazafy 03f47c8f2f
Slashing protection refactor - EIP 3076 (#2094)
* Create CLI tool for slashing export

* Use SQLite as a DB instead of a KV-store

* Keeps v1 and v2 DBs around

* Uses the same schema as Lighthouse v1.1.0

* Passes all interchange tests + skeleton of finalization pruning

* Removes tests that would violate v5 / minimal slashing DB and MinSlot rules

* Migration tool added using low-watermark scheme for faster migration of large number of validators
2021-02-09 17:23:06 +02:00
nbc-bump-bot[bot] 93cfc2d446
auto-bump nim-libp2p (#2300)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-02-08 08:32:47 +01:00
Jacek Sieka ece50c4706
bump nim-json-rpc (fixes https://github.com/status-im/nim-json-rpc/issues/93) (#2299) 2021-02-07 22:30:45 +01:00
nbc-bump-bot[bot] 0b560cc6fb
auto-bump nim-libp2p (#2296)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-02-06 09:50:08 +01:00
Zahary Karadjov 1394ef37fb
Tentative fix for the freezing Eth1 monitor activity 2021-02-04 17:01:47 +02:00
Ștefan Talpalaru 80c11546ff Windows binary release
CI: use both cores on GitHub Actions and set timeouts for the local testnet tests
2021-02-04 10:25:44 +02:00
Eugene Kabanov c4c04197c5
Bump nim-http-utils. (#2290) 2021-02-03 19:05:30 +01:00
Ștefan Talpalaru cadb1aae15
bump submodules (#2279)
* bump submodules

* undo a couple of bumps (32-bit failure)

* launch_local_testnet.sh: timeout and various fixes

* undo nim-http-utils bump (breaks RPC for VC)
2021-02-02 19:02:52 +01:00
Kim De Mey 73d9c2aa53
Add enr auto update cli option and bump nim-eth (#2278) 2021-02-02 09:07:21 +01:00
Zahary Karadjov fa99c3b417 Fix #2261
Also bumps Confutils to allow setting the hidden --web3-mode param
(to allow testing the eth1 syncing without validators)
2021-01-30 01:32:20 +02:00
Kim De Mey a3678d6bf0
Bump eth2-testnets for mainnet nimbus bootstrap nodes (#2276) 2021-01-29 14:07:38 +01:00
nbc-bump-bot[bot] 3e89aa0f4f
auto-bump nim-libp2p (#2275)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-01-29 09:48:27 +01:00
Dmitriy Ryajov 1117993a24
Libp2p connection limits (#2272)
* max peers gets set as connection limits

* pass max connections
2021-01-28 07:31:29 +01:00
tersec 4de9875cdf
Revert "pass max peers to libp2p (#2265)" (#2270)
This reverts commit 64e4d74f87.
2021-01-26 17:53:52 +01:00
Dmitriy Ryajov 64e4d74f87
pass max peers to libp2p (#2265) 2021-01-26 10:35:22 +01:00
Mamy Ratsimbazafy 70a03658e3
Block validation flow v2 + Batch (serial) sig verification (#2250)
* bump nim-blscurve

* Outline the block validation flow

* introduce the SigVerified types, pass the tests

* Split clearance/quarantine to prepare for batch crypto verif

* Add a batch signature collector

* Make clearance use SigVerified block and split verification between crypto and state transition

* Always use signedBeaconBlock for the onBlockAdded callback

* RANDAO signing_root is the epoch instead of the full block

* Support skipping BLS for testing

* Fix compilation of the validator client

* Try to fix strange errors MacOS and Jenkins (Clang, unknown type name br_hmac_drbg_context in stdlib_assertions.nim.c)

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561819858

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828025

* onBlockAdded callback should use TrustedSignedBeaconBlock https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561837261

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828946

* Use the application RNG: https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561815336

* Improve codegen of conversion zero-cost)

* Quick fixes with loadWithCache after #2259 (TODO: graceful error since pubkey validations is now done first in signatures_batch)

* Graceful handle rogue pubkeys and signatures now that those are lazy-loaded
2021-01-25 20:45:48 +02:00
nbc-bump-bot[bot] f0f292f509
auto-bump nim-libp2p (#2245)
Co-authored-by: = <dryajov@users.noreply.github.com>
2021-01-22 08:43:22 +01:00
Ștefan Talpalaru 9a0cac61c8 only ignore untracked files in submodules 2021-01-21 16:09:28 +02:00
Jacek Sieka 76f15302a7 better timesharing in eth2 processor
* use `idleAsync` to more evenly divide cpu attention when syncing in
particular - this gives networking better latency
* more strict exception handling in eth2_processor
2021-01-18 22:27:34 +02:00
nbc-bump-bot[bot] a0689e4237
auto-bump nim-libp2p (#2239)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-01-15 21:15:58 +09:00
nbc-bump-bot[bot] bb0a022968
auto-bump nim-libp2p (#2238)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-01-15 15:50:14 +09:00
Giovanni Petrantoni 295e3c9c73
Topics validation and direct peers (#2237)
* pick the right libp2p branch

* add topics validation
2021-01-15 04:17:06 +00:00
nbc-bump-bot[bot] ce64da1fcd
auto-bump nim-libp2p (#2235)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2021-01-15 01:39:09 +00:00
Kim De Mey 66d8f317cd
Use async queryRandom instead of inefficient randomNodes to discover nodes (#2211) 2021-01-14 08:58:13 +01:00
nbc-bump-bot[bot] f5efcc8669
auto-bump nim-libp2p (#2228)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-01-12 04:27:32 +01:00
nbc-bump-bot[bot] be3fac8495
auto-bump nim-libp2p (#2206)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2021-01-11 09:53:20 +01:00
Jacek Sieka 93b434d0ac
bumps (#2214)
* bumps

several modules pinned to merged branches

* faststreams master broken, go back
2021-01-07 13:12:37 +01:00
Ștefan Talpalaru e58a355a1c
CI release: ARM and ARM64 builds (#2213) 2021-01-07 10:19:29 +01:00
Jacek Sieka 52ca12c79f
bump nat, avoid one more finalizer (#2209) 2020-12-31 15:58:31 +01:00
Giovanni Petrantoni ed24f60f70
remove async from sub/unsub (#2197)
* remove await/async from sub/unsub

* fix unsubscribe wrong key (missed _snappy)

* use the right libp2p commit hash

* remove unused async

* fix inspector

* fix subnet calculation in RPC and insert broadcast attestations into node's pool

* unify codepaths to ensure only mostly-checked-to-be-valid attestations enter the pool, even from node's own broadcasts

* update attestation pool tests for new validateAttestation param

Co-authored-by: Dustin Brody <tersec@users.noreply.github.com>
2020-12-24 09:48:52 +01:00
Jacek Sieka 0f8a3a5ae8
checkpoint database at end of each slot (#2195)
* checkpoint database at end of each slot

To avoid spending time on synchronizing with the file system while doing
processing, the manual checkpointing mode turns off fsync during
processing and instead checkpoints the database when the slot has ended.

From an sqlite perspecitve, in WAL mode this guaranees database
consistency but may lead to data loss which is fine - anything missing
from the beacon chain database can be recovered on the next startup.

* log sync status and delay in slot start message

* bump
2020-12-18 22:01:24 +01:00
Jacek Sieka 452042b17b
msgid formatting, libp2p bump (#2198)
* msgid formatting, libp2p bump

* fix libp2p commit
2020-12-18 18:45:27 +00:00
Jacek Sieka ca2a09d424 bump secp 2020-12-16 19:41:12 +02:00
Ștefan Talpalaru ff23ca27c2
bump vendor/nimbus-benchmarking 2020-12-16 14:11:11 +01:00
Jacek Sieka b0086c5d99
bump libp2p (#2187)
* bump libp2p

* also stew
2020-12-16 09:36:51 +01:00
Jacek Sieka 4e191a06ac use stew/leb128
* 3 varint parsers -> 1
* 1/3 memory usage / allocations when writing stuff to libp2p
2020-12-15 17:06:15 +02:00
Ștefan Talpalaru e52c88386b
build warnings (#2179) 2020-12-14 16:45:31 +00:00
Kim De Mey f49fcb9df3
Bump nim-eth for discovery v5 ip limits (#2085) 2020-12-11 14:35:08 +01:00
Ștefan Talpalaru e59437c5e2
new Make target: "benchmarks" (#2174)
It runs some benchmarks, collects the output and generates HTML charts.
2020-12-11 12:12:43 +01:00
Mamy Ratsimbazafy 2b3b219c77
Bump blst: perf + 32-bit platforms support + pure C fallback + drop Miracl fallback tests (#2152)
* bump BLST

* Cached subgroup checks

* Bump BLST - BLST now has pure C fallback for all arch (Miracl unneeded)

* Drop testing Miracl fallback in Nimbus

* Charting uncharted waters: 32-bit should imply -mno-adx
2020-12-10 10:18:07 +01:00
Jacek Sieka c4d9d5f64e
bump nim-stew (no change) 2020-12-09 17:24:25 +01:00
Jacek Sieka 0b33183341
bumps (#2154)
* avoid dangling cstring returns in Result, fixing some low-risk crash
potential
* bump libp2p for connection race fix and a few cleanups
2020-12-07 17:09:05 +01:00
Zahary Karadjov 4feb0a308e Fix #2125 (ETH status bar display); Bump LibP2P 2020-12-02 00:03:59 +02:00
Zahary Karadjov 4eaf29875a Less risk for Eth1-induced delays in block proposal 2020-12-01 15:38:50 +02:00
Zahary Karadjov 38f7558e50 Work around a strange codegen issue to fix local sim in CI; Bump LibP2P 2020-12-01 15:38:00 +02:00
Jacek Sieka 7e4fd325b5
bump nim-libp2p 2020-11-29 14:47:19 +01:00
= e1fb2d98fb auto-bump nim-libp2p 2020-11-28 09:20:02 +00:00
Jacek Sieka d16e127daf
Peer stuff (#2084)
* Revert "Revert "Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065)" (#2082)"

This reverts commit 7cc3dc8027.

* fix nil disconnectedFut crash

* fixes

don't resetPeer, it causes peer miscounts

* disconnect disconnecting peers

...when there's a race.

* avoid connection spamming

* never decrease SeenTable timeout
* only recover ENR for known peers

* seen only when really disconnected
2020-11-26 20:23:45 +01:00
Giovanni Petrantoni 9b9aa93cd4
Seenttl by spec (#2083)
* add rpc call to get chronos futures at runtime

* add seenTTL gossip parameter

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-11-26 13:08:22 +01:00
Jacek Sieka 7cc3dc8027
Revert "Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065)" (#2082)
This reverts commit d041287a4f.
2020-11-26 09:05:23 +01:00
Eugene Kabanov d041287a4f
Full "node" RPC calls implementation and fixes to peer lifetime states. (#2065)
* Initial commit.

* Fix log lines and compilation error.

* Add get_v1_node_peers() implementation.

* Fix peer's lifetime states.

* Use the most recent multiaddress.

* Fix assign NewPeerScore again.
Fix compilation error with last seen address.
Fix Peer upgraded log line place.

* syncing, health, peer_count, peer_id and fixes for identity.

* Fix compilation problems.

* Move object declaration to callsigs.
Fix identity addresses fields.

* Finish node RPC calls.

* Avoid leak of lifetime future.

* Bump chronos.

* Fix json generator problem.
2020-11-26 08:09:59 +01:00
Zahary Karadjov 7fdafa65ee
New bootstrap nodes for Mainnet 2020-11-25 21:27:40 +02:00
zah 372c9b798c
Fix the corrupted database state on Pyrmont nodes; Add mainnet genesis (#2056)
* Handle some web3 timeouts better

* Add support for developer .env files

* Eth1 improvements; Mainnet genesis state

Notable changes:

* The deposits table have been removed from the database. The client
  will no longer process all deposits on start-up.

* The network metadata now includes a "state snapshot" of the deposit
  contract. This allows the client to skip syncing deposits made prior
  to the snapshot (i.e. genesis). Suitable metadata added for Pyrmont
  and Mainnet.

* The Eth1 monitor won't be started unless there are validators attached
  to the node.

* The genesis detection code is now optional and disabled by default

* Bugfix: The client should not produce blocks that will fail validation
  when it hasn't downloaded the latest deposits yet

* Bugfix: Work around the database corruption affecting Pyrmont nodes

* Remove metadata for Toledo and Medalla
2020-11-24 22:21:47 +01:00
Jacek Sieka f2722ed866
auto-bump nim-libp2p (#2064)
Co-authored-by: = <dryajov@users.noreply.github.com>
2020-11-24 12:38:18 +01:00
nbc-bump-bot[bot] 0856f0539a
auto-bump nim-libp2p (#2036)
Co-authored-by: = <dryajov@users.noreply.github.com>
2020-11-23 10:38:34 +01:00
Zahary Karadjov d232f16b40 Add more eth1-related logging 2020-11-20 17:13:51 +02:00
Zahary Karadjov e370e80aee
Add more Pyrmont bootstrap nodes 2020-11-18 14:40:45 +02:00
Eugene Kabanov 28369574b7
Bump `nim-chronos` to fix write() deadlocks. (#2037) 2020-11-18 11:39:15 +01:00
Zahary Karadjov ebfacf597c
Review and fix some usages of Future.cancel; Bump Chronos 2020-11-18 00:27:21 +02:00
Jacek Sieka 4479c0a9f1
fix some libp2p channel hangs (#2032)
* fix some libp2p channel hangs

* make sure no concurrent reads are issues by awaiting cancellations
* bumps with fixes

* merge bump
2020-11-17 16:07:05 +01:00
Jacek Sieka a6b188bfd4
misc fixes (#2027)
* log when database is loading (to avoid confusion)
* generate network keys later during startup
* fix quarantine not scheduling chain of parents for download and
increase size to one epoch
* log validator count, enr and peerid more clearly on startup
2020-11-16 20:15:43 +01:00
Jacek Sieka 5b011b65ae add rpc call to get chronos futures at runtime 2020-11-16 12:31:39 +02:00
Zahary Karadjov eb76d650f6 Add genesis state for Pyrmont 2020-11-15 21:40:40 +02:00
kdeme d02d8f8d51 Drop UseDiscv51 flag and only support discovery v5.1 2020-11-14 19:12:30 +02:00
tersec 602f344525
remove v0.12.3 EF test vectors to save 2.7GB of space in CI instances (#2017) 2020-11-13 22:16:56 +01:00
Jacek Sieka ab8f8ccaba
clean up dependencies (#2008)
* clean up dependencies

* use non-prce-mode for metrics
* clean up obsolete snappy and gflags deps from manuals

* conditional pcre
2020-11-13 16:00:45 +01:00
Zahary Karadjov 80ca7a2d9f Implement #1768 2020-11-12 22:29:43 +02:00
Zahary Karadjov 8ad45de65b
Add makefile targets for Pyrmont 2020-11-12 21:51:15 +02:00
Zahary Karadjov 08da65edea
Latest toledo metadata 2020-11-10 14:11:31 +02:00
Zahary Karadjov 52043a8fe7
Toledo genesis state 2020-11-10 00:56:46 +02:00
tersec a010ef2061
update tests from v1.0.0-rc.0 to v1.0.0 (#1976) 2020-11-09 17:44:18 +00:00
Zahary Karadjov 2343934bc8
Add backwards compatibility target for signing_process 2020-11-09 14:40:06 +02:00
Zahary Karadjov 94e9ce3c1f Add metadata for the toledo network 2020-11-09 11:38:52 +02:00
Jacek Sieka aaf20c1554
bump libp2p (#1963) 2020-11-06 13:18:16 +01:00
Zahary Karadjov 18639c3eff Don't require requests that might fail on non-archive Geth nodes 2020-11-03 23:23:10 +02:00
Jacek Sieka ee663d9fc2
move libp2p back a few commits (#1948)
to see if it helps with connectivity issues
2020-11-03 17:04:16 +01:00
Jacek Sieka 937decb0e9
Revert "Gossip11 post audit (#1917)" (#1947)
This reverts commit 63173ab2c1.

It appears the cluster is having trouble staying connected - since the culprit is unknown, this is a first step on the way to what was stable.

Notably, this does not fully revert libp2p itself, merely the gossip version.
2020-11-03 11:45:40 +01:00
nbc-bump-bot[bot] 446f8cc738
auto-bump nim-libp2p (#1934)
Co-authored-by: = <dryajov@users.noreply.github.com>
2020-11-01 23:04:05 +01:00
nbc-bump-bot[bot] 9e3ca74ee0
auto-bump nim-libp2p (#1930)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2020-10-31 15:06:18 +09:00
Giovanni Petrantoni 63173ab2c1
Gossip11 post audit (#1917)
* bump libp2p gossip

* use gossip 1.1

* use proper libp2p commit hash

* switch to master libp2p, has audit fixes now
2020-10-30 14:26:35 +00:00
nbc-bump-bot[bot] 8d998a5a8b
auto-bump nim-libp2p (#1914)
Co-authored-by: = <dryajov@users.noreply.github.com>
2020-10-30 13:35:01 +01:00
Ștefan Talpalaru 2396417581
bump nim-libbacktrace (#1799)
so stack traces work with LTO
2020-10-27 13:09:03 +01:00
Eugene Kabanov eee01a32d5
Regression fix of eth2_network_simulation on Windows. (#1900)
* Concentrate all sensitive writeFile/createPath calls in one place.
Fix eth2_network_simulation for Windows.

* Remove artifacts.

* fix import

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-10-27 12:04:17 +01:00
Mamy Ratsimbazafy 677f099110
BLS signature v4 (#1912) 2020-10-27 09:37:04 +01:00
Kim De Mey 8102d5dc4d
Bump eth2-testnets for more discv5.1 bootstrap nodes on medalla (#1911)
And improve logging UX a bit by ignoring comments or newlines
2020-10-27 08:44:46 +01:00
Kim De Mey 756b62fdf5
Bump nim-eth to get UseDiscv51 flag (#1895)
* Bump nim-eth to get UseDiscv51 flag

* Switch medalla to discovery v5.1, other targets to v5.0

* Bump nim-eth for better discv5.1 logging

* Bump eth2-testnets for updated medalla bootnodes
2020-10-23 18:59:58 +02:00
tersec decfd66a78
bump libp2p: seq[byte] message IDs and improve reject/ignore handling (#1898)
* bump libp2p: use seq[byte] message IDs and improve reject/ignore handling

* bump libp2p to pick up debug message details
2020-10-21 20:03:12 +00:00
tersec a136c2e95a
bump libp2p; integrate pubsub.ValidationResult into extended validation (#1893) 2020-10-20 12:31:20 +00:00
Zahary Karadjov dbc90e998a Add Disc v5.1 bootstrap nodes for Zinken
Also prepares the Makefiles for supporting Medalla
through a separate binary
2020-10-18 12:44:52 +03:00
Zahary Karadjov d98be078c3 Address #1690 Min password entropy 2020-10-16 19:34:26 +03:00
cheatfate 7ad59b1852 Bump nim-stew. 2020-10-15 20:18:33 +03:00
cheatfate 84fe294c00 Fix protection of data folder and security files using Windows ACL. 2020-10-15 20:18:33 +03:00
Zahary Karadjov ce1fda1195 Store the deposits and the immutable validator data in Sqlite 2020-10-15 20:15:51 +03:00
Zahary Karadjov e6320e5881 Address #1584 Don't keep all deposits in memory (persist them to disk) 2020-10-15 20:15:51 +03:00
Zahary Karadjov 8ce0fc3a89 Address #1687
This addresses the issues by detecting and rejecting keystores with
incorrect PBKDF2 and SCrypt params. It also bumps the version of
nim-json-serialization to include a bugfix for incorrect parsing
of json files featuring comments.
2020-10-15 20:15:51 +03:00
Mamy Ratsimbazafy bcd9088114
Bump blscurve/blst (missed in #1775) (#1870) 2020-10-15 14:25:12 +02:00
Ștefan Talpalaru 1ae3fb90ba
Linux AMD64 binary distribution (#1844)
* Linux AMD64 binary distribution

The builds are reproducible, as long as our base Docker Hub image
remains available.

tl;dr: `make dist`

* use UTC dates
2020-10-15 12:19:41 +00:00
tersec 513ba72b9a
add v1.0.0-rc.0 support behind compile-time flag (#1852)
* add v1.0.0-rc.0 support behind compile-time flag

* keep runtime presets consistent
2020-10-13 17:21:25 +00:00
tersec 15c989f70d
bump eth2-testnets (#1856) 2020-10-12 12:40:01 +00:00
Zahary Karadjov 5bf2cd623a
Add a genesis state for zinken 2020-10-10 13:26:26 +03:00
Zahary Karadjov 00a8a68671
Address #1695
Better error messages when the beacon node is asked to listen on a
reserved port (0) or an already taken one.
2020-10-09 16:39:03 +03:00
Zahary Karadjov d534570cf6
Use latest Zinken metadata (correct genesis time and bootstrap nodes) 2020-10-08 11:20:06 +03:00
Zahary Karadjov b46c8410a6 Retire Spadina and prepare for the Zinken network 2020-10-07 18:12:21 +03:00
Mamy Ratsimbazafy 6b9090398e
[JSON-RPC] Prevent silent uint conversion (#1819)
* Prevent silent uint conversion

* bump json-rpc
2020-10-07 15:02:54 +02:00
nbc-bump-bot[bot] 078c626554
auto-bump nim-libp2p (#1806)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2020-10-07 12:52:25 +02:00
Ștefan Talpalaru e9b2253fac
bump nim-nat-traversal and nimbus-build-system (#1824)
* bump nim-nat-traversal and nimbus-build-system

The "clean-cross" target has been moved to the latter.

* Windows fix
2020-10-07 12:01:27 +02:00
Zahary Karadjov aed291128a Add support for starting from weak subjectivity checkpoints
Also removes the `genesis.ssz` file stored in the data folder.
The `medalla-fast-sync` target has been adapted to use the new features.
2020-10-07 09:32:03 +03:00
Zahary Karadjov fec4b5014d Address #1694 2020-10-06 15:46:22 +03:00
Jacek Sieka f484d06f6b
bump json-rpc (#1813)
gets rid of ugly inf logs
2020-10-06 10:50:15 +02:00
cheatfate 5555d39ea9 Bump nim-stew. 2020-10-05 22:19:50 +03:00
cheatfate c5c788a9db Secure network key file and data directory. 2020-10-05 22:19:50 +03:00
Jacek Sieka 596600058a
bump nimcrypto (#1802) 2020-10-03 11:21:45 +02:00
Mamy Ratsimbazafy b57693ec0d
Logging update (#1795)
* Fix discovery log message trigger

* Bump chronicles - include https://github.com/status-im/nim-chronicles/pull/89 for better NOTICE/WARNING color
2020-10-03 08:35:45 +02:00
nbc-bump-bot[bot] 7a5f389f64
auto-bump nim-libp2p (#1738)
Co-authored-by: = <sinkingsugar@users.noreply.github.com>
2020-10-03 08:21:38 +02:00
Zahary Karadjov f5340998b9 Add a fuzzing test for the beacon node CLI parser 2020-10-02 12:42:32 +03:00
Mamy Ratsimbazafy a455dc1bc0
Bump confutils to allow overriding CLI options https://github.com/status-im/nim-confutils/pull/22 (#1781) 2020-09-30 16:37:29 +02:00
Mamy Ratsimbazafy 98661914f1
Revert "bump nim-libbacktrace (#1769)" (#1773)
This reverts commit de1b3315e8.
2020-09-28 22:37:18 +02:00
Ștefan Talpalaru de1b3315e8
bump nim-libbacktrace (#1769)
* bump nim-libbacktrace

so it works with LTO

* macOS fix
2020-09-28 21:23:53 +02:00
Mamy Ratsimbazafy 9a39bda5f1
BLSCurve EIP2333 updates for Spadina (breaks old keystore recovery) (#1742)
* BLSCurve EIP2333 updates for Spadina (breaks old keystore recovery)

* Workaround blst.h and vect.h having conflicting definition of limb_t
2020-09-25 19:52:02 +02:00
Jacek Sieka 7837646079
anonymize libp2p messages (#1756)
* anonymize libp2p messages

* bump
2020-09-25 18:40:30 +02:00
Jacek Sieka b3a9afa0b1
libp2p: limit max gossip writes (#1739)
* libp2p: limit max gossip writes

* bump
2020-09-24 19:03:17 +02:00
tersec 0eb53f2802
avoid unpacking phase 1 tests to reduce Azure CI disk usage (#1736) 2020-09-24 17:16:00 +02:00
Viktor Kirilov 410a72667b fixed interop with geth after updating nim-web3 2020-09-24 16:07:57 +03:00
Mamy Ratsimbazafy 1f5b487fc6
Github Actions CI (#1693)
* Clean up PR + bump nimbus-build-system

* pcre on 32-bit + Improve env variable handling + cache mingw

* Add badge + fix setting env variable

* Auto cancel if commit becomes outdated

* fix shell for deriving env variable

* Add more cancellation points

* Add finalization tests to Github Action

* Fix case

* change cancel actions + fixes for windows and finalization

* have to use matrix variable for cache path/key

* ARCH_OVERRIDE=LATFORM issue rebuild cache

* Update scripts - deactivate workflows with identified issues + reactivate caching

* workaround mac getopt

* Disable all aAVX512f extensions (Error: invalid register for .seh_savexmm in Cygwin)

* Fix cross compile of libminiupmp #1723

* Cache fetch-dlls to avoid being a drag on nim-lang.org

* Fix windows downloading DLLs twice and set CFLAGS env variable for Linux32

* fix silly yaml mistake

* .

* reactivate win32 after https://github.com/status-im/nim-beacon-chain/pull/1726

* Comment out minimal tests for now
2020-09-24 13:45:34 +02:00
nbc-bump-bot[bot] 4ded755d32
auto-bump nim-libp2p (#1732)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2020-09-24 09:57:15 +02:00
nbc-bump-bot[bot] d7bfef3c78
auto-bump nim-libp2p (#1711)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2020-09-24 07:26:56 +02:00
Jacek Sieka e1c177cdd1
bump libp2p (#1721)
gossipsub 1.1 can be enabled with -d:nbc_gossipsub_11
2020-09-22 19:34:34 +02:00
Zahary Karadjov 73724b2c4b Add attacknet-beta1-mc-0 2020-09-22 18:51:51 +03:00
Zahary Karadjov 10cfd1ec37 Adds the Spedina testnet; Upgrades the spec presets to spec v0.12.3
Other changes:

* Removed all references to Altona
* Refactored the Makefile to avoid code duplication
2020-09-22 18:51:51 +03:00
tersec 3190c695b0
minimal v0.12.3 update (#1716) 2020-09-21 15:58:35 +00:00
Jacek Sieka f8e782fb5f
bump nim-eth (#1708)
fixes WAL file growth in sqlite - might cause nbc to freeze once for a
minute or two while clearing the wal
2020-09-21 11:07:59 +02:00
Jacek Sieka 59a3862f6f
bump nim-stew, get bitops intrinsics 2020-09-21 08:46:29 +02:00
Ștefan Talpalaru 2b6c287dbb
update: delete "nimcache" dirs (#1706) 2020-09-21 08:39:13 +02:00
Mamy Ratsimbazafy ea4ec6a785
Upgrade blst (#1682)
* Bump BLST

* Test for https://github.com/supranational/blst/issues/22 regression

* Use SHA256 from BLST + bump nim-blscurve to reenable fno-tree-vectorize

* SHA256 on non-blst platforms import fixes

* import fixes again

* can't prefix with nimcrypto

* address review comment [skip ci]

* {.noInit.} on the digests
2020-09-18 16:55:55 +02:00
nbc-bump-bot[bot] df685f0c12
auto-bump nim-libp2p (#1659)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2020-09-16 14:22:19 +02:00
nbc-bump-bot[bot] 88eb0c06c7
auto-bump nim-libp2p (#1606)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2020-09-15 16:17:25 +02:00
Eugene Kabanov 617c295839
Bump chronos to fix memory leaks. (#1648) 2020-09-15 11:45:00 +02:00
tersec aca1a318f2
cleanly close kvstore databases and bump nim-eth (#1630)
* cleanly close kvstore databases

* close databases for all subcommands and during error conditions
2020-09-12 05:35:58 +00:00
Kim De Mey 5fff800cf8
Bump nim-eth to use lrucache for discovery sessions (#1622) 2020-09-11 17:46:48 +00:00
Eugene Kabanov 9db05c484d
Bump chronos to help investigate Index problem. (#1625) 2020-09-11 07:12:12 +02:00
Jacek Sieka 7c21fa983a
bump libp2p, introduce AsyncQueue BufferStream (#1599)
* Syncing workers now not bound to peers.
Sync status is now printed in statusbar.

* Add `SyncQueue.outSlot` to statusbar too.

* Add `inRangeEvent` and `rangeAge` parameter.

* Fix rangeAge is not depends on SyncQueue latest slot.
Fix syncManager to start from latest local head slot.

* Add notInRange event.

* bump libp2p, introduce AsyncQueue BufferStream

* bump

Co-authored-by: cheatfate <eugene.kabanov@status.im>
2020-09-10 08:22:39 +02:00
Jacek Sieka 4c3dd853e1
bump libp2p (#1609) 2020-09-07 14:17:15 +02:00
nbc-bump-bot[bot] c5f8ef8e40
auto-bump nim-libp2p (#1604)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2020-09-05 08:28:03 +02:00
Jacek Sieka f61fdf93d3
bump chronos (#1598)
* bump chronos

this reverts backwards-incompatible changes to CancelledError and
resores other functionality.

chronos metrics are lost and will need to be reimplemented.

* bump
2020-09-04 21:27:54 +02:00
Ștefan Talpalaru aa3298f0ea
support spaces in $(MAKE) (#1603) 2020-09-04 08:33:37 +02:00
nbc-bump-bot[bot] 6000a49e84
auto-bump nim-libp2p (#1484)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2020-09-01 11:26:20 +02:00
tersec d9f926b33d
bump nim-eth (#1579) 2020-08-31 10:50:48 +02:00
Ștefan Talpalaru ecc70c6405 remove BUILD_LOG_LEVEL 2020-08-27 18:44:10 +03:00
Jacek Sieka 2081c4f505
avoid unnecessary seq allocations (#1573)
"sigh"
2020-08-27 08:32:51 +02:00
Zahary Karadjov 207397775b
Update the Nimbus bootstrap nodes 2020-08-26 21:22:21 +03:00
Ștefan Talpalaru ab22bad188
bump vendor/nim-metrics (#1565) 2020-08-25 09:26:56 +02:00
Ștefan Talpalaru ddd8489de0
bump vendor/nim-blscurve (#1560)
and log the BLS backend
2020-08-24 14:56:45 +02:00
Zahary Karadjov 8809f8d200 Implement 'wallets restore' and 'wallets list'; Update 'nextAccount' properly after making deposits multiple times 2020-08-22 00:16:18 +03:00
Ștefan Talpalaru 1d1e0217ca bump vendor/nim-nat-traversal 2020-08-22 00:09:45 +03:00
Zahary Karadjov 2c19e3f8cd
[skip ci] Use GOSSIP_MAX_SIZE when snappy decoding in the inspector as well; Bumps 2020-08-19 14:33:52 +03:00
Zahary Karadjov 3433c77c35 Prevent Snappy decompression bombs 2020-08-19 10:13:04 +03:00
Zahary Karadjov 4cf54eadf9
Bump libp2p in order to restore interop with Lighthouse 2020-08-18 20:06:43 +03:00
Zahary Karadjov 17ca72cf55 Bump nim-libp2p 2020-08-17 17:24:36 +03:00
Jacek Sieka 8c56a01706 restore NBC finalization
the root cause is a deadlock where the bufferstream space is exhausted
and libp2p stops reading from the socket, eventually timing it out
2020-08-17 17:24:36 +03:00
Dmitriy Ryajov 87f983c639 use split out pubsub 2020-08-17 17:24:36 +03:00
Mamy Ratsimbazafy 454b9d0724
Bump nim-blscurve (#1491)
* Bump BLSCurve

* Use unified aggregation API

* use new blscurve with unified aggregate API

* bump

* fix toRaw

* replace state_sim combine with AggregateSignature

* Fix 32-bit

* Fix 32-bit for real and test deactivating ccache for fno-tree-lopp-vectorize flag

* change compilation switches to narrow down Linux issue

* Use -fno-tree-vectorize to disable both tree-loop-vectorize and tree-slp-vectorize

* blscurve now disables both Loop and SLP vectorization

* Add tests for the miracl/milagro fallback

* Travis has max log size of 4MB

* Test with Miracl in the finalization test

* fix state_sim log level

* Coment out the slow fallback tests
2020-08-15 19:33:58 +02:00
Eugene Kabanov 37cb0a2249
Bump nimcrypto (#1492) 2020-08-14 21:21:55 +02:00
Jacek Sieka 936440fccd
use libp2p peer events to track peer (#1468)
this resolves some peer counting issues that were happening because the
lifetime future in PeerInfo was unreliable (multiple PeerInfo instances
existed per peer)

In addition, this solves another race condition: when connecting to a
peer and later dialling that protocol, it is not certain that the same
connection will be used if there's a concurrent incoming peer connection
ongoing - better not make too many assumptions about who sent statuses
when.
2020-08-10 12:58:34 +02:00
Ștefan Talpalaru cd0f4a2d23
bump vendor/nim-metrics 2020-08-09 16:38:30 +02:00
Ștefan Talpalaru 5e9c91381e
bump submodules (#1472)
* fork News

* bump submodules
2020-08-08 18:00:19 +02:00
nbc-bump-bot[bot] 3d39757321
auto-bump nim-libp2p (#1461)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2020-08-07 18:29:13 +02:00
nbc-bump-bot[bot] 8d14e0aca6
auto-bump nim-libp2p (#1446)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2020-08-06 12:45:14 +02:00
Jacek Sieka 6d3f1b93be bump 2020-08-05 19:34:59 +03:00
Jacek Sieka 221f372dbc use peer id in a number of places 2020-08-05 19:34:59 +03:00
nbc-bump-bot[bot] b13b04bd7b
auto-bump nim-libp2p (#1444)
Co-authored-by: = <dryajov@users.noreply.github.com>
2020-08-05 13:02:14 +02:00
nbc-bump-bot[bot] ae072d39e6
auto-bump nim-libp2p (#1427)
Co-authored-by: = <stefantalpalaru@users.noreply.github.com>
2020-08-05 03:16:10 +02:00
Zahary Karadjov 25474d055b
Bump eth2-testnets 2020-08-04 17:27:19 +03:00
nbc-bump-bot[bot] 771ed7044a
auto-bump nim-libp2p (#1412)
Co-authored-by: = <dryajov@users.noreply.github.com>
2020-08-04 04:54:19 +00:00
Dmitriy Ryajov 52d9d269d7
bump libp2p (delayed send, without hooks) (#1413)
* use `switch.isConnected`

* libp2p

* add timeout to publish

* use isConnected

* adjust timeouts

* latest libp2p master

* do not drop peers
2020-08-03 16:43:22 +00:00
Zahary Karadjov 8c2a9a4950
Switch back to the old undepleted Infura key; Bake in the Medalla genesis state 2020-08-03 03:59:45 +03:00
Zahary Karadjov d052623381
Fix a trivial issue with 'make medalla' and add more logging 2020-08-03 00:19:25 +03:00
Zahary Karadjov c293254ded Add 'deposits import' command; Switch to NJS when loading the keystores and improve the data validation 2020-08-02 23:00:43 +03:00
Zahary Karadjov 2a94f6eca6 medalla-deposit target capable of executing mass deposits 2020-08-02 23:00:43 +03:00
Jacek Sieka 25d7a17745
Revert "bump"
This reverts commit adbcde07f2.
2020-08-02 11:57:46 +02:00
Jacek Sieka adbcde07f2
bump 2020-08-02 11:46:12 +02:00
Ștefan Talpalaru 8ab9174208
bump vendor/nim-chronicles (#1405) 2020-07-31 01:48:58 +02:00
Ștefan Talpalaru f2512a0c71
Nim-1.2.6-RC1 (#1398)
* Nim-1.2.6-RC1

* Azure: bump cache key
2020-07-30 14:08:49 +02:00
Zahary Karadjov 0c60e452e7 Allow the deposit contract deployment block to be specified as a number (needed for Medalla) 2020-07-28 18:13:01 +03:00
Zahary Karadjov 40ea9e9886 Integrate the Wallet support with the Deposits creation; Produce Launchpad-compatible deposits 2020-07-28 07:36:25 +03:00
Zahary Karadjov fcd412f7a1 Finish the 'create wallet' command; Addresses #1319 2020-07-28 07:36:25 +03:00
tersec 20a2525390
v0.12.2 beacon chain protocol update (#1378) 2020-07-27 12:59:57 +02:00
Zahary Karadjov 5a4f6b7e5e
Restore the SSZ fuzzign tests to working state 2020-07-22 16:20:07 +03:00
Jacek Sieka 6712c7bdfd
Bumps2 (#1336)
* bumps
2020-07-17 07:02:10 +00:00
Ștefan Talpalaru 9571cb5f7a
bump vendor/nimbus-build-system 2020-07-16 21:08:27 +02:00
Ștefan Talpalaru 88a9629c2b
bump submodules 2020-07-14 03:01:23 +02:00
Dmitriy Ryajov b0470d7318
bumps (#1289)
Co-authored-by: Jacek Sieka <jacek@status.im>
2020-07-13 22:05:04 +02:00
Jacek Sieka d6897409e4
bump-eth (#1310) 2020-07-13 16:58:56 +02:00
Kim De Mey 80cba4db91
Bump nim-eth (#1305) 2020-07-13 12:46:05 +02:00
Eugene Kabanov c544a3b250
Bump nimcrypto. (#1301)
* Bump nimcrypto.

* Add more recent version
2020-07-11 15:14:23 +02:00
Zahary Karadjov c06c60dcaf
Merge branch 'devel' into cli-reforms 2020-07-10 16:23:27 +03:00
Kim De Mey 396825344c
Bump nim-eth and adjust code for it (#1293) 2020-07-10 11:17:15 +02:00
Zahary Karadjov 540b2828b2
Adapt the local sim scripts to use the new run-time presets 2020-07-10 01:08:54 +03:00
Zahary Karadjov 318b225ccd
Merge devel and resolve the conflicts 2020-07-08 15:36:03 +03:00
Zahary Karadjov c4af4e2f35
Working test suite with run-time presets 2020-07-08 02:02:14 +03:00
Jacek Sieka 6fe0a623f5
Crypto rng (#1284)
* use bearssl rng throughout

* bump

* bump

* move keygen out of crypto
2020-07-07 17:51:02 +02:00
Zahary Karadjov 98761afc64
Run beacon_node --network=altona without initial Eth1 syncing 2020-07-02 18:52:48 +03:00
Zahary Karadjov e342b96d2e
Embed the Altona metadata in the NBC executable 2020-07-02 18:14:11 +03:00
cheatfate 76c12d493e Add check for late attestation sent messages in logtrace tool. 2020-07-02 10:36:04 +03:00
Kim De Mey c43bf48de2
Bump vendor/nim-eth (#1268) 2020-07-01 16:05:42 +02:00
Jacek Sieka ef2f037571
bump libp2p (#1267) 2020-07-01 13:41:40 +02:00
Dmitriy Ryajov 76a9708dff
bumping libp2p to latest master (#1258) 2020-06-30 09:43:39 +02:00
tersec b4db2ad693
remove v0.11.3 support; add block_sim to CI (#1253)
* remove v0.11.3 support; add block_sim to CI

* rm stray PERSISTENT_COMMITTEE_PERIOD

* remove TopicFilter.InteropAttestations

* bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
Zahary Karadjov 31e31bb30c Switch to monitoring of Blocks; More accurate genesis detection and faster Eth1 syncing 2020-06-29 02:18:48 +03:00
Jacek Sieka 816779733e
use the eth2 message id for gossip (in logging too) (#1246)
* use the eth2 message id for gossip (in logging too)

* bump

* add spec link
2020-06-28 22:06:50 +02:00
Jacek Sieka 70d468ac2b
bump libp2p (#1232) 2020-06-26 21:01:00 +02:00
Zahary Karadjov 384e512031 Implement EIP-2386 wallets; Progress towards a CLI for interactive wallet creation
For more information:
4494da0966/EIPS/eip-2386.md (specification)
2020-06-24 23:06:05 +03:00
Zahary Karadjov f20f077827 Introduce some short-cuts in the Eth1 monitoring for faster syncing 2020-06-24 19:22:52 +03:00
tersec 12ee61421f
bump nim-blscurve to switch default to v0.12.x BLS (#1228) 2020-06-24 14:53:16 +00:00
Jacek Sieka 1d709c09f4
secp: requiresInit (#1210)
* secp: requiresInit

* bump
2020-06-22 21:40:19 +02:00
Zahary Karadjov a2e28bef27 Further removal of enum support; Bump modules 2020-06-22 20:14:14 +03:00
Ștefan Talpalaru c06682f588
Makefile: move "nat-libs" to "deps" 2020-06-22 17:08:29 +02:00
Zahary Karadjov 7211a99086 Smooth out the differences between Ganache and Infura; Working local sim and Altona target 2020-06-22 17:30:04 +03:00
Zahary Karadjov 14274587cf More user-friendly logging during mainchain monitoring 2020-06-22 17:30:04 +03:00
Zahary Karadjov ffb613d3c6 Usability enhancements for Tmux
* Press 'q' to quit Tmux
* In case of errors, Tmux will wait for a keypress before closing a pane
2020-06-22 17:30:04 +03:00
Zahary Karadjov 3d9aff5d23 Fix some logical typos introduced in the rebase 2020-06-22 17:30:04 +03:00
Zahary Karadjov 1def383ad1 Reword the Tmux setup to handle all simulation scenarios in more visible way 2020-06-22 17:30:04 +03:00
Zahary Karadjov e9d68e2f7b Add simulation mode that bootstraps from an Eth1 ganache instance 2020-06-22 17:30:04 +03:00
Zahary Karadjov 7d3160b0b5 Local sim now includes a prometheus server, a Ganache instance and a simulation dashboard server (in Tmux mode) 2020-06-22 17:30:04 +03:00
Jacek Sieka a661ecbae1
bump libp2p (#1209) 2020-06-21 18:45:28 +02:00
Ștefan Talpalaru e61776b9ab
Merge pull request #1105 from status-im/Nim-1-2-2
Nim-1.2.2
2020-06-17 19:28:11 +02:00
Jacek Sieka 073046f8ab bump libp2p 2020-06-17 13:21:12 +00:00
Ștefan Talpalaru e072997e97
Nim-1.2.2 2020-06-17 14:42:22 +02:00
Viktor Kirilov 00537b391c updated news - includes a fix for the test suite of nim-web3 2020-06-17 11:15:55 +03:00
Ștefan Talpalaru 9335533503 bump submodules 2020-06-16 04:27:20 +00:00
tersec f3fd7e17a3 Revert "bump libp2p (#1173)"
This reverts commit 95a8a0e81c.
2020-06-15 14:31:28 +00:00
Jacek Sieka 95a8a0e81c
bump libp2p (#1173) 2020-06-14 13:45:05 +02:00
Ștefan Talpalaru cd80fab5a9
add link to nimbus-build-system docs 2020-06-13 02:01:19 +02:00
Kim De Mey 68a8b7d969
Filter discovery nodes on forkId (#1162) 2020-06-12 16:14:18 +02:00
Jacek Sieka f9c5769d42
bump libp2p 2020-06-11 10:12:52 +02:00
Jacek Sieka 016cc22173
show peer info on connect (#1155) 2020-06-11 07:14:26 +02:00
Ștefan Talpalaru 4517bf5dd4
Merge pull request #1156 from status-im/bump
bump submodules
2020-06-09 22:29:16 +02:00
Dustin Brody 8f7541959a allow compile-time switching between v0.11.3 and v0.12.1 2020-06-09 18:39:05 +00:00
Dustin Brody cc13153fc6 initial 0.12.1 beacon chain spec 2020-06-09 18:39:05 +00:00
Ștefan Talpalaru 2a70f8c9d1
bump submodules 2020-06-09 20:00:33 +02:00
李婷婷 9c1da836f7 fix all comments 2020-06-10 00:50:10 +08:00
李婷婷 77364548e2 rebase from devel 2020-06-10 00:29:29 +08:00
Viktor Kirilov b3710bfc0b updated nim-json-rpc to use instead of in logScope 2020-06-09 17:07:47 +03:00
Ștefan Talpalaru 63d8966316 bump submodules 2020-06-06 17:20:54 +00:00
Dmitriy Ryajov 99b2d6758a
upping libp2p (#1135) 2020-06-06 10:35:36 +02:00
Zahary Karadjov 83eeff4da9
Bump chronos 2020-06-05 20:10:43 +03:00
Zahary Karadjov 1e51ee1d01
Bump nim-eth 2020-06-05 19:37:42 +03:00
Zahary Karadjov 0c78fc39e7
Use the latest LibP2P 2020-06-05 19:34:12 +03:00
Zahary Karadjov 32bc4d9096 Use {.push raises.} in spec/datatypes
This was an attempt to trigger a previously reported crash, but
everything turned out to be fine.
2020-06-05 19:33:30 +03:00
Eugene Kabanov b22e5b6d1d
Fix #1108 (#1119) 2020-06-04 13:32:50 +02:00
Jacek Sieka 56ffb696be
reorder ssz (#1099)
* reorder ssz

* split into hash_trees and ssz_serialization, roughly, for hashing and
IO
* move bitseqs into ssz (from stew)
* clean up imports

* docs, imports
2020-06-03 15:52:02 +02:00
Zahary Karadjov ef90a45509
Bump testutils 2020-06-01 16:58:28 +03:00
Zahary Karadjov 0bcdabfcdf Detect another invalid SSZ input found through fuzzing
The first offset of an SSZ object should always have a fixed constant
value. Otherwise, some unused bytes may appear between the fixed portion
and the dynamic portion.

Please note that this fix shutds down the minimal forward compatibility
currently supported by the SSZ format (and thus, the expected behavior
must be clarified in the SSZ spec).
2020-05-30 14:47:35 +03:00
Zahary Karadjov 8d1fb2f3a6 Bump nim-eth to fix 'make witti' 2020-05-29 21:13:58 +03:00
Kim De Mey e33c8d9067
Bump nim-eth and accompanying discv5 cleanup (#1081) 2020-05-29 12:03:29 +02:00
Ștefan Talpalaru 4b731b266d
bump vendor/nimbus-build-system 2020-05-29 00:18:47 +02:00
Mamy André-Ratsimbazafy 49c33f0890 Use v0.12 ready BLS curve 2020-05-28 15:09:56 +00:00
Dmitriy Ryajov e238e8186c
bumping libp2p to latest master (#1075) 2020-05-28 06:38:07 +02:00
Ștefan Talpalaru 489565b5ab
Makefile: small testnet targets refactoring (#1078) 2020-05-28 06:28:10 +02:00
Zahary Karadjov 47864c7c6c
Fix wrong bump 2020-05-27 20:14:31 +03:00
Zahary Karadjov fe2bcb20d2
Bump modules to fix 'make witti-dev' 2020-05-27 20:13:18 +03:00
Zahary Karadjov 28128f4d2f Add a handler for the Goodbye message
The lack of body of `goodbye` in sync_protocol.nim was preventing
the respective LibP2P protocol to be mounted and advertised on the
network.

Adding a body fixes that, but I've also made some changes in the
P2P protocol codegen that will prevent the issue from happening
again (no body is now considered the equivalent of having an empty
body).
2020-05-26 22:17:26 +03:00
Zahary Karadjov b6c9de968f
Fix some build problems and warnings 2020-05-25 20:33:03 +03:00
Zahary Karadjov f41d823d07
Bump modules 2020-05-25 13:41:42 +03:00
Zahary Karadjov 833f19e942 Reform the networking layer in order to handle the new stricter SSZ API 2020-05-24 19:00:34 +03:00
Ștefan Talpalaru f16fb31060 bump vendor/nimbus-build-system 2020-05-22 13:51:37 +00:00
Ștefan Talpalaru e4173860f0
bump vendor/nimbus-build-system (#1050)
* bump vendor/nimbus-build-system

* Jenkins: switch to Bash for the "Build" stage
2020-05-21 00:02:05 +02:00
Ștefan Talpalaru 46a99c35f0
bump vendor/nimbus-build-system 2020-05-20 15:55:31 +02:00
Zahary Karadjov 202bbbab6c Bump some modules 2020-05-20 15:51:45 +03:00
Zahary Karadjov 52c3ebf631 Fuzzing tests for SSZ 2020-05-20 15:51:45 +03:00
Ștefan Talpalaru 383b22795c
bump submodules (#1043) 2020-05-20 06:57:39 +02:00
Dustin Brody 6c941b1ff7 spec v0.11.3 update 2020-05-19 17:16:55 +00:00
Zahary Karadjov a99977b772 Progress towards #991; Enable the distinct ssz.List type 2020-05-19 15:53:03 +03:00
tersec 8432932c11
update to spec v0.11.2 to keep compatible with Schlesi (#1036)
* update to spec v0.11.2 to keep compatible with Schlesi

* update spec/state_transition_epoch spec references to v0.11.2

* bump other spec refs
2020-05-18 23:05:27 +00:00
Dmitriy Ryajov 398166369b bump libp2p to latest master 2020-05-18 21:51:03 +00:00
Jacek Sieka 6be7d64e8c
bump libp2p (#1031) 2020-05-18 10:11:21 +02:00
Ștefan Talpalaru c232fd0b3d
bump vendor/nimbus-build-system 2020-05-15 14:27:21 +02:00
Jacek Sieka 6c9e2d98a8 bump libp2p 2020-05-14 16:41:19 +03:00
Jacek Sieka a605c7244e simplify libp2p snappy
* handle a few more exceptions gracefully (in libp2p also)
* unify libp2p varint parsing
* decompress directly into seq
* avoid seq slice
* stop oversized snappy processing earlier (lowers risk)
2020-05-14 16:41:19 +03:00
Viktor Kirilov 926d1c546e updated news, also no longer allowing event handlers to raise anything other than a Defect in the Web3 API 2020-05-14 14:10:26 +03:00
Zahary Karadjov e537ed94d6
Bump some modules 2020-05-13 12:19:58 +03:00
Zahary Karadjov 75c1c6a95c Enable Snappy by default (using LibP2P steams for now)
This refactors the newly added Snappy streaming back-ends trying to
make them more similar and to reduce the code duplication to a minimum.
2020-05-13 12:18:42 +03:00
Zahary Karadjov f055fad08a Make the Snappy FastStreams integration optional by duplicating it for LibP2P streams 2020-05-13 12:18:42 +03:00
Zahary Karadjov 7e846a0bce Implement sszSize 2020-05-13 12:18:42 +03:00
Zahary Karadjov dbe0010504 Bump nim-snappy 2020-05-13 12:18:42 +03:00
Zahary Karadjov 15f0153441 Cosmetic improvements 2020-05-13 12:18:42 +03:00
Zahary Karadjov 9538b60704 Integrate the async Snappy implementation 2020-05-13 12:18:42 +03:00
Zahary Karadjov a739d7e8d6 Adapt SSZ to the latest FastStreams API 2020-05-13 12:18:42 +03:00
Jacek Sieka 0d1a602cf5
bump libp2p 2020-05-11 21:06:58 +02:00
Ștefan Talpalaru 2d78523423
fix Grafana dashboards (#1006)
* fix Grafana dashboards

and refactor a couple of scripts

* start.sh: allow running standalone script
2020-05-11 18:35:53 +00:00
Jacek Sieka fb2e0ddbec
sync fixes (#1005)
* sync fixes

* fix Status message finalized info
* work around sync starting before initial status exchange
* don't fail block on deposit signature check failure (fixes #989)
* print ForkDigest and Version nicely
* dump incoming blocks
* fix crash when libp2p peer connection is closed
* update chunk size to 16 to work around missing blocks when syncing

* bump libp2p

* bump libp2p

* better deposit skip message
2020-05-11 18:08:52 +00:00
Ștefan Talpalaru a7a50824a1
more metrics (#1004) 2020-05-11 06:25:49 +00:00
Jacek Sieka 592d6f65c3
bump (#1001) 2020-05-09 16:47:14 +02:00
Ștefan Talpalaru c572f61129
bump vendor/nim-metrics 2020-05-09 01:13:57 +02:00
Ștefan Talpalaru fc9d6bc1b3
enable SO_REUSEADDR in libp2p transports 2020-05-08 21:50:20 +02:00
Eugene Kabanov 3ffd0d7216
Fix gossip messages not delivered. (#983) 2020-05-06 15:42:59 +00:00
Jacek Sieka e65f5c86da
vendor: bump (#954) 2020-04-30 18:27:04 +02:00
Zahary Karadjov 667bdf066a Bump Chronicles to fix a minor bug; Add a placeholder for disabled topics 2020-04-28 13:49:30 +03:00
Zahary Karadjov 7237a370ba Implement https://github.com/status-im/nim-beacon-chain/issues/872
Please see the newly added 'schlesi-dev' Makefile target.
It demonstrates how the log level can be specified for individual topics.

Additionally, when connecting to testnets like 'schlesi' there will be
two additional log files produced in the working directory:

* json-log.txt
* text-log.txt (in the textblocks format)
2020-04-28 13:49:30 +03:00
Jacek Sieka 898df9ba45
kvstore: port to nim-eth (#938) 2020-04-27 18:36:28 +02:00
Zahary Karadjov 740b76d152 Spec-compliant implementation of Eth1 monitoring; Eth1-enabled local sim
BEWARE! This commit will trigger a stack overflow during local sim
2020-04-26 13:04:53 +03:00
Jacek Sieka fe30b24aba
bump (#929) 2020-04-24 11:47:51 +02:00
Viktor Kirilov 0079242457 again exported the right overloads for serializing hashes properly to json (and not as byte arrays) and updating nim-eth 2020-04-23 20:46:13 +03:00
Jacek Sieka 6729d3c032
kvstore: fix raising, be harsher on database errors (#923)
* kvstore: fix raising, be harsher on database errors

* bump stew/serialization
2020-04-23 08:27:35 +02:00
Zahary Karadjov ae9ca33aca
Disable the faststreams destructors 2020-04-22 21:33:04 +03:00
Jacek Sieka ed74770451
spec: regulate exceptions (#913)
* spec: regulate exceptions

* a few more simple raises
2020-04-22 07:53:02 +02:00
Ștefan Talpalaru 2f74951c04
bump submodules (#912) 2020-04-21 16:54:32 +02:00
Giovanni Petrantoni b441a373db Bump libp2p submodule 2020-04-21 10:46:21 +09:00
Ștefan Talpalaru 1babc00255
new initial submodule update strategy (#904)
* new initial submodule update strategy

* Azure: fix command order
2020-04-19 01:25:21 +02:00
Jacek Sieka 8b6c51e78d
secp: convenience api (#897) 2020-04-17 15:29:49 +02:00
Ștefan Talpalaru 6975869ad6
bump submodules (#899) 2020-04-17 01:04:46 +02:00
Zahary Karadjov e7259ce7ec
Bump faststreams 2020-04-17 00:23:11 +03:00
Zahary Karadjov a4830976ab
hotfix: fix the selecting of the syncing protocol handlers 2020-04-16 15:01:19 +03:00
Ștefan Talpalaru 7b8d6ea61c
bump vendor/nimbus-build-system (#894)
* bump vendor/nimbus-build-system

* change beacon_node banner

* beacon_node: update copyright years

* version.nim is imported in beacon_chain.nimble
2020-04-16 00:20:27 +02:00
Zahary Karadjov 99f9bc4eee Fix the inspector build 2020-04-16 01:01:18 +03:00
Zahary Karadjov 672f690bf6 Implement most of the v0.11.1 networking changes 2020-04-16 01:01:18 +03:00
Zahary Karadjov 8d639d9bd7 Avoid accidental memory allocations in SSZ 2020-04-16 01:01:18 +03:00
Zahary Karadjov ca49494356 Fix test_fixture_ssz_generic_types 2020-04-16 01:01:18 +03:00
Zahary Karadjov f34fd46e3a Add snappy and protobuf-serialization; Use the latest FastStreams API 2020-04-16 01:01:18 +03:00
Ștefan Talpalaru 7435c2010f
remove last traces of Go support
You'll have to run `chmod -R 755 vendor/go && rm -rf vendor/go` locally.
2020-04-15 14:38:42 +02:00
Ștefan Talpalaru b7a32a17ba
bump submodules
and remove failing syncManagerGroupRecoveryTest
2020-04-14 18:21:56 +02:00
Jacek Sieka bf2355d877
bump modules (#885) 2020-04-13 08:57:55 +02:00
Jacek Sieka afa08c8e3c
crypto: cleanup (#882)
* crypto: cleanup

* fix several Defect-on-user-input
* make crypto interface more similar to secp
* use `crypto.nim` in all of nbc

* digest: raises

* fix

* vendor
2020-04-11 10:51:07 +02:00
Ștefan Talpalaru a7fe5e4218
Azure: change NimBinaries cache key
- also bump submodules to match "devel"
2020-04-08 17:33:13 +02:00
Ștefan Talpalaru 2308dc9520
bump Nim target commit to latest devel 2020-04-08 14:36:03 +02:00
Zahary Karadjov 779349c11d
Compile NBC with Nim 1.2 RC 2020-04-03 22:04:03 +03:00
Jacek Sieka d180724d41
Bump eth (#857)
* vendor: bump stew
* vendor: bump nim-eth
* vendor: bump web3 too
2020-04-03 11:12:38 +02:00
Ștefan Talpalaru eaf4594f77
bump submodules 2020-04-01 23:05:49 +02:00
Ștefan Talpalaru 3ccb2dcb88
Merge branch 'discv5-ip-checks' into bump 2020-04-01 22:48:17 +02:00
Dmitriy Ryajov f811ed63be
use pkill in start.sh to fix mac os (#845) 2020-04-01 10:53:05 +00:00
Ștefan Talpalaru 3239b62904
move "--import:libbacktrace" to config.nims
This simplifies the `./env.sh nim c -r ...` scenario.
2020-03-31 14:53:41 +02:00
kdeme 8b9c8a692e
Discv5: Don't pass ip address when external ip is not known 2020-03-31 12:02:13 +02:00
tersec f5f939bd31
0.11.1 beacon chain spec update (#836)
* initial 0.11.1 spec commit; no test regressions and finalizes in eth2_network_simulation

* with BLS 0.10/0.11 available, stop skipping attester slashing, proposer slashing, and voluntary exist operations fixture tests

* switch param orders to group state.{fork, genesis_validators_root}; bump spec/datatypes spec version for network purposes

* mark attestation construction and broadcast and some minimal/mainnet constants as 0.11.1-compatible; remove phase 1 sharding constants from minimal which don't exist in that preset
2020-03-30 11:31:44 +00:00
Dmitriy Ryajov 9b07b87ca5
bump libp2p (#837)
Both `make eth2_network_simulation` and `./scripts/launch_local_testnet.sh --nodes 4 --disable-htop -- --verify-finalization --stop-at-epoch=7` work locally.
2020-03-28 19:20:52 +00:00
Zahary Karadjov 86a3bea1ec Use the latest Chronos and Libp2p 2020-03-24 22:14:40 +02:00
kdeme b80a5b90df Adjustments to store bootnode enrs in discovery object + bump nim-eth 2020-03-24 22:14:40 +02:00
Ștefan Talpalaru 1752e1c2b6 bump submodules 2020-03-23 14:57:26 +02:00
Zahary Karadjov 5868afe341
Restore compilation with -d:p2pdump 2020-03-22 19:28:11 +02:00
Zahary Karadjov d589cb7964 Fix the build with Json logging enabled 2020-03-22 19:00:09 +02:00
Zahary Karadjov 7d2381240f Bump nim-eth and nim-json-rpc 2020-03-22 19:00:09 +02:00
Ștefan Talpalaru 4f0caeceed
Merge branch 'bump' into devel 2020-03-18 20:20:10 +01:00
tersec 2a2d6ea6f7
remove vestiges of rocksdb (#813) 2020-03-18 19:43:51 +01:00
Ștefan Talpalaru b0e7e248d5
bump submodules 2020-03-18 17:20:34 +01:00
Dustin Brody 75003c22e1 drop 0.10.1 EF test vectors 2020-03-17 12:06:56 +02:00
tersec 33687c3e41
remove lmdb (#809) 2020-03-14 11:33:37 +01:00
Zed f7862a4cc2 Update testutils submodule 2020-03-13 14:38:59 +00:00
Mamy André-Ratsimbazafy 04234bf90e Fix second stack-smashing (https://github.com/status-im/nim-blscurve/issues/43#issuecomment-598357932) 2020-03-13 12:20:41 +00:00
Mamy André-Ratsimbazafy 28dc8a6a29 Fix mock signatures/validator keys and stack smashing 2020-03-13 12:20:41 +00:00
Mamy André-Ratsimbazafy 985298be1e update blscurve with privtopub, serialize and exportRaw primitives 2020-03-13 12:20:41 +00:00
Mamy André-Ratsimbazafy c418bf2b24 Update vendor/nim-blscurve 2020-03-13 12:20:41 +00:00
Dustin Brody 158f3a5c13 bump EF test vectors to 0.11.0 (keeping 0.10.1) and clean up leftover tests/official/fixtures/ directory 2020-03-13 11:40:05 +00:00
Ștefan Talpalaru 68f166800d bump vendor/nim-metrics 2020-03-13 08:21:36 +00:00
Ștefan Talpalaru 1caafba79c
Merge pull request #803 from status-im/misc/bump-libp2p
bumping libp2p to latest master
2020-03-12 01:57:11 +01:00
Dmitriy Ryajov 8d6bfd7ce7 bumping chronos to latest master 2020-03-11 10:53:36 -06:00
kdeme 2e5899c6d7 Bump nim-eth for discv5 fixes 2020-03-11 16:05:11 +00:00
Dmitriy Ryajov 50df0051a8 bumping libp2p to latest master 2020-03-11 10:02:28 -06:00
Jacek Sieka cf2dd002f6
bump stew 2020-03-10 12:54:06 +01:00
Jacek Sieka 1f1f06e8d9
Revert "bump modules"
This reverts commit e93bb880cf.
2020-03-06 23:46:34 +01:00
Jacek Sieka e93bb880cf
bump modules 2020-03-06 23:12:25 +01:00
Jacek Sieka 435ce15b04
bump modules 2020-03-05 17:52:26 +01:00
kdeme 02717a6354 Use randomNodes from discv5 instead of lookup 2020-03-05 13:17:34 +02:00
Zahary Karadjov 2623ac191f
Force the nodes in the local sim to announce 127.0.0.1 in discovery 2020-02-27 20:45:26 +02:00
Ștefan Talpalaru c2c570fcb0
bump submodules
plus a few small modifications
2020-02-26 01:14:20 +01:00
Zahary Karadjov e4ed51a144
Add nim-testutils 2020-02-24 15:39:50 +02:00
Ștefan Talpalaru 8b8a99135f
bump vendor/nim-eth
and improve htop processes in launch_local_testnet.sh
2020-02-23 01:56:29 +01:00
Zahary Karadjov fa0f3b4468
Use the specified extip address in the advertised ENR records 2020-02-22 20:54:15 +02:00
Dmitriy Ryajov 08915ed597 bump libp2p and increase bootstrap timeout 2020-02-21 12:03:19 -06:00
Ștefan Talpalaru c10d53faaf
bump submodules 2020-02-21 01:35:54 +01:00
Ștefan Talpalaru fbb16d8aa3
bump vendor/nim-libbacktrace 2020-02-20 22:06:40 +01:00
Jacek Sieka c25f8d4adf
sqlite kvstore backend (#749) 2020-02-20 11:49:34 +01:00
Zahary Karadjov 55886ff0ac
More proper fix replacing hotfix 052a6ac3bc
nim-eth was bumped to take advantage of a fixed `==` for public eth keys
2020-02-19 11:59:38 +02:00
Zahary Karadjov 2d53f42fc4 Switch to the merged PR in Chronos and nim-eth 2020-02-18 12:53:49 +02:00
Zahary Karadjov b2d5aba749 Advertise the TCP port of the bootstrap node properly 2020-02-18 12:53:49 +02:00
Zahary Karadjov 2c814db750 Temporary switch to supporting only ENR bootstrap addresses
Turns out the DiscV5 code relies heavily on the presence of ENR
records at the moment, so we cannot drive it with ENodes. @kdeme
is working on refactoring that will relax these requirements.
2020-02-18 12:53:49 +02:00
Zahary Karadjov d2f59ebab9 Switch to ENR records in the bootstrap_nodes.txt file 2020-02-18 12:53:49 +02:00
Zahary Karadjov a620c83916 Handle status bar spanning multiple lines (relevant for the Tmux network sim) 2020-02-18 12:53:49 +02:00
Ștefan Talpalaru 14b207e22d
bump vendor/nim-libp2p 2020-02-18 01:13:53 +01:00
Mamy André-Ratsimbazafy 59bfc1c7b5 Rebrand "fixtures" -> scenario + move to "vendor" 2020-02-17 15:26:11 +00:00
Ștefan Talpalaru e72dfeed73
nim-libbacktrace: don't build the C++ wrapper 2020-02-15 12:58:58 +01:00
Ștefan Talpalaru e5629ebc1f
bump vendor/nim-libbacktrace 2020-02-14 12:50:59 +01:00
Ștefan Talpalaru 06322385d9
lightweight stack traces
- plus some light test runner refactoring and some Makefile cosmetic changes
- compile tools with LOG_LEVEL=TRACE in CI
- bump a couple of submodules
2020-02-13 17:54:53 +01:00
Ștefan Talpalaru 1ce6d6bf31
fix Azure Pipelines 64-bit build 2020-02-12 02:05:24 +01:00
Ștefan Talpalaru 649d691081
bump submodules 2020-02-11 23:16:02 +01:00
Ștefan Talpalaru b2b284dd25
fork NimYAML
because of https://github.com/flyx/NimYAML/issues/77
2020-02-11 22:21:07 +01:00
Jacek Sieka 5cea471c4b
bump libp2p 2020-02-08 20:19:33 +01:00
Dmitriy Ryajov 0bc16518ed bumping libp2p to latest 2020-02-05 20:59:32 +01:00
Zahary Karadjov d634eba3fd
Produce a json log file in the data dir of each node in the local network sim 2020-02-04 16:09:21 +01:00
Ștefan Talpalaru 2a7ef4e5e5
Nim 1.0.6 2020-01-31 22:26:10 +01:00
Ștefan Talpalaru 28264aa293
Nim 1.0.6 RC 2020-01-31 22:26:07 +01:00
Zahary Karadjov 1f1a244f3f Implement more checks and wire up the invalid SSZ tests 2020-01-29 10:10:28 +02:00
Zahary Karadjov 70a387d1c7
Bump Chronos 2020-01-28 22:29:18 +02:00
Zahary Karadjov 257771d9af
Bump Chronos 2020-01-28 00:59:37 +02:00
Jacek Sieka 4568d290b4
result -> stew/result 2020-01-24 09:36:04 +01:00
Zahary Karadjov 9daa10ee6d Bump modules 2020-01-21 01:26:57 +02:00
Zahary Karadjov 4848b12050 Detect and use Lighthouse's boot_enr.yaml files 2020-01-21 01:26:57 +02:00
Zahary Karadjov 52878405b7 Integrate Discovery V5 and support ENR bootstrap records 2020-01-21 01:26:57 +02:00
Zahary Karadjov ff140689da Bump nim-libp2p and nim-eth (switched to enr branch) 2020-01-21 01:26:57 +02:00
Jacek Sieka 2a67ac3c05 trie -> kv store
* simplify data storage to key-value, tries are not relevant for NBC
* locked-down version of lmdb dependency
* easier to build / maintain on various platforms
2020-01-20 13:39:37 +00:00
Ștefan Talpalaru 6a9baf9688
bump vendor/nim-libp2p 2020-01-13 19:53:42 +01:00
Jacek Sieka 746659bdc6
bump stew, fix warnings (#655) 2019-12-20 14:25:33 +01:00
Jacek Sieka b994da78a7 Move some attestation/block logic out of beacon node
* state data cache in block pool
* keep head state around
* more attestation logic in attestation pool
* first fork choice tests (!)
* fix fork choice (it's still likely broken / out of date)
2019-12-19 16:17:45 +00:00
Ștefan Talpalaru bc8264a043
bump nim-eth & nim-nat-traversal 2019-12-12 02:35:15 +01:00
Ștefan Talpalaru 6dd54393ed
bump vendor/nim-nat-traversal 2019-12-10 16:46:31 +01:00
Jacek Sieka 2d5605524a
bump submodules 2019-12-10 15:52:15 +01:00
Jacek Sieka a3145f0f5c cleanups
* move stuff out of beaconnode
* fix use-after-free in query
2019-12-10 14:47:52 +00:00
Zahary Karadjov 10e6d4842f Local sim finalizes with native LibP2P 2019-12-10 15:02:16 +02:00
Zahary Karadjov 889031453f The beacon node can be compiled with the LibP2P native back-end
This hasn't been tested yet even in local sim.
2019-12-10 15:02:16 +02:00
Ștefan Talpalaru f8e2496b06
bump vendor/nimbus-build-system [skip ci] 2019-12-03 17:18:35 +01:00
Zahary Karadjov ee2448b221 Simple validator onboarding
On your very first connection to each testnet, you'll be asked to
become a validator. Please consult our private repo for a Goerli
Eth1 private key that you can use for deposits.

Other changes:

* Added a simple wrapper ./connect-to-testnet script calling the
  nims file in the correct environment. No extension was used to
  make the command the same on Unix and Windows.

* Bumped a number of modules with fixes from this week

* `make testnet0` and `make testnet1` will no longer delete your
  existing database. This is considered a more appropriate behavior
  for testing forward sync.
2019-12-03 02:23:17 +02:00
Ștefan Talpalaru 7682a010a9 remove getCurrentExceptionMsg() 2019-12-02 22:06:42 +02:00
Ștefan Talpalaru 84c189fd1b Nim 1.0.4 (#604)
* Nim 1.0.4

* Azure: try to fix cache key

* mock_validator_keys.nim: fix index out of bounds

* Travis: make Nim building verbose
2019-11-29 10:17:18 +01:00
Ștefan Talpalaru d5f20846ad Jenkinsfile setup 2019-11-28 17:32:46 +02:00
Zahary Karadjov 34ed2b0fa4 Bump nim-eth 2019-11-26 02:45:25 +02:00
Yuriy Glukhov 2e875ea17e Better connection error handling in MainchainMonitor 2019-11-25 13:45:53 +02:00
Ștefan Talpalaru 38077162b9
bump vendor/nimbus-build-system 2019-11-18 09:18:20 +01:00
Ștefan Talpalaru 99164bbfdd
bump vendor/nim-nat-traversal and vendor/nimbus-build-system 2019-11-14 17:52:01 +01:00
Zahary Karadjov 5a6a8e9520 Finalizing setup for testnet1; Auto-restart of the docker containers after testnet resets 2019-11-12 19:08:12 +00:00
Zahary Karadjov 7fe18732d5 Bump confutils 2019-11-12 19:08:12 +00:00
Zahary Karadjov ac5aabd569 Use the latest confutils 2019-11-11 23:29:36 +00:00
Zahary Karadjov fb180a1313 A `query` CLI command for inspecting the state of the database (WIP) 2019-11-11 23:29:36 +00:00
Ștefan Talpalaru 3192d1fb4d
bump vendor/nim-metrics 2019-11-10 18:57:51 +01:00
Ștefan Talpalaru c1a63035b9 beacon_node: "--version" option [skip ci]
fixes #466
2019-11-09 17:43:42 +00:00
Mamy André-Ratsimbazafy 9be6bfce35 bump nimyaml 2019-11-07 17:22:15 +00:00
Yuriy Glukhov 9b39c792d0 Revert f79f9deeb3 a0fbf5464a 94a9c51d8a (#523) 2019-11-06 16:56:54 +02:00
cheatfate 94a9c51d8a
Initial commit. 2019-11-06 16:41:23 +02:00
Ștefan Talpalaru 03f57aab43
bump vendor/nim-libp2p 2019-11-05 23:31:01 +01:00
Ștefan Talpalaru f8e8dc6c7a
bump vendor/nim-web3 2019-11-05 15:15:31 +01:00
Zahary Karadjov 14374504cf
Finalization experiements
* Switched to FloodSub
* Testnet0 switched to the minimal preset. All validators deployed on the master node.
2019-11-05 03:13:16 +02:00
Zahary Karadjov a6f350f35f
Bump modules 2019-10-30 18:35:52 +02:00
Yuriy Glukhov abbe4075fa Fixed deposit contract deployment 2019-10-30 17:23:34 +02:00
Ștefan Talpalaru f91918f1b6
bump vendor/nimbus-build-system 2019-10-29 16:10:05 +01:00
Ștefan Talpalaru c22d02e9a3
bump vendor/nimbus-build-system 2019-10-29 15:30:39 +01:00
Ștefan Talpalaru 18d32f4882
Merge branch 'master' of github.com:status-im/nim-beacon-chain 2019-10-29 15:16:29 +01:00
Ștefan Talpalaru bc9d0dd8de
bump vendor/nim-confutils 2019-10-29 15:16:13 +01:00
Zahary Karadjov f92db449e1
nims script for connecting to testnets published in the eth2-testnets repo
See https://github.com/ethereum/eth2.0-pm/pull/93 for more info
2019-10-29 01:07:31 +02:00
Zahary Karadjov ba0037738b
Removed all code related to the old network metadata files
Also switched to a more "standard" naming convention for the
command-line parameters.
2019-10-29 01:04:52 +02:00
Yuriy Glukhov 9d3889cbab
WIP Goerli testnet 2019-10-28 15:51:17 +02:00
Zahary Karadjov 1bcd045a2f
Bump modules 2019-10-28 15:00:25 +02:00
Zahary Karadjov 97fdb3399c
Switch to the final Nim v1.0.2 2019-10-28 15:00:25 +02:00
Zahary Karadjov e3380ea8f4
Bump modules 2019-10-28 15:00:25 +02:00
andri lim 47799a2b8a
bump stint 2019-10-28 15:00:25 +02:00
Zahary Karadjov 4bccbdce71
Use confutils compatible with Nim v1.0 2019-10-28 15:00:25 +02:00
Zahary Karadjov a876488075
Add nim-bearssl 2019-10-28 15:00:25 +02:00
Zahary Karadjov dfe3a6f0fb
Switch to Nim v1.0.2 2019-10-28 15:00:25 +02:00
Ștefan Talpalaru 3a2fc249a2
simulation: metric visualisation 2019-10-23 20:57:59 +02:00
Zahary Karadjov 021661180b
Use the latest nim-eth 2019-10-23 19:37:06 +03:00
Zahary Karadjov d3f88929da Integrate nim-prompt 2019-10-07 23:56:34 +09:00
Mamy André-Ratsimbazafy c1918d1253 bump again for unary var having precedence over | for tpe constraint 2019-09-26 17:53:59 +03:00
Mamy André-Ratsimbazafy c6c4cf82dd bum blscurve to fix in-place init signature - f9a29c2af9 2019-09-26 17:53:59 +03:00
Mamy André-Ratsimbazafy 5619f9e4e6 update nim-blscurve to integrate the Domain changes 2019-09-26 17:53:59 +03:00
Jacek Sieka ef92ea1321
bump modules 2019-09-11 20:29:51 -04:00
Zahary Karadjov c3adac44f2
Use Secp256k1 keys in accordance to the interop spec 2019-09-10 10:11:56 -04:00
Zahary Karadjov 9dec05f9c9 Sending chunked responses 2019-09-10 09:49:54 -04:00
Zahary Karadjov 2bbfa8c877 Changes related to the new ETH interop spec
* Hello is no longer a handshake message
  (all handshakes related code was deleted for clarity)

* Deal with the single-parameter inlining defined in the new spec
2019-09-10 09:49:54 -04:00
Yuriy Glukhov d4c480ff5f
Eth1 validators (#393) 2019-09-09 18:59:02 +03:00
Ștefan Talpalaru 026d569d8e bump go-libp2p-daemon to support Go 1.13 (#397)
* bump go-libp2p-daemon to support Go 1.13

* bump vendor/nimbus-build-system [skip ci]
2019-09-07 16:23:13 -04:00
zah 93cdb43f1e Integrate nim-metrics and add some metrics from the ETH2 spec (#394) 2019-09-07 19:48:05 +02:00
Ștefan Talpalaru 94ade77c99
move "go-checks" to nimbus-build-system [skip ci] 2019-09-05 13:58:42 +02:00
Ștefan Talpalaru 9547553066
Makefile: "nat-libs" is now included in "deps" 2019-09-04 19:19:39 +02:00
Ștefan Talpalaru 4d27a41924
move some submodules to nimbus-build-system 2019-08-28 15:16:44 +02:00
Ștefan Talpalaru 59fa03be89
process_lfs.sh: proper error message when `git-lfs` is missing
(and bump 2 submodules)
2019-08-27 17:01:19 +02:00
Ștefan Talpalaru 60dfa7081b
move build_p2pd.sh to nimbus-build-system 2019-08-25 15:44:45 +02:00
Ștefan Talpalaru c91727e7e5
bump submodules 2019-08-24 21:56:35 +02:00
Mamy André-Ratsimbazafy 39403ca39f
bump nim-serialization 2019-08-24 19:13:53 +02:00
Ștefan Talpalaru a19734e47f
bump vendor/nimbus-build-system 2019-08-23 20:26:35 +02:00
Ștefan Talpalaru fe18fcdc06
env.sh: exec -> source
bump vendor/nimbus-build-system
2019-08-23 19:07:50 +02:00
Ștefan Talpalaru 569985746a bump submodule 2019-08-23 14:36:29 +03:00
Ștefan Talpalaru 69ac78b2e0 new build system and submodules
- better p2pd building, with CI caching
2019-08-23 14:36:29 +03:00