Commit Graph

90 Commits

Author SHA1 Message Date
Jacek Sieka a0b8b818f3
work around type issues 2020-06-01 14:52:52 +02:00
Jacek Sieka 4fc9a84c57
support array of uint64 2020-06-01 14:52:52 +02:00
Jacek Sieka f1dcee2749
wip: cache
* cache lists and arrays of complex objects (5x block processing speed
on ncli_db)

trivial baseline cache that stores tree in flat memory structure
2020-06-01 14:52:50 +02:00
Zahary Karadjov 0bcdabfcdf Detect another invalid SSZ input found through fuzzing
The first offset of an SSZ object should always have a fixed constant
value. Otherwise, some unused bytes may appear between the fixed portion
and the dynamic portion.

Please note that this fix shutds down the minimal forward compatibility
currently supported by the SSZ format (and thus, the expected behavior
must be clarified in the SSZ spec).
2020-05-30 14:47:35 +03:00
Zahary Karadjov 1fd87a2255 Remove SszReader.maxObjectSize 2020-05-28 19:15:16 +03:00
Zahary Karadjov d4fa2c8913 Avoid an unnecessary copy in readSszValue 2020-05-28 19:15:16 +03:00
Jacek Sieka 693bc15919 avoid some RVO bugs 2020-05-28 19:15:16 +03:00
Jacek Sieka f53b55cbe0 SSZ cleanup
* be stricter about SSZ length prefix
* compute zeroHash list at compile time
* remove SSZ schema stuff
* move SSZ navigation to ncli
* cleanup a few leftover openArray uses
2020-05-28 19:15:16 +03:00
Jacek Sieka 0d54073ec9 properly work around nim type bug in ssz
the cast worked around the bug at compile time by means of casting, but
introduced a runtime error, because the pre-cast type was still being
used during deserialization
2020-05-26 14:12:42 +03:00
Zahary Karadjov a8003e7e38 More complete integration of the List type; Detect more invalid inputs; Enable more tests 2020-05-24 19:00:34 +03:00
Zahary Karadjov accd5fe954 Don't use StackArray in ssz; Drop the support for strings 2020-05-24 19:00:34 +03:00
Zahary Karadjov a99977b772 Progress towards #991; Enable the distinct ssz.List type 2020-05-19 15:53:03 +03:00
Zahary Karadjov a739d7e8d6 Adapt SSZ to the latest FastStreams API 2020-05-13 12:18:42 +03:00
Jacek Sieka fb2e0ddbec
sync fixes (#1005)
* sync fixes

* fix Status message finalized info
* work around sync starting before initial status exchange
* don't fail block on deposit signature check failure (fixes #989)
* print ForkDigest and Version nicely
* dump incoming blocks
* fix crash when libp2p peer connection is closed
* update chunk size to 16 to work around missing blocks when syncing

* bump libp2p

* bump libp2p

* better deposit skip message
2020-05-11 18:08:52 +00:00
Jacek Sieka c74ba5c0c6
ssz: move ref support outside (#943)
* ssz: move ref support outside

Instead of allocating ref's inside SSZ, move it to separate helper:

* makes `ref` allocations explicit
* less magic inside SSZ
* `ref` in nim generally means reference whereas SSZ was loading as
value - if a type indeed used references it would get copies instead of
references to a single value on roundtrip which is unexpected

TODO: EF tests would benefit from some refactoring since they all do the
same thing practically..

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2020-04-29 22:12:07 +02:00
tersec be475a82d7
catch invalid 0-raw-byte Bitlist SSZ inputs during parsing (#936)
* catch invalid 0-raw-byte Bitlist SSZ inputs during parsing

* bump a couple more spec refs to v0.11.1
2020-04-26 16:26:53 +00:00
Zahary Karadjov fdcbfdff05 Pass the test suite with a BeaconState ref type 2020-04-26 13:04:53 +03:00
tersec 7bc18423fa
detect improper 0-length lists of variable-sized objects in SSZ reading (#928)
* detect improper 0-length lists of variable-sized objects in SSZ reading
2020-04-23 19:39:23 +00:00
Jacek Sieka ed74770451
spec: regulate exceptions (#913)
* spec: regulate exceptions

* a few more simple raises
2020-04-22 07:53:02 +02:00
Dustin Brody 4f970a067a fix variable offset bounding/comparison 2020-04-17 16:37:52 +03:00
Dustin Brody 71e802c740 harden SSZ reader 2020-04-17 16:37:52 +03:00
Zahary Karadjov 8d639d9bd7 Avoid accidental memory allocations in SSZ 2020-04-16 01:01:18 +03:00
Zahary Karadjov f34fd46e3a Add snappy and protobuf-serialization; Use the latest FastStreams API 2020-04-16 01:01:18 +03:00
Dustin Brody e7de8aae20 fix/restore 0.11.1 BeaconState genesis interop: (a) use GENESIS_FORK_VERSION properly in BeaconState init; (b) use GENESIS_FORK_VERSION to be compatible with zcli/zrnt initial eth1 deposits; (c) let List[foo] work with 64-bit ints, so BeaconState.validators SSZ-serializes properly; and (d) update test_interop to use new spec/crypto API 2020-04-07 13:16:55 +03:00
Jacek Sieka 7902d070cd prefer endians2 2020-03-05 18:13:28 +02:00
Zahary Karadjov 4fb654f2af Remove some SSZ compilation warnings 2020-02-18 12:53:49 +02:00
Zahary Karadjov 1f1a244f3f Implement more checks and wire up the invalid SSZ tests 2020-01-29 10:10:28 +02:00
Zahary Karadjov 182593ccb5 Tenative fix for the test suite 2020-01-29 10:10:28 +02:00
Zahary Karadjov 2a3c237bbb Handle malformed SSZ inputs properly 2020-01-29 10:10:28 +02:00
Jacek Sieka fd4de5de0f
start untangling beaconnode (#589)
* Move BeaconNode type to its own file (fewer imports)
* disentangle sync protocol/request manager
* fix some old nimisms
* de-fear some logs
* simplify eth1 data production
* add stack tracing to release builds
* drop release compile flag for testnet
2019-11-25 15:36:25 +01:00
Jacek Sieka 14712bbbdc
random warning fixes 2019-11-15 12:05:18 +01:00
Zahary Karadjov 22591deced Safer testnet restarts; Working CLI queries for inspecting the genesis states
When the connect_to_testnet script is invoked it will first verify that
the genesis file of the testnet hasn't changed. If it has changed, any
previously created database associated with the testnet will be erased.

To facilitate this, the genesis file of each network is written to the
data folder of the beacon node. The beacon node will refuse to start if
it detects a discrepancy between the data folder and any state snapshot
specified on the command-line.

Since the testnet sharing spec requires us to use SSZ snapshots, the Json
support is now phased out. To help with the transition and to preserve the
functionality of the multinet scripts, the beacon node now supports a CLI
query command that can extract any data from the genesis state. This is
based on new developments in the SSZ navigators.
2019-11-11 23:29:36 +00:00
Zahary Karadjov a644839b79 Dynamic SSZ navigator 2019-11-11 23:29:36 +00:00
Mamy André-Ratsimbazafy 5f6d60abce remove a lot of XDeclaredButNotUsed spam from SSZ 2019-11-07 15:22:04 +00:00
Zahary Karadjov a336285922 Fix a potential OOB error in SSZ deserialization 2019-09-27 23:29:58 +03:00
Zahary Karadjov 2bbfa8c877 Changes related to the new ETH interop spec
* Hello is no longer a handshake message
  (all handshakes related code was deleted for clarity)

* Deal with the single-parameter inlining defined in the new spec
2019-09-10 09:49:54 -04:00
zah 7ebf685fa3 Work-around for #373 (#385) 2019-09-05 10:27:26 +00:00
Mamy André-Ratsimbazafy ad55889707
Fix SSZ typo fieldCaseDisciminator (declared in nim-serialization) 2019-08-24 16:52:58 +02:00
Zahary Karadjov 5ce50b3aca
Implement the latest networking spec
https://github.com/ethereum/eth2.0-specs/pull/1328
2019-08-07 05:22:28 +03:00
Zahary Karadjov 398ea55801
Implement the latest SSZ specification and integrate the official SSZ test suite 2019-08-05 03:18:48 +03:00