Commit Graph

2 Commits

Author SHA1 Message Date
Jamie Lokier 57de56bab6
Sync: Add packet tracing to `blockchain_sync` network calls
Using the same packet tracing format to match `protocol_eth65`.
There aren't many calls, and this makes them clear.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-27 14:12:57 +01:00
Jamie Lokier c435409292
Sync: Move `blockchain_sync` code and use it with `eth/65`
Move `blockchain_sync.nim` from `nim-eth` to `nimbus-eth1`.

This lets `blockchain_sync` use the `eth/65` protocol to synchronise with more
modern peers than before.

Practically, the effect is the sync process runs more quickly and reliably than
before.  It finds usable peers, and they are up to date.

Note, this is mostly old code, and it mostly performs "classic sync", the
original Ethereum method.  Here's a summary of this code:

- It decides on a blockchain canonical head by sampling a few peers.
- Starting from block 0 (genesis), it downloads each block header and
  block, mostly in order.
- After it downloads each block, it executes the EVM transactions in that block
  and updates state trie from that, before going to the next block.
- This way the database state is updated by EVM executions in block order,
  and new state is persisted to the trie database after each block.

Even though it mentions Geth "fast sync" (comments near end of file), and has
some elements, it isn't really.  The most obvious missing part is this code
_doesn't download a state trie_, it calculates all state from block 0.
Geth "fast sync" has several parts:

1. Find an agreed common chain among several peers to treat as probably secure,
   and a sufficiently long suffix to provide "statistical economic consensus"
   when it is validated.
2. Perform a subset of PoW calculations, skipping forward over a segment to
   verify some of the PoWs according to a pattern in the relevant paper.
3. Download the state trie from the block at the start of that last segment.
4. Execute only the blocks/transactions in that last segment, using the
   downloaded state trie, to fill out the later states and properly validate the
   blocks in the last segment.

Some other issues with `blockchain_sync` code:

- If it ever reaches the head of the chain, it doesn't follow new blocks with
  increasing block numbers, at least not rapidly.
- If the chain undergoes a reorg, this code won't fetch a block number it has
  already fetched, so it can't accept the reorg.  It will end up conflicted
  with peers. This hasn't mattered because the development focus has been on
  the bulk of the catching up process, not the real-time head and reorgs.
- So it probably doesn't work correctly when it gets close to the head due to
  many small reorgs, though it might for subtle reasons.
- Some of the network message handling isn't sufficiently robust, and it
  discards some replies that have valid data according to specification.
- On rare occasions the initial query mapping block hash to number can
  fail (because the peer's state changes).
- It makes some assumptions about the state of peers based on their responses
  which may not be valid (I'm not convinced they are).  The method for working
  out "trusted" peers that agree a common chain prefix is clever.  It compares
  peers by asking each peer if it has the header matching another peer's
  canonical head block by hash.  But it's not clear that merely knowing about a
  block constitutes agreement about the canonical chain.  (If it did, query by
  block number would give the same answer more authoritatively.)

Nonetheless, being able to run this sync process on `eth/65` is useful.

<# interactive rebase in progress; onto 66532e8a

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-27 14:12:53 +01:00