Commit Graph

98 Commits

Author SHA1 Message Date
Jordan Hrycaj 103656dbb5 TxPool implementation
details:
  For documentation, see comments in the file tx_pool.nim.

  For prettified manual pages run 'make docs' in the nimbus directory and
  point your web browser to the newly created 'docs' directory.
2022-01-22 08:26:57 +02:00
Jordan Hrycaj cfe955c962
Feature/implement poa processing (#748)
* re-shuffled Clique functions

why:
  Due to the port from the go-sources, the interface logic is not optimal
  for nimbus. The main visible function is currently snapshot() and most
  of the _procurement_ of this function result has been moved to a
  sub-directory.

* run eip-225 Clique test against p2p/chain.persistBlocks()

why:
  Previously, loading the test block chains was fugdged with the purpose
  only to fill the database. As it is now clear how nimbus works on
  Goerli, the same can be achieved with a more realistic scenario.

details:
  Eventually these tests will be pre-cursor to the reply tests for the
  Goerli chain supporting TDD approach with more simple cases.

* fix exception annotations for executor module

why:
  needed for exception tracking

details:
  main annoyance are vmState methods (in state.nim) which potentially
  throw a base level Exception (a proc would only throws CatchableError)

* split p2p/chain into sub-modules and fix exception annotations

why:
  make space for implementing PoA stuff

* provide over-loadable Clique PRNG

why:
  There is a PRNG provided for generating reproducible number sequences.
  The functions which employ the PRNG to generate time slots were ported
  ported from the go-implementation. They are currently unused.

* implement trusted signer assembly in p2p/chain.persistBlocks()

details:
  * PoA processing moved there at the end of a transaction. Currently,
   there is no action (eg. transaction rollback) if this fails.
  * The unit tests with staged blocks work ok. In particular, there should
    be tests with to-be-rejected blocks.
  * TODO: 1.Optimise throughput/cache handling; 2.Verify headers

* fix statement cast in pool.nim

* added table features to LRU cache

why:
  Clique uses the LRU cache using a mixture of volatile online items
  from the LRU cache and database checkpoints for hard synchronisation.
  For performance, Clique needs more table like features.

details:
  First, last, and query key added, as well as efficient random delete
  added. Also key-item pair iterator added for debugging.

* re-factored LRU snapshot caching

why:
  Caching was sub-optimal (aka. bonkers) in that it skipped over memory
  caches in many cases and so mostly rebuild the snapshot from the
  last on-disk checkpoint.

details;
  The LRU snapshot toValue() handler has been moved into the module
  clique_snapshot. This is for the fact that toValue() is not supposed
  to see the whole LRU cache database. So there must be a higher layer
  working with the the whole LRU cache and the on-disk checkpoint
  database.

also:
  some clean up

todo:
  The code still assumes that the block headers are valid in itself. This
  is particular important when an epoch header (aka re-sync header) is
  processed as it must contain the PoA result of all previous headers.

  So blocks need to be verified when they come in before used for PoA
  processing.

* fix some snapshot cache fringe cases

why:
  Must not index empty sequences in clique_snapshot module
2021-07-14 16:13:27 +01:00
Jordan Hrycaj fbff3aea68
Feature/goerli replay clique poa (#743)
* extract unused clique/mining support into separate file

why:
  mining is currently unsupported by nimbus

* Replay first 51840 transactions from Goerli block chain

why:
  Currently Goerli is loaded but the block headers are not verified.
  Replaying allows real data PoA development.

details:
  Simple stupid gzipped dump/undump layer for debugging based on
  the zlib module (no nim-faststream support.)

  This is a replay running against p2p/chain.persistBlocks() where
  the data were captured from.

* prepare stubs for PoA engine

* split executor source into sup-modules

why:
  make room for updates, clique integration should go into
  executor/update_poastate.nim

* Simplify p2p/executor.processBlock() function prototype

why:
  vmState argument always wraps basicChainDB

* split processBlock() into sub-functions

why:
  isolate the part where it will support clique/poa

* provided additional processTransaction() function prototype without _fork_ argument

why:
  with the exception of some tests, the _fork_ argument is always derived
  from the other prototype argument _vmState_

details:
  similar situation with makeReceipt()

* provide new processBlock() version explicitly supporting PoA

details:
  The new processBlock() version supporting PoA is the general one also
  supporting non-PoA networks, it needs an additional _Clique_ descriptor
  function argument for PoA state (if any.)
  The old processBlock() function without the _Clique_ descriptor argument
  retorns an error on PoA networgs (e.g. Goerli.)

* re-implemented Clique descriptor as _ref object_

why:
  gives more flexibility when moving around the descriptor object

details:
  also cleaned up a bit the clique sources

* comments for clarifying handling of Clique/PoA state descriptor
2021-07-06 14:14:45 +01:00
jangko db8988fe64
EIP-1559: Fee market change for ETH 1.0 chain
Transaction and BlockHeader already updated in nim-eth repo
to support EIP-1559

EIP-1559 header validation and gasLimit validation
already implemented in previous commit

This commit deals with block validation:
- Effective gasPrice per EIP-1559
- new miner reward based on priorityFee
2021-06-30 20:30:39 +07:00
jangko 5159ad7aac
preparation for London hard fork
This preparation is needed for subsequent
EIPs included in London.

- Add London to Fork enum
- Block number to fork
- Parsing London fork in chain config
- Prepare gas costs table for London
- Prepare EVM opcode dispatcher for London
- Block rewards for London
- Prepare hive script for London
2021-06-29 07:34:45 +07:00
jangko 4a188788bd
preparation for EIP-1559 implementation
- unify signTx in test_helper and signTransaction in rpc_utils
  and put it into transaction.nim
- clean up mess by previous EIP-2930
2021-06-29 07:33:48 +07:00
Jamie Lokier a4f92a6543
Transaction: Use same log level for all block rejection causes
Block validation failure isn't an error, it's correct rejection of a bad block
from the network.  All conditions that lead to block rejection return a simple
boolean.

When a block is rejected, most reasons log at `debug` level.  Only `stateRoot`
mismatch shouts a loud, highlighted, multi-line error message with big red
`error` alert.

Historically this was to assist EVM development, because it was more likely to
be a Nimbus EVM bug than a real bad block.  But now the EVM is in good shape,
has a large and thorough testsuite, and `stateRoot` mismatch is more likely to
be a real bad block that should be rejected with less fuss.

If there's a genuine EVM bug, we'll still get an alert: Consensus failure will
quickly become obvious, and the block where it happens is easily fetched.

So a big, loud error is no longer useful, and it became a problem during tests.
Recently a few hundred tests were added that trigger it, and now successful
test output is filled with attention-grabbing errors which aren't really errors
or particularly useful.

Since it's not really an error, the original motivation is now backwards, and
other reasons warn at `debug` level, make this like the others.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-08 18:07:50 +01:00
Jamie Lokier aee0fe39d2
EVM: Remove `vm_types2` everywhere, use common forks list instead
File `vm_types2` is obsolete.  Remove this file and divert all imports to the
common forks list outside the EVM, or in some cases they don't need it anyway.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-08 15:36:31 +01:00
Jamie Lokier 775231eef1
EVM: Apply EIP-6 in the code (affects both vm and vm2)
The rationale in EIP-6[1] for changing names to `selfDestruct` applies to code
as much as it does to specs.  Also, Ethereum uses the new names consistently,
so it's useful for our code to match the terms used in later EIP specs and
testsuite entries.

This change is straightforward, and is a prerequisite for patches to come that
do things with the `selfDestruct` fields.

[1] https://eips.ethereum.org/EIPS/eip-6
Hudson Jameson, "EIP-6: Renaming SUICIDE opcode," Ethereum Improvement
Proposals, no. 6, November 2015.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-08 15:36:30 +01:00
Jordan Hrycaj ce8e5511e3 backport from test_blockchain_json, see issue #666 2021-05-24 07:57:21 +01:00
jangko 79044f1e92
eip2718: test_blockchain_json pass test 2021-05-15 18:09:35 +07:00
Jamie Lokier c7e1cb61ee
Transaction: Make transaction validation use new function txCallEvm
Split out and move the EVM setup and call in `processTransaction` to
`call_evm`.  This is the last part of the main program which calls the EVM
to be moved.  (There's still test code.)

While we're here, move the EIP2929 access list setup too, as the similarity
to `rpcInitialAccessListEIP2929` is obvious.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-04 00:56:03 +01:00
Jamie Lokier 4187eb1959
Transaction: Prepare txRefundGas to support txCallEvm
There's only one call left to `refundGas(Transaction, ...)`, and the
similarity to the tail of `rpcEstimateGas` is obvious.

Gather this into `call_evm`: `refundGas` -> `txRefundGas`.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-03 19:51:20 +01:00
Jamie Lokier 52fd8b8129
Transaction: Prepare txSetupComputation to support txCallEvm
After recent changes, there's only one call left to `setupComputation`, and
it's just a variant like `rpcSetupComputation` but for transaction processing.
The similarity to `rpcSetupComputation` is obvious.

Gather this into `call_evm`: `setupComputation` -> `txSetupComputation`.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-03 19:51:20 +01:00
Jamie Lokier 7eb4471004
Bugfix: Avoid numeric overflow when validating transaction value
It's possible for `tx.value` in the transaction to have a deliberately
constructed large 256-bit value, such that adding `gasLimit * gasPrice` to it
overflows to a small value.

Prior to this patch, the code would allow such a transaction to pass
validation, even though such a large transfer cannot be valid.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-03 19:34:22 +01:00
Jordan Hrycaj dfc93a74ad
moved validateTransaction() to executor
why:
  not part of VM (see andri's requested change at #573)
2021-04-07 15:13:28 +01:00
Jordan Hrycaj 827b8c9c81
reset explicit import paths for local modules
why:
  it was convenient to have relocatable source modules when writing the
  vm interface wrappers. this patch moves it back to the standard.

also:
  there are no deep links into the vm folder anymore which leaves some
  room for manoeuvring inside
2021-04-01 12:53:22 +01:00
Jordan Hrycaj 00ba7a2718
merge vm_forks and vm_opcode_values => vm_type2
why:
  all types, but they cannot be merged int vm_types because of a circular
  dependency.
2021-03-31 17:53:15 +01:00
Jordan Hrycaj 7b5d00307c
provide vm_precompiles as import/export wrapper
details:
  moved original vm/precompiles.nim => vm/nvm_precompiles.nim
2021-03-31 16:47:15 +01:00
Jordan Hrycaj eee24de450
provide vm_message as import/export wrapper
details:
  moved original vm/message.nim => vm/nvm_message.nim
2021-03-31 16:47:02 +01:00
Jordan Hrycaj cf3a356d76
provide vm_computation as import/export wrapper
details:
  moved original vm/computation.nim => vm/nvm_computation.nim
2021-03-31 16:38:10 +01:00
Jordan Hrycaj 3a3e4d5707
provide vm_forks as import/export wrapper
details:
  moved original vm/interpreter/vm_forks.nim => vm/interpreter/nvm_forks.nim
2021-03-31 16:03:34 +01:00
jangko 3db535aa39
EIP2929 implementation 2021-01-11 14:56:42 +07:00
jangko c68aa47464
fixes fork comparison related code 2020-11-19 11:59:53 +07:00
jangko f401622782
cache miner address instead of recalculation 2020-06-22 07:48:23 +07:00
jangko 2385df7bae
miner address calculation for PoA consensus engine 2020-06-19 20:24:09 +07:00
jangko bd7e1fe2e5
reuse VMState and AccountsCache for better performance 2020-06-18 13:16:38 +07:00
jangko eabacb0a33
allow nimbus to read geth database 2020-06-18 13:16:25 +07:00
jangko 7c026e1b48
modify VMState to enable witness data collection 2020-06-06 10:05:11 +07:00
jangko 844071033a
fix at various places related to missing accounts_cache.persist call 2020-06-01 13:45:32 +07:00
jangko 71514a0a66
replace state_db with accounts_cache 2020-05-30 10:14:59 +07:00
Jacek Sieka 4ade5797ee
rlp: don't use ranges / experimental features (#495) 2020-04-20 20:12:44 +02:00
andri lim af02a3b1b2
reduce unused import warnings 2020-04-15 19:05:57 +07:00
andri lim 87bae2bb78
switch to new toFork 2020-04-12 18:02:59 +07:00
andri lim 6591893158
change FkGlacierMuir to FkMuirGlacier [skip ci] 2020-04-11 16:02:15 +07:00
andri lim 266e0ddb1e
room for EIP-1283 2020-03-24 17:21:13 +07:00
andri lim 40a7b7da1b fixes glacier muir block reward 2020-02-20 09:21:35 +02:00
andri lim 270854a5aa simplify 'gasUsed' and 'refundGas' 2020-02-12 17:53:26 +02:00
andri lim d9991b1e8b simplify cash 2020-02-12 17:53:26 +02:00
andri lim 55494f06e5
move 'validateTransaction' from GST into 'processTransaction' 2020-01-24 19:52:55 +07:00
andri lim ea4f851f80 move create contract incNonce to 'applyMessage' 2020-01-23 18:07:44 +02:00
andri lim ec8abf40e4 move 'tx.getRecipient' into 'setupComputation' 2020-01-23 18:07:44 +02:00
andri lim 73c60fe694 unifying contract collision detection 2020-01-20 18:36:58 +02:00
andri lim 63e886655b fixes coinbase suicide bugs 2020-01-20 18:36:58 +02:00
andri lim 5e6018b20b [skip ci] collectTouchedAccounts reduction 2020-01-19 19:17:37 +02:00
andri lim 9dbcc7bb69 improve comment in 'processBlock' 2019-12-10 12:34:48 +02:00
andri lim 1ffb992674 multi root state trie implementation 2019-12-10 12:34:48 +02:00
andri lim 97e89b3afe implement 'getCommittedStorage' 2019-11-14 10:11:35 +00:00
andri lim 4cf3931f34 add istanbul block reward 2019-11-12 15:51:48 +00:00
andri lim 26ee138ea5 fixes GST regression 2019-09-26 12:33:43 +03:00