Commit Graph

2274 Commits

Author SHA1 Message Date
Jamie Lokier e60ad129a2
Transaction: New object `TransactionHost` for "EVMC host services"
`TransactionHost` represents the interface to the EVM from the application once
we've fully transitioned to EVMC.  It represents a managed EVM call, and the
"EVMC host" side of the host<->EVM boundary.

This holds transaction state which sits outside the EVM, but a pointer to this
is passed around by the EVM as _opaque_ type `evmc_host_context`.

To the EVM, this offers "host services", which manage account state that the
EVM cannot do, such as balance transfers, call costs, storage, logs, gas
refunds, nested calls and new contract addresses.  The EVM has no direct access
to the account state database or network; it's all via "host services".

To the application (host side), this object represents a managed EVM call, with
inputs, a method to run, outputs, hidden transaction state, and later async
scheduling.  It is to replace `Computation` on the application side, while
`Computation` will remain but just be for the EVM side.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-28 12:12:02 +01:00
Jamie Lokier ee88e0266a EVMC: Bump `nim-evmc` module for type improvements (API 7.5 branch)
This bump is required by `TransactionHost` coming next.

There have been many changes to `nim-evmc` since earlier developments on
`nimbus-eth1`.  The types in the API have changed, and there is no longer a
need to cast the function types.

This commit bumps the submodule while keeping with EVMC API 7.5 for now.
`.gitmodules` is updated to follow the module branch `api-version-7.5`.

Submodule changes:

  > Nim: Function casts no longer required, type-checking finally works
  > Nim: Make the test host use exactly the type signatures in the API
  > add github action script
  > Nim: Export `evmc_create_vm_name_fn`
  > Nim: Use `var` instead of `ptr` for arguments that are pass-by-ref
  > Nim: Fix missing `ptr` in signature of `execute`
  > Nim: Use `ptr` in `release` for API consistency
  > Nim: Disable "Imported and not used" warning in test program
  > Nim: Remove unnecessary exports from the Nim example host/VM
  > Nim: Simplify Nim C++ codegen bug workaround
  > Nim: Add type and documentation for `evmc_create_example_fn`
  > Nim: Workaround Nim compiler bug with Nim 1.2.10
  > Nim: Workaround Nim compiler bug with `uint32`-sized bitset
  > Nim: Use a more varied bit pattern in `get_capabilities` test
  > Nim: Convert `evmc_capabilities` to Nim bitset
  > Nim: Convert `evmc_flags` to Nim bitset
  > Nim: Compile time verify that target C enums are `int` sized
  > Nim: Recognise that target C enums are not always `cint`
  > Nim: Make `evmc_status_code` a normal enum
  > Nim: Use `csize_t` in test program and Nim example host/VM
  > Nim: Use `csize_t` in the VM support API
  > Nim: Use `csize_t` in appropriate places to match EVMC API
  > Nim: Use binary-compatible C99 `bool` without target assumptions
  > Version (README): Announce we support EVMC 7.5.0 and Istanbul fork
  > Doc: Tweak GitHub badges on README
  > Doc: Make title fit on GitHub page without wrapping
  > Doc: README edits
  > Doc: Update the README for 2021 - the year of Nimbus!
  > License: Add licensing info about third-party files
  > License: Update license text in README
  > License: Fix incorrect name of Apache v2 license file and tweak
  > License: Add missing MIT license file
  > Nim: Add and update copyright headers
  > Nim: Update Nim API to follow EVMC API 7.5.0
  > Nim: Update doc comments and fix formatting
  > Make the tests work with the new C++ mini-EVM in EVMC 7.5.0
  > Change C++ `example_host` to workaround Nim -> C call type mismatch
  > The EVMC package expects C++14 now
  > Upgrade C++ `example_host` in `tests/evmc_c` to EVMC 7.5.0
  > Upgrade C++ `example_vm` in `tests/evmc_c` to EVMC 7.5.0
  > Upgrade C/C++ headers in `tests/evmc_c` to EVMC 7.5.0

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-28 13:14:17 +03:00
Jamie Lokier 537cac1bf5
EVM: Move where `continuation` is cleared to fix a potential stall
This fixes a bug spotted by @mjfh that was introduced by commit 2a7ccceb:

    try:
      if not c.continuation.isNil:
        (c.continuation)()
        c.continuation = nil
      c.selectVM(fork)
    except CatchableError as e:
      ...

The call to `(c.continuation)()` was moved by 2a7ccceb inside the `try` so
that, like all the Op functions do already, if the continuation raises, the
interpreter's general catch turns the exception into a an error status result.

But if the continuation raises an exception, `continuation` is not cleared in
the next line, and at the next resumption the continuation is called again.
It may loop doing this.

This doesn't currently happen because the continuations don't really raise, but
it's still a correctness issue.

This fix also allows a continuation to spawn a second continuation, if it
encounters a second suspension point.  This also doesn't happen currently,
but the pattern will become useful with async EVM.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-27 12:16:37 +01:00
Jamie Lokier f4de9d3c80
Tests: Use `ZERO_ADDRESS` in precompile tests
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-27 12:16:27 +01:00
Jamie Lokier 78233a4edc
Tests: Explain rules of "to" for call/create, use `ZERO_ADDRESS`
The conditions mentioned in the old TODO comment have been checked.  All
fixtures have either 40 hex digits or empty string for "to".  There is a test
with all-zeros, and it means send to that account, not contract creation.
Empty string means contract creation.

This patch does not change the relaxed parsing where fewer than 40 digits is
accepted.  We should probably be stricter about this.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-27 12:15:02 +01:00
Jamie Lokier 90a961243e
Clear up meaning of `ZERO_ADDRESS`, delete `CREATE_CONTRACT_ADDRESS`
There is no valid `CREATE_CONTRACT_ADDRESS`.  Some places on the internet say
account zero means contract creation, but that's not correct.

Transactions to `ZERO_ADDRESS` are legitimate transfers to that account, not
contract creations.  They are used to "burn" Eth.  People also send Eth to
address zero by accident, unrecoverably, due to poor user interface issues.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-27 12:14:51 +01:00
Jamie Lokier 745129c4ac
Transaction: Don't calculate `contractAddress` redundantly
Each place in `call_evm` that sets up an EVM call calculates the new contract
address for contract creations.  But it's redundant, because `newComputation`
ignores the provided value and does the calculation again.

Remove the unused address calculation.

This is also a step to merging different entry points and EVMC.  This change
ends up with the same value in both `msg.contractAddress` and `msg.codeAddress`
for every entry point, and this is good because it matches the EVMC message
structure, where they are replaced by only one value called `msg.destination`

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-27 12:01:47 +01:00
Jamie Lokier fa74dc909e
Fixtures: Verify EVM continuation is clear after `c.executeOpcodes`
`c.executeOpcodes` is called by some JSON fixture tests.  These tests bypass
some of the setup and return, and because of this call, continuations aren't
processed either.  Opcodes that use continuations will behave incorrectly.

The opcodes used in these particular tests don't use continuations currently,
so just add some assertions to verify this remains the case.

This is only used by local tests, and the call to `c.executeOpcodes` will be
replaced by the common entry point (that handles things like this correctly in
all cases) so we don't need to spend more time on this.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-27 11:54:44 +01:00
jangko 5fc57e4093
add validateKinship in persistBlocks of nimbus/p2p/chain.nim
put jordan's work #668 into effect, and this bring down
failing consensus test cases from 59 to 44
2021-05-27 16:28:26 +07:00
Ștefan Talpalaru 2539bd9904
CI: limit "push" events to the master branch 2021-05-26 21:55:32 +02:00
jangko 91b50235b0 add assertion guard in test_blockchain_json's parseBlock function
we don't want any surprice because of unrecognized key in test fixture
classified as `hasException`.
2021-05-26 15:46:04 +01:00
Jordan Hrycaj e5947f4db6 Deep copy semantics for LRU cache
why:
  follows standard nim semantics

details:
  changed Table to TableRef in previous patch which was the
  wrong choice (see andri's comment.)
2021-05-26 11:12:52 +01:00
Jordan Hrycaj 7b72109afa Use sorted RLP serialisation for LRU cache
why:
  previously, table data were stored with the table iterator. while
  loading a table with permuted entries will always reconstruct equivalent
  tables (in the sense of `==`), serialisation data are not comparable.

  this patch produces always the same serialised data for equivalent
  tables.
2021-05-26 07:58:12 +01:00
Jordan Hrycaj b83b47e541 LRU cache tests makeover
why:
  source-local unit tests would hardly be triggered by github CI as rightly
  criticised with the last patch.

details:
  source-local unit tests have been moved to tests folder.

  this version also contains rlp serialisation code so rlp encode/decode
  will apply tranparently. this is not needed in p2p/validate but will be
   useful with the clique protocol.
2021-05-26 07:58:12 +01:00
jangko 396f3e9909
add missing poaEngine configuration in config.nim
later we will use real engine configuration it if become available to us
2021-05-24 14:35:47 +07:00
Jordan Hrycaj 179cc75c32 Update LRU complexity comment
why:
  hash tables might worst-case degrade into linear mode, so it is not
  strictly O(1)
2021-05-24 07:57:21 +01:00
Jordan Hrycaj 1965ea027b updated LRU cache to run with complexity O(1)
why:
  to be used in Clique consensus protocol which suggests 4k cache entries.
  the previous implementation used OrderTable[] which has complexity O(n)
  for deleting entries.
2021-05-24 07:57:21 +01:00
Jordan Hrycaj a5e0bb6139 use general lru_cache for EpochHashCache
why:
  generic implementation will be also be used elsewhere
2021-05-24 07:57:21 +01:00
Jordan Hrycaj 3663b1603f pulled out cache logic into separate file
why:
  handy to re-use, eg. for upcoming clique implementation
2021-05-24 07:57:21 +01:00
Jordan Hrycaj d6a5cecb98 re-wrote validation with exceptionless functions
why:
  exceptions were from test code should be avoided in production code
2021-05-24 07:57:21 +01:00
Jordan Hrycaj 40c7bdfc06 update lookup cache management
details:
  enable fifo behaviour, using cache as argument
2021-05-24 07:57:21 +01:00
Jordan Hrycaj ce8e5511e3 backport from test_blockchain_json, see issue #666 2021-05-24 07:57:21 +01:00
jangko cbed4a368b
hive: update current state of the tests
from: ethereum/sync:           0 pass,     1 fail,     1 total
to  : ethereum/sync:           2 pass,     2 fail,     4 total
2021-05-20 18:47:11 +07:00
jangko 19f343b979
hive: use HIVE_NETWORK_ID to initialize networkId if available
devp2p/eth simulator is using this HIVE_NETWORK_ID to configure client
2021-05-20 14:04:17 +07:00
jangko b82061bf46
don't mix customBootNodes and bootNodes usage 2021-05-20 14:04:17 +07:00
jangko a0d10f5728
drop PublicNetwork enum usage and replace it with NetworkId
we cannot limit the `--networkid` switch to values available in
`PublicNetwork` enum. it should able to accept very wide range of
custom NetworkId.
2021-05-20 14:04:16 +07:00
jangko d12e7d22bf
update hive stat
additional simulators can run:
- `devp2p/discv4`
- `devp2p/eth`

from:
devp2p/discv4:           0 pass,    14 fail,    14 total
devp2p/eth:              0 pass,     1 fail,     1 total

to:
devp2p/discv4:           3 pass,    11 fail,    14 total
devp2p/eth:              1 pass,     8 fail,     9 total
2021-05-19 17:51:13 +07:00
jangko 2fd2665026
update hive client script
add enode.sh
use HIVE_BOOTNODES flag in nimbus.sh
2021-05-19 17:51:12 +07:00
jangko ec1af91370
implement "net_nodeInfo" rpc
fixes #569
2021-05-19 16:35:13 +07:00
Ștefan Talpalaru 82f99dc3c0
Windows: support more Mingw-w64 versions
This reverts commit 295c1409f6.
2021-05-19 00:42:56 +02:00
Jordan Hrycaj 91e24e3581
collected wisdom after working with hive/docker (#669) 2021-05-18 16:38:35 +01:00
jangko 18553156b1
hive: update hive flags usage checklist in nimbus.sh
additional flags we use:
- [x] HIVE_NETWORK_ID              network ID number to use for the eth protocol
- [x] HIVE_LOGLEVEL                client loglevel (0-5)
- [x] HIVE_GRAPHQL_ENABLED         enables graphql on port 8545
2021-05-18 09:31:11 +07:00
jangko cc1034aecf
hive: fixes mapper.jq related to DAO fork
hive always set 'HIVE_FORK_DAO_VOTE' to '1'.

set 'daoForkSupport' to 'true' if only 'HIVE_FORK_DAO_BLOCK'
is set by hive, not depends only on 'HIVE_FORK_DAO_VOTE'.

This is because 'nimbus-eth1' internal is different compared to 'geth'.
2021-05-18 09:23:12 +07:00
jangko b02095a45a
hive: update current state of the tests
additional simulators can be run:
- `ethereum/rpc`
- `smoke/network`
- `smoke/genesis`

number of passess and fails:
from: ethereum/consensus:  27353 pass,   892 fail, 28245 total
to  : ethereum/consensus:  28186 pass,    59 fail, 28245 total

new:
- smoke/genesis:           3 pass,     0 fail,     3 total
- smoke/network:           1 pass,     0 fail,     1 total
- smoke/clique:            0 pass,     1 fail,     1 total
2021-05-18 09:01:11 +07:00
jangko 0ecf9fe1af
add more query fields and resolvers to graphql api
after EIP2718/EIP2930, we have additional fields:

type AccessTuple {
  address: Address!
  storageKeys : [Bytes32!]
}

type Transaction {
  r: BigInt!
  s: BigInt!
  v: BigInt!
  # Envelope transaction support
  type: Int
  accessList: [AccessTuple!]
}

close #606
2021-05-18 07:32:03 +07:00
jangko af1537fcab
fixes and add hive simulators written in nim
now we have hive simulators written in nim:
- ethereum/consensus
- ethereum/graphql

Using these simulators, we can debug test cases without have to run
hive or docker.
2021-05-17 20:48:22 +07:00
jangko d2b47139e1
fixes `importRlpBlocks` in conf_utils.nim
now we are importing block one by one to satisfy
some of hive test cases.

we also catch exception instead of letting it terminate the
process.
2021-05-17 18:43:44 +07:00
jangko 43bc010c9c
remove test_vm_json
reason:
- they already covered by test_blockchain_json
  and test_generalstate_json.
- new ethereum/tests already remove them,
  nothing to tests anymore
2021-05-17 11:14:34 +07:00
jangko 6055e0bd0f
update tests logs
recent EIP2718 and EIP2930 inclusion and ethereum/tests update
add and remove many entries
2021-05-17 11:14:33 +07:00
jangko 77272544c0
fixes test_helper's jsonTestImpl
when a skipped test case not in the skip list
but skipped using `testStatusIMPL`, the generated
markdown file list it as failed.

now that bug is fixed.
2021-05-17 11:14:33 +07:00
jangko df9bae3517
bump nim-eth submodule
eip2718: protect against malformed rlp in tx sequence decoder
2021-05-17 11:14:33 +07:00
jangko 224b1e833b
bump ethereum/tests submodule
reason: allow EIP-2930 test cases to participate in test
2021-05-17 01:30:44 +07:00
jangko 79a52b11fd
fixes test_blockchain_json: better error message
also remove local EthBlock type and use EthBlock from nim-eth/common
2021-05-17 01:29:03 +07:00
jangko 76543da456
disable EIP-2537: Precompile for BLS12-381 curve operations
reason: not included in berlin hard fork

but we keep the code around, for future inclusion
2021-05-17 01:29:03 +07:00
jangko 3ccc4642f2
disable EIP-2315: Simple Subroutines for the EVM
reason: not included in berlin hard fork
2021-05-17 01:29:03 +07:00
jangko 6fc3df637c
reenable EIP-2565: modExp gas cost
now it's officially included in berlin hard fork
2021-05-17 01:28:31 +07:00
jangko 01a27ff328
EIP-2930: optional access list
the new AccessListTx contains address and storage keys
that will go into global access list. and this come with
prices.... in ether
2021-05-16 19:54:48 +07:00
jangko a2712c5c7a
eip2718: fixes test_rpc.nim 2021-05-15 18:09:36 +07:00
jangko a2f77e8627
eip2718: rename vm2 ChainId op code to ChainIdOp
this is to avoid clash with ChainId type
imported from eth/common
2021-05-15 18:09:36 +07:00
jangko 96d1969017
bump nim-eth: eip2718 typed transaction envelope
these series of commits labelled eip2718 will pave the way for
eip2930: optional access lists
2021-05-15 18:09:36 +07:00