Commit Graph

10 Commits

Author SHA1 Message Date
Jamie Lokier cf49b8d4fc
EVMC: Option `--evm`, load third-party EVM as a shared library
This patch adds:

- Load and use a third-party EVM in a shared library, instead of Nimbus EVM.
- New option `--evm` to specify which library to load.
- The library and this loader conforms to the [EVMC]
  (https://evmc.ethereum.org/) 9.x specification.

Any third-party EVM which is compatible with EVMC version 9.x and supports EVM1
contract code will be accepted.  The operating system's shared library format
applies.  These are `.so*` files on Linux, `.dll` files on Windows and `.dylib`
files on Mac.

The alternative EVM can be selected in two ways:

- Nimbus command line option `--evm:<path>`.
- Environment variable `NIMBUS_EVM=<path>`.

The reason for an environment variable is this allows all the test programs to
run with a third-party EVM as well.  Some don't parse command line options.

There are some limitations to be aware of:

- The third-party EVM must use EVMC version 9.x, no other major version.
  EVMC 9.x supports EIP-1559 / London fork and older transactions.

- Nested `*CALL` and `CREATE*` operations don't use the third-party EVM yet.
  These call the built-in Nimbus EVM.  This mixing of different EVMs between
  levels is explicitly allowed in specs, so there is no problem doing it.

- The third-party EVM doesn't need to support precompiles, because those are
  nested calls, which use the built-in Nimbus EVM.

- Third-party EVMs execute contracts correctly, but fail the final `rootHash`
  match.  The reason is that some account state changes, which are correct, are
  currently inside the Nimbus EVM and need to be moved to EVMC host logic.
  *This is a known work in progress*.  The EVM execution itself is fine.

Test results using "evmone" third-party EVM:

- [evmone](https://github.com/ethereum/evmone) has been tested.  Only on
  Linux but it "should" work on Windows and Mac equally well.

- [Version 0.8.1](https://github.com/ethereum/evmone/releases/tag/v0.8.1) was
  used because it is compatible with EVMC 9.x, which is required for the
  EIP-1559 / London fork, which Nimbus supports.  Version 0.8.0 could be used
  but it looks like an important bug was fixed in 0.8.1.

- evmone runs fine and the trace output looks good.  The calls and arguments
  are the same as the built-in Nimbus EVM for tests that have been checked
  manually, except evmone skips some calls that can be safely skipped.

- The final `rootHash` is incorrect, due to the *work in progress* mentioned
  above which is not part of the evmone execution.  Due to this, it's possible
  to try evmone and verify expected behaviours, which also validates our own
  EVMC implementation, but it can't be used as a full substitute yet.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-12-10 16:32:18 +00:00
Jamie Lokier 6ef9bfd21b
EVMC: Byte-endian conversions for 256-bit numeric values
Perform byte-endian conversion for 256-bit numeric values, but not 256-bit
hashes.  These conversions are necessary for EVMC binary compatibility.

In new EVMC, all host-side conversions are explicit, calling `flip256`.

These conversions are performed in the EVMC "glue" code, which deals with the
binary interface, so the host services aren't aware of conversions.

We intend to skip these conversions when Nimbus host calls Nimbus EVM, even
when it's a shared library, using a negotiated EVMC extension.  But for now
we're focused on correctness and cross-validation with third party EVMs.

The overhead of endian conversion is not too high because most EVMC host calls
access the database anyway.  `getTxContext` does not, so the conversions from
that are cached here.  Also, well-optimised EVMs don't call it often.

It is arguable whether endian conversion should occur for storage slots (`key`).

In favour of no conversion: Slot keys are 32-byte blobs, and this is clear in
the EVMC definition where slot keys are `evmc_bytes32` (not `evmc_uint256be`),
meaning treating as a number is _not_ expected by EVMC.  Although they are
often small numbers, sometimes they are a hash from the contract code plus a
number.  Slot keys are hashed on the host side with Keccak256 before any
database calls, so the host side does not look at them numerically.

In favour of conversion: They are often small numbers and it is helpful to log
them as such, rather than a long string of zero digits with 1-2 non-zero.  The
representation in JSON has leading zeros removed, like a number rather than a
32-byte blob.  There is also an interesting space optimisation when the keys
are used unhashed in storage.

Nimbus currently treats slot keys on the host side as numbers, and the tests
pass when endian conversion is done.  So to remain consistent with other parts
of Nimbus we convert slot keys.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-12-10 16:23:27 +00:00
Jamie Lokier 8fcd8354b1
EVMC: Use the same host interface for nested calls as top-level
Prior to this patch, top-level EVM executions and nested EVM executions did
their `getStorage` and other requests using a completely different set of host
functions.  It was just unfinished, to get top-level "new" EVMC working.

This finishes the job - it stops using the old methods.  Effect:

- Functionality added at the EVMC host level will be used by all EVM calls.
  (The target here is Beam Sync).

- The old set of functions are no longer used, so they can be removed.

- When EVMC host call tracing is enabled (`showTxCalls = true`), it traces
  the calls from nested EVM executions as well as top-level.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-08-17 17:18:26 +01:00
Jamie Lokier b783756ff3
EVMC: Make `hostInterface` a statically initialised global
We've been filling a "vtable"-like at run time, but it's not necessary.

The new object is a global `let x = evmc_host_interface(...)`, we assume it's
initialised before the first use, and we take its address with `.unsafeAddr`.

(If we use `ref evmc_host_interface`, Nim decides (correctly) that the
functions which use it aren't GC-safe because it's a global.)

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-08-17 17:18:26 +01:00
Jamie Lokier 3047c839dc
EVMC: Improve host call tracing and fix nested call C stack usage
This combines two things, a C stack usage change with EVM nested calls
via EVMC, and changes to host call tracing.

Feature-wise, the tracing is improved:

- Storage keys and values are make more sense.
- The message/result/context objects are shown with all relevant fields.
- `call` trace is split into entry/exit, so these can be shown around the
  called contract's operations, instead of only showing the `call` parameters
  after the nested call is finished.
- Nested calls are indented, which helps to highlight the flow.
- C stack usage considerably reduced in nested calls when more functionality
  is enabled (either tracing here, or other things to come).

This will seem like a minor patch, but C stack usage was the real motivation,
after plenty of time in the debugger.

Nobody cares about stack when `showTxCalls` (you can just use a big stack when
debugging).  But these subtle changes around the `call` path were found to be
necessary for passing all tests when the EVMC nested call code is completed,
and that's a prerequisite for many things: async EVM, dynamic EVM, Beam Sync,
and to fix https://github.com/status-im/nimbus-eth1/issues/345.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-08-12 07:48:56 +07:00
Jamie Lokier 74f53c7761
EVMC: Add missing EIP-2929 (Berlin) functions to EVMC host
The update for London (EIP-1559) in 1cdb30df ("bump nim-emvc with evmc revision
8.0.0 to 9.0.0") really bumped EVMC ABI version from 7.5 up to 9.

In other words, it skipped Berlin, going direct from Istanbul to London.

That was accompanied by EVMC changes in 05e9b891 ("EIP-3198: add baseFee op
code in nim-evm"), which added the API changes needed for London.

But the missing Berlin functions weren't added in the move to London.

As a result, our EVMC host became incompatible with Berlin, London, and really
all revisions of the ABI, and if a third party EVM was loaded, it crashed.

This commit adds the missing Berlin host support, and makes our ABI
binary-compatible with real EVMC again.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-08-11 19:47:34 +07:00
Jamie Lokier a5dc0bd283
EVMC: Using `{.show.}` trace all calls from the host into the EVM
Show calls from the host into the EVM.  Shows the call, `evmc_message` fields,
and `evmc_result` fields when the call returns.

(When `show_tx_calls` is manually set to true.)

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-08 18:29:40 +01:00
Jamie Lokier 0e2bc8408d
Transaction: Run all computations via EVMC `execute`
1. Send all EVM executions through the EVMC `execute` function.

   It leads to the same place in the end as calling `Computation` before, but
   `execute` is the API function used by all EVMC implementations, and it is
   very explicit what data is passed back and forth.

2. As a consequence this starts using the new `host_services` code from EVM, so
   this is a significant change to the paths used for account state processing.

3. Because we will have to remove the `newComputation` call on the host side,
   anticipating that the contract code is now saved in `host` instead of being
   copied around.  As it's saved in `host`, there is no need to pass it
   separately to `evmcExecComputation`.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-08 18:29:39 +01:00
Jamie Lokier e8a79e7246
EVMC: Reference EVMC entry point so it's linked into the program
Even though `evmc_create_nimbus_evm` is called, it fails at link time because
the definition of that function isn't included unless it is pulled in
explicitly.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-08 18:29:39 +01:00
Jamie Lokier 0b19f42158
EVMC: Binary compatibility glue on the host side
1. This provides the necessary type adjustments for host services to be
   (optionally) callable via the EVMC binary-compatible interface.  This layer
   is stashed away in a glue module so the host services continue to use
   appropriate Nim types, and are still callable directly.

   Inlining is used to ensure there should be no real overhead, including stack
   frame size for the `call` function.  Note, `import` must be used for
   `{.inline.}` to work effectively.

2. This also provides a key call in the other direction, the version of host to
   EVM `execute` that is called on the host side.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-08 18:29:39 +01:00