2021-05-20 09:15:00 +00:00
|
|
|
# Nimbus - Binary compatibility on the host side of the EVMC API interface
|
|
|
|
#
|
|
|
|
# Copyright (c) 2019-2021 Status Research & Development GmbH
|
|
|
|
# Licensed under either of
|
|
|
|
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
|
|
|
|
# * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
|
|
|
|
# at your option. This file may not be copied, modified, or distributed except according to those terms.
|
|
|
|
|
|
|
|
#{.push raises: [Defect].}
|
|
|
|
|
|
|
|
when not declaredInScope(included_from_host_services):
|
|
|
|
{.error: "Do not import this file directly, import host_services instead".}
|
|
|
|
|
EVMC: Option `--evm`, load third-party EVM as a shared library
This patch adds:
- Load and use a third-party EVM in a shared library, instead of Nimbus EVM.
- New option `--evm` to specify which library to load.
- The library and this loader conforms to the [EVMC]
(https://evmc.ethereum.org/) 9.x specification.
Any third-party EVM which is compatible with EVMC version 9.x and supports EVM1
contract code will be accepted. The operating system's shared library format
applies. These are `.so*` files on Linux, `.dll` files on Windows and `.dylib`
files on Mac.
The alternative EVM can be selected in two ways:
- Nimbus command line option `--evm:<path>`.
- Environment variable `NIMBUS_EVM=<path>`.
The reason for an environment variable is this allows all the test programs to
run with a third-party EVM as well. Some don't parse command line options.
There are some limitations to be aware of:
- The third-party EVM must use EVMC version 9.x, no other major version.
EVMC 9.x supports EIP-1559 / London fork and older transactions.
- Nested `*CALL` and `CREATE*` operations don't use the third-party EVM yet.
These call the built-in Nimbus EVM. This mixing of different EVMs between
levels is explicitly allowed in specs, so there is no problem doing it.
- The third-party EVM doesn't need to support precompiles, because those are
nested calls, which use the built-in Nimbus EVM.
- Third-party EVMs execute contracts correctly, but fail the final `rootHash`
match. The reason is that some account state changes, which are correct, are
currently inside the Nimbus EVM and need to be moved to EVMC host logic.
*This is a known work in progress*. The EVM execution itself is fine.
Test results using "evmone" third-party EVM:
- [evmone](https://github.com/ethereum/evmone) has been tested. Only on
Linux but it "should" work on Windows and Mac equally well.
- [Version 0.8.1](https://github.com/ethereum/evmone/releases/tag/v0.8.1) was
used because it is compatible with EVMC 9.x, which is required for the
EIP-1559 / London fork, which Nimbus supports. Version 0.8.0 could be used
but it looks like an important bug was fixed in 0.8.1.
- evmone runs fine and the trace output looks good. The calls and arguments
are the same as the built-in Nimbus EVM for tests that have been checked
manually, except evmone skips some calls that can be safely skipped.
- The final `rootHash` is incorrect, due to the *work in progress* mentioned
above which is not part of the evmone execution. Due to this, it's possible
to try evmone and verify expected behaviours, which also validates our own
EVMC implementation, but it can't be used as a full substitute yet.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-12-05 11:20:27 +00:00
|
|
|
import evmc/evmc, ./evmc_dynamic_loader
|
2021-05-20 09:15:00 +00:00
|
|
|
|
|
|
|
template toHost(p: evmc_host_context): TransactionHost =
|
|
|
|
cast[TransactionHost](p)
|
|
|
|
|
|
|
|
proc accountExists(p: evmc_host_context, address: var evmc_address): c99bool {.cdecl.} =
|
|
|
|
toHost(p).accountExists(address.fromEvmc)
|
|
|
|
|
|
|
|
proc getStorage(p: evmc_host_context, address: var evmc_address,
|
|
|
|
key: var evmc_bytes32): evmc_bytes32 {.cdecl.} =
|
EVMC: Byte-endian conversions for 256-bit numeric values
Perform byte-endian conversion for 256-bit numeric values, but not 256-bit
hashes. These conversions are necessary for EVMC binary compatibility.
In new EVMC, all host-side conversions are explicit, calling `flip256`.
These conversions are performed in the EVMC "glue" code, which deals with the
binary interface, so the host services aren't aware of conversions.
We intend to skip these conversions when Nimbus host calls Nimbus EVM, even
when it's a shared library, using a negotiated EVMC extension. But for now
we're focused on correctness and cross-validation with third party EVMs.
The overhead of endian conversion is not too high because most EVMC host calls
access the database anyway. `getTxContext` does not, so the conversions from
that are cached here. Also, well-optimised EVMs don't call it often.
It is arguable whether endian conversion should occur for storage slots (`key`).
In favour of no conversion: Slot keys are 32-byte blobs, and this is clear in
the EVMC definition where slot keys are `evmc_bytes32` (not `evmc_uint256be`),
meaning treating as a number is _not_ expected by EVMC. Although they are
often small numbers, sometimes they are a hash from the contract code plus a
number. Slot keys are hashed on the host side with Keccak256 before any
database calls, so the host side does not look at them numerically.
In favour of conversion: They are often small numbers and it is helpful to log
them as such, rather than a long string of zero digits with 1-2 non-zero. The
representation in JSON has leading zeros removed, like a number rather than a
32-byte blob. There is also an interesting space optimisation when the keys
are used unhashed in storage.
Nimbus currently treats slot keys on the host side as numbers, and the tests
pass when endian conversion is done. So to remain consistent with other parts
of Nimbus we convert slot keys.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-11-24 15:54:59 +00:00
|
|
|
toHost(p).getStorage(address.fromEvmc, key.flip256.fromEvmc).toEvmc.flip256
|
2021-05-20 09:15:00 +00:00
|
|
|
|
|
|
|
proc setStorage(p: evmc_host_context, address: var evmc_address,
|
|
|
|
key, value: var evmc_bytes32): evmc_storage_status {.cdecl.} =
|
EVMC: Byte-endian conversions for 256-bit numeric values
Perform byte-endian conversion for 256-bit numeric values, but not 256-bit
hashes. These conversions are necessary for EVMC binary compatibility.
In new EVMC, all host-side conversions are explicit, calling `flip256`.
These conversions are performed in the EVMC "glue" code, which deals with the
binary interface, so the host services aren't aware of conversions.
We intend to skip these conversions when Nimbus host calls Nimbus EVM, even
when it's a shared library, using a negotiated EVMC extension. But for now
we're focused on correctness and cross-validation with third party EVMs.
The overhead of endian conversion is not too high because most EVMC host calls
access the database anyway. `getTxContext` does not, so the conversions from
that are cached here. Also, well-optimised EVMs don't call it often.
It is arguable whether endian conversion should occur for storage slots (`key`).
In favour of no conversion: Slot keys are 32-byte blobs, and this is clear in
the EVMC definition where slot keys are `evmc_bytes32` (not `evmc_uint256be`),
meaning treating as a number is _not_ expected by EVMC. Although they are
often small numbers, sometimes they are a hash from the contract code plus a
number. Slot keys are hashed on the host side with Keccak256 before any
database calls, so the host side does not look at them numerically.
In favour of conversion: They are often small numbers and it is helpful to log
them as such, rather than a long string of zero digits with 1-2 non-zero. The
representation in JSON has leading zeros removed, like a number rather than a
32-byte blob. There is also an interesting space optimisation when the keys
are used unhashed in storage.
Nimbus currently treats slot keys on the host side as numbers, and the tests
pass when endian conversion is done. So to remain consistent with other parts
of Nimbus we convert slot keys.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-11-24 15:54:59 +00:00
|
|
|
toHost(p).setStorage(address.fromEvmc, key.flip256.fromEvmc, value.flip256.fromEvmc)
|
2021-05-20 09:15:00 +00:00
|
|
|
|
|
|
|
proc getBalance(p: evmc_host_context,
|
|
|
|
address: var evmc_address): evmc_uint256be {.cdecl.} =
|
EVMC: Byte-endian conversions for 256-bit numeric values
Perform byte-endian conversion for 256-bit numeric values, but not 256-bit
hashes. These conversions are necessary for EVMC binary compatibility.
In new EVMC, all host-side conversions are explicit, calling `flip256`.
These conversions are performed in the EVMC "glue" code, which deals with the
binary interface, so the host services aren't aware of conversions.
We intend to skip these conversions when Nimbus host calls Nimbus EVM, even
when it's a shared library, using a negotiated EVMC extension. But for now
we're focused on correctness and cross-validation with third party EVMs.
The overhead of endian conversion is not too high because most EVMC host calls
access the database anyway. `getTxContext` does not, so the conversions from
that are cached here. Also, well-optimised EVMs don't call it often.
It is arguable whether endian conversion should occur for storage slots (`key`).
In favour of no conversion: Slot keys are 32-byte blobs, and this is clear in
the EVMC definition where slot keys are `evmc_bytes32` (not `evmc_uint256be`),
meaning treating as a number is _not_ expected by EVMC. Although they are
often small numbers, sometimes they are a hash from the contract code plus a
number. Slot keys are hashed on the host side with Keccak256 before any
database calls, so the host side does not look at them numerically.
In favour of conversion: They are often small numbers and it is helpful to log
them as such, rather than a long string of zero digits with 1-2 non-zero. The
representation in JSON has leading zeros removed, like a number rather than a
32-byte blob. There is also an interesting space optimisation when the keys
are used unhashed in storage.
Nimbus currently treats slot keys on the host side as numbers, and the tests
pass when endian conversion is done. So to remain consistent with other parts
of Nimbus we convert slot keys.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-11-24 15:54:59 +00:00
|
|
|
toHost(p).getBalance(address.fromEvmc).toEvmc.flip256
|
2021-05-20 09:15:00 +00:00
|
|
|
|
|
|
|
proc getCodeSize(p: evmc_host_context,
|
|
|
|
address: var evmc_address): csize_t {.cdecl.} =
|
|
|
|
toHost(p).getCodeSize(address.fromEvmc)
|
|
|
|
|
|
|
|
proc getCodeHash(p: evmc_host_context,
|
|
|
|
address: var evmc_address): evmc_bytes32 {.cdecl.} =
|
|
|
|
toHost(p).getCodeHash(address.fromEvmc).toEvmc
|
|
|
|
|
|
|
|
proc copyCode(p: evmc_host_context, address: var evmc_address, code_offset: csize_t,
|
|
|
|
buffer_data: ptr byte, buffer_size: csize_t): csize_t {.cdecl.} =
|
|
|
|
toHost(p).copyCode(address.fromEvmc, code_offset, buffer_data, buffer_size)
|
|
|
|
|
|
|
|
proc selfDestruct(p: evmc_host_context, address,
|
|
|
|
beneficiary: var evmc_address) {.cdecl.} =
|
|
|
|
toHost(p).selfDestruct(address.fromEvmc, beneficiary.fromEvmc)
|
|
|
|
|
|
|
|
proc call(p: evmc_host_context, msg: var evmc_message): evmc_result {.cdecl.} =
|
EVMC: Byte-endian conversions for 256-bit numeric values
Perform byte-endian conversion for 256-bit numeric values, but not 256-bit
hashes. These conversions are necessary for EVMC binary compatibility.
In new EVMC, all host-side conversions are explicit, calling `flip256`.
These conversions are performed in the EVMC "glue" code, which deals with the
binary interface, so the host services aren't aware of conversions.
We intend to skip these conversions when Nimbus host calls Nimbus EVM, even
when it's a shared library, using a negotiated EVMC extension. But for now
we're focused on correctness and cross-validation with third party EVMs.
The overhead of endian conversion is not too high because most EVMC host calls
access the database anyway. `getTxContext` does not, so the conversions from
that are cached here. Also, well-optimised EVMs don't call it often.
It is arguable whether endian conversion should occur for storage slots (`key`).
In favour of no conversion: Slot keys are 32-byte blobs, and this is clear in
the EVMC definition where slot keys are `evmc_bytes32` (not `evmc_uint256be`),
meaning treating as a number is _not_ expected by EVMC. Although they are
often small numbers, sometimes they are a hash from the contract code plus a
number. Slot keys are hashed on the host side with Keccak256 before any
database calls, so the host side does not look at them numerically.
In favour of conversion: They are often small numbers and it is helpful to log
them as such, rather than a long string of zero digits with 1-2 non-zero. The
representation in JSON has leading zeros removed, like a number rather than a
32-byte blob. There is also an interesting space optimisation when the keys
are used unhashed in storage.
Nimbus currently treats slot keys on the host side as numbers, and the tests
pass when endian conversion is done. So to remain consistent with other parts
of Nimbus we convert slot keys.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-11-24 15:54:59 +00:00
|
|
|
# This would contain `flip256`, but `call` is special. The C stack usage
|
|
|
|
# must be kept small for deeply nested EVM calls. To ensure small stack,
|
|
|
|
# `flip256` must be handled at `host_call_nested`, not here.
|
2021-05-20 09:15:00 +00:00
|
|
|
toHost(p).call(msg)
|
|
|
|
|
|
|
|
proc getTxContext(p: evmc_host_context): evmc_tx_context {.cdecl.} =
|
EVMC: Byte-endian conversions for 256-bit numeric values
Perform byte-endian conversion for 256-bit numeric values, but not 256-bit
hashes. These conversions are necessary for EVMC binary compatibility.
In new EVMC, all host-side conversions are explicit, calling `flip256`.
These conversions are performed in the EVMC "glue" code, which deals with the
binary interface, so the host services aren't aware of conversions.
We intend to skip these conversions when Nimbus host calls Nimbus EVM, even
when it's a shared library, using a negotiated EVMC extension. But for now
we're focused on correctness and cross-validation with third party EVMs.
The overhead of endian conversion is not too high because most EVMC host calls
access the database anyway. `getTxContext` does not, so the conversions from
that are cached here. Also, well-optimised EVMs don't call it often.
It is arguable whether endian conversion should occur for storage slots (`key`).
In favour of no conversion: Slot keys are 32-byte blobs, and this is clear in
the EVMC definition where slot keys are `evmc_bytes32` (not `evmc_uint256be`),
meaning treating as a number is _not_ expected by EVMC. Although they are
often small numbers, sometimes they are a hash from the contract code plus a
number. Slot keys are hashed on the host side with Keccak256 before any
database calls, so the host side does not look at them numerically.
In favour of conversion: They are often small numbers and it is helpful to log
them as such, rather than a long string of zero digits with 1-2 non-zero. The
representation in JSON has leading zeros removed, like a number rather than a
32-byte blob. There is also an interesting space optimisation when the keys
are used unhashed in storage.
Nimbus currently treats slot keys on the host side as numbers, and the tests
pass when endian conversion is done. So to remain consistent with other parts
of Nimbus we convert slot keys.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-11-24 15:54:59 +00:00
|
|
|
# This would contain `flip256`, but due to this result being cached in
|
|
|
|
# `getTxContext`, it's better to do `flip256` when filling the cache.
|
2021-05-20 09:15:00 +00:00
|
|
|
toHost(p).getTxContext()
|
|
|
|
|
|
|
|
proc getBlockHash(p: evmc_host_context, number: int64): evmc_bytes32 {.cdecl.} =
|
|
|
|
# TODO: `HostBlockNumber` is 256-bit unsigned. It should be changed to match
|
|
|
|
# EVMC which is more sensible.
|
|
|
|
toHost(p).getBlockHash(number.uint64.u256).toEvmc
|
|
|
|
|
|
|
|
proc emitLog(p: evmc_host_context, address: var evmc_address,
|
|
|
|
data: ptr byte, data_size: csize_t,
|
|
|
|
topics: ptr evmc_bytes32, topics_count: csize_t) {.cdecl.} =
|
|
|
|
toHost(p).emitLog(address.fromEvmc, data, data_size,
|
|
|
|
cast[ptr HostTopic](topics), topics_count)
|
|
|
|
|
2021-08-05 01:52:40 +00:00
|
|
|
proc accessAccount(p: evmc_host_context,
|
|
|
|
address: var evmc_address): evmc_access_status {.cdecl.} =
|
|
|
|
toHost(p).accessAccount(address.fromEvmc)
|
|
|
|
|
|
|
|
proc accessStorage(p: evmc_host_context, address: var evmc_address,
|
|
|
|
key: var evmc_bytes32): evmc_access_status {.cdecl.} =
|
EVMC: Byte-endian conversions for 256-bit numeric values
Perform byte-endian conversion for 256-bit numeric values, but not 256-bit
hashes. These conversions are necessary for EVMC binary compatibility.
In new EVMC, all host-side conversions are explicit, calling `flip256`.
These conversions are performed in the EVMC "glue" code, which deals with the
binary interface, so the host services aren't aware of conversions.
We intend to skip these conversions when Nimbus host calls Nimbus EVM, even
when it's a shared library, using a negotiated EVMC extension. But for now
we're focused on correctness and cross-validation with third party EVMs.
The overhead of endian conversion is not too high because most EVMC host calls
access the database anyway. `getTxContext` does not, so the conversions from
that are cached here. Also, well-optimised EVMs don't call it often.
It is arguable whether endian conversion should occur for storage slots (`key`).
In favour of no conversion: Slot keys are 32-byte blobs, and this is clear in
the EVMC definition where slot keys are `evmc_bytes32` (not `evmc_uint256be`),
meaning treating as a number is _not_ expected by EVMC. Although they are
often small numbers, sometimes they are a hash from the contract code plus a
number. Slot keys are hashed on the host side with Keccak256 before any
database calls, so the host side does not look at them numerically.
In favour of conversion: They are often small numbers and it is helpful to log
them as such, rather than a long string of zero digits with 1-2 non-zero. The
representation in JSON has leading zeros removed, like a number rather than a
32-byte blob. There is also an interesting space optimisation when the keys
are used unhashed in storage.
Nimbus currently treats slot keys on the host side as numbers, and the tests
pass when endian conversion is done. So to remain consistent with other parts
of Nimbus we convert slot keys.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-11-24 15:54:59 +00:00
|
|
|
toHost(p).accessStorage(address.fromEvmc, key.flip256.fromEvmc)
|
2021-08-05 01:52:40 +00:00
|
|
|
|
2021-08-09 16:38:59 +00:00
|
|
|
let hostInterface = evmc_host_interface(
|
|
|
|
account_exists: accountExists,
|
|
|
|
get_storage: getStorage,
|
|
|
|
set_storage: setStorage,
|
|
|
|
get_balance: getBalance,
|
|
|
|
get_code_size: getCodeSize,
|
|
|
|
get_code_hash: getCodeHash,
|
|
|
|
copy_code: copyCode,
|
|
|
|
selfdestruct: selfDestruct,
|
|
|
|
call: call,
|
|
|
|
get_tx_context: getTxContext,
|
|
|
|
get_block_hash: getBlockHash,
|
|
|
|
emit_log: emitLog,
|
|
|
|
access_account: accessAccount,
|
|
|
|
access_storage: accessStorage,
|
|
|
|
)
|
2021-05-20 09:15:00 +00:00
|
|
|
|
EVMC: Improve host call tracing and fix nested call C stack usage
This combines two things, a C stack usage change with EVM nested calls
via EVMC, and changes to host call tracing.
Feature-wise, the tracing is improved:
- Storage keys and values are make more sense.
- The message/result/context objects are shown with all relevant fields.
- `call` trace is split into entry/exit, so these can be shown around the
called contract's operations, instead of only showing the `call` parameters
after the nested call is finished.
- Nested calls are indented, which helps to highlight the flow.
- C stack usage considerably reduced in nested calls when more functionality
is enabled (either tracing here, or other things to come).
This will seem like a minor patch, but C stack usage was the real motivation,
after plenty of time in the debugger.
Nobody cares about stack when `showTxCalls` (you can just use a big stack when
debugging). But these subtle changes around the `call` path were found to be
necessary for passing all tests when the EVMC nested call code is completed,
and that's a prerequisite for many things: async EVM, dynamic EVM, Beam Sync,
and to fix https://github.com/status-im/nimbus-eth1/issues/345.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-08-09 14:54:38 +00:00
|
|
|
proc evmcExecComputation*(host: TransactionHost): EvmcResult {.inline.} =
|
|
|
|
host.showCallEntry(host.msg)
|
|
|
|
|
EVMC: Option `--evm`, load third-party EVM as a shared library
This patch adds:
- Load and use a third-party EVM in a shared library, instead of Nimbus EVM.
- New option `--evm` to specify which library to load.
- The library and this loader conforms to the [EVMC]
(https://evmc.ethereum.org/) 9.x specification.
Any third-party EVM which is compatible with EVMC version 9.x and supports EVM1
contract code will be accepted. The operating system's shared library format
applies. These are `.so*` files on Linux, `.dll` files on Windows and `.dylib`
files on Mac.
The alternative EVM can be selected in two ways:
- Nimbus command line option `--evm:<path>`.
- Environment variable `NIMBUS_EVM=<path>`.
The reason for an environment variable is this allows all the test programs to
run with a third-party EVM as well. Some don't parse command line options.
There are some limitations to be aware of:
- The third-party EVM must use EVMC version 9.x, no other major version.
EVMC 9.x supports EIP-1559 / London fork and older transactions.
- Nested `*CALL` and `CREATE*` operations don't use the third-party EVM yet.
These call the built-in Nimbus EVM. This mixing of different EVMs between
levels is explicitly allowed in specs, so there is no problem doing it.
- The third-party EVM doesn't need to support precompiles, because those are
nested calls, which use the built-in Nimbus EVM.
- Third-party EVMs execute contracts correctly, but fail the final `rootHash`
match. The reason is that some account state changes, which are correct, are
currently inside the Nimbus EVM and need to be moved to EVMC host logic.
*This is a known work in progress*. The EVM execution itself is fine.
Test results using "evmone" third-party EVM:
- [evmone](https://github.com/ethereum/evmone) has been tested. Only on
Linux but it "should" work on Windows and Mac equally well.
- [Version 0.8.1](https://github.com/ethereum/evmone/releases/tag/v0.8.1) was
used because it is compatible with EVMC 9.x, which is required for the
EIP-1559 / London fork, which Nimbus supports. Version 0.8.0 could be used
but it looks like an important bug was fixed in 0.8.1.
- evmone runs fine and the trace output looks good. The calls and arguments
are the same as the built-in Nimbus EVM for tests that have been checked
manually, except evmone skips some calls that can be safely skipped.
- The final `rootHash` is incorrect, due to the *work in progress* mentioned
above which is not part of the evmone execution. Due to this, it's possible
to try evmone and verify expected behaviours, which also validates our own
EVMC implementation, but it can't be used as a full substitute yet.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-12-05 11:20:27 +00:00
|
|
|
let vm = evmcLoadVMCached()
|
2021-05-20 09:15:00 +00:00
|
|
|
if vm.isNil:
|
EVMC: Option `--evm`, load third-party EVM as a shared library
This patch adds:
- Load and use a third-party EVM in a shared library, instead of Nimbus EVM.
- New option `--evm` to specify which library to load.
- The library and this loader conforms to the [EVMC]
(https://evmc.ethereum.org/) 9.x specification.
Any third-party EVM which is compatible with EVMC version 9.x and supports EVM1
contract code will be accepted. The operating system's shared library format
applies. These are `.so*` files on Linux, `.dll` files on Windows and `.dylib`
files on Mac.
The alternative EVM can be selected in two ways:
- Nimbus command line option `--evm:<path>`.
- Environment variable `NIMBUS_EVM=<path>`.
The reason for an environment variable is this allows all the test programs to
run with a third-party EVM as well. Some don't parse command line options.
There are some limitations to be aware of:
- The third-party EVM must use EVMC version 9.x, no other major version.
EVMC 9.x supports EIP-1559 / London fork and older transactions.
- Nested `*CALL` and `CREATE*` operations don't use the third-party EVM yet.
These call the built-in Nimbus EVM. This mixing of different EVMs between
levels is explicitly allowed in specs, so there is no problem doing it.
- The third-party EVM doesn't need to support precompiles, because those are
nested calls, which use the built-in Nimbus EVM.
- Third-party EVMs execute contracts correctly, but fail the final `rootHash`
match. The reason is that some account state changes, which are correct, are
currently inside the Nimbus EVM and need to be moved to EVMC host logic.
*This is a known work in progress*. The EVM execution itself is fine.
Test results using "evmone" third-party EVM:
- [evmone](https://github.com/ethereum/evmone) has been tested. Only on
Linux but it "should" work on Windows and Mac equally well.
- [Version 0.8.1](https://github.com/ethereum/evmone/releases/tag/v0.8.1) was
used because it is compatible with EVMC 9.x, which is required for the
EIP-1559 / London fork, which Nimbus supports. Version 0.8.0 could be used
but it looks like an important bug was fixed in 0.8.1.
- evmone runs fine and the trace output looks good. The calls and arguments
are the same as the built-in Nimbus EVM for tests that have been checked
manually, except evmone skips some calls that can be safely skipped.
- The final `rootHash` is incorrect, due to the *work in progress* mentioned
above which is not part of the evmone execution. Due to this, it's possible
to try evmone and verify expected behaviours, which also validates our own
EVMC implementation, but it can't be used as a full substitute yet.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-12-05 11:20:27 +00:00
|
|
|
warn "No EVM"
|
2021-05-20 09:15:00 +00:00
|
|
|
# Nim defaults are fine for all other fields in the result object.
|
EVMC: Improve host call tracing and fix nested call C stack usage
This combines two things, a C stack usage change with EVM nested calls
via EVMC, and changes to host call tracing.
Feature-wise, the tracing is improved:
- Storage keys and values are make more sense.
- The message/result/context objects are shown with all relevant fields.
- `call` trace is split into entry/exit, so these can be shown around the
called contract's operations, instead of only showing the `call` parameters
after the nested call is finished.
- Nested calls are indented, which helps to highlight the flow.
- C stack usage considerably reduced in nested calls when more functionality
is enabled (either tracing here, or other things to come).
This will seem like a minor patch, but C stack usage was the real motivation,
after plenty of time in the debugger.
Nobody cares about stack when `showTxCalls` (you can just use a big stack when
debugging). But these subtle changes around the `call` path were found to be
necessary for passing all tests when the EVMC nested call code is completed,
and that's a prerequisite for many things: async EVM, dynamic EVM, Beam Sync,
and to fix https://github.com/status-im/nimbus-eth1/issues/345.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-08-09 14:54:38 +00:00
|
|
|
result = EvmcResult(status_code: EVMC_INTERNAL_ERROR)
|
|
|
|
host.showCallReturn(result)
|
|
|
|
return
|
2021-05-20 09:15:00 +00:00
|
|
|
|
|
|
|
let hostContext = cast[evmc_host_context](host)
|
2021-08-17 16:18:07 +00:00
|
|
|
host.hostInterface = hostInterface.unsafeAddr
|
2021-05-20 09:15:00 +00:00
|
|
|
|
|
|
|
# Without `{.gcsafe.}:` here, the call via `vm.execute` results in a Nim
|
|
|
|
# compile-time error in a far away function. Starting here, a cascade of
|
|
|
|
# warnings takes place: "Warning: '...' is not GC-safe as it performs an
|
|
|
|
# indirect call here [GCUnsafe2]", then a list of "Warning: '...' is not
|
|
|
|
# GC-safe as it calls '...'" at each function up the call stack, to a high
|
|
|
|
# level function `persistBlocks` where it terminates compilation as an error
|
|
|
|
# instead of a warning.
|
|
|
|
#
|
|
|
|
# It is tempting to annotate all EVMC API functions with `{.cdecl, gcsafe.}`,
|
|
|
|
# overriding the function signatures from the Nim EVMC module. Perhaps we
|
|
|
|
# will do that, though it's conceptually dubious, as the two sides of the
|
|
|
|
# EVMC ABI live in different GC worlds (when loaded as a shared library with
|
|
|
|
# its own Nim runtime), very similar to calling between threads.
|
|
|
|
#
|
|
|
|
# TODO: But wait: Why does the Nim EVMC test program compile fine without
|
|
|
|
# any `gcsafe`, even with `--threads:on`?
|
|
|
|
{.gcsafe.}:
|
2021-08-09 16:38:59 +00:00
|
|
|
result = vm.execute(vm, hostInterface.unsafeAddr, hostContext,
|
EVMC: Improve host call tracing and fix nested call C stack usage
This combines two things, a C stack usage change with EVM nested calls
via EVMC, and changes to host call tracing.
Feature-wise, the tracing is improved:
- Storage keys and values are make more sense.
- The message/result/context objects are shown with all relevant fields.
- `call` trace is split into entry/exit, so these can be shown around the
called contract's operations, instead of only showing the `call` parameters
after the nested call is finished.
- Nested calls are indented, which helps to highlight the flow.
- C stack usage considerably reduced in nested calls when more functionality
is enabled (either tracing here, or other things to come).
This will seem like a minor patch, but C stack usage was the real motivation,
after plenty of time in the debugger.
Nobody cares about stack when `showTxCalls` (you can just use a big stack when
debugging). But these subtle changes around the `call` path were found to be
necessary for passing all tests when the EVMC nested call code is completed,
and that's a prerequisite for many things: async EVM, dynamic EVM, Beam Sync,
and to fix https://github.com/status-im/nimbus-eth1/issues/345.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-08-09 14:54:38 +00:00
|
|
|
evmc_revision(host.vmState.fork), host.msg,
|
|
|
|
if host.code.len > 0: host.code[0].unsafeAddr else: nil,
|
|
|
|
host.code.len.csize_t)
|
2021-05-24 17:00:38 +00:00
|
|
|
|
EVMC: Improve host call tracing and fix nested call C stack usage
This combines two things, a C stack usage change with EVM nested calls
via EVMC, and changes to host call tracing.
Feature-wise, the tracing is improved:
- Storage keys and values are make more sense.
- The message/result/context objects are shown with all relevant fields.
- `call` trace is split into entry/exit, so these can be shown around the
called contract's operations, instead of only showing the `call` parameters
after the nested call is finished.
- Nested calls are indented, which helps to highlight the flow.
- C stack usage considerably reduced in nested calls when more functionality
is enabled (either tracing here, or other things to come).
This will seem like a minor patch, but C stack usage was the real motivation,
after plenty of time in the debugger.
Nobody cares about stack when `showTxCalls` (you can just use a big stack when
debugging). But these subtle changes around the `call` path were found to be
necessary for passing all tests when the EVMC nested call code is completed,
and that's a prerequisite for many things: async EVM, dynamic EVM, Beam Sync,
and to fix https://github.com/status-im/nimbus-eth1/issues/345.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-08-09 14:54:38 +00:00
|
|
|
host.showCallReturn(result)
|
EVMC: Option `--evm`, load third-party EVM as a shared library
This patch adds:
- Load and use a third-party EVM in a shared library, instead of Nimbus EVM.
- New option `--evm` to specify which library to load.
- The library and this loader conforms to the [EVMC]
(https://evmc.ethereum.org/) 9.x specification.
Any third-party EVM which is compatible with EVMC version 9.x and supports EVM1
contract code will be accepted. The operating system's shared library format
applies. These are `.so*` files on Linux, `.dll` files on Windows and `.dylib`
files on Mac.
The alternative EVM can be selected in two ways:
- Nimbus command line option `--evm:<path>`.
- Environment variable `NIMBUS_EVM=<path>`.
The reason for an environment variable is this allows all the test programs to
run with a third-party EVM as well. Some don't parse command line options.
There are some limitations to be aware of:
- The third-party EVM must use EVMC version 9.x, no other major version.
EVMC 9.x supports EIP-1559 / London fork and older transactions.
- Nested `*CALL` and `CREATE*` operations don't use the third-party EVM yet.
These call the built-in Nimbus EVM. This mixing of different EVMs between
levels is explicitly allowed in specs, so there is no problem doing it.
- The third-party EVM doesn't need to support precompiles, because those are
nested calls, which use the built-in Nimbus EVM.
- Third-party EVMs execute contracts correctly, but fail the final `rootHash`
match. The reason is that some account state changes, which are correct, are
currently inside the Nimbus EVM and need to be moved to EVMC host logic.
*This is a known work in progress*. The EVM execution itself is fine.
Test results using "evmone" third-party EVM:
- [evmone](https://github.com/ethereum/evmone) has been tested. Only on
Linux but it "should" work on Windows and Mac equally well.
- [Version 0.8.1](https://github.com/ethereum/evmone/releases/tag/v0.8.1) was
used because it is compatible with EVMC 9.x, which is required for the
EIP-1559 / London fork, which Nimbus supports. Version 0.8.0 could be used
but it looks like an important bug was fixed in 0.8.1.
- evmone runs fine and the trace output looks good. The calls and arguments
are the same as the built-in Nimbus EVM for tests that have been checked
manually, except evmone skips some calls that can be safely skipped.
- The final `rootHash` is incorrect, due to the *work in progress* mentioned
above which is not part of the evmone execution. Due to this, it's possible
to try evmone and verify expected behaviours, which also validates our own
EVMC implementation, but it can't be used as a full substitute yet.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-12-05 11:20:27 +00:00
|
|
|
|
|
|
|
# This code assumes fields, methods and types of ABI version 9, and must be
|
|
|
|
# checked for compatibility if the `import evmc/evmc` major version is updated.
|
|
|
|
when EVMC_ABI_VERSION != 9:
|
|
|
|
{.error: ("This code assumes EVMC_ABI_VERSION 9;" &
|
|
|
|
" update the code to use EVMC_ABI_VERSION " & $EVMC_ABI_VERSION).}
|