2021-05-24 08:53:53 +00:00
|
|
|
# Nimbus - Services available to EVM code that is run for a transaction
|
|
|
|
#
|
|
|
|
# Copyright (c) 2019-2021 Status Research & Development GmbH
|
|
|
|
# Licensed under either of
|
|
|
|
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
|
|
|
|
# * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
|
|
|
|
# at your option. This file may not be copied, modified, or distributed except according to those terms.
|
|
|
|
|
|
|
|
#{.push raises: [Defect].}
|
|
|
|
|
|
|
|
import
|
|
|
|
sets, stint, chronicles, stew/ranges/ptr_arith,
|
|
|
|
eth/common/eth_types,
|
Transaction: EVMC fix, `CREATE2` salt is a 256-bit blob not a number
This changes fixes a bug in `CREATE2` ops when used with EVMC.
Because it changes the salt type, it affects non-EVMC code as well.
The salt was passed through EVMC with the wrong byte order, although this went
unnoticed as the Nimbus host flipped the byte order before using it.
This was found when running Nimbus with third-party EVM,
["evmone"](https://github.com/ethereum/evmone).
There are different ways to remedy this.
If treated as a number, Nimbus EVM would byte-flip the value when calling EVMC,
then Nimbus host would flip the received value. Finally, it would be flipped a
third time when generating the address in `generateSafeAddress`. The first two
flips can be eliminated by negotiation (like other numbers), but there would
always be one flip.
As a bit pattern, Nimbus EVM would flip the same way it does when dealing with
hashes on the stack (e.g. with `getBlockHash`). Nimbus host wouldn't flip at
all - and when using third-party EVMs there would be no flips in Nimbus.
Because this value is not for arithmetic, any bit pattern is valid, and there
shouldn't be any flips when using a third-party EVM, the bit-pattern
interpretation is favoured. The only flip is done in Nimbus EVM (and might be
eliminated in an optimised version).
As suggested, we'll define a new "opaque 256 bits" type to hold this value.
(Similar to `Hash256`, but the salt isn't necessarily a hash.)
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-09 22:32:42 +00:00
|
|
|
".."/[vm_types, vm_computation, utils],
|
EVMC: Improve host call tracing and fix nested call C stack usage
This combines two things, a C stack usage change with EVM nested calls
via EVMC, and changes to host call tracing.
Feature-wise, the tracing is improved:
- Storage keys and values are make more sense.
- The message/result/context objects are shown with all relevant fields.
- `call` trace is split into entry/exit, so these can be shown around the
called contract's operations, instead of only showing the `call` parameters
after the nested call is finished.
- Nested calls are indented, which helps to highlight the flow.
- C stack usage considerably reduced in nested calls when more functionality
is enabled (either tracing here, or other things to come).
This will seem like a minor patch, but C stack usage was the real motivation,
after plenty of time in the debugger.
Nobody cares about stack when `showTxCalls` (you can just use a big stack when
debugging). But these subtle changes around the `call` path were found to be
necessary for passing all tests when the EVMC nested call code is completed,
and that's a prerequisite for many things: async EVM, dynamic EVM, Beam Sync,
and to fix https://github.com/status-im/nimbus-eth1/issues/345.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-08-09 14:54:38 +00:00
|
|
|
./host_types, ./host_trace
|
2021-05-24 08:53:53 +00:00
|
|
|
|
|
|
|
proc evmcResultRelease(res: var EvmcResult) {.cdecl, gcsafe.} =
|
|
|
|
dealloc(res.output_data)
|
|
|
|
|
|
|
|
proc beforeExecCreateEvmcNested(host: TransactionHost,
|
|
|
|
m: EvmcMessage): Computation {.inline.} =
|
|
|
|
# TODO: use evmc_message to avoid copy
|
|
|
|
let childMsg = Message(
|
|
|
|
kind: CallKind(m.kind),
|
|
|
|
depth: m.depth,
|
|
|
|
gas: m.gas,
|
|
|
|
sender: m.sender.fromEvmc,
|
|
|
|
value: m.value.fromEvmc,
|
|
|
|
data: @(makeOpenArray(m.inputData, m.inputSize.int))
|
|
|
|
)
|
Transaction: EVMC fix, `CREATE2` salt is a 256-bit blob not a number
This changes fixes a bug in `CREATE2` ops when used with EVMC.
Because it changes the salt type, it affects non-EVMC code as well.
The salt was passed through EVMC with the wrong byte order, although this went
unnoticed as the Nimbus host flipped the byte order before using it.
This was found when running Nimbus with third-party EVM,
["evmone"](https://github.com/ethereum/evmone).
There are different ways to remedy this.
If treated as a number, Nimbus EVM would byte-flip the value when calling EVMC,
then Nimbus host would flip the received value. Finally, it would be flipped a
third time when generating the address in `generateSafeAddress`. The first two
flips can be eliminated by negotiation (like other numbers), but there would
always be one flip.
As a bit pattern, Nimbus EVM would flip the same way it does when dealing with
hashes on the stack (e.g. with `getBlockHash`). Nimbus host wouldn't flip at
all - and when using third-party EVMs there would be no flips in Nimbus.
Because this value is not for arithmetic, any bit pattern is valid, and there
shouldn't be any flips when using a third-party EVM, the bit-pattern
interpretation is favoured. The only flip is done in Nimbus EVM (and might be
eliminated in an optimised version).
As suggested, we'll define a new "opaque 256 bits" type to hold this value.
(Similar to `Hash256`, but the salt isn't necessarily a hash.)
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-09 22:32:42 +00:00
|
|
|
return newComputation(host.vmState, childMsg,
|
|
|
|
cast[ContractSalt](m.create2_salt))
|
2021-05-24 08:53:53 +00:00
|
|
|
|
|
|
|
proc afterExecCreateEvmcNested(host: TransactionHost, child: Computation,
|
|
|
|
res: var EvmcResult) {.inline.} =
|
|
|
|
if not child.shouldBurnGas:
|
|
|
|
res.gas_left = child.gasMeter.gasRemaining
|
|
|
|
|
|
|
|
if child.isSuccess:
|
|
|
|
host.computation.merge(child)
|
|
|
|
res.status_code = EVMC_SUCCESS
|
|
|
|
res.create_address = child.msg.contractAddress.toEvmc
|
|
|
|
else:
|
|
|
|
res.status_code = if child.shouldBurnGas: EVMC_FAILURE else: EVMC_REVERT
|
|
|
|
if child.output.len > 0:
|
|
|
|
# TODO: can we move the ownership of seq to raw pointer?
|
|
|
|
res.output_size = child.output.len.uint
|
|
|
|
res.output_data = cast[ptr byte](alloc(child.output.len))
|
|
|
|
copyMem(res.output_data, child.output[0].addr, child.output.len)
|
|
|
|
res.release = evmcResultRelease
|
|
|
|
|
|
|
|
proc beforeExecCallEvmcNested(host: TransactionHost,
|
|
|
|
m: EvmcMessage): Computation {.inline.} =
|
|
|
|
let childMsg = Message(
|
|
|
|
kind: CallKind(m.kind),
|
|
|
|
depth: m.depth,
|
|
|
|
gas: m.gas,
|
|
|
|
sender: m.sender.fromEvmc,
|
|
|
|
codeAddress: m.destination.fromEvmc,
|
|
|
|
contractAddress: if m.kind == EVMC_CALL:
|
|
|
|
m.destination.fromEvmc
|
|
|
|
else:
|
|
|
|
host.computation.msg.contractAddress,
|
|
|
|
value: m.value.fromEvmc,
|
|
|
|
data: @(makeOpenArray(m.inputData, m.inputSize.int)),
|
|
|
|
flags: if m.isStatic: emvcStatic else: emvcNoFlags,
|
|
|
|
)
|
|
|
|
return newComputation(host.vmState, childMsg)
|
|
|
|
|
|
|
|
proc afterExecCallEvmcNested(host: TransactionHost, child: Computation,
|
|
|
|
res: var EvmcResult) {.inline.} =
|
|
|
|
if not child.shouldBurnGas:
|
|
|
|
res.gas_left = child.gasMeter.gasRemaining
|
|
|
|
|
|
|
|
if child.isSuccess:
|
|
|
|
host.computation.merge(child)
|
|
|
|
res.status_code = EVMC_SUCCESS
|
|
|
|
else:
|
|
|
|
res.status_code = if child.shouldBurnGas: EVMC_FAILURE else: EVMC_REVERT
|
|
|
|
|
|
|
|
if child.output.len > 0:
|
|
|
|
# TODO: can we move the ownership of seq to raw pointer?
|
|
|
|
res.output_size = child.output.len.uint
|
|
|
|
res.output_data = cast[ptr byte](alloc(child.output.len))
|
|
|
|
copyMem(res.output_data, child.output[0].addr, child.output.len)
|
|
|
|
res.release = evmcResultRelease
|
|
|
|
|
|
|
|
# The next three functions are designed so `callEvmcNested` uses very small C
|
|
|
|
# stack usage for each level of nested EVM calls.
|
|
|
|
#
|
|
|
|
# To keep the C stack usage small when there are deeply nested EVM calls,
|
|
|
|
# `callEvmcNested` must use as little stack as possible, going from the EVM
|
|
|
|
# which calls it to the nested EVM which it calls.
|
|
|
|
#
|
|
|
|
# First, `callEvmcNested` itself is `template` so it is inlined to the caller
|
|
|
|
# at Nim level, not C level. Only at Nim level is inlining guaranteed across
|
|
|
|
# `import`. This saves a C stack frame, which matters because some C compilers
|
|
|
|
# reserve space for 1-3 copies of the large `EvmcResult` return value.
|
|
|
|
#
|
|
|
|
# Second, the complicated parts of preparation and return are done in
|
|
|
|
# out-of-line functions `beforeExecEvmcNested` and `afterExecEvmcNested`. They
|
|
|
|
# are annotated with `{.noinline.}` to make sure they are out-of-line. The
|
|
|
|
# annotation ensures they don't contribute to the stack frame of
|
|
|
|
# `callEvmcNested`, because otherwise the compiler can optimistically inline.
|
|
|
|
# (Even across modules when using `-flto`).
|
|
|
|
#
|
|
|
|
# The functions `beforeExecEvmcNested` and `afterExecEvmcNested` can use as
|
|
|
|
# much stack as they like.
|
|
|
|
|
|
|
|
proc beforeExecEvmcNested(host: TransactionHost, msg: EvmcMessage): Computation
|
|
|
|
# This function must be declared with `{.noinline.}` to make sure it doesn't
|
|
|
|
# contribute to the stack frame of `callEvmcNested` below.
|
|
|
|
{.noinline.} =
|
EVMC: Byte-endian conversions for 256-bit numeric values
Perform byte-endian conversion for 256-bit numeric values, but not 256-bit
hashes. These conversions are necessary for EVMC binary compatibility.
In new EVMC, all host-side conversions are explicit, calling `flip256`.
These conversions are performed in the EVMC "glue" code, which deals with the
binary interface, so the host services aren't aware of conversions.
We intend to skip these conversions when Nimbus host calls Nimbus EVM, even
when it's a shared library, using a negotiated EVMC extension. But for now
we're focused on correctness and cross-validation with third party EVMs.
The overhead of endian conversion is not too high because most EVMC host calls
access the database anyway. `getTxContext` does not, so the conversions from
that are cached here. Also, well-optimised EVMs don't call it often.
It is arguable whether endian conversion should occur for storage slots (`key`).
In favour of no conversion: Slot keys are 32-byte blobs, and this is clear in
the EVMC definition where slot keys are `evmc_bytes32` (not `evmc_uint256be`),
meaning treating as a number is _not_ expected by EVMC. Although they are
often small numbers, sometimes they are a hash from the contract code plus a
number. Slot keys are hashed on the host side with Keccak256 before any
database calls, so the host side does not look at them numerically.
In favour of conversion: They are often small numbers and it is helpful to log
them as such, rather than a long string of zero digits with 1-2 non-zero. The
representation in JSON has leading zeros removed, like a number rather than a
32-byte blob. There is also an interesting space optimisation when the keys
are used unhashed in storage.
Nimbus currently treats slot keys on the host side as numbers, and the tests
pass when endian conversion is done. So to remain consistent with other parts
of Nimbus we convert slot keys.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-11-24 15:54:59 +00:00
|
|
|
# `call` is special. Most host functions do `flip256` in `evmc_host_glue`
|
|
|
|
# and `show` in `host_services`, but `call` needs to minimise C stack used
|
|
|
|
# by nested EVM calls. Just `flip256` in glue's `call` adds a lot of
|
|
|
|
# stack: +65% in tests, enough to blow our 750kiB test stack target and
|
|
|
|
# crash. Easily avoided by doing `flip256` and `show` out-of-line here.
|
|
|
|
var msg = msg # Make a local copy that's ok to modify.
|
|
|
|
msg.value = flip256(msg.value)
|
EVMC: Improve host call tracing and fix nested call C stack usage
This combines two things, a C stack usage change with EVM nested calls
via EVMC, and changes to host call tracing.
Feature-wise, the tracing is improved:
- Storage keys and values are make more sense.
- The message/result/context objects are shown with all relevant fields.
- `call` trace is split into entry/exit, so these can be shown around the
called contract's operations, instead of only showing the `call` parameters
after the nested call is finished.
- Nested calls are indented, which helps to highlight the flow.
- C stack usage considerably reduced in nested calls when more functionality
is enabled (either tracing here, or other things to come).
This will seem like a minor patch, but C stack usage was the real motivation,
after plenty of time in the debugger.
Nobody cares about stack when `showTxCalls` (you can just use a big stack when
debugging). But these subtle changes around the `call` path were found to be
necessary for passing all tests when the EVMC nested call code is completed,
and that's a prerequisite for many things: async EVM, dynamic EVM, Beam Sync,
and to fix https://github.com/status-im/nimbus-eth1/issues/345.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-08-09 14:54:38 +00:00
|
|
|
host.showCallEntry(msg)
|
2021-08-17 16:18:07 +00:00
|
|
|
let c = if msg.kind == EVMC_CREATE or msg.kind == EVMC_CREATE2:
|
|
|
|
beforeExecCreateEvmcNested(host, msg)
|
|
|
|
else:
|
|
|
|
beforeExecCallEvmcNested(host, msg)
|
|
|
|
when defined(evmc_enabled):
|
|
|
|
c.host.init(cast[ptr nimbus_host_interface](host.hostInterface),
|
|
|
|
cast[typeof(c.host.context)](host))
|
|
|
|
host.saveComputation.add(host.computation)
|
|
|
|
host.computation = c
|
|
|
|
return c
|
2021-05-24 08:53:53 +00:00
|
|
|
|
|
|
|
proc afterExecEvmcNested(host: TransactionHost, child: Computation,
|
|
|
|
kind: EvmcCallKind): EvmcResult
|
|
|
|
# This function must be declared with `{.noinline.}` to make sure it doesn't
|
|
|
|
# contribute to the stack frame of `callEvmcNested` below.
|
|
|
|
{.noinline.} =
|
2021-08-17 16:18:07 +00:00
|
|
|
host.computation = host.saveComputation[^1]
|
|
|
|
host.saveComputation[^1] = nil
|
|
|
|
host.saveComputation.setLen(host.saveComputation.len - 1)
|
2021-05-24 08:53:53 +00:00
|
|
|
if kind == EVMC_CREATE or kind == EVMC_CREATE2:
|
|
|
|
afterExecCreateEvmcNested(host, child, result)
|
|
|
|
else:
|
|
|
|
afterExecCallEvmcNested(host, child, result)
|
EVMC: Improve host call tracing and fix nested call C stack usage
This combines two things, a C stack usage change with EVM nested calls
via EVMC, and changes to host call tracing.
Feature-wise, the tracing is improved:
- Storage keys and values are make more sense.
- The message/result/context objects are shown with all relevant fields.
- `call` trace is split into entry/exit, so these can be shown around the
called contract's operations, instead of only showing the `call` parameters
after the nested call is finished.
- Nested calls are indented, which helps to highlight the flow.
- C stack usage considerably reduced in nested calls when more functionality
is enabled (either tracing here, or other things to come).
This will seem like a minor patch, but C stack usage was the real motivation,
after plenty of time in the debugger.
Nobody cares about stack when `showTxCalls` (you can just use a big stack when
debugging). But these subtle changes around the `call` path were found to be
necessary for passing all tests when the EVMC nested call code is completed,
and that's a prerequisite for many things: async EVM, dynamic EVM, Beam Sync,
and to fix https://github.com/status-im/nimbus-eth1/issues/345.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-08-09 14:54:38 +00:00
|
|
|
host.showCallReturn(result, kind.isCreate)
|
2021-05-24 08:53:53 +00:00
|
|
|
|
|
|
|
template callEvmcNested*(host: TransactionHost, msg: EvmcMessage): EvmcResult =
|
EVMC: Byte-endian conversions for 256-bit numeric values
Perform byte-endian conversion for 256-bit numeric values, but not 256-bit
hashes. These conversions are necessary for EVMC binary compatibility.
In new EVMC, all host-side conversions are explicit, calling `flip256`.
These conversions are performed in the EVMC "glue" code, which deals with the
binary interface, so the host services aren't aware of conversions.
We intend to skip these conversions when Nimbus host calls Nimbus EVM, even
when it's a shared library, using a negotiated EVMC extension. But for now
we're focused on correctness and cross-validation with third party EVMs.
The overhead of endian conversion is not too high because most EVMC host calls
access the database anyway. `getTxContext` does not, so the conversions from
that are cached here. Also, well-optimised EVMs don't call it often.
It is arguable whether endian conversion should occur for storage slots (`key`).
In favour of no conversion: Slot keys are 32-byte blobs, and this is clear in
the EVMC definition where slot keys are `evmc_bytes32` (not `evmc_uint256be`),
meaning treating as a number is _not_ expected by EVMC. Although they are
often small numbers, sometimes they are a hash from the contract code plus a
number. Slot keys are hashed on the host side with Keccak256 before any
database calls, so the host side does not look at them numerically.
In favour of conversion: They are often small numbers and it is helpful to log
them as such, rather than a long string of zero digits with 1-2 non-zero. The
representation in JSON has leading zeros removed, like a number rather than a
32-byte blob. There is also an interesting space optimisation when the keys
are used unhashed in storage.
Nimbus currently treats slot keys on the host side as numbers, and the tests
pass when endian conversion is done. So to remain consistent with other parts
of Nimbus we convert slot keys.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-11-24 15:54:59 +00:00
|
|
|
# `call` is special. The C stack usage must be kept small for deeply nested
|
|
|
|
# EVM calls. To ensure small stack, this function must use `template` to
|
|
|
|
# inline at Nim level (same for `host.call(msg)`). `{.inline.}` is not good
|
|
|
|
# enough. Due to object return it ends up using a lot more stack. (Note
|
|
|
|
# that template parameters `host` and `msg` are multiple-evaluated here;
|
|
|
|
# simple expressions must be used when calling.)
|
2021-05-24 08:53:53 +00:00
|
|
|
let child = beforeExecEvmcNested(host, msg)
|
|
|
|
child.execCallOrCreate()
|
|
|
|
afterExecEvmcNested(host, child, msg.kind)
|