2021-04-08 14:52:10 +00:00
|
|
|
# Nimbus
|
|
|
|
|
# Copyright (c) 2018 Status Research & Development GmbH
|
|
|
|
|
# Licensed under either of
|
|
|
|
|
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or
|
|
|
|
|
# http://www.apache.org/licenses/LICENSE-2.0)
|
|
|
|
|
# * MIT license ([LICENSE-MIT](LICENSE-MIT) or
|
|
|
|
|
# http://opensource.org/licenses/MIT)
|
|
|
|
|
# at your option. This file may not be copied, modified, or distributed except
|
|
|
|
|
# according to those terms.
|
|
|
|
|
|
|
|
|
|
import
|
2022-12-02 04:35:41 +00:00
|
|
|
std/[tables, json, sets],
|
|
|
|
|
chronos,
|
Added basic async capabilities for vm2. (#1260)
* Added basic async capabilities for vm2.
This is a whole new Git branch, not the same one as last time
(https://github.com/status-im/nimbus-eth1/pull/1250) - there wasn't
much worth salvaging. Main differences:
I didn't do the "each opcode has to specify an async handler" junk
that I put in last time. Instead, in oph_memory.nim you can see
sloadOp calling asyncChainTo and passing in an async operation.
That async operation is then run by the execCallOrCreate (or
asyncExecCallOrCreate) code in interpreter_dispatch.nim.
In the test code, the (previously existing) macro called "assembler"
now allows you to add a section called "initialStorage", specifying
fake data to be used by the EVM computation run by that test. (In
the long run we'll obviously want to write tests that for-real use
the JSON-RPC API to asynchronously fetch data; for now, this was
just an expedient way to write a basic unit test that exercises the
async-EVM code pathway.)
There's also a new macro called "concurrentAssemblers" that allows
you to write a test that runs multiple assemblers concurrently (and
then waits for them all to finish). There's one example test using
this, in test_op_memory_lazy.nim, though you can't actually see it
doing so unless you uncomment some echo statements in
async_operations.nim (in which case you can see the two concurrently
running EVM computations each printing out what they're doing, and
you'll see that they interleave).
A question: is it possible to make EVMC work asynchronously? (For
now, this code compiles and "make test" passes even if ENABLE_EVMC
is turned on, but it doesn't actually work asynchronously, it just
falls back on doing the usual synchronous EVMC thing. See
FIXME-asyncAndEvmc.)
* Moved the AsyncOperationFactory to the BaseVMState object.
* Made the AsyncOperationFactory into a table of fn pointers.
Also ditched the plain-data Vm2AsyncOperation type; it wasn't
really serving much purpose. Instead, the pendingAsyncOperation
field directly contains the Future.
* Removed the hasStorage idea.
It's not the right solution to the "how do we know whether we
still need to fetch the storage value or not?" problem. I
haven't implemented the right solution yet, but at least
we're better off not putting in a wrong one.
* Added/modified/removed some comments.
(Based on feedback on the PR.)
* Removed the waitFor from execCallOrCreate.
There was some back-and-forth in the PR regarding whether nested
waitFor calls are acceptable:
https://github.com/status-im/nimbus-eth1/pull/1260#discussion_r998587449
The eventual decision was to just change the waitFor to a doAssert
(since we probably won't want this extra functionality when running
synchronously anyway) to make sure that the Future is already
finished.
2022-11-01 15:35:46 +00:00
|
|
|
json_rpc/rpcclient,
|
2022-12-02 04:35:41 +00:00
|
|
|
"."/[stack, memory, code_stream],
|
2021-06-01 11:27:05 +00:00
|
|
|
./interpreter/[gas_costs, op_codes],
|
2022-12-02 04:35:41 +00:00
|
|
|
../db/accounts_cache,
|
|
|
|
|
../common/[common, evmforks]
|
2021-04-08 14:52:10 +00:00
|
|
|
|
2022-09-26 04:56:54 +00:00
|
|
|
when defined(evmc_enabled):
|
|
|
|
|
import
|
|
|
|
|
./evmc_api
|
|
|
|
|
|
|
|
|
|
# Select between small-stack recursion and no recursion. Both are good, fast,
|
|
|
|
|
# low resource using methods. Keep both here because true EVMC API requires
|
|
|
|
|
# the small-stack method, but Chronos `async` is better without recursion.
|
|
|
|
|
const vm_use_recursion* = defined(evmc_enabled)
|
|
|
|
|
|
2021-04-08 14:52:10 +00:00
|
|
|
type
|
|
|
|
|
VMFlag* = enum
|
|
|
|
|
ExecutionOK
|
|
|
|
|
GenerateWitness
|
|
|
|
|
ClearCache
|
|
|
|
|
|
|
|
|
|
BaseVMState* = ref object of RootObj
|
|
|
|
|
prevHeaders* : seq[BlockHeader]
|
2022-12-02 04:35:41 +00:00
|
|
|
com* : CommonRef
|
2022-01-18 16:19:32 +00:00
|
|
|
parent* : BlockHeader
|
|
|
|
|
timestamp* : EthTime
|
|
|
|
|
gasLimit* : GasInt
|
2022-04-08 04:54:11 +00:00
|
|
|
fee* : Option[UInt256]
|
2022-02-27 05:21:46 +00:00
|
|
|
prevRandao* : Hash256
|
2022-06-14 06:01:51 +00:00
|
|
|
blockDifficulty*: UInt256
|
2021-04-08 14:52:10 +00:00
|
|
|
flags* : set[VMFlag]
|
|
|
|
|
tracer* : TransactionTracer
|
|
|
|
|
logEntries* : seq[Log]
|
|
|
|
|
receipts* : seq[Receipt]
|
2021-10-28 09:42:39 +00:00
|
|
|
stateDB* : AccountsCache
|
2021-04-08 14:52:10 +00:00
|
|
|
cumulativeGasUsed*: GasInt
|
|
|
|
|
touchedAccounts*: HashSet[EthAddress]
|
2021-05-11 09:05:58 +00:00
|
|
|
selfDestructs* : HashSet[EthAddress]
|
2021-04-08 14:52:10 +00:00
|
|
|
txOrigin* : EthAddress
|
|
|
|
|
txGasPrice* : GasInt
|
|
|
|
|
gasCosts* : GasCosts
|
2022-12-02 04:35:41 +00:00
|
|
|
fork* : EVMFork
|
2021-04-08 14:52:10 +00:00
|
|
|
minerAddress* : EthAddress
|
Added basic async capabilities for vm2. (#1260)
* Added basic async capabilities for vm2.
This is a whole new Git branch, not the same one as last time
(https://github.com/status-im/nimbus-eth1/pull/1250) - there wasn't
much worth salvaging. Main differences:
I didn't do the "each opcode has to specify an async handler" junk
that I put in last time. Instead, in oph_memory.nim you can see
sloadOp calling asyncChainTo and passing in an async operation.
That async operation is then run by the execCallOrCreate (or
asyncExecCallOrCreate) code in interpreter_dispatch.nim.
In the test code, the (previously existing) macro called "assembler"
now allows you to add a section called "initialStorage", specifying
fake data to be used by the EVM computation run by that test. (In
the long run we'll obviously want to write tests that for-real use
the JSON-RPC API to asynchronously fetch data; for now, this was
just an expedient way to write a basic unit test that exercises the
async-EVM code pathway.)
There's also a new macro called "concurrentAssemblers" that allows
you to write a test that runs multiple assemblers concurrently (and
then waits for them all to finish). There's one example test using
this, in test_op_memory_lazy.nim, though you can't actually see it
doing so unless you uncomment some echo statements in
async_operations.nim (in which case you can see the two concurrently
running EVM computations each printing out what they're doing, and
you'll see that they interleave).
A question: is it possible to make EVMC work asynchronously? (For
now, this code compiles and "make test" passes even if ENABLE_EVMC
is turned on, but it doesn't actually work asynchronously, it just
falls back on doing the usual synchronous EVMC thing. See
FIXME-asyncAndEvmc.)
* Moved the AsyncOperationFactory to the BaseVMState object.
* Made the AsyncOperationFactory into a table of fn pointers.
Also ditched the plain-data Vm2AsyncOperation type; it wasn't
really serving much purpose. Instead, the pendingAsyncOperation
field directly contains the Future.
* Removed the hasStorage idea.
It's not the right solution to the "how do we know whether we
still need to fetch the storage value or not?" problem. I
haven't implemented the right solution yet, but at least
we're better off not putting in a wrong one.
* Added/modified/removed some comments.
(Based on feedback on the PR.)
* Removed the waitFor from execCallOrCreate.
There was some back-and-forth in the PR regarding whether nested
waitFor calls are acceptable:
https://github.com/status-im/nimbus-eth1/pull/1260#discussion_r998587449
The eventual decision was to just change the waitFor to a doAssert
(since we probably won't want this extra functionality when running
synchronously anyway) to make sure that the Future is already
finished.
2022-11-01 15:35:46 +00:00
|
|
|
asyncFactory* : AsyncOperationFactory
|
2021-04-08 14:52:10 +00:00
|
|
|
|
|
|
|
|
TracerFlags* {.pure.} = enum
|
|
|
|
|
EnableTracing
|
|
|
|
|
DisableStorage
|
|
|
|
|
DisableMemory
|
|
|
|
|
DisableStack
|
|
|
|
|
DisableState
|
|
|
|
|
DisableStateDiff
|
|
|
|
|
EnableAccount
|
2022-10-15 15:58:23 +00:00
|
|
|
DisableReturnData
|
2021-04-08 14:52:10 +00:00
|
|
|
|
|
|
|
|
TransactionTracer* = object
|
|
|
|
|
trace*: JsonNode
|
|
|
|
|
flags*: set[TracerFlags]
|
|
|
|
|
accounts*: HashSet[EthAddress]
|
2022-04-08 04:54:11 +00:00
|
|
|
storageKeys*: seq[HashSet[UInt256]]
|
2021-04-08 14:52:10 +00:00
|
|
|
|
|
|
|
|
Computation* = ref object
|
|
|
|
|
# The execution computation
|
|
|
|
|
vmState*: BaseVMState
|
|
|
|
|
msg*: Message
|
|
|
|
|
memory*: Memory
|
|
|
|
|
stack*: Stack
|
|
|
|
|
returnStack*: seq[int]
|
|
|
|
|
gasMeter*: GasMeter
|
|
|
|
|
code*: CodeStream
|
|
|
|
|
output*: seq[byte]
|
|
|
|
|
returnData*: seq[byte]
|
|
|
|
|
error*: Error
|
|
|
|
|
touchedAccounts*: HashSet[EthAddress]
|
2021-05-11 09:05:58 +00:00
|
|
|
selfDestructs*: HashSet[EthAddress]
|
2021-04-08 14:52:10 +00:00
|
|
|
logEntries*: seq[Log]
|
|
|
|
|
savePoint*: SavePoint
|
|
|
|
|
instr*: Op
|
|
|
|
|
opIndex*: int
|
2022-09-26 04:56:54 +00:00
|
|
|
when defined(evmc_enabled):
|
|
|
|
|
host*: HostContext
|
|
|
|
|
child*: ref nimbus_message
|
|
|
|
|
res*: nimbus_result
|
|
|
|
|
else:
|
|
|
|
|
parent*, child*: Computation
|
Added basic async capabilities for vm2. (#1260)
* Added basic async capabilities for vm2.
This is a whole new Git branch, not the same one as last time
(https://github.com/status-im/nimbus-eth1/pull/1250) - there wasn't
much worth salvaging. Main differences:
I didn't do the "each opcode has to specify an async handler" junk
that I put in last time. Instead, in oph_memory.nim you can see
sloadOp calling asyncChainTo and passing in an async operation.
That async operation is then run by the execCallOrCreate (or
asyncExecCallOrCreate) code in interpreter_dispatch.nim.
In the test code, the (previously existing) macro called "assembler"
now allows you to add a section called "initialStorage", specifying
fake data to be used by the EVM computation run by that test. (In
the long run we'll obviously want to write tests that for-real use
the JSON-RPC API to asynchronously fetch data; for now, this was
just an expedient way to write a basic unit test that exercises the
async-EVM code pathway.)
There's also a new macro called "concurrentAssemblers" that allows
you to write a test that runs multiple assemblers concurrently (and
then waits for them all to finish). There's one example test using
this, in test_op_memory_lazy.nim, though you can't actually see it
doing so unless you uncomment some echo statements in
async_operations.nim (in which case you can see the two concurrently
running EVM computations each printing out what they're doing, and
you'll see that they interleave).
A question: is it possible to make EVMC work asynchronously? (For
now, this code compiles and "make test" passes even if ENABLE_EVMC
is turned on, but it doesn't actually work asynchronously, it just
falls back on doing the usual synchronous EVMC thing. See
FIXME-asyncAndEvmc.)
* Moved the AsyncOperationFactory to the BaseVMState object.
* Made the AsyncOperationFactory into a table of fn pointers.
Also ditched the plain-data Vm2AsyncOperation type; it wasn't
really serving much purpose. Instead, the pendingAsyncOperation
field directly contains the Future.
* Removed the hasStorage idea.
It's not the right solution to the "how do we know whether we
still need to fetch the storage value or not?" problem. I
haven't implemented the right solution yet, but at least
we're better off not putting in a wrong one.
* Added/modified/removed some comments.
(Based on feedback on the PR.)
* Removed the waitFor from execCallOrCreate.
There was some back-and-forth in the PR regarding whether nested
waitFor calls are acceptable:
https://github.com/status-im/nimbus-eth1/pull/1260#discussion_r998587449
The eventual decision was to just change the waitFor to a doAssert
(since we probably won't want this extra functionality when running
synchronously anyway) to make sure that the Future is already
finished.
2022-11-01 15:35:46 +00:00
|
|
|
pendingAsyncOperation*: Future[void]
|
2021-04-08 14:52:10 +00:00
|
|
|
continuation*: proc() {.gcsafe.}
|
|
|
|
|
|
|
|
|
|
Error* = ref object
|
|
|
|
|
info*: string
|
|
|
|
|
burnsGas*: bool
|
|
|
|
|
|
|
|
|
|
GasMeter* = object
|
|
|
|
|
gasRefunded*: GasInt
|
|
|
|
|
gasRemaining*: GasInt
|
|
|
|
|
|
|
|
|
|
CallKind* = enum
|
|
|
|
|
evmcCall = 0, # CALL
|
|
|
|
|
evmcDelegateCall = 1, # DELEGATECALL
|
|
|
|
|
evmcCallCode = 2, # CALLCODE
|
|
|
|
|
evmcCreate = 3, # CREATE
|
|
|
|
|
evmcCreate2 = 4 # CREATE2
|
|
|
|
|
|
|
|
|
|
MsgFlags* = enum
|
|
|
|
|
emvcNoFlags = 0
|
|
|
|
|
emvcStatic = 1
|
|
|
|
|
|
|
|
|
|
Message* = ref object
|
|
|
|
|
kind*: CallKind
|
|
|
|
|
depth*: int
|
|
|
|
|
gas*: GasInt
|
|
|
|
|
sender*: EthAddress
|
|
|
|
|
contractAddress*: EthAddress
|
|
|
|
|
codeAddress*: EthAddress
|
|
|
|
|
value*: UInt256
|
|
|
|
|
data*: seq[byte]
|
|
|
|
|
flags*: MsgFlags
|
Added basic async capabilities for vm2. (#1260)
* Added basic async capabilities for vm2.
This is a whole new Git branch, not the same one as last time
(https://github.com/status-im/nimbus-eth1/pull/1250) - there wasn't
much worth salvaging. Main differences:
I didn't do the "each opcode has to specify an async handler" junk
that I put in last time. Instead, in oph_memory.nim you can see
sloadOp calling asyncChainTo and passing in an async operation.
That async operation is then run by the execCallOrCreate (or
asyncExecCallOrCreate) code in interpreter_dispatch.nim.
In the test code, the (previously existing) macro called "assembler"
now allows you to add a section called "initialStorage", specifying
fake data to be used by the EVM computation run by that test. (In
the long run we'll obviously want to write tests that for-real use
the JSON-RPC API to asynchronously fetch data; for now, this was
just an expedient way to write a basic unit test that exercises the
async-EVM code pathway.)
There's also a new macro called "concurrentAssemblers" that allows
you to write a test that runs multiple assemblers concurrently (and
then waits for them all to finish). There's one example test using
this, in test_op_memory_lazy.nim, though you can't actually see it
doing so unless you uncomment some echo statements in
async_operations.nim (in which case you can see the two concurrently
running EVM computations each printing out what they're doing, and
you'll see that they interleave).
A question: is it possible to make EVMC work asynchronously? (For
now, this code compiles and "make test" passes even if ENABLE_EVMC
is turned on, but it doesn't actually work asynchronously, it just
falls back on doing the usual synchronous EVMC thing. See
FIXME-asyncAndEvmc.)
* Moved the AsyncOperationFactory to the BaseVMState object.
* Made the AsyncOperationFactory into a table of fn pointers.
Also ditched the plain-data Vm2AsyncOperation type; it wasn't
really serving much purpose. Instead, the pendingAsyncOperation
field directly contains the Future.
* Removed the hasStorage idea.
It's not the right solution to the "how do we know whether we
still need to fetch the storage value or not?" problem. I
haven't implemented the right solution yet, but at least
we're better off not putting in a wrong one.
* Added/modified/removed some comments.
(Based on feedback on the PR.)
* Removed the waitFor from execCallOrCreate.
There was some back-and-forth in the PR regarding whether nested
waitFor calls are acceptable:
https://github.com/status-im/nimbus-eth1/pull/1260#discussion_r998587449
The eventual decision was to just change the waitFor to a doAssert
(since we probably won't want this extra functionality when running
synchronously anyway) to make sure that the Future is already
finished.
2022-11-01 15:35:46 +00:00
|
|
|
|
|
|
|
|
LazyDataSource* = ref object of RootObj
|
|
|
|
|
ifNecessaryGetStorage*: proc(c: Computation, slot: UInt256): Future[void] {.gcsafe.}
|
2022-12-02 04:35:41 +00:00
|
|
|
|
Added basic async capabilities for vm2. (#1260)
* Added basic async capabilities for vm2.
This is a whole new Git branch, not the same one as last time
(https://github.com/status-im/nimbus-eth1/pull/1250) - there wasn't
much worth salvaging. Main differences:
I didn't do the "each opcode has to specify an async handler" junk
that I put in last time. Instead, in oph_memory.nim you can see
sloadOp calling asyncChainTo and passing in an async operation.
That async operation is then run by the execCallOrCreate (or
asyncExecCallOrCreate) code in interpreter_dispatch.nim.
In the test code, the (previously existing) macro called "assembler"
now allows you to add a section called "initialStorage", specifying
fake data to be used by the EVM computation run by that test. (In
the long run we'll obviously want to write tests that for-real use
the JSON-RPC API to asynchronously fetch data; for now, this was
just an expedient way to write a basic unit test that exercises the
async-EVM code pathway.)
There's also a new macro called "concurrentAssemblers" that allows
you to write a test that runs multiple assemblers concurrently (and
then waits for them all to finish). There's one example test using
this, in test_op_memory_lazy.nim, though you can't actually see it
doing so unless you uncomment some echo statements in
async_operations.nim (in which case you can see the two concurrently
running EVM computations each printing out what they're doing, and
you'll see that they interleave).
A question: is it possible to make EVMC work asynchronously? (For
now, this code compiles and "make test" passes even if ENABLE_EVMC
is turned on, but it doesn't actually work asynchronously, it just
falls back on doing the usual synchronous EVMC thing. See
FIXME-asyncAndEvmc.)
* Moved the AsyncOperationFactory to the BaseVMState object.
* Made the AsyncOperationFactory into a table of fn pointers.
Also ditched the plain-data Vm2AsyncOperation type; it wasn't
really serving much purpose. Instead, the pendingAsyncOperation
field directly contains the Future.
* Removed the hasStorage idea.
It's not the right solution to the "how do we know whether we
still need to fetch the storage value or not?" problem. I
haven't implemented the right solution yet, but at least
we're better off not putting in a wrong one.
* Added/modified/removed some comments.
(Based on feedback on the PR.)
* Removed the waitFor from execCallOrCreate.
There was some back-and-forth in the PR regarding whether nested
waitFor calls are acceptable:
https://github.com/status-im/nimbus-eth1/pull/1260#discussion_r998587449
The eventual decision was to just change the waitFor to a doAssert
(since we probably won't want this extra functionality when running
synchronously anyway) to make sure that the Future is already
finished.
2022-11-01 15:35:46 +00:00
|
|
|
AsyncOperationFactory* = ref object of RootObj
|
|
|
|
|
lazyDataSource*: LazyDataSource
|
