- fixes non repeatable directive error message
- refactor nim caching in github action ci
- refactor github action ci matrix
- extract miniz to it's own repo
This is the main patch which removes Whisper code from `nimbus-eth1` code.
It removes all configuration, help, startup, JSON-RPC calls and most types.
Note, there is still Whisper functionality in `nim-eth`. Also, the "wrapper"
under `wrappers/` isn't dealt with by this change, but it's not built by
default (and might not currently work).
Signed-off-by: Jamie Lokier <jamie@shareable.org>
This is the patch which removes Whisper functionality from `nimbus-eth1`,
even though the code has yet to be removed after.
After this change, enabling Whisper has no effect. Its configuration is
ignored and it won't be started.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
Remove the C and Go example wrappers that call Nimbus as a library, by removing
the entire `wrappers/` directory. They are removed because they only wrap
Whisper protocol support, which has been removed as it is obsolete.
The only thing wrapped were Whisper functions, even though there were separate
`go_wrapper_example` and `go_wrapper_whisper_example` programs. The wrappers
don't build without Whisper in Nimbus, and without it, there isn't enough left
for them to be useful examples.
Also remove support for building the whole of Nimbus as a library, because
there is nothing left using it. These targets are gone from the Makefile:
- `wrappers`
- `wrappers-static`
- `libnimbus.so`
- `libnimbus.a`
The code isn't really gone, because it remains available in Git history. It
may be useful someday, so a comment has been left in the Makefile for future
generations:
> This note is kept so that anyone wanting to build Nimbus as a library or call
> from C or Go will know it has been done before. The previous working version
> can be found in Git history. Look for the `nimbus-eth1` commit that adds
> this comment and removes `wrappers/*`.
Also worth a note, the library support was not tested on Windows, and the
shared library support was only tested on Linux.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
With Whisper support gone, the wrappers cannot be built. More detail in the
patch that removes the wrappers.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
This commit turns Whisper off by default, without changing anything else.
So this can be cherry-picked if you just want to disable Whisper without
removing it.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
There's no need for macro `toSymbolName` to convert fork enum values to their
presentation texts (logging etc) then re-parse them back to a fork enum value.
`asFork` is already used in the same function and works without these steps,
so use it consistently.
Same applies to `op.toSymbolName` and `asOp`.
This makes the code simpler, and removes a text pattern-matching requirement.
The patch has been checked to confirm it doesn't change the compiled code.
Motivation: The forks list will be removed from VM because it is used outside
the VM as well. Doing so highlighted vm2's `toSymbolName`. It's not needed,
and it's best if the VM doesn't constrain text strings used outside the VM
Signed-off-by: Jamie Lokier <jamie@shareable.org>
The last caller of `setupComputation` is gone, now that it's been replaced by
the single entry point for all EVM calls, `runComputation`.
With this removal, EVM's `Computation` type should no longer be used anywhere
outside the call module (except in some tests and the EVM itself).
Signed-off-by: Jamie Lokier <jamie@shareable.org>
Simplify transaction validations to use `runComputation`; drop other code.
Getting everything right up to this point to pass all the tests was trickier
than it looks.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
Simplify how JSON fixtures tests are run to use `runComputation`.
Drop other code.
These use the `noTransfer` option, which is similar enough to calling
`c.executeOpcodes()` instead of `c.execComputation()`.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
Add another flag to disable a processing step when a call doesn't come from
a real transaction:
- `noTransfer`: Don't update balances, nonces, code.
This is to support VM fixtures tests which require account balances and nonces
to be unchanged when running the account's code.
These tests call `c.executeOpcodes()`, an internal function of the EVM, instead
of the usual `c.execComputation()`. It goes direct to the bytecode dispatcher,
skipping parts of `Computation` that are normally called.
But we can't keep calling `c.executeOpcodes()` and have a single entry point to
the VM, let alone an EVMC entry point.
`noTransfer` provides similar enough behaviour to calling `c.executeOpcodes()`
that these tests can use the new single entry point like everything else.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
Simplify `estimateGas` to use `runComputation`; drop other code.
The RPC/GraphQL `estimateGas` operation is quite different from the `call`
operation. It is much more like ordinary transaction execution than `call`,
though there are still enough differences that tx validation cannot be used.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
Simplify `call` to use `runComputation`; drop other code.
The RPC/GraphQL `call` operation differs in many ways from regular transaction
calls. The following flags are set, to disable various steps in processing.
All four are true (disabling the corresponding step) for `call`:
- `noIntrinsic`: Don't charge intrinsic gas.
- `noAccessList`: Don't initialise EIP2929 access list.
- `noGasCharge`: Don't charge sender account for gas.
- `noRefund`: Don't apply gas refund/burn rule.
Interestingly, the RPC/GraphQL `estimateGas` operation doesn't behave so
differently from regular transactions. It might be that not all these steps
should be disabled for `call` either. But until we investigate what
RPC/GraphQL clients are expecting, keep the same behaviour as before.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
The following four flags are added, to change various steps in EVM processing
when a call doesn't come from a real transaction:
- `noIntrinsic`: Don't charge intrinsic gas.
- `noAccessList`: Don't initialise EIP2929 access list.
- `noGasCharge`: Don't charge sender account for gas.
- `noRefund`: Don't apply gas refund/burn rule.
This is to support RPC and GraphQL `call` operations, which behave differently
in some ways from regular transaction calls, and to support some test suites.
In EVMC terms, all these alterations can be performed on the host side.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
Calculate extra intrinsic gas for an EIP-2930 transaction with access list.
While we're here, do the rest of the intrinsic gas calculation. Make it clear,
explicit and in one place. (Previous code delegated parts of the calculation
to `transaction.nim` but had to do the rest locally due to mismatched types.)
Signed-off-by: Jamie Lokier <jamie@shareable.org>
New entry point `runComputation`, for all EVM calls.
(Later the intent is `runComputationAsync`.)
As noted in commit 297d789, there are six entry points calling EVM computation,
with different parameters and expecting different behaviours. Parameters were
dealt with in `setupComputation`. Behaviours are unified in `runComputation`,
with options passed via `CallParams`.
This code performs the steps used when validating a transaction. Options for
non-standard behaviour for RPC, GraphQL and tests to be added as required.
This replaces `setupComputation`, `execComputation` and `executeOpcodes`
(other than its own calls). As a result `Computation` and other EVM types are
no longer referenced in the main program, and many imports can be dropped.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
Change fixtures tests to use shared `setupComputation` instead of
their own slightly different variant.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
Change RPC/GraphQL calls to the EVM to use shared `setupComputation`
instead of their own special variant.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
Reverts part of commit 2539bd9 while keeping the intention of it:
To avoid duplicate CI runs when making and updating a PR.
Disabling `push` means we cannot push to a branch and see the CI results
directly without making a PR, which some of us use. There are many situations
where this is useful, and "[WIP]" PRs are not appropriate for all.
Disabling `pull_request` has a similar effect, removing duplicate CI.
It is known that `pull_request` _is_ needed when a third party sends a
PR (because it's not committed to the repo yet). But this is rare at the
moment, and there's a workaround: A committer can push the third party change
to a branch, triggering CI.
So re-enable `push`, disable `pull_request`, and we'll see if the latter
missing causes problems in practice. Won't know until we try it.
Note: This might be interim until `workflow_dispatch` is working better.
Perhaps that needs more configuration. Currently, `workflow_dispatch` is kind
of useless for CI tests, because it doesn't result in any CI indicator
associated with a commit or branch. Even the actions page doesn't show the
name of the branch, just a less-than-useful generic "CI" for these actions.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
There are currently six entry points to running an EVM computation, all with
slightly different parameters, and expecting slightly different EVM behaviours.
First step in merging them is a common `setupComputation` that replaces all
the different `*...SetupComputation` functions.
This uses the `TransactionHost` type because it's a step towards using that
type for all EVM calls using only EVMC. For now an EVMC message is created
then translated to EVM-internal `Message`. It is done this way to build up
the new interface in stages where all tests pass at each stage.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
`TransactionHost` represents the interface to the EVM from the application once
we've fully transitioned to EVMC. It represents a managed EVM call, and the
"EVMC host" side of the host<->EVM boundary.
This holds transaction state which sits outside the EVM, but a pointer to this
is passed around by the EVM as _opaque_ type `evmc_host_context`.
To the EVM, this offers "host services", which manage account state that the
EVM cannot do, such as balance transfers, call costs, storage, logs, gas
refunds, nested calls and new contract addresses. The EVM has no direct access
to the account state database or network; it's all via "host services".
To the application (host side), this object represents a managed EVM call, with
inputs, a method to run, outputs, hidden transaction state, and later async
scheduling. It is to replace `Computation` on the application side, while
`Computation` will remain but just be for the EVM side.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
This bump is required by `TransactionHost` coming next.
There have been many changes to `nim-evmc` since earlier developments on
`nimbus-eth1`. The types in the API have changed, and there is no longer a
need to cast the function types.
This commit bumps the submodule while keeping with EVMC API 7.5 for now.
`.gitmodules` is updated to follow the module branch `api-version-7.5`.
Submodule changes:
> Nim: Function casts no longer required, type-checking finally works
> Nim: Make the test host use exactly the type signatures in the API
> add github action script
> Nim: Export `evmc_create_vm_name_fn`
> Nim: Use `var` instead of `ptr` for arguments that are pass-by-ref
> Nim: Fix missing `ptr` in signature of `execute`
> Nim: Use `ptr` in `release` for API consistency
> Nim: Disable "Imported and not used" warning in test program
> Nim: Remove unnecessary exports from the Nim example host/VM
> Nim: Simplify Nim C++ codegen bug workaround
> Nim: Add type and documentation for `evmc_create_example_fn`
> Nim: Workaround Nim compiler bug with Nim 1.2.10
> Nim: Workaround Nim compiler bug with `uint32`-sized bitset
> Nim: Use a more varied bit pattern in `get_capabilities` test
> Nim: Convert `evmc_capabilities` to Nim bitset
> Nim: Convert `evmc_flags` to Nim bitset
> Nim: Compile time verify that target C enums are `int` sized
> Nim: Recognise that target C enums are not always `cint`
> Nim: Make `evmc_status_code` a normal enum
> Nim: Use `csize_t` in test program and Nim example host/VM
> Nim: Use `csize_t` in the VM support API
> Nim: Use `csize_t` in appropriate places to match EVMC API
> Nim: Use binary-compatible C99 `bool` without target assumptions
> Version (README): Announce we support EVMC 7.5.0 and Istanbul fork
> Doc: Tweak GitHub badges on README
> Doc: Make title fit on GitHub page without wrapping
> Doc: README edits
> Doc: Update the README for 2021 - the year of Nimbus!
> License: Add licensing info about third-party files
> License: Update license text in README
> License: Fix incorrect name of Apache v2 license file and tweak
> License: Add missing MIT license file
> Nim: Add and update copyright headers
> Nim: Update Nim API to follow EVMC API 7.5.0
> Nim: Update doc comments and fix formatting
> Make the tests work with the new C++ mini-EVM in EVMC 7.5.0
> Change C++ `example_host` to workaround Nim -> C call type mismatch
> The EVMC package expects C++14 now
> Upgrade C++ `example_host` in `tests/evmc_c` to EVMC 7.5.0
> Upgrade C++ `example_vm` in `tests/evmc_c` to EVMC 7.5.0
> Upgrade C/C++ headers in `tests/evmc_c` to EVMC 7.5.0
Signed-off-by: Jamie Lokier <jamie@shareable.org>
This fixes a bug spotted by @mjfh that was introduced by commit 2a7ccceb:
try:
if not c.continuation.isNil:
(c.continuation)()
c.continuation = nil
c.selectVM(fork)
except CatchableError as e:
...
The call to `(c.continuation)()` was moved by 2a7ccceb inside the `try` so
that, like all the Op functions do already, if the continuation raises, the
interpreter's general catch turns the exception into a an error status result.
But if the continuation raises an exception, `continuation` is not cleared in
the next line, and at the next resumption the continuation is called again.
It may loop doing this.
This doesn't currently happen because the continuations don't really raise, but
it's still a correctness issue.
This fix also allows a continuation to spawn a second continuation, if it
encounters a second suspension point. This also doesn't happen currently,
but the pattern will become useful with async EVM.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
The conditions mentioned in the old TODO comment have been checked. All
fixtures have either 40 hex digits or empty string for "to". There is a test
with all-zeros, and it means send to that account, not contract creation.
Empty string means contract creation.
This patch does not change the relaxed parsing where fewer than 40 digits is
accepted. We should probably be stricter about this.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
There is no valid `CREATE_CONTRACT_ADDRESS`. Some places on the internet say
account zero means contract creation, but that's not correct.
Transactions to `ZERO_ADDRESS` are legitimate transfers to that account, not
contract creations. They are used to "burn" Eth. People also send Eth to
address zero by accident, unrecoverably, due to poor user interface issues.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
Each place in `call_evm` that sets up an EVM call calculates the new contract
address for contract creations. But it's redundant, because `newComputation`
ignores the provided value and does the calculation again.
Remove the unused address calculation.
This is also a step to merging different entry points and EVMC. This change
ends up with the same value in both `msg.contractAddress` and `msg.codeAddress`
for every entry point, and this is good because it matches the EVMC message
structure, where they are replaced by only one value called `msg.destination`
Signed-off-by: Jamie Lokier <jamie@shareable.org>
`c.executeOpcodes` is called by some JSON fixture tests. These tests bypass
some of the setup and return, and because of this call, continuations aren't
processed either. Opcodes that use continuations will behave incorrectly.
The opcodes used in these particular tests don't use continuations currently,
so just add some assertions to verify this remains the case.
This is only used by local tests, and the call to `c.executeOpcodes` will be
replaced by the common entry point (that handles things like this correctly in
all cases) so we don't need to spend more time on this.
Signed-off-by: Jamie Lokier <jamie@shareable.org>
why:
previously, table data were stored with the table iterator. while
loading a table with permuted entries will always reconstruct equivalent
tables (in the sense of `==`), serialisation data are not comparable.
this patch produces always the same serialised data for equivalent
tables.
why:
source-local unit tests would hardly be triggered by github CI as rightly
criticised with the last patch.
details:
source-local unit tests have been moved to tests folder.
this version also contains rlp serialisation code so rlp encode/decode
will apply tranparently. this is not needed in p2p/validate but will be
useful with the clique protocol.
jangko
Jamie Lokier
Ștefan Talpalaru
Jordan Hrycaj