Commit Graph

987 Commits

Author SHA1 Message Date
Jamie Lokier ef7773daa6
Whisper: Remove Whisper-specific hexstring/JSON/key storage support
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 18:12:48 +01:00
Jamie Lokier 613f06e61c
Whisper: Remove all the main Whisper code (config, startup, RPC etc)
This is the main patch which removes Whisper code from `nimbus-eth1` code.
It removes all configuration, help, startup, JSON-RPC calls and most types.

Note, there is still Whisper functionality in `nim-eth`.  Also, the "wrapper"
under `wrappers/` isn't dealt with by this change, but it's not built by
default (and might not currently work).

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 18:12:48 +01:00
Jamie Lokier f5c69f372a
Whisper: Remove all code which starts Whisper running
This is the patch which removes Whisper functionality from `nimbus-eth1`,
even though the code has yet to be removed after.

After this change, enabling Whisper has no effect.  Its configuration is
ignored and it won't be started.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 18:12:48 +01:00
Jamie Lokier 774f697c73
Whisper: Disable Whisper (Shh) protocol by default
This commit turns Whisper off by default, without changing anything else.

So this can be cherry-picked if you just want to disable Whisper without
removing it.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 18:12:47 +01:00
Jamie Lokier e2689792b0
vm2: Remove `toSymbolName` unnecessary symbol-text-symbol conversion
There's no need for macro `toSymbolName` to convert fork enum values to their
presentation texts (logging etc) then re-parse them back to a fork enum value.
`asFork` is already used in the same function and works without these steps,
so use it consistently.

Same applies to `op.toSymbolName` and `asOp`.

This makes the code simpler, and removes a text pattern-matching requirement.
The patch has been checked to confirm it doesn't change the compiled code.

Motivation: The forks list will be removed from VM because it is used outside
the VM as well.  Doing so highlighted vm2's `toSymbolName`.  It's not needed,
and it's best if the VM doesn't constrain text strings used outside the VM

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 16:54:38 +01:00
Jamie Lokier 8f9c593dac
EVM: Remove `vm_types2` unused `op_codes`/`opcode_values` re-exports
Everything builds with this section removed, all build options.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 16:54:38 +01:00
Jamie Lokier fdebf6c1f7
EVM: Remove unused file `vm_message`
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 16:54:38 +01:00
Jamie Lokier 5e718bcbe2
EVM: Remove most unused imports of `vm_*` files
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 16:54:38 +01:00
Jamie Lokier beb750b8df
Transaction: Remove no longer used `setupComputation`
The last caller of `setupComputation` is gone, now that it's been replaced by
the single entry point for all EVM calls, `runComputation`.

With this removal, EVM's `Computation` type should no longer be used anywhere
outside the call module (except in some tests and the EVM itself).

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 16:54:34 +01:00
Jamie Lokier 8b4f5a1103 Transaction: Change transactions to go through unified EVM runner
Simplify transaction validations to use `runComputation`; drop other code.

Getting everything right up to this point to pass all the tests was trickier
than it looks.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 18:40:41 +03:00
Jamie Lokier bf6569bdeb Assembler: Change assembler tests to go through unified EVM runner
Simplify how assembler tests are run to use `runComputation`; drop other code.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 18:40:41 +03:00
Jamie Lokier 9211a15c0a Fixtures: Change JSON fixture tests to go through unified EVM runner
Simplify how JSON fixtures tests are run to use `runComputation`.
Drop other code.

These use the `noTransfer` option, which is similar enough to calling
`c.executeOpcodes()` instead of `c.execComputation()`.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 18:40:41 +03:00
Jamie Lokier 5fb3c51e5e Transaction: Skip balance & nonce updates option in `runComputation`
Add another flag to disable a processing step when a call doesn't come from
a real transaction:

- `noTransfer`: Don't update balances, nonces, code.

This is to support VM fixtures tests which require account balances and nonces
to be unchanged when running the account's code.

These tests call `c.executeOpcodes()`, an internal function of the EVM, instead
of the usual `c.execComputation()`.  It goes direct to the bytecode dispatcher,
skipping parts of `Computation` that are normally called.

But we can't keep calling `c.executeOpcodes()` and have a single entry point to
the VM, let alone an EVMC entry point.

`noTransfer` provides similar enough behaviour to calling `c.executeOpcodes()`
that these tests can use the new single entry point like everything else.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 18:40:41 +03:00
Jamie Lokier deffa20b07
RPC and GraphQL: Change `estimateGas` to go through unified EVM runner
Simplify `estimateGas` to use `runComputation`; drop other code.

The RPC/GraphQL `estimateGas` operation is quite different from the `call`
operation.  It is much more like ordinary transaction execution than `call`,
though there are still enough differences that tx validation cannot be used.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-29 08:23:20 +01:00
Jamie Lokier b16aa2f1f7
RPC and GraphQL: Change `call` op to go through unified EVM runner
Simplify `call` to use `runComputation`; drop other code.

The RPC/GraphQL `call` operation differs in many ways from regular transaction
calls.  The following flags are set, to disable various steps in processing.
All four are true (disabling the corresponding step) for `call`:

- `noIntrinsic`:  Don't charge intrinsic gas.
- `noAccessList`: Don't initialise EIP2929 access list.
- `noGasCharge`:  Don't charge sender account for gas.
- `noRefund`:     Don't apply gas refund/burn rule.

Interestingly, the RPC/GraphQL `estimateGas` operation doesn't behave so
differently from regular transactions.  It might be that not all these steps
should be disabled for `call` either.  But until we investigate what
RPC/GraphQL clients are expecting, keep the same behaviour as before.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-29 08:23:20 +01:00
Jamie Lokier 306c8e92c2
Transaction: `runComputation` options for non-standard EVM behaviour
The following four flags are added, to change various steps in EVM processing
when a call doesn't come from a real transaction:

- `noIntrinsic`:  Don't charge intrinsic gas.
- `noAccessList`: Don't initialise EIP2929 access list.
- `noGasCharge`:  Don't charge sender account for gas.
- `noRefund`:     Don't apply gas refund/burn rule.

This is to support RPC and GraphQL `call` operations, which behave differently
in some ways from regular transaction calls, and to support some test suites.

In EVMC terms, all these alterations can be performed on the host side.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-29 08:23:19 +01:00
Jamie Lokier 12bf0fd346
Transaction: EIP-2930 (Berlin): Extra intrinsic gas for access list
Calculate extra intrinsic gas for an EIP-2930 transaction with access list.

While we're here, do the rest of the intrinsic gas calculation.  Make it clear,
explicit and in one place.  (Previous code delegated parts of the calculation
to `transaction.nim` but had to do the rest locally due to mismatched types.)

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-29 06:54:19 +01:00
Jamie Lokier f6cc6e3ed1
Transaction: EIP-2930 (Berlin): Per-transaction extra access list
Add `accessList` to the `runComputation` API for EIP-2930 transactions.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-29 06:53:11 +01:00
Jamie Lokier 76031ccbe4
Transaction: EIP-2929 (Berlin): Initial access list
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-29 06:53:11 +01:00
Jamie Lokier c6d50a0ef7
Transaction: Unified runner `runComputation` for all EVM call types
New entry point `runComputation`, for all EVM calls.
(Later the intent is `runComputationAsync`.)

As noted in commit 297d789, there are six entry points calling EVM computation,
with different parameters and expecting different behaviours.  Parameters were
dealt with in `setupComputation`.  Behaviours are unified in `runComputation`,
with options passed via `CallParams`.

This code performs the steps used when validating a transaction.  Options for
non-standard behaviour for RPC, GraphQL and tests to be added as required.

This replaces `setupComputation`, `execComputation` and `executeOpcodes`
(other than its own calls).  As a result `Computation` and other EVM types are
no longer referenced in the main program, and many imports can be dropped.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-29 06:53:11 +01:00
Jamie Lokier 94f95efd9e
Transaction: Move `setupComputation` to its own file
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-29 06:53:11 +01:00
Jamie Lokier 4e51a5bb35
Fixtures: Change fixture tests to use shared `setupComputation`
Change fixtures tests to use shared `setupComputation` instead of
their own slightly different variant.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-28 16:48:09 +01:00
Jamie Lokier a5385e5344 RPC/GraphQL: Change RPC/GraphQL to use shared `setupComputation`
Change RPC/GraphQL calls to the EVM to use shared `setupComputation`
instead of their own special variant.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-28 16:10:19 +01:00
Jamie Lokier 8b33cbe568
Assembler: Change assembler tests to use shared `setupComputation`
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-28 15:50:02 +01:00
Jamie Lokier c655d59b5f
Assembler: Rearrange logic where assembler tests call EVM
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-28 15:49:58 +01:00
Jamie Lokier 0d3117344a
Transaction: Change tx validation to use shared `setupComputation`
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-28 15:22:06 +01:00
Jamie Lokier 297d789d21
Transaction: Merge entry point for `*...SetupComputation` functions
There are currently six entry points to running an EVM computation, all with
slightly different parameters, and expecting slightly different EVM behaviours.

First step in merging them is a common `setupComputation` that replaces all
the different `*...SetupComputation` functions.

This uses the `TransactionHost` type because it's a step towards using that
type for all EVM calls using only EVMC.  For now an EVMC message is created
then translated to EVM-internal `Message`.  It is done this way to build up
the new interface in stages where all tests pass at each stage.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-28 12:12:10 +01:00
Jamie Lokier e60ad129a2
Transaction: New object `TransactionHost` for "EVMC host services"
`TransactionHost` represents the interface to the EVM from the application once
we've fully transitioned to EVMC.  It represents a managed EVM call, and the
"EVMC host" side of the host<->EVM boundary.

This holds transaction state which sits outside the EVM, but a pointer to this
is passed around by the EVM as _opaque_ type `evmc_host_context`.

To the EVM, this offers "host services", which manage account state that the
EVM cannot do, such as balance transfers, call costs, storage, logs, gas
refunds, nested calls and new contract addresses.  The EVM has no direct access
to the account state database or network; it's all via "host services".

To the application (host side), this object represents a managed EVM call, with
inputs, a method to run, outputs, hidden transaction state, and later async
scheduling.  It is to replace `Computation` on the application side, while
`Computation` will remain but just be for the EVM side.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-28 12:12:02 +01:00
Jamie Lokier 537cac1bf5
EVM: Move where `continuation` is cleared to fix a potential stall
This fixes a bug spotted by @mjfh that was introduced by commit 2a7ccceb:

    try:
      if not c.continuation.isNil:
        (c.continuation)()
        c.continuation = nil
      c.selectVM(fork)
    except CatchableError as e:
      ...

The call to `(c.continuation)()` was moved by 2a7ccceb inside the `try` so
that, like all the Op functions do already, if the continuation raises, the
interpreter's general catch turns the exception into a an error status result.

But if the continuation raises an exception, `continuation` is not cleared in
the next line, and at the next resumption the continuation is called again.
It may loop doing this.

This doesn't currently happen because the continuations don't really raise, but
it's still a correctness issue.

This fix also allows a continuation to spawn a second continuation, if it
encounters a second suspension point.  This also doesn't happen currently,
but the pattern will become useful with async EVM.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-27 12:16:37 +01:00
Jamie Lokier 90a961243e
Clear up meaning of `ZERO_ADDRESS`, delete `CREATE_CONTRACT_ADDRESS`
There is no valid `CREATE_CONTRACT_ADDRESS`.  Some places on the internet say
account zero means contract creation, but that's not correct.

Transactions to `ZERO_ADDRESS` are legitimate transfers to that account, not
contract creations.  They are used to "burn" Eth.  People also send Eth to
address zero by accident, unrecoverably, due to poor user interface issues.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-27 12:14:51 +01:00
Jamie Lokier 745129c4ac
Transaction: Don't calculate `contractAddress` redundantly
Each place in `call_evm` that sets up an EVM call calculates the new contract
address for contract creations.  But it's redundant, because `newComputation`
ignores the provided value and does the calculation again.

Remove the unused address calculation.

This is also a step to merging different entry points and EVMC.  This change
ends up with the same value in both `msg.contractAddress` and `msg.codeAddress`
for every entry point, and this is good because it matches the EVMC message
structure, where they are replaced by only one value called `msg.destination`

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-27 12:01:47 +01:00
Jamie Lokier fa74dc909e
Fixtures: Verify EVM continuation is clear after `c.executeOpcodes`
`c.executeOpcodes` is called by some JSON fixture tests.  These tests bypass
some of the setup and return, and because of this call, continuations aren't
processed either.  Opcodes that use continuations will behave incorrectly.

The opcodes used in these particular tests don't use continuations currently,
so just add some assertions to verify this remains the case.

This is only used by local tests, and the call to `c.executeOpcodes` will be
replaced by the common entry point (that handles things like this correctly in
all cases) so we don't need to spend more time on this.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-27 11:54:44 +01:00
jangko 5fc57e4093
add validateKinship in persistBlocks of nimbus/p2p/chain.nim
put jordan's work #668 into effect, and this bring down
failing consensus test cases from 59 to 44
2021-05-27 16:28:26 +07:00
Jordan Hrycaj e5947f4db6 Deep copy semantics for LRU cache
why:
  follows standard nim semantics

details:
  changed Table to TableRef in previous patch which was the
  wrong choice (see andri's comment.)
2021-05-26 11:12:52 +01:00
Jordan Hrycaj 7b72109afa Use sorted RLP serialisation for LRU cache
why:
  previously, table data were stored with the table iterator. while
  loading a table with permuted entries will always reconstruct equivalent
  tables (in the sense of `==`), serialisation data are not comparable.

  this patch produces always the same serialised data for equivalent
  tables.
2021-05-26 07:58:12 +01:00
Jordan Hrycaj b83b47e541 LRU cache tests makeover
why:
  source-local unit tests would hardly be triggered by github CI as rightly
  criticised with the last patch.

details:
  source-local unit tests have been moved to tests folder.

  this version also contains rlp serialisation code so rlp encode/decode
  will apply tranparently. this is not needed in p2p/validate but will be
   useful with the clique protocol.
2021-05-26 07:58:12 +01:00
jangko 396f3e9909
add missing poaEngine configuration in config.nim
later we will use real engine configuration it if become available to us
2021-05-24 14:35:47 +07:00
Jordan Hrycaj 179cc75c32 Update LRU complexity comment
why:
  hash tables might worst-case degrade into linear mode, so it is not
  strictly O(1)
2021-05-24 07:57:21 +01:00
Jordan Hrycaj 1965ea027b updated LRU cache to run with complexity O(1)
why:
  to be used in Clique consensus protocol which suggests 4k cache entries.
  the previous implementation used OrderTable[] which has complexity O(n)
  for deleting entries.
2021-05-24 07:57:21 +01:00
Jordan Hrycaj a5e0bb6139 use general lru_cache for EpochHashCache
why:
  generic implementation will be also be used elsewhere
2021-05-24 07:57:21 +01:00
Jordan Hrycaj 3663b1603f pulled out cache logic into separate file
why:
  handy to re-use, eg. for upcoming clique implementation
2021-05-24 07:57:21 +01:00
Jordan Hrycaj d6a5cecb98 re-wrote validation with exceptionless functions
why:
  exceptions were from test code should be avoided in production code
2021-05-24 07:57:21 +01:00
Jordan Hrycaj 40c7bdfc06 update lookup cache management
details:
  enable fifo behaviour, using cache as argument
2021-05-24 07:57:21 +01:00
Jordan Hrycaj ce8e5511e3 backport from test_blockchain_json, see issue #666 2021-05-24 07:57:21 +01:00
jangko b82061bf46
don't mix customBootNodes and bootNodes usage 2021-05-20 14:04:17 +07:00
jangko a0d10f5728
drop PublicNetwork enum usage and replace it with NetworkId
we cannot limit the `--networkid` switch to values available in
`PublicNetwork` enum. it should able to accept very wide range of
custom NetworkId.
2021-05-20 14:04:16 +07:00
jangko ec1af91370
implement "net_nodeInfo" rpc
fixes #569
2021-05-19 16:35:13 +07:00
jangko 0ecf9fe1af
add more query fields and resolvers to graphql api
after EIP2718/EIP2930, we have additional fields:

type AccessTuple {
  address: Address!
  storageKeys : [Bytes32!]
}

type Transaction {
  r: BigInt!
  s: BigInt!
  v: BigInt!
  # Envelope transaction support
  type: Int
  accessList: [AccessTuple!]
}

close #606
2021-05-18 07:32:03 +07:00
jangko d2b47139e1
fixes `importRlpBlocks` in conf_utils.nim
now we are importing block one by one to satisfy
some of hive test cases.

we also catch exception instead of letting it terminate the
process.
2021-05-17 18:43:44 +07:00
jangko 76543da456
disable EIP-2537: Precompile for BLS12-381 curve operations
reason: not included in berlin hard fork

but we keep the code around, for future inclusion
2021-05-17 01:29:03 +07:00