Commit Graph

2419 Commits

Author SHA1 Message Date
Jamie Lokier f3f872d707
GraphQL: Typo in error message for GraphQL "call" request
Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-03 15:08:50 +01:00
Jamie Lokier 6dd14e4f4f
GraphQL: Move EVM-calling function makeCall to rpcMakeCall
`makeCall` used by GraphQL is another way to setup and call the EVM.
Move it to `transaction/call_evm`.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-03 10:59:13 +01:00
Jamie Lokier 8bda81496a
RPC: Move EVM-calling function estimateGas to rpcEstimateGas
`estimateGas` used by JSON-RPC is another way to setup and call the EVM,
also used by GraphQL.  Move it to `transaction/call_evm`.

This function has too much direct knowledge of details that shouldn't be used
outside transaction handling code, details we need to change when changing the
db and transaction memory layer.

Moving this one exposed quite a bit of abstraction leakage, as it calls
directly to the hexary trie db around `processTransaction`.

It looks like the _intended_ functionality of `estimateGas` is similar to
`rpcDoCall` with the only real difference being to not store the final state.
It looks like the extra stuff in `estimateGas` compared with `doCall` is a
messy workaround for computation not exposing the right API ("don't save final
state") for RPC to use.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-03 10:59:10 +01:00
Jamie Lokier 2732af99eb
RPC: Move EVM-calling function doCall to rpcDoCall
`doCall` used by JSON-RPC is another way to setup and call the EVM.
Move it to `transaction/call_evm`.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-03 10:59:07 +01:00
Jamie Lokier 7c3b7ab7a8
RPC: Gather EVM-calling functions to one place; rpcSetupComputation
Start gathering the functions that call the EVM into one place,
`transaction/call_evm.nim`.

This is first of a series of changes to gather all ways the EVM is called to
one place.  Duplicate, slightly different setup functions have accumulated over
time, each with some knowledge of EVM internals.  When they are brought
together, these methods will be changed to use a single entry point to the EVM,
allowing the entry point to be refactored, EVMC to be completed, and async
concurrency to be implemented on top.  This also simplifies the callers.

First, a helper function used by RPC and GraphQL to make EVM calls without
permanently modifying the account state.  `setupComputation` ->
`rpcSetupComputation`.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-03 10:59:05 +01:00
jangko 24ce283498
bump nim-graphql from 0.2.2 to 0.2.4
bug fixes:
- fixes copyTree for nkMap node
- fixes queries with variables reusability
- fixes non nullable variables with no runtime value
2021-05-02 15:25:29 +07:00
jangko 86edf449cf
bump nim-graphql from 0.2.0 to 0.2.2
nim-graphql v0.2.2 have numerous bugfixes, but notable ones are:
- only one non-introspection field are allowed in subscription root
- @skip and @include should not allowed for subscription root
- more descriptive error message for playground ethapi, fixes
- fixes GraphiQL client complaint about our instrospection system is old
- graphql http server enhancements: gzip encoding and chunked transfer
2021-05-01 15:08:36 +07:00
jangko e6d7d6188c
`processArguments` now can have alternate OptParser instead of fixed one
the `processArguments` now have overloaded proc, one with opt param and one without.
the OptParser now can be passed to `opt` param.
this is useful in scenario where in test code we need to simulate something
without using real command line arguments.
2021-04-30 12:56:19 +07:00
jangko 6049073c69
fixes nimbus.sh script for hive integration related to graphql
- add `chain.rlp` import code
- add --graphql and --graphqlbind to enable graphql service
2021-04-30 12:56:19 +07:00
jangko 68e70ebdca
fixes hard fork block number initialization in `processCustomGenesisConfig`
rather than initialize it to 0, those block numbers
are initialized to high(BlockNumber). this will fix
issue when imported genesis.json doesn't contains all
forks' blockNumber.
2021-04-30 12:56:18 +07:00
jangko 287f1b2ba0
fixes `importRlpBlock` algorithm
it will skip blocks with blockNumber <= than current
head blockNumber
2021-04-30 12:56:18 +07:00
jangko 6e79fa3708
bump nim-graphql from 0.1.2 to 0.2.0
- fixes http server response status code
- fixes `__schema.types` and `__schema.directives` implementation
- fixes 'getOperation' in executor.nim
- web ui(graphiql) for http server
2021-04-30 12:56:14 +07:00
Jamie Lokier 2a7ccceb3e
EVM: Make continuation exceptions behave as they did before
The account database code is not supposed to raise exceptions in the EVM, and
the behaviour is not well defined if it does.  It isn't compliant with EVMC
spec either.  But that will be dealt with properly when the account state-cache
is dealt with, as there is some work to be done on it.

Meanwhile, if it raises in code under `chainTo` and then `(continuation)()`,
the behaviour was changed slightly by the stack-shrink patches.

Before those patches, an exception after the recursion-point was converted to
`c.setError` "Opcode Dispatch Error" in `executeOpcodes.  After, it would
propagate out, a different behaviour.  (It still correctly walked the chain of
`c.dispose()` calls to clean up.)

It's easy to restore the original behaviour just by moving the continuation
call, so let's do that.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-30 11:32:42 +07:00
Jamie Lokier 8161de1a8f
Limit stack size on Windows like other targets
We can't use `ulimit -s` to limit stack size on Windows.  Even though Bash
accepts `ulimit -s` and the numbers change it has no effect and is not passed
to child processes.

(See https://public-inbox.org/git/alpine.DEB.2.21.1.1709131448390.4132@virtualbox/)

Instead, set it when building the test executable, following, a suggestion from
@stefantalpalaru.
https://github.com/status-im/nimbus-eth1/pull/598#discussion_r621107128

To ensure no conflict with `config.nims`, `-d:windowsNoSetStack` is used.  This
proved unnecessary in practice because the command-line option is passed to the
linker after the config file option.  But given we don't have an automated test
to verify linker behaviour, it's best not to rely on the option order, neither
how the linker treats it, or whether Nim will always send them in that order.

Testing:

This has been verified by using a smaller limit.  At 200k, all `ENABLE_EVMC=0`
OS targets passed as expected, and all `ENABLE_EVMC=1` OS targets failed with
expected kinds of errors due to stack overflow, including Windows.
(400k wasn't small enough; 32-bit x86 Windows passed on that).

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-30 01:00:31 +01:00
Jamie Lokier 6b65467af7
Sync: Use stable nodekey/enode to accelerate sync testing
Make `run-nimbus-sync` look for and use `~/.nimbus/$TESTNET/nimbus/nodekey`
during Ethereum sync tests.  This is a private key which identifies the node.

If you have created that file, its contents should be a hex nodekey, same
format as Geth.  In fact you can use Geth to generate one.  If found,
`run-nimbus-sync` will use it as the nodekey, instead of Nimbus's default,
which is a random nodekey each time it is run.

Using the same nodekey for each run allows us to add the corresponding
`enode:...` URL (public key) as a trusted peer to the dedicated Geth instances,
using Geth's `admin.addTrustedPeer`.

This ensures Geth will almost always accept our connections, which is very
helpful for sync testing, instead of waiting a long time for a good peer.

Indeed, without this we might never get a willing good peer, due to reputation
effects while working on new sync methods.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-29 21:25:26 +01:00
Jordan Hrycaj 5d0d44c38f re-named compu_helper.nim => computation.nim
why:
  exports all except one of the original computation.nim functional
  objects
2021-04-28 15:24:14 +03:00
Jordan Hrycaj a86308c079 merged contents of computations.nim int interpreter_dispatch.nim
why:
  only two public functions left: executeOpcodes() and execCallOrCreate()
  where the former one was originally in interpreter_dispatch.nim and
  the latter one calls this one.

  improves maintainability
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 49afac46b7 move dispatcher case switch from interpterer_dispatcher.nim into separate file
why:
  insulate for improving maintenance
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 028a3d6a37 removed redundant source file: interpreter.nim
why:
  works as import concentrator for state_transaction.nim for
  vm_internals.nim interface.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 0a4c34f13b removed circular import dependencies
overview:
  can be verified by running "make check_vm2 X=0" in the nimbus directory
  (be patient when running it.) the X=0 flag is necessary if there is a
  native NIM compiler which may bail out at some vendor imports.

details:
  when compiling state_transaction.nim, the nim flag vm2_enabled must
  be set in order to avoid implicit import of native VM definitions.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj caabc9c292 removed kludge and simplified sources oph_call.nim and oph_create.nim
why:
  kludge not needed anymore for oph_handlers.nim sub-sources and sources
  that rely on oph_handlers.nim (but not state_transactions.nim which
  relies on computation.nim.)
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 77518446d9 shift functions from computation.nim => compu_helper.nim
why:
  insulate exec functions in computation.nim
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 73900270db extracted macros from oph_helpers.nim => oph_gen_handlers.nim
why:
  imports mostly need to import only one of either
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 45558282f7 merged oph_*_kludge.nim sources => single oph_kludge.nim 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 69a1ee5fc8 re-named some v2state_transactions.nim to its original name without the v2
also:
  re-integrated stack_defs.nim back into stack.nim

why:
  the v2 prefix of the file name was used as a visual aid when
  comparing vm2 against vm sources
2021-04-28 15:24:14 +03:00
Jordan Hrycaj e6eee3f3a6 prepared v2state_transaction.nim so it can be compiled locally
details:
  use the -d:kludge:1 flag for syntax check
2021-04-28 15:24:14 +03:00
Jordan Hrycaj ff6921eb1a re-named some v2*.nim sources to its original name *.nim (without the v2)
why:
  the v2 prefix of the file name was used as a visual aid when
  comparing vm2 against vm sources

details:
  all renamed v2*.nim sources compile locally with the -d:kludge:1 flag
  set or without (some work with either)

  only sources not renamed yet: v2state_transactions.nim
2021-04-28 15:24:14 +03:00
Jordan Hrycaj bca6e791aa provide experimental op handler switch -d:lowmem:1 for low memory C compiler
why:
  on 32bit windows 7, there seems to be a 64k memory ceiling for the gcc
  compiler which was exceeded on some test platform.

details:
  compiling VM2 for low memory C compiler can be triggered with
  "make ENABLE_VM2LOWMEM". this comes with a ~24% longer execution time
  of the test suite against old VM and optimised VM2.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 1b3117edbd re-implemented handler-call statement by doubly nested case statement
why:
  the new implementation lost more then 25% execution time on the test
  suite when compared to the original VM. so the handler call and the
  surrounding statements have been wrapped in a big case statement similar
  to the original VM implementation. on Linux/x64, the execution time of
  the new VM2 seems to be on par with the old VM.

details:
  on Linux/x64, computed goto works and is activated with the -d:release
  flag. here the execution time of the new VM2 was tested short of 0.02%
  better than the old VM. without the computed goto, it is short of
  0.4% slower than the old VM.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 3ed234e0a1 clean up cyclic-import-breaker function stubs where possible for op handlers
why:
  using function stubs made it possible to check the syntax of an op
  handler source file by compiling this very file. this was previously
  impossible due cyclic import/include mechanism.

details:
  only oph_call.nim, oph_create.nim and subsequently op_handlers.nim
  still need the -d:kludge:1 flag for syntax check compiling. this flag
  also works with interpreter_dispatch.nim which imports op_handlers.nim.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 59d7ba1f1e print compiler warning about the VM used
why:
  handy to have confirmation about which of the three different VMs
  is activated
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 18587f5496 move setupTxContext() from v2state.nim to v2state_transactions.nim
why:
  removes circular dependency in v2state.nim which is more used than
  v2state_transactions.nim
2021-04-28 15:24:14 +03:00
Jordan Hrycaj a86bcefc7a re-named v2gas_costs.nim to its original name v2gas_costs.nim
why:
  the v2 prefix of the file name was used as a visual aid when
  comparing vm2 against vm sources
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 72b36e154b eliminated v2opcode_values, v2forks in favour of op_codes, forks_list 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 01b96df99f applies jamie's patch for eliminating recursion
original comment:
  This patch eliminates recursion entirely from the EVM when ENABLE_EVMC=0.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj b2ce6d9e70 re-arrange functions from v2computation.nim and interpreter_dispatch.nim
why:
  step towards breaking circular dependency

details:
  some functions from v2computation.nim have been extracted into
  compu_helper.nim which does not explicitly back-import
  v2computation.nim. all non recursive op handlers now import this source
  file rather than v2computation.nim.

  recursive call/create op handler still need to import v2computation.nim.

  the executeOpcodes() function from interpreter_dispatch.nim has been
  moved to v2computation.nim which allows for <import> rather than
  <include> the interpreter_dispatch.nim source.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 9b70ab5f8f update handler prototype using call-by-reference argument
why:
  this allows for passing back information which can eventually be
  used for reducing use of exceptions

caveat:
  call/create currently needs to un-capture the call-by-reference
  (wrapper) argument using the Computation reference inside
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 06b34a4a56 remove obsolete files
why:
  opcodes_impl.nim was fully replaced by functionality from
  op_handlers.nim
2021-04-28 15:24:14 +03:00
Jordan Hrycaj b388e966cc simplify interpreter_dispatch.nim code
details:
  replace generated macro loop/switch by explicit call using the
  fork/op handler matrix (encapsulated via opHandlersRun() function)
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 2fb18bf88c install new op handlers
details:
  in the source file interpreter_dispatch.nim, op handlers imported from
  opcodes_impl.nim are replaced by table entries from op_handlers.nim
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 51587208b4 verifying new op handler tables layout against original tables
why:
  the previous approach was replacing the function-lets in
  opcode_impl.nim by the particulate table handlers. the test
  functions will verify the the handler functions are sort of
  correct but not the assignments in the fork tables.

  the handler names of old and new for tables are checked here.

caveat:
  verifying tables currently takes a while at compile time.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 4e2af7937b clean up code for call handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 4ac32d360b re-integrated call op handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 1bdbfda37f re-integrated/added Create and Create2 handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 1bb6ef43a1 re-integrated handlers with op codes 0xf2/return ..0xff/selfdestruct 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 7436e516fd re-integrated/added EIP2929 handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 716bd64419 re-integrated 0x60..0xaf (push, dup, swap, log) op handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj f4bc9c4561 updated circular dependency klugde
why:
  removing <when> clauses and replacing gas calculation by stubs
  makes up for better reading of the code
2021-04-28 15:24:14 +03:00
Jordan Hrycaj d373ab6460 re-integrated 0x5# op handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 0f1c7cee43 re-integrated 0x4# op handlers 2021-04-28 15:24:14 +03:00