Commit Graph

2262 Commits

Author SHA1 Message Date
jangko e6d7d6188c
`processArguments` now can have alternate OptParser instead of fixed one
the `processArguments` now have overloaded proc, one with opt param and one without.
the OptParser now can be passed to `opt` param.
this is useful in scenario where in test code we need to simulate something
without using real command line arguments.
2021-04-30 12:56:19 +07:00
jangko 6049073c69
fixes nimbus.sh script for hive integration related to graphql
- add `chain.rlp` import code
- add --graphql and --graphqlbind to enable graphql service
2021-04-30 12:56:19 +07:00
jangko 68e70ebdca
fixes hard fork block number initialization in `processCustomGenesisConfig`
rather than initialize it to 0, those block numbers
are initialized to high(BlockNumber). this will fix
issue when imported genesis.json doesn't contains all
forks' blockNumber.
2021-04-30 12:56:18 +07:00
jangko 287f1b2ba0
fixes `importRlpBlock` algorithm
it will skip blocks with blockNumber <= than current
head blockNumber
2021-04-30 12:56:18 +07:00
jangko 6e79fa3708
bump nim-graphql from 0.1.2 to 0.2.0
- fixes http server response status code
- fixes `__schema.types` and `__schema.directives` implementation
- fixes 'getOperation' in executor.nim
- web ui(graphiql) for http server
2021-04-30 12:56:14 +07:00
Jamie Lokier 2a7ccceb3e
EVM: Make continuation exceptions behave as they did before
The account database code is not supposed to raise exceptions in the EVM, and
the behaviour is not well defined if it does.  It isn't compliant with EVMC
spec either.  But that will be dealt with properly when the account state-cache
is dealt with, as there is some work to be done on it.

Meanwhile, if it raises in code under `chainTo` and then `(continuation)()`,
the behaviour was changed slightly by the stack-shrink patches.

Before those patches, an exception after the recursion-point was converted to
`c.setError` "Opcode Dispatch Error" in `executeOpcodes.  After, it would
propagate out, a different behaviour.  (It still correctly walked the chain of
`c.dispose()` calls to clean up.)

It's easy to restore the original behaviour just by moving the continuation
call, so let's do that.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-30 11:32:42 +07:00
Jamie Lokier 8161de1a8f
Limit stack size on Windows like other targets
We can't use `ulimit -s` to limit stack size on Windows.  Even though Bash
accepts `ulimit -s` and the numbers change it has no effect and is not passed
to child processes.

(See https://public-inbox.org/git/alpine.DEB.2.21.1.1709131448390.4132@virtualbox/)

Instead, set it when building the test executable, following, a suggestion from
@stefantalpalaru.
https://github.com/status-im/nimbus-eth1/pull/598#discussion_r621107128

To ensure no conflict with `config.nims`, `-d:windowsNoSetStack` is used.  This
proved unnecessary in practice because the command-line option is passed to the
linker after the config file option.  But given we don't have an automated test
to verify linker behaviour, it's best not to rely on the option order, neither
how the linker treats it, or whether Nim will always send them in that order.

Testing:

This has been verified by using a smaller limit.  At 200k, all `ENABLE_EVMC=0`
OS targets passed as expected, and all `ENABLE_EVMC=1` OS targets failed with
expected kinds of errors due to stack overflow, including Windows.
(400k wasn't small enough; 32-bit x86 Windows passed on that).

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-30 01:00:31 +01:00
Jamie Lokier 6b65467af7
Sync: Use stable nodekey/enode to accelerate sync testing
Make `run-nimbus-sync` look for and use `~/.nimbus/$TESTNET/nimbus/nodekey`
during Ethereum sync tests.  This is a private key which identifies the node.

If you have created that file, its contents should be a hex nodekey, same
format as Geth.  In fact you can use Geth to generate one.  If found,
`run-nimbus-sync` will use it as the nodekey, instead of Nimbus's default,
which is a random nodekey each time it is run.

Using the same nodekey for each run allows us to add the corresponding
`enode:...` URL (public key) as a trusted peer to the dedicated Geth instances,
using Geth's `admin.addTrustedPeer`.

This ensures Geth will almost always accept our connections, which is very
helpful for sync testing, instead of waiting a long time for a good peer.

Indeed, without this we might never get a willing good peer, due to reputation
effects while working on new sync methods.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-29 21:25:26 +01:00
Jordan Hrycaj 5d0d44c38f re-named compu_helper.nim => computation.nim
why:
  exports all except one of the original computation.nim functional
  objects
2021-04-28 15:24:14 +03:00
Jordan Hrycaj a86308c079 merged contents of computations.nim int interpreter_dispatch.nim
why:
  only two public functions left: executeOpcodes() and execCallOrCreate()
  where the former one was originally in interpreter_dispatch.nim and
  the latter one calls this one.

  improves maintainability
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 49afac46b7 move dispatcher case switch from interpterer_dispatcher.nim into separate file
why:
  insulate for improving maintenance
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 028a3d6a37 removed redundant source file: interpreter.nim
why:
  works as import concentrator for state_transaction.nim for
  vm_internals.nim interface.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 0a4c34f13b removed circular import dependencies
overview:
  can be verified by running "make check_vm2 X=0" in the nimbus directory
  (be patient when running it.) the X=0 flag is necessary if there is a
  native NIM compiler which may bail out at some vendor imports.

details:
  when compiling state_transaction.nim, the nim flag vm2_enabled must
  be set in order to avoid implicit import of native VM definitions.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj caabc9c292 removed kludge and simplified sources oph_call.nim and oph_create.nim
why:
  kludge not needed anymore for oph_handlers.nim sub-sources and sources
  that rely on oph_handlers.nim (but not state_transactions.nim which
  relies on computation.nim.)
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 77518446d9 shift functions from computation.nim => compu_helper.nim
why:
  insulate exec functions in computation.nim
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 73900270db extracted macros from oph_helpers.nim => oph_gen_handlers.nim
why:
  imports mostly need to import only one of either
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 45558282f7 merged oph_*_kludge.nim sources => single oph_kludge.nim 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 69a1ee5fc8 re-named some v2state_transactions.nim to its original name without the v2
also:
  re-integrated stack_defs.nim back into stack.nim

why:
  the v2 prefix of the file name was used as a visual aid when
  comparing vm2 against vm sources
2021-04-28 15:24:14 +03:00
Jordan Hrycaj e6eee3f3a6 prepared v2state_transaction.nim so it can be compiled locally
details:
  use the -d:kludge:1 flag for syntax check
2021-04-28 15:24:14 +03:00
Jordan Hrycaj ff6921eb1a re-named some v2*.nim sources to its original name *.nim (without the v2)
why:
  the v2 prefix of the file name was used as a visual aid when
  comparing vm2 against vm sources

details:
  all renamed v2*.nim sources compile locally with the -d:kludge:1 flag
  set or without (some work with either)

  only sources not renamed yet: v2state_transactions.nim
2021-04-28 15:24:14 +03:00
Jordan Hrycaj bca6e791aa provide experimental op handler switch -d:lowmem:1 for low memory C compiler
why:
  on 32bit windows 7, there seems to be a 64k memory ceiling for the gcc
  compiler which was exceeded on some test platform.

details:
  compiling VM2 for low memory C compiler can be triggered with
  "make ENABLE_VM2LOWMEM". this comes with a ~24% longer execution time
  of the test suite against old VM and optimised VM2.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 1b3117edbd re-implemented handler-call statement by doubly nested case statement
why:
  the new implementation lost more then 25% execution time on the test
  suite when compared to the original VM. so the handler call and the
  surrounding statements have been wrapped in a big case statement similar
  to the original VM implementation. on Linux/x64, the execution time of
  the new VM2 seems to be on par with the old VM.

details:
  on Linux/x64, computed goto works and is activated with the -d:release
  flag. here the execution time of the new VM2 was tested short of 0.02%
  better than the old VM. without the computed goto, it is short of
  0.4% slower than the old VM.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 3ed234e0a1 clean up cyclic-import-breaker function stubs where possible for op handlers
why:
  using function stubs made it possible to check the syntax of an op
  handler source file by compiling this very file. this was previously
  impossible due cyclic import/include mechanism.

details:
  only oph_call.nim, oph_create.nim and subsequently op_handlers.nim
  still need the -d:kludge:1 flag for syntax check compiling. this flag
  also works with interpreter_dispatch.nim which imports op_handlers.nim.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 59d7ba1f1e print compiler warning about the VM used
why:
  handy to have confirmation about which of the three different VMs
  is activated
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 18587f5496 move setupTxContext() from v2state.nim to v2state_transactions.nim
why:
  removes circular dependency in v2state.nim which is more used than
  v2state_transactions.nim
2021-04-28 15:24:14 +03:00
Jordan Hrycaj a86bcefc7a re-named v2gas_costs.nim to its original name v2gas_costs.nim
why:
  the v2 prefix of the file name was used as a visual aid when
  comparing vm2 against vm sources
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 72b36e154b eliminated v2opcode_values, v2forks in favour of op_codes, forks_list 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 01b96df99f applies jamie's patch for eliminating recursion
original comment:
  This patch eliminates recursion entirely from the EVM when ENABLE_EVMC=0.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj b2ce6d9e70 re-arrange functions from v2computation.nim and interpreter_dispatch.nim
why:
  step towards breaking circular dependency

details:
  some functions from v2computation.nim have been extracted into
  compu_helper.nim which does not explicitly back-import
  v2computation.nim. all non recursive op handlers now import this source
  file rather than v2computation.nim.

  recursive call/create op handler still need to import v2computation.nim.

  the executeOpcodes() function from interpreter_dispatch.nim has been
  moved to v2computation.nim which allows for <import> rather than
  <include> the interpreter_dispatch.nim source.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 9b70ab5f8f update handler prototype using call-by-reference argument
why:
  this allows for passing back information which can eventually be
  used for reducing use of exceptions

caveat:
  call/create currently needs to un-capture the call-by-reference
  (wrapper) argument using the Computation reference inside
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 06b34a4a56 remove obsolete files
why:
  opcodes_impl.nim was fully replaced by functionality from
  op_handlers.nim
2021-04-28 15:24:14 +03:00
Jordan Hrycaj b388e966cc simplify interpreter_dispatch.nim code
details:
  replace generated macro loop/switch by explicit call using the
  fork/op handler matrix (encapsulated via opHandlersRun() function)
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 2fb18bf88c install new op handlers
details:
  in the source file interpreter_dispatch.nim, op handlers imported from
  opcodes_impl.nim are replaced by table entries from op_handlers.nim
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 51587208b4 verifying new op handler tables layout against original tables
why:
  the previous approach was replacing the function-lets in
  opcode_impl.nim by the particulate table handlers. the test
  functions will verify the the handler functions are sort of
  correct but not the assignments in the fork tables.

  the handler names of old and new for tables are checked here.

caveat:
  verifying tables currently takes a while at compile time.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 4e2af7937b clean up code for call handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 4ac32d360b re-integrated call op handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 1bdbfda37f re-integrated/added Create and Create2 handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 1bb6ef43a1 re-integrated handlers with op codes 0xf2/return ..0xff/selfdestruct 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 7436e516fd re-integrated/added EIP2929 handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 716bd64419 re-integrated 0x60..0xaf (push, dup, swap, log) op handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj f4bc9c4561 updated circular dependency klugde
why:
  removing <when> clauses and replacing gas calculation by stubs
  makes up for better reading of the code
2021-04-28 15:24:14 +03:00
Jordan Hrycaj d373ab6460 re-integrated 0x5# op handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj 0f1c7cee43 re-integrated 0x4# op handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj a9719f2dae added comments for better readability
and:
  reduced some import clause arguments
2021-04-28 15:24:14 +03:00
Jordan Hrycaj fb94aa8a35 re-integrated 0x3# op handlers 2021-04-28 15:24:14 +03:00
Jordan Hrycaj fda676062f integrated current op handlers into opcodes_impl.nim (tbc.)
why:
  integration tests will verify op code handlers which wher rephrased
  from the very opcodes_impl module into the handler tables.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 5e7d4ac9c5 experimental op handlers table (tbc.)
details:
  the op handler table is accessible via op_handlers.nim module

  op handler function implementations are found in the op_handlers/
  sub-directory

kludge:
  for development and pre-testing, any new module can be individually
  compiled setting the kludge flag using -d:kludge:1. this causes some
  proc/func replacements in turn allowing for omitting imports that would
  otherwise cause a circular dependency. otherwise individual compilation
  would fail.

  in order to prove the overall correctness of the code, the
  op_handlers.nim is imported by opcodes_impl.nim when compiling all,
  nimbus or test.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj f37591ca35 need to remove vm2_defined sentinel
why:
  subsequent development will compile sources as main without setting
  the vm2_enabled flag. also, the doc generator would fail an vm2 without
  setting the flag for the vm2 files.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj fd7b1bd040 update doc generator
why:
  generally, there is no role for libbacktrace when docs are generated

  for vm2, undo settings of config.nim and provide the "kludge" flag, so
  circular import/include dependencies can be taken care of (not only)
  for generating docs
2021-04-28 15:24:14 +03:00
Jordan Hrycaj 5c01b3548d extract forks definition (all but rename v2forks.nim)
why:
  new name forks_list.nim file name matches additional documentation
  file names.

details:
  v2forks.nim remains a hollowed out shell serving as interface file.
2021-04-28 15:24:14 +03:00