Commit Graph

140 Commits

Author SHA1 Message Date
jangko 69f2a0f95a
config: replace stdlib parseOpt with nim-confutils
fixes #581
2021-09-18 17:34:46 +07:00
jangko 48d497580a
config: remove last instance of getConfiguration usage from nimbus code
this is a preparation for migration to confutils based config
although there is still some getConfiguration usage in tests code
it will be removed after new config arrived
2021-09-08 21:25:14 +07:00
jangko c9cfebfa97
config: rearrange getConfiguration usage
avoid using getConfiguration inside object construction and
replace it with passing suitable param
2021-09-08 08:07:10 +07:00
jangko 9108301eef
config: remove global rng from NimbusConfiguration
move the rng to EthContext
2021-09-07 22:02:29 +07:00
jangko 34972c6cea
config: remove accounts management from NimbusConfiguration
a new AccountsManager and EthContext is created for managing
keystore and accounts

this is a preparation for new config using ConfUtils
2021-09-07 22:02:29 +07:00
jangko 14d2edcb26
chain config preset: add london block number
MainNet     12_965_000
RopstenNet  10_499_401
RinkebyNet   8_897_988
GoerliNet    5_062_605
2021-09-02 12:24:04 +07:00
jangko 77092641b5
add websocket rpc server 2021-08-06 07:32:19 +07:00
jangko 5e87624315
config: copy chainId to networkid if networkid not set in cli
although they are technically different, but in reality,
many networks are using the same id for ChainId dan NetworkId.
in this commit, we set networkid from config file's chainId.
2021-08-06 07:31:02 +07:00
jangko 1da4346295
config: fixes bug networkid parser
previously it mistakenly parse into the `result`
now it correctly parse networkId into res.
2021-08-06 07:31:01 +07:00
jangko c69e57e5d4
config: fix config file parser
- allow clique period and epoch to be configured via config file
- this also activate poaEngine mode
- remove clique period configuration from cli to reduce confusion
- fix #786
2021-08-06 07:30:53 +07:00
jangko c142119487
add new command line options: ws, wsbind, and wsapi
new command line options:
  --ws                    Enable the Websocket JSON-RPC server
  --wsbind:<value>        Set address:port pair(s) (comma-separated) Websocket JSON-RPC server will bind to (default: localhost:8546)
  --wsapi:<value>         Enable specific set of Websocket RPC API from list (comma-separated) (available: eth, debug)

fixes #770
2021-07-31 19:27:13 +07:00
jangko 6015a4e029
add new command line options: clique-period, engine-signer, and import-key
new command line options:
  --clique-period:<value> Enables clique support. value is block time in seconds
  --engine-signer:<value> Enables mining. value is EthAddress in hex
  --import-key:<path>     Import unencrypted 32 bytes hex private key file

fixes #771
2021-07-31 19:16:30 +07:00
Jordan Hrycaj ca07c40a48
Fearture/poa clique tuning (#765)
* Provide API

details:
  API is bundled via clique.nim.

* Set extraValidation as default for PoA chains

why:
  This triggers consensus verification and an update of the list
  of authorised signers. These signers are integral part of the
  PoA block chain.

todo:
  Option argument to control validation for the nimbus binary.

* Fix snapshot state block number

why:
  Using sub-sequence here, so the len() function was wrong.

* Optional start where block verification begins

why:
  Can speed up time building loading initial parts of block chain. For
  PoA, this allows to prove & test that authorised signers can be
  (correctly) calculated starting at any point on the block chain.

todo:
  On Goerli around blocks #193537..#197568, processing time increases
  disproportionally -- needs to be understand

* For Clique test, get old grouping back (7 transactions per log entry)

why:
  Forgot to change back after troubleshooting

* Fix field/function/module-name misunderstanding

why:
  Make compilation work

* Use eth_types.blockHash() rather than utils.hash() in Clique modules

why:
  Prefer lib module

* Dissolve snapshot_misc.nim

details:
  .. into clique_verify.nim (the other source file clique_unused.nim
  is inactive)

* Hide unused AsyncLock in Clique descriptor

details:
  Unused here but was part of the Go reference implementation

* Remove fakeDiff flag from Clique descriptor

details:
  This flag was a kludge in the Go reference implementation used for the
  canonical tests. The tests have been adapted so there is no need for
  the fakeDiff flag and its implementation.

* Not observing minimum distance from epoch sync point

why:
  For compiling PoA state, the go implementation will walk back to the
  epoch header with at least 90000 blocks apart from the current header
  in the absence of other synchronisation points.

  Here just the nearest epoch header is used. The assumption is that all
  the checkpoints before have been vetted already regardless of the
  current branch.

details:
  The behaviour of using the nearest vs the minimum distance epoch is
  controlled by a flag and can be changed at run time.

* Analysing processing time (patch adds some debugging/visualisation support)

why:
  At the first half million blocks of the Goerli replay, blocks on the
  interval #194854..#196224 take exceptionally long to process, but not
  due to PoA processing.

details:
  It turns out that much time is spent in p2p/excecutor.processBlock()
  where the elapsed transaction execution time is significantly greater
  for many of these blocks.

  Between the 1371 blocks #194854..#196224 there are 223 blocks with more
  than 1/2 seconds execution time whereas there are only 4 such blocks
  before and 13 such after this range up to #504192.

* fix debugging symbol in clique_desc (causes CI failing)

* Fixing canonical reference tests

why:
  Two errors were introduced earlier but ovelooked:
   1. "Remove fakeDiff flag .." patch was incomplete
   2. "Not observing minimum distance .." introduced problem w/tests 23/24

details:
  Fixing 2. needed to revert the behaviour by setting the
  applySnapsMinBacklog flag for the Clique descriptor. Also a new
  test was added to lock the new behaviour.

* Remove cruft

why:
  Clique/PoA processing was intended to take place somewhere in
  executor/process_block.processBlock() but was decided later to run
  from chain/persist_block.persistBlock() instead.

* Update API comment

* ditto
2021-07-30 15:06:51 +01:00
jangko 5159ad7aac
preparation for London hard fork
This preparation is needed for subsequent
EIPs included in London.

- Add London to Fork enum
- Block number to fork
- Parsing London fork in chain config
- Prepare gas costs table for London
- Prepare EVM opcode dispatcher for London
- Block rewards for London
- Prepare hive script for London
2021-06-29 07:34:45 +07:00
Jamie Lokier aee0fe39d2
EVM: Remove `vm_types2` everywhere, use common forks list instead
File `vm_types2` is obsolete.  Remove this file and divert all imports to the
common forks list outside the EVM, or in some cases they don't need it anyway.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-08 15:36:31 +01:00
Jamie Lokier 613f06e61c
Whisper: Remove all the main Whisper code (config, startup, RPC etc)
This is the main patch which removes Whisper code from `nimbus-eth1` code.
It removes all configuration, help, startup, JSON-RPC calls and most types.

Note, there is still Whisper functionality in `nim-eth`.  Also, the "wrapper"
under `wrappers/` isn't dealt with by this change, but it's not built by
default (and might not currently work).

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 18:12:48 +01:00
Jamie Lokier 774f697c73
Whisper: Disable Whisper (Shh) protocol by default
This commit turns Whisper off by default, without changing anything else.

So this can be cherry-picked if you just want to disable Whisper without
removing it.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 18:12:47 +01:00
jangko 396f3e9909
add missing poaEngine configuration in config.nim
later we will use real engine configuration it if become available to us
2021-05-24 14:35:47 +07:00
jangko b82061bf46
don't mix customBootNodes and bootNodes usage 2021-05-20 14:04:17 +07:00
jangko a0d10f5728
drop PublicNetwork enum usage and replace it with NetworkId
we cannot limit the `--networkid` switch to values available in
`PublicNetwork` enum. it should able to accept very wide range of
custom NetworkId.
2021-05-20 14:04:16 +07:00
jangko f2491e6307
fixes crappy custom genesis and chain config parser
instead of using stdlib/json, now we switch to json_serialization
the result is much tidier code and more robust when parsing
optional fields.

fixes #635
2021-05-13 16:04:08 +07:00
jangko a57ac65c8c
fixes --customnetwork parser
now it can ignore an optional fork e.g. MuirGlacier
2021-05-12 19:05:32 +07:00
jangko 97f4226171
update berlin fork number in config.nim
also update test_forkid because of berlin changes
2021-05-12 17:24:27 +07:00
jangko e6d7d6188c
`processArguments` now can have alternate OptParser instead of fixed one
the `processArguments` now have overloaded proc, one with opt param and one without.
the OptParser now can be passed to `opt` param.
this is useful in scenario where in test code we need to simulate something
without using real command line arguments.
2021-04-30 12:56:19 +07:00
jangko 68e70ebdca
fixes hard fork block number initialization in `processCustomGenesisConfig`
rather than initialize it to 0, those block numbers
are initialized to high(BlockNumber). this will fix
issue when imported genesis.json doesn't contains all
forks' blockNumber.
2021-04-30 12:56:18 +07:00
Jamie Lokier 90cefc7a3d
Shell: Update help text to match reality
Capitalisation:
- The option is lower case `--logmetrics` but help said `--logMetrics`
- Same for `--logmetricsiterval`
- Same for `--metricsserver` and `--metricsserverport`

Ethereum network selection:
- Moved out into their own, cleaner help section
- Added help for `--mainnet`, `--goerli` and `--kovan`
- Moved `--networkid` and `--customnetwork` to this section as well

Other:
- Reworded or formatted some help lines for clarity and consistency

Changed options:
- Renamed `--metricserver` to `--metrics`
- Renamed `--matricsserverport` to `--metricsport`
- Removed Morden network; this didn't have an option, but could be
  selected with `--networkid:2` and then fail to work

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-27 11:11:13 +01:00
jangko 34536b0d25
add --graphql and --graphqlbind to cli parser 2021-04-24 10:58:05 +07:00
Jacek Sieka 3147df0dcd
switch to chronos metrics, remove insecure (#580)
* switch to chronos metrics, remove insecure

See https://github.com/status-im/nimbus-eth2/pull/2468

also fixes pcre linking for real, and adds some random build flags that
help nimbus-eth2 stay afloat

* fix help

* don't omit frame pointers on windows
2021-04-09 09:26:06 +02:00
Jordan Hrycaj 2eb46ca221
Merge pull request #573 from status-im/feature/isolate-evms
Feature/isolate evms
2021-04-08 08:00:34 +01:00
jangko a923016a12
allow missing chainId in chain config 2021-04-08 08:52:41 +07:00
Jordan Hrycaj 00ba7a2718
merge vm_forks and vm_opcode_values => vm_type2
why:
  all types, but they cannot be merged int vm_types because of a circular
  dependency.
2021-03-31 17:53:15 +01:00
Jordan Hrycaj 3a3e4d5707
provide vm_forks as import/export wrapper
details:
  moved original vm/interpreter/vm_forks.nim => vm/interpreter/nvm_forks.nim
2021-03-31 16:03:34 +01:00
jangko 2cd081495b implement '--import': import rlp encoded block(s), validate, write to db and quit 2021-03-23 10:37:00 +07:00
jangko 8e4b917fd3 allow custom net parser to skip chain config and only parse genesis data 2021-02-16 10:37:42 +07:00
jangko 8486c1834c fixes #558, use distinct uint for ChainId and NetworkId to prevent confusion 2021-02-14 11:01:27 +07:00
jangko 2566cebfe0 fixes #548, custom network json parser now behaves like geth 2021-02-11 18:19:33 +07:00
jangko c68aa47464
fixes fork comparison related code 2020-11-19 11:59:53 +07:00
jangko f82dff64fa
implement more eth rpc and keystore management 2020-07-23 14:54:32 +07:00
jangko 032c29288a
fix git revision string in config.nim 2020-07-22 18:40:12 +07:00
jangko 165f9fea2e
reduce warnings 2020-07-21 13:15:06 +07:00
jangko ab5c763a84
move rng to configuration 2020-07-21 00:16:59 +07:00
jangko 845671bf0a
fix compilation error following breaking changes in nim-eth 2020-07-20 13:50:05 +07:00
jangko 12ddfee675
fix compilation error related to lib-secp256k1 changes 2020-06-24 17:07:33 +07:00
Jacek Sieka 4ade5797ee
rlp: don't use ranges / experimental features (#495) 2020-04-20 20:12:44 +02:00
Ștefan Talpalaru a783b096fe
bump vendor/nimbus-build-system (#491)
* bump vendor/nimbus-build-system

- add the Nim compiler header to the Nimbus header
- also support the USE_LIBBACKTRACE env var

* "go-checks" target no longer available
2020-04-16 00:21:58 +02:00
andri lim e324a7342d
move toFork to config.nim 2020-04-12 17:07:09 +07:00
andri lim a864967c4f
more to update chain config [skip ci] 2020-04-11 16:59:46 +07:00
andri lim d69ede6060 add goerli testnet genesis data[skip ci] 2020-04-09 19:13:17 +03:00
andri lim 1ad0dcd586
fix testnet config 2020-04-08 16:53:26 +07:00
Jacek Sieka c1899711c9
keep up with nim-eth (#483)
* keep up with nim-eth
2020-04-07 11:53:50 +02:00
Jacek Sieka 1d472cf090
Eth keys (#482)
* bump nim-eth, fix deprecated calls
2020-04-05 15:12:48 +02:00
acolytec3 45cda8bab0 Add support for custom genesis blocks 2020-01-20 06:35:35 -05:00
kdeme 9aae2f9463 No more blind except 2019-12-05 13:02:21 +01:00
Ștefan Talpalaru 5a18227562
add Git revision to NimbusHeader 2019-11-10 18:44:40 +01:00
Ștefan Talpalaru c1621a518b
bump vendor/nim-metrics 2019-11-10 18:28:55 +01:00
kdeme 1e716a960a Refactor bootnodes and whispernodes 2019-08-16 08:51:40 +02:00
Ștefan Talpalaru 1d7e14dc1b
refactor the C and Go wrapper build system
- moved "nimbus/api" to "wrappers"
- renamed files
- replaced the build scripts with Makefile targets
- set the rpath relative to the test binary's location so it can look
  for libnimbus.so there at runtime
- libnimbus.so.0 required on Linux, apparently
- compiled all the Nimbus code with `--app:lib`, not just one file (this
  required skipping a proc in "nimbus/config.nim" because it uses an API
  that's unavailable in libraries)
- removed static linking from the Go wrapper. It doesn't make sense at a
  global level, when using a shared Nimbus library. To selectively link
  static libraries, we should probably be specifying them as *.a. I did
  build a static libnimbus.a, as a test, but it insisted on dlopen-ing a
  shared version of itself which looked too ugly to continue.
2019-08-02 18:13:47 +02:00
kdeme f2f2f57040 Add new bootnodes (#353)
* Add new bootnodes and remove legacy bootnodes

* Clean-up network selection
2019-07-30 11:00:55 +02:00
Ștefan Talpalaru 409d771a50 metrics: put the HTTP server under -d:insecure
- also fix an option parsing bug
- bump vendor/nim-eth and vendor/nim-metrics
2019-07-19 15:17:51 +03:00
Ștefan Talpalaru 0f3d05bf68 metrics: HTTP server disabled by default
- metric logging added
- new Nimbus options: --metricsServer, --metricsServerPort:<value>,
  --logMetrics, --logMetricsInterval:<value>
2019-07-19 15:17:51 +03:00
Ștefan Talpalaru 2d9f62530b metrics HTTP server replacing the periodic "stats" logging
and example prometheus.yml file to use it as a scraping target
2019-07-19 15:17:51 +03:00
kdeme f62816c4f3 Use of BlockNumber to be able to switch between UInt256 and int64 2019-07-10 20:56:05 +02:00
kdeme 46fb9dc1b2 Add staticnodes, protocols, and whisper config options (#319) 2019-04-23 07:49:49 -06:00
Ștefan Talpalaru d4aff04cbd
NAT port mapping 2019-04-17 03:56:28 +02:00
Ștefan Talpalaru 50504cf553 Nimbus: runtime log level selection and logfile option 2019-03-26 13:20:01 +02:00
andri lim 6c891d08e6
put doAssert(false) at nimbus configuration cmdEnd 2019-03-19 08:55:30 +07:00
Mamy Ratsimbazafy e1812eb7ce
Prepare for AD2 rebranding to chronos (#225)
* Prepare for AD2 rebranding to chronos - https://github.com/status-im/nim-asyncdispatch2/pull/20

* fix nimble path to chronos

* nim-eth and nim-chronos merged into nimble packages

* fix nimble package name
2019-02-06 19:03:36 +01:00
Yuriy Glukhov 481c6cf4ed Use nim-eth (#224) 2019-02-05 20:15:50 +01:00
Zahary Karadjov c98e337090 remove the need for gcsafe overrides 2019-01-15 18:59:24 +02:00
Ștefan Talpalaru a431ceed11 add "--threads:on" to tests and main binary
- some "funcs" were no longer considered side-effect free, so I made
  them procs
- added {.base.} to some base methods to avoid a deprecation warning
2019-01-15 18:59:24 +02:00
andri lim 9f590a22e8 fix persist tool 2019-01-15 15:30:25 +02:00
Ștefan Talpalaru 89044b84e7 update Nimbus header and refine db backend picking
- dynamically generated copyright year interval
- added the db backend to the header
- documented the db-backend-changing define, made it case insensitive
  and ensured wrong values would trigger compilation errors
2019-01-07 01:21:29 +02:00
andri lim 40f094f6d5 fix datadir typo 2018-12-28 15:24:57 +07:00
Ștefan Talpalaru 115843487c
general-state testsuite changes and better fork selection
- skipped the tests in allowedFailInCurrentBuild()
- replaced doAssert() with check() in testFixtureIndexes() so we can see
  both hash values on failure
- checking filename extension for JSON tests to avoid editor swap files
- replaced the duplicated block values in the main net's ChainConfig
  with values from forkBlocks
- allowed overriding the current fork in computations, because the old
  strategy of only looking at the block number doesn't work with JSON tests
  where the block number is usually 1
- explicitly pass the fork to gasCosts() and use it for conditional cost
  calculation
- fixed a logic error in the CREATE opcode
- fixed VM selection based on current fork in updateOpcodeExec()
- single point of control for supported forks in tests (just one fork, at the
  moment)
- 44 new test failures (that were probably passing for the wrong reasons)
2018-12-07 18:18:29 +01:00
andri lim 4885445735 add prune cli switch 2018-12-05 21:58:34 +07:00
andri lim 21070d510d add TransactionKey to persistTransactions and add more fat to debug_traceTransaction 2018-11-28 09:25:36 +07:00
andri lim a2b9167e2b extend rpc api cli and implement debug_traceTransaction stub 2018-11-22 13:40:09 +07:00
Zahary Karadjov 41adca1abb Minimal changes required for building the LES branch 2018-10-16 03:10:01 +03:00
Yuriy Glukhov 7a732040ba Added comments 2018-09-26 13:31:48 +03:00
Yuriy Glukhov 291fddee71 Better cmdline flags handling 2018-09-26 13:31:48 +03:00
Zahary Karadjov 4e323df363 Implement a --datadir command-line options
By default, the database files will be written in the
platform-specific application data folder:

$HOME/AppData/Roaming/Nimbus/DB
$HOME/Library/Application Support/Nimbus/DB
$HOME/.cache/nimbus/db
2018-09-25 02:06:20 +03:00
Yuriy Glukhov 0a247a58f3 New nim compat 2018-09-06 13:16:13 +03:00
Yuriy Glukhov db62ef132e More comments addressed 2018-08-03 14:10:07 +03:00
Yuriy Glukhov 04f404fb22 Fixed toPublicNetwork 2018-08-02 11:18:12 +03:00
Yuriy Glukhov c4f4a37d2c Added genesis initialization 2018-08-01 15:50:44 +03:00
Yuriy Glukhov f6f355b6c8 Workaround for nim issue 7833 2018-07-26 16:14:16 +03:00
Zahary Karadjov 7f4d97dca0 add the ETH and LES protocol implementations to the nimbus build 2018-07-23 00:34:43 +03:00
cheatfate 533bd6caf4 Integrated P2P & RPC servers. 2018-06-20 20:27:32 +03:00
Zahary Karadjov eacf13ed97 ethp2p was renamed to eth_p2p 2018-06-16 22:14:31 +03:00
Yuriy Glukhov 51b572d8b8 Some renames 2018-05-28 13:22:28 +03:00