Commit Graph

169 Commits

Author SHA1 Message Date
KonradStaniec b43709f473
Add missing nimbus-eth2 modules (#1224) 2022-09-14 15:10:05 +02:00
KonradStaniec 9d10f8fbae
Add mpt proof verification to proxy (#1213)
* Add mpt proof verification to proxy
2022-09-06 18:14:50 +02:00
KonradStaniec caa5e009ff
Initial Binary for light client proxy (#1202)
* Initial Binary for light client proxy
2022-08-26 13:54:10 +02:00
Nikolay Mitev 1e4f138574 Add nimbus-eth2 as submodule. Add launch_local_testnet script to CI. 2022-08-18 18:06:38 +03:00
Jordan Hrycaj c123e1eb93
Updated account scheduler (#1124)
* Using `IntervalSet` type data for `LeafRange`

* Updated log ticker

* Update to `eth67`

details:
  Disabled by default, use `ENABLE_LEGACY_ETH66=0` to enable
  No support for `Get/NodeData` dialogue via eth, anymore

* Dissolved fetch/common.nim

details;
  the log/ticker part becomes ticker.nim
  the interval range management is merged into fetch.nim

* Updated account scheduler

why:
  The previous scheduler fetched each account once (for different state
  roots.) The updated scheduler re-calibrates after a change of the state
  root and potentially (until told otherwise) fetches all possible
  accounts.

* Fix `high(P)` fringe cases in `IntervalSet` handling

why:
  The `high(P)` value for a point type `P` cannot be represented with
  half open intervals `[a,b)` for a,b points of `P`. So this single value
  needs extra treatment which was slightly wrong.

* Updated docu/comments

also:
  rebased

* Update scheduler

details:
  Change the `pivot` management when creating new accounts lists. It is
  strictly increasing (and wrapping around) depending on last updated
  accounts list.
2022-06-16 09:58:50 +01:00
Jacek Sieka 1d6a9951d6
move rocksdb support to eth1 (#927)
* move rocksdb support to eth1

only used here / causes unnecessary build deps
2022-04-06 07:28:19 +02:00
Kim De Mey 8c3f210526
Add blockwalk tool to test range of blocks on Portal network (#1020) 2022-03-30 08:55:38 +02:00
Jordan Hrycaj 4696a53302
Enable optional chunked RLPx messages (#1010)
* Enable optional chunked RLPx messages

why:
  Legacy feature used by Nethermind

details:
  Disable with make flag: ENABLE_CHUNKED_RLPX=0

* Rebase & bump nim-eth

* Fix default behaviour

why:
  Got lost somehow. Comments do not match GNU make code directives.
2022-03-29 10:19:32 +01:00
Ștefan Talpalaru 51bc1cf87f
dist: precompiled binaries and Docker images (#1015)
* dist: precompiled binaries and Docker images

The builds are reproducible, the binaries are portable and statically link librocksdb.

This took some patching. Upstream PR: https://github.com/facebook/rocksdb/pull/9752

32-bit ARM is missing as a target because two different GCC versions
fail with an ICE when trying to cross-compile RocksDB. Using Clang
instead is too much trouble for a platform that nobody should be using
anyway.

(Clang doesn't come with its own target headers and libraries, can't be
easily convinced to use the ones from GCC, so it needs an fs image from
a 32-bit ARM distro - at which point I stopped caring).

* CI: disable reproducibility test
2022-03-27 13:21:15 +02:00
KonradStaniec cccd1e4640
Add tooling necessary to improve utp-testing (#931)
* Add tooling necessary to improve utp-testing
2022-01-19 16:06:23 +01:00
Kim De Mey 63b0945b52
Use json rpc client to run tests on portal testnet (#899)
Tests have also been added & adjusted. Original ones were actually
failing but unnoticed due to mistake in port passed in curl command.
2021-12-03 09:51:25 +01:00
Kim De Mey 424fd405e9
Move Portal wire protocol from nim-eth to fluffy (#749)
* Move Portal wire protocol from nim-eth to fluffy

* Rename fluffy make targets

* Move all_fluffy_tests a folder down
2021-07-09 13:34:16 +02:00
KonradStaniec a59a2c61b6
[FEATURE] Add support for handling experimental api call (#746)
* [FEATURE] Add support for handling experimental api call
2021-07-07 11:04:18 +02:00
Kim De Mey f8b3922eb5
Change name from nlpn to fluffy (#734) 2021-06-28 17:53:13 +02:00
Kim De Mey aef7a25174
Add nlpn (#714)
* Add start of nlpn client

* Fix some error handling in nlpn

* Clean-up GOARCH from nlpn.yml and add vendor to include paths
2021-06-16 21:18:45 +02:00
Jamie Lokier ecb0654da7
Whisper: Remove C and Go wrappers and Nimbus as a library
Remove the C and Go example wrappers that call Nimbus as a library, by removing
the entire `wrappers/` directory.  They are removed because they only wrap
Whisper protocol support, which has been removed as it is obsolete.

The only thing wrapped were Whisper functions, even though there were separate
`go_wrapper_example` and `go_wrapper_whisper_example` programs.  The wrappers
don't build without Whisper in Nimbus, and without it, there isn't enough left
for them to be useful examples.

Also remove support for building the whole of Nimbus as a library, because
there is nothing left using it.  These targets are gone from the Makefile:

- `wrappers`
- `wrappers-static`
- `libnimbus.so`
- `libnimbus.a`

The code isn't really gone, because it remains available in Git history.  It
may be useful someday, so a comment has been left in the Makefile for future
generations:

> This note is kept so that anyone wanting to build Nimbus as a library or call
> from C or Go will know it has been done before.  The previous working version
> can be found in Git history.  Look for the `nimbus-eth1` commit that adds
> this comment and removes `wrappers/*`.

Also worth a note, the library support was not tested on Windows, and the
shared library support was only tested on Linux.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 18:12:48 +01:00
Jordan Hrycaj bca6e791aa provide experimental op handler switch -d:lowmem:1 for low memory C compiler
why:
  on 32bit windows 7, there seems to be a 64k memory ceiling for the gcc
  compiler which was exceeded on some test platform.

details:
  compiling VM2 for low memory C compiler can be triggered with
  "make ENABLE_VM2LOWMEM". this comes with a ~24% longer execution time
  of the test suite against old VM and optimised VM2.
2021-04-28 15:24:14 +03:00
Jordan Hrycaj b4f8450968 provide identical copy of vm folder => vm2, activated by make flag ENABLE_VM2=1
why:
  vm2 enabled by ENABLE_VM2=1 behaves as vm without ENABLE_EVMC=1 until
  it doesn't in some future fatch set. this leaves some wiggle room
  to work on a vm copy without degrading the original implementation.

details:
  + additional make flag ENABLE_VM2=1 (or ENABLE_VM2=0 to explicitely disable)
  + when both flags ENABLE_EVMC=1 and ENABLE_VM2=1 are present, the former
    flag ENABLE_EVMC=1 takes precedence, this is implemented at the NIM
    compiler level for -d:evmc_enabled and -d:vm2_enabled
2021-04-23 14:04:06 +03:00
Jacek Sieka 3147df0dcd
switch to chronos metrics, remove insecure (#580)
* switch to chronos metrics, remove insecure

See https://github.com/status-im/nimbus-eth2/pull/2468

also fixes pcre linking for real, and adds some random build flags that
help nimbus-eth2 stay afloat

* fix help

* don't omit frame pointers on windows
2021-04-09 09:26:06 +02:00
Jacek Sieka ebb65e9c20
Compile without pcre (#574) 2021-04-01 09:36:13 +02:00
jangko b6ad47f3a4 fixes evmc bug and add github action job to test evmc 2021-01-20 11:50:07 +07:00
jangko 8c5c967715 bump submodules 2021-01-20 11:50:07 +07:00
jangko 91eb5db346
save some CI time by combining tools build 2020-07-21 11:28:03 +07:00
Ștefan Talpalaru 75b5d4effe
Makefile: move "nat-libs" to "deps" 2020-06-22 17:12:12 +02:00
kdeme 769228418e Remove wakunode and waku rpc code from repository 2020-05-07 20:49:14 +03:00
Ștefan Talpalaru 73e9199ebf
new initial submodule update strategy (#494)
* new initial submodule update strategy

* Azure: increase timeout

* Makefile: change comment [skip ci]
2020-04-18 14:56:40 +02:00
Ștefan Talpalaru a783b096fe
bump vendor/nimbus-build-system (#491)
* bump vendor/nimbus-build-system

- add the Nim compiler header to the Nimbus header
- also support the USE_LIBBACKTRACE env var

* "go-checks" target no longer available
2020-04-16 00:21:58 +02:00
Ștefan Talpalaru 71129cc2ab
nim-libbacktrace: don't build the C++ wrapper 2020-02-15 13:00:39 +01:00
Ștefan Talpalaru 8568ec35ea
enable -march=native by default
plus some drive-by fixes and cosmetic changes
2020-02-13 20:18:27 +01:00
Ștefan Talpalaru 06e0eb80e6
lightweight stack traces
Also converted the top-level nim.cfg into a config.nims, in the process.
2020-02-11 04:33:21 +01:00
kdeme 776f924e39 Small fixes/improvements 2020-01-23 12:39:36 +02:00
kdeme a03d0be31f Add start_network.nim to start different types of networks 2020-01-23 12:39:36 +02:00
kdeme 48391744ff Add metrics config files generation for wakusim 2020-01-23 12:39:36 +02:00
kdeme 574c5ec0af Add lightNode and logMetrics options 2020-01-23 12:39:36 +02:00
kdeme 7b80b313e4 Start implementation of waku node 2020-01-23 12:39:36 +02:00
kdeme 9472cf31a9
Remove -linkshared & update Nix file to be consistent 2019-11-27 11:06:58 +01:00
kdeme 2705826cc2
Seperate nimcache for libnimbus.so and libnimbus.a targets + keep output
arg
2019-11-26 22:48:35 +01:00
kdeme c2105d0ed7
Workaround for nim file output issue + add compilation of static whisper example 2019-11-26 16:10:37 +01:00
kdeme 23baeaa15d Trying out some Whisper RPC alike API 2019-11-19 11:02:00 +00:00
kdeme ef540fdb14 Make go wrapper_example subscribe work with c gateway function 2019-11-19 11:02:00 +00:00
Ștefan Talpalaru 35bdfbc159
Makefile: define SHELL for when submodules haven't been fetched yet [skip ci] 2019-09-10 02:48:34 +02:00
Ștefan Talpalaru d96f36c51d
fix "wrappers-static" [skip ci] 2019-09-05 14:31:20 +02:00
Ștefan Talpalaru b4007b941e
Makefile: "nat-libs" is now included in "deps" 2019-09-04 19:14:54 +02:00
Ștefan Talpalaru 2314b4d593
use nimbus-build-system
Also remove the nim-beacon-chain submodule and its specific
dependencies.
2019-08-22 02:02:32 +02:00
Ștefan Talpalaru 33494634aa
CI: LFS support for the beacon-chain tests
- Travis: switch language
- AppVeyor: disable `mingw32-make go-checks` for now
- `make update`: export GIT_LFS_SKIP_SMUDGE=1
- CI: disable the LFS script inside Makefile,
  since we run it separately (and git-lfs is not installed in the Travis
  macOS image if we found a valid cache).
2019-08-15 01:35:07 +02:00
Ștefan Talpalaru c22c1d303f
fix the `go-checks` target on macOS and add it to CI 2019-08-07 18:08:48 +02:00
Ștefan Talpalaru b3969b9531
add `git submodule sync` to `make update` 2019-08-03 22:52:08 +02:00
Ștefan Talpalaru c861539501
add "Building: vendor/go/bin/p2pd" message 2019-08-03 14:51:16 +02:00
Ștefan Talpalaru f6121c10f0
various changes:
- re-enable Nim's cache globally
- add a new "nimcache" subdir for the libraries
- make sure Go doesn't try to link libnimbus.a in the dynamically linked
  wrapper example
- always delete the static archive before recreating it
- rename wrapper.nim/.h to libnimbus.nim/.h
- better hygiene in libnimbus.h (include guards and C++ support)
- remove MainnetBootnodes copy, since we do include "config.nim" after all
2019-08-02 18:59:47 +02:00
Ștefan Talpalaru f63014bddb
libnimbus.a and statically linked wrapper examples 2019-08-02 18:13:50 +02:00
Ștefan Talpalaru c75e491d76
more renaming 2019-08-02 18:13:50 +02:00
Ștefan Talpalaru 1d7e14dc1b
refactor the C and Go wrapper build system
- moved "nimbus/api" to "wrappers"
- renamed files
- replaced the build scripts with Makefile targets
- set the rpath relative to the test binary's location so it can look
  for libnimbus.so there at runtime
- libnimbus.so.0 required on Linux, apparently
- compiled all the Nimbus code with `--app:lib`, not just one file (this
  required skipping a proc in "nimbus/config.nim" because it uses an API
  that's unavailable in libraries)
- removed static linking from the Go wrapper. It doesn't make sense at a
  global level, when using a shared Nimbus library. To selectively link
  static libraries, we should probably be specifying them as *.a. I did
  build a static libnimbus.a, as a test, but it insisted on dlopen-ing a
  shared version of itself which looked too ugly to continue.
2019-08-02 18:13:47 +02:00
Ștefan Talpalaru 6bcf4effbb
Makefile: check the Go compiler version 2019-07-11 15:34:22 +02:00
Zahary Karadjov d5fe4488de
Merge branch 'libp2p-testnet' into devel 2019-06-24 05:45:37 +03:00
Zahary Karadjov 7785b31431 For some reason, git's env variables are not defined inside the docker build env 2019-06-20 19:23:46 +03:00
Zahary Karadjov eb825051fd [Makefile] support Nimble packages with an inner src directory 2019-06-20 18:52:28 +03:00
Ștefan Talpalaru 9d8aa105ef
build_nim.sh: refactoring and CI caching 2019-06-12 13:53:34 +02:00
Ștefan Talpalaru 688fbd2145 nim_needs_rebuilding.sh and build_nim.sh 2019-06-12 11:42:31 +03:00
Ștefan Talpalaru a0066abf95
CI: cache the Nim compiler binaries between builds 2019-06-10 02:05:32 +02:00
Ștefan Talpalaru dd29d66b8c
ctags: add "func" symbols 2019-05-01 11:06:10 +02:00
Ștefan Talpalaru da5900b706
Makefile: fix the fetch-dlls target 2019-04-24 20:59:16 +02:00
Ștefan Talpalaru 39a2c5d3c7
bump submodules 2019-04-19 22:27:21 +02:00
Ștefan Talpalaru d2f0a3827d
Makefile: check if the C compiler exists 2019-04-19 13:38:37 +02:00
Ștefan Talpalaru 859968cdf2
Makefile fix 2019-04-19 12:49:14 +02:00
Ștefan Talpalaru 20d18e3af3
Windows: delete nimbus.nims on `mingw32-make update` 2019-04-17 18:37:30 +02:00
Ștefan Talpalaru 0412ecfc4f
libminiupnpc.a and libnatpmp.a: fix Windows compilation 2019-04-17 14:42:56 +02:00
Ștefan Talpalaru 8a3ef31ec4
tools and tests end up linking to the NAT libraries too 2019-04-17 04:19:56 +02:00
Ștefan Talpalaru d4aff04cbd
NAT port mapping 2019-04-17 03:56:28 +02:00
Ștefan Talpalaru b212d2ee21
Windows: copy all DLLs to vendor/nim-beacon-chain/build/ 2019-04-12 23:59:20 +02:00
Ștefan Talpalaru 7b1e031f89
we no longer need to rebuild Nimble 2019-04-12 15:33:12 +02:00
Ștefan Talpalaru b6f6795a17
rebuild the Nim compiler after the corresponding submodule is updated 2019-04-12 14:39:37 +02:00
Ștefan Talpalaru 8150199581
Makefile: low verbosity by default 2019-03-29 18:08:39 +01:00
Ștefan Talpalaru 3908c0e570
Makefile: rebuild Nimble in release mode 2019-03-29 16:29:04 +01:00
Ștefan Talpalaru 12a9904710
Makefile: echo "Building the Nim compiler." 2019-03-29 15:27:19 +01:00
Ștefan Talpalaru bb61c08fc2
Makefile: remove the "eth2_network_simulation" target
from the Nimbus Makefile, since it's now in teh nim-beacon-chain one.

Also bump vendor/nim-beacon-chain and vendor/nim-chronos
2019-03-28 16:32:03 +01:00
Ștefan Talpalaru 535cf1772c
Makefile: moved some reusable vars in common.mk for nim-beacon-chain
- also introduced TOOLS_DIRS for readability and ease  of maintenance
2019-03-26 19:23:07 +01:00
Ștefan Talpalaru 9f0c945336 CI: use a release build for the test suite (#278)
* CI: use a release build for the test suite

Makefile: the "testsuite" target is no longer needed

* rename EXTRA_NIM_PARAMS to NIMFLAGS
2019-03-16 18:54:30 -06:00
Jacek Sieka 9bd5b02f93
ci: make everything (#270)
* ci: make everything
* remove tools from test target, do avoid duplicate work in CI
2019-03-14 14:11:31 -06:00
Ștefan Talpalaru de3cf9ad2f
Makefile: don't override the test suite's loglevel by default 2019-03-12 23:50:49 +01:00
Ștefan Talpalaru 83d811ca1c Makefile: allow arbitrary nim params through EXTRA_NIM_PARAMS 2019-03-12 13:29:10 +02:00
Ștefan Talpalaru fba31263d4 Makefile: specify the Chronicles log level 2019-03-12 13:29:10 +02:00
Ștefan Talpalaru e1e56211ba
update libp2p-related submodules and its build process 2019-03-07 01:11:48 +01:00
Ștefan Talpalaru 4326d98434
testsuite: build the tools in parallel 2019-02-28 00:07:23 +01:00
Ștefan Talpalaru 36575d9451
Makefile: make "test-libp2p-daemon" a phony target 2019-02-27 13:58:55 +01:00
andri lim 14d1c28982 new member of premix tool set 2019-02-27 13:44:01 +02:00
Ștefan Talpalaru 12d18a1fd6 build p2pd and run some client tests
(with `make V=0 test-libp2p-daemon`)
2019-02-27 13:33:18 +02:00
Ștefan Talpalaru 2abf779668 fix rebase 2019-02-24 10:41:05 +02:00
Ștefan Talpalaru 9806110979 Makefile: better "github-ssh" target 2019-02-24 10:41:05 +02:00
Ștefan Talpalaru 5fcfa70be4 test-reproducibility: coloured messages 2019-02-24 10:41:05 +02:00
Ștefan Talpalaru e9565258a6 Windows: disable timestamps in PE headers 2019-02-24 10:41:05 +02:00
Ștefan Talpalaru a96976cd0a custom nimcache dir 2019-02-24 10:41:05 +02:00
Ștefan Talpalaru 05535591f1 quiet compiler building 2019-02-24 10:41:05 +02:00
Ștefan Talpalaru 0f656125ac macOS doesn't have "md5sum" 2019-02-24 10:41:05 +02:00
Ștefan Talpalaru 90dcb20d35 let the Makefile manage the symlink 2019-02-24 10:41:05 +02:00
Ștefan Talpalaru da79a7e996 very simple reproducibility test
Additional changes:
- Makefile verbosity control
- nimble.sh can now run in parallel on the same *.nimble file
- nimble.sh no longer used in the Makefile, in favour of a nimbus.nims
  symlink that eliminates race risks in parallel jobs
- nimbus.nimble takes extra params in the command line, with the caveat
  that they also apply to nim
- setCommand() replaced with exec(), to avoid splitting param strings
2019-02-24 10:41:05 +02:00
Ștefan Talpalaru e8d654ed0f Windows: allow architecture override 2019-02-24 10:41:05 +02:00
Ștefan Talpalaru 50860e3817 Makefile: delete tool binaries in the "clean" target 2019-02-24 10:41:05 +02:00
Ștefan Talpalaru 63b8ce8c00 Windows: take into account the .exe suffix 2019-02-24 10:41:05 +02:00
Ștefan Talpalaru 69440a6b4c Makefile comments 2019-02-24 10:41:05 +02:00
Ștefan Talpalaru 74196c87ea don't bother rebuilding Nimble, since we don't use it 2019-02-24 10:41:05 +02:00