Commit Graph

77 Commits

Author SHA1 Message Date
Kim De Mey 392a540eb5
Remove unused import of config to avoid select_backend db import (#1018)
* Remove unused import of config to avoid select_backend db import

- Importing nimbus-eth1 config.nim causes import of select_backend
which will default cause an import of kvstore_rocksdb and thus a
require rocksdb. Remove unused one to avoid rocksdb dependency
for Fluffy.
- Remove some whitespace in bridge_client (to make fluffy CI
trigger for sure).

* Use specific cache keys for fluffy CI workflow

* Disable Fluffy CI reproducibility test
2022-03-29 15:51:48 +02:00
Ștefan Talpalaru 51bc1cf87f
dist: precompiled binaries and Docker images (#1015)
* dist: precompiled binaries and Docker images

The builds are reproducible, the binaries are portable and statically link librocksdb.

This took some patching. Upstream PR: https://github.com/facebook/rocksdb/pull/9752

32-bit ARM is missing as a target because two different GCC versions
fail with an ICE when trying to cross-compile RocksDB. Using Clang
instead is too much trouble for a platform that nobody should be using
anyway.

(Clang doesn't come with its own target headers and libraries, can't be
easily convinced to use the ones from GCC, so it needs an fs image from
a 32-bit ARM distro - at which point I stopped caring).

* CI: disable reproducibility test
2022-03-27 13:21:15 +02:00
Kim De Mey 0060462dc0
Sharing block header data around in a Portal history network (PoC) (#960)
* Sharing block header data around in a Portal history network (PoC)

- Rework PortalStream to have an instance per PortalProtocol (this
needs to be improved eventually). Each instance uses the same
UtpDiscv5Protocol instance.
- Add processContent on receival of accepted data
- Add dumb neighborhoodGossip: dumb in the sense that it only
offers one piece of content at a time.
- Add to / adjust populate_db to also allow for propagation of
the data and add debug rpc call: portal_history_propagate
- Add eth_rpc_client
- Add eth_getBlockbyHash (no txs or uncles) to eth API
- Add additional test to test_portal_testnet which loads 5 block
headers to 1 node, and offers this data to few nodes, which should
propagate it over the network further. Next query every node for
this data.

* Adjust paths on which Fluffy CI is triggered

* Add documentation on the local testnet
2022-02-11 14:43:10 +01:00
KonradStaniec cf249109fa
Add utp test to ci (#952) 2022-02-03 13:11:54 +01:00
Jordan Hrycaj 261c0b51a7
Redesign of BaseVMState descriptor (#923)
* Redesign of BaseVMState descriptor

why:
  BaseVMState provides an environment for executing transactions. The
  current descriptor also provides data that cannot generally be known
  within the execution environment, e.g. the total gasUsed which is
  available not before after all transactions have finished.

  Also, the BaseVMState constructor has been replaced by a constructor
  that does not need pre-initialised input of the account database.

also:
  Previous constructor and some fields are provided with a deprecated
  annotation (producing a lot of noise.)

* Replace legacy directives in production sources

* Replace legacy directives in unit test sources

* fix CI (missing premix update)

* Remove legacy directives

* chase CI problem

* rebased

* Re-introduce 'AccountsCache' constructor optimisation for 'BaseVmState' re-initialisation

why:
  Constructing a new 'AccountsCache' descriptor can be avoided sometimes
  when the current state root is properly positioned already. Such a
  feature existed already as the update function 'initStateDB()' for the
  'BaseChanDB' where the accounts cache was linked into this desctiptor.

  The function 'initStateDB()' was removed and re-implemented into the
  'BaseVmState' constructor without optimisation. The old version was of
  restricted use as a wrong accounts cache state would unconditionally
  throw an exception rather than conceptually ask for a remedy.

  The optimised 'BaseVmState' re-initialisation has been implemented for
  the 'persistBlocks()' function.

also:
  moved some test helpers to 'test/replay' folder

* Remove unused & undocumented fields from Chain descriptor

why:
  Reduces attack surface in general & improves reading the code.
2022-01-18 16:19:32 +00:00
Kim De Mey 73b4a73423
Bump nim-eth and nim-ssz-serialization vendor modules (#926)
- nim-eth: uTP updates, ENR improvemens and extra discv5 metrics
- nim-ssz-serialization: require BitList pairs
- update github actions cache keys to invalidate cache
2022-01-12 21:41:23 +01:00
Kim De Mey 903350bdde
Add local testnet script and required json-rpc calls (#891)
- Add basic discv5 and portal json-rpc calls and activate them in
fluffy
- Renames in the rpc folder
- Add local testnet script and run this script in CI
- bump nim-eth
2021-11-24 08:45:55 +01:00
jangko 2b6658b0d6
ci: clear github actions cache in linux/macos runner
it seemed that github actions macOs image have changed
(version changed from 20211002.1 to 20211011.2)
which made cached rocksdb not findable.
2021-10-20 08:27:53 +07:00
Jordan Hrycaj ec9354d2d0
Jordan/poa voting header (#782)
* Provide PoA voting header generator

why:
  Handy for hive/smoke test

details:
  Header generator is a re-implementation of the generator previously
  used for the canonical reference tests.

* try fixing ci out-of-mem condition

why:
  for some reason, the ci began behaving like a real win7/i386 machine
  where gcc is limited to 64k optimiser space

* fix comments, typos ..
2021-08-03 08:15:32 +01:00
Kim De Mey 424fd405e9
Move Portal wire protocol from nim-eth to fluffy (#749)
* Move Portal wire protocol from nim-eth to fluffy

* Rename fluffy make targets

* Move all_fluffy_tests a folder down
2021-07-09 13:34:16 +02:00
KonradStaniec a59a2c61b6
[FEATURE] Add support for handling experimental api call (#746)
* [FEATURE] Add support for handling experimental api call
2021-07-07 11:04:18 +02:00
Kim De Mey f8b3922eb5
Change name from nlpn to fluffy (#734) 2021-06-28 17:53:13 +02:00
Kim De Mey e7745a6456
Add nlpn readme (#718)
* Do not run nlpn CI on md files in nlpn folder

* Add nlpn README.md
2021-06-17 17:05:00 +02:00
Kim De Mey aef7a25174
Add nlpn (#714)
* Add start of nlpn client

* Fix some error handling in nlpn

* Clean-up GOARCH from nlpn.yml and add vendor to include paths
2021-06-16 21:18:45 +02:00
Jamie Lokier cf36bdb801
Whisper (CI): Remove GitHub actions for building `wrappers*`
With Whisper support gone, the wrappers cannot be built.  More detail in the
patch that removes the wrappers.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-06-01 18:12:47 +01:00
Jamie Lokier 2fe2f23e70
CI: Restore "push" events but disable "pull_request"
Reverts part of commit 2539bd9 while keeping the intention of it:
To avoid duplicate CI runs when making and updating a PR.

Disabling `push` means we cannot push to a branch and see the CI results
directly without making a PR, which some of us use.  There are many situations
where this is useful, and "[WIP]" PRs are not appropriate for all.

Disabling `pull_request` has a similar effect, removing duplicate CI.

It is known that `pull_request` _is_ needed when a third party sends a
PR (because it's not committed to the repo yet).  But this is rare at the
moment, and there's a workaround: A committer can push the third party change
to a branch, triggering CI.

So re-enable `push`, disable `pull_request`, and we'll see if the latter
missing causes problems in practice.  Won't know until we try it.

Note: This might be interim until `workflow_dispatch` is working better.
Perhaps that needs more configuration.  Currently, `workflow_dispatch` is kind
of useless for CI tests, because it doesn't result in any CI indicator
associated with a commit or branch.  Even the actions page doesn't show the
name of the branch, just a less-than-useful generic "CI" for these actions.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-05-28 15:33:35 +01:00
Ștefan Talpalaru 45da426bb8
CI: enable manual running (#695)
Since we think it needs to be on master to have any effect, sending it to master to test.
2021-05-28 15:14:01 +01:00
Ștefan Talpalaru 2539bd9904
CI: limit "push" events to the master branch 2021-05-26 21:55:32 +02:00
Jamie Lokier e03c483afa CI: Skip GitHub actions for `hive_integration` files
Files underneath `hive_integration` are not used to build anything,
so don't trigger CI when these files change.

Most of them are manually copied into Hive `hive/clients/nimbus`.
The rest are instructions and a standalone helper program not used by CI.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-27 21:43:51 +03:00
Jamie Lokier 2416d23507 CI: Skip GitHub actions for documentation-only changes
The CI takes a long time (sometimes hours) and shouldn't hold up
documentation-only commits.  We may also find CI for other changes starts
faster, if we're not using up as many credits.  When we process documentation
into something nicer (like `nimbus.guide`) we'll need to revisit this.

Syntax from [example ignoring paths](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#example-ignoring-paths).

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-04-27 11:23:37 +01:00
jangko 74e6c86316
uncomment linux-amd64-evmc and windows-amd64-evmc entries in gh action matrix 2021-04-27 14:24:01 +07:00
Jordan Hrycaj b4f8450968 provide identical copy of vm folder => vm2, activated by make flag ENABLE_VM2=1
why:
  vm2 enabled by ENABLE_VM2=1 behaves as vm without ENABLE_EVMC=1 until
  it doesn't in some future fatch set. this leaves some wiggle room
  to work on a vm copy without degrading the original implementation.

details:
  + additional make flag ENABLE_VM2=1 (or ENABLE_VM2=0 to explicitely disable)
  + when both flags ENABLE_EVMC=1 and ENABLE_VM2=1 are present, the former
    flag ENABLE_EVMC=1 takes precedence, this is implemented at the NIM
    compiler level for -d:evmc_enabled and -d:vm2_enabled
2021-04-23 14:04:06 +03:00
Ștefan Talpalaru 0c34a8e365
CI: use MSYS2 on Windows 2021-04-09 17:19:40 +07:00
jangko b6ad47f3a4 fixes evmc bug and add github action job to test evmc 2021-01-20 11:50:07 +07:00
jangko 8c5c967715 bump submodules 2021-01-20 11:50:07 +07:00
jangko 6b8c5c9e91 fix gh action script for linux i386 2021-01-16 22:41:21 +07:00
jangko b22828806f add github action script 2021-01-15 14:03:10 +07:00