Commit Graph

2517 Commits

Author SHA1 Message Date
jangko 960539df81
unify evm call for both json-rpc and graphql
also fixes rpcEstimateGas for both of json-rpc and graphql
2021-10-31 12:24:40 +07:00
jangko baf508f6ae
move stateDB from VMState to chainDB
previously, every time the VMState was created, it will also create
new stateDB, and this action will nullify the advantages of cached accounts.

the new changes will conserve the accounts cache if the executed blocks
are contiguous. if not the stateDB need to be reinited.

this changes also allow rpcCallEvm and rpcEstimateGas executed properly
using current stateDB instead of creating new one each time they are called.
2021-10-28 18:57:08 +07:00
jangko cec628e620
cleanup: remove unused accessLogs code from vm_state
they are not used anywhere at present, nor in the future
2021-10-28 11:30:18 +07:00
jangko f0a0c1b878
cleanup: replace unnecessary initHexaryTrie with emptyRlpHash
it's simple non-sense
2021-10-28 10:48:28 +07:00
jangko e1abf81cac
cleanup: remove nimbus/rpc/key_storage.nim
this is an unused file
2021-10-28 10:32:41 +07:00
jangko df85cd779b
cleanup: remove setupComputation from vm2
setupComputation already replaced by runComputation
in transaction/call_common.nim
2021-10-28 10:30:56 +07:00
Kim De Mey 0ba3c682e4
Use SSZ code from nim-ssz-serialization module (#875) 2021-10-23 14:28:12 +02:00
Kim De Mey 02e8b2eb12
Clean up old unused code and unused imports (#877) 2021-10-22 21:36:53 +02:00
Kim De Mey 92dfb02ad7
Use ByteList.init instead of List 2048 (#876) 2021-10-22 17:49:23 +02:00
Kim De Mey 4b498356fc
Improve portal wire logging on sending/receiving messages (#874) 2021-10-22 11:03:00 +02:00
Kim De Mey 4b85c73f43
Update portal wire protocol ids to latest spec (#873) 2021-10-20 22:31:05 +02:00
jangko 2b6658b0d6
ci: clear github actions cache in linux/macos runner
it seemed that github actions macOs image have changed
(version changed from 20211002.1 to 20211011.2)
which made cached rocksdb not findable.
2021-10-20 08:27:53 +07:00
Jamie Lokier 5a5edb392a Bugfix: Incorrect processing of self-destructed, new contract
Fixes #868 "Gas usage consensus error at Mainnet block 6001128", and equivalent
on other networks.  Mainnet sync is able to continue past 6001128 after this.

Here's a trace:

```
TRC 2021-09-29 15:13:21.532+01:00 Persisting blocks                  file=persist_blocks.nim:43 fromBlock=6000961 toBlock=6001152
...
DBG 2021-09-29 15:14:35.925+01:00 gasUsed neq cumulativeGasUsed      file=process_block.nim:68 gasUsed=7999726 cumulativeGasUsed=7989726
TRC 2021-09-29 15:14:35.925+01:00 peer disconnected                  file=blockchain_sync.nim:407 peer=<PEER:IP>
```

Similar output is seen at many blocks in the range 6001128..6001204.

The bug is when handling a combination of `CREATE` or `CREATE2`, along with
`SELFDESTRUCT` applied to the new contract address.

Init code for a contract can't return non-empty code and do `SELFDESTRUCT` at
the same time, because `SELFDESTRUCT` returns empty data.

But it is possible to return non-empty code in a newly created, self-destructed
account if the init code calls `DELEGATECALL` or `CALLCODE` to other code which
uses `SELFDESTRUCT`.

In this case we must still charge gas and write the code.  This shows on
Mainnet blocks 6001128..6001204, where the gas difference matters.  The code
must be written because the new code can be called later in the transaction
too, before self-destruction wipes the account at the end.

There are actually three semantic changes here for a self-destructed, new
contract:

- Gas is charged.
- The code is written to the account.
- It can fail due to insufficient gas.

This patch almost exactly reverts a15805e4 "fix applyCreateMessage" from
2019-02-28.  I wonder what that fixed.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-10-19 14:24:46 +01:00
Jamie Lokier 242dfdd5ac
Bugfix: Off by 1 in EIP-170 code size checks in `stateless`
Fixes an off by 1 error where `EIP170_CODE_SIZE_LIMIT` was being treated as the
lowest invalid value by EVM code, but the highest valid value by witness code.

To remove confusion, this is renamed to `EIP170_MAX_CODE_SIZE` with value
0x6000, which matches the name (`MAX_CODE_SIZE`) and value used for this limit
in [EIP-170](https://eips.ethereum.org/EIPS/eip-170).

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-10-19 10:30:53 +01:00
Jamie Lokier 40fbed49cf
Sync fix: `GetBlockBodies` logic preventing sync, dropping peers
Fixes #864 "Sync progress stops at Goerli block 4494913", and equivalent on
other networks.

The block body fetcher in `blockchain_sync.nim` had an incorrect assumption
about how peers respond to `GetBlockBodies`.  It was issuing requests for N
block bodies and incorrectly handling replies which contained fewer than N
bodies.

Having received up to 192 headers in a batch, it split the range into smaller
`GetBlockBodies` requests, fetched each reply, then combined replies.  The
effect was Nimbus requested batches of 128+64 block bodies, received gaps in
the reply sequence, then aborted.

That meant it repeatedly fetched data, then discarded it, and fetched it again,
dropping good peers in the process.

Aborted and restarted batches occurred with earlier blocks too, but this became
more pronounced until there were no suitable peers at batch 4494913..4495104.

Here's a trace:

```
TRC 2021-09-29 02:40:24.977+01:00 Requesting block headers                   file=blockchain_sync.nim:224 start=4494913 count=192 peer=<ENODE>
TRC 2021-09-29 02:40:24.977+01:00 >> Sending eth.GetBlockHeaders (0x03)      file=protocol_eth65.nim:51 peer=<PEER> startBlock=4494913 max=192
TRC 2021-09-29 02:40:25.005+01:00 << Got reply eth.BlockHeaders (0x04)       file=protocol_eth65.nim:51 peer=<PEER> count=192
TRC 2021-09-29 02:40:25.007+01:00 >> Sending eth.GetBlockBodies (0x05)       file=protocol_eth65.nim:51 peer=<PEER> count=128
TRC 2021-09-29 02:40:25.209+01:00 << Got reply eth.BlockBodies (0x06)        file=protocol_eth65.nim:51 peer=<PEER> count=13
TRC 2021-09-29 02:40:25.210+01:00 >> Sending eth.GetBlockBodies (0x05)       file=protocol_eth65.nim:51 peer=<PEER> count=64
TRC 2021-09-29 02:40:25.290+01:00 << Got reply eth.BlockBodies (0x06)        file=protocol_eth65.nim:51 peer=<PEER> count=64
WRN 2021-09-29 02:40:25.306+01:00 Bodies len != headers.len                  file=blockchain_sync.nim:276 bodies=77 headers=192
TRC 2021-09-29 02:40:25.306+01:00 peer disconnected                          file=blockchain_sync.nim:403 peer=<PEER>
TRC 2021-09-29 02:40:25.306+01:00 Finished obtaining blocks                  file=blockchain_sync.nim:303 peer=<PEER>
```

In practice, for modern peers, Nimbus received shorter replies than it assumed
depending on the block sizes on the chain.  Geth/Erigon has 2MiB `BlockBodies`
response size soft limit.  OpenEthereum has 4MiB.

Up to Berlin (EIP-2929), Nimbus's fetcher failed often, but there were still
some peers serving what Nimbus needed.

Just after the start of Berlin, at batch 4494913..4495104 on Goerli, zero peers
responded with full size replies for the whole batch, so Nimbus couldn't
progress past that point.  But there was already a problem happening before
that for large blocks, dropping good peers and repeatedly fetching the same
block data.

Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-10-19 10:20:26 +01:00
jangko d93a8bc4a1
make macro_assembler to use the same testEvmCall
it also allow the macro_assembler to write more test
of CALL and CREATE family which previously not possible
2021-10-14 15:10:12 +07:00
jangko 71273f2f4c
remove error field from evm CallResult 2021-10-14 15:10:12 +07:00
jangko 08f8652790
remove noTransfer field from evm CallParams 2021-10-14 15:10:11 +07:00
jangko eb2251ec37
simplify evm call of test_precompiles
first step towards evm call variation reduction
2021-10-14 15:10:11 +07:00
Kim De Mey b77034c870
Update portal wire protocol according to latest spec changes (#860)
This is not complete, in order to be according to specification
two SSZ Union types need to be used.
2021-10-13 21:35:54 +02:00
jangko 5c4c1784a0
fix missing EIP-799 extra range check in header validation 2021-10-12 11:06:39 +07:00
Kim De Mey 447deb29ff
Add HistoryNetwork that uses Portal wire protocol (#856)
* Add HistoryNetwork that uses Portal wire protocol

* Further implement getContent and adjust naming of distance calc calls
2021-10-09 13:22:03 +02:00
KonradStaniec c62905db25
Re enable using custom distance function (#854) 2021-10-05 21:16:33 +02:00
Kim De Mey b58920f29f
Set config default descriptions (#852)
* Update fluffy config to set defaults descriptions

* Update portalcli config to set defaults descriptions
2021-09-29 13:58:55 +02:00
Kim De Mey 1d74ef4b98
Fix typo in README.md (#850) 2021-09-29 13:04:51 +02:00
jangko 09dbe15eba
update test logs 2021-09-29 10:55:32 +07:00
jangko 460432f154
fixes EIP1559 tx gasCost validation
pre EIP1559 max(gasCost) is tx.gasLimit * tx.gasPrice
the new EIP1559 max(gasCost) before the transaction can be executed is
tx.gasLimit * tx.maxFeePerGas
2021-09-29 10:55:32 +07:00
jangko d8dedb0469
bump ethereum/tests fixtures 2021-09-29 10:55:31 +07:00
Kim De Mey 51626c5831
Add a basic ContentDB for portal networks (#848)
* Add a basic ContentDB for Portal networks

* Use ContentDB in StateNetwork

* Avoid probably some form of sandwich problem by re-exporting kvstore_sqlite3 from content_db
2021-09-28 19:58:41 +02:00
jangko 44394d9ffd
fixes nimbus evm tracer, add missing networkParams when constructing chainDB 2021-09-26 10:45:52 +07:00
jangko 8552dda7bb
premix: fixes premix parser dan graphql_downloader
premix parser:
- fix incorrect tx type comparison
- fix chainId parser to support both graphql and json returned data

graphql_downloadse:
- allow downloader to skip parsing transactions
2021-09-26 10:38:41 +07:00
Kim De Mey 8e26acc4eb
Move tools and add common types (#846)
* Move some duplicated types to common_types

* Move portalcli to tools/ and update the READMEs
2021-09-24 17:18:54 +02:00
Kim De Mey 16a36453b5
Rework ContentId, ContentKey, GetContent to be more clear (#845)
* Small clarification in naming and note where GetContent is actually needed

* Rework contentId and contentKey
2021-09-24 11:22:07 +02:00
Kim De Mey 785a3b47b0
Allow for passing Portal specific bootstrap nodes (#844)
* Allow for passing Portal specific bootstrap nodes

* Fix to also replaceNode when decodeMessage fails

* Add portal bootstrap node tests and reorder test cases
2021-09-23 14:26:41 +02:00
Kim De Mey 6f6345a022
Move Portal wire protocol out of state specific code (#843)
* Allow for custom protocol id in portal wire protocol

* Move Portal wire protocol out of state specific code
2021-09-22 17:07:14 +02:00
Kim De Mey 11c120e400
Update wire protocol of state network spec (#829)
* Update wire protocol of state network spec

* BytesN should be SSZ Vector not List

* Add specific lists and bitlists for content keys
2021-09-22 11:28:04 +02:00
jangko 908dc21478
evm: fixes EIP2929 opcodes
op balanceEIP2929, extCodeHashEIP2929, extCodeSizeEIP2929, and
extCodeCopyEIP2929 are fixed due to their wrong gasConsume
position
2021-09-22 11:58:06 +07:00
Kim De Mey 0135189e4c
Oops, isMainModule will not run of course for all_tests (#841) 2021-09-21 18:38:08 +02:00
Kim De Mey 1afdfdf2ff
Disable mixed usage of cliBuilder to fix incorrect test failure reporting (#836) 2021-09-21 14:35:41 +02:00
jangko e23300e2cc
fix premix tools due to recent changes
- persist: fix related to new config
- parser: add EIP 2930 and EIP 1559 features in parseBlockHeader
  and parseTransaction
- add graphql_downloader as a supplement json-rpc downloader
2021-09-21 13:35:52 +07:00
jangko 7757b03d0b
vendor: bump nim-graphql version from 0.2.18 to 0.2.20
features:
- playground: add support to run inside docker
- add secure mode to graphql http server
- add secure mode to graphql http client
- httpclient: add unzip dan chunked transfer of HTTP 1.1

bugfixes:
- fix accept-encoding bug in httpserver
2021-09-21 13:35:52 +07:00
jangko f3d194c05e
grpahql: add EIP-2718 and EIP-1559 features to graphql API
EIP-2718:
- chainID: Long! of Query
- chainID: Long of Transaction

EIP-1559:
- baseFeePerGas: BigInt of Block
- effectiveGasPrice: BigInt of Transaction
- maxFeePerGas: BigInt of Transaction
- maxPriorityFeePerGas: BigInt of Transaction
2021-09-21 13:35:49 +07:00
Kim De Mey 7a42d037f2
Disable the custom distance functions for Portal Protocol (#832)
The Portal Network tests fail with the custom functions,
on one part due to the fact that the portal tests still
uses the xor functions, but also because the
`neighboursAtDistances` call appears to filter them out.
The reverse calculation might be off.
2021-09-20 12:55:03 +02:00
jangko be91bb349b
hive: fixes readme.md and nimbus.sh
the introduction of new config requires the new command line options
be applied to the commands in nimbus.sh
2021-09-18 17:37:37 +07:00
jangko a3badea928
config: fix new config based on input from jamie and zahary 2021-09-18 17:34:51 +07:00
jangko 69f2a0f95a
config: replace stdlib parseOpt with nim-confutils
fixes #581
2021-09-18 17:34:46 +07:00
KonradStaniec 6192cd7dc1
Use custom distance function in state network (#831)
* Use custom distance calculator in state network
2021-09-16 16:13:36 +02:00
KonradStaniec 8f683bd318
bump nim-eth module (#828)
* bump-nim-eth

* fix breaking changes

* Increase result limit
2021-09-15 11:24:03 +02:00
KonradStaniec 58b11e683d
Implement custom distance function (#827)
* Implement custom distance function

* More docs about function equivalence
2021-09-14 13:36:55 +02:00
sacha 749069da5c
Update README.md
Add recent highlights
2021-09-14 00:48:25 +02:00