Commit Graph

1470 Commits

Author SHA1 Message Date
Péter Szilágyi 9eba3a9fff
cmd/geth, core/rawdb: seamless freezer consistency, friendly removedb 2019-05-16 17:01:56 +03:00
Péter Szilágyi 536b3b416c
cosensus, core, eth, params, trie: fixes + clique history cap 2019-05-16 10:39:35 +03:00
gary rong 37d280da41
core, cmd, vendor: fixes and database inspection tool (#15)
* core, eth: some fixes for freezer

* vendor, core/rawdb, cmd/geth: add db inspector

* core, cmd/utils: check ancient store path forceily

* cmd/geth, common, core/rawdb: a few fixes

* cmd/geth: support windows file rename and fix rename error

* core: support ancient plugin

* core, cmd: streaming file copy

* cmd, consensus, core, tests: keep genesis in leveldb

* core: write txlookup during ancient init

* core: bump database version
2019-05-16 10:39:34 +03:00
Martin Holst Swende 42c746d6f4
freezer: disable compression on hashes and difficulties (#14)
* freezer: disable compression on hashes and difficulties

* core/rawdb: address review concerns

* core/rawdb: address review concerns
2019-05-16 10:39:33 +03:00
Martin Holst Swende 331de17e4d
core/rawdb: support starting offset for future deletion 2019-05-16 10:39:33 +03:00
gary rong 80469bea0c
all: integrate the freezer with fast sync
* all: freezer style syncing

core, eth, les, light: clean up freezer relative APIs

core, eth, les, trie, ethdb, light: clean a bit

core, eth, les, light: add unit tests

core, light: rewrite setHead function

core, eth: fix downloader unit tests

core: add receipt chain insertion test

core: use constant instead of hardcoding table name

core: fix rollback

core: fix setHead

core/rawdb: remove canonical block first and then iterate side chain

core/rawdb, ethdb: add hasAncient interface

eth/downloader: calculate ancient limit via cht first

core, eth, ethdb: lots of fixes

* eth/downloader: print ancient disable log only for fast sync
2019-05-16 10:39:32 +03:00
rjl493456442 b6cac42e9f
core/rawdb: add file lock for freezer 2019-05-16 10:39:31 +03:00
Martin Holst Swende b69bdc2a4f
freezer: implement split files for data
* freezer: implement split files for data

* freezer: add tests

* freezer: close old head-file when opening next

* freezer: fix truncation

* freezer: more testing around close/open

* rawdb/freezer: address review concerns

* freezer: fix minor review concerns

* freezer: fix remaining concerns + testcases around truncation

* freezer: docs

* freezer: implement multithreading

* core/rawdb: fix freezer nitpicks + change offsets to uint32

* freezer: preopen files, simplify lock constructs

* freezer: delete files during truncation
2019-05-16 10:39:30 +03:00
Péter Szilágyi 006c21efc7
cmd, core, eth, les, node: chain freezer on top of db rework 2019-05-16 10:39:29 +03:00
Péter Szilágyi 9effd64290
core, eth, trie: bloom filter for trie node dedup during fast sync (#19489)
* core, eth, trie: bloom filter for trie node dedup during fast sync

* eth/downloader, trie: address review comments

* core, ethdb, trie: restart fast-sync bloom construction now and again

* eth/downloader: initialize fast sync bloom on startup

* eth: reenable eth/62 until we properly remove it
2019-05-13 15:28:01 +03:00
Péter Szilágyi 751effa35e
core: fix formatting error (trailing whitepace) 2019-05-13 14:07:55 +03:00
PilkyuJung ec2131c8d3 core: move error variable to error.go (#19560)
* move error variable to error.go

* Update error.go

Edit "Genesis" to "genesis"
2019-05-13 13:23:32 +03:00
Péter Szilágyi 6ec6b29051
core: fix import errors on clique crashes + empty blocks (#19544)
* core: fix import errors on clique crashes + empty blocks

* cosensus/clique, core: add test for the mirrored state issue

* core: address todo question wrt log count

* core: raise a louder warning for non-clique known blocks
2019-05-10 17:04:10 +03:00
gary rong c113723fdb core: handle importing known blocks more gracefully (#19417)
* core: import known blocks if they can be inserted as canonical blocks

* core: insert knowns blocks

* core: remove useless

* core: doesn't process head block in reorg function
2019-05-08 14:30:36 +03:00
Martin Holst Swende c8cf360f29
core: fix canonicality confusion (#19514)
* core: add tests for canonicality confusion

* core: delete stale future canon number mappings during reorg to shorter+heavier chain
2019-05-07 14:26:00 +02:00
유용환 abeba0a1de core/rawdb: fix typo (#19526) 2019-05-04 12:17:47 +02:00
Martin Holst Swende 4c90efdf57 consensus,core,miner: avoid overhead of creating a new block (#19301)
* consensus,core,miner: avoid overhead of creating a new block

* consensus: nitpick dot

* consensus: fix some comment formatting nits
2019-04-30 16:42:36 +03:00
Hongbin Mao 504f88b65b core/rawdb: typo fix storea => stores (#19498)
* typo fix

* change to stores
2019-04-26 12:22:21 +03:00
Matthew Halpern 937417527c core: lookup txs by block number instead of block hash (#19431)
* core: lookup txs by block number instead of block hash

Transaction hashes now store a reference to their corresponding
block number as opposed to their hash. In benchmarks this was
shown to reduce storage by over 12 GB.

The main limitation of this approach is that transactions on
non-canonical blocks could never be looked up, however that is
currently not supported.

The database version has been upgraded to version 5 and the
transaction lookup process is backwards-compatible with the
prior two transaction lookup formats prexisting in the
database instance. Tests have been added to ensure this.

* core/rawdb: tiny review nit fixes
2019-04-25 17:24:55 +03:00
gary rong ce9a289fa4
core/types: fix cummulative gas bug and legacy decoding tests 2019-04-16 09:50:11 +03:00
Péter Szilágyi 7221cb1434
core, eth, les, light: scope receipt functionality a bit cleaner 2019-04-15 13:42:26 +03:00
Matthew Halpern 6b0ddd141e
core, eth, les, light: store transaction receipts without txHash and gasCost 2019-04-15 13:15:39 +03:00
Martin Holst Swende d763b49ae3
consensus,core: shortcut uncle validation 2019-04-08 10:57:15 +02:00
gary rong 36f81118f6 core/state: fix state iterator (#19127)
* core/state: fix state iterator

* core: fix state iterator more elegant
2019-04-05 09:44:02 +03:00
Péter Szilágyi 43631aa1d6
core: minor code polishes + rebase fixes 2019-04-04 16:29:25 +03:00
rjl493456442 690bd8a417
core: re-omit new log event when logs rebirth 2019-04-04 14:17:43 +03:00
hydai 9b3601cfce core/vm: fix typos in comments (#19391) 2019-04-04 12:30:10 +02:00
Martin Holst Swende 36b78abe61
core/vm: instruction tests (#16327)
This PR makes it easy to generate and execute testcases for VM arithmetic operations. By enabling and running the testcase TestWriteExpectedValues, a set of json files are created which contain input and output for each arith operation.
The test TestJsonTestcases executes all of those tests.

While meaningless as is, this PR makes it less risky to make changes (optimizations) to the vm operations, since there will be a larger body of testcases.
2019-04-04 11:19:38 +02:00
Martin Holst Swende 0b4fe8d192 all: simplify timestamps to uint64 (#19372)
* all: simplify timestamps to uint64

* tests: update definitions

* clef, faucet, mobile: leftover uint64 fixups

* ethash: fix tests

* graphql: update schema for timestamp

* ethash: remove unused variable
2019-04-02 23:28:48 +03:00
Péter Szilágyi e14f8a408c
Merge pull request #19328 from karalabe/preload
core: prefetch next block state concurrently
2019-04-02 17:03:12 +03:00
Runchao Han c4109d790f core: fix typo in insertChain method doc (#19371) 2019-04-02 13:01:02 +03:00
Péter Szilágyi 92faf1bf7a
Merge pull request #19348 from LiangMa/overflowPR
core/vm: Correct the Memory Gas Overflow condition
2019-04-01 17:12:13 +03:00
Péter Szilágyi 9294b8f10f
core/vm: polish gas PR, fix tests, make table driven 2019-04-01 17:10:42 +03:00
Péter Szilágyi ed34a5e08a
cmd, core, eth: support disabling the concurrent state prefetcher 2019-04-01 11:52:11 +03:00
Péter Szilágyi bb9631c399
core: prefetch next block state concurrently 2019-04-01 11:06:15 +03:00
贺鹏飞 fbe7caf136
core: cache tx signature before obtaining lock 2019-03-29 12:01:29 +02:00
Liang Ma 157f09e5b6 core/vm: Correct the Memory Gas Overflow condition
previous overflow condition is too big to use.
0x7FFFFFFFF squre operation is overflowed uint64.

0x7FFFFFFFF^2 = 0x3F FFFF FFF0 0000 0001
2019-03-28 21:04:31 +00:00
Brent 7fb89697fd core/types: add block location fields to receipt (#17662)
Solves #15210 without changing consensus, in a backwards compatible way,
by adding tx inclusion information to the Receipt struct.
2019-03-27 13:39:25 +01:00
Péter Szilágyi a732c93309
core: 3rd try on splitting the trie metrics correctly 2019-03-27 13:02:04 +02:00
Martin Holst Swende 59e1953246 core, ethdb, trie: mode dirty data to clean cache on flush (#19307)
This PR is a more advanced form of the dirty-to-clean cacher (#18995),
where we reuse previous database write batches as datasets to uncache,
saving a dirty-trie-iteration and a dirty-trie-rlp-reencoding per block.
2019-03-26 15:48:31 +01:00
Péter Szilágyi b8b4fb004c
Merge pull request #19308 from holiman/fix_reset_txpool
core: make txpool handle reorg due to setHead
2019-03-26 11:00:35 +02:00
Péter Szilágyi 435020f9b3
core: split trie op metrics from the correct chain metrics 2019-03-25 16:27:46 +02:00
Péter Szilágyi 86989e3fcd
core: split out detailed trie access metrics from insertion time (#19316)
* core: split out detailed trie access metrics from insertion time

* cmd, core, metrics: support expensive optional metrics
2019-03-25 10:01:18 +02:00
Martin Holst Swende 650ad19c2d
core: make txpool handle reorg due to setHead 2019-03-21 11:42:56 +01:00
Péter Szilágyi 91eec1251c
cmd, core, eth, trie: get rid of trie cache generations (#19262)
* cmd, core, eth, trie: get rid of trie cache generations

* core, trie: get rid of remainder of cache gen boilerplate
2019-03-14 15:25:12 +02:00
Corey Lin 768b4c2e6b asm: remove unused parameter for function Lex (#18058) 2019-03-14 10:35:55 +01:00
Péter Szilágyi 4f457859a2
core: use headers only where blocks are unnecessary 2019-03-13 12:32:47 +02:00
Martin Holst Swende 7504dbd6eb core/vm: 64 bit memory and gas calculations (#19210)
* core/vm: remove function call for stack validation from evm runloop

* core/vm: separate gas  calc into static + dynamic

* core/vm: optimize push1

* core/vm: reuse pooled bigints for ADDRESS, ORIGIN and CALLER

* core/vm: use generic error message for jump/jumpi, to avoid string interpolation

* testdata: fix tests for new error message

* core/vm: use 64-bit memory calculations

* core/vm: fix error in memory calculation

* core/vm: address review concerns

* core/vm: avoid unnecessary use of big.Int:BitLen()
2019-03-12 11:40:05 +02:00
Sheldon da5de012c3 state: fix emptyStatet to emptyRoot (#19254) 2019-03-12 11:14:24 +02:00
Péter Szilágyi 054412e335
all: clean up and proerly abstract database access 2019-03-06 13:35:03 +02:00
Péter Szilágyi 1bc7f3f906
core/types: fix receipt legacy decoding 2019-02-28 14:16:36 +02:00
Samuel Marks e43bc36226
travis, appveyor, Dockerfile: upgrade to Go 1.12 2019-02-27 14:21:02 +02:00
Felföldi Zsolt c2003ed63b les, les/flowcontrol: improved request serving and flow control (#18230)
This change

- implements concurrent LES request serving even for a single peer.
- replaces the request cost estimation method with a cost table based on
  benchmarks which gives much more consistent results. Until now the
  allowed number of light peers was just a guess which probably contributed
  a lot to the fluctuating quality of available service. Everything related
  to request cost is implemented in a single object, the 'cost tracker'. It
  uses a fixed cost table with a global 'correction factor'. Benchmark code
  is included and can be run at any time to adapt costs to low-level
  implementation changes.
- reimplements flowcontrol.ClientManager in a cleaner and more efficient
  way, with added capabilities: There is now control over bandwidth, which
  allows using the flow control parameters for client prioritization.
  Target utilization over 100 percent is now supported to model concurrent
  request processing. Total serving bandwidth is reduced during block
  processing to prevent database contention.
- implements an RPC API for the LES servers allowing server operators to
  assign priority bandwidth to certain clients and change prioritized
  status even while the client is connected. The new API is meant for
  cases where server operators charge for LES using an off-protocol mechanism.
- adds a unit test for the new client manager.
- adds an end-to-end test using the network simulator that tests bandwidth
  control functions through the new API.
2019-02-26 12:32:48 +01:00
gary rong 7fd0ccaa68 core: remove unnecessary fields in logs, receipts and tx lookups (#17106)
* core: remove unnecessary fields in log

* core: bump blockchain database version

* core, les: remove unnecessary fields in txlookup

* eth: print db version explicitly

* core/rawdb: drop txlookup entry struct wrapper
2019-02-21 15:14:35 +02:00
Martin Holst Swende 8577b5b020 core: more tests for sidechain import, fixes #19105 (#19113)
* blockchain: more tests for sidechain import, fixes #19105

* core/blockchain: rework import of pruned canon blocks and canon-prepended sidechains

* core/blockchain: minor clarity change

* core/blockchain: remove unused method
2019-02-21 12:36:49 +02:00
Péter Szilágyi 2e8a5e5659
core/vm: remove unused constants (#19095) 2019-02-19 19:49:45 +02:00
Matthew Halpern bf42535d31 core: remove redundant parentheses (#19106) 2019-02-19 13:25:42 +02:00
Matthew Halpern c5eccaefb8 core/vm: remove unused constants 2019-02-18 07:48:19 -08:00
Matthew Halpern 3930f3795b core: remove unused function (#19097) 2019-02-18 13:27:31 +02:00
needkane a1f366ecf6 core/vm: update annotation (#19050) 2019-02-18 12:14:49 +01:00
Matthew Halpern 2b75fa9d61 core: enforce camel case variable names (#19058) 2019-02-14 21:14:05 +02:00
Péter Szilágyi 940e317094
core: fix pruner panic when importing low-diff-large-sidechain 2019-02-08 11:56:25 +02:00
Martin Holst Swende da1efdae0c
core: repro #18977 2019-02-08 10:29:51 +02:00
Péter Szilágyi d6225ab846
cmd/utils, eth: relinquish GC cache to read cache in archive mode 2019-02-07 14:36:25 +02:00
atsushi-ishibashi 81801ccc2b core/state: more memory efficient preimage allocation (#16663) 2019-02-07 10:44:45 +01:00
Martin Holst Swende a0fd4b7e57 core/vm: unshadow err to make it visible in tracers(#18504) 2019-02-04 15:26:21 +01:00
Péter Szilágyi 822dc1bfb1
Merge pull request #18121 from karalabe/goerli
cmd, core, params: add support for Goerli
2019-02-04 15:21:24 +02:00
Péter Szilágyi b0ed083ead
cmd, core, params: add support for Goerli 2019-02-04 14:53:12 +02:00
Martin Holst Swende ec3432bccb
core: fix error in block iterator (#18986) 2019-02-04 13:30:19 +01:00
Gus 4eee99aac5 core/types: remove use of package unsafe 2019-01-29 10:17:40 +01:00
b00ris 769657060e les: implement ultralight client (#16904)
For more information about this light client mode, read
https://hackmd.io/s/HJy7jjZpm
2019-01-24 12:18:26 +01:00
Nalin Bhardwaj b8f9b3779f core/vm: fix typos and use ExpGas for EXP (#18400)
This replaces the GasSlowStep constant with params.ExpGas.
Both constants have value 10.
2019-01-24 12:14:02 +01:00
Martin Holst Swende ecb781297b core, cmd/puppeth: implement constantinople fix, disable EIP-1283 (#18486)
This PR adds a new fork which disables EIP-1283. Internally it's called Petersburg,
but the genesis/config field is ConstantinopleFix.

The block numbers are:

    7280000 for Constantinople on Mainnet
    7280000 for ConstantinopleFix on Mainnet
    4939394 for ConstantinopleFix on Ropsten
    9999999 for ConstantinopleFix on Rinkeby (real number decided later)

This PR also defaults to using the same ConstantinopleFix number as whatever
Constantinople is set to. That is, it will default to mainnet behaviour if ConstantinopleFix
is not set.This means that for private networks which have already transitioned
to Constantinople, this PR will break the network unless ConstantinopleFix is
explicitly set!
2019-01-24 11:36:30 +01:00
silence 66f0c464cc core: only cache non-nil receipts from the database (#18447)
receipts may be null for very short time in some condition. For this case, we should not add the null value into cache. Because you will not get the right result if you keep requesting that receipt.
2019-01-17 17:49:12 +02:00
Péter Szilágyi f25f776c9f
core, light: get rid of the dual mutexes, hard to reason with 2019-01-11 15:27:47 +02:00
gary rong d5cad488be core, eth: fix database version (#18429)
* core, eth: fix database version

* eth: polish error message
2019-01-11 13:49:12 +02:00
Péter Szilágyi c1c4301121
Merge pull request #18371 from jeremyschlatter/patch-1
core/types: update incorrect comment
2019-01-04 10:14:17 +02:00
Dave McGregor 33d233d3e1
vendor, crypto, swarm: switch over to upstream sha3 package 2019-01-04 09:26:07 +02:00
Jeremy Schlatter 6537ab5dd3
core/types: update incorrect comment 2018-12-28 17:58:03 -08:00
Jordan Krage 27ce4eb78b core: sanitize more TxPoolConfig fields (#17210)
* core: sanitize more TxPoolConfig fields

* core: fix TestTransactionPendingMinimumAllowance
2018-12-20 14:00:58 +01:00
Shuai Qi fe26b2f366 core/state: rename 'new' variable (#18301) 2018-12-14 14:55:03 +01:00
qd-ethan 9940d93a43 Comment error (#18303) 2018-12-14 11:15:31 +01:00
Martin Holst Swende 83a9a73b89 cmd/geth, core, eth: implement Constantinople override flag (#18273)
* geth/core/eth: implement constantinople override flag

* les: implemnent constantinople override flag for les clients

* cmd/geth, eth, les: fix typo, move flag to experimentals
2018-12-11 14:19:03 +02:00
Martin Holst Swende 42a914a84f
cmd/evm, core/vm, eth: implement api methods to do stdjson dump to local filesystem 2018-12-10 12:33:50 +02:00
Paweł Bylica de39513ced core, internal, eth, miner, les: Take VM config from BlockChain (#17955)
Until this commit, when sending an RPC request that called `NewEVM`, a blank `vm.Config`
would be taken so as to set some options, based on the default configuration. If some extra
configuration switches were passed to the blockchain, those would be ignored.

This PR adds a function to get the config from the blockchain, and this is what is now used
for RPC calls.

Some subsequent changes need to be made, see https://github.com/ethereum/go-ethereum/pull/17955#pullrequestreview-182237244
for the details of the discussion.
2018-12-06 14:34:49 +01:00
Martin Holst Swende 7c657fc789 tests, core: update tests and make STATICCALL cause touch-delete (#18187) 2018-11-29 10:51:57 +02:00
Martin Holst Swende bfed28a421 core: more detailed metrics for block processing (#18119) 2018-11-28 10:29:05 +02:00
Jaynti Kanani 38592a13a3 fix mixHash/nonce for parity compatible network (#18166) 2018-11-26 09:59:04 +01:00
mr_franklin 2a113f6d72 core: return error if repair block failed (#18126)
* core: return error if repair block failed

* make error a bit shorter
2018-11-23 11:16:14 +02:00
Péter Szilágyi 3ba0418a9a
Merge pull request #17973 from holiman/splitter2
core: better side-chain importing
2018-11-22 15:01:10 +02:00
Martin Holst Swende e0d091e090 core: better printout of receipts in bad block reports (#18156)
* core/blockchain: better printout of receipts in bad block reports

* fix splleing
2018-11-22 11:00:16 +02:00
Péter Szilágyi 333b5fb123
core: polish side chain importer a bit 2018-11-21 13:19:56 +02:00
mr_franklin 3fd87f2193 core: fix comment typo (#18144) 2018-11-21 12:52:02 +02:00
Martin Holst Swende 493903eede
core: better side-chain importing 2018-11-20 12:28:43 +02:00
Péter Szilágyi accc0fab4f
core, eth/downloader: fix ancestor lookup for fast sync 2018-11-16 13:21:20 +02:00
Péter Szilágyi 434dd5bc00
cmd, core, eth, light, trie: add trie read caching layer 2018-11-15 12:22:13 +02:00
gary rong 1212c7b844 core: fix default trie cache limit (#17860) 2018-11-12 18:06:34 +02:00
Corey Lin 1ff152f3a4 rawdb: remove unused parameter for WritePreimages func (#18059)
* rawdb: remove unused parameter for WritePreimages func and modify a
spelling mistake

* rawdb: update the doc for function WritePreimages
2018-11-09 12:51:07 +02:00
Felix Lange 870efeef01
core/state: remove lock (#18065)
The lock in StateDB is useless. It's only held in Copy, but Copy is safe
for concurrent use because all it does is read.
2018-11-08 21:37:19 +01:00
Péter Szilágyi d0675e9d9c
Merge pull request #17982 from holiman/polish_contantinople_extcodehash
core/vm: check empty in extcodehash
2018-11-08 14:26:04 +02:00