Commit Graph

10614 Commits

Author SHA1 Message Date
Martin Holst Swende 8577b5b020 core: more tests for sidechain import, fixes #19105 (#19113)
* blockchain: more tests for sidechain import, fixes #19105

* core/blockchain: rework import of pruned canon blocks and canon-prepended sidechains

* core/blockchain: minor clarity change

* core/blockchain: remove unused method
2019-02-21 12:36:49 +02:00
Enrique Fynn 628a0bde3f common/fdlimit: Fix compilation error in freebsd, Raise() returns uint64 (#19141)
cast limit.Cur (int64) to uint64, fix test and compilation error
2019-02-21 11:16:51 +02:00
Matthew Halpern fbedf62f3d swarm/storage: fix loop bound for database cleanup (#19085)
The current loop continuation condition is always true as a uint8
is always being checked whether it is less than 255 (its maximum
value). Since the loop starts with the value 1, the loop termination
can be guarranteed to exit once the value overflows to 0.
2019-02-21 06:37:32 +01:00
Janoš Guljaš 9d5e10f5bb cmd/swarm/global-store: use kill instead interrupt in tests (#19142) 2019-02-20 22:58:25 +01:00
Ferenc Szabo e38b227ce6 Ci race detector handle failing tests (#19143)
* swarm/storage: increase mget timeout in common_test.go

 TestDbStoreCorrect_1k sometimes timed out with -race on Travis.

--- FAIL: TestDbStoreCorrect_1k (24.63s)
    common_test.go:194: testStore failed: timed out after 10s

* swarm: remove unused vars from TestSnapshotSyncWithServer

nodeCount and chunkCount is returned from setupSim and those values
we use.

* swarm: move race/norace helpers from stream to testutil

As we will need to use the flag in other packages, too.

* swarm: refactor TestSwarmNetwork case

Extract long running test cases for better visibility.

* swarm/network: skip TestSyncingViaGlobalSync with -race

As panics on Travis.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7e351b]

* swarm: run TestSwarmNetwork with fewer nodes with -race

As otherwise we always get test failure with `network_test.go:374:
context deadline exceeded` even with raised `Timeout`.

* swarm/network: run TestDeliveryFromNodes with fewer nodes with -race

Test on Travis times out with 8 or more nodes if -race flag is present.

* swarm/network: smaller node count for discovery tests with -race

TestDiscoveryPersistenceSimulationSimAdapters failed on Travis with
`-race` flag present. The failure was due to extensive memory usage,
coming from the CGO runtime. Using a smaller node count resolves the
issue.

=== RUN   TestDiscoveryPersistenceSimulationSimAdapter
==7227==ERROR: ThreadSanitizer failed to allocate 0x80000 (524288) bytes of clock allocator (error code: 12)
FATAL: ThreadSanitizer CHECK failed: ./gotsan.cc:6976 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
FAIL    github.com/ethereum/go-ethereum/swarm/network/simulations/discovery     804.826s

* swarm/network: run TestFileRetrieval with fewer nodes with -race

Otherwise we get a failure due to extensive memory usage, as the CGO
runtime cannot allocate more bytes.

=== RUN   TestFileRetrieval
==7366==ERROR: ThreadSanitizer failed to allocate 0x80000 (524288) bytes of clock allocator (error code: 12)
FATAL: ThreadSanitizer CHECK failed: ./gotsan.cc:6976 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
FAIL	github.com/ethereum/go-ethereum/swarm/network/stream	155.165s

* swarm/network: run TestRetrieval with fewer nodes with -race

Otherwise we get a failure due to extensive memory usage, as the CGO
runtime cannot allocate more bytes ("ThreadSanitizer failed to
allocate").

* swarm/network: skip flaky TestGetSubscriptionsRPC on Travis w/ -race

Test fails a lot with something like:
 streamer_test.go:1332: Real subscriptions and expected amount don't match; real: 0, expected: 20

* swarm/storage: skip TestDB_SubscribePull* tests on Travis w/ -race

Travis just hangs...

ok  	github.com/ethereum/go-ethereum/swarm/storage/feed/lookup	1.307s
keepalive
keepalive
keepalive

or panics after a while.

Without these tests the race detector job is now stable. Let's
invetigate these tests in a separate issue:
https://github.com/ethersphere/go-ethereum/issues/1245
2019-02-20 22:57:42 +01:00
lash d36e974ba3 swarm/network: Keep span across roundtrip (#19140)
* swarm/newtork: WIP Span request span until delivery and put

* swarm/storage: Introduce new trace across single fetcher lifespan

* swarm/network: Put span ids for sendpriority in context value

* swarm: Add global span store in tracing

* swarm/tracing: Add context key constants

* swarm/tracing: Add comments

* swarm/storage: Remove redundant fix for filestore

* swarm/tracing: Elaborate constants comments

* swarm/network, swarm/storage, swarm:tracing: Minor cleanup
2019-02-20 14:50:37 +01:00
lash 460d206f30 swarm/network: Use actual remote peer ip in underlay (#19137)
* swarm/network: Logline to see handshake addr

* swarm/network: Replace remote ip in handshake uaddr

* swarm/network: Add test for enode uaddr rewrite method

* swarm/network: Remove redundance pointer return from sanitize

* swarm/network: Obeying the linting machine

* swarm/network: Add panic comment

(travis trigger take 1)
2019-02-20 14:46:00 +01:00
Janoš Guljaš ba2dfa5ce4 swarm/network/stream: fix a goroutine leak in Registry (#19139)
* swarm/network/stream: fix a goroutine leak in Registry

* swarm/network, swamr/network/stream: Kademlia close addr count and depth change chans

* swarm/network/stream: rename close channel to quit

* swarm/network/stream: fix sync between NewRegistry goroutine and Close method
2019-02-20 14:45:25 +01:00
Dan Kinsley f49f95e2b0 accounts/abi: mutex lock in TransactionByHash and code cleanup (#19133) 2019-02-20 08:08:54 +01:00
Matthew Halpern d3ccedc767 p2p/simulations: enforce camel case variable names (#19053) 2019-02-19 19:50:59 +02:00
Péter Szilágyi 2e8a5e5659
core/vm: remove unused constants (#19095) 2019-02-19 19:49:45 +02:00
Matthew Halpern 8af6c9e6a2 eth: extract check for tracing transaction in block file (#19107)
Simplifies the transaction presense check to use a function to
determine if the transaction is present in the block provided
to trace, which originally had a redundant parenthesis and used
a `exist` flag to dictate control flow.
2019-02-19 19:49:24 +02:00
Matthew Halpern 514a9472ad trie: prefer nil slices over zero-length slices (#19084) 2019-02-19 14:50:11 +01:00
Matthew Halpern f1537b774c p2p/discover: make maximum packet size a constant (#19061) 2019-02-19 12:27:29 +01:00
Matthew Halpern bf42535d31 core: remove redundant parentheses (#19106) 2019-02-19 13:25:42 +02:00
Jeremy Schlatter b5e5b3567c crypto: fix build when CGO_ENABLED=0 (#19121)
Package crypto works with or without cgo, which is great. However, to make it
work without cgo required setting the build tag `nocgo`. It's common to disable
cgo by instead just setting the environment variable `CGO_ENABLED=0`. Setting
this environment variable does _not_ implicitly set the build tag `nocgo`. So
projects that try to build the crypto package with `CGO_ENABLED=0` will fail. I
have done this myself several times. Until today, I had just assumed that this
meant that this package requires cgo.

But a small build tag change will make this case work. Instead of using `nocgo`
and `!nocgo`, we can use `!cgo` and `cgo`, respectively. The `cgo` build tag is
automatically set if cgo is enabled, and unset if it is disabled.
2019-02-19 12:18:37 +01:00
HackyMiner f7f6a46029 eth, node: use APPDATA env to support cygwin/msys correctly (#17786)
This changes default location of the data directory to use the LOCALAPPDATA
environment variable, resolving issues with remote home directories an improving
compatibility with Cygwin.

Fixes #2239 
Fixes #2237 
Fixes #16437
2019-02-19 12:15:15 +01:00
Felix Lange d2256244c4 rpc: fixup change to not verify websocket origin (#19128)
This corrects the previous change which broke the build and
was pushed by accident.
2019-02-19 12:58:32 +02:00
Felix Lange 26d3a8ca80 rpc: skip websocket origin check if there is no origin header 2019-02-19 11:49:43 +01:00
Hsien-Tang Kao c283d9b5e8 signer/core: handle JSON unmarshal error (#19123) 2019-02-19 10:48:19 +02:00
Hsien-Tang Kao 4a090a1bab accounts/abi: fix error message format (#19122) 2019-02-19 10:34:55 +02:00
Guillaume Ballet 9c7e65c435 accounts: fix typos from the `SignData` merge (#19119) 2019-02-19 10:33:42 +02:00
Matthew Halpern c5eccaefb8 core/vm: remove unused constants 2019-02-18 07:48:19 -08:00
lash d88c6ce6b0 swarm: Reinstate Pss Protocol add call through swarm service (#19117)
* swarm: Reinstate Pss Protocol add call through swarm service

* swarm: Even less self
2019-02-18 16:44:50 +01:00
Matthew Halpern 3c62f965e7 eth: remove redundant parentheses (#19108) 2019-02-18 17:42:22 +02:00
Matthew Halpern 2a0e1bb32b crypto/ecies: remove unused function (#19096) 2019-02-18 14:09:07 +02:00
Matthew Halpern f57c80d22e metrics: remove redundant type specifiers (#19090) 2019-02-18 13:37:31 +02:00
Matthew Halpern 7b3e3d549e node: prefer nil slices over zero-length slices (#19083) 2019-02-18 13:31:22 +02:00
Matthew Halpern 3930f3795b core: remove unused function (#19097) 2019-02-18 13:27:31 +02:00
needkane a1f366ecf6 core/vm: update annotation (#19050) 2019-02-18 12:14:49 +01:00
Matthew Halpern df7c4618cd signer/core: remove unused function (#19099) 2019-02-18 12:10:28 +01:00
holisticode 62d7688d0a cmd/swarm/swarm-smoke: Trigger chunk debug on timeout (#19101)
* cmd/swarm/swarm-smoke: first version trigger has-chunks on timeout

* cmd/swarm/swarm-smoke: finalize trigger to chunk debug

* cmd/swarm/swarm-smoke: fixed httpEndpoint for trigger

* cmd/swarm/swarm-smoke: port

* cmd/swarm/swarm-smoke: ws not rpc

* cmd/swarm/swarm-smoke: added debug output

* cmd/swarm/swarm-smoke: addressed PR comments

* cmd/swarm/swarm-smoke: renamed track-timeout and track-chunks
2019-02-18 12:05:22 +01:00
Péter Szilágyi b6ce358a9b
Merge pull request #19114 from holiman/update_bigcache
vendor: update bigcache
2019-02-18 12:43:41 +02:00
Péter Szilágyi c0b9c763bb
build: explicitly force .xz compression (old debuild picks gzip) (#19118) 2019-02-18 12:12:26 +02:00
Péter Szilágyi 75a931470e travis.yml: add launchpad SSH public key (#19115) 2019-02-18 10:24:11 +01:00
Martin Holst Swende 37e5a908e7
vendor: update bigcache 2019-02-18 09:33:05 +01:00
Ferenc Szabo 50b872bf05 p2p, swarm: fix node up races by granular locking (#18976)
* swarm/network: DRY out repeated giga comment

I not necessarily agree with the way we wait for event propagation.
But I truly disagree with having duplicated giga comments.

* p2p/simulations: encapsulate Node.Up field so we avoid data races

The Node.Up field was accessed concurrently without "proper" locking.
There was a lock on Network and that was used sometimes to access
the  field. Other times the locking was missed and we had
a data race.

For example: https://github.com/ethereum/go-ethereum/pull/18464
The case above was solved, but there were still intermittent/hard to
reproduce races. So let's solve the issue permanently.

resolves: ethersphere/go-ethereum#1146

* p2p/simulations: fix unmarshal of simulations.Node

Making Node.Up field private in 13292ee897e345045fbfab3bda23a77589a271c1
broke TestHTTPNetwork and TestHTTPSnapshot. Because the default
UnmarshalJSON does not handle unexported fields.

Important: The fix is partial and not proper to my taste. But I cut
scope as I think the fix may require a change to the current
serialization format. New ticket:
https://github.com/ethersphere/go-ethereum/issues/1177

* p2p/simulations: Add a sanity test case for Node.Config UnmarshalJSON

* p2p/simulations: revert back to defer Unlock() pattern for Network

It's a good patten to call `defer Unlock()` right after `Lock()` so
(new) error cases won't miss to unlock. Let's get back to that pattern.

The patten was abandoned in 85a79b3ad3,
while fixing a data race. That data race does not exist anymore,
since the Node.Up field got hidden behind its own lock.

* p2p/simulations: consistent naming for test providers Node.UnmarshalJSON

* p2p/simulations: remove JSON annotation from private fields of Node

As unexported fields are not serialized.

* p2p/simulations: fix deadlock in Network.GetRandomDownNode()

Problem: GetRandomDownNode() locks -> getDownNodeIDs() ->
GetNodes() tries to lock -> deadlock

On Network type, unexported functions must assume that `net.lock`
is already acquired and should not call exported functions which
might try to lock again.

* p2p/simulations: ensure method conformity for Network

Connect* methods were moved to p2p/simulations.Network from
swarm/network/simulation. However these new methods did not follow
the pattern of Network methods, i.e., all exported method locks
the whole Network either for read or write.

* p2p/simulations: fix deadlock during network shutdown

`TestDiscoveryPersistenceSimulationSimAdapter` often got into deadlock.
The execution was stuck on two locks, i.e, `Kademlia.lock` and
`p2p/simulations.Network.lock`. Usually the test got stuck once in each
20 executions with high confidence.

`Kademlia` was stuck in `Kademlia.EachAddr()` and `Network` in
`Network.Stop()`.

Solution: in `Network.Stop()` `net.lock` must be released before
calling `node.Stop()` as stopping a node (somehow - I did not find
the exact code path) causes `Network.InitConn()` to be called from
`Kademlia.SuggestPeer()` and that blocks on `net.lock`.

Related ticket: https://github.com/ethersphere/go-ethereum/issues/1223

* swarm/state: simplify if statement in DBStore.Put()

* p2p/simulations: remove faulty godoc from private function

The comment started with the wrong method name.

The method is simple and self explanatory. Also, it's private.
=> Let's just remove the comment.
2019-02-18 07:38:14 +01:00
gluk256 12ca3b172a swarm/pss: refactoring (#19110)
* swarm/pss: split pss and keystore

* swarm/pss: moved whisper to keystore

* swarm/pss: goimports fixed
2019-02-17 06:29:41 +01:00
Martin Holst Swende 4f85c2b88b
trie: fix error in node decoding (#19111) 2019-02-16 16:16:12 +01:00
Elad 5b8ae7885e swarm/storage: fix influxdb gc metrics report (#19102) 2019-02-15 07:41:42 +01:00
Péter Szilágyi 7f6b05aa43
Merge pull request #19087 from karalabe/ios-fix-take-2
vendor: pull in upstream syscall fixes for non-linux/arm64
2019-02-15 02:01:28 +02:00
Matthew Halpern fa87929a2f cmd: prefer nil slices over zero-length slices (#19077) 2019-02-15 01:02:11 +02:00
Matthew Halpern e26a119c9b console: prefer nil slices over zero-length slices (#19076) 2019-02-15 00:59:54 +02:00
Péter Szilágyi 9d3ea8df1c
vendor: pull in upstream syscall fixes for non-linux/arm64 2019-02-15 00:51:34 +02:00
Matthew Halpern 2b75fa9d61 core: enforce camel case variable names (#19058) 2019-02-14 21:14:05 +02:00
holisticode 2af24724dd swarm/network: Saturation check for healthy networks (#19071)
* swarm/network: new saturation for  implementation

* swarm/network: re-added saturation func in Kademlia as it is used elsewhere

* swarm/network: saturation with higher MinBinSize

* swarm/network: PeersPerBin with depth check

* swarm/network: edited tests to pass new saturated check

* swarm/network: minor fix saturated check

* swarm/network/simulations/discovery: fixed renamed RPC call

* swarm/network: renamed to isSaturated and returns bool

* swarm/network: early depth check
2019-02-14 19:01:50 +01:00
Péter Szilágyi fab8c5a1cd
Merge pull request #19072 from karalabe/update-syscalls
vendor: update syscalls dependency
2019-02-14 18:42:02 +02:00
Péter Szilágyi dcc045f03c
vendor: update syscalls dependency 2019-02-14 18:14:28 +02:00
Felix Lange ba90a4aaa4 common/fdlimit: fix windows build (#19068) 2019-02-14 17:32:22 +02:00
Matthew Halpern 325334f61a light: enforce camel case variable names (#19054) 2019-02-14 17:18:32 +02:00