Commit Graph

714 Commits

Author SHA1 Message Date
Matt Joiner 39bde7237e Try to count IPv6 connections 2018-02-13 00:48:21 +11:00
Matt Joiner dd9244c01d bencode: Unmarshal now returns an error on unused trailing bytes 2018-02-13 00:21:28 +11:00
Matt Joiner 57216bd299 Standardize on krpc types, and extend PEX message type for IPv6 2018-02-11 15:13:00 +11:00
Matt Joiner 8d1995c4d4 Support IPv6 blocklists
Existing packed blocklists need to be regenerated.
2018-02-11 15:11:26 +11:00
Matt Joiner 924b916b2a Expose Client.Closed 2018-02-08 23:57:53 +11:00
Matt Joiner f57437201e Send bitfield messages before other initial messages 2018-02-05 17:41:53 +11:00
Matt Joiner e0c2265cb0 Support sending HaveAll 2018-02-04 19:10:25 +11:00
Matt Joiner ea8659c951 Move initiateConn and openNewConns onto Torrent 2018-02-04 12:59:23 +11:00
Matt Joiner efecf88d3c Use flip buffering for connection writing 2018-02-03 15:09:38 +11:00
Matt Joiner 085e676ef0 Generate default peerExtensionBytes using helpers 2018-02-03 13:36:17 +11:00
Matt Joiner 9567aa901c Add missing increments to some stats 2018-02-03 11:53:11 +11:00
Matt Joiner fe991128ed Rework connection stat reconciliation with Torrent and refactor doppleganger handling 2018-02-02 19:07:20 +11:00
Matt Joiner b0c1f99509 Merge a bunch of stuff into ConnStats and refactor connection.upload 2018-02-02 19:04:56 +11:00
Matt Joiner 97ffe39561 Move sendChunk onto connection 2018-02-02 13:26:04 +11:00
Matt Joiner 1a7708b484 sortimports 2018-01-31 16:42:40 +11:00
Matt Joiner 77806c1f49 More development of the new logging interface 2018-01-29 18:22:21 +11:00
Matt Joiner 53e32ca9dd Start using new log package 2018-01-28 16:07:11 +11:00
Matt Joiner e08d11a4b9 Update Client.newConnection with changed rateLimitedReader interface 2018-01-28 15:42:37 +11:00
Matt Joiner 51b3374015 Add very tentative UPnP NAT traversal
Relates to #230
2018-01-25 13:14:20 +11:00
Matt Joiner 9285470cc6 Improve some log messages 2018-01-25 13:10:52 +11:00
Matt Joiner 56109b387d Formatting and comments 2018-01-06 16:37:40 +11:00
Matt Joiner e13b0eccbf Make peerID a public type
Wanted it applied to Client Status output
2018-01-06 15:50:45 +11:00
Matt Joiner c44ee5fec4 Use peerID type more, and make its Stringer output nicer 2018-01-06 12:34:31 +11:00
Matt Joiner 8f7720edb2 Merge commit '6ab65a49a8a72dea1a28968b2ab42a85fd4566ec'
# Conflicts:
#	config.go
2017-11-08 20:23:15 +11:00
Matt Joiner 3aa1e8f3ef Add a test for behaviour after getting metadata_size and before GotInfo
#208
2017-11-08 19:31:10 +11:00
Matt Joiner 8712cc25d0 Make Client.PeerID return [20]byte 2017-11-08 19:28:37 +11:00
Matt Joiner 7eb62eec0d Move some code around 2017-11-08 15:00:18 +11:00
themihai 6ab65a49a8 feat(config): Allows the torrent client to customise the client identity and connection config 2017-11-07 20:14:13 +02:00
Matt Joiner 9757f04cf8 Apply megacheck to torrent package 2017-11-07 16:11:59 +11:00
Matt Joiner 82093937d5 Ignore expected error 2017-11-06 14:01:07 +11:00
Matt Joiner 6aad8041ab Implement new request strategy
The new strategy, 2, has the fastest connection download by priority in order, and all other pieces stick to a randomized ordering that's stable per connection.
2017-09-23 15:25:47 +10:00
Matt Joiner bb53c97d38 Move uploading to the connection writer 2017-09-18 12:09:08 +10:00
Matt Joiner ed0fa62340 Comment and formatting 2017-09-17 00:45:12 +10:00
Matt Joiner b39df82fbc Include peer data with half open connections 2017-09-15 19:10:09 +10:00
Matt Joiner 846ac2d33e Herp 2017-09-15 12:56:54 +10:00
Matt Joiner d4576a56d9 Add some expvars to track uTP/TCP and PHE fallbacks for outgoing connections 2017-09-15 12:56:15 +10:00
Matt Joiner 881f1a7e35 Support plaintext crypto method for protocol header encryption 2017-09-13 18:20:20 +10:00
Matt Joiner 110764480e Avoid triggering writer cond where possible 2017-09-01 12:09:41 +10:00
Matt Joiner 13e79039f2 Generate requests, cancels and interest state in the connection writer 2017-08-31 23:48:52 +10:00
Matt Joiner 986b9093a0 Make connection.writer pull from a buffer 2017-08-31 16:26:45 +10:00
Matt Joiner 0388ec1eee Synchronize Torrent.Info 2017-08-29 15:16:53 +10:00
Matt Joiner a8a1ea755d Remove unused functions 2017-08-26 13:23:04 +10:00
Matt Joiner 401974dd8d dht.ServerConfig.StartingNodes is now a function
Additionally cmd/torrentfs uses the global bootstrap nodes for its DHT. Aids with https://github.com/anacrolix/torrent/issues/180
2017-08-25 16:28:27 +10:00
Matt Joiner 97cb41a5c9 Provide default starting nodes to DHT if default torrent client config is used
https://github.com/anacrolix/torrent/issues/178
2017-08-18 11:14:28 +10:00
Matt Joiner 76c60ffa77 Try a state-delta function for updating request state
Also adds Torrent.networkingEnabled, though it isn't yet useful.
2017-08-18 01:51:02 +10:00
Matt Joiner 170bb72c34 Augment dialed connection timeouts with context.Context
Works toward a dial timeout for half open connections. https://github.com/anacrolix/torrent/issues/169
2017-08-16 17:05:05 +10:00
Matt Joiner 9bec8f6a95 Use net.Dialer for TCP 2017-08-16 15:48:30 +10:00
Matt Joiner 8317417910 Use DialContext instead of DialTimeout with utp 2017-08-16 15:35:17 +10:00
Matt Joiner 2d59d48015 Some tidying in Client.dialFirst 2017-08-16 10:32:25 +10:00
Matt Joiner 5099d8e33f DHT bootstrap is not asynchronous 2017-08-10 11:18:48 +10:00
Matt Joiner 2624281842 Changes for dht-cleanup 2017-07-21 00:40:49 +10:00
mlmhl 6286c11bed Do not add nodes if dht is disabled 2017-06-29 14:28:07 +08:00
Matt Joiner ae8b03d586 Use go-libutp if cgo is enabled 2017-06-16 18:08:24 +10:00
Matt Joiner 49648b9ae1 Close implicit Client default storage on Client.Close
Fixes #158
2017-06-01 22:57:08 +10:00
luffyma(马林) d8023a5e39 bug fix: race condition between outgoing connection and incoming connection 2017-05-10 18:31:19 +08:00
Matt Joiner 40b96d0e4e Avoid allocating skeys for receiving encrypted handshakes 2017-04-04 18:41:08 +10:00
Chris Walker 9d96cd659f fix `Spec.Storage` and allow per-torrent dir
`TorrentSpec.Storage` was not honored when calling `Client.AddTorrentSpec`
and the configured `cfg.DefaultStorage` was always used. Now if you construct
your `TorrentSpec` you can specify any `StorageImpl`

Also, the most common use case for custom storage being per-torrent paths for
FileStorage, this adds a `pathMaker` function to the File implementation that
allows customization, along with the default (always use base path) and my use
case (which seemed common enough from the Gitter chat) which is infohash based
subdirectories.

All Public methods have not changed signature, but 1 private method did, hence
the test update.
2017-03-16 14:40:21 +00:00
Matt Joiner 9cf09dba8a Remove unused funcs and idents 2017-01-01 11:03:02 +11:00
Matt Joiner 186b5073d2 Move method queuePieceCheck 2017-01-01 11:02:37 +11:00
Matt Joiner d48c7c8e36 sort imports 2017-01-01 11:01:41 +11:00
Matt Joiner b68d7cd08e dht/… moved to github.com/anacrolix/dht 2016-12-14 11:43:37 +11:00
Matt Joiner 8b50f5fa7a Fix deadlock in Client.onDHTAnnouncePeer 2016-11-27 14:43:21 +11:00
Matt Joiner f451a39b69 Fix harmless race conditions in Client.WriteStatus introduced way back 2016-11-27 14:26:45 +11:00
Matt Joiner 4a7fbf6170 Add peers received from received announce_peer DHT messages to the Client
Addresses #133
2016-11-27 00:05:19 +11:00
Matt Joiner aa697794cc Remove unused function 2016-11-25 17:24:29 +11:00
Matt Joiner 4807c9e29a Make Torrent.conns a map
Can't remember if I've tried this before. But dropping arbitrary connections while iterating established conns will become much simpler.
2016-11-23 11:48:44 +11:00
Matt Joiner ebbd555e7b Move a bunch of Client methods onto more appropriate types 2016-11-22 21:12:53 +11:00
Matt Joiner 0d9348c5d1 Move Client.downloadedChunk to connection.go 2016-11-22 14:16:18 +11:00
Matt Joiner d4cbdc5c38 Add download rate limiting
Fixes #121.
2016-10-10 17:29:39 +11:00
Matt Joiner ed0dbba384 Make newConnection a method on Client 2016-10-10 16:55:56 +11:00
Matt Joiner c34234bf34 Split connection.rw into separate Reader and Writer
This will make it easier to write hooks for Read and Write separately.
2016-10-10 16:30:51 +11:00
Matt Joiner 836bb344f3 Add really simple upload rate limiting and test 2016-10-10 14:58:29 +11:00
Matt Joiner 76c24e70b5 Improve code comments 2016-10-10 14:57:34 +11:00
Matt Joiner 510c03770b Reorder the fields of struct Client 2016-10-10 00:04:14 +11:00
Matt Joiner de761fb506 Do chunk pooling at Torrent instead of connection level 2016-10-05 15:57:00 +11:00
Matt Joiner 3b5f123645 Fix Client.receiveSkeys
Fixes #114. Loop variables in Go are ****ed.
2016-09-21 21:04:52 +10:00
Matt Joiner 7856cc6329 Add some more debug logging 2016-09-21 21:00:18 +10:00
Matt Joiner 9126db177b Add Config.{Force,PreferNo}Encryption 2016-09-16 12:42:41 +10:00
Matt Joiner 2fc814b5eb Move Client.connectionLoop to connection.mainReadLoop 2016-09-11 14:32:56 +10:00
Matt Joiner 5d7882a79d Remove unused trackerTier and shuffleTier 2016-09-11 14:12:03 +10:00
Matt Joiner f47fd8c969 Move TorrentSpec stuff into its own file 2016-09-11 14:09:40 +10:00
Matt Joiner 1e919dd6b1 Rework storage interfaces to make them simpler to implement
This allows lots of behaviour to be baked into the new Client, Torrent and Piece wrappers, rather than duplicating (badly) them in all the backend implementations.
2016-09-02 15:10:57 +10:00
Matt Joiner 8b17d2a63a Rework Reader position changes affecting piece priorities
Torrent.updatePiecePriorities is always followed by Torrent.piecePriorityChanged, so move it into the former function. Also drop the separate mutex for Reader, and add a missing Reader.posChanged call.
2016-08-30 15:41:26 +10:00
Matt Joiner 7557c9bd4e Only parse the config listen addr if we have to
This fixes a broken benchmark
2016-08-30 14:19:29 +10:00
Matt Joiner 2a1cef7c9e Remove the InfoEx type, and don't generate its infohash on the fly
Fixes #106.
2016-08-26 20:29:05 +10:00
Matt Joiner 00f406753a Fix build error with missinggo.Sorted 2016-07-30 00:48:15 +10:00
Matt Joiner 4143b7470d Begin exposing all the magic that Client.WriteStatus has so people can parse Client state their own way
It wasn't possible to move Client.WriteStatus to an external package to verify it doesn't depend on same-package access to data because then it can't be used in the tests, and it's extremely useful there. So I've settled for not locking the Client, and trying to use all the public methods. It's a work in progress.
2016-07-30 00:37:52 +10:00
Matt Joiner 3c75f68413 Kick connections that contribute the final piece of missing metadata that fails to validate
Failing to get metainfo was resetting Torrent.metadataBytes, which caused arriving metadata data to appear to belong to nonexistent pieces, despite passing the check that we'd in fact requested them from the sending connection. This was unnecessarily noisy. Instead return an error if we got all the data, but couldn't set the Torrent's info bytes, that should propagate out and cause the connection that contributed the final missing piece to be dropped. This will at least provide some improved resistance to unfriendly behaviour on the network.
2016-07-24 00:34:40 +10:00
Matt Joiner 14e10ad58d Use utp.Socket.CloseNow to close a Client's uTP socket on Close
Fixes https://github.com/anacrolix/utp/issues/15.
2016-07-24 00:24:44 +10:00
Matt Joiner 980cd69ab2 Break up the DHT announcer code into smaller functions, and add a 5 minute delay between announces 2016-07-23 22:38:31 +10:00
Matt Joiner f5d58b3dc6 Unused *Client argument to Torrent.writeStatus 2016-07-12 21:23:20 +10:00
Matt Joiner fbe0ded844 Add connection read stats 2016-07-12 16:42:04 +10:00
Matt Joiner a4e140b939 New slices package 2016-07-12 16:40:14 +10:00
Matt Joiner 838c35f3ea Solve the mystery bug with TestMaxEstablishedConns
Adding Torrent by infohash wasn't triggering connections to be accepted, when both TCP and uTP were enabled.
2016-07-07 14:49:18 +10:00
Matt Joiner 40a78b7cdd Improve logging heuristic for sending chunks of pieces that go missing 2016-07-06 14:03:11 +10:00
Matt Joiner 38048672a7 Rework locking in Client.acceptConnections 2016-07-06 00:38:43 +10:00
Matt Joiner 326b36545b Prepare to allow max conns per torrent to be configured 2016-07-05 16:23:17 +10:00
Matt Joiner 6b447c8d2b Track closing of outbound TCP connections
Got all inbound and outbound UTP, missed this one I think.
2016-06-20 17:51:40 +10:00
Matt Joiner cee200a5a2 Add piece completion storage
Toward fixing https://github.com/anacrolix/torrent/issues/50.
2016-06-20 17:51:05 +10:00
Matt Joiner b84298351a Move a few methods to Torrent, and track how many DHT announces occur
I suspect that DHT announces may occur too regularly, or without a break for torrents that aren't able to get over the lower water pending peer mark.
2016-06-15 15:29:47 +10:00
Matt Joiner c2bdb31bb3 Client.announceTorrentDHT -> Torrent.announceDHT 2016-06-15 15:13:53 +10:00
Matt Joiner 060ff72175 Improve the logic for which address to use for DHT 2016-05-24 19:46:24 +10:00
Matt Joiner 7a4b491ede Tidy up the listen function 2016-05-24 19:45:42 +10:00
lovedboy ca8d9de944 add return when utpsock's err is not nil 2016-05-24 13:24:29 +08:00
lovedboy 0b36de4be9 Merge branch 'master' of https://github.com/lovedboy/torrent 2016-05-24 13:19:55 +08:00
lovedboy fd1b4dc489 listenedAddr fix when TCP or UDP bind dynamic port 2016-05-24 13:18:04 +08:00
Matt Joiner b55e67dc22 Ban IPs that contribute to failed piece checks 2016-05-24 02:09:47 +10:00
Matt Joiner c6ec935e53 Update Torrent.wantPeersEvent in several places 2016-05-23 10:19:14 +10:00
Matt Joiner 7a60873ec8 Don't create an empty tracker URL if both AnnounceList and Announce are empty 2016-05-22 23:43:55 +10:00
Matt Joiner 20870ec4ff Rework torrent tracker scraping
New scrapers are added anytime trackers are added to a torrent. In the future they will also be stopped as soon as they're removed. All trackers are concurrently scraped, the old-style of sticking to one tracker that works is abandoned for now.
2016-05-22 22:45:08 +10:00
Matt Joiner 8cdcbe9d40 Fix the tracker method receivers and names 2016-05-19 23:58:00 +10:00
Matt Joiner 7600ac70e3 Move torrent tracker methods into torrent.go 2016-05-19 22:49:37 +10:00
Matt Joiner 73a648fb32 Torrent.writeStatus: Include human readable torrent size 2016-05-19 17:15:10 +10:00
Matt Joiner a69044b9ea Break out KRPC stuff from dht into new package 2016-05-17 16:40:08 +10:00
Matt Joiner cf55055d12 Improve the note about chunk writing concurrency
Everytime I see memory spike I suspect this, and it isn't it.
2016-05-17 02:12:06 +10:00
Matt Joiner 02ea8b1081 Rename storage.I->Client 2016-05-16 21:50:43 +10:00
Matt Joiner cbac2ecaff Stop returning errors from the 2 callstacks to connectionLoop where errors are expected 2016-05-16 19:50:10 +10:00
Matt Joiner c0283d3314 Return an error from Torrent.setMetadataSize
This should mean connections sending ridiculous metadata sizes get dropped.
2016-05-16 18:48:56 +10:00
Matt Joiner d2951c6c4b Move requestPendingMetadata onto connection from Client 2016-05-16 18:46:38 +10:00
Matt Joiner 2b46a17b39 Bit of a tidy in the tracker code 2016-05-12 12:43:37 +10:00
Matt Joiner 528836ab4c Get rid of obsoleted Torrent.ceaseNetworking, and clean up Torrent.closing 2016-05-11 21:44:55 +10:00
Matt Joiner dce3a7f675 Rework Client listeners
In particular, if the ListenAddr used a dynamic port ":0", and both TCP and uTP were enabled. If the TCP listen succeeded, and the uTP did not, the TCP listener was leaked, and another port number was not tried.
2016-05-11 21:11:52 +10:00
Matt Joiner 948552b282 Remove unused Client.pruneTimer 2016-05-11 21:08:55 +10:00
Matt Joiner 715a74985f Wasn't using ChunkSize from TorrentSpec 2016-05-09 23:00:20 +10:00
Matt Joiner a45a09b877 Rework setting of info bytes 2016-05-09 15:47:39 +10:00
Matt Joiner cee49435e0 Add Client.AddTorrentInfoHash, and some other tidying
Remove Client.bannedTorrents, which shouldn't have been in use anymore. Also fix Client.newTorrent.
2016-05-09 14:37:29 +10:00
Matt Joiner 69643860ea Move some behaviour into Client.newConnection 2016-05-07 18:57:38 +10:00
Matt Joiner 0fd73396fd missinggo.Event changed, connection.writeOptimizer changes 2016-05-07 18:56:44 +10:00
Matt Joiner b5812bb0b7 Remove the last of the "config dir" stuff
This just conflates the Client. It should be done orthogonally.
2016-05-05 23:05:28 +10:00
Matt Joiner 2d160b0419 metainfo.InfoEx.Hash becomes a function, UpdateBytes is added, and Bytes used in marshalling
Fixes #80.
2016-05-05 22:40:38 +10:00
Matt Joiner f6b83130c9 Move a bunch of Torrent methods from client.go 2016-05-03 16:47:11 +10:00
Matt Joiner d47ba4a08f Rework handling of metadata data messages
There are some dubious logs probably resulting from poor ordering of some code for handling incoming metadata data messages.
2016-05-03 14:59:54 +10:00
Matt Joiner 726e763c46 Doc and logging improvements 2016-05-03 14:58:26 +10:00
Seth Hoenig 9f4bf70e17 refactor struct identifiers to follow conventional go names 2016-04-18 23:11:11 -05:00
Matt Joiner da4d40ed7e Merge all our global consts and vars into global.go
For easier referral to other parties.
2016-04-14 17:19:07 +10:00
Matt Joiner f83771b660 Apply ineffassign linter 2016-04-04 15:39:26 +10:00
Matt Joiner 82ee860e8d Apply unconvert linter 2016-04-04 15:28:25 +10:00
Matt Joiner 1566e2afdc Remove deadcode 2016-04-04 15:18:51 +10:00
Matt Joiner 8f164ae956 Remove some of the magic ConfigDir stuff
This might become a helper. Torrent file cache still remains.
2016-04-04 14:04:48 +10:00
Matt Joiner fa511154e4 Move Magnet and friends into metainfo package 2016-04-04 13:48:39 +10:00
Matt Joiner b80bb6393a Use metainfo.Hash for piece and info hashes 2016-04-04 13:01:31 +10:00
Matt Joiner 00f33c848e Adding a torrent without info after waitAccept has blocked wouldn't trigger further accepts immediately 2016-04-03 22:06:25 +10:00
Matt Joiner 22ad654420 Fix deadlocks in WriteStatus due to incomplete renaming 2016-04-03 20:54:14 +10:00
Matt Joiner d3a1c79c79 Merge Torrent and torrent types 2016-04-03 18:40:43 +10:00
Matt Joiner 08830b97a1 Make everything on type torrent private 2016-04-03 16:50:53 +10:00
Matt Joiner bc8fb2a165 If we fail to write to a piece, check its completion state 2016-04-03 16:36:57 +10:00
Matt Joiner 44f5bb73c5 Improve log messages 2016-04-03 16:36:24 +10:00
Matt Joiner 5856b91af0 Fix missinggo import 2016-03-30 19:11:55 +11:00
Matt Joiner a5b54f21a1 Make opening a torrent in storage an explicit method
This is storage types where opening can fail, like mmap
2016-03-28 22:40:29 +11:00
Matt Joiner 775cf53809 Get mmap storage working 2016-03-28 21:57:04 +11:00