Full-featured BitTorrent client package and utilities
Go to file
Matt Joiner bbeb69bb5f Ignore cached piece completion state when verifying data
Forcing data to be checked via VerifyData, and probably other places where piece checks are queued were being short-circuited due to cached piece completion. If a piece check is queued, or verifyData called, the cached completion state should be ignored.
2019-07-25 14:10:21 +10:00
.circleci Update CircleCI to use go1.12 instead of master 2019-04-24 09:35:09 +10:00
.github Create FUNDING.yml 2019-06-24 14:12:14 +10:00
bencode bencode: Decode singleton lists of the expected type 2019-06-13 13:07:37 +10:00
cmd Comments 2019-07-19 13:04:10 +10:00
fs Change the default client port 2019-07-19 16:19:21 +10:00
internal/testutil Switch to goimports import sorting 2018-11-02 23:12:01 +11:00
iplist Switch to goimports import sorting 2018-11-02 23:12:01 +11:00
logonce Add a prefix to logonce logger output 2018-09-03 11:48:31 +10:00
metainfo metainfo: Fix test 2018-11-27 11:05:32 +11:00
mmap_span mmap_span: Synchronize access to memory maps to avoid race when unmapping 2018-01-06 16:39:52 +11:00
mse Comments 2019-07-19 13:04:10 +10:00
peer_protocol Always return errors when handshakes or dialing fails 2019-07-19 16:15:46 +10:00
storage Switch to goimports import sorting 2018-11-02 23:12:01 +11:00
testdata Add the fs test torrent to the repo 2017-12-29 13:50:26 +11:00
tracker Fix announcing to S3 HTTP trackers 2019-07-17 18:12:11 +10:00
util/dirwatch Switch to goimports import sorting 2018-11-02 23:12:01 +11:00
.gitignore Ignore ".torrent.bolt.db" which is generated by unit tests 2019-06-21 14:53:57 +10:00
LICENSE Update LICENSE 2018-01-27 11:21:33 +11:00
Peer.go Move IpPort to missinggo 2018-11-16 10:35:30 +11:00
Peers.go Switch to goimports import sorting 2018-11-02 23:12:01 +11:00
README.md Add astronomer rating badge 2019-07-22 22:42:05 +10:00
TODO Update TODO file 2018-04-14 21:44:50 +10:00
bad_storage.go Some test tidying and improvements 2018-01-29 18:22:21 +11:00
bep40.go Move IpPort to missinggo 2018-11-16 10:35:30 +11:00
bep40_test.go Remove unnecessary blank line 2019-01-30 15:18:00 +11:00
client.go Add some commented out logging 2019-07-19 16:17:38 +10:00
client_test.go Change the default client port 2019-07-19 16:19:21 +10:00
closewrapper.go Add initial connection tracking 2018-11-21 17:02:22 +11:00
config.go Improve comments on some config fields 2019-07-23 11:30:05 +10:00
conn_stats.go allow ConnStats being marsheled to json 2019-05-29 15:05:31 +10:00
connection.go Track concurrent chunk writes 2019-06-13 12:18:24 +10:00
connection_test.go Move IpPort to missinggo 2018-11-16 10:35:30 +11:00
doc.go List implemented BEPs 2016-07-11 13:53:00 +10:00
example_test.go Add File.NewReader 2018-01-06 16:37:13 +11:00
file.go Remove unused code 2018-11-27 23:57:17 +11:00
file_test.go Add File.Cancel 2016-02-05 01:18:54 +11:00
global.go Track concurrent chunk writes 2019-06-13 12:18:24 +10:00
go.mod Always return errors when handshakes or dialing fails 2019-07-19 16:15:46 +10:00
go.sum Always return errors when handshakes or dialing fails 2019-07-19 16:15:46 +10:00
handshake.go Rework header obfuscation and add tests for fallbacks 2019-07-19 13:23:36 +10:00
handshake_test.go Support AllowedFast and enable fast extension 2018-02-05 00:18:38 +11:00
issue97_test.go Switch to goimports import sorting 2018-11-02 23:12:01 +11:00
issue211_test.go Switch to goimports import sorting 2018-11-02 23:12:01 +11:00
listen.go Remove unused code 2018-11-27 23:57:17 +11:00
main_test.go Fix TestSetMaxEstablishedConn and allow it to be run with -count > 1 2018-02-11 15:14:31 +11:00
misc.go Move IpPort to missinggo 2018-11-16 10:35:30 +11:00
misc_test.go Remove excess newline from log call 2018-06-13 10:56:09 +10:00
multiless.go Move multiless to missinggo 2019-03-03 14:31:08 +11:00
network_test.go Use the new firewall callback support in go-libutp 2018-07-25 17:11:09 +10:00
networks.go Improve network handling and only listen networks we will use 2018-11-28 10:30:21 +11:00
peerid.go Change a bunch of Stringer and status stuff 2018-02-05 17:47:47 +11:00
peerid_test.go Change a bunch of Stringer and status stuff 2018-02-05 17:47:47 +11:00
piece.go Reduce allocations for Piece.hash 2019-01-30 17:54:02 +11:00
piecestate.go Remove vestigial comment 2018-05-02 19:23:05 +10:00
portfwd.go Law of Demeter Client.mu 2018-07-25 13:42:28 +10:00
prioritized_peers.go Add tests for prioritizedPeers 2018-04-14 21:44:03 +10:00
prioritized_peers_test.go Move IpPort to missinggo 2018-11-16 10:35:30 +11:00
protocol.go Abstract out making a cancel message 2017-09-18 13:42:42 +10:00
ratelimitreader.go Fix download rate limiter and add test 2018-01-28 15:28:45 +11:00
reader.go When failing to read stored data, try updating only the completion state for the failed piece 2019-04-09 12:57:54 +10:00
reader_test.go Switch to goimports import sorting 2018-11-02 23:12:01 +11:00
rlreader_test.go sortimports 2018-01-31 16:42:40 +11:00
socket.go Improve network handling and only listen networks we will use 2018-11-28 10:30:21 +11:00
spec.go Add error return value to metainfo.MetaInfo.UnmarshalInfo 2016-09-20 12:32:15 +10:00
t.go Fix race condition in Torrent.SetDisplayName 2019-03-20 11:01:56 +11:00
torrent.go Ignore cached piece completion state when verifying data 2019-07-25 14:10:21 +10:00
torrent_stats.go Merge a bunch of stuff into ConnStats and refactor connection.upload 2018-02-02 19:04:56 +11:00
torrent_test.go Always return errors when handshakes or dialing fails 2019-07-19 16:15:46 +10:00
tracker_scraper.go Send tracker stopped event from the tracker scraper routine 2019-07-17 11:56:25 +10:00
utp.go Add support for disabling IPv4 and IPv4 peers 2018-02-16 10:46:11 +11:00
utp_go.go Use the new firewall callback support in go-libutp 2018-07-25 17:11:09 +10:00
utp_libutp.go Use the new firewall callback support in go-libutp 2018-07-25 17:11:09 +10:00
utp_test.go Use the new firewall callback support in go-libutp 2018-07-25 17:11:09 +10:00
worst_conns.go Calculate peer priority lazily for worse conn comparison 2018-07-15 12:57:52 +10:00

README.md

torrent

Join the chat at https://gitter.im/anacrolix/torrent GoDoc CircleCI Astronomer Rating

This repository implements BitTorrent-related packages and command-line utilities in Go. The emphasis is on use as a library from other projects. It's been used 24/7 in production by downstream services since late 2014. The implementation was specifically created to explore Go's concurrency capabilities, and to include the ability to stream data directly from the BitTorrent network. To this end it supports seeking, readaheads and other features exposing torrents and their files with the various Go idiomatic io package interfaces. This is also demonstrated through torrentfs.

There is support for protocol encryption, DHT, PEX, uTP, and various extensions. There are several data storage backends provided: blob, file, bolt, and mmap, to name a few. You can write your own to store data for example on S3, or in a database.

Some noteworthy package dependencies that can be used for other purposes include:

Installation

Install the library package with go get github.com/anacrolix/torrent, or the provided cmds with go get github.com/anacrolix/torrent/cmd/....

Library examples

There are some small examples in the package documentation.

Downstream projects

There are several web-frontends and Android clients among the known public projects:

Help

Communication about the project is primarily through Gitter and the issue tracker.

Command packages

Here I'll describe what some of the packages in ./cmd do.

Note that the godo command which is invoked in the following examples builds and executes a Go import path, like go run. It's easier to use this convention than to spell out the install/invoke cycle for every single example.

torrent

Downloads torrents from the command-line. This first example does not use godo.

$ go get github.com/anacrolix/torrent/cmd/torrent
# Now 'torrent' should be in $GOPATH/bin, which should be in $PATH.
$ torrent 'magnet:?xt=urn:btih:KRWPCX3SJUM4IMM4YF5RPHL6ANPYTQPU'
ubuntu-14.04.2-desktop-amd64.iso [===================================================================>]  99% downloading (1.0 GB/1.0 GB)
2015/04/01 02:08:20 main.go:137: downloaded ALL the torrents
$ md5sum ubuntu-14.04.2-desktop-amd64.iso
1b305d585b1918f297164add46784116  ubuntu-14.04.2-desktop-amd64.iso
$ echo such amaze
wow

torrentfs

torrentfs mounts a FUSE filesystem at -mountDir. The contents are the torrents described by the torrent files and magnet links at -metainfoDir. Data for read requests is fetched only as required from the torrent network, and stored at -downloadDir.

$ mkdir mnt torrents
$ godo github.com/anacrolix/torrent/cmd/torrentfs -mountDir=mnt -metainfoDir=torrents &
$ cd torrents
$ wget http://releases.ubuntu.com/14.04.2/ubuntu-14.04.2-desktop-amd64.iso.torrent
$ cd ..
$ ls mnt
ubuntu-14.04.2-desktop-amd64.iso
$ pv mnt/ubuntu-14.04.2-desktop-amd64.iso | md5sum
996MB 0:04:40 [3.55MB/s] [========================================>] 100%
1b305d585b1918f297164add46784116  -

torrent-magnet

Creates a magnet link from a torrent file. Note the extracted trackers, display name, and info hash.

$ godo github.com/anacrolix/torrent/cmd/torrent-magnet < ubuntu-14.04.2-desktop-amd64.iso.torrent
magnet:?xt=urn:btih:546cf15f724d19c4319cc17b179d7e035f89c1f4&dn=ubuntu-14.04.2-desktop-amd64.iso&tr=http%3A%2F%2Ftorrent.ubuntu.com%3A6969%2Fannounce&tr=http%3A%2F%2Fipv6.torrent.ubuntu.com%3A6969%2Fannounce