Full-featured BitTorrent client package and utilities
Go to file
Matt Joiner 506996d038 Fix panic in update requests timer func on closed conn 2021-10-25 16:16:06 +11:00
.circleci Don't bench with -race 2021-02-24 13:48:05 +11:00
.github/workflows Set Go CI workflow to use Go 1.17 2021-08-30 11:19:16 +10:00
analysis Add peer upload order analysis 2021-01-28 16:31:13 +11:00
bencode Limit decoded bencode string lengths to 32 bits 2021-09-29 16:56:01 +10:00
cmd cmd/torrent: Use updated args module 2021-10-20 16:41:33 +11:00
common
fs Fixes for wasm 2021-06-23 17:24:50 +10:00
internal Extract chansync to github.com/anacrolix/chansync 2021-06-07 13:01:40 +10:00
iplist Remove old-style build tags 2021-10-07 09:12:17 +11:00
logonce
metainfo Update fuzz to use new standard library support 2021-09-29 11:51:03 +10:00
mmap_span Fix excessive logging from mmap-backed storage (anacrolix/torrent#436) 2020-11-12 08:57:42 +11:00
mse Drop bradfitz/iter dependency (#605) 2021-09-14 13:46:50 +10:00
peer_protocol Fix incomplete Integer.Read 2021-10-19 14:07:23 +11:00
request-strategy Optimize chunk calculations in request strategy 2021-10-19 14:08:56 +11:00
segments
storage Remove old-style build tags 2021-10-07 09:12:17 +11:00
test Add to tests for Torrent.Complete 2021-10-07 13:38:29 +11:00
testdata Merge branch 'fuse-email-issue' 2021-02-22 10:05:32 +11:00
tracker Fix http announce of infohash containing ' ' bytes (#591) 2021-09-12 14:11:59 +10:00
types Extract request strategy stuff into a separate module 2021-06-07 13:01:39 +10:00
util/dirwatch Add linter CI (#542) 2021-08-16 11:11:31 +10:00
version Update and expose default client identifiers 2021-06-25 11:29:59 +10:00
webseed Treat 404 responses from webseed peers as fatal 2021-02-18 14:36:08 +11:00
webtorrent Fix race running and closing webtorrent tracker clients 2021-10-25 16:15:42 +11:00
.deepsource.toml fixed code quality issues using DeepSource 2021-01-30 15:23:49 +11:00
.gitignore Ignore request strategy run gobs and envrcs 2021-10-19 14:08:56 +11:00
.golangci.yml Update staticcheck go target to 1.16 2021-08-19 16:06:24 +10:00
LICENSE
NOTES.md Add NOTES 2021-09-20 12:31:44 +10:00
README.md cmd/torrent: Use anacrolix/args and merge several other cmds in as subcommands 2021-09-15 10:54:14 +10:00
SECURITY.md Create SECURITY.md 2021-05-14 10:17:19 +10:00
TODO
bad_storage.go Rework storage.TorrentImpl to support shared capacity key 2021-06-07 13:01:39 +10:00
bep40.go
bep40_test.go
callbacks.go Expose more callbacks and Request and ChunkSpec 2021-01-28 14:23:22 +11:00
client-nowasm_test.go Remove old-style build tags 2021-10-07 09:12:17 +11:00
client.go Fix panic in update requests timer func on closed conn 2021-10-25 16:16:06 +11:00
client_test.go Add reasons for updateRequests to be triggered 2021-10-19 14:08:56 +11:00
config.go Eliminate `expect` dependency (#667) 2021-09-24 00:10:30 +10:00
conn_stats.go Track chunks read for webseed peers too 2021-06-07 13:01:40 +10:00
deferrwl.go lockWithDeferreds: inlineable Unlock() (#603) 2021-09-17 13:08:00 +10:00
dht.go Expose AnacrolixDhtServerWrapper 2021-06-07 13:01:40 +10:00
dialer.go Tidy up the Dialer interface 2021-06-21 13:29:26 +10:00
doc.go
example_test.go
file.go Add reasons for updateRequests to be triggered 2021-10-19 14:08:56 +11:00
file_test.go Use roaring.Bitmap directly for completed pieces 2021-08-16 11:07:10 +10:00
global.go go fmt ./... 2021-10-05 17:48:52 +11:00
go.mod cmd/torrent: Use updated args module 2021-10-20 16:41:33 +11:00
go.sum cmd/torrent: Use updated args module 2021-10-20 16:41:33 +11:00
handshake.go Add the ReceiveEncryptedHandshakeSkeys callback 2020-11-05 13:28:45 +11:00
handshake_test.go
ipport.go Possible fix for GitHub code scanning warning 2021-05-14 15:43:13 +10:00
issue97_test.go Avoid allocation putting undirtiedChunksIter into interface 2021-10-19 14:08:56 +11:00
issue211_test.go Remove old-style build tags 2021-10-07 09:12:17 +11:00
listen.go Optimize `LoopbackListenHost()` (#615) 2021-09-14 21:54:37 +10:00
main_test.go Replace internal/tmproot with Testing.TB.TempDir 2021-02-22 14:30:09 +11:00
misc.go Rework Reader waiting 2021-09-04 23:07:32 +10:00
misc_test.go Drop bradfitz/iter dependency (#605) 2021-09-14 13:46:50 +10:00
multiless.go Fixes for wasm 2021-06-23 17:24:50 +10:00
network_test.go Fixes for wasm 2021-06-23 17:24:50 +10:00
networks.go
peer-conn-msg-writer.go Avoid reallocating keep alive timer on each pass 2021-08-22 20:05:53 +10:00
peer-impl.go Add low requests check 2021-10-21 10:48:43 +11:00
peer_info.go Don't expose Peer network through RemoteAddr 2021-01-25 15:49:48 +11:00
peer_infos.go
peerconn.go Fix panic when sustaining unrejected requests after being choked 2021-10-25 13:00:56 +11:00
peerconn_test.go Use roaring bitmap for pending pieces 2021-10-19 14:08:56 +11:00
peerid.go
peerid_test.go
pending-requests.go Use a flat slice for pending request counts 2021-10-19 14:08:56 +11:00
pex.go Inlineable `addrIndex` (#673) 2021-09-27 12:41:20 +10:00
pex_test.go Add linter CI (#542) 2021-08-16 11:11:31 +10:00
pexconn.go PEX: impede full-meching in tracker-less swarms by adding a cooldown minute 2021-02-24 08:29:18 +11:00
pexconn_test.go Create default constructor for Client (#567) 2021-09-14 23:01:20 +10:00
piece.go Avoid allocation putting undirtiedChunksIter into interface 2021-10-19 14:08:56 +11:00
piecestate.go Track piece marking state separately 2020-11-21 13:44:23 +11:00
portfwd.go Big logging cleanup to improve experience from README 2021-05-04 12:44:51 +10:00
prioritized_peers.go
prioritized_peers_test.go
protocol.go Expose more callbacks and Request and ChunkSpec 2021-01-28 14:23:22 +11:00
ratelimitreader.go
reader.go go fmt ./... 2021-10-05 17:48:52 +11:00
reader_test.go Replace internal/tmproot with Testing.TB.TempDir 2021-02-22 14:30:09 +11:00
requesting.go Fix panic when sustaining unrejected requests after being choked 2021-10-25 13:00:56 +11:00
requesting_test.go Add some tests verifying request map ordering 2021-09-18 13:50:55 +10:00
rlreader_test.go Drop bradfitz/iter dependency (#605) 2021-09-14 13:46:50 +10:00
roaring.go Track dirty chunks in a single bitmap on Torrent 2021-09-20 18:52:54 +10:00
socket.go Fixes for wasm 2021-06-23 17:24:50 +10:00
spec.go go fmt 2021-10-19 14:08:56 +11:00
struct_test.go Export Peer 2021-01-25 15:49:48 +11:00
t.go Add reasons for updateRequests to be triggered 2021-10-19 14:08:56 +11:00
testing.go Set testing keep alive timeout to 1 millisecond 2021-08-19 13:37:00 +10:00
torrent.go Demote Torrent.runHandshookConnLoggingErr level to debug 2021-10-21 11:40:06 +11:00
torrent_mmap_test.go Remove old-style build tags 2021-10-07 09:12:17 +11:00
torrent_pending_pieces.go
torrent_stats.go Add TorrentStats.PiecesComplete 2021-10-07 13:50:13 +11:00
torrent_test.go Add reasons for updateRequests to be triggered 2021-10-19 14:08:56 +11:00
tracker_scraper.go Rework Reader waiting 2021-09-04 23:07:32 +10:00
url-net-addr.go Generate a RemoteAddr for webseed Peers 2021-01-25 15:49:48 +11:00
utp.go
utp_go.go Fix go:build directives 2021-07-14 14:35:52 +10:00
utp_libutp.go Remove old-style build tags 2021-10-07 09:12:17 +11:00
utp_test.go
webrtc.go Upgrade to pion/webrtc@v3 2021-02-21 13:46:37 -08:00
webseed-peer.go Add low requests check 2021-10-21 10:48:43 +11:00
worst_conns.go Export Peer 2021-01-25 15:49:48 +11:00
wstracker.go Fix race running and closing webtorrent tracker clients 2021-10-25 16:15:42 +11:00

README.md

torrent

PkgGoDev

This repository implements BitTorrent-related packages and command-line utilities in Go. The emphasis is on use as a library from other projects. It's been used 24/7 in production by downstream services since late 2014. The implementation was specifically created to explore Go's concurrency capabilities, and to include the ability to stream data directly from the BitTorrent network. To this end it supports seeking, readaheads and other features exposing torrents and their files with the various Go idiomatic io package interfaces. This is also demonstrated through torrentfs.

There is support for protocol encryption, DHT, PEX, uTP, and various extensions. There are several data storage backends provided: blob, file, bolt, mmap, and sqlite, to name a few. You can write your own to store data for example on S3, or in a database.

Some noteworthy package dependencies that can be used for other purposes include:

Installation

Install the library package with go get github.com/anacrolix/torrent, or the provided cmds with go install github.com/anacrolix/torrent/cmd/...@latest.

Library examples

There are some small examples in the package documentation.

Mentions

Downstream projects

There are several web-frontends and Android clients among the known public projects:

Help

Communication about the project is primarily through Discussions and the issue tracker.

Command packages

Here I'll describe what some of the packages in ./cmd do. See installation to make them available.

torrent

torrent download

Downloads torrents from the command-line.

$ torrent download 'magnet:?xt=urn:btih:KRWPCX3SJUM4IMM4YF5RPHL6ANPYTQPU'
... lots of jibba jabber ...
downloading "ubuntu-14.04.2-desktop-amd64.iso": 1.0 GB/1.0 GB, 1989/1992 pieces completed (1 partial)
2015/04/01 02:08:20 main.go:137: downloaded ALL the torrents
$ md5sum ubuntu-14.04.2-desktop-amd64.iso
1b305d585b1918f297164add46784116  ubuntu-14.04.2-desktop-amd64.iso
$ echo such amaze
wow

torrent metainfo magnet

Creates a magnet link from a torrent file. Note the extracted trackers, display name, and info hash.

$ torrent metainfo testdata/debian-10.8.0-amd64-netinst.iso.torrent magnet
magnet:?xt=urn:btih:4090c3c2a394a49974dfbbf2ce7ad0db3cdeddd7&dn=debian-10.8.0-amd64-netinst.iso&tr=http%3A%2F%2Fbttracker.debian.org%3A6969%2Fannounce

See torrent metainfo --help for other metainfo related commands.

torrentfs

torrentfs mounts a FUSE filesystem at -mountDir. The contents are the torrents described by the torrent files and magnet links at -metainfoDir. Data for read requests is fetched only as required from the torrent network, and stored at -downloadDir.

$ mkdir mnt torrents
$ torrentfs -mountDir=mnt -metainfoDir=torrents &
$ cd torrents
$ wget http://releases.ubuntu.com/14.04.2/ubuntu-14.04.2-desktop-amd64.iso.torrent
$ cd ..
$ ls mnt
ubuntu-14.04.2-desktop-amd64.iso
$ pv mnt/ubuntu-14.04.2-desktop-amd64.iso | md5sum
996MB 0:04:40 [3.55MB/s] [========================================>] 100%
1b305d585b1918f297164add46784116  -