Full-featured BitTorrent client package and utilities
Go to file
guoguangwu 7036f221f4 chore: remove refs to deprecated io/ioutil 2023-07-10 00:20:13 +10:00
.circleci Don't bench with -race 2021-02-24 13:48:05 +11:00
.github Run torrentfs CI on macos 2023-05-28 16:28:50 +10:00
analysis gorond ./... 2022-11-15 23:31:27 +11:00
bencode chore: remove refs to deprecated io/ioutil 2023-07-10 00:20:13 +10:00
cmd chore: remove refs to deprecated io/ioutil 2023-07-10 00:20:13 +10:00
common
dialer Move a bunch of stuff into subpackages 2022-11-15 23:31:26 +11:00
fs fs/test.sh: Handle torrentfs failures synchronously 2023-05-28 13:52:50 +10:00
internal chore: remove refs to deprecated io/ioutil 2023-07-10 00:20:13 +10:00
iplist gofumpt 2021-11-08 14:47:01 +11:00
logonce
metainfo chore: remove refs to deprecated io/ioutil 2023-07-10 00:20:13 +10:00
mmap_span Fixes for storage tests on Windows 2023-05-28 00:41:35 +10:00
mse chore: remove refs to deprecated io/ioutil 2023-07-10 00:20:13 +10:00
peer_protocol Fix incorrect EOF when decoding some peer protocol message types 2023-05-29 19:01:01 +10:00
request-strategy Camel case requestStrategy package 2023-04-29 14:54:49 +10:00
segments gofumpt 2021-11-08 14:47:01 +11:00
smartban Implement smart ban using generics 2022-01-22 18:43:04 +11:00
storage chore: remove refs to deprecated io/ioutil 2023-07-10 00:20:13 +10:00
test chore: remove refs to deprecated io/ioutil 2023-07-10 00:20:13 +10:00
testdata Add sintel.torrent 2021-12-12 15:01:50 +11:00
tests/issue-798 gorond 2023-01-04 01:26:31 +11:00
tracker chore: remove refs to deprecated io/ioutil 2023-07-10 00:20:13 +10:00
typed-roaring Use reusable roaring iterators 2022-05-12 10:42:16 +10:00
types Move a bunch of stuff into subpackages 2022-11-15 23:31:26 +11:00
util/dirwatch gorond ./... 2022-11-15 23:31:27 +11:00
version Update and expose default client identifiers 2021-06-25 11:29:59 +10:00
webseed Don't use path.Join in default webseed path escaper 2022-12-25 18:20:42 +11:00
webtorrent Add WebRTC ICE servers config (#824) 2023-05-27 20:27:24 +10:00
.deepsource.toml Update .deepsource.toml 2022-02-21 21:23:20 +00:00
.gitignore Ignore request strategy run gobs and envrcs 2021-10-19 14:08:56 +11:00
.golangci.yml Update staticcheck go target to 1.16 2021-08-19 16:06:24 +10:00
LICENSE
NOTES.md Add more hole punching NOTES 2023-05-09 15:45:52 +10:00
README.md Add exatorrent and finish tidying downstream projects 2022-06-25 17:28:23 +10:00
SECURITY.md Create SECURITY.md 2021-05-14 10:17:19 +10:00
TODO
bad_storage.go Rework storage.TorrentImpl to support shared capacity key 2021-06-07 13:01:39 +10:00
bep40.go
bep40_test.go Use webrtc local addr for webrtc conn peer priority 2022-07-12 15:52:03 +10:00
callbacks.go Expose more callbacks and Request and ChunkSpec 2021-01-28 14:23:22 +11:00
client-nowasm_test.go gorond 2023-05-28 12:04:57 +10:00
client-stats.go Attribute accepted connection to holepunching when connect message is late 2023-05-28 13:53:22 +10:00
client.go Attribute accepted connection to holepunching when connect message is late 2023-05-28 13:53:22 +10:00
client_test.go More Windows test fixes 2023-05-28 00:41:36 +10:00
config.go Add WebRTC ICE servers config (#824) 2023-05-27 20:27:24 +10:00
conn_stats.go Track chunks read for webseed peers too 2021-06-07 13:01:40 +10:00
deferrwl.go lockWithDeferreds: inlineable Unlock() (#603) 2021-09-17 13:08:00 +10:00
dht.go Don't wait for DHT pings 2021-12-23 14:00:00 +11:00
dial-pool.go Attempt holepunch after initial dial fails 2023-05-11 13:03:54 +10:00
dialer.go Move a bunch of stuff into subpackages 2022-11-15 23:31:26 +11:00
doc.go New go fmt on doc comments 2022-12-07 10:44:05 +11:00
example_test.go
file.go use rLock where can, part2 (#767) 2022-07-13 20:04:03 +10:00
file_test.go gofumpt 2021-11-08 14:47:01 +11:00
global.go Add many more extension bit definitions 2023-05-09 15:46:53 +10:00
go.mod Test and fix closed sqlite storage panicking during piece hashing 2023-06-26 10:24:06 +10:00
go.sum Test and fix closed sqlite storage panicking during piece hashing 2023-06-26 10:24:06 +10:00
handshake.go Add the ReceiveEncryptedHandshakeSkeys callback 2020-11-05 13:28:45 +11:00
handshake_test.go
ipport.go Fix addrPortOrZero for unix sockets on Windows 2023-05-28 00:41:36 +10:00
issue97_test.go Test fixes for Windows 2023-05-28 00:41:35 +10:00
issue211_test.go chore: remove refs to deprecated io/ioutil 2023-07-10 00:20:13 +10:00
listen.go Optimize `LoopbackListenHost()` (#615) 2021-09-14 21:54:37 +10:00
main_test.go Tweak logging 2023-05-18 21:25:58 +10:00
misc.go gofumpt 2023-02-18 18:35:22 +11:00
misc_test.go Drop bradfitz/iter dependency (#605) 2021-09-14 13:46:50 +10:00
netip-addrport.go go1.19 compatibility 2023-05-23 20:55:30 +10:00
network_test.go Test fixes for Windows 2023-05-28 00:41:35 +10:00
networks.go
ordered-bitmap.go Import generics as g 2023-04-04 19:12:26 +10:00
otel.go webtorrent: Create data channel earlier per webrtc examples 2022-07-12 16:15:50 +10:00
peer-conn-msg-writer.go Fix write error handling 2022-12-25 19:24:16 +11:00
peer-impl.go Sleep webseed peers after unhandled errors 2022-12-31 11:27:47 +11:00
peer.go Move PeerClientName and PeerExtensionIDs to PeerConn 2023-05-19 14:41:27 +10:00
peer_info.go Don't expose Peer network through RemoteAddr 2021-01-25 15:49:48 +11:00
peer_infos.go
peerconn.go Fix logging reported in #841 2023-05-28 12:04:49 +10:00
peerconn_test.go chore: unnecessary use of fmt.Sprintf 2023-07-09 22:16:22 +10:00
peerid.go Move a bunch of stuff into subpackages 2022-11-15 23:31:26 +11:00
peerid_test.go
pending-requests.go Revert "Use a flat slice for pending request counts" 2021-11-25 22:55:02 +11:00
pending-requests_test.go Comment out pending requests tests and asserts 2021-11-25 22:59:20 +11:00
pex.go Reintroduce torrent-wide PEX throttling 2023-05-28 20:07:33 +10:00
pex_test.go Use netip.AddrPort in PEX code and filter unusable addrs much sooner 2023-05-16 17:55:53 +10:00
pexconn.go Reintroduce torrent-wide PEX throttling 2023-05-28 20:07:33 +10:00
pexconn_test.go Use netip.AddrPort in PEX code and filter unusable addrs much sooner 2023-05-16 17:55:53 +10:00
piece.go gorond ./... 2022-11-15 23:31:27 +11:00
piecestate.go Track piece marking state separately 2020-11-21 13:44:23 +11:00
portfwd.go Big logging cleanup to improve experience from README 2021-05-04 12:44:51 +10:00
prioritized-peers.go Rename prioritized_peers go files 2023-04-29 14:54:49 +10:00
prioritized-peers_test.go Rename prioritized_peers go files 2023-04-29 14:54:49 +10:00
protocol.go Expose more callbacks and Request and ChunkSpec 2021-01-28 14:23:22 +11:00
ratelimitreader.go
reader.go Fix races around Reader and Torrent.Drop when Torrent is closed 2022-04-11 15:44:24 +10:00
reader_test.go Replace internal/tmproot with Testing.TB.TempDir 2021-02-22 14:30:09 +11:00
request-strategy-impls.go Add and use typed roaring bitmap 2022-05-09 12:05:50 +10:00
requesting.go Don't panic if changing interest fills the write buffer 2023-05-19 14:09:27 +10:00
requesting_test.go gorond test files 2023-05-12 13:47:47 +10:00
reuse_test.go Improve test assertion for Linux 2023-05-17 13:45:18 +10:00
rlreader_test.go Drop bradfitz/iter dependency (#605) 2021-09-14 13:46:50 +10:00
roaring.go Add and use typed roaring bitmap 2022-05-09 12:05:50 +10:00
smartban.go Import generics as g 2023-04-04 19:12:26 +10:00
socket.go Don't dial out TCP from the listen port 2023-05-09 15:45:52 +10:00
sockopts.go Dial TCP with the listener's local addr 2023-05-09 15:45:52 +10:00
sockopts_unix.go Dial TCP with the listener's local addr 2023-05-09 15:45:52 +10:00
sockopts_wasm.go Dial TCP with the listener's local addr 2023-05-09 15:45:52 +10:00
sockopts_windows.go Dial TCP with the listener's local addr 2023-05-09 15:45:52 +10:00
sources.go Don't log using torrent source as warning for context cancellation 2023-04-04 19:13:14 +10:00
spec.go WebSeed PathEscaper API tweaks 2022-04-26 10:46:01 +10:00
stats.go Add Client.Stats with ActiveHalfOpenAttempts 2023-05-09 15:46:52 +10:00
string-addr.go Expose StringAddr 2023-04-03 15:10:54 +10:00
struct_test.go Export Peer 2021-01-25 15:49:48 +11:00
t.go More Windows test fixes 2023-05-28 00:41:36 +10:00
test_test.go Add TestHaveAllThenBitfield 2021-12-23 14:00:00 +11:00
testing.go Fix logging reported in #841 2023-05-28 12:04:49 +10:00
torrent-piece-request-order.go fixes anacrolix/torrent#795 (#807) 2023-05-19 14:23:34 +10:00
torrent-stats.go Add Client.Stats with ActiveHalfOpenAttempts 2023-05-09 15:46:52 +10:00
torrent.go Log bad tracker URL error 2023-06-06 10:40:15 +10:00
torrent_mmap_test.go Revert "Remove old-style build tags" 2021-11-01 11:26:58 +11:00
torrent_test.go Add many more extension bit definitions 2023-05-09 15:46:53 +10:00
tracker_scraper.go Rename some HTTP identifiers to Http 2022-11-29 10:45:38 +11:00
undirtied-chunks-iter.go Use reusable roaring iterators 2022-05-12 10:42:16 +10:00
undirtied-chunks-iter_test.go Use reusable roaring iterators 2022-05-12 10:42:16 +10:00
url-net-addr.go Generate a RemoteAddr for webseed Peers 2021-01-25 15:49:48 +11:00
ut-holepunching.go Attempt holepunch after initial dial fails 2023-05-11 13:03:54 +10:00
ut-holepunching_test.go Skip test failures due to Go uTP implementation 2023-05-19 12:33:54 +10:00
utp.go
utp_go.go Pass client logger to anacrolix/go-libutp sockets (#722) 2022-02-15 16:18:32 +11:00
utp_libutp.go Pass client logger to anacrolix/go-libutp sockets (#722) 2022-02-15 16:18:32 +11:00
utp_test.go gorond test files 2023-05-12 13:47:47 +10:00
webrtc.go gorond ./... 2022-11-15 23:31:27 +11:00
webseed-peer.go Sleep webseed peers after unhandled errors 2022-12-31 11:27:47 +11:00
worse-conns.go Move PeerClientName and PeerExtensionIDs to PeerConn 2023-05-19 14:41:27 +10:00
worse-conns_test.go Optimize the use of worstConnSlice again 2021-11-29 10:07:48 +11:00
wstracker.go Add WebRTC ICE servers config (#824) 2023-05-27 20:27:24 +10:00

README.md

torrent

PkgGoDev

This repository implements BitTorrent-related packages and command-line utilities in Go. The emphasis is on use as a library from other projects. It's been used 24/7 in production by downstream services since late 2014. The implementation was specifically created to explore Go's concurrency capabilities, and to include the ability to stream data directly from the BitTorrent network. To this end it supports seeking, readaheads and other features exposing torrents and their files with the various Go idiomatic io package interfaces. This is also demonstrated through torrentfs.

There is support for protocol encryption, DHT, PEX, uTP, and various extensions. There are several data storage backends provided: blob, file, bolt, mmap, and sqlite, to name a few. You can write your own to store data for example on S3, or in a database.

Some noteworthy package dependencies that can be used for other purposes include:

Installation

Install the library package with go get github.com/anacrolix/torrent, or the provided cmds with go install github.com/anacrolix/torrent/cmd/...@latest.

Library examples

There are some small examples in the package documentation.

Mentions

Downstream projects

There are several web-frontends, sites, Android clients and supporting services among the known public projects:

Help

Communication about the project is primarily through Discussions and the issue tracker.

Command packages

Here I'll describe what some of the packages in ./cmd do. See installation to make them available.

torrent

torrent download

Downloads torrents from the command-line.

$ torrent download 'magnet:?xt=urn:btih:KRWPCX3SJUM4IMM4YF5RPHL6ANPYTQPU'
... lots of jibba jabber ...
downloading "ubuntu-14.04.2-desktop-amd64.iso": 1.0 GB/1.0 GB, 1989/1992 pieces completed (1 partial)
2015/04/01 02:08:20 main.go:137: downloaded ALL the torrents
$ md5sum ubuntu-14.04.2-desktop-amd64.iso
1b305d585b1918f297164add46784116  ubuntu-14.04.2-desktop-amd64.iso
$ echo such amaze
wow

torrent metainfo magnet

Creates a magnet link from a torrent file. Note the extracted trackers, display name, and info hash.

$ torrent metainfo testdata/debian-10.8.0-amd64-netinst.iso.torrent magnet
magnet:?xt=urn:btih:4090c3c2a394a49974dfbbf2ce7ad0db3cdeddd7&dn=debian-10.8.0-amd64-netinst.iso&tr=http%3A%2F%2Fbttracker.debian.org%3A6969%2Fannounce

See torrent metainfo --help for other metainfo related commands.

torrentfs

torrentfs mounts a FUSE filesystem at -mountDir. The contents are the torrents described by the torrent files and magnet links at -metainfoDir. Data for read requests is fetched only as required from the torrent network, and stored at -downloadDir.

$ mkdir mnt torrents
$ torrentfs -mountDir=mnt -metainfoDir=torrents &
$ cd torrents
$ wget http://releases.ubuntu.com/14.04.2/ubuntu-14.04.2-desktop-amd64.iso.torrent
$ cd ..
$ ls mnt
ubuntu-14.04.2-desktop-amd64.iso
$ pv mnt/ubuntu-14.04.2-desktop-amd64.iso | md5sum
996MB 0:04:40 [3.55MB/s] [========================================>] 100%
1b305d585b1918f297164add46784116  -