logos-storage/bittorrent-codex-docs
1
0
Fork 0
mirror of https://github.com/logos-storage/bittorrent-codex-docs.git synced 2026-01-02 13:03:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

Updates docs

Browse Source
This commit is contained in:
Marcin Czenko 2024-10-29 05:04:02 +01:00
parent 8a0c95a577
commit d454ed27f0
No known key found for this signature in database
GPG Key ID: 33DEA0C8E30937C0
21 changed files with 264 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.obsidian/plugins/query-control/data.json vendored
View File

@ -4,5 +4,5 @@
"defaultHideTitle": true,
"defaultHideResults": false,
"defaultRenderMarkdown": true,
"defaultSortOrder": "2"
"defaultSortOrder": "0"
}

8
10 Notes/BitTorrent Clients Usage Statistics.md
View File

@ -2,16 +2,18 @@
tags:
- bittorrent
---
#bittorrent
Below is the quick summary generated by ChatGPT. The following data source has been indicated when generating this summary:
The following data source has been used when generating this summary:
1. https://vpnalert.com/resources/torrenting-statistics/
2. https://torrentfreak.com/bittorrent-surges-to-150-million-monthly-users-120109/
3. https://en.wikipedia.org/wiki/Usage_share_of_BitTorrent_clients
As of 2024, the popularity of BitTorrent clients can be quantified through various statistics, primarily focused on user counts and traffic. Here’s a breakdown of some of the leading clients:
1. **uTorrent and BitTorrent**: These two clients, owned by the same company, collectively have around **150 million active monthly users**. uTorrent alone is particularly popular and widely used worldwide, especially in countries like Russia, India, and the United States.
2. **qBittorrent**: This open-source client is another popular choice, especially among users looking for an ad-free experience. qBittorrent receives around **6 million visits monthly**, with most users coming from countries like China, the United States, and Russia. It’s particularly popular with younger users, with over 35% of its visitors aged 25-34 .
2. [[qBittorrent]]: This open-source client is another popular choice, especially among users looking for an ad-free experience. [[qBittorrent]] receives around **6 million visits monthly**, with most users coming from countries like China, the United States, and Russia. It’s particularly popular with younger users, with over 35% of its visitors aged 25-34 .
3. [[Deluge (BitTorrent)]]: Although smaller, Deluge remains popular with an estimated **272,800 monthly visitors**. The majority of its users are from the United States, Brazil, and France. It attracts a similar age demographic as other clients, with a strong preference among those aged 25-34 .
These numbers highlight the reach and usage patterns of the most popular BitTorrent clients. While uTorrent and BitTorrent dominate in terms of sheer volume, qBittorrent has a dedicated user base that values its open-source and ad-free nature, and Deluge maintains a smaller but loyal following.
These numbers highlight the reach and usage patterns of the most popular BitTorrent clients. While uTorrent and BitTorrent dominate in terms of sheer volume, [[qBittorrent]] has a dedicated user base that values its open-source and ad-free nature, and Deluge maintains a smaller but loyal following.

9
10 Notes/BitTorrent chapter from book P2P and Grids to Services on the Web.md
View File

@ -5,5 +5,14 @@ authors:
- Ian J. Taylor and Andrew B. Harrison
link: https://link.springer.com/chapter/10.1007/978-1-84800-123-7_13
related-to:
- "[[Learn BitTorrent]]"
---
#book
| authors | Ian J. Taylor and Andrew B. Harrison |
| ---------- | -------------------------------------------------------------- |
| link | https://link.springer.com/chapter/10.1007/978-1-84800-123-7_13 |
| related-to | [[Learn BitTorrent]] |
Poor graphics, expensive (around €30,-).

2
10 Notes/Codex-BitTorrent Integration Intro.md
View File

@ -3,7 +3,7 @@ tags:
- bittorrent
---
Let's gather some background data about [[Learn BitTorrent]] clients. This is something that ChatGPT is quite good at so I compiled a bit of data you can easily check for yourself. If something looks funky, please feel free to adjust and clarify, but just for having quick check of where we are, it seems to be a reasonable overview. I also saw @Giuliano Mega immediately jumping into [[Deluge (BitTorrent)]] (also @Mark sees that as a good start), so for my personal conviction we are diving into the right thing I wanted to have some arguments on why - however short and imprecise it is not.
Let's gather some background data about [[Learn BitTorrent|BitTorrent]] clients. This is something that ChatGPT is quite good at so I compiled a bit of data you can easily check for yourself. If something looks funky, please feel free to adjust and clarify, but just for having quick check of where we are, it seems to be a reasonable overview. I also saw @Giuliano Mega immediately jumping into [[Deluge (BitTorrent)]] (also @Mark sees that as a good start), so for my personal conviction we are diving into the right thing I wanted to have some arguments on why - however short and imprecise it is not.
The most popular BitTorrent clients as of recent years include:

6
10 Notes/Comparison of BitTorrent clients.md
View File

@ -2,7 +2,9 @@
tags:
- bittorrent
---
On Wikipedia we can get a decent overview of [[Learn BitTorrent]] clients: https://en.wikipedia.org/wiki/Comparison_of_BitTorrent_clients.
#bittorrent
On Wikipedia we can get a decent overview of [[Learn BitTorrent|BitTorrent]] clients: https://en.wikipedia.org/wiki/Comparison_of_BitTorrent_clients.
> It should be noted here, that Wikipedia sources are often quite outdated.
@ -12,4 +14,4 @@ From this comparative, it does feel appropriate to further constraint our resear
- [[Transmission]]
- [[Deluge (BitTorrent)]]
All clients have similar capabilities and provide a similar set of features. Looking at look-and-feel of the website, Transmission looks the most attractive. If I would be a regular, non-dirty-hacker user, I would be attracted by [[Transmission]]. Other clients feel to me more niche, yet I would put [[qBitTorrent]] on the second position. [[Deluge (BitTorrent)]], still potentially attractive, with significantly lower user-base, being quite feature complete, still looks a bit ”hackie”. Both [[Deluge (BitTorrent)]] and [[qBittorrent]] depend on [[libtorrent]], while [[Transmission]] is using its own BitTorrent protocol implementation.
All clients have similar capabilities and provide a similar set of features. Looking at look-and-feel of the website, Transmission looks the most attractive. If I would be a regular, non-dirty-hacker user, I would be attracted by [[Transmission]]. Other clients feel to me more niche, yet I would put [[qBitTorrent]] on the second position. [[Deluge (BitTorrent)]], still potentially attractive, with significantly lower user-base, being quite feature complete, still looks a bit ”hackie”. Both [[Deluge (BitTorrent)]] and [[qBittorrent]] depend on [[libtorrent-rasterbar]], while [[Transmission]] is using its own BitTorrent protocol implementation.

2
10 Notes/Deluge (BitTorrent).md
View File

@ -2,7 +2,7 @@
tags:
- bittorrent
---
Deluge is a [[Learn BitTorrent]] client.
Deluge is a [[Learn BitTorrent|BitTorrent]] client.
Official link: https://deluge-torrent.org.
Git: https://git.deluge-torrent.org/deluge (they are not on GitHub!)

7
10 Notes/Incentives Build Robustness in BitTorrent.md
View File

@ -7,6 +7,13 @@ link: https://stuker.com/wp-content/uploads/import/i-1fd3ae7c5502dfddfe8b2c7acde
related-to:
- "[[Learn BitTorrent]]"
---
#paper
| authors | Bram Cohen |
| ---------- | -------------------------------------------------------------------------------------------------- |
| link | https://stuker.com/wp-content/uploads/import/i-1fd3ae7c5502dfddfe8b2c7acdefaa5e-bittorrentecon.pdf |
| related-to | [[Learn BitTorrent]] |
Original "BitTorrent" paper by [[Bram Cohen]].
![[i-1fd3ae7c5502dfddfe8b2c7acdefaa5e-bittorrentecon.pdf]]

36
10 Notes/Installing Qt on Ubuntu 24.04.1 LTS.md Normal file
View File

@ -0,0 +1,36 @@
---
tags:
- qt/ubuntu
---
#qt/ubuntu
The following guide was also helpful: [https://web.stanford.edu/dept/cs_edu/resources/qt/install-linux](https://web.stanford.edu/dept/cs_edu/resources/qt/install-linux).
If you previously installed:
```bash
sudo apt install --no-install-recommends qtbase5-dev qttools5-dev libqt5svg5-dev
```
Uninstall it with:
```bash
sudo apt remove qtbase5-dev qttools5-dev libqt5svg5-dev
```
### The actual steps to install Qt
1. [Download Qt](https://www.qt.io/download-qt-installer)
2. As indicated in the tutorial above, in order to install Qt your machine, you need to create account.
3. Your installer will have name like this: `qt-online-installer-linux-x64-4.8.1.run`. Make it executable (`chmod 755 qt-online-installer-linux-x64-4.8.1.run`) and run it.
4. In the installer provide your Qt credentials.
5. Select the installation option *Qt 6.8 for desktop development* below:
![[install-qt-ubuntu.png]]
6. It is possible that the installer will ask you to install some additional packages:
```bash
sudo apt install libxcb-cursor0 libxcb-cursor-dev
```
7. That's should be it. Qt should be installed and ready for use.

8
10 Notes/Learn BitTorrent.md
View File

@ -4,7 +4,15 @@ tags:
urls:
- https://www.bittorrent.org
- https://www.bittorrent.com
related-to:
- "[[Comparison of BitTorrent clients]]"
---
#bittorrent
| urls | https://www.bittorrent.org, https://www.bittorrent.com |
| ---------- | ------------------------------------------------------ |
| relates-to | [[Comparison of BitTorrent clients]] |
In order to imagine, what do we mean by *BitTorrent - Codex Integration*, we need to gain some basic understanding of BitTorrent. This is no time nor space to create a comprehensive introduction to BitTorrent here (we need to learn it as we go), so, here I just gathered some resources I use to understand the protocol, and by doing it to understand what does it mean for Codex to integrate with BitTorrent clients.
### Specs

52
10 Notes/Linux Machine.md Normal file
View File

@ -0,0 +1,52 @@
#ubuntu #linux #intel-nuc
In our tests we used Linux machine running on `IntelNUC11ATKC2` with `Ubuntu 24.04.1 LTS`, `16.0 GiB` RAM and `500 GB` SSD storage `WD Blue SN570 NVMe M.2 2280`.
```bash
lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 39 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Vendor ID: GenuineIntel
Model name: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
CPU family: 6
Model: 140
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
Stepping: 1
CPU(s) scaling MHz: 13%
CPU max MHz: 4700.0000
CPU min MHz: 400.0000
BogoMIPS: 5606.40
Flags: [...]
Virtualization features:
Virtualization: VT-x
Caches (sum of all):
L1d: 192 KiB (4 instances)
L1i: 128 KiB (4 instances)
L2: 5 MiB (4 instances)
L3: 12 MiB (1 instance)
NUMA:
NUMA node(s): 1
NUMA node0 CPU(s): 0-7
Vulnerabilities:
Gather data sampling: Mitigation; Microcode
Itlb multihit: Not affected
L1tf: Not affected
Mds: Not affected
Meltdown: Not affected
Mmio stale data: Not affected
Reg file data sampling: Not affected
Retbleed: Not affected
Spec rstack overflow: Not affected
Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; RSB filling; PBRSB-eIBRS SW sequence;
BHI SW loop, KVM SW loop
Srbds: Not affected
Tsx async abort: Not affected
```

7
10 Notes/The Bittorrent P2P File-Sharing System - Measurements And Analysis.md
View File

@ -7,6 +7,13 @@ link: http://nil.csail.mit.edu/6.824/2015/papers/pouwelse-btmeasure.pdf
related-to:
- "[[Learn BitTorrent]]"
---
#paper
| authors | J.A. Pouwelse, P . Garbacki, D.H.J. Epema, H.J. Sips |
| ---------- | ----------------------------------------------------------------- |
| link | http://nil.csail.mit.edu/6.824/2015/papers/pouwelse-btmeasure.pdf |
| related-to | [[Learn BitTorrent]] |
Paper from the creators of the [[Tribler]] protocol.
![[pouwelse-btmeasure.pdf]]

11
10 Notes/The World of Peer-to-Peer (P2P).md
View File

@ -3,11 +3,18 @@ tags:
- book
authors:
- "[[Panic]]"
related-to:
- "[[Learn BitTorrent]]"
url:
- https://en.wikibooks.org/wiki/The_World_of_Peer-to-Peer_(P2P)
related-to:
- "[[Learn BitTorrent]]"
---
#book
| authors | [[Panic]] |
| ---------- | ------------------------------------------------------------- |
| link | https://en.wikibooks.org/wiki/The_World_of_Peer-to-Peer_(P2P) |
| related-to | [[Learn BitTorrent]] |
From this book we are particularly interested in the chapter [2.15 BitTorrent](https://en.wikibooks.org/wiki/The_World_of_Peer-to-Peer_(P2P)/Networks_and_Protocols/BitTorrent).
I have included the complete PDF of that book here:

2
10 Notes/Transmission.md
View File

@ -5,7 +5,7 @@ tags:
Offcial URL: https://transmissionbt.com
Repository: https://github.com/transmission/transmission
[[Learn BitTorrent]] client. One of the three popular clients, the other are [[Deluge (BitTorrent)]] and [[qBittorrent]], with the most attractive looking website. Comparing to the other two, Transmission does not depend on [[libtorrent]], but rather has its own implementation of the BitTorrent protocol.
[[Learn BitTorrent]] client. One of the three popular clients, the other are [[Deluge (BitTorrent)]] and [[qBittorrent]], with the most attractive looking website. Comparing to the other two, Transmission does not depend on [[libtorrent-rasterbar]], but rather has its own implementation of the BitTorrent protocol.
> As noted in GitHub’s REDME, [Transmission's documentation](https://github.com/transmission/transmission/blob/main/docs/README.md) is currently out-of-date, but the team has recently begun a new project to update it and is looking for volunteers. If you're interested, please feel free to submit pull requests!

2
10 Notes/Tribler.md
View File

@ -1,6 +1,6 @@
---
link: https://www.tribler.org/index.html
---
This is a protocol based on [[Learn BitTorrent]]. Created at the [[University of Delft]] in the frame of the [[I-Share]] project I was part of as a Ph.D. student (2004-2009).
This is a protocol based on [[Learn BitTorrent|BitTorrent]]. Created at the [[University of Delft]] in the frame of the [[I-Share]] project I was part of as a Ph.D. student (2004-2009).
Read more: https://www.tribler.org/about.html.

36
10 Notes/libtorrent-rasterbar.md Normal file
View File

@ -0,0 +1,36 @@
---
tags:
- bittorrent
link: http://www.libtorrent.org/
source: https://github.com/arvidn/libtorrent
related-to:
- "[[Deluge (BitTorrent)]]"
- "[[qBittorrent]]"
---
#bittorrent
| link | http://www.libtorrent.org/ |
| ---------- | ---------------------------------------- |
| source | https://github.com/arvidn/libtorrent |
| related-to | [[Deluge (BitTorrent)]], [[qBittorrent]] |
Open Source BitTorrent library.
Languages:
![[Pasted image 20241023160207.png]]
### Building
I have followed the instructions from [https://www.libtorrent.org/building.html](https://www.libtorrent.org/building.html).
I've followed sections *building from git*, *building with boost build*, and then immediately *Step 4: Installing libtorrent*. Documentation not really clear - hard to see the logical order.
Building steps:
```bash
git clone --recurse-submodules https://github.com/arvidn/libtorrent.git
sudo apt install libboost-tools-dev libboost-dev libboost-system-dev
echo "using gcc ;" >>~/user-config.jam
b2 crypto=openssl cxxstd=14 release
sudo b2 install --prefix=/usr/local
```

14
10 Notes/libtorrent.md
View File

@ -1,14 +0,0 @@
---
tags:
- bittorrent
---
Open Source BitTorrent library.
Link: http://www.libtorrent.org/.
Repository: https://github.com/arvidn/libtorrent
Used by [[Deluge (BitTorrent)]] and [[qBittorrent]].
Languages:
![[Pasted image 20241023160207.png]]

81
10 Notes/qBittorrent.md
View File

@ -1,18 +1,91 @@
---
tags:
- bittorrent
link: https://www.qbittorrent.org
source: https://github.com/qbittorrent/qBittorrent/
related-to:
- "[[libtorrent-rasterbar]]"
---
[[Learn BitTorrent]] client.
#bittorrent
| link | https://www.qbittorrent.org |
| ---------- | ------------------------------------------- |
| source | https://github.com/qbittorrent/qBittorrent/ |
| related-to | [[libtorrent-rasterbar]] |
A [[Learn BitTorrent|BitTorrent]] client that is said to be developed by [volunteers](https://www.qbittorrent.org/team) in their spare time.
From Wikipedia:
> **qBittorrent** is a [cross-platform](https://en.wikipedia.org/wiki/Cross-platform "Cross-platform") [free and open-source](https://en.wikipedia.org/wiki/Free_and_open-source "Free and open-source") [BitTorrent client](https://en.wikipedia.org/wiki/BitTorrent_client "BitTorrent client")written in [native](https://en.wikipedia.org/wiki/Native_application "Native application") [C++](https://en.wikipedia.org/wiki/C%2B%2B "C++"). It relies on [Boost](https://en.wikipedia.org/wiki/Boost_(C%2B%2B_libraries) "Boost (C++ libraries)"), [OpenSSL](https://en.wikipedia.org/wiki/OpenSSL "OpenSSL"), [zlib](https://en.wikipedia.org/wiki/Zlib "Zlib"), [Qt](https://en.wikipedia.org/wiki/Qt_(software) "Qt (software)") 6 toolkit and the [libtorrent](https://en.wikipedia.org/wiki/Libtorrent "Libtorrent")-rasterbar library (for the torrent back-end), with an optional search engine written in [Python](https://en.wikipedia.org/wiki/Python_(programming_language) "Python (programming language)").[8](https://en.wikipedia.org/wiki/QBittorrent#cite_note-8)[9](https://en.wikipedia.org/wiki/QBittorrent#cite_note-9).
> **qBittorrent** is a [cross-platform](https://en.wikipedia.org/wiki/Cross-platform "Cross-platform") [free and open-source](https://en.wikipedia.org/wiki/Free_and_open-source "Free and open-source") [BitTorrent client](https://en.wikipedia.org/wiki/BitTorrent_client "BitTorrent client")written in [native](https://en.wikipedia.org/wiki/Native_application "Native application") [C++](https://en.wikipedia.org/wiki/C%2B%2B "C++"). It relies on [Boost](https://en.wikipedia.org/wiki/Boost_(C%2B%2B_libraries) "Boost (C++ libraries)"), [OpenSSL](https://en.wikipedia.org/wiki/OpenSSL "OpenSSL"), [zlib](https://en.wikipedia.org/wiki/Zlib "Zlib"), [Qt](https://en.wikipedia.org/wiki/Qt_(software) "Qt (software)") 6 toolkit and the [[libtorrent-rasterbar]] library (for the torrent back-end), with an optional search engine written in [Python](https://en.wikipedia.org/wiki/Python_(programming_language) "Python (programming language)").[8](https://en.wikipedia.org/wiki/QBittorrent#cite_note-8)[9](https://en.wikipedia.org/wiki/QBittorrent#cite_note-9).
Note about macOS support:
> The macOS version is **barely supported,** because we don't have active macOS developers/contributors.
> The project is in need of macOS developers. If you are a macOS developer willing to help, just go to our bug tracker for a list of macOS related issues. Or try to fix bugs that you yourself have discovered and annoy you.
Languages:
I am not sure if that information is completely up-to-date. I did not test building the client on macOS, yet, there seem to be instructions available: [Compilation macOS (x86_64, arm64, cross compilation)](https://github.com/qbittorrent/qBittorrent/wiki/Compilation-macOS-(x86_64,-arm64,-cross-compilation)).
![[Pasted image 20241023155058.png]]
Languages:
![[Pasted image 20241023155058.png]]
### How does it look like?
![[qBittorrent-ubuntu.png]]
### Building
Machine: [[Linux Machine]].
Building instructions: [Compilation Debian, Ubuntu, and derivatives](https://github.com/qbittorrent/qBittorrent/wiki/Compilation-Debian,-Ubuntu,-and-derivatives).
When trying to build I was having problems concerning Qt libraries. Following the tutorial, I have installed recommended Qt packages:
```bash
sudo apt install --no-install-recommends qtbase5-dev qttools5-dev libqt5svg5-dev
```
Unfortunately, this did not work. When subsequently running:
```bash
cmake -G "Ninja" -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_INSTALL_PREFIX=/usr/local
```
I was getting the following error:
```bash
CMake Error at cmake/Modules/CheckPackages.cmake:49 (find_package):
By not providing "FindQt6.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "Qt6", but
CMake did not find one.
Could not find a package configuration file provided by "Qt6" (requested
version 6.5.0) with any of the following names:
Qt6Config.cmake
qt6-config.cmake
Add the installation prefix of "Qt6" to CMAKE_PREFIX_PATH or set "Qt6_DIR"
to a directory containing one of the above files. If "Qt6" provides a
separate development package or SDK, be sure it has been installed.
Call Stack (most recent call first):
CMakeLists.txt:56 (include)
```
To fix this, first needed to [[Installing Qt on Ubuntu 24.04.1 LTS|instal Qt]], then I have modified `./CMakeLists.txt` by adding the following two lines after line `15`:
```cmake
# version requirements - older versions may work, but you are on your own
set(minBoostVersion 1.76)
set(minQt6Version 6.5.0)
set(minOpenSSLVersion 3.0.2)
set(minLibtorrent1Version 1.2.19)
set(minLibtorrentVersion 2.0.10)
set(minZlibVersion 1.2.11)
# added the following two lines:
set(Qt6_DIR "~/Qt/6.8.0/gcc_64/lib/cmake/Qt6/")
set(Qt6GuiTools_DIR "~/Qt/6.8.0/gcc_64/lib/cmake/Qt6GuiTools/")
```
After that I was able to successfully compile the client.

5
90 Extras/91 Templates/Authors Table.md Normal file
View File

@ -0,0 +1,5 @@
| authors | |
| ---------- | --- |
| link | |
| related-to | |

BIN
90 Extras/92 Assets/install-qt-ubuntu.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 240 KiB

BIN
90 Extras/92 Assets/qBittorrent-ubuntu.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 259 KiB

6
README.md
View File

@ -2,6 +2,7 @@
tags:
- bittorrent
---
#bittorrent
## BitTorrent/Codex Integration
In this repository your will find a complete documentation of our BitTorrent/Codex investigation.
@ -30,5 +31,6 @@ The vault includes the `.obsidian` folder, but does not include publishing setti
### Published site
For the duration of the project, I am also publishing the documentation at [https://publish.obsidian.md/bittorrent](https://publish.obsidian.md/bittorrent).
This will be removed after conclusion has been reached.
For the duration of the project, I am also publishing the documentation at [https://publish.obsidian.md/bittorrent](https://publish.obsidian.md/bittorrent). Published site allows you to access all the docs, but it is highly recommended to use [Obsidian App](https://obsidian.md/download) for a complete experience. Currently, for instance, we need to duplicate note properties in a table, in order for those to be visible at the published site.
The site will be removed after conclusion has been reached.