Commit Graph

8812 Commits

Author SHA1 Message Date
iczero 89189adb24 [Core] Stop using removed disk.num_blocks_cache_hits stat
Removed in libtorrent v2.0.0:
569d47c391
2021-10-03 14:19:17 +01:00
Calum Lind a5a7da4a1a [Core] Use external_address in external_ip alert handler
Unit tests are segfaulting and this is a result of a subsequent call to
pop_alert creating a dangling pointer to alert.message method.

Replace alert.message call with correct external_address property that
doesn't require any parsing and wonder trigger segfault to expired
pointer.

https://github.com/arvidn/libtorrent/issues/6437
2021-10-03 14:16:57 +01:00
Calum Lind 1e6cc03946 [Lint] Fix spelling mistakes
A quick fix of some of the mistakes caught by codespell.
Updated readme with new IRC server

Useful to add it as part of linting checks.
2021-09-21 21:43:53 +01:00
Calum Lind d8526ba653 [UI] Add magnet icons for copy and add actions
Added new magnet icons with a consistent naming scheme
Run script to update all icons

Co-authored-by: Matias Wilkman <matias.wilkman@gmail.com>
2021-09-21 20:38:08 +01:00
Gargaj c38f913948 [WebUI] Add menu option to copy magnet URI 2021-09-21 20:35:24 +01:00
Calum Lind 0659fe4641 [Core] Export torrent get_magnet_uri method
Returns a generated magnet uri from torrent info
2021-09-21 20:27:38 +01:00
Calum Lind 10501db63d [i18n] Update translation PO files from Launchpad 2021-09-14 22:02:56 +01:00
Calum Lind 2a312159b9 [GTKUI] Fix unhandled error with empty clipboard
If the primary clipboard was empty the fallback resulted in an unhandled
error due to missing arguments.

Fixed by using SELECTION_PRIMARY as fallback clipboard
2021-09-14 21:56:03 +01:00
Calum Lind cb75192df4 [CI] Add core dump capture to GH job
Add further debugging to trace segfaults with lt 1.2
2021-09-10 19:06:21 +01:00
Calum Lind 588f600ba2 [#3310|Core] Change logging invalid session status key to debug
Users were complaining about logs being flooded with `Session status key not valid`
which was a result of the Stats plugin using the wrong status keys.

Fixed by changing to debug log level since not useful in warning level
if spamming.
2021-08-29 16:07:53 +01:00
iczero ea609cd3e0 [#3310|Stats] Fix constant session status key warnings
Fixed logs flooded with:

    [WARNING ][deluge.core.core              :655 ] Session status key not valid: num_connections
    [WARNING ][deluge.core.core              :655 ] Session status key not valid: dht_cache_nodes
2021-08-29 16:05:36 +01:00
Calum Lind 4b6c7d01b2 [#3478|Core] Fix loading magnet with resume_data and no metadata
Since libtorrent 1.2.10 magnets save resume_data even with metadata not
yet downloaded. Unfortunately when using the deprecated
add_torrent_params key resume_data results in an error  "missing
info-hash from URI"

The problem is due to lt session requiring an info_hash in
add_torrent_params but resume_data does not set or override this key and
resume_data overrides the add_torrent_params.url with an empty string.

The workaround is to specify the info_hash in add_torrent_params. We
require sha1_hash object or bytes and use of bytearray to maintain
python2 compatability.

https://dev.deluge-torrent.org/ticket/3478
2021-08-29 15:58:48 +01:00
Calum Lind b89b2c45b1 [Console] Fix using windows-curses on Windows
The console tests are still failing on Windows due to an issue where the
sys args are not being correctly replaced in the tests so the pytest
args are being passed to console.
2021-08-01 08:48:27 +01:00
Calum Lind e38f1173cf [Notifications] Fix email KeyError with status name
Fix user reported error:

    Notification failure using email:
    [Failure instance: Traceback: <class 'KeyError'>: 'name'

This was due to using empty dict used with get_status where a list of
keys is now required or the all_keys parameter is used.

Fixes: #3303
2021-08-01 08:43:37 +01:00
Calum Lind e1e0999de6 [Tests] Fix skipping torrent test for libtorrent >= 1.2
Test is still failing with libtorrent 2 so also skip.
2021-08-01 08:35:17 +01:00
Calum Lind 5c9378ac5e [Tests] Fix incorrent twisted defer import
With the release Twisted 21.7.0 there was an import error running the
tests due to defer incorrectly imported via twisted.internet.tasks module.
2021-07-31 22:08:23 +01:00
Calum Lind f075f391cb [CI] Add catchsegv to get a backtrace for segfaults
Encountering random libtorrent segfault with GitHub action so add
catchsegv when running tests to get more information.
2021-07-31 22:08:23 +01:00
Calum Lind 8fb25f71f3 [Install] Update and fix python optional requirements
* Added required dependency setuptools to install_requires
* Remove optional dependency ipaddress from install_requires
* Created extras_require in setup.py. The optional dependencies should
not be included in install_requires so that users can either install
forked dependencies or remove problematic ones. Updated documentation to
detail how to install these optional dependencies.
* Fixed README badge

Refs:
 * https://dev.deluge-torrent.org/ticket/3470
 * https://dev.deluge-torrent.org/ticket/3282
 * https://dev.deluge-torrent.org/ticket/3353
2021-07-31 22:08:23 +01:00
Calum Lind a3332079db [GTKUI] Remove deprecated GTK attributes 2021-07-25 17:42:05 +01:00
Calum Lind 0d6eec7a33 [i18n] Refactor loading libintl library
Handle different names for libintl library on MacOS and Windows with
fallback.
2021-07-25 16:47:14 +01:00
Calum Lind f16afc59ba Ignore TypeError with custom Twisted logging
The modification of Python logging _findCaller args in Python 3.8 raises
TypeError in our custom Twisted Logger with Twisted <= 19 versions.

The actual issue for the custom logger was fixed in 18.9 so added a
version check to avoid usage.

Refs:
 - https://twistedmatrix.com/trac/ticket/7927
 - 6b894744e4
2021-07-25 13:28:18 +01:00
Calum Lind e5388048a9 [CI/CD] Add github actions to replace Travis
Due to new limitations for open-source projects on Travis we are
switching to GitHub actions.

* Notes about system site-packages

We had many problems with accessing system python packages on Travis for
libtorrent and GTK and the problems are harder on Github since there is
no more access. For now copying the python libtorrent binary into the
deluge source is the workaround. There is a pip package that could be
used in future.

Fixed failing tests with libtorrent 1.2 which required a non-zero length
file in torrent and workarounds for async alert delay.
2021-07-25 13:27:26 +01:00
Calum Lind 5374d237a7 [AutoAdd|3295] Correctly fix auto-adding magnets
Properly fix adding magnets, first attempted in previous commit 2e466101fc

add_torrent_magnet does not return a deferred so wrap in maybeDeferred.

Fixed broken test due to new deluge website icon
2021-07-24 11:25:25 +01:00
RFBomb 2e466101fc [AutoAdd] Fix magnet missing applied labels
The autoadd function does not apply labels to torrents that are added via magnet files.
Those magnet files are also renamed ".Magnet.Invalid".

Here are two threads discussing the issue, which still exists.

    https://forum.deluge-torrent.org/viewtopic.php?t=55539
    https://dev.deluge-torrent.org/ticket/3295

Here is what Deluged.log shows when the problem occurs:

21:51:38 [ERROR   ][deluge_autoadd.core           :333 ] Cannot Autoadd magnet: /Torrents/TorrentFiles/FileName.magnet: Torrent already in session (e1e0f33b656cb74532dcddc04f2ec52771ef1c26).
21:56:38 [ERROR   ][deluge_autoadd.core           :333 ] Cannot Autoadd magnet: /Torrents/TorrentFiles/FileName2.magnet: Torrent already in session (ef839d84d113cc35719b6fd616a4d8e220de7d32).

After looking at the code, what appears to be happening is the magnet link is added, but then a second scan of the folder occurs. Since the magnet file was never renamed, it will attempt to add it again, error out, then rename the file "magnet.invalid".

The only difference between the torrents working properly and magnets having the issue is the two lines I copy-pasted into the magnet IF statement. This should resolve the issue.
2021-04-17 17:24:58 +01:00
DjLegolas 8676a0d2a0 [Core] Improve on_alert_tracker_error for lt >= 1.2
libtorrent 1.2 added endpoint struct to each tracker, to prevent false
updates we will need to verify that at least one endpoint to the errored
tracker is working.  if there is at least one working, it will not set
the tracker status to
error and set it to `Announce OK`. otherwise, it will use the error
message from the alert.

Refs: https://dev.deluge-torrent.org/ticket/3384
2021-04-17 16:55:54 +01:00
Calum Lind 3ec23ad96b [#3388|WebUI] Fix md5sums in torrent files breaking file listing
Torrents containing md5sum optional hashes are not being decoded and so
causes errors in the json_api when the TorrentInfo is returned:

    Object of type bytes is not JSON serializable

Fixed by removing all optional hashes from the paths returned from
TorrentInfo and only including the required path keys. The optional
hashes are unused by Deluge so simplify by removing.

Fixed Windows path issue in TorrentInfo by ensuring conversion to posix paths.

Refs:

http://wiki.bitcomet.com/inside_bitcomet
http://wiki.depthstrike.com/index.php/P2P:Protocol:Specifications:Optional_Hashes
https://wiki.theory.org/index.php/BitTorrentSpecification
2021-03-24 10:26:08 +00:00
Hans Ole Hatzel dcd3918f36 [WebUI] Add test for torrent files containing md5sums
Some torrent files built with py3createtorrent fail to produce a
file listing in the WebUI when uploading them.
This made it impossible to add such files.
Specifically this is caused by the additional metadata when using
py3createtorrent with the `--md5` flag.
2021-03-24 10:26:08 +00:00
Calum Lind 08c7f1960f [CI/CD] Fix Tox SSL error in Windows Travis job
The tox sdist-make step failed with SSL: CERTIFICATE_VERIFY_FAILED so
fix by install certifi to ensure updated SSL certificates are available.
2021-03-24 10:25:30 +00:00
Calum Lind 8a4ec493c0 [CI/CD] Add Travis windows build
* Added APPDATA to tox passenv so it is available to common module.
* Fixed windows path issue in httpdownloader tests
* Skipped torrentmanager test due to the following error from loading a
Linux pickled state file with a different line ending.
    ModuleNotFoundError: No module named 'deluge.core.torrentmanager\r'
* Removed appveyor build
2021-02-23 10:41:46 +00:00
Calum Lind 4d970754a4 [#3440] Fix httpdownloader reencoding torrent file downloads
Torrent downloads from rutracker responds with the header:

    Content-Type: application/x-bittorrent; charset=Windows-1251

The problem is that httpdownloader was using the charset to re-encode
the downloaded file, corrupting the binary torrent file download.

Fixed by only re-encoding text content types, since it is very rare
that non-text content types would actually have a non-utf8 codeset and
if there is a requirement we would need to determine it on a type by
type basis.
2021-02-20 21:18:32 +00:00
Calum Lind f331b6c754 [GTKUI] Fix torrentdetails tab bar position not saving
The GTKUI tests were failing and the saved config for the tab bar
position was not being restored.

Fixed by moving the setting of notebook.tabs_pos to TorrentDetail init.

Replaced more deprecated methods that were showing up in tests.
2021-02-20 17:29:36 +00:00
Calum Lind 1022448e4f [#3441|GTKUI] Add a torrentdetails tabs position menu
The tabs placement for the torrentdetails notebook might not be to
everyone's liking so add a menu item to configure it.

Default the position back to top.
2021-02-20 15:22:08 +00:00
Calum Lind 291540b601 Hide pygame community banner in console
Notifications plugin uses pygame for sound notifications however pygame
show a console message "Hello from the pygame community." whenever
starting deluge from console.

Refs: https://stackoverflow.com/a/55769463/175584
2021-02-20 14:11:14 +00:00
Calum Lind 092d07b68e [#3337|Core] Fix lt listen_interfaces not comma-separated
A typo meant that the interfaces supplied to libtorrent were not
comma-separated.

Refs: https://github.com/arvidn/libtorrent/blob/RC_1_1/include/libtorrent/string_util.hpp#L70
2021-02-06 17:26:30 +00:00
Calum Lind da5d5bee20 [#3325|Core] Fix unable to remove magnet with delete_copies enabled
Users were encountering the following error while attempting to delete
magnet torrents and had the config 'Delete copy of torrent file'
enabled. This was due to removing a magnet before the metadata was
downloaded and the torrent.filename was still set to None so raises
exceptions when string operations are performed with it.

  File "/usr/lib/python3/dist-packages/deluge/core/torrent.py", line 1317, in delete_torrentfile
    os.path.join(self.config['torrentfiles_location'], self.filename)
  ...
  TypeError: join() argument must be str or bytes, not 'NoneType'

Fixed by both setting a default empty string for self.filename and only
deleting the torrent file copy if filename is set.
2021-02-05 20:33:19 +00:00
Vasilij Schneidermann 6d9dc9bd42 [WebUI] Add country flag alt/title for accessibility
This allows viewing the country in textual form by hovering the flag
image and displays it if the image couldn't be loaded.
2021-01-29 18:49:24 +00:00
bendikro 937afd921c [Tests] Fix console tests sometimes failing due to hard coded port
The tests in ConsoleUIWithDaemonBaseTestCase could fail to the hard coded
port 58900 being busy.
Fix by using the proper port found in self.listen_port

Also convert unnecessary use of assertTrue where more appropriate
assert functions should be used, such as assertEqual and assertIn
2021-01-29 18:46:42 +00:00
EFS a4da8d29f8 [WebUI] Fix tracker icon download error
Encoutering an error when webui attempts to download tracker icon:

    Error occurred downloading file from "http://b'acg.rip'/": invalid
    hostname: b'acg.rip'

Fixed by ensuring the request.tracker_name is decoded from bytes before
looking up the icon name.
2021-01-29 18:31:52 +00:00
neeshy 8ec5ca9d08 [Console] Fix setting 'Skip' priority on console
Selecting priorities 'Low' and 'Skip' on console will both set the
actual priority to 'Low'.

Fixed typo in previous commit 6655fe67c3
2021-01-29 18:26:35 +00:00
scudre 9c90136f57 [#3439] Execute plugin fails to run on Windows
Fixed TypeError: a bytes-like object is required, not 'str'
2021-01-29 18:17:35 +00:00
Calum Lind 610a1bb313 [Lint] Update pre-commit hook and isort versions
* Fixed black hook requiring Py3.6 to installed locally. Will now assume
Py3.6+ in installed.
 * Added isort traceback in pre-commit flake8 hook fails
 * Updated versions of Black, Prettier and isort
 * Keep Flake8 at 3.7.9 due to E402 issue: https://gitlab.com/pycqa/flake8/-/issues/638
 * New pyproject config for isort v5 with fixes for Python 2 imports.
 * Fixed travis config to run Python 3.6 for lint run. Replaced the
virtualenv with_system_site_packages config with Travis specific Python
config value so lint run doesn't attempt to append with_system_site_packages
to Python 3.6 command.
2021-01-24 20:40:20 +00:00
Calum Lind 23a48dd01c [#3309|GTK] Fix cmp function for None types
Comparisons on Python 3 are much stricter resulting in the following
error comparing with None:

    TypeError: '>' not supported between instances of 'NoneType' and 'str'

Fix this by getting the type of the other value and getting it's default
value.
2020-04-30 14:30:37 +01:00
Calum Lind d02fa72e80 [Console] Fix hostlist status lookup errors
If a host in hostlist failed DNS lookup or other issue it was returning
a tuple instead of deferred. Fix this in hostlist by returning a
defer.succeed.

A race condition with BaseMode was also encountered when
update_hosts_status calls update_select_host_popup and
ConnectionManager does not have a rows attribute. Fix this by init
BaseMode before update_hosts_status and remove already called
update_select_host_popup.
2020-04-27 16:24:33 +01:00
Calum Lind 62d8749e74 [#3348] Fix TypeError adding peers to torrents
Python3 has stricter type checking and passing a port as string results
in libtorrent raising a TypeError.

Fixed by casting port to int, along with refactoring to ensure ipv6 is
correctly parsing and a useful error is output to user with invalid ip
or port details.

https://dev.deluge-torrent.org/ticket/3348
2020-04-25 13:16:04 +01:00
Nitzan Raz 76f0bf2e04 Ctl+Q to quit Deluge GTK without killing daemon 2020-04-25 13:08:54 +01:00
Calum Lind 635f6d970d [Config] Fix loading config with double-quotes in string
If a password or other string contained a double-quote then the config
would fail to be loaded on startup and reset.

This occurred due to fixing a similar issue with curly braces for #3079
in commit 33e9545cd4 and the checking for double-quotes had unforseen
consequences.

To resolve both these issues the code to check for json objects in
config files was simplified and utilises the json module raw_decode
method to ensure the extracted string indexes are json objects.
2020-04-25 10:49:08 +01:00
bendikro 672e3c42a8 [Tests] Add pytest markers to tox.ini
Remove pytest warnings due to unknown markers
2020-04-23 17:19:37 +01:00
bendikro c1110e4ef3 [Tests] Fix tests failing when deluged fails to listen
Commit b32c5d824 changed the logged message in deluge/core/daemon_entry.py
when libtorrent fails to listen on the given port, without updating the
trigger expression in deluge/tests/common.py:start_core to match the new output.

Fix by updating the trigger match expressions to match the new log output
2020-04-23 17:19:37 +01:00
bendikro 742c8a941a [Tests] Fix PytestDeprecationWarning from pytest
Accessing pytest.config is deprecated and produces:
PytestDeprecationWarning: the pytest.config global is deprecated. Please use
request.config or pytest_configure (if you're a pytest plugin) instead.

Fix by using a pytest.fixture
2020-04-23 17:19:37 +01:00
bendikro 3427ae4b90 [GTK] Remove PyGIWarning in gtk3/files_tab.py
Remove warning: PyGIWarning: Gtk was imported without specifying a version first
2020-04-23 17:17:20 +01:00