Commit Graph

8800 Commits

Author SHA1 Message Date
Calum Lind b89b2c45b1 [Console] Fix using windows-curses on Windows
The console tests are still failing on Windows due to an issue where the
sys args are not being correctly replaced in the tests so the pytest
args are being passed to console.
2021-08-01 08:48:27 +01:00
Calum Lind e38f1173cf [Notifications] Fix email KeyError with status name
Fix user reported error:

    Notification failure using email:
    [Failure instance: Traceback: <class 'KeyError'>: 'name'

This was due to using empty dict used with get_status where a list of
keys is now required or the all_keys parameter is used.

Fixes: #3303
2021-08-01 08:43:37 +01:00
Calum Lind e1e0999de6 [Tests] Fix skipping torrent test for libtorrent >= 1.2
Test is still failing with libtorrent 2 so also skip.
2021-08-01 08:35:17 +01:00
Calum Lind 5c9378ac5e [Tests] Fix incorrent twisted defer import
With the release Twisted 21.7.0 there was an import error running the
tests due to defer incorrectly imported via twisted.internet.tasks module.
2021-07-31 22:08:23 +01:00
Calum Lind f075f391cb [CI] Add catchsegv to get a backtrace for segfaults
Encountering random libtorrent segfault with GitHub action so add
catchsegv when running tests to get more information.
2021-07-31 22:08:23 +01:00
Calum Lind 8fb25f71f3 [Install] Update and fix python optional requirements
* Added required dependency setuptools to install_requires
* Remove optional dependency ipaddress from install_requires
* Created extras_require in setup.py. The optional dependencies should
not be included in install_requires so that users can either install
forked dependencies or remove problematic ones. Updated documentation to
detail how to install these optional dependencies.
* Fixed README badge

Refs:
 * https://dev.deluge-torrent.org/ticket/3470
 * https://dev.deluge-torrent.org/ticket/3282
 * https://dev.deluge-torrent.org/ticket/3353
2021-07-31 22:08:23 +01:00
Calum Lind a3332079db [GTKUI] Remove deprecated GTK attributes 2021-07-25 17:42:05 +01:00
Calum Lind 0d6eec7a33 [i18n] Refactor loading libintl library
Handle different names for libintl library on MacOS and Windows with
fallback.
2021-07-25 16:47:14 +01:00
Calum Lind f16afc59ba Ignore TypeError with custom Twisted logging
The modification of Python logging _findCaller args in Python 3.8 raises
TypeError in our custom Twisted Logger with Twisted <= 19 versions.

The actual issue for the custom logger was fixed in 18.9 so added a
version check to avoid usage.

Refs:
 - https://twistedmatrix.com/trac/ticket/7927
 - 6b894744e4
2021-07-25 13:28:18 +01:00
Calum Lind e5388048a9 [CI/CD] Add github actions to replace Travis
Due to new limitations for open-source projects on Travis we are
switching to GitHub actions.

* Notes about system site-packages

We had many problems with accessing system python packages on Travis for
libtorrent and GTK and the problems are harder on Github since there is
no more access. For now copying the python libtorrent binary into the
deluge source is the workaround. There is a pip package that could be
used in future.

Fixed failing tests with libtorrent 1.2 which required a non-zero length
file in torrent and workarounds for async alert delay.
2021-07-25 13:27:26 +01:00
Calum Lind 5374d237a7 [AutoAdd|3295] Correctly fix auto-adding magnets
Properly fix adding magnets, first attempted in previous commit 2e466101fc

add_torrent_magnet does not return a deferred so wrap in maybeDeferred.

Fixed broken test due to new deluge website icon
2021-07-24 11:25:25 +01:00
RFBomb 2e466101fc [AutoAdd] Fix magnet missing applied labels
The autoadd function does not apply labels to torrents that are added via magnet files.
Those magnet files are also renamed ".Magnet.Invalid".

Here are two threads discussing the issue, which still exists.

    https://forum.deluge-torrent.org/viewtopic.php?t=55539
    https://dev.deluge-torrent.org/ticket/3295

Here is what Deluged.log shows when the problem occurs:

21:51:38 [ERROR   ][deluge_autoadd.core           :333 ] Cannot Autoadd magnet: /Torrents/TorrentFiles/FileName.magnet: Torrent already in session (e1e0f33b656cb74532dcddc04f2ec52771ef1c26).
21:56:38 [ERROR   ][deluge_autoadd.core           :333 ] Cannot Autoadd magnet: /Torrents/TorrentFiles/FileName2.magnet: Torrent already in session (ef839d84d113cc35719b6fd616a4d8e220de7d32).

After looking at the code, what appears to be happening is the magnet link is added, but then a second scan of the folder occurs. Since the magnet file was never renamed, it will attempt to add it again, error out, then rename the file "magnet.invalid".

The only difference between the torrents working properly and magnets having the issue is the two lines I copy-pasted into the magnet IF statement. This should resolve the issue.
2021-04-17 17:24:58 +01:00
DjLegolas 8676a0d2a0 [Core] Improve on_alert_tracker_error for lt >= 1.2
libtorrent 1.2 added endpoint struct to each tracker, to prevent false
updates we will need to verify that at least one endpoint to the errored
tracker is working.  if there is at least one working, it will not set
the tracker status to
error and set it to `Announce OK`. otherwise, it will use the error
message from the alert.

Refs: https://dev.deluge-torrent.org/ticket/3384
2021-04-17 16:55:54 +01:00
Calum Lind 3ec23ad96b [#3388|WebUI] Fix md5sums in torrent files breaking file listing
Torrents containing md5sum optional hashes are not being decoded and so
causes errors in the json_api when the TorrentInfo is returned:

    Object of type bytes is not JSON serializable

Fixed by removing all optional hashes from the paths returned from
TorrentInfo and only including the required path keys. The optional
hashes are unused by Deluge so simplify by removing.

Fixed Windows path issue in TorrentInfo by ensuring conversion to posix paths.

Refs:

http://wiki.bitcomet.com/inside_bitcomet
http://wiki.depthstrike.com/index.php/P2P:Protocol:Specifications:Optional_Hashes
https://wiki.theory.org/index.php/BitTorrentSpecification
2021-03-24 10:26:08 +00:00
Hans Ole Hatzel dcd3918f36 [WebUI] Add test for torrent files containing md5sums
Some torrent files built with py3createtorrent fail to produce a
file listing in the WebUI when uploading them.
This made it impossible to add such files.
Specifically this is caused by the additional metadata when using
py3createtorrent with the `--md5` flag.
2021-03-24 10:26:08 +00:00
Calum Lind 08c7f1960f [CI/CD] Fix Tox SSL error in Windows Travis job
The tox sdist-make step failed with SSL: CERTIFICATE_VERIFY_FAILED so
fix by install certifi to ensure updated SSL certificates are available.
2021-03-24 10:25:30 +00:00
Calum Lind 8a4ec493c0 [CI/CD] Add Travis windows build
* Added APPDATA to tox passenv so it is available to common module.
* Fixed windows path issue in httpdownloader tests
* Skipped torrentmanager test due to the following error from loading a
Linux pickled state file with a different line ending.
    ModuleNotFoundError: No module named 'deluge.core.torrentmanager\r'
* Removed appveyor build
2021-02-23 10:41:46 +00:00
Calum Lind 4d970754a4 [#3440] Fix httpdownloader reencoding torrent file downloads
Torrent downloads from rutracker responds with the header:

    Content-Type: application/x-bittorrent; charset=Windows-1251

The problem is that httpdownloader was using the charset to re-encode
the downloaded file, corrupting the binary torrent file download.

Fixed by only re-encoding text content types, since it is very rare
that non-text content types would actually have a non-utf8 codeset and
if there is a requirement we would need to determine it on a type by
type basis.
2021-02-20 21:18:32 +00:00
Calum Lind f331b6c754 [GTKUI] Fix torrentdetails tab bar position not saving
The GTKUI tests were failing and the saved config for the tab bar
position was not being restored.

Fixed by moving the setting of notebook.tabs_pos to TorrentDetail init.

Replaced more deprecated methods that were showing up in tests.
2021-02-20 17:29:36 +00:00
Calum Lind 1022448e4f [#3441|GTKUI] Add a torrentdetails tabs position menu
The tabs placement for the torrentdetails notebook might not be to
everyone's liking so add a menu item to configure it.

Default the position back to top.
2021-02-20 15:22:08 +00:00
Calum Lind 291540b601 Hide pygame community banner in console
Notifications plugin uses pygame for sound notifications however pygame
show a console message "Hello from the pygame community." whenever
starting deluge from console.

Refs: https://stackoverflow.com/a/55769463/175584
2021-02-20 14:11:14 +00:00
Calum Lind 092d07b68e [#3337|Core] Fix lt listen_interfaces not comma-separated
A typo meant that the interfaces supplied to libtorrent were not
comma-separated.

Refs: https://github.com/arvidn/libtorrent/blob/RC_1_1/include/libtorrent/string_util.hpp#L70
2021-02-06 17:26:30 +00:00
Calum Lind da5d5bee20 [#3325|Core] Fix unable to remove magnet with delete_copies enabled
Users were encountering the following error while attempting to delete
magnet torrents and had the config 'Delete copy of torrent file'
enabled. This was due to removing a magnet before the metadata was
downloaded and the torrent.filename was still set to None so raises
exceptions when string operations are performed with it.

  File "/usr/lib/python3/dist-packages/deluge/core/torrent.py", line 1317, in delete_torrentfile
    os.path.join(self.config['torrentfiles_location'], self.filename)
  ...
  TypeError: join() argument must be str or bytes, not 'NoneType'

Fixed by both setting a default empty string for self.filename and only
deleting the torrent file copy if filename is set.
2021-02-05 20:33:19 +00:00
Vasilij Schneidermann 6d9dc9bd42 [WebUI] Add country flag alt/title for accessibility
This allows viewing the country in textual form by hovering the flag
image and displays it if the image couldn't be loaded.
2021-01-29 18:49:24 +00:00
bendikro 937afd921c [Tests] Fix console tests sometimes failing due to hard coded port
The tests in ConsoleUIWithDaemonBaseTestCase could fail to the hard coded
port 58900 being busy.
Fix by using the proper port found in self.listen_port

Also convert unnecessary use of assertTrue where more appropriate
assert functions should be used, such as assertEqual and assertIn
2021-01-29 18:46:42 +00:00
EFS a4da8d29f8 [WebUI] Fix tracker icon download error
Encoutering an error when webui attempts to download tracker icon:

    Error occurred downloading file from "http://b'acg.rip'/": invalid
    hostname: b'acg.rip'

Fixed by ensuring the request.tracker_name is decoded from bytes before
looking up the icon name.
2021-01-29 18:31:52 +00:00
neeshy 8ec5ca9d08 [Console] Fix setting 'Skip' priority on console
Selecting priorities 'Low' and 'Skip' on console will both set the
actual priority to 'Low'.

Fixed typo in previous commit 6655fe67c3
2021-01-29 18:26:35 +00:00
scudre 9c90136f57 [#3439] Execute plugin fails to run on Windows
Fixed TypeError: a bytes-like object is required, not 'str'
2021-01-29 18:17:35 +00:00
Calum Lind 610a1bb313 [Lint] Update pre-commit hook and isort versions
* Fixed black hook requiring Py3.6 to installed locally. Will now assume
Py3.6+ in installed.
 * Added isort traceback in pre-commit flake8 hook fails
 * Updated versions of Black, Prettier and isort
 * Keep Flake8 at 3.7.9 due to E402 issue: https://gitlab.com/pycqa/flake8/-/issues/638
 * New pyproject config for isort v5 with fixes for Python 2 imports.
 * Fixed travis config to run Python 3.6 for lint run. Replaced the
virtualenv with_system_site_packages config with Travis specific Python
config value so lint run doesn't attempt to append with_system_site_packages
to Python 3.6 command.
2021-01-24 20:40:20 +00:00
Calum Lind 23a48dd01c [#3309|GTK] Fix cmp function for None types
Comparisons on Python 3 are much stricter resulting in the following
error comparing with None:

    TypeError: '>' not supported between instances of 'NoneType' and 'str'

Fix this by getting the type of the other value and getting it's default
value.
2020-04-30 14:30:37 +01:00
Calum Lind d02fa72e80 [Console] Fix hostlist status lookup errors
If a host in hostlist failed DNS lookup or other issue it was returning
a tuple instead of deferred. Fix this in hostlist by returning a
defer.succeed.

A race condition with BaseMode was also encountered when
update_hosts_status calls update_select_host_popup and
ConnectionManager does not have a rows attribute. Fix this by init
BaseMode before update_hosts_status and remove already called
update_select_host_popup.
2020-04-27 16:24:33 +01:00
Calum Lind 62d8749e74 [#3348] Fix TypeError adding peers to torrents
Python3 has stricter type checking and passing a port as string results
in libtorrent raising a TypeError.

Fixed by casting port to int, along with refactoring to ensure ipv6 is
correctly parsing and a useful error is output to user with invalid ip
or port details.

https://dev.deluge-torrent.org/ticket/3348
2020-04-25 13:16:04 +01:00
Nitzan Raz 76f0bf2e04 Ctl+Q to quit Deluge GTK without killing daemon 2020-04-25 13:08:54 +01:00
Calum Lind 635f6d970d [Config] Fix loading config with double-quotes in string
If a password or other string contained a double-quote then the config
would fail to be loaded on startup and reset.

This occurred due to fixing a similar issue with curly braces for #3079
in commit 33e9545cd4 and the checking for double-quotes had unforseen
consequences.

To resolve both these issues the code to check for json objects in
config files was simplified and utilises the json module raw_decode
method to ensure the extracted string indexes are json objects.
2020-04-25 10:49:08 +01:00
bendikro 672e3c42a8 [Tests] Add pytest markers to tox.ini
Remove pytest warnings due to unknown markers
2020-04-23 17:19:37 +01:00
bendikro c1110e4ef3 [Tests] Fix tests failing when deluged fails to listen
Commit b32c5d824 changed the logged message in deluge/core/daemon_entry.py
when libtorrent fails to listen on the given port, without updating the
trigger expression in deluge/tests/common.py:start_core to match the new output.

Fix by updating the trigger match expressions to match the new log output
2020-04-23 17:19:37 +01:00
bendikro 742c8a941a [Tests] Fix PytestDeprecationWarning from pytest
Accessing pytest.config is deprecated and produces:
PytestDeprecationWarning: the pytest.config global is deprecated. Please use
request.config or pytest_configure (if you're a pytest plugin) instead.

Fix by using a pytest.fixture
2020-04-23 17:19:37 +01:00
bendikro 3427ae4b90 [GTK] Remove PyGIWarning in gtk3/files_tab.py
Remove warning: PyGIWarning: Gtk was imported without specifying a version first
2020-04-23 17:17:20 +01:00
bendikro 034db27936 [GTK] Add more width to outgoing ports spinbuttons in network preferences
The spinbuttons would sometimes be truncated.
Fix by increasing the width
2020-04-23 17:17:20 +01:00
bendikro 1e3c624613 [GTK] Destroy the dialog before running the callback
Currently, the dialog window is displayed until after the callback has returned.
The result is that if a new dialog is opened from the callback, the first dialog
is still displayed until the new dialog is destroyed.

Fix by destroying the dialog before running the callback.
2020-04-23 17:17:20 +01:00
bendikro 3519f341d4 [GTK] Fix showing correct error on libtorrent import error
The exception string "No module named libtorrent" was changed to
"No module named 'libtorrent'" in python 3.3, which results in a
"unknown Import Error" message being displayed instead of the
message meant for libtorrent import error.

Change to raising LibtorrentImportError in _libtorrent.py and
catch this error to display libtorrent specific import errors.
2020-04-23 17:17:20 +01:00
neeshy d6c96d6291 Fix warning related to gettext 2020-04-23 17:14:24 +01:00
Alex Knaust 15c250e152 Fix template config.ui naming in create_plugin script. 2020-04-20 00:01:16 -07:00
Calum Lind eb57412601 [Tests] Fix tox, pytest and travis issues
* Error occurring with Pytest 5.4 so pin to below that version.
 * Fix minor issues with Travis config.
 * Use full command-switches for pytest in tox config.
 * Remove pin for pip as issue with pip-wheel-metadata was fixed in 19.3
 * Remove tox-venv as causing issues of incompatible packages installed.
   The latest versions of the virtualenv package should handle these
   duties.
2020-04-12 17:37:42 +01:00
Calum Lind 2f1c008a26 [Console] Fix AttributeError setting config values
GitHub user JohnDoee reported that config settings are not decoded
correctly, this error can be reproduced with a command like

    deluge-console -c /config/ "config --set download_location /downloads"

    > AttributeError: 'str' object has no attribute 'decode'

The tokenize code was using 'string-escape' to decode strings but there
is no direct replacement in Python 3 but also unnecessary. However the
tokenize code is complex and buggy and not really suitable for the task
of evaluating config values.

A better alternative is to evaluate the config values using the json
decoder with some additional logic to allow for previous syntax usage,
such as parentheses.

Added a comprehensive set of tests to check for potential config values
passed in from command line.
2019-11-28 12:38:02 +00:00
minus 5e06aee5c8 [Logging] Fix findCaller with unknown source
In case no source was found, a 3-tuple was returned instead of a 4-tuple
in Python 3
2019-11-19 17:44:48 +01:00
minus 351664ec07 [Logging] Fix Python 3.8 compatibility
Deluge's logger class extends Python's `logging.Logger`. Since Python
3.8, it takes an additional argument `stacklevel`.
The implementation in Deluge does not support that. Work around the
problem by ignoring additional arguments.
2019-11-19 17:44:47 +01:00
Calum Lind 5f1eada3ea [Tests] Skip buggy pytest 5.2.3
Plugins test fails due to:

https://github.com/pytest-dev/pytest/issues/6194
2019-11-15 20:55:22 +00:00
Calum Lind bde4e4443e [Lint] Fix Black and Flake8 issues
For a single element unpack black now also encloses with parentheses to
make it clearer: https://github.com/psf/black/issues/1108

Fix flake8 warnings
2019-11-13 15:44:46 +00:00
Anders Jensen ed4bc5fa17 [Core] Fix potential "dictionary changed size during iteration" on shutdown 2019-11-12 15:40:38 +00:00