As PEP 594 says, cgi module is marked as deprecated
in python 3.11, and will be removed in 3.13
(actually removed at least in 3.13 rc1).
As suggested on PEP 594, replace cgi.parse_header
with email.message.EmailMessage introduced in python 3.6.
Updated test modify test_download_with_rename_sanitised
- With RFC2045 specification, Content-Disposition filenames
parameter containing slash (directory separator) must be
quoted, so changing as such.
Ref: https://peps.python.org/pep-0594/#deprecated-modules
Ref: https://peps.python.org/pep-0594/#cgi
Closes: https://github.com/deluge-torrent/deluge/pull/462
The test_pause_not_pop_alert test passes on Linux but is consistently failing
in CI pipeline for Windows:
AssertionError: Expected 'mock' to not have been called. Called 1 times.
Calls: [call.deferred.cancel(),
call(LtAlertMock(type=1, name='mock_alert1', message='Alert 1'))].
Disabling the test until it can be resolved.
We are currently creating a copy of each alert to avoid segfaults when
the next pop_alerts invalidates the lt alert objects we are holding
for handler callbacks.
However these alert copies are not deep enough since need to also
resolve the alert object methods e.g. message() therefore it is still
possible to result in lt segfaults.
The workaround is to check for any handlers not called, give them some
more time and eventually discard if still not handled.
Ref: https://github.com/arvidn/libtorrent/issues/6437
This spawns a thread in alertmanager to call wait_for_alert in a thread.
This reduces latency to deluge responding to events.
And removes all `hasattr` checks in Component
Closes: https://github.com/deluge-torrent/deluge/pull/221
For new UI features to be added, one should make sure the backend daemon
is supported and add fallback in case it doesn't.
Here we add the ability to get the daemon version from the `Client`
class and also check the version against a desired version.
Closes: https://github.com/deluge-torrent/deluge/pull/427
Add support for v2 torrents in create_torrent, but keep the old
default of only adding the v1 metadata.
Unify the single-file and directory cases to avoid code
duplication.
V2 torrents require files to be piece-aligned. The same for
hybrid v1/v2 ones. To handle both cases of piece-aligned and
non-aligned files, always read the files in piece-aligned
chunks. Re-slice the buffer if needed (for v1-only multi-file
torrents).
Also, had to adapt to progress event. It now depends on the
number of bytes hashed rather than the number of pieces. To
avoid sending and excessive amount of event when handling a
directory with many small files, add a mechanism to limit
event period at 1 per piece_length.
Closes: https://github.com/deluge-torrent/deluge/pull/430
When adding a torrent in the add torrents dialog, containing only
a single directory with a single file inside, the directory is not
included as a prefix to the filename.
The prefix is added only if there are multiple files inside the directory.
Fix by adding the prefix also when there is only one file inside a dir.
Closes: https://dev.deluge-torrent.org/ticket/3602
Closes: https://github.com/deluge-torrent/deluge/pull/424
Encountered a problem with dht_error alert not returning the correct
alert name using Python type.
This should likely be fixed in libtorrent but we should be using
the alert.what method to determine alert type/name.
Since the alert name does not include the `_alert` suffix, strip this
when registering alerts.
Since all of our async tests should be run with twisted, it's
annoying to have to decorate them with pytest_twisted.ensureDeferred.
Forgetting to do this will cause the test to pass without actually
running. This changes the behavior to detect all coroutine function
tests and mark them to be run by pytest_twisted the same way the
decorator does.
Closes: https://github.com/deluge-torrent/deluge/pull/414
Modified test functions to be async.
Used pytest_twisted_ensuredeferred_for_class decorator to avoid needed
ensureDeferred for each test within the class. There might be a way to
do this with fixtures so likely to be improvements for use in all test
classes.
Used Mock in component subclass for simpler tracking of event method calls
Daemon logfile were being stored in dir where tests were started from
which cluttered up local dev env.
Entry point logfiles are stored in config dir since getting tmp_path in
set_up was a bit too tricky.
Simplify the code for setting the base path, both via headers and
config. Replaced putchild since it is not recommended for dynamic
paths and overriding getChildWithDefault provides a proper solution.
This also fixes recursive base path problem with previous code where
appending base paths to URL would still return Deluge web e.g.
http://localhost:8112/deluge/deluge/deluge
Removed getChild override by consolidating empty path conditional to
getChildWithDefault. This simplifies and combines the returning of
TopLevel resource for root path or base path.
Added workaround for test logfile error with forwardslash in filename
Error in test_rename_unicode:
TypeError: object MagicMock can't be used in 'await' expression
Fixed by using AsyncMock that can be awaited.
Added backport asyncmock for Python 3.7
Magnets with trackers specified with tr.x param were not being unquoted
so unusable raw tracker string was being set.
Fixed by unquoting tracker and adding test
See-also: https://dev.deluge-torrent.org/ticket/2716
The changes to remove deprecated lt methods didn't account for magnet
trackers so magnets are missing trackers when added.
Previously the addition of trackers was handled by libtorrent when a url
was passed in add_torrent_params. The url parameter is deprecated so
instead we need to add both the info_hash and trackers.
Trac: https://dev.deluge-torrent.org/ticket/3530
Just trying to clean up some of the more complicated callback logic.
Notable changes:
* The test was awaiting a DeferredList. By default that will eat
exceptions and just add them to the result list (including test
assertion exceptions.) Added fireOnOneErrback=True to make sure that
wasn't happening. * Moved the logic for multiple calls to await the
same response into torrentmanager from core, so no matter where the
prefetch is called from it will wait for the original call.
* Implemented the multiple calls with an explicit queue of waiting
callbacks, rather than a callback callback chain. * Moved to one inline
async function rather than split into a main and callback after alert
function.
* Added some more type hints to the stuff I changed.
Adjusted test since we are using prefetch as an async function now
we have to schedule the alert to come after we start awaiting the
prefetch call.
Closes: https://github.com/deluge-torrent/deluge/pull/368
Temporary torrent files are not deleted by add_torrent_url. Not as big a
problem as with tracker icons pages but should be removed after use.
Fixed by updating the method to use async and a try..finally cleanup
block.
Perhaps could be refactored to not require temporary files and instead
store the downloaded torrent as object for passing to add_torrent_file.
Trac: https://dev.deluge-torrent.org/ticket/3167
Temporary files created while download host html page are no cleaned up
if the download fails.
Fixed by adding a 'finally' step in the callback chain to delete any
created temporary files.
Added tests to ensure the temporary files are deleted, using a fixture
that creates a known filename for the test.
Trac: https://dev.deluge-torrent.org/ticket/3167
GitHub CI tests on Windows failing for get_windows_interface_name so
remove the fragile tests since not a requirement to be this specific
with testing whether name exists for these methods relying on standard
lib or 3rd-party libs.
The SNI icon test is failing due to seo.com removing their favicon.
A new test to replace it would be needed to check SNI support. Ideally
new tests would not rely on external sites.
Notable changes:
* Prettier >=2.3 with more consistent js assignments
* Black now formats docstrings
* Added isort to list of autoformaters
* Update flake8 config for v4
Ref: https://prettier.io/blog/2021/05/09/2.3.0.html
As part of the process of adding support to LT 2.0, we should stop using
all deprecated function, as some (if not all) were removed.
For this process, we should use the LT 1.2 upgrade (guide)[1].
The change includes:
* stop using file entries directly
* start using the torrent handle's set/unset flags
* stop using url key in add_torrent_params (for magnet)
* stop accessing resume_data from save_resume_data_alert
* stop using deprecated session status keys in UI
[1] https://libtorrent.org/upgrade_to_1.2-ref.html
Closes: https://dev.deluge-torrent.org/ticket/3499
Closes: https://github.com/deluge-torrent/deluge/pull/342
In a6840296, a refactor to the `config` class was introduced.
The change included an internal wrapper for `reactor.callLater`, for lazy
import, but didn't wrap it correctly and therefor, no args/kwargs were
passed to the wrapped method.
Furthermore, the exception was silently ignored.
This caused changes to be ignored and not applied, including
`preferencesmanager._on_config_value_change` callback.
Closes: https://github.com/deluge-torrent/deluge/pull/372
Our file priority test was using time.sleep to wait until libtorrent
had processed the command. This was sometimes not long enough and the
test would fail. On libtorrent 2.0.3+ there is an alert when the
process has finished, switch to waiting for that in this test to make
the test more consistent. On older libtorrent, make the delay a bit
longer, to try to make the test more consistent there as well.
Closes: https://github.com/deluge-torrent/deluge/pull/373
- Clean up callback hell by making more code inline
- Use async/await syntax as it has more modern niceties than inlineCallbacks
- Also gets us closer if we want to transition to asyncio in the future
- `await` is usable in places that `yield` is not. e.g. `return await thething` `func(await thething, 'otherparam')`
- IDEs know async semantics of async/await syntax to help more than with `inlineCallbacks`
- `maybe_coroutine` decorator has nice property (over `ensureDeferred`) that when used in a chain of other coroutines, they won't be wrapped in deferreds on each level, and so traceback will show each `await` call leading to the error.
- All async functions wrapped in `maybe_coroutine` are 100% backwards compatible with a regular Deferred returning function. Whether called from a coroutine or not.
- Use Deferred type hints as strings since older versions of twisted
(<21.7) don't support generic Deferred type hinting.
Added a new Config class parameter `log_mask_funcs` to enable config
instances hide sensitive information that would normally appear in
config debug logs.
Added mask password function to hostlist to replace passwords with '*'s
in logs.
Closes: https://github.com/deluge-torrent/deluge/pull/363
* Refactored duplication with setting config key and logging
* Simplified lazy importing reactor for callLater. This lazy importing
is required for testing and also prevents Gtk UI lockup if reactor
imported in Config.
* Fixed saving config to file when setting a key that doesn't exist yet.
This was due to returning early in the set_item method.
* Added a `default` arg to set_item to prevent saving to file when only
setting a default value for a key in init.
* Moved casting value to existing key type from set_item to dedicated
function.
Convert all the twisted.trial tests to pytest_twisted. Also move off of unittest.TestCase as well. Seems there were several tests which weren't actually testing what they should, and also some code that wasn't doing what the broken test said it should.
Goals:
Remove twisted.trial tests
Move to pytest fixtures, rather than many classess and subclasses with setup and teardown functions
Move away from self.assertX to assert style tests
FIx broken tests
Going forward I think these should be the goals when adding/modifying tests:
* Don't use BaseTest or set_up tear_down methods any more. Fixtures should be used either in the test module/class, or make/improve the ones available in conftest.py
* For sure don't use unittest or twisted.trial, they mess up the pytest stuff.
* Prefer pytest_twisted.ensureDeferred with an async function over inlineCallbacks.
- I think the async function syntax is nicer, and it helps catch silly mistakes, e.g. await None is invalid, but yield None isn't, so if some function returns an unexpected thing we try to await on, it will be caught earlier. (I struggled debugging a test for quite a while, then caught it immediately when switching to the new syntax)
- Once the maybe_coroutine PR goes in, using the async syntax can also improve tracebacks when debugging tests.
Things that should probably be cleaned up going forward:
* Remove BaseTestCase
* Remove the subclasses like DaemonBase in favor of new fixtures.
* I think there are some other utility subclasses that could be removed too
* Perhaps use parameterization in the ui_entry tests, rather that the weird combination of subclasses and the set_var fixture I mixed in.
* Convert some of the callback stuff to pytest_twisted.ensureDeferred tests, just for nicer readability
Details relating to pytest fixtures conftest.py in root dir:
* https://github.com/pytest-dev/pytest/issues/5822#issuecomment-697331920
* https://docs.pytest.org/en/latest/deprecations.html#pytest-plugins-in-non-top-level-conftest-files
Closes: https://github.com/deluge-torrent/deluge/pull/354
Libtorrent now supports interface names instead of just IP address so
add new common functions to validate user input.
* Added is_interface that will verify if a libtorrent interface of name
or IP address.
* Added is_interface_name to verify that the name supplied is a valid
network interface name in the operating system.
On Windows sock.if_nameindex() is only supported on 3.8+ and does not
return a uuid (required by libtorrent) so use ifaddr package. Using git
commit version for ifaddr due to adapter name decode bug in v0.1.7.
On other OSes attempt to use stdlib and fallback to ifaddr if installed
otherwiser return True.
* Added tests for is_interface & is_interface_name
* Updated UIs with change from address to interface
* Updated is_ipv6 and is_ipv4 to used inet_pton; now supported on
Windows.
Ref: https://github.com/pydron/ifaddr/pull/32
Closes: https://github.com/deluge-torrent/deluge/pull/338
