Commit Graph

43 Commits

Author SHA1 Message Date
Calum Lind c7dc60571e
[CI] Fix windows packaging build
With recent update to pyinstaller 6.0 the libraries are now placed in an
`_internal` folder within the bundle. This has resulted in the failure
to create copies of libssl.

However after examining the new _internal dir it appears that the x64
lib are now created so this step is no longer required.
2023-11-24 20:50:15 +00:00
Calum Lind b2005ecd78
[Tests] Fix console tests stalling by pinning pytest to 7.4.2
Tests were stalling in deluge_ui_entry with pytest 7.4.3 likely due to
changes in the way it handles stderr but it is not clean the extact
issue.

For now we will pin the pytest version and look to fix the issue later.

Reference: https://docs.pytest.org/en/stable/changelog.html#pytest-7-4-3-2023-10-24
2023-11-19 23:14:44 +00:00
Calum Lind e24081a17e
[CI] Bump Pyinstaller version for Windows packaging
Pyinstaller was pinned due to problems with v5 but these should now be
resolved with latest version.

Bump Twisted to latest minor version

Closes: https://github.com/deluge-torrent/deluge/pull/435
2023-09-18 19:56:31 +01:00
Calum Lind 6c9b058d81
[CI/CD] Fix Pillow packaging build errors
Pillow have dropped 32-bit wheels from v10 onwards so force install of
older v9 wheels since we don't want to build from src.

Refs: https://github.com/python-pillow/Pillow/issues/6941#issuecomment-1604058002
2023-09-18 14:55:53 +01:00
Calum Lind 7082d9cec4
[CI] Fix packaging errors with Python 3.7
The latest Pillow 10 does not support Py3.7 therefore wheels are no
longer available and we need to specify previous major version.

Older versions of setuptools do not correctly determine the Twisted
requirement for zope.interface>5 on Python 3.7 so ensure latest
installed. For the CD builds we don't want any surprises so keep the
setuptools version pinned.

Refs: https://pillow.readthedocs.io/en/stable/installation.html
Closes: https://github.com/deluge-torrent/deluge/pull/433
2023-09-18 13:00:43 +01:00
Calum Lind d8b586e6ba
[CI] Bump image and action versions
Updated the core dump storage to be enabled only when needed.
Fixed missing catchsegv on Ubuntu 22.04 by installing glibc-tools
2023-05-30 19:45:57 +01:00
Calum Lind 905a7dc3bc
[Tests] Only pin libtorrent for tests
Bump versions for Windows packaging
2023-05-30 17:11:13 +01:00
Calum Lind 40a66278a3
[Common] Replace pkg_resources with importlib for resource path finding
With an existing Deluge package installed on the system errors were
occuring trying to start a development instance in virtualenv.

Fixed by replacing usage of deprecated pkg_resource for finding
non-python data files. Includes fallback for Python 3.7/3.8 but drops
Python 3.6 support.

The plugins are still using pkg_resources since they are distributed as
eggs and importlib extracts those data files differently to
pkg_resources so requires a different solution, either as a file stream
or manually cached when plugins are installed.

Closes: https://github.com/deluge-torrent/deluge/pull/403

Co-authored-by: DjLegolas <djlegolas@protonmail.com>
2023-05-29 12:08:34 +01:00
Calum Lind 930cf87103
[Lint] Update pre-commit apps to latest versions
Also update github CI action versions
2023-02-24 14:59:15 +00:00
Calum Lind 2a945de069
[CI] Fix tests stalling/timing out
GitHub pytest runner stalling with the following error:

    [Errno 2] No such file or directory: '/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/deluge/plugins'

This is related to the editable install of Deluge via pip

    pip install -e .

and the custom resource_filename in deluge.common is the source of the
problem where a DistInfoDistribution returns a different path to
EggInfoDistribution.

Working egg-info install

    >>> pkg_resources.get_distribution('Deluge')
    deluge 2.1.1.dev8 (/home/user/deluge)

    >>> type(pkg_resources.get_distribution('Deluge'))
    <class 'pkg_resources.EggInfoDistribution'>

    >>> pkg_resources.resource_filename('deluge', 'plugins')
    '/home/user/deluge/deluge/plugins'

This can identified by the `deluge.egg-info` directory in source
directory.

Broken dist-info install

    >>> pkg_resources.get_distribution('Deluge')
    deluge 2.1.1.dev8 (/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages)

    >>> type(pkg_resources.get_distribution('Deluge'))
    <class 'pkg_resources.DistInfoDistribution'>

    >>> pkg_resources.resource_filename('deluge', 'plugins')
    '/home/user/deluge/deluge/plugins'

This can be worked around by setting an env var that enables legacy mode
but long-term need to replace the custom resource_filename and replace
usage of pkg_resources.

https://setuptools.pypa.io/en/latest/userguide/development_mode.html#legacy-behavior
https://setuptools.pypa.io/en/latest/pkg_resources.html
2022-12-01 22:39:10 +00:00
Calum Lind d0acd3e06e
[CI] Fix installing enchant for github docs workflow
The enchant package was renamed for version 2 to enchant-2 and original
enchant package removed in Ubuntu 22.04 so docs workflow failed

Fixed by using latest package and specifying ubuntu version to avoid
unexpected failures in future.
2022-12-01 13:11:53 +00:00
Calum Lind f78506161d
[CI] Fix failing Windows Python 3.10 tests
A recent dependency change caused the tests running on GitHub Actions
under Python 3.10.5 on Windows to fail when starting pytest run:

    ...
    INTERNALERROR>   File "<frozen importlib._bootstrap>", line 123, in acquire
    INTERNALERROR> KeyError: xxxx

The cause seems to have been a newer version of chardet package released
recently.

* Fixed by pinning chardet to v4
* Also pin Windows version to 2019 to match packaging workflow

See-also: https://github.com/deluge-torrent/deluge/actions/runs/2578427588
Issue: https://github.com/chardet/chardet/issues/265
2022-06-29 15:07:23 +01:00
Calum Lind f1ec68704d
[CD] Cleanup pip cache in Win builds
- Use the more reliable setup-python cache
- Move the pip install to GTK install step for consistency
- Don't update pip to ensure consistent version
2022-06-13 19:37:41 +01:00
Martin Hertz ae3fbcca77
[Packaging] Pinned Pyinstaller to v4.10 and readme update
Pin Pyinstaller to latest v4.x until issue of aborting upon missing typelibs for various unbuilt gst-modules can be properly investigated and resolved. Specific error for one of the modules being:

`36738 INFO: Loading module hook 'hook-gi.repository.Gst.py' from 'C:\\hostedtoolcache\\windows\\Python\\3.9.13\\x64\\lib\\site-packages\\PyInstaller\\hooks'...
Traceback (most recent call last):
  File "<string>", line 7, in <module>
gi.repository.GLib.GError: g-irepository-error-quark: Typelib file for namespace 'Gst', version '1.0' not found (0)
36870 ERROR: gi repository 'GIRepository 2.0' not found. Please make sure corresponding package is installed.
Traceback (most recent call last):
  File "<string>", line 4, in <module>
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\gi\__init__.py", line 139, in require_version
    raise ValueError('Namespace %s not available' % namespace)
ValueError: Namespace Gst not available`

Added `--no-index` to ensure pip doesn't install from Pypi

Closes: https://github.com/deluge-torrent/deluge/pull/386
2022-06-13 19:31:54 +01:00
Calum Lind 6a10e8f3cd
[Packaging] Bump Win dependencies
* Update Twisted and libtorrent to latest releases
* Update to v3 github actions that now use node 16
2022-06-12 20:05:04 +01:00
Calum Lind 5acb57b5af
[CI] Use setup-python action pip cache
Replace custom pip cache that didn't work correctly on Windows with
option to use pip cache in setup-python action
2022-03-02 13:08:02 +00:00
DjLegolas a954348567
[CI] Use libtorrent pypi install
Up until now, the linux installation source of libtorrent was launchpad,
because there was no other source, and we wanted a debug version of lt.

With pypi wheel versions now available use lt in the requirements.txt
file.

Closes: https://github.com/deluge-torrent/deluge/pull/364
2022-03-02 12:55:19 +00:00
DjLegolas 13be64d355
[CI] Changed tested python version to 3.7 and 3.10
We cannot add python 3.6 because there is no precompiled version of it to used.
Therefor, will be using 3.7 as the minimum version in CI.
In addition, dropped version limits from pytest.
2022-03-02 12:53:29 +00:00
Calum Lind b29829f571
[CI] Fix package build error
Not all dependencies were installed due to adding a comment in the
middle of the pip install command

Also need to specify Twisted extras to match requirement.txt
2022-02-09 20:39:48 +00:00
Calum Lind d4f8775f44
[CI] Use working dir to shorten commands
Making the workflows more readable
2022-02-09 19:53:17 +00:00
Calum Lind 50647ab3a5
[CI] Replace custom twisted package with pre-release
Fix for Windows simulate error has been merged and in 22.2.0rc

Ref: https://github.com/twisted/twisted/pull/1679
2022-02-09 19:53:17 +00:00
Calum Lind 0fbb3882f2
[CI] Fix checkout action missing fetch depth
Need a fetch depth greater than 1 to find latest tag.
2022-02-01 06:45:10 +00:00
Calum Lind 73394f1fc5
[CI] Fix run manual packaging workflow for tag
To allow packaging any commit the workflow needs to separately checkout
the source code from the current code containing the packaging scripts.
2022-01-30 17:42:15 +00:00
Calum Lind 9b043cf2c1
[CI] Allow manual specifying tag in packaging workflow 2022-01-30 17:03:33 +00:00
tbkizle 2cad0f46f2
[CI] Add pygame to windows package build/spec file
pygame is required by notification plugin for sounds
2022-01-21 13:12:23 +00:00
Calum Lind 5931d0cc0b
[CI] Fix package job not running with PR label
The job would only run when the PR was labeled since
`github.event.label.name` only available when `labeled` type event
recieved
2022-01-21 12:53:54 +00:00
Calum Lind 9d4ca77ef7
[CI] Cleanup packaging dependencies 2022-01-21 12:53:54 +00:00
Calum Lind 15d2d27a53
[CI] Specify github windows server version
To ensure builds don't break avoid using windows-latest

Refs: https://github.com/actions/virtual-environments/issues/4856
2022-01-21 10:16:15 +00:00
Calum Lind d62362d6ae
[CI] Improve packaging workflow
Include arch in artifacts so they can be downloaded separately

Added libtorrent 2.0 to matrix since users often request latest
libtorrent.

Renamed workflow to make it's purpose clearer
2022-01-20 15:31:15 +00:00
Calum Lind 2316088f5c
[CI] Remove PR specified branch
This branch name is the head name not the base name so prevents the job
running unless submitted has branch name that matches
2022-01-14 13:07:22 +00:00
Calum Lind d14310078b
[CI] Fix typo in CD 2022-01-13 22:46:02 +00:00
Calum Lind 1696c69776
[CI] Fix windows build tag exclude
Fixes error:

    you may only define one of `tags` and `tags-ignore` for a single event
2022-01-13 22:36:56 +00:00
Calum Lind 5f96ea4217
[CI] Restrict creating Windows installer
Limit the running of this job by only running on develop, tags and pull
requests that have label 'windows'
2022-01-13 22:23:25 +00:00
tbkizle 490fb898af
Build With Patched Twisted Build
Fixes TypeError in simulate call

Ref: https://twistedmatrix.com/trac/ticket/9660
Ref: https://github.com/twisted/twisted/pull/1679
2022-01-13 22:23:25 +00:00
tbkizle 560a52a443
Fix OpenSSL For Libtorrent
libtorrent + pyinstaller requires a lib(ssl/crypto)-1_1.dll and
lib(ssl/crypto)-1_1-x64.dll odd quirk but solveable by just having
two copies. Maybe later compiling our own libtorrent.
2022-01-13 22:23:25 +00:00
tbkizle b9a208f18f
Update Windows Packaging
* Rename instances of win32 to generic win or the appropriate bit where applicable
* Remove files used in GTK2
* Add spec file for use with PyInstaller
* Remove Python bbfreeze Script
* Add Github Action To Build Releases
* Add Modified script to make files used by NSIS
* Update Readme

Closes: https://github.com/deluge-torrent/deluge/pull/331
2022-01-13 22:23:08 +00:00
DjLegolas 7f5857296e [CI] Upgrade Windows python version to 3.8 (same as linux) 2021-12-29 20:06:10 +02:00
DjLegolas 88ffd1b843
[Servers] Moved check_ssl_keys and generate_ssl_keys to crypto_utils.py
With this change, we drop a core dependency from the UI. This will help group together
all related functionality in one place, i.e. all security related functions.

Also updated testssl.sh version to 3.0.6 (SECURITY_TEST)

Closes: deluge-torrent/deluge#288
2021-12-20 22:09:08 +00:00
Calum Lind bca0aa3532
[CI] Replace pypi deluge-libtorrent with libtorrent
* Remove certifi since included in requirements.txt
* Remove old travis config
2021-12-12 21:49:31 +00:00
Calum Lind cb75192df4 [CI] Add core dump capture to GH job
Add further debugging to trace segfaults with lt 1.2
2021-09-10 19:06:21 +01:00
Calum Lind f075f391cb [CI] Add catchsegv to get a backtrace for segfaults
Encountering random libtorrent segfault with GitHub action so add
catchsegv when running tests to get more information.
2021-07-31 22:08:23 +01:00
Calum Lind 8fb25f71f3 [Install] Update and fix python optional requirements
* Added required dependency setuptools to install_requires
* Remove optional dependency ipaddress from install_requires
* Created extras_require in setup.py. The optional dependencies should
not be included in install_requires so that users can either install
forked dependencies or remove problematic ones. Updated documentation to
detail how to install these optional dependencies.
* Fixed README badge

Refs:
 * https://dev.deluge-torrent.org/ticket/3470
 * https://dev.deluge-torrent.org/ticket/3282
 * https://dev.deluge-torrent.org/ticket/3353
2021-07-31 22:08:23 +01:00
Calum Lind e5388048a9 [CI/CD] Add github actions to replace Travis
Due to new limitations for open-source projects on Travis we are
switching to GitHub actions.

* Notes about system site-packages

We had many problems with accessing system python packages on Travis for
libtorrent and GTK and the problems are harder on Github since there is
no more access. For now copying the python libtorrent binary into the
deluge source is the workaround. There is a pip package that could be
used in future.

Fixed failing tests with libtorrent 1.2 which required a non-zero length
file in torrent and workarounds for async alert delay.
2021-07-25 13:27:26 +01:00