Commit Graph

103 Commits

Author SHA1 Message Date
burnettk 57cb064025 upgrade ruff and add a few rules 2023-12-05 19:43:41 -05:00
jasquat 18600189c8 Feature/background proc with celery (#788)
* WIP: some initial test code to test out celery w/ burnettk

* some cleanup for celery and added base model to put tasks waiting on timers

* removed dup bpmn file

* some more cleanup and added strategy to queue instructions

* some minor code changes w/ burnettk

* remove the unused next_task key from api calls since nobody uses it w/ burnettk essweine

* added migration for future tasks and added test to make sure we are inserting into it w/ burnettk essweine

* ensure future task run at time can be updated w/ burnettk

* added table to queue instructions for end user in w/ burnettk

* added test to ensure we are storing instructions for end users w/ burnettk

* added progress page to display new instructions to user

* ignore dup instructions on db insert w/ burnettk

* some more updates for celery w/ burnettk

* some pyl and test fixes w/ burnettk

* fixed tests w/ burnettk

* WIP: added in page to show instructions on pi show page w/ burnettk

* pi show page is fully using not interstitial now w/ burnettk

* fixed broken test w/ burnettk

* moved background processing items to own module w/ burnettk

* fixed apscheduler start script

* updated celery task queue to handle future tasks and upgraded black and set its line-length to match ruff w/ burnettk

* added support to run future tasks using countdown w/ burnettk

* build image for celery branch w/ burnettk

* poet does not exist in the image w/ burnettk

* start blocking scheduler should always start the scheduler w/ burnettk

* add init and stuff for this branch

* make this work not just on my mac

* send other args to only

* added running status for process instance and use that on fe to go to show page and added additional identifier to locking system to isolate celery workers better w/ burnettk

* fixed typing error that typeguard found, not sure why mypy did not w/ burnettk

* do not check for no instructions on interstitial page for cypress tests on frontend w/ burnettk

* do not queue process instances twice w/ burnettk

* removed bad file w/ burnettk

* queue tasks using strings to avoid circular imports when attmepting to queue w/ burnettk

* only queue imminent new timer events and mock celery

* some keyboard shortcut support on frontend and added ability to force run a process instance over the api w/ burnettk

* some styles added for the shortcut menu w/ burnettk

* pyl w/ burnettk

* fixed test w/ burnettk

* removed temporary celery script and added support for celery worker in run server locally w/ burnettk

* cleaned up migrations w/ burnettk

* created new migration to clean up old migrations

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-12-05 11:41:59 -05:00
Kevin Burnett 353ba4cd31 avoid print statements, generally (#784)
* avoid print statements, generally

* remove test for empty main script

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-12-01 06:24:11 -08:00
Kevin Burnett cfe00a45d4 reset to page 1 when status changes to fix #765 (#769)
* reset to page 1 when status changes to fix #765 w/ jasquat

* upgrade connexion and werkzeug to fix snyk w/ jasquat

* fix all security issues like a boss w/ jasquat

* whoops, still no resolution for cryptography w/ jasquat

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-11-28 08:58:54 -08:00
jbirddog abe865c19b Data Store CRUD interface (#689) 2023-11-24 13:27:46 -05:00
burnettk f524b78368 fixes from fixit 2 2023-11-20 11:04:33 -05:00
jasquat fc503ec76b feature/secrets-encryption-cleaning (#673)
* removed simple-crypt and cleaned up usage of keys for encryption w/ burnettk

* renamed var to SPIFFWORKFLOW_BACKEND_ENCRYPTION_KEY w/ burnettk

* pyl w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-16 10:12:47 -05:00
jasquat a025aaa017 upgraded cryptography to satisfy snyk and added ignore for werkzeug issue since we cannot do anything about it now w/ burnettk 2023-10-26 11:28:37 -04:00
jasquat 2f9aa12906 Feature/error boundary (#552)
* WIP for error boundary support w/ burnettk

* updates to catch error boundary events and send them back out w/ burnettk

* fixed broken test w/ burnettk

* use connector-http and spiffworkflow-proxy from main w/ burnettk

* updated smtp and slack connectors in connector-proxy-demo w/ burnettk

* added more tests for the service task service call connector to test errors w/ burnettk

* added spiffworkflow-connector-command and some code clean up

* updated the connectors in the proxy demo

* use SpiffWorkflow from git instead of locally

* some cleanup while code reviewing

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-18 14:00:12 -04:00
jasquat cba4a19d3b Feature/remove group service (#529)
* removed group service in favor of user service and messing around with ruff and pre-commeit w/ burnettk

* pyl can succeed now w/ burnettk

* removed debug code w/ burnettk

* pyl

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-05 13:27:38 -04:00
jasquat a84750741e updated SpiffWorkflow for depth fix (#519)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-28 12:51:48 -07:00
jasquat 47f94dccbb Feature/nonetype debug stuff (#513)
* some debug items w/ burnettk

* removed some db commits from workflow execution service so the passed in save is the only thing that actually commits the transaction

* set max depth to 50000 for now w/ burnettk

* pyl w/ burnettk

* use temp maxdepth fix in SpiffWorkflow w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-27 17:04:47 -04:00
jbirddog 9925105a5e Support lazy loading call activities with human tasks (#450)
* Switch to local wheel

* Fix to load the call activity by spec

* Added log

* Logging

* WIP - getting the right tasks it seems from the db, need to get into the workflow

* Clean up, revert things

* Working thanks dan

* Getting bin_pyl to pass

* run_pyl.

* Re-poet install

* Revert file

* Remove logging lines

* Getting bin_pyl to pass

* Clean up, revert things

* Bump wheel

* Use version off pypi

---------

Co-authored-by: danfunk <daniel.h.funk@gmail.com>
2023-09-07 10:33:38 -04:00
jbirddog 2b363f4783 Oauth admin (#434)
* Adding dependencies

* Disconnect from /v1/auths for auth list. hardcoded for now.

* Revert changes

* WIP

* Getting hardcoded v2 auths into the frontend

* Better url for v2 oauth

* Pass the auth token from the frontend, don't verify token to start the auth process

* Manually verify the token from the querystring

* WIP

* WIP

* WIP, refactor SPIFF_SECRET handling, move dependencies

* Construct remote_app

* WIP

* WIP

* WIP

* WIP

* Ugly but getting the grant screen

* WIP

* WIP

* Github oauth ok

* Verify token, save access token

* Let secret name work with regex

* Getting bin_pyl to pass

* New component

* Load up the current config in an editor

* Getting bin_pyl to pass

* End point to update auth config

* Linting

* Adding configuration model

* Adding configuration model

* Prep to read config from db

* Read config from the db

* Save/reload poor man's styling

* Getting bin_pyl to pass

* Getting bin_pyl to pass

* Getting bin_pyl to pass

* Better handling of invalid json

* Getting bin_pyl to pass
2023-08-21 21:58:55 +05:30
jasquat 3f6fb4d9f5 Feature/pin spiff in ci (#428)
* attempt to use the locked version of SpiffWorkflow in ci w/ burnettk

* fixed regex for spiffworkflow revision replacement

* make sure we also update the lockfile when getting new spiff

* install poetry before we attempt to update hte pyproject

* hardcoding spiffworkflow revision as a test w/ burnettk

* try running tests in ci from bash script w/ burnettk

* print working dir in ci w/ burnettk

* fixed location of instance dir w/ burnettk

* run with mysql in ci

* run typeguard with bash script as well w/ burnettk

* fixed postgres test w/ burnettk

* clean up github action file w/ burnettk

* fixed postgres test again w/ burnettk

* pyl

* attempt to remove nox from ci completely

* omit safety for now to test coverage

* fixed how coverage is being called from not nox

* allow running safety and macos again

* renamed run_not_nox to run_ci_session w/ burnettk

* attempt to only upload if matrix says to w/ burnettk

* attempt to install mysqlclient prereqs for mac and remove noxfile stuff w/ burnettk

* added back the constraints file w/ burnettk

* moved the contributing file to the root of arena w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-08-03 11:49:30 -04:00
jasquat 84369a1b20 return the process instance early from the interstitial if it is susp… (#407)
* return the process instance early from the interstitial if it is suspended or terminated

* added a test to make sure the interstitial page returns the process instance if suspended or termianted w/ burnettk

* randomize tests and cleaned up the pyproject file a little bit w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-07-25 22:57:53 +05:30
jasquat c936c02db0 use mysqlclient instead of mysql-connect w/ burnettk (#358)
* use mysqlclient instead of mysql-connect w/ burnettk

* updated some comments w/ burnettk

* runs tests in parallel in run_pyl

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-07-12 19:44:49 +05:30
jasquat e4bf22f5cf Feature/interstitial summary part 3 (#367)
* throw an error if backend gives a 500 from interstitial page w/ burnettk

* turn unexpected errors for the interstitial page into event stream responses to avoid error misdirection

* pyl

* raise the api_error from the original and make the render_data method private

* Feature/interstitial summary (#337)

* * Process instance logs and messages are now components rather than pages, and are included within tabs on the process instance page, along with the diagram.
* Removed the Zoom and Move modules when showing the readonly
  Diagram.  Assured this readonly view is resized to fit the space when possible.
* Checkbox Widget no longer displays a duplicate label.
* CSS Tweaks
  * All pages are limited to a max display width of 1440, with auto margins to center the main content on the page.
  * "Show" pages, like ProcessInstanceShow, TaskShow have the primary content limited to 1000, also with auto-margins.
  * Paragraphs, headings, blockquotes, list items are limited to a width of 640.
  * Reduced margin bottom on all breadcrumbs.
  * Slightly reduced the width and margin of tiles
  * ordered lists and unordered lists show numbers and bullets now.
* End user Instructions component can, optionally, auto-collapse, so that only a portion is displayed, along with a toggle.  This is how it is set up for the ProcessInstanceShow page.
* Greatly reduced the lag in the interstitial page when doing a re-direct.

* run_pyl

* kill console

* wait for permissionsLoaded too since we are using ability.can

* Previous change removed the top level Messages page - this re-adds it.

* I am always, ALWAYS shocked at how I can not wrap my head around when and where to use "useEffect".

This should cause the show/hide Instructions toggle to only show when useful.

* Minor cleanup on the process instance properties display.

* linting

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>

* Fix a linting error.

* minor permissions related fixes for the new ui.

* Revert "Revert "Feature/better subworkflow management (#331)""

This reverts commit 48dcde8faf.

* do not execute tasks from the process instance show page w/ burnettk

* pyl w/ burnettk

* Very minor UI tweak to resolve a number of issues mentioned in SPIFF-316 Notion Ticket:

* Markdown links now open in a new window
* Tables on the home pages are now contained within Grids to better align them with the other content that is in a grid.
* Right aligned the "process instance list link" button to it is flush right with the table underneath
* Gave a little more breathing room the content on the info/metadata in the process instance view .

* updated docker image build action to contain the appropriate version info w/ burnettk

* properly resize to the available columns depending on s/m/l

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: Dan Funk <daniel.h.funk@gmail.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-07-03 10:54:54 +05:30
Kevin Burnett b13b716424 parallel tests with xdist (#335)
* parallel tests with xdist

* add pytest-xdist as dev dep

* put back spiff

* update messaging

* get more in line with main

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-26 22:37:41 +05:30
Kevin Burnett 7de0d97303 Feature/better subworkflow management part 2 (#341)
* Revert "Revert "Feature/better subworkflow management (#331)""

This reverts commit 48dcde8faf.

* updated SpiffWorkflow to fix infinite loop in task trace w/ burnettk

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-20 10:37:49 -04:00
burnettk 48dcde8faf Revert "Feature/better subworkflow management (#331)"
This reverts commit 23af5cbbe4.
2023-06-15 15:10:48 -04:00
jasquat 23af5cbbe4 Feature/better subworkflow management (#331)
* WIP: some updates to support new spiff w/ burnettk

* unit tests are passing

* all tests except message tests are passing

* fixed usage of catch message event w/ burnettk

* messages are working again w/ burnettk

* uncommented remaining message tests w/ burnettk

* fixed cypress tests w/ burnettk

* use main for spiffworkflow

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-15 10:31:42 -04:00
Kevin Burnett 35543cc362 add mccabe linter and auto fixes (#302)
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-06-08 14:26:33 -07:00
jbirddog 122d1efbda Bump flask for safety (#304)
* Bump flask for safety

* let snyk check flask again w/ burnettk

* attempt to use the same revision for front w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-06 17:33:48 -07:00
jasquat 5dfd96ca9e upgrade cryptography for security w/ burnettk 2023-06-02 12:09:24 -04:00
burnettk 397d99b5c0 enable flake8-bandit, cut off all http requests at 15 seconds to avoid hanging process, ignore xml lib spiff uses 2023-05-30 19:53:26 -04:00
burnettk aae043e0a4 add pylint 2023-05-30 19:39:15 -04:00
burnettk a567b990e6 enable pep8-naming and use ruff for autofixer 2023-05-30 07:15:49 -04:00
burnettk 6bbabd15f4 instead of actually adding pyright configs that do not work, document 2023-05-29 11:52:35 -04:00
burnettk 4b21f02355 get rid of some pyright diagnostics 2023-05-27 22:33:53 -04:00
burnettk 6a328cdb43 actually stop running tests against 3.9 2023-05-27 18:34:04 -04:00
burnettk be9859752e drop python 3.9 and remove unused libs 2023-05-27 18:17:51 -04:00
burnettk f8d4a6382b get compatible with ruff pyupgrade 2023-05-27 14:59:52 -04:00
burnettk 2b2875cb6f let ruff sort imports and ditch duplicative pre-commit linters 2023-05-26 20:01:08 -04:00
burnettk cf40546a09 get compatible with default ruff ruleset 2023-05-26 13:07:18 -04:00
burnettk 8d20ef6956 remove sphinx from pyproject since we do not use it in this way 2023-05-18 18:42:40 -04:00
danfunk 2bdf4e1cfc Remove incorrectly added myst-parser from backend. 2023-05-18 16:30:38 -04:00
danfunk 670a1d5363 initial set of docs for SpiffWorkflow 2023-05-18 10:49:44 -04:00
jasquat de8a718a30 removed flask-bpmn from pyproject since it is no longer used w/ burnettk 2023-05-10 17:09:51 -04:00
danfunk cc55f4413b run_pyl
use latest spiffworkflow in main
2023-05-10 15:42:07 -04:00
danfunk 4354efe536 minor fix to spiffworkflow backend. 2023-05-10 14:51:13 -04:00
danfunk b9a8b97fda Various updates to get the reset working correctly. 2023-05-10 10:49:39 -04:00
Dan 30a26de38d wip 2023-05-09 15:02:05 -04:00
jbirddog b89527ab80 Integrate spiff-element-units 0.3.0 for (some) lazy call activities (#239) 2023-05-04 14:15:13 -04:00
jasquat 418ec7d9c9 updated typeguard and fixed issues w/ burnettk 2023-05-01 15:26:29 -04:00
jbirddog 1dd445de68 spiff-element-units integration (#214) 2023-04-17 14:49:38 -04:00
jasquat 106b03d32b merged in main and resolved conflicts w/ burnettk 2023-04-07 14:59:49 -04:00
jasquat e12db76038 some cleanup, updated SpiffWorkflow, and added some notes to pick back up where we left off with the more efficient method w/ burnettk 2023-04-06 10:57:52 -04:00
jasquat c5b85fd404 WIP - test is still passing, no longer processing all tasks w/ burnettk 2023-04-05 16:07:35 -04:00
jasquat a7521cba65 run snyk on backend and added marshmallow dep to remove warning w/ burnettk 2023-03-28 16:55:13 -04:00