Commit Graph

175 Commits

Author SHA1 Message Date
jbirddog 1c0570c147
Bump Restricted Python to 7.x, needed for Python 3.12 support (#872) 2024-01-10 10:46:28 -05:00
jbirddog ab39569cac
Add direct dependency on jsonschema (#868) 2024-01-10 08:18:34 -05:00
Kevin Burnett e1fefa9fc7
nplusone (#824)
* n plus one checks

* tests passing with nplusone

* satisfy probably-wrong json schema

* remove useless comment and consolidate poetry dev deps

* not actually going to add this dependency, but leave docs for next guy

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-12-20 07:18:20 -08:00
jasquat c00d810704 Feature/data object category (#796)
* WIP: some initial code for category in data objects

* attempt to get the data object for the given bpmn process and instance of it w/ burnettk

* updates for data objects

* fixed tests

* made suggestions by code rabbit and moved logic to get process data file out of shared method since it has a completely different implentation

* remove commented out code

* updated SpiffWorkflow for data object category

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-12-11 07:42:01 -08:00
burnettk 57cb064025 upgrade ruff and add a few rules 2023-12-05 19:43:41 -05:00
jasquat 18600189c8 Feature/background proc with celery (#788)
* WIP: some initial test code to test out celery w/ burnettk

* some cleanup for celery and added base model to put tasks waiting on timers

* removed dup bpmn file

* some more cleanup and added strategy to queue instructions

* some minor code changes w/ burnettk

* remove the unused next_task key from api calls since nobody uses it w/ burnettk essweine

* added migration for future tasks and added test to make sure we are inserting into it w/ burnettk essweine

* ensure future task run at time can be updated w/ burnettk

* added table to queue instructions for end user in w/ burnettk

* added test to ensure we are storing instructions for end users w/ burnettk

* added progress page to display new instructions to user

* ignore dup instructions on db insert w/ burnettk

* some more updates for celery w/ burnettk

* some pyl and test fixes w/ burnettk

* fixed tests w/ burnettk

* WIP: added in page to show instructions on pi show page w/ burnettk

* pi show page is fully using not interstitial now w/ burnettk

* fixed broken test w/ burnettk

* moved background processing items to own module w/ burnettk

* fixed apscheduler start script

* updated celery task queue to handle future tasks and upgraded black and set its line-length to match ruff w/ burnettk

* added support to run future tasks using countdown w/ burnettk

* build image for celery branch w/ burnettk

* poet does not exist in the image w/ burnettk

* start blocking scheduler should always start the scheduler w/ burnettk

* add init and stuff for this branch

* make this work not just on my mac

* send other args to only

* added running status for process instance and use that on fe to go to show page and added additional identifier to locking system to isolate celery workers better w/ burnettk

* fixed typing error that typeguard found, not sure why mypy did not w/ burnettk

* do not check for no instructions on interstitial page for cypress tests on frontend w/ burnettk

* do not queue process instances twice w/ burnettk

* removed bad file w/ burnettk

* queue tasks using strings to avoid circular imports when attmepting to queue w/ burnettk

* only queue imminent new timer events and mock celery

* some keyboard shortcut support on frontend and added ability to force run a process instance over the api w/ burnettk

* some styles added for the shortcut menu w/ burnettk

* pyl w/ burnettk

* fixed test w/ burnettk

* removed temporary celery script and added support for celery worker in run server locally w/ burnettk

* cleaned up migrations w/ burnettk

* created new migration to clean up old migrations

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-12-05 11:41:59 -05:00
Kevin Burnett cfe00a45d4 reset to page 1 when status changes to fix #765 (#769)
* reset to page 1 when status changes to fix #765 w/ jasquat

* upgrade connexion and werkzeug to fix snyk w/ jasquat

* fix all security issues like a boss w/ jasquat

* whoops, still no resolution for cryptography w/ jasquat

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-11-28 08:58:54 -08:00
jbirddog abe865c19b Data Store CRUD interface (#689) 2023-11-24 13:27:46 -05:00
jasquat fc503ec76b feature/secrets-encryption-cleaning (#673)
* removed simple-crypt and cleaned up usage of keys for encryption w/ burnettk

* renamed var to SPIFFWORKFLOW_BACKEND_ENCRYPTION_KEY w/ burnettk

* pyl w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-16 10:12:47 -05:00
jbirddog eb3fd9e2af Bump SpiffWorkflow (#590)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-26 12:27:35 -04:00
jasquat a025aaa017 upgraded cryptography to satisfy snyk and added ignore for werkzeug issue since we cannot do anything about it now w/ burnettk 2023-10-26 11:28:37 -04:00
jasquat fe4dc14b8d Feature/docker CVE issues (#558)
* updated Dockerfile to try to remove security vulnerabilities w/ burnettk

* we require curl for health checks w/ burnettk

* try to scan docker image in ci

* use Dockerfile from backend w/ burnettk

* continue-on-error w/ burnettk

* attempt to elevate permissions of snyk w/ burnettk

* added snyk security github workflow w/ burnettk

* fixed location of constraints w/ burnettk

* add in or true for snyk tests w/ burnettk

* sent the snyk token w/ burnettk

* specify the directory for the sarif file w/ burnettk

* updated spiffworkflow-connector-command for snyk issue w/ burnettk

* updated sql statements sanitize input

* ignore issues for debug_controller and check frontend with snyk w/ burnettk

* updated babel and electron for snyk w/ burnettk

* some more updates to fix vulnerabilities w/ burnettk

* prune repeated deps for frontend builds since

* uncomment ci code so it runs again and use node for frontend base image w/ burnettk

* fixed backend image name w/ burnettk

* pyl w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-19 14:22:52 -04:00
jasquat 2f9aa12906 Feature/error boundary (#552)
* WIP for error boundary support w/ burnettk

* updates to catch error boundary events and send them back out w/ burnettk

* fixed broken test w/ burnettk

* use connector-http and spiffworkflow-proxy from main w/ burnettk

* updated smtp and slack connectors in connector-proxy-demo w/ burnettk

* added more tests for the service task service call connector to test errors w/ burnettk

* added spiffworkflow-connector-command and some code clean up

* updated the connectors in the proxy demo

* use SpiffWorkflow from git instead of locally

* some cleanup while code reviewing

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-18 14:00:12 -04:00
jasquat b6a4d40f74 updated urllib3 for snyk 2023-10-18 11:21:53 -04:00
jasquat 4d842e8dbf Improvement/better serialization (#540)
* using new spiffworkflow locally and the db can be recreated w/ burnettk

* tests are passing w/ burnettk

* added version 3 data migration for typenames on tasks and bpmn processes w/ burnettk

* pyl w/ burnettk

* switch SpiffWorkflow back to main

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-12 14:14:02 -04:00
jasquat 1b06d9c987 updated urllib3 for snyk 2023-10-03 14:42:01 -04:00
jasquat a84750741e updated SpiffWorkflow for depth fix (#519)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-28 12:51:48 -07:00
jasquat 47f94dccbb Feature/nonetype debug stuff (#513)
* some debug items w/ burnettk

* removed some db commits from workflow execution service so the passed in save is the only thing that actually commits the transaction

* set max depth to 50000 for now w/ burnettk

* pyl w/ burnettk

* use temp maxdepth fix in SpiffWorkflow w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-27 17:04:47 -04:00
jasquat b035191964 updated cryptography for snyk w/ burnettk 2023-09-22 10:36:06 -04:00
jasquat 1826cc4b6c Improvement/flexible task iteration (#507)
* updated to use new spiff branch and fixed broken tests w/ burnettk essweine

* updated spiffworkflow with new main build w/ burnettk essweine

* initial updates to new spiff branch w/ burnettk essweine

* more updates for new spiff w/ burnettk essweine

* fixed some linting issues w/ burnettk essweine

* fixed some failing tests w/ burnettk

* updated spiffworkflow for cancel fix w/ burnettk

* Improvement/flexible task iteration 2 (#504)

* wip

* consistent failure, mostly

* removing test code and tests

* removing unused test bpmn files

* removing unused test bpmn files

* minor cleanup of commented code

* spaces and unused imports

* go back to spiff on main

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>

* lint

* updated test to reflect storing predicted tasks w/ burnettk

* add some orders so postgres does not do whatever it wants, and clear log

* fix lint

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: danfunk <daniel.h.funk@gmail.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-09-21 14:47:18 -07:00
jasquat 9312b8cf91 pass a class to the modal to put it back (#483)
* pass a class to the modal to put it back

* update gitpython for safety

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-12 15:39:32 -04:00
burnettk e4d50f850e update spiff to get a couple bug fixes 2023-09-11 10:32:34 -04:00
jasquat 912886f3e6 Improvement/task and spec relations (#477)
* updated to use new spiff branch and fixed broken tests w/ burnettk essweine

* updated spiffworkflow with new main build w/ burnettk essweine

* work around dictionary changed size during iteration error in parallel tests

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-09-08 12:54:43 -04:00
jbirddog 9925105a5e Support lazy loading call activities with human tasks (#450)
* Switch to local wheel

* Fix to load the call activity by spec

* Added log

* Logging

* WIP - getting the right tasks it seems from the db, need to get into the workflow

* Clean up, revert things

* Working thanks dan

* Getting bin_pyl to pass

* run_pyl.

* Re-poet install

* Revert file

* Remove logging lines

* Getting bin_pyl to pass

* Clean up, revert things

* Bump wheel

* Use version off pypi

---------

Co-authored-by: danfunk <daniel.h.funk@gmail.com>
2023-09-07 10:33:38 -04:00
Dan Funk 8c53432ee6 Feature/bug fixes for quick merge on mi ca (#462)
* upgrade to correct multi-instance error bug in BPMN-JS-Spiffworkflow

* update the git service to re-build cache after doing a git pull when a web-hook is called.

* update to SpiffWorkflow - which includes 'replace root with spec start'

* updating the lock file.
2023-08-31 21:10:03 +05:30
jasquat 6094a63084 updated restrictedpython for security 2023-08-31 11:03:17 -04:00
jasquat de67f1ba72 update SpiffWorkflow to remove Root task specs and removed Root specific code from backend w/ burnettk (#444)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-08-22 08:55:09 +05:30
Kevin Burnett 68ac5868b7 bump spiff to change task internal data, add new method and new exceptions (#441)
* bump spiff to change task internal data, add new method and new exceptions

* bump bpmn-js-spiffworkflow for MI updates

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-08-21 22:23:02 +05:30
jbirddog 2b363f4783 Oauth admin (#434)
* Adding dependencies

* Disconnect from /v1/auths for auth list. hardcoded for now.

* Revert changes

* WIP

* Getting hardcoded v2 auths into the frontend

* Better url for v2 oauth

* Pass the auth token from the frontend, don't verify token to start the auth process

* Manually verify the token from the querystring

* WIP

* WIP

* WIP, refactor SPIFF_SECRET handling, move dependencies

* Construct remote_app

* WIP

* WIP

* WIP

* WIP

* Ugly but getting the grant screen

* WIP

* WIP

* Github oauth ok

* Verify token, save access token

* Let secret name work with regex

* Getting bin_pyl to pass

* New component

* Load up the current config in an editor

* Getting bin_pyl to pass

* End point to update auth config

* Linting

* Adding configuration model

* Adding configuration model

* Prep to read config from db

* Read config from the db

* Save/reload poor man's styling

* Getting bin_pyl to pass

* Getting bin_pyl to pass

* Getting bin_pyl to pass

* Better handling of invalid json

* Getting bin_pyl to pass
2023-08-21 21:58:55 +05:30
Kevin Burnett 9313a9f73a Feature/event payloads part 2 (#401)
* Revert "Revert "Feature/event payloads (#393)""

This reverts commit 95fafb7af1.

* Revert "Revert "poet not available in container""

This reverts commit 140220498c.

* Revert "Revert "Run event payloads data migration from background processor (#399)""

This reverts commit 2afced3a51.

* Revert "Revert "using new spiff api to get info about events. w/ elizabeth""

This reverts commit af857fee22.

* Revert "Revert "fix tests for waiting_event_can_be_skipped""

This reverts commit 886e6bd42a.

* push image for preview env

* default scripts to localhost w/ burnettk

* use the bugfix/update-split-task-inputs spiffworkflow branch w/ burnettk

* removed debug json files

* use main for spiffworkflow

* do not attempt to highlight non-diagram boundary items w/ burnettk

* updated SpiffWorkflow to fix multiple signal event issue w/ burnettk

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-08-10 18:24:49 +05:30
jasquat 3f6fb4d9f5 Feature/pin spiff in ci (#428)
* attempt to use the locked version of SpiffWorkflow in ci w/ burnettk

* fixed regex for spiffworkflow revision replacement

* make sure we also update the lockfile when getting new spiff

* install poetry before we attempt to update hte pyproject

* hardcoding spiffworkflow revision as a test w/ burnettk

* try running tests in ci from bash script w/ burnettk

* print working dir in ci w/ burnettk

* fixed location of instance dir w/ burnettk

* run with mysql in ci

* run typeguard with bash script as well w/ burnettk

* fixed postgres test w/ burnettk

* clean up github action file w/ burnettk

* fixed postgres test again w/ burnettk

* pyl

* attempt to remove nox from ci completely

* omit safety for now to test coverage

* fixed how coverage is being called from not nox

* allow running safety and macos again

* renamed run_not_nox to run_ci_session w/ burnettk

* attempt to only upload if matrix says to w/ burnettk

* attempt to install mysqlclient prereqs for mac and remove noxfile stuff w/ burnettk

* added back the constraints file w/ burnettk

* moved the contributing file to the root of arena w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-08-03 11:49:30 -04:00
jasquat 9c1eedee91 Merge branch 'main' of github.com:sartography/spiff-arena 2023-08-02 11:05:59 -04:00
jasquat 9fa6182bfb updated python packages for ci w/ burnettk 2023-08-02 11:05:55 -04:00
burnettk f971a298f1 bump cryptography 2023-08-02 10:34:21 -04:00
jasquat 299f4a6823 return the process instance when interstitial process is complete and… (#421)
* return the process instance when interstitial process is complete and favor redirecting to it on the frontend

* upgrade certifi for snyk check

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-07-28 20:34:39 +05:30
jasquat 84369a1b20 return the process instance early from the interstitial if it is susp… (#407)
* return the process instance early from the interstitial if it is suspended or terminated

* added a test to make sure the interstitial page returns the process instance if suspended or termianted w/ burnettk

* randomize tests and cleaned up the pyproject file a little bit w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-07-25 22:57:53 +05:30
burnettk 95fafb7af1 Revert "Feature/event payloads (#393)"
This reverts commit f1c5f1ea21.
2023-07-19 11:44:06 -04:00
burnettk 2afced3a51 Revert "Run event payloads data migration from background processor (#399)"
This reverts commit ab5926aab1.
2023-07-19 11:43:42 -04:00
burnettk 4638adbaef update cryptography 2023-07-18 17:12:25 -04:00
Kevin Burnett ab5926aab1 Run event payloads data migration from background processor (#399)
* move data migration code out of bin so it can be reused in background processor

* sleep for 5 minutes and update bpmn js to pull in some fixes from elizabeth

* update spiff to pull in parser update to make it act like before

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-07-18 10:50:07 -07:00
jasquat f1c5f1ea21 Feature/event payloads (#393)
* some initial changes for event payload changes in spiff

* fixed tests for new spiffworkflow with event payloads w/ burnettk essweine

* pyl w/ burnettk essweine

* updated SpiffWorkflow from branch

* switched SpiffWorkflow back to main w/ burnettk

* added base for migration script to upgrade db w/ burnettk essweine

* some updates to script w/ burnettk

* script has been written, needs to be tested

* pyl w/ burnettk

* updates to migration script so it can work w/ burnettk

* pyl w/ burnettk

* added comment to data migration file

* run the version 1 3 migration on app boot w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-07-18 06:56:24 -07:00
jbirddog 1134e39dd1 Bump SpiffWorkflow to fix call activity references (#383) 2023-07-12 19:48:08 +05:30
jasquat c936c02db0 use mysqlclient instead of mysql-connect w/ burnettk (#358)
* use mysqlclient instead of mysql-connect w/ burnettk

* updated some comments w/ burnettk

* runs tests in parallel in run_pyl

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-07-12 19:44:49 +05:30
burnettk b7aa82bd08 fix security issue 2023-07-11 15:57:11 -04:00
jasquat 7c3c3af057 also check human task table for completed by user when determining if… (#370)
* also check human task table for completed by user when determining if an instance is associated with a user

* update pygments

* added some comments for clarity w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-07-07 20:22:57 +05:30
Kevin Burnett b13b716424 parallel tests with xdist (#335)
* parallel tests with xdist

* add pytest-xdist as dev dep

* put back spiff

* update messaging

* get more in line with main

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-26 22:37:41 +05:30
jasquat 1f848f6992 updated spiffworkflow to fix terminate instance error (#344)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-22 09:16:55 -04:00
Kevin Burnett 7de0d97303 Feature/better subworkflow management part 2 (#341)
* Revert "Revert "Feature/better subworkflow management (#331)""

This reverts commit 48dcde8faf.

* updated SpiffWorkflow to fix infinite loop in task trace w/ burnettk

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-20 10:37:49 -04:00
burnettk 48dcde8faf Revert "Feature/better subworkflow management (#331)"
This reverts commit 23af5cbbe4.
2023-06-15 15:10:48 -04:00
jasquat 23af5cbbe4 Feature/better subworkflow management (#331)
* WIP: some updates to support new spiff w/ burnettk

* unit tests are passing

* all tests except message tests are passing

* fixed usage of catch message event w/ burnettk

* messages are working again w/ burnettk

* uncommented remaining message tests w/ burnettk

* fixed cypress tests w/ burnettk

* use main for spiffworkflow

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-15 10:31:42 -04:00