Commit Graph

186 Commits

Author SHA1 Message Date
Kevin Burnett ab5926aab1 Run event payloads data migration from background processor (#399)
* move data migration code out of bin so it can be reused in background processor

* sleep for 5 minutes and update bpmn js to pull in some fixes from elizabeth

* update spiff to pull in parser update to make it act like before

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-07-18 10:50:07 -07:00
jasquat f1c5f1ea21 Feature/event payloads (#393)
* some initial changes for event payload changes in spiff

* fixed tests for new spiffworkflow with event payloads w/ burnettk essweine

* pyl w/ burnettk essweine

* updated SpiffWorkflow from branch

* switched SpiffWorkflow back to main w/ burnettk

* added base for migration script to upgrade db w/ burnettk essweine

* some updates to script w/ burnettk

* script has been written, needs to be tested

* pyl w/ burnettk

* updates to migration script so it can work w/ burnettk

* pyl w/ burnettk

* added comment to data migration file

* run the version 1 3 migration on app boot w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-07-18 06:56:24 -07:00
jbirddog 1134e39dd1 Bump SpiffWorkflow to fix call activity references (#383) 2023-07-12 19:48:08 +05:30
jasquat c936c02db0 use mysqlclient instead of mysql-connect w/ burnettk (#358)
* use mysqlclient instead of mysql-connect w/ burnettk

* updated some comments w/ burnettk

* runs tests in parallel in run_pyl

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-07-12 19:44:49 +05:30
burnettk b7aa82bd08 fix security issue 2023-07-11 15:57:11 -04:00
jasquat 7c3c3af057 also check human task table for completed by user when determining if… (#370)
* also check human task table for completed by user when determining if an instance is associated with a user

* update pygments

* added some comments for clarity w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-07-07 20:22:57 +05:30
Kevin Burnett b13b716424 parallel tests with xdist (#335)
* parallel tests with xdist

* add pytest-xdist as dev dep

* put back spiff

* update messaging

* get more in line with main

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-26 22:37:41 +05:30
jasquat 1f848f6992 updated spiffworkflow to fix terminate instance error (#344)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-22 09:16:55 -04:00
Kevin Burnett 7de0d97303 Feature/better subworkflow management part 2 (#341)
* Revert "Revert "Feature/better subworkflow management (#331)""

This reverts commit 48dcde8faf.

* updated SpiffWorkflow to fix infinite loop in task trace w/ burnettk

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-20 10:37:49 -04:00
burnettk 48dcde8faf Revert "Feature/better subworkflow management (#331)"
This reverts commit 23af5cbbe4.
2023-06-15 15:10:48 -04:00
jasquat 23af5cbbe4 Feature/better subworkflow management (#331)
* WIP: some updates to support new spiff w/ burnettk

* unit tests are passing

* all tests except message tests are passing

* fixed usage of catch message event w/ burnettk

* messages are working again w/ burnettk

* uncommented remaining message tests w/ burnettk

* fixed cypress tests w/ burnettk

* use main for spiffworkflow

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-15 10:31:42 -04:00
jasquat 90a2a0ca0b look up human tasks with the guid instead of the task spec identifier w/ burnettk (#317)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-08 14:26:48 -07:00
jbirddog 122d1efbda Bump flask for safety (#304)
* Bump flask for safety

* let snyk check flask again w/ burnettk

* attempt to use the same revision for front w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-06 17:33:48 -07:00
jasquat 5dfd96ca9e upgrade cryptography for security w/ burnettk 2023-06-02 12:09:24 -04:00
danfunk 7743b6c87f Documentation Updates
- updated conf.py with some new settings
- updated the documentation
- provided an example of section heading
- added a few additional requirements
2023-06-02 08:25:39 -04:00
burnettk be9859752e drop python 3.9 and remove unused libs 2023-05-27 18:17:51 -04:00
burnettk 2b2875cb6f let ruff sort imports and ditch duplicative pre-commit linters 2023-05-26 20:01:08 -04:00
jasquat a58a9532fc ensure we delete related items when deleting a process instance w/ burnettk 2023-05-26 10:51:24 -04:00
burnettk ecca0453f4 bump requests to mitigate security issue 2023-05-23 07:19:09 -04:00
burnettk 8d20ef6956 remove sphinx from pyproject since we do not use it in this way 2023-05-18 18:42:40 -04:00
danfunk 93eb5131cd update spiffworkflow 2023-05-15 16:55:52 -04:00
jasquat 3144862ebc updated spiffworkflow to for raw strings in regexes w/ burnettk 2023-05-12 10:09:46 -04:00
jasquat 1066ae4b8a updated spiffworkflow to fix typo w/ burnettk 2023-05-12 09:55:10 -04:00
jasquat bb50c8a285 updated spiffworkflow to use newest main 2023-05-11 16:20:25 -04:00
jasquat 233d8a9832 using the new spiff branch and tests are now passing w/ burnettk 2023-05-11 12:41:52 -04:00
jasquat de8a718a30 removed flask-bpmn from pyproject since it is no longer used w/ burnettk 2023-05-10 17:09:51 -04:00
jasquat ca27217a25 updated spiffworkflow w/ burnettk 2023-05-10 16:43:08 -04:00
jasquat da52b68c84 updated Spiffworkflow and updated terminated test to ensure tasks are deleted from the db w/ burnettk 2023-05-10 16:39:03 -04:00
jbirddog b89527ab80 Integrate spiff-element-units 0.3.0 for (some) lazy call activities (#239) 2023-05-04 14:15:13 -04:00
jasquat 418ec7d9c9 updated typeguard and fixed issues w/ burnettk 2023-05-01 15:26:29 -04:00
jbirddog 5f8ff4b5aa Bump spiffworkflow (#235) 2023-05-01 11:24:04 -04:00
Dan 4cbf4e456b a whole bunch of front end and backend changes that got mangled. 2023-04-26 16:22:50 -04:00
jasquat 2dc3b0a76e updated SpiffWorkflow w/ burnettk 2023-04-20 14:49:34 -04:00
jasquat 8cc2d56dbd merged in main and resolved conflicts w/ burnettk 2023-04-20 14:10:23 -04:00
jbirddog ff0202f4d4 Called elements (#218) 2023-04-19 18:07:15 -04:00
jasquat 9254d19b74 display event errors in the frontend using errorDisplay w/ burnettk 2023-04-19 15:20:19 -04:00
jasquat 6e931470d2 updated SpiffWorkflow w/ burnettk 2023-04-18 15:24:29 -04:00
jbirddog 1dd445de68 spiff-element-units integration (#214) 2023-04-17 14:49:38 -04:00
burnettk 6fc1e2be96 bump spiff 2023-04-13 15:31:40 -04:00
jasquat 79a0505ddb Merge remote-tracking branch 'origin/main' into feature/prometheus-metrics 2023-04-11 13:23:06 -04:00
jasquat 14c8f38287 updated spiffworkflow 2023-04-10 13:06:14 -04:00
jasquat 5c2f0ef9de using spiffworkflow run-boundary-events-from-engine-steps branch and tests pass w/ burnettk 2023-04-10 12:22:33 -04:00
jasquat 9b3206f66c pyl w/ burnettk 2023-04-07 16:13:01 -04:00
jasquat 106b03d32b merged in main and resolved conflicts w/ burnettk 2023-04-07 14:59:49 -04:00
jasquat f6bcf130e7 updated spiffworkflow to send events to top level workflow w/ burnettk 2023-04-07 14:53:05 -04:00
jasquat e12db76038 some cleanup, updated SpiffWorkflow, and added some notes to pick back up where we left off with the more efficient method w/ burnettk 2023-04-06 10:57:52 -04:00
jasquat c5b85fd404 WIP - test is still passing, no longer processing all tasks w/ burnettk 2023-04-05 16:07:35 -04:00
jasquat 0bbe52714f updated to use spiff main 2023-04-03 09:46:00 -04:00
jasquat 343aae0628 rewind test passed w/ burnettk 2023-03-31 15:14:25 -04:00
jasquat a7521cba65 run snyk on backend and added marshmallow dep to remove warning w/ burnettk 2023-03-28 16:55:13 -04:00
jasquat 98a935e222 use session delete branch of sqlalchemy stubs w/ burnettk 2023-03-28 16:28:41 -04:00
jasquat a878790393 add back in sqlalchemy stubs w/ burnettk 2023-03-28 16:05:43 -04:00
jasquat e7fef5b6c4 updated sqlalchemy to 2.0 due to safety complaints w/ burnettk 2023-03-28 15:56:00 -04:00
burnettk 16b837daca downgrade setuptools closer to where it was to avoid deprecation warnings 2023-03-28 14:58:16 -04:00
burnettk 3371148d4a fix four of five safety issues 2023-03-28 14:51:08 -04:00
burnettk fa5ded0f26 get backend working with new spiff 2023-03-28 10:33:53 -04:00
burnettk 4a092a8e2a tests have been broken in ci, and this should fix it 2023-03-27 17:02:29 -04:00
burnettk 152be789ec another take on metrics 2023-03-22 22:40:56 -04:00
jasquat e8e4eab44b use main branch of spiffworkflow w/ burnettk 2023-03-15 16:15:15 -04:00
burnettk c591dbfefe no op cipher for testing 2023-03-12 18:20:29 -04:00
jasquat 2ef0bf0065 added loop to task test bpmn file w/ burnettk 2023-03-10 17:15:27 -05:00
jbirddog 8f1975d7e2 Bump SpiffWorkflow (#171) 2023-03-10 10:21:07 -05:00
burnettk d13001efff bump sentry version 2023-03-07 22:56:48 -05:00
Jon Herron f994c2f5d1 Poetry install 2023-03-06 15:43:46 -05:00
jbirddog 7c0b7d252e Couple dependency changes (#165) 2023-03-06 10:12:54 -05:00
jasquat 9bc99e2ef7 updates SpiffWorkflow w/ burnettk 2023-03-01 17:25:03 -05:00
Dan afec26fbd5 Bumping spiffworkflow lib changes for messages 2023-03-01 16:40:56 -05:00
jasquat d5acd5a16d added new table and some notes on how to get a delta w/ burnettk jbirddog 2023-02-28 16:30:52 -05:00
Dan 43e41fe848 Point to the latest spiffworkflow 2023-02-27 14:40:47 -05:00
Dan f16150b02b Merging main 2023-02-27 14:17:10 -05:00
Jon Herron a766e7d9a6 poetry remove orjson 2023-02-27 09:35:51 -05:00
Dan 0d468a825a run_pyl (part 2) 2023-02-23 15:09:22 -05:00
Dan 0f3ef00d72 BPMN.io -- Just show the message names not the ids - to assure we are only exposing the names.
SpiffWorkflow -
    - start_messages function should return message names, not ids.
    - don't catch external thrown messages within the same workflow process
    - add an expected value to the Correlation Property Model so we can use this well defined class as an external communication tool (rather than building an arbitrary dictionary)
    - Added a "get_awaiting_correlations" to an event, so we can get a list of the correlation properties related to the workflows currently defined correlation values.
    - workflows.waiting_events() function now returns the above awaiting correlations as the value on returned message events
 Backend
    - Dropping MessageModel and MessageCorrelationProperties - at least for now.  We don't need them to send / receive messages though we may eventually want to track the messages and correlations defined across the system - these things (which are ever changing) should not be directly connected to the Messages which may be in flux - and the cross relationships between the tables could cause unexpected and unceissary errors.  Commented out the caching logic so we can turn this back on later.
    - Slight improvement to API Errors
    - MessageInstances are no longer in a many-to-many relationship with Correlations - Each message instance has a unique set of message correlations specific to the instance.
    - Message Instances have users, and can be linked through a "counterpart_id" so you can see what send is connected to what recieve.
    - Message Correlations are connected to  recieving message instances.  It is not to a process instance, and not to a message model.  They now include the expected value and retrieval expression required to validate an incoming message.
    - A process instance is not connected to message correlations.
    - Message Instances are not always tied to a process instance (for example, a Send Message from an API)
    - API calls to create a message use the same logic as all other message catching code.
    - Make use of the new waiting_events() method to check for any new recieve messages in the workflow (much easier than
    churning through all of the tasks)
    - One giant mother of a migration.
2023-02-23 13:53:03 -05:00
burnettk 55f3b765ee update spiff 2023-02-23 11:02:40 -05:00
Dan f2ca47e798 bump spiffworkflow hash 2023-02-17 13:08:45 -05:00
burnettk 01668368e7 all tests pass w/ spiff lib upgrade, thanks for the help elizabeth 2023-02-15 22:56:12 -05:00
Dan 76bd40cb09 Update Spiffworkflow so we have the better error messages on service tasks. 2023-02-12 16:19:40 -05:00
burnettk 4934014ba9 simplify spiff integration post serializer update, w/ elizabeth and jon 2023-02-02 15:40:01 -05:00
jbirddog a815863727 Allow for different Python Environments when executing scripts within SpiffWorkflow (#121) 2023-02-02 10:24:55 -05:00
jasquat 54f795da7b some initial updates w/ burnettk 2023-01-27 12:51:19 -05:00
jasquat 9fb2b01e3c added pylint back to lock file 2023-01-25 17:35:22 -05:00
jasquat cc85146678 upgraded bandit and flake8 w/ burnettk 2023-01-25 15:44:53 -05:00
jasquat 2d57c3d45f strip off newlines and spaces when importing secrets and pinning spiffworkflow to working version until we can debug issues with new one w/ burnettk 2023-01-23 17:18:47 -05:00
Dan cf1fcf9966 updating spiffworkflow version, and fixing the run_pyl (by removing the corft I stuck in earlier) 2023-01-19 13:55:35 -05:00
jasquat dd8427bd32 pin SpiffWorkflow to passing version for CI tests w/ burnettk 2023-01-19 11:10:36 -05:00
jasquat b5be1745d8 Merge pull request #105 from sartography/feature/avoid_logs_when_call_activity_waiting
do not write to logs when a task is inheriting data from the parent w…
2023-01-19 11:08:51 -05:00
burnettk edf11f76cf downgrade spiff and upgrade some other stuff 2023-01-19 07:59:26 -05:00
burnettk cd07be3e28 upgrade certifi to fix security vulnerability 2023-01-19 07:38:57 -05:00
jasquat 7194864e6d do not write to logs when a task is inheriting data from the parent w/ burnettk 2023-01-17 12:23:51 -05:00
jasquat 6b5087c128 debugging cookies w/ burnettk 2023-01-11 14:52:12 -05:00
jasquat c18eafa812 fixed typeguard tests w/ burnettk 2023-01-09 15:53:13 -05:00
jasquat 924856a547 updated SpiffWorkflow w/ burnettk 2023-01-05 17:35:59 -05:00
burnettk d1e911950d Merge remote-tracking branch 'origin/main' into feature/process_api_blueprint_refactor 2022-12-29 20:41:50 -05:00
Dan d4c1055077 Minor fix to SpiffWorkflow library 2022-12-29 13:51:23 -05:00
jasquat 79e21882d4 update flask-bpmn w/ burnettk 2022-12-27 16:16:53 -05:00
jasquat d9d6cb4006 update flask-bpmn w/ burnettk 2022-12-27 13:55:02 -05:00
jasquat 3a9f86c25b updated flask-bpmn so it does not send invalid token exceptions to sentry w/ burnettk 2022-12-16 13:24:52 -05:00
burnettk f920edbb56 bump nox stuff and spiff 2022-12-05 09:08:56 -05:00
burnettk a8357c73f4 Merge remote-tracking branch 'origin/main' into new_report 2022-12-02 16:29:16 -05:00
Dan 9e3c504384 getting the collect hit_policy to work correctly. 2022-12-02 15:32:33 -05:00