Commit Graph

1949 Commits

Author SHA1 Message Date
burnettk 350daf0b3d allow task_name to be long 2023-02-27 22:01:08 -05:00
burnettk a1ec494ab3 let env var work on windows and fixing typing issue on python 3.9 w/ messaging stuff 2023-02-27 19:16:06 -05:00
jasquat 45feb38168 recreated all migrations to resolve postgres migration failures w/ burnettk 2023-02-27 17:27:53 -05:00
jasquat 400f3ced73 Merge pull request #160 from sartography/feature/fix_sqlite_ci
Feature/fix sqlite ci
2023-02-27 17:26:26 -05:00
jasquat b2782f8975 run poetry install for sqlite w/ burnettk 2023-02-27 17:21:34 -05:00
jasquat 5a873a9e2d do not run mysql commands if not doing mysql w/ burnettk 2023-02-27 17:19:03 -05:00
jasquat 338bc6a8ad do not override env vars w/ burnettk 2023-02-27 17:15:21 -05:00
jasquat dca7815045 syntax error w/ burnettk 2023-02-27 17:10:52 -05:00
jasquat edfbc77c30 attempt to set the sample process model dir w/ burnettk 2023-02-27 17:10:14 -05:00
jasquat c428158de6 attempt to run migrations from scratch for sqlite in ci since it does not support the migrations w/ burnettk 2023-02-27 17:00:34 -05:00
jasquat 35c93e3f7e do not return human tasks from errored process instances w/ burnettk 2023-02-27 16:11:26 -05:00
jasquat 09a46712d4 added debug logging when adding spiff step details w/ burnettk 2023-02-27 15:53:56 -05:00
burnettk 3add069267 Squashed 'bpmn-js-spiffworkflow/' changes from f1f008e3e..a547888ef
a547888ef run_pyl
53378437d BPMN.io -- Just show the message names not the ids - to assure we are only exposing the names. SpiffWorkflow -     - start_messages function should return message names, not ids.     - don't catch external thrown messages within the same workflow process     - add an expected value to the Correlation Property Model so we can use this well defined class as an external communication tool (rather than building an arbitrary dictionary)     - Added a "get_awaiting_correlations" to an event, so we can get a list of the correlation properties related to the workflows currently defined correlation values.     - workflows.waiting_events() function now returns the above awaiting correlations as the value on returned message events  Backend     - Dropping MessageModel and MessageCorrelationProperties - at least for now.  We don't need them to send / receive messages though we may eventually want to track the messages and correlations defined across the system - these things (which are ever changing) should not be directly connected to the Messages which may be in flux - and the cross relationships between the tables could cause unexpected and unceissary errors.  Commented out the caching logic so we can turn this back on later.     - Slight improvement to API Errors     - MessageInstances are no longer in a many-to-many relationship with Correlations - Each message instance has a unique set of message correlations specific to the instance.     - Message Instances have users, and can be linked through a "counterpart_id" so you can see what send is connected to what recieve.     - Message Correlations are connected to  recieving message instances.  It is not to a process instance, and not to a message model.  They now include the expected value and retrieval expression required to validate an incoming message.     - A process instance is not connected to message correlations.     - Message Instances are not always tied to a process instance (for example, a Send Message from an API)     - API calls to create a message use the same logic as all other message catching code.     - Make use of the new waiting_events() method to check for any new recieve messages in the workflow (much easier than     churning through all of the tasks)     - One giant mother of a migration.

git-subtree-dir: bpmn-js-spiffworkflow
git-subtree-split: a547888ef102046bab983b9004f07d4398e9a1bc
2023-02-27 14:59:39 -05:00
burnettk 9d81078ef6 Merge commit '3add06926765126577a100845c8d3247323e5f4c' 2023-02-27 14:59:39 -05:00
burnettk 10febdb14a Merge commit '7a5f961b2bbff62f525805d6302822a411024f7b' 2023-02-27 14:59:38 -05:00
burnettk 7a5f961b2b Squashed 'spiffworkflow-frontend/' changes from c22d22ba5..4fd946be4
4fd946be4 Merging main
453bdb246 run_pyl
b5a8ff01b Needed an additional check for empty correlation keys - which on a RECEIVE message, should always match anything.
b583f84cc lint w/ burnettk
fd4d7d13b removed some unused code from task and fixed the logs table a bit w/ burnettk
6f17e71e6 avoid using task-data endpoint for task data and only use it to get tasks based on spiff step instead
669c29595 removed task-data endpoints since we no longer need them w/ burnettk
5f25fffe0 added api to get task data and do not return from task data list anymore w/ burnettk
373c4f184 Merge remote-tracking branch 'origin/main' into feature/message_fixes
e40c12ac8 turn on sentry detailed tracing for task-data w/ burnettk
d7861aae2 Merge branch 'main' into feature/message_fixes
0a9f2480d work in progress - * Link between message instance and correlations is now a link table and many-to-many relationships as recommended by SQLAlchemy * Use the correlation keys, not the process id when accepting api messages.

git-subtree-dir: spiffworkflow-frontend
git-subtree-split: 4fd946be4ff716bfd1664aca8d75cb6f709b53eb
2023-02-27 14:59:38 -05:00
burnettk 8b109c548e Merge commit 'c4142702d403668629c418208bf505d86f8ac24e' 2023-02-27 14:59:37 -05:00
burnettk c4142702d4 Squashed 'spiffworkflow-backend/' changes from 6cae736ac..0bfe4400f
0bfe4400f Merge branch 'main' of github.com:sartography/spiff-arena into main
fbe97dd90 Point to the latest spiffworkflow
805400312 fixed conflict with db migrations w/ burnettk
6e48c8205 Merge remote-tracking branch 'origin/main' into feature/script_get_last_user_completing_task
3a6cc9c2b added script to get process initiator w/ burnettk
37770dbca run_pyl
8a068cdb2 Merging main
bf25c1cef run_pyl
4ca6e3f27 Needed an additional check for empty correlation keys - which on a RECEIVE message, should always match anything.
0fb88c50d remove unwanted test files w/ burnettk
89d92687d script to get last user completing a task is working w/ burnettk
98f056e5c Merge remote-tracking branch 'origin/main' into feature/script_get_last_user_completing_task
46af55a60 poetry remove orjson
28a0db097 wip for get_last_user_completing_task script task
2596cfeb1 postgres really will just order however it wants if you do not specify an order_by clause
5e339b2fb add ppg.ba4.sme and ba5
873cfdfc1 fix postgres db name and comment out debug job
f54d6ae4d removed some unused code from task and fixed the logs table a bit w/ burnettk
7c74e216a run_pyl
f53c85924 skip failing test if postgres and added comment about cause w/ burnettk
fbe2237f1 # SpiffWorkflow: 1) Type Safe checking on correlation properties (no more str()) 2) A running workflows Correlations are once again at the key level.
2677736c2 look users up by service and username instead of service_id since usernames have to be unique anyway w/ burnettk
22d1f8bbb avoid using task-data endpoint for task data and only use it to get tasks based on spiff step instead
698fb5d5c put back the task data code when getting tasks
b52f97453 Merge remote-tracking branch 'origin/main' into feature/task_data_api_refactor
9c4a17c93 removed commented out code w/ burnettk
3357fbef4 removed task-data endpoints since we no longer need them w/ burnettk
3b64afb65 turn off profiling again
fcbcfd0ae add two users and update one
2ee7cba09 BPMN Parser was returning all retrieval expressions, rather than the ones specific to a correlation property, as was intended. Adding a correlation cache - so we have a reference of all the messages and properties (though still lacking a description of keys) Adding yet another migration, maybe should squash em.
6658e26bb added api to get task data and do not return from task data list anymore w/ burnettk
1fd3261d7 run_pyl (part 2)
f6c63eb1c Merge branch 'main' of github.com:sartography/spiff-arena
5c5262a31 added comment about refactoring getting task data w/ burnettk jbirddog
6d4aa9043 lint
e4c0ed7e1 add test users
adfb0644f Adding Migration.
ec36290f2 remove task size check since it can take a long time to run and we do not do anything with it w/ burnettk jbirddog
9b0b95f95 Merge remote-tracking branch 'origin/main' into feature/message_fixes
f3d124a70 run_pyl
be6ac8743 BPMN.io -- Just show the message names not the ids - to assure we are only exposing the names. SpiffWorkflow -     - start_messages function should return message names, not ids.     - don't catch external thrown messages within the same workflow process     - add an expected value to the Correlation Property Model so we can use this well defined class as an external communication tool (rather than building an arbitrary dictionary)     - Added a "get_awaiting_correlations" to an event, so we can get a list of the correlation properties related to the workflows currently defined correlation values.     - workflows.waiting_events() function now returns the above awaiting correlations as the value on returned message events  Backend     - Dropping MessageModel and MessageCorrelationProperties - at least for now.  We don't need them to send / receive messages though we may eventually want to track the messages and correlations defined across the system - these things (which are ever changing) should not be directly connected to the Messages which may be in flux - and the cross relationships between the tables could cause unexpected and unceissary errors.  Commented out the caching logic so we can turn this back on later.     - Slight improvement to API Errors     - MessageInstances are no longer in a many-to-many relationship with Correlations - Each message instance has a unique set of message correlations specific to the instance.     - Message Instances have users, and can be linked through a "counterpart_id" so you can see what send is connected to what recieve.     - Message Correlations are connected to  recieving message instances.  It is not to a process instance, and not to a message model.  They now include the expected value and retrieval expression required to validate an incoming message.     - A process instance is not connected to message correlations.     - Message Instances are not always tied to a process instance (for example, a Send Message from an API)     - API calls to create a message use the same logic as all other message catching code.     - Make use of the new waiting_events() method to check for any new recieve messages in the workflow (much easier than     churning through all of the tasks)     - One giant mother of a migration.
ce449971a do not call serialize if we can use the cached bpmn_json instead w/ burnettk
15d720f94 Merge branch 'main' of github.com:sartography/spiff-arena
8d8347068 turn on sentry detailed tracing for task-data w/ burnettk
cdf5f4053 update spiff
5c1ea3c93 run_pyl
384c272af * SpiffWorkflow event_definitions wanted to return a message event's correlation properties mested within correlation keys.  But messages are directly related to properties, not to keys - and it forced a number of conversions that made for tricky code.  So Messages now contain a dictionary of correlation properties only. * SpiffWorkflow did not serialize correlations - so they were lost between save and retrieve.
f45103406 Allow people to run commands like "flask db upgrade" without setting specific environment variables like FLASK_SESSION_SECRET_KEY everytime - they just need to add in their own /instance/config.py with their local configuration.
b169c3a87 * Re-work message tests so I could wrap my simple head around what was happening - just needed an example that made sense to me. * Clear out complex get_message_instance_receive how that many-to-many works. * Create decent error messages when correlations fail * Move correlation checks into the MessageInstance class * The APIError could bomb out ugly if it hit a workflow exception with not Task Spec.
a39590912 failing test.
6db600caa Merge branch 'main' into feature/message_fixes
4942a728b work in progress - * Link between message instance and correlations is now a link table and many-to-many relationships as recommended by SQLAlchemy * Use the correlation keys, not the process id when accepting api messages.

git-subtree-dir: spiffworkflow-backend
git-subtree-split: 0bfe4400f4191214b8972977438ceb35a9f5b3c3
2023-02-27 14:59:37 -05:00
burnettk 89196daed5 Squashed 'SpiffWorkflow/' changes from 7b39b2235..b3235fad5
b3235fad5 Merging main
09623ca61 # SpiffWorkflow: 1) Type Safe checking on correlation properties (no more str()) 2) A running workflows Correlations are once again at the key level.
d6806f69d maintain a way to access the correlations in relation to the correlation keys
065a86cde BPMN Parser was returning all retrieval expressions, rather than the ones specific to a correlation property, as was intended. Adding a correlation cache - so we have a reference of all the messages and properties (though still lacking a description of keys) Adding yet another migration, maybe should squash em.
9e8832c93 Merge remote-tracking branch 'origin/main' into feature/message_fixes
8efa922ae run_pyl
72a7e535a BPMN.io -- Just show the message names not the ids - to assure we are only exposing the names. SpiffWorkflow -     - start_messages function should return message names, not ids.     - don't catch external thrown messages within the same workflow process     - add an expected value to the Correlation Property Model so we can use this well defined class as an external communication tool (rather than building an arbitrary dictionary)     - Added a "get_awaiting_correlations" to an event, so we can get a list of the correlation properties related to the workflows currently defined correlation values.     - workflows.waiting_events() function now returns the above awaiting correlations as the value on returned message events  Backend     - Dropping MessageModel and MessageCorrelationProperties - at least for now.  We don't need them to send / receive messages though we may eventually want to track the messages and correlations defined across the system - these things (which are ever changing) should not be directly connected to the Messages which may be in flux - and the cross relationships between the tables could cause unexpected and unceissary errors.  Commented out the caching logic so we can turn this back on later.     - Slight improvement to API Errors     - MessageInstances are no longer in a many-to-many relationship with Correlations - Each message instance has a unique set of message correlations specific to the instance.     - Message Instances have users, and can be linked through a "counterpart_id" so you can see what send is connected to what recieve.     - Message Correlations are connected to  recieving message instances.  It is not to a process instance, and not to a message model.  They now include the expected value and retrieval expression required to validate an incoming message.     - A process instance is not connected to message correlations.     - Message Instances are not always tied to a process instance (for example, a Send Message from an API)     - API calls to create a message use the same logic as all other message catching code.     - Make use of the new waiting_events() method to check for any new recieve messages in the workflow (much easier than     churning through all of the tasks)     - One giant mother of a migration.
cb2ff8a93 * SpiffWorkflow event_definitions wanted to return a message event's correlation properties mested within correlation keys.  But messages are directly related to properties, not to keys - and it forced a number of conversions that made for tricky code.  So Messages now contain a dictionary of correlation properties only. * SpiffWorkflow did not serialize correlations - so they were lost between save and retrieve.
d4852a1a5 * Re-work message tests so I could wrap my simple head around what was happening - just needed an example that made sense to me. * Clear out complex get_message_instance_receive how that many-to-many works. * Create decent error messages when correlations fail * Move correlation checks into the MessageInstance class * The APIError could bomb out ugly if it hit a workflow exception with not Task Spec.

git-subtree-dir: SpiffWorkflow
git-subtree-split: b3235fad598ee3c4680a23f26adb09cdc8f2807b
2023-02-27 14:59:36 -05:00
burnettk cbe181f693 Merge commit '89196daed573f0244ce32ad4f86e5fe7bd876d64' 2023-02-27 14:59:36 -05:00
Jakub Sokołowski 3b7857739a ci: add basic jenkinsfile for building Docker images
This is the first and most basic version of a Jenkinsfile we can use.

Since we can't just hardcode things like docker name, tag, or
credentials for pushing to a Docker registry it needs to be a parameter.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-27 20:46:15 +01:00
Dan 24839a34dd Merge branch 'main' of github.com:sartography/spiff-arena into main 2023-02-27 14:40:52 -05:00
Dan 43e41fe848 Point to the latest spiffworkflow 2023-02-27 14:40:47 -05:00
jasquat c0af69b11a Merge pull request #159 from sartography/feature/script_get_last_user_completing_task
Feature/script get last user completing task
2023-02-27 14:35:11 -05:00
jasquat 7c12709fdd fixed conflict with db migrations w/ burnettk 2023-02-27 14:33:55 -05:00
jasquat dfac872606 Merge remote-tracking branch 'origin/main' into feature/script_get_last_user_completing_task 2023-02-27 14:28:23 -05:00
jasquat 29ada38904 added script to get process initiator w/ burnettk 2023-02-27 14:28:19 -05:00
Dan e00fbdb490 run_pyl 2023-02-27 14:21:22 -05:00
Dan f16150b02b Merging main 2023-02-27 14:17:10 -05:00
Dan 8bebd09880 Squashed 'SpiffWorkflow/' changes from 2ca6ebf80..7b39b2235
7b39b2235 Merge pull request #300 from sartography/bugfix/remove-minidom-dependency
0642d48b1 remove minidom

git-subtree-dir: SpiffWorkflow
git-subtree-split: 7b39b223562eb510dd68c8d451922721ebb721a7
2023-02-27 14:06:23 -05:00
Dan 3e6ad50156 Merge commit '8bebd0988057d61015675e6ae931069d0a7d2b40' into main 2023-02-27 14:06:23 -05:00
Dan 150b95070c run_pyl 2023-02-27 14:05:00 -05:00
Dan e060e28c28 Needed an additional check for empty correlation keys - which on a RECEIVE message, should always match anything.
When finding messages to match a send, assure they are RECIEVE messages.

Fix some of the json output for messages sent to the front end
2023-02-27 14:01:21 -05:00
jasquat 6a4dc0f14e remove unwanted test files w/ burnettk 2023-02-27 12:08:40 -05:00
jasquat 9f84564457 script to get last user completing a task is working w/ burnettk 2023-02-27 12:08:07 -05:00
jasquat e07a122674 Merge remote-tracking branch 'origin/main' into feature/script_get_last_user_completing_task 2023-02-27 10:45:04 -05:00
Kevin Burnett a5e5ad56ad Merge pull request #157 from sartography/orjson
poetry remove orjson
2023-02-27 06:47:19 -08:00
Jon Herron a766e7d9a6 poetry remove orjson 2023-02-27 09:35:51 -05:00
burnettk e9b3ababc1 wip for get_last_user_completing_task script task 2023-02-25 23:31:23 -05:00
burnettk f7dc076f75 postgres really will just order however it wants if you do not specify an order_by clause 2023-02-25 22:35:46 -05:00
burnettk 647d5ee92a add ppg.ba4.sme and ba5 2023-02-24 23:42:19 -05:00
burnettk 5eb9da00f1 fix postgres db name and comment out debug job 2023-02-24 16:57:53 -05:00
jasquat ff1df1145b lint w/ burnettk 2023-02-24 16:29:02 -05:00
jasquat b71419616d removed some unused code from task and fixed the logs table a bit w/ burnettk 2023-02-24 16:18:49 -05:00
Dan ff069cb862 run_pyl 2023-02-24 15:02:03 -05:00
jasquat 425cf3f8c1 skip failing test if postgres and added comment about cause w/ burnettk 2023-02-24 14:57:17 -05:00
Dan 0dc2bc3316 # SpiffWorkflow:
1) Type Safe checking on correlation properties (no more str())
2) A running workflows Correlations are once again at the key level.

# Backend
1) Both send and receive messages can have correlation_keys - and we compare these to each other to quickly assure a match (if they both exist - otherwise we fall back to comparing the properties on the receive to the sending messages payload)
2) Cleaned up the migrations to just one file
2023-02-24 14:53:22 -05:00
jasquat 7e1f7942c3 added remaining dependabot items and commented them out and an auto merger w/ burnettk 2023-02-24 14:40:09 -05:00
burnettk 9a466ab65e commit dependabot from backend 2023-02-24 14:31:10 -05:00
jasquat c66a078879 updated actions checkout in images as well w/ burnettk 2023-02-24 14:27:59 -05:00