Commit Graph

1951 Commits

Author SHA1 Message Date
jasquat 11384fe8fa
always save the serialized bpmn definition for now w/ burnettk 2023-03-01 15:01:29 -05:00
Dan Funk 38a4fe18f4
Merge pull request #161 from sartography/feature/messages_again_and_again
workflow.catch() was nice, in that it is where we could send events a…
2023-03-01 14:24:34 -05:00
Dan ddca493011 Don't attempt to gather the augmented methods if no task is provided -- if we aren't working within the context of a task, we are not working in a context where augmented methods can work (at least not all of them). This was causing an error when attepting to use the custom engine to execute extraction expressions on messages. 2023-03-01 13:46:20 -05:00
jasquat e9ce360a10
pyl 2023-03-01 12:35:08 -05:00
jasquat 28fd2a3966
build docker images for this branch w/ burnettk 2023-03-01 12:27:28 -05:00
jasquat 3865fefd0c
fixed get call activity task data w/ burnettk 2023-03-01 12:23:04 -05:00
jasquat e9f8364b1a
store the process instance data id on the process instance and not the other way around w/ burnettk 2023-03-01 11:39:03 -05:00
Kevin Burnett 27446e7a1e
Merge pull request #158 from sartography/ci/add-jenkinsfile
ci: add basic jenkinsfile for building Docker images
2023-03-01 08:01:23 -08:00
jasquat 8266d3f14b
all backend tests except for report tests are now passing 2023-03-01 10:58:12 -05:00
jasquat b23cdec804
unit tests are passing except for test_process_instance_report which cannot work currently 2023-03-01 10:36:11 -05:00
jasquat e4d7b067bc
save full_bpmn_json in a var 2023-03-01 09:55:20 -05:00
jasquat 25269133c1
must add data to db 2023-03-01 09:42:41 -05:00
jasquat 32f04e9a75
most unit tests are passing now and the use of bpmn_json is almost gone in src 2023-03-01 09:22:38 -05:00
Dan e6d32a4af7 workflow.catch() was nice, in that it is where we could send events and messages. With this change sending an event to catch will behave incorrectly for BPMN Messages. Only sending it to the right method will create the desired result. It also adds a lot of additional code. Would love a careful review of this, and any optimizations anyone can think of. 2023-02-28 21:08:04 -05:00
jasquat 8e286cba91
some logic to attempt to use the new bpmn json tables w/ burnettk 2023-02-28 17:46:14 -05:00
jasquat 828c042397
added new table and some notes on how to get a delta w/ burnettk jbirddog 2023-02-28 16:30:52 -05:00
jasquat f8af24f0a1
added config to turn on sentry profiling w/ burnettk 2023-02-28 11:23:50 -05:00
jasquat 7c4c04d3c9
remove docker push image from backend in favor of docker image from dedicated action file 2023-02-28 09:22:50 -05:00
jasquat 5e359cac3b
fixed python 3.9 syntax issue 2023-02-28 08:56:44 -05:00
burnettk 7c83ce9c3a
make metadata header bigger 2023-02-27 22:36:31 -05:00
burnettk a9a01f33c6
slam migrations file one last-ish time, increase task_name column length 2023-02-27 22:03:42 -05:00
burnettk 7216af1ce8
allow task_name to be long 2023-02-27 22:01:08 -05:00
burnettk 4c7bc90a8f
let env var work on windows and fixing typing issue on python 3.9 w/ messaging stuff 2023-02-27 19:16:06 -05:00
jasquat 45b634a6df
recreated all migrations to resolve postgres migration failures w/ burnettk 2023-02-27 17:27:53 -05:00
jasquat bc5e6d695c
Merge pull request #160 from sartography/feature/fix_sqlite_ci
Feature/fix sqlite ci
2023-02-27 17:26:26 -05:00
jasquat f00059369c
run poetry install for sqlite w/ burnettk 2023-02-27 17:21:34 -05:00
jasquat 5c5e36c067
do not run mysql commands if not doing mysql w/ burnettk 2023-02-27 17:19:03 -05:00
jasquat 9ffdbe4572
do not override env vars w/ burnettk 2023-02-27 17:15:21 -05:00
jasquat f0ea9076e2
syntax error w/ burnettk 2023-02-27 17:10:52 -05:00
jasquat 43201f6597
attempt to set the sample process model dir w/ burnettk 2023-02-27 17:10:14 -05:00
jasquat c927210c8d
attempt to run migrations from scratch for sqlite in ci since it does not support the migrations w/ burnettk 2023-02-27 17:00:34 -05:00
jasquat aafeed409c
do not return human tasks from errored process instances w/ burnettk 2023-02-27 16:11:26 -05:00
jasquat 4d7b0038fc
added debug logging when adding spiff step details w/ burnettk 2023-02-27 15:53:56 -05:00
burnettk 3add069267 Squashed 'bpmn-js-spiffworkflow/' changes from f1f008e3e..a547888ef
a547888ef run_pyl
53378437d BPMN.io -- Just show the message names not the ids - to assure we are only exposing the names. SpiffWorkflow -     - start_messages function should return message names, not ids.     - don't catch external thrown messages within the same workflow process     - add an expected value to the Correlation Property Model so we can use this well defined class as an external communication tool (rather than building an arbitrary dictionary)     - Added a "get_awaiting_correlations" to an event, so we can get a list of the correlation properties related to the workflows currently defined correlation values.     - workflows.waiting_events() function now returns the above awaiting correlations as the value on returned message events  Backend     - Dropping MessageModel and MessageCorrelationProperties - at least for now.  We don't need them to send / receive messages though we may eventually want to track the messages and correlations defined across the system - these things (which are ever changing) should not be directly connected to the Messages which may be in flux - and the cross relationships between the tables could cause unexpected and unceissary errors.  Commented out the caching logic so we can turn this back on later.     - Slight improvement to API Errors     - MessageInstances are no longer in a many-to-many relationship with Correlations - Each message instance has a unique set of message correlations specific to the instance.     - Message Instances have users, and can be linked through a "counterpart_id" so you can see what send is connected to what recieve.     - Message Correlations are connected to  recieving message instances.  It is not to a process instance, and not to a message model.  They now include the expected value and retrieval expression required to validate an incoming message.     - A process instance is not connected to message correlations.     - Message Instances are not always tied to a process instance (for example, a Send Message from an API)     - API calls to create a message use the same logic as all other message catching code.     - Make use of the new waiting_events() method to check for any new recieve messages in the workflow (much easier than     churning through all of the tasks)     - One giant mother of a migration.

git-subtree-dir: bpmn-js-spiffworkflow
git-subtree-split: a547888ef102046bab983b9004f07d4398e9a1bc
2023-02-27 14:59:39 -05:00
burnettk 7ab479c1ef
Merge commit '3add06926765126577a100845c8d3247323e5f4c' 2023-02-27 14:59:39 -05:00
burnettk 7cd7ca5e3c
Merge commit '7a5f961b2bbff62f525805d6302822a411024f7b' 2023-02-27 14:59:38 -05:00
burnettk 7a5f961b2b Squashed 'spiffworkflow-frontend/' changes from c22d22ba5..4fd946be4
4fd946be4 Merging main
453bdb246 run_pyl
b5a8ff01b Needed an additional check for empty correlation keys - which on a RECEIVE message, should always match anything.
b583f84cc lint w/ burnettk
fd4d7d13b removed some unused code from task and fixed the logs table a bit w/ burnettk
6f17e71e6 avoid using task-data endpoint for task data and only use it to get tasks based on spiff step instead
669c29595 removed task-data endpoints since we no longer need them w/ burnettk
5f25fffe0 added api to get task data and do not return from task data list anymore w/ burnettk
373c4f184 Merge remote-tracking branch 'origin/main' into feature/message_fixes
e40c12ac8 turn on sentry detailed tracing for task-data w/ burnettk
d7861aae2 Merge branch 'main' into feature/message_fixes
0a9f2480d work in progress - * Link between message instance and correlations is now a link table and many-to-many relationships as recommended by SQLAlchemy * Use the correlation keys, not the process id when accepting api messages.

git-subtree-dir: spiffworkflow-frontend
git-subtree-split: 4fd946be4ff716bfd1664aca8d75cb6f709b53eb
2023-02-27 14:59:38 -05:00
burnettk 0d2c1e2835
Merge commit '3ea0819aae44dcd135a071889b970f0919ef880e' 2023-02-27 14:59:37 -05:00
burnettk 3ea0819aae Squashed 'spiffworkflow-backend/' changes from 6cae736ac..0bfe4400f
0bfe4400f Merge branch 'main' of github.com:sartography/spiff-arena into main
fbe97dd90 Point to the latest spiffworkflow
805400312 fixed conflict with db migrations w/ burnettk
6e48c8205 Merge remote-tracking branch 'origin/main' into feature/script_get_last_user_completing_task
3a6cc9c2b added script to get process initiator w/ burnettk
37770dbca run_pyl
8a068cdb2 Merging main
bf25c1cef run_pyl
4ca6e3f27 Needed an additional check for empty correlation keys - which on a RECEIVE message, should always match anything.
0fb88c50d remove unwanted test files w/ burnettk
89d92687d script to get last user completing a task is working w/ burnettk
98f056e5c Merge remote-tracking branch 'origin/main' into feature/script_get_last_user_completing_task
46af55a60 poetry remove orjson
28a0db097 wip for get_last_user_completing_task script task
2596cfeb1 postgres really will just order however it wants if you do not specify an order_by clause
5e339b2fb add ppg.ba4.sme and ba5
873cfdfc1 fix postgres db name and comment out debug job
f54d6ae4d removed some unused code from task and fixed the logs table a bit w/ burnettk
7c74e216a run_pyl
f53c85924 skip failing test if postgres and added comment about cause w/ burnettk
fbe2237f1 # SpiffWorkflow: 1) Type Safe checking on correlation properties (no more str()) 2) A running workflows Correlations are once again at the key level.
2677736c2 look users up by service and username instead of service_id since usernames have to be unique anyway w/ burnettk
22d1f8bbb avoid using task-data endpoint for task data and only use it to get tasks based on spiff step instead
698fb5d5c put back the task data code when getting tasks
b52f97453 Merge remote-tracking branch 'origin/main' into feature/task_data_api_refactor
9c4a17c93 removed commented out code w/ burnettk
3357fbef4 removed task-data endpoints since we no longer need them w/ burnettk
3b64afb65 turn off profiling again
fcbcfd0ae add two users and update one
2ee7cba09 BPMN Parser was returning all retrieval expressions, rather than the ones specific to a correlation property, as was intended. Adding a correlation cache - so we have a reference of all the messages and properties (though still lacking a description of keys) Adding yet another migration, maybe should squash em.
6658e26bb added api to get task data and do not return from task data list anymore w/ burnettk
1fd3261d7 run_pyl (part 2)
f6c63eb1c Merge branch 'main' of github.com:sartography/spiff-arena
5c5262a31 added comment about refactoring getting task data w/ burnettk jbirddog
6d4aa9043 lint
e4c0ed7e1 add test users
adfb0644f Adding Migration.
ec36290f2 remove task size check since it can take a long time to run and we do not do anything with it w/ burnettk jbirddog
9b0b95f95 Merge remote-tracking branch 'origin/main' into feature/message_fixes
f3d124a70 run_pyl
be6ac8743 BPMN.io -- Just show the message names not the ids - to assure we are only exposing the names. SpiffWorkflow -     - start_messages function should return message names, not ids.     - don't catch external thrown messages within the same workflow process     - add an expected value to the Correlation Property Model so we can use this well defined class as an external communication tool (rather than building an arbitrary dictionary)     - Added a "get_awaiting_correlations" to an event, so we can get a list of the correlation properties related to the workflows currently defined correlation values.     - workflows.waiting_events() function now returns the above awaiting correlations as the value on returned message events  Backend     - Dropping MessageModel and MessageCorrelationProperties - at least for now.  We don't need them to send / receive messages though we may eventually want to track the messages and correlations defined across the system - these things (which are ever changing) should not be directly connected to the Messages which may be in flux - and the cross relationships between the tables could cause unexpected and unceissary errors.  Commented out the caching logic so we can turn this back on later.     - Slight improvement to API Errors     - MessageInstances are no longer in a many-to-many relationship with Correlations - Each message instance has a unique set of message correlations specific to the instance.     - Message Instances have users, and can be linked through a "counterpart_id" so you can see what send is connected to what recieve.     - Message Correlations are connected to  recieving message instances.  It is not to a process instance, and not to a message model.  They now include the expected value and retrieval expression required to validate an incoming message.     - A process instance is not connected to message correlations.     - Message Instances are not always tied to a process instance (for example, a Send Message from an API)     - API calls to create a message use the same logic as all other message catching code.     - Make use of the new waiting_events() method to check for any new recieve messages in the workflow (much easier than     churning through all of the tasks)     - One giant mother of a migration.
ce449971a do not call serialize if we can use the cached bpmn_json instead w/ burnettk
15d720f94 Merge branch 'main' of github.com:sartography/spiff-arena
8d8347068 turn on sentry detailed tracing for task-data w/ burnettk
cdf5f4053 update spiff
5c1ea3c93 run_pyl
384c272af * SpiffWorkflow event_definitions wanted to return a message event's correlation properties mested within correlation keys.  But messages are directly related to properties, not to keys - and it forced a number of conversions that made for tricky code.  So Messages now contain a dictionary of correlation properties only. * SpiffWorkflow did not serialize correlations - so they were lost between save and retrieve.
f45103406 Allow people to run commands like "flask db upgrade" without setting specific environment variables like FLASK_SESSION_SECRET_KEY everytime - they just need to add in their own /instance/config.py with their local configuration.
b169c3a87 * Re-work message tests so I could wrap my simple head around what was happening - just needed an example that made sense to me. * Clear out complex get_message_instance_receive how that many-to-many works. * Create decent error messages when correlations fail * Move correlation checks into the MessageInstance class * The APIError could bomb out ugly if it hit a workflow exception with not Task Spec.
a39590912 failing test.
6db600caa Merge branch 'main' into feature/message_fixes
4942a728b work in progress - * Link between message instance and correlations is now a link table and many-to-many relationships as recommended by SQLAlchemy * Use the correlation keys, not the process id when accepting api messages.

git-subtree-dir: spiffworkflow-backend
git-subtree-split: 0bfe4400f4191214b8972977438ceb35a9f5b3c3
2023-02-27 14:59:37 -05:00
burnettk f3d02daf7a Squashed 'SpiffWorkflow/' changes from 7b39b2235..b3235fad5
b3235fad5 Merging main
09623ca61 # SpiffWorkflow: 1) Type Safe checking on correlation properties (no more str()) 2) A running workflows Correlations are once again at the key level.
d6806f69d maintain a way to access the correlations in relation to the correlation keys
065a86cde BPMN Parser was returning all retrieval expressions, rather than the ones specific to a correlation property, as was intended. Adding a correlation cache - so we have a reference of all the messages and properties (though still lacking a description of keys) Adding yet another migration, maybe should squash em.
9e8832c93 Merge remote-tracking branch 'origin/main' into feature/message_fixes
8efa922ae run_pyl
72a7e535a BPMN.io -- Just show the message names not the ids - to assure we are only exposing the names. SpiffWorkflow -     - start_messages function should return message names, not ids.     - don't catch external thrown messages within the same workflow process     - add an expected value to the Correlation Property Model so we can use this well defined class as an external communication tool (rather than building an arbitrary dictionary)     - Added a "get_awaiting_correlations" to an event, so we can get a list of the correlation properties related to the workflows currently defined correlation values.     - workflows.waiting_events() function now returns the above awaiting correlations as the value on returned message events  Backend     - Dropping MessageModel and MessageCorrelationProperties - at least for now.  We don't need them to send / receive messages though we may eventually want to track the messages and correlations defined across the system - these things (which are ever changing) should not be directly connected to the Messages which may be in flux - and the cross relationships between the tables could cause unexpected and unceissary errors.  Commented out the caching logic so we can turn this back on later.     - Slight improvement to API Errors     - MessageInstances are no longer in a many-to-many relationship with Correlations - Each message instance has a unique set of message correlations specific to the instance.     - Message Instances have users, and can be linked through a "counterpart_id" so you can see what send is connected to what recieve.     - Message Correlations are connected to  recieving message instances.  It is not to a process instance, and not to a message model.  They now include the expected value and retrieval expression required to validate an incoming message.     - A process instance is not connected to message correlations.     - Message Instances are not always tied to a process instance (for example, a Send Message from an API)     - API calls to create a message use the same logic as all other message catching code.     - Make use of the new waiting_events() method to check for any new recieve messages in the workflow (much easier than     churning through all of the tasks)     - One giant mother of a migration.
cb2ff8a93 * SpiffWorkflow event_definitions wanted to return a message event's correlation properties mested within correlation keys.  But messages are directly related to properties, not to keys - and it forced a number of conversions that made for tricky code.  So Messages now contain a dictionary of correlation properties only. * SpiffWorkflow did not serialize correlations - so they were lost between save and retrieve.
d4852a1a5 * Re-work message tests so I could wrap my simple head around what was happening - just needed an example that made sense to me. * Clear out complex get_message_instance_receive how that many-to-many works. * Create decent error messages when correlations fail * Move correlation checks into the MessageInstance class * The APIError could bomb out ugly if it hit a workflow exception with not Task Spec.

git-subtree-dir: SpiffWorkflow
git-subtree-split: b3235fad598ee3c4680a23f26adb09cdc8f2807b
2023-02-27 14:59:36 -05:00
burnettk dffce43fc3
Merge commit 'f3d02daf7ada5d2544898e2867cf74b39a5ba099' 2023-02-27 14:59:36 -05:00
Jakub Sokołowski c939bb24d8
ci: add basic jenkinsfile for building Docker images
This is the first and most basic version of a Jenkinsfile we can use.

Since we can't just hardcode things like docker name, tag, or
credentials for pushing to a Docker registry it needs to be a parameter.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-27 20:46:15 +01:00
Dan 0b3fcf1daa Merge branch 'main' of github.com:sartography/spiff-arena into main 2023-02-27 14:40:52 -05:00
Dan 5dfe1d73d1 Point to the latest spiffworkflow 2023-02-27 14:40:47 -05:00
jasquat ae4626b2f6
Merge pull request #159 from sartography/feature/script_get_last_user_completing_task
Feature/script get last user completing task
2023-02-27 14:35:11 -05:00
jasquat 24ee23f28c
fixed conflict with db migrations w/ burnettk 2023-02-27 14:33:55 -05:00
jasquat 3cc0ad1fb8
Merge remote-tracking branch 'origin/main' into feature/script_get_last_user_completing_task 2023-02-27 14:28:23 -05:00
jasquat df17c51453
added script to get process initiator w/ burnettk 2023-02-27 14:28:19 -05:00
Dan 6401481f99 run_pyl 2023-02-27 14:21:22 -05:00
Dan f65ed14df3 Merging main 2023-02-27 14:17:10 -05:00