Commit Graph

380 Commits

Author SHA1 Message Date
jasquat 7b3eb4730f fixed broken test w/ burnettk 2023-03-20 10:55:57 -04:00
jasquat f8856db032 removed spiff_logging w/ burnettk 2023-03-17 15:49:16 -04:00
jasquat 77e97d7a57 tasks can be skipped and manually executed w/ burnettk 2023-03-17 15:35:27 -04:00
jasquat 88df3bd5c3 added process instance event table 2023-03-17 13:20:06 -04:00
jasquat 3461056beb Merge remote-tracking branch 'origin/main' into feature/use_tasks_as_logs 2023-03-17 12:40:26 -04:00
jbirddog 90aed76e21 UAT bug fix: suspended/terminated instances automagically resume (#186) 2023-03-17 11:55:11 -04:00
jasquat e0dbb6a768 updated log list view in frontend w/ burnettk 2023-03-17 10:30:29 -04:00
jasquat 363428df09 added bpmn_name columns to definition tables and added test for simple logs as well 2023-03-17 10:00:59 -04:00
burnettk 6bfce59fe3 lint 2023-03-16 22:59:42 -04:00
jasquat 4b96a0c453 add in missing fields to logs 2023-03-16 17:58:43 -04:00
jasquat d75d66c33f pyl w/ burnettk 2023-03-16 16:18:02 -04:00
jasquat 5c0d72ab91 tests are now passing w/ burnettk 2023-03-16 16:14:41 -04:00
jasquat 80d9fa8ad6 merged in save_to_task_data branch and pyl with some cleanup w/ burnettk 2023-03-16 12:50:52 -04:00
burnettk d36256fef7 fix two issues, one where we were not sorting after globbing, and another where we forgot about process groups 2023-03-16 12:34:44 -04:00
jasquat b0656f961f use box script engine to save everything into task data for time being w/ burnettk 2023-03-16 11:17:21 -04:00
jasquat bbdac3c586 some initial code to use tasks for logs 2023-03-16 09:30:25 -04:00
burnettk dc355e90e9 try to fix postgres and mysql group by 2023-03-16 00:02:02 -04:00
jasquat e305b22b5a increase line length from 88 to 119 for black w/ burnettk 2023-03-15 16:24:08 -04:00
jasquat 96989c676e fixed up tests and ran pyl w/ burnettk 2023-03-15 16:10:23 -04:00
jasquat bc619af0a0 check data when running main spiff test w/ burnettk 2023-03-15 15:38:58 -04:00
jasquat 2e9fa0e4b4 pyl and cleaned up debug code w/ burnettk 2023-03-15 12:15:48 -04:00
jasquat ceca4d1333 unit tests are passing w/ burnettk 2023-03-15 11:25:15 -04:00
jbirddog 764eb35d1b Move process instance locking to new queue table (#177) 2023-03-14 13:12:01 -04:00
jasquat 812e93047c fixed tests and added some comments for future fixes we will need with spiff without loop reset 2023-03-11 13:22:30 -05:00
jasquat 2ef0bf0065 added loop to task test bpmn file w/ burnettk 2023-03-10 17:15:27 -05:00
jasquat 7806e6c460 tests are now passing w/ burnettk 2023-03-10 16:52:57 -05:00
jasquat e13e703825 check task data on spiff tasks 2023-03-10 13:29:23 -05:00
jasquat 66a6c0449a added subprocess and call activity to task data bpmn file and the test is passing now 2023-03-10 10:46:40 -05:00
burnettk eeadd9296f Merge remote-tracking branch 'origin/main' into feature/save_tasks_one_at_a_time 2023-03-09 22:07:11 -05:00
jasquat 7e44c90fbb the primary test is passing now but subprocesses and call activities are probably broken w/ burnettk 2023-03-09 17:16:44 -05:00
Dan 3879ea4f3a run_pyl 2023-03-09 16:10:31 -05:00
Dan 79a17ec829 Adding a new test for error handing to assure this doesn't break in the future, and cleaning up the message call event.
Will also need to update the error handling BPMN process so it provides correlation keys.  We should add a task that will
alert you when you create a message object without setting correlation keys - as they are required per the specification.
2023-03-09 15:27:35 -05:00
jasquat 0f23a1b4d9 Merge remote-tracking branch 'origin/main' into feature/split_up_bpmn_json 2023-03-07 10:48:37 -05:00
Jon Herron ce5969e9ca Getting ./bin/pyl to pass 2023-03-06 20:42:25 -05:00
Jon Herron ec53ab8717 Fix secret getting reset when decrypted in get_secret 2023-03-06 20:40:08 -05:00
Jon Herron 3361bb0cd2 Fixed tests 2023-03-06 19:27:24 -05:00
jasquat 628dc14d7c make sure we set the subprocesses correctly in the bpmn dict w/ burnettk 2023-03-06 13:03:18 -05:00
jasquat 2cdacfa03b some stuff is passing but still needs the process_instance_data w/ burnettk 2023-03-03 16:51:24 -05:00
jasquat 40a3515d4e Merge remote-tracking branch 'origin/main' into feature/split_up_bpmn_json 2023-03-03 13:04:26 -05:00
jbirddog b4546e86e6 Special case file data during user form submission (#164) 2023-03-03 12:24:18 -05:00
jasquat d7f4c8ba95 clean up bpmn process definition save code a bit w/ burnettk 2023-03-03 11:24:28 -05:00
jasquat 0c6e9a63ba unit tests are passing with the new spec tables 2023-03-03 10:08:14 -05:00
jasquat cf9b796e1a avoid creating new processors if it is not necessary 2023-03-02 10:00:08 -05:00
jasquat 90b1772215 do not require task to be given to evaluate a task unless that script specifically needs it w/ burnettk 2023-03-01 17:18:58 -05:00
jasquat d1b8de9ea3 pyl 2023-03-01 12:35:08 -05:00
jasquat ab50e7ac03 all backend tests except for report tests are now passing 2023-03-01 10:58:12 -05:00
jasquat 035475f58f unit tests are passing except for test_process_instance_report which cannot work currently 2023-03-01 10:36:11 -05:00
jasquat d5acd5a16d added new table and some notes on how to get a delta w/ burnettk jbirddog 2023-02-28 16:30:52 -05:00
jasquat 09a46712d4 added debug logging when adding spiff step details w/ burnettk 2023-02-27 15:53:56 -05:00
jasquat dfac872606 Merge remote-tracking branch 'origin/main' into feature/script_get_last_user_completing_task 2023-02-27 14:28:23 -05:00
jasquat 29ada38904 added script to get process initiator w/ burnettk 2023-02-27 14:28:19 -05:00
Dan e00fbdb490 run_pyl 2023-02-27 14:21:22 -05:00
Dan f16150b02b Merging main 2023-02-27 14:17:10 -05:00
jasquat 9f84564457 script to get last user completing a task is working w/ burnettk 2023-02-27 12:08:07 -05:00
burnettk e9b3ababc1 wip for get_last_user_completing_task script task 2023-02-25 23:31:23 -05:00
burnettk f7dc076f75 postgres really will just order however it wants if you do not specify an order_by clause 2023-02-25 22:35:46 -05:00
jasquat b71419616d removed some unused code from task and fixed the logs table a bit w/ burnettk 2023-02-24 16:18:49 -05:00
Dan ff069cb862 run_pyl 2023-02-24 15:02:03 -05:00
jasquat 425cf3f8c1 skip failing test if postgres and added comment about cause w/ burnettk 2023-02-24 14:57:17 -05:00
Dan 0dc2bc3316 # SpiffWorkflow:
1) Type Safe checking on correlation properties (no more str())
2) A running workflows Correlations are once again at the key level.

# Backend
1) Both send and receive messages can have correlation_keys - and we compare these to each other to quickly assure a match (if they both exist - otherwise we fall back to comparing the properties on the receive to the sending messages payload)
2) Cleaned up the migrations to just one file
2023-02-24 14:53:22 -05:00
jasquat 7a1e712c14 avoid using task-data endpoint for task data and only use it to get tasks based on spiff step instead 2023-02-24 09:21:21 -05:00
jasquat a4c1eb96b5 removed task-data endpoints since we no longer need them w/ burnettk 2023-02-23 17:07:53 -05:00
Dan 7c12dffe41 Merge remote-tracking branch 'origin/main' into feature/message_fixes 2023-02-23 14:26:04 -05:00
Dan 0030a46938 run_pyl 2023-02-23 14:17:22 -05:00
Dan 0f3ef00d72 BPMN.io -- Just show the message names not the ids - to assure we are only exposing the names.
SpiffWorkflow -
    - start_messages function should return message names, not ids.
    - don't catch external thrown messages within the same workflow process
    - add an expected value to the Correlation Property Model so we can use this well defined class as an external communication tool (rather than building an arbitrary dictionary)
    - Added a "get_awaiting_correlations" to an event, so we can get a list of the correlation properties related to the workflows currently defined correlation values.
    - workflows.waiting_events() function now returns the above awaiting correlations as the value on returned message events
 Backend
    - Dropping MessageModel and MessageCorrelationProperties - at least for now.  We don't need them to send / receive messages though we may eventually want to track the messages and correlations defined across the system - these things (which are ever changing) should not be directly connected to the Messages which may be in flux - and the cross relationships between the tables could cause unexpected and unceissary errors.  Commented out the caching logic so we can turn this back on later.
    - Slight improvement to API Errors
    - MessageInstances are no longer in a many-to-many relationship with Correlations - Each message instance has a unique set of message correlations specific to the instance.
    - Message Instances have users, and can be linked through a "counterpart_id" so you can see what send is connected to what recieve.
    - Message Correlations are connected to  recieving message instances.  It is not to a process instance, and not to a message model.  They now include the expected value and retrieval expression required to validate an incoming message.
    - A process instance is not connected to message correlations.
    - Message Instances are not always tied to a process instance (for example, a Send Message from an API)
    - API calls to create a message use the same logic as all other message catching code.
    - Make use of the new waiting_events() method to check for any new recieve messages in the workflow (much easier than
    churning through all of the tasks)
    - One giant mother of a migration.
2023-02-23 13:53:03 -05:00
jasquat c5d611f03b disable strict host checking for git commands w/ burnettk 2023-02-21 16:42:11 -05:00
jasquat e5b83d0919 fixed tests w/ burnettk 2023-02-21 15:28:54 -05:00
Dan 790483a421 run_pyl 2023-02-20 12:34:42 -05:00
Dan 5f6a61c93f * SpiffWorkflow event_definitions wanted to return a message event's correlation properties mested within correlation keys. But messages are directly related to properties, not to keys - and it forced a number of conversions that made for tricky code. So Messages now contain a dictionary of correlation properties only.
* SpiffWorkflow did not serialize correlations - so they were lost between save and retrieve.

* When comparing Correlation Property values - we are storing these values as strings in the database and can't convert them back to integers later, so I'm changing everying everywhere to compare after conversion to a string.  Don't feel great about this one.
* By using an SQL Alchemy join table, there is a lot of db queries we don't need to write.
* A few handy fucntions on db models to make it easier to work with correlations.
* Updated tests because I changed some of the BPMN models we were testing against.
* Database migration to use the new constraint names with the alternate form of the join table between correlation mesages to instance messages.
2023-02-20 11:50:35 -05:00
Dan 2a800e844f * Re-work message tests so I could wrap my simple head around what was happening - just needed an example that made sense to me.
* Clear out complex get_message_instance_receive how that many-to-many works.
* Create decent error messages when correlations fail
* Move correlation checks into the MessageInstance class
* The APIError could bomb out ugly if it hit a workflow exception with not Task Spec.
2023-02-18 13:09:58 -05:00
Dan ecbd30af00 failing test. 2023-02-17 12:10:32 -05:00
Dan 8e257600af work in progress -
* Link between message instance and correlations is now a link table and many-to-many relationships as recommended by SQLAlchemy
* Use the correlation keys, not the process id when accepting api messages.
2023-02-17 10:45:01 -05:00
Jon Herron 491d30b9b3 The End Event has a spiff step 2023-02-13 14:13:34 -05:00
Dan Funk 3ba9827670 Merge pull request #132 from sartography/feature/no_more_current_user
Feature/no more current user
2023-02-08 18:23:25 -05:00
Dan 18c481e3d4 run_pyl 2023-02-08 17:10:57 -05:00
Dan eb39b61488 remove the "current_user" from being added to the task_data. 2023-02-08 15:53:14 -05:00
burnettk d5a8475be6 pyl 2023-02-08 14:02:17 -05:00
Dan b5cd11ef37 Use a mock when making external calls in tests. 2023-02-08 13:14:42 -05:00
Dan c270ababd9 run_pyl 2023-02-08 12:00:27 -05:00
Dan 443a855782 Assure that when something goes wrong calling a service task that we get as much good information about the problem as possible. 2023-02-08 11:53:20 -05:00
jasquat 559d9a78d8 renamed development env to local_development and testing to unit_testing w/ burnettk 2023-02-07 15:02:47 -05:00
jasquat 5452b48d08 use our json encoder to dump the user for get_current_user script w/ burnettk 2023-02-06 14:11:15 -05:00
burnettk 2c3b88042e move towards returning dict in get_current_user 2023-02-06 10:30:38 -05:00
burnettk 11a9e740af show that hiding nested fields works as well 2023-02-03 15:47:35 -05:00
burnettk 1247189bf8 make form schema and form ui schema both dicts, add support for hiding fields based on task data 2023-02-03 15:40:14 -05:00
jbirddog a815863727 Allow for different Python Environments when executing scripts within SpiffWorkflow (#121) 2023-02-02 10:24:55 -05:00
burnettk cb4429e837 folks who can start instances can also view their logs 2023-01-31 17:11:11 -05:00
jasquat bec1b43cfb tests are passing now w/ burnettk 2023-01-27 13:20:22 -05:00
jasquat 54f795da7b some initial updates w/ burnettk 2023-01-27 12:51:19 -05:00
jasquat 9e3640b169 set the correct type for task since SpiffTask and a backend task are not the same 2023-01-26 07:35:13 -05:00
jasquat 13b442f7f8 Merge remote-tracking branch 'origin/main' into feature/jinja_errors 2023-01-25 17:30:55 -05:00
Dan b2fb0dd79f run_pyl had various recommendations that I find a bit of a pain in the butt, but that I did anyway. 2023-01-25 16:43:28 -05:00
jasquat ddb55a0ce1 ensure we are passing the primary file name to the delete file test 2023-01-25 16:20:55 -05:00
Dan a38ecc2e60 Merge remote-tracking branch 'origin/main' into feature/jinja_errors 2023-01-25 15:38:52 -05:00
jasquat ada389b572 allow removing users from groups when refreshing permissions w/ burnettk 2023-01-25 15:12:48 -05:00
Dan d63f0c2a91 Making sure we create informative messages when encountering jinja2 syntax errors. 2023-01-25 14:13:21 -05:00
jasquat d86c083dfe do not allow deleting primary bpmn file and do not allow instantiating models without a primary bpmn file w/ burnettk 2023-01-24 15:15:48 -05:00
jasquat 4e0ddc7ea5 removed uniqueness constraint from human task so we can loopback to a previous task with a gateway w/ burnettk 2023-01-24 11:03:55 -05:00
jasquat 6371703637 ensure order of permissions in tests 2023-01-24 10:23:47 -05:00
jasquat 102413326a added locking system for process instances so hopefully background jobs will not take instances currently being run by the user w/ burnettk 2023-01-23 16:45:07 -05:00