Commit Graph

132 Commits

Author SHA1 Message Date
mike cullerton b1dca3f212 FileModel => DocumentModel
Remove unused imports
2022-04-13 08:34:10 -04:00
Dan 0455506e2b Loading latest chages from Spiff whic refactored the way the PythonExpressionEngine works.
This cleans up the _evaluate method which previously accepted arbitrary args and kwargs, and now requires an expression, a context to which exectute it, and, optionally, the current task being executed if the DMN is being executed as a part of a BusinessRuleTask in a BPMN diagram.

This also cleans up several bits of duplicated code.

There is also a bit of code here to assure that the current user is included when running the master workflow.
2022-03-04 15:34:36 -05:00
Dan 3944d00e88 Removing an unused / rogue variable.
Adding a logger warning for very large workflow json data.
2022-02-25 09:59:41 -05:00
Dan e61ca471db remove all the @timeit from method calls. 2022-02-24 21:06:22 -05:00
Dan dde8873c9e 1. defer the loading of the bpmn_json when loading the workflow model. It can be stupidly larger. (see models/workflow.py)
2. Shave a little more time off the reset by not re-creating the whole WorkflowProcessor

cleaning up all the firsttime/sincetime statements.
2022-02-24 20:56:57 -05:00
Dan 3442655bd1 Performance Improvements:
1. Avoid ever re-generating the list of scripts that can be used in a script task.  Terribly expensive as we call eval constantly, and it never ever changes once the app starts. (see script.py changes, and comments)
2. Cache the DocumentStatus list in the flask session, so we calculate it at most once per API Call.  It's at least .25 seconds per call.  (see study_sevice)
3. We called UserFileService.get_files_for_study (which runs a db query EVERY time) for every possible document type.  Now we run the query once (study service line 321)
4. When returning a workflow, we looped through every single task in that workflow's navigation, and called the expensive spiff_task_to_api_task just to figure out it's proper display name. We run a much faster and more efficient method to calculate the display name naow (see workflow_service on lie 680, and 799)
5. A hellton of @timeit and sincetime() calls, that I want to leave in, to help debug any slowness on production.
2022-02-24 14:25:42 -05:00
Dan c3e52f960b Minor stuff to help digging into validation of current workflows. 2022-02-18 17:20:04 -05:00
Dan f2b6008e5f Fixes 619 - look up models were being built incorrectly, and repeatedly, and sometimes bombed out all together.
bonus: resond with a valid error message when an invalid task_id is requested (don't just 500)
2022-02-17 11:04:50 -05:00
Dan 0ad9843569 Dealing with some changes in SpiffWorkflow's latest release. I think this gets all tests passing again, and behaving as expected. 2022-02-16 10:46:46 -05:00
Dan cc6f80bea9 fixing some paths. 2022-02-15 16:25:52 -05:00
Dan 093741fa90 By default the system will start up the scheduled tasks, which is usually what you want, but in development
it can be kind of irritating for this stuff to be spinning up when you are trying to debug something, so just set
PROCESS_WAITING_TASKS to false in instance/config.py and voila!!
2022-02-15 14:07:09 -05:00
Dan 648383f2ea for some reason jenkins doesn't like the spec_category_reorder test, so just doing it a different way. 2022-02-11 11:37:25 -05:00
Dan f12c4aba52 Refactor the workflow_spec_service so it doesn't cache anything, it always reads what it needs from the file system. 2022-02-09 21:06:55 -05:00
Dan 89e02c6207 really need to avoid any kind of caching in the WorkflowSpecService 2022-02-09 13:37:57 -05:00
Dan cc915ac25a refactoring the study service. 2022-02-09 08:50:00 -05:00
Dan 4b8d193fb9 a few more tests passing 2022-02-07 14:58:25 -05:00
Dan 2383c7d76d WIP - lots of random changes, trying to swtich over 2022-02-04 14:50:31 -05:00
Dan 4ec6e403f5 1. Created a UserFileService, so it's clear what we use when for File Services, UserFiles, SpecFiles, and ReferenceFiles each function differently.
2. Reference Files and Spec Files are written to disk, they do not exist in the database at all.
2022-02-02 12:59:56 -05:00
Dan 8529465322 Removed the method get_spec_data_files completly - using get_spec_files and get_spec_data to get this information instead.
Only load the spec data files if you are creating a new workflow, otherwise just deserialize the json.
Removed the stuff about calculaing the version of the spec, as we don't use it.
2022-01-25 16:10:54 -05:00
mike cullerton 68820c67cb Removed (almost) all references to WorkflowSpecDependencyFile
(There is still a call in the lookup service, but we need to decide how to fix that)
2022-01-19 16:12:54 -05:00
mike cullerton b99ed73951 Remove unused imports 2022-01-12 15:00:26 -05:00
mike cullerton 9cc91f92c3 *** WIP ***
cleanup - removing commented code
2022-01-11 15:55:08 -05:00
mike cullerton dc27f795c8 *** WIP ***
Committing because it is Friday afternoon, and my computer is acting flaky
2022-01-07 15:34:51 -05:00
mike cullerton 86a6039dc8 *** WIP ***
**Many** tests are failing!

Committing so I can merge dev into this branch
2022-01-06 11:46:54 -05:00
Dan 3480e1c8ab use a signal, in addition to firing off a cancel notify when a workflow is restarted. 2021-12-23 15:30:09 -05:00
Dan bd3176eea6 Fixes #578 - issue with us displaying an incorrect end event in some cases. 2021-12-22 16:27:52 -05:00
Dan ad93b5fae6 We were not correctly handing the possibility of there only being a waiting task (and no ready tasks).
We also had a problem with the validator throwing a 500 when it was trying tell us it would never finish validation.
2021-11-30 12:44:35 -05:00
Dan 4d1f4429ff Fixing a bug in enum_label that was not working when lookup up an select list from a spreadsheet.
Assure we raise more thoughtful error messages when running getting exceptions in engine tasks.
Field Options should always be available now due to a fix in Spiffworkflow.
2021-11-22 12:26:30 -05:00
Dan 84ce24243f add an enum_label script that will return the label given a value selection. 2021-10-21 13:57:49 -04:00
alicia pritchett 58e75a1c8f Change one condition to compare only user tasks, not non-user-tasks 2021-10-13 12:08:20 -04:00
alicia pritchett 1ef18f23aa one solution for the Parallel nav order problem 2021-10-11 11:50:03 -04:00
Dan 84680ea846 Fixing multiple issues that came out of Study Info, as we debugged issue #474 related to navigating back to a previous task.
There was a problem with the python script engine as well that wasn't handling the de-serialize properly and didn't correctly pick back up on the script engine, and the renaming of methods in PythonScriptEngine created some conflicts with the way we override functions.
We were not handling ldap looks up efficiently, and this was also breaking in Study Info.

Finally we had a bug in SpiffWorkflow that did not allow us to reset back to the previous task in some cases where nested call activities happen far later in the process and are currently active when the reset is created.
2021-10-06 12:17:57 -04:00
Dan 92b9fea08d fixing a stupid mistake that made all the people I care about suffer yesterday afternoon. 2021-09-29 10:06:17 -04:00
Kelly McDonald 960d302c03 Finish #367 - Make sure that we only use a primary_id if it is in the workflow that we are currently working on - otherwise it is a library and might have its own primary id. 2021-08-10 09:15:38 -04:00
Kelly McDonald 217e61eed3 next_task returns the next available task, except when the task is completed when it returns the EndEvent -
The problem was that it was returning the first EndEvent it found, not the last one. This caused a problem when we had a CallActivity which has its own EndEvent.

Fixes #399
2021-07-27 09:19:08 -04:00
Dan Funk 3c52453c26
Merge branch 'dev' into add-name-error-hint-8 2021-07-07 11:35:24 -04:00
Dan Funk db9b228735
Merge pull request #334 from sartography/WaitingEventsHotPatch
Make a change so that anything that has a waiting event is labeled in…
2021-07-07 11:27:10 -04:00
Dan 2a45f2fcda Merge branch 'dev' into add-name-error-hint-8 2021-07-07 11:24:00 -04:00
Dan fb54edac1c Adding additional details to error messages, and cleaning up the cruft around these messages to keep them clear and succinct.
Most noteable is the addition of the line on which the error occurs for script tasks.  It will report the line number and pass back the content of
the line that failed.
The validator only returns the first error it encounters, as it's clear that all we ever get right now is two of the same error.
Did a lot of work between this and spiffworkflow to remove all the places where we obfuscate or drop details as we converted between workflowExceptions and APIExceptions.
Dropped the python levenshtein dependency, in favor of just rolling a simple one ourselves in Spiffworkflow.
2021-07-07 00:53:49 -04:00
Kelly McDonald 9803a04d6d Make a change so that anything that has a waiting event is labeled in the database as waiting, even if it is sitting around waiting on a user input task that is ready 2021-07-06 11:46:47 -04:00
mike cullerton db44882733 When we encounter a name error running a script, look in the task data and see if there is a variable with a similar name.
If a variable with a similar name exists, add it to the ApiError as a hint
2021-07-01 15:38:45 -04:00
Mike Cullerton 98d641f54d
Merge branch 'dev' into partial_testing 2021-06-30 16:13:31 -04:00
NWalker4483 508ef1ff2b Readded exit_at to workflow param 2021-06-30 11:15:48 -04:00
Dan 9a32fadc2f Merge branch 'dev' into 346-waiting-task-schedule
# Conflicts:
#	Pipfile.lock
#	crc/services/workflow_service.py
2021-06-30 10:14:37 -04:00
Dan 07eb3f9ca8 Moving metrics into SpiffWorkflow so we can run the performance metrics deeply across both systems simultaniously.
Upgrading libraries.
Fixing deprication issue with Pandas and numpy.
We can only process xlsx files now, plain oldschool xls is fully removed.
2021-06-18 16:41:55 -04:00
Kelly McDonald 394906d840 Add refresh_waiting_tasks to the processor do_engine_steps so that if we have a waiting task that has met its condition prior to the scheduler running, it will go ahead and proceed 2021-06-10 08:33:24 -04:00
Kelly McDonald 5f722d675f Add a function that gets runs via a background scheduler to look for any workflows that are in a 'waiting' state - it runs the update waiting tasks and does do_engine_steps
We have a test for the function that runs, but an assumption was made that the scheduler module has its own unit tests and we do not need to test that.

fixes #346
2021-06-09 10:42:34 -04:00
Kelly McDonald 0d8d251e8e Temp save to test dev 2021-06-09 09:24:37 -04:00
Dan ef7ee284b2 Prefer tasks that share a parent over just the the next available task when returning the next_task in the workflow processor. 2021-05-19 16:26:16 -04:00
Dan Funk 34759a2f3d
Merge branch 'dev' into feature/performance-refactor 2021-05-04 11:30:13 -04:00