Commit Graph

66 Commits

Author SHA1 Message Date
Dan Funk 6caf44544c Merge branch 'dev' into cr-connect-92-scripting-enhancements
# Conflicts:
#	crc/static/bpmn/ind_supplement/ind_supplement.bpmn
#	tests/workflow/test_workflow_service.py
2020-07-20 13:01:23 -04:00
Dan Funk 06430550c8 Dropping the RRT-Data-Fix, it should have come out already, but had a failing test, so pulling it out now rather than delve into what is going wrong with obsolete code. 2020-07-20 11:39:50 -04:00
Carlos Lopez fcb772c900 Reporting to Sentry all captured exceptions and enabling multiple environments 2020-07-19 16:40:33 -06:00
Aaron Louie e82532aad8 Updates IND workflow spec. Fixes validation process to return lookups for enum values. 2020-07-17 11:51:21 -04:00
Aaron Louie d67105332f Merge branch 'dev' into feature/lanes_and_roles 2020-07-16 14:00:53 -04:00
Aaron Louie d0279a11e5 Fixes failing unit test 2020-07-15 17:44:46 -04:00
Dan Funk fa7608053a fixing a failed test. 2020-07-14 22:23:54 -04:00
Dan Funk d85ca1ce51 Whenever a workflow is loaded or updated, add events to the TaskEvent table with assignments for the next set of ready tasks, along with who should complete those tasks.
Add the lane information to the Task model.
Drop the foreign key constraint on the user_uid in the task log, as we might create tasks for users before they ever log into the system.
Add a new endpoint to the API called task events.  It should be possible to query this and get a list of all tasks that need a users attention.
The task events returned include detailed information about the workflow and study as sub-models
Rename all the actions in event log to things that are easier to pass over the api as arguments, make this backwards compatible, updating existing names in the database via the migration.
Throughly test the navigation and task details as control of the workflow is passed between two lanes.
2020-07-14 22:16:44 -04:00
Aaron Louie d71ff80eac Adds end and start events 2020-07-14 13:42:52 -04:00
Dan Funk 9077ff3ebf It is not possible to use task_data for an auto-complete field. It's too expensive an operation to provide that feature on the backend, and the data already fully resides on the front end anyway. Task-data can be used to populate enum fields if needed, so it can populate dropdowns, radios and checkboxes, just not auto-complete. 2020-07-14 11:38:48 -04:00
Dan Funk a48322ef6a Partial work on CR Connect Roles.
Adding checks in the API to assure the correct person is completeing a task based on the task's lane.
Adding lane to the Navigation item.
Adding a check to assure that unique user ids can be identified using task.data
Added some additional ldap entries to make testing and development easier.
Removed a big chunk of duplicate code from task_tests_api
Modified some of the helper functions to make it easier to test as specific users.
Added some additional bpmn models to the tests for testing lanes and roles.
2020-07-14 10:29:25 -04:00
Aaron Louie 07066b8a16 Looks up enum options from task data 2020-07-13 17:46:28 -04:00
Aaron Louie b7920989ed WIP: Adds Camunda property for retrieving enum field options from task data. 2020-07-10 14:48:38 -04:00
Aaron Louie 7f0b35ff27 Merge branch 'dev' into feature/enum_lookup_data 2020-07-10 10:27:51 -04:00
Carlos Lopez a302208890 Updating test for study with approvals 2020-07-07 08:32:41 -06:00
Aaron Louie 517cd8fc91 Merge branch 'dev' into feature/enum_lookup_data 2020-07-06 16:31:19 -04:00
Dan Funk 419c05e46c
Merge branch 'dev' into fix/more-info-in-exceptions 2020-07-06 12:51:24 -04:00
Dan Funk a5cef8775e * Modifying the StudyInfo script to return both "investigators" and "roles", the investigators argument will return only the current active investigators according to the protocol builder. The "roles" argument returns all possible roles - it is what "investigators" use to be.
* The Task.title returned to the front end will now attempt to process the "display_name" property for dot-notation syntax, making it possible to use this for multi-instance tasks,
  but will work in all cases where we want he title to change based on values in the data model.
* Fixing a bug the test_study_api where it wasn't updated when we made recent changes to the different states of a study.
*
2020-07-06 12:09:21 -04:00
Aaron Louie 1d9fef380a Merge branch 'fix/more-info-in-exceptions' into feature/enum_lookup_data 2020-07-03 14:22:11 -04:00
Aaron Louie f344fe60c0 WIP: Adds enum field lookup option data 2020-07-03 12:20:10 -04:00
Carlos Lopez 817333f26e Adding proper exception information 2020-07-02 16:10:33 -06:00
Aaron Louie a996c81508 Gets mi_index out of Spiff task internal data 2020-06-28 11:35:35 -04:00
Aaron Louie 848ad563d3 Only filters by mi_index if task has a mult_instance_index 2020-06-26 12:47:42 -04:00
Aaron Louie f9f3003ef0 Filters by multi-instance index, if applicable 2020-06-26 12:31:37 -04:00
Carlos Lopez 5327b469f6 Merge branch 'rrt/dev' into feature/emails-enhancement 2020-06-24 21:46:53 -06:00
Dan Funk 6aec15cc7c Shifting to a different model, where the TaskEvents store ONLY the form data submitted for that task.
In order to allow proper deletion of tasks, we no longer merge data returned from the front end, we set it directly as the task_data.
When returning data to the front end, we take any previous form submission and merge it into the current task data, allowing users to keep their previous submissions.
There is now an "extract_form_data" method that does it's best job to calculate what form data might have changed from the front end.
2020-06-19 08:22:53 -04:00
Carlos Lopez e947f40ec7 Merge branch 'rrt/dev' into feature/emails-enhancement 2020-06-17 20:10:11 -06:00
Dan Funk 3b57adb84c Continuing a major refactor. Some important points:
* TaskEvents now contain the data for each event as it was when the task was completed.
* When loading a task for the front end, if the task was completed previously, we take that data, and overwrite it with the lastest data, allowing users to see previously entered values.
* Pulling in the Admin branch, as there are changes in that branch that are critical to seeing what is happening when we do this thing.
* Moved code for converting a workflow to an API ready data stricture into the Workflow service where it belongs, and out of the API.
* Hard resets just convert to using the latest spec, they don't try to keep the data from the last task.  There is a better way.
* Moving to a previous task does not attept to keep the data from the last completed task.
* Added a function that will fix all the existing RRT data by adding critical data into the TaskEvent model. This can be called with from the flask command line tool.
2020-06-17 17:11:15 -04:00
Dan Funk 1b9166dcb7 Cleaning up the merge, which resulted in some lost code. 2020-06-16 13:34:21 -04:00
Dan Funk c7d8eaff30 Merge branch 'dev' into feature/refactor_data_loading 2020-06-16 13:15:43 -04:00
Aaron Louie 2a84d5196a filter, not filter_by 2020-06-12 14:13:27 -04:00
Aaron Louie e3126620b3 Eschews obfuscation 2020-06-12 14:09:08 -04:00
Aaron Louie 561e254315 Prevents non-admin users from editing each others' tasks. Fixes bug where test user uid was not being set from token. Moves complete form and get workflow API test utility methods into BaseTest. 2020-06-12 13:46:10 -04:00
Carlos Lopez e9e805b2c9 Storing emails in database 2020-06-09 22:57:56 -06:00
Dan Funk e102214809 minor cleanup of error codes. 2020-06-03 15:03:22 -04:00
Dan Funk c7484267e1 For the main approval endpoints - we now group the approvals by study. So you get one record back for each study, but it may have other approvals along with it as "related_approvals".
We now cache the LDAP records - so we look in our own database for the record before calling out to ldap for the details when given a straight up computing id like dhf8r.

Added "date_approved" to the approval model.

And moved the approver and primary investigator into real associated models to make it easier to dump.

Fixed a problem with the validation that was causing it to throw incorrect errors on valid workflows. Getting it to behave a little more like the front end behaves, and respecting the read-only fields.  But it was mainly to do with always returning all the data with each form submission.
2020-06-02 18:17:00 -04:00
Dan Funk 1db9401166 Don't put all the data into Spiff Tasks on a reload or backtrack, just store the data that gets submitted each time in the task log, and use that.
This should correct issues with parallel tasks and other complex areas - so we don't have tasks seeing data that isn't along their path.
2020-06-01 17:42:39 -04:00
Dan Funk b6bf843f6e used 'name' rather than 'value' in the lookup options during validation, causing a disconnect with how this is processed on the front end. 2020-06-01 11:00:56 -04:00
Dan Funk 2bc735a3f0 Kelly wrote a beautiful method for resetting the workflow that doesn't loose data when reset inside a parallel task, all I needed to do was use it.
Catching TypeErrors and reporting them back to the UI so we don't 500 in a bad way (but we still 500)
2020-05-31 13:48:00 -04:00
Dan Funk 1f0e8741ba Run the validation twice, once completing all of the data, and a second time, completing only the required fields.
Also, add a helper method to reduce boiler plate code around Workflow Exceptions.
2020-05-30 17:26:27 -04:00
Dan Funk 860c475b29 Fill out repeating sections during validation process.
Also, when returning error messages, attempt to include the task data for the task that caused the error.
Also, when attempting to delete any file, respond with an API error explaining the issue, and log the details.
2020-05-30 15:37:04 -04:00
Dan Funk afb6be7c60 re-working the way the redirects function, so we pass arguments as a get parameter. Just trying to get rid of the weird lag on production.
I noticed the validation sometimes looks ahead for files, so looking at all the tasks now, not just the ready tasks for the lookup field.
Ran into an issue with validation where a workflow model was required, so I create one and delete it.  Another refactor for another day.
2020-05-29 04:42:48 -04:00
Dan Funk 11413838a7 Faster lookup fields. We were parsing the spec each time to get details about how to search. We're just grabbing the workflow id and task id now and building that straight into the full text search index for faster lookups. Should be peppy.
Another speed improvement - data in the FileDataModel is deferred, and not queried until it is specifically used, as the new data structures need to use this model frequently.
2020-05-29 01:39:39 -04:00
Dan Funk dba41f4759 Ludicrously stupid launch in a refactor of the way all files work in the system at a time where I crave sleep and peace above all other things.
Added a File class, that we wrap around the FileModel so the api endpoints don't change, but File no longer holds refences to versions or dates of the file_data model, we
figure this out based on a clean database structure.

The ApprovalFile is directly related to the file_data_model - so no chance that a reviewer would review the incorrect version of a file.py

Noticed that our FileType enum called "bpmn" "bpmm", hope this doesn't screw someone up.

Workflows are directly related to the data_models that create the workflow spec it needs.  So the files should always be there.  There are no more hashes, and thus no more hash errors where it can't find the files to rebuild the workflow.py

Not much to report here, other than I broke every single test in the system at one point.  So I'm super concerned about this, and will be testing it a lot before creating the pull request.
2020-05-28 20:03:50 -04:00
Dan Funk ccbf374b40 Loads of bug fixes.
Modifed the request_approval to take a list of arguments, which works better for us... today.
UpdateStudy correctly handles validation.
WorkflowService correctly populates random values from lookup tables.
And several fixes down in Spiffworkflow, including a big bug where only the last item in a decision table made it through.
2020-05-26 20:06:50 -04:00
Dan Funk be057e8758 Adding an "UpdateStudy" task that is able to update the data on the study model, useful for setting core data points on the model, such as setting the Primary Investigator, or altering the Study Title.
Fixing a bug where the validation of forms did not correctly process auto-complete fields.
Fixing a bug where the approvals script and the update study script could not process dot notation correctly.

Moved populate_random_data into the WorkflowService where it makes more sense.
2020-05-25 15:30:06 -04:00
Aaron Louie 481a9ed04c Gets image ids from task data and Injects images into jinja docx 2020-05-19 21:51:54 -04:00
Dan Funk 951710d762 ldap lookup.
Refactored calls into a new lookup_service to keep things tidy.

New keys for all enum/auto-complete fields:
    PROP_OPTIONS_FILE = "spreadsheet.name"
    PROP_OPTIONS_VALUE_COLUMN = "spreadsheet.value.column"
    PROP_OPTIONS_LABEL_COL = "spreadsheet.label.column"
    PROP_LDAP_LOOKUP = "ldap.lookup"
    FIELD_TYPE_AUTO_COMPLETE = "autocomplete"
2020-05-19 16:11:43 -04:00
Dan Funk de435bd961 the heck with camel case, what the heck TypeScript? Get a grip. This is a python API. 2020-05-15 16:38:37 -04:00
Dan Funk 53255ef35e massive overhaul of the Workflow API endpoint.
No Previous Task, No Last Task, No Task List.  Just the current task, and the Navigation.
Use the token endpoint to set the current task, even if it is a "READY" task in the api.
Previous Task can be set by identifying the prior task in the Navigation (I'm hoping)
Prefering camel case to snake case on all new apis.  Maybe clean the rest up later.
2020-05-15 15:54:53 -04:00