Commit Graph

102 Commits

Author SHA1 Message Date
Dan Funk fd0adb1d43 Updated the study status to use a different enumeration. Migration correctly handles modifying the enum.
INCOMPLETE = 'Incomplete in Protocol Builder',
  ACTIVE = 'Active / Ready to roll',
  HOLD = 'On Hold',
  OPEN = 'Open - this study is in progress',
  ABANDONED = 'Abandoned, it got deleted in Protocol Builder'
2020-04-21 17:13:30 -04:00
Dan Funk c8b817e98a Be careful not to clean out all the data when a form is submitted, just merge the data in as it comes. 2020-04-17 13:51:48 -04:00
Dan Funk dc2895cb05 Allow configurators to upload xls files into a workflow for defining enumrations of values for dropdown lists in forms. Fixing lots of tests.
Found a problem where the documentation for elements was being processed BEFORE data was loaded from a script.  There still may be some issues here.

Ran into an issue with circular dependencies - handling it with a new workflow_service, and pulling computational logic out of the api_models - it was the right thing to do.
2020-04-15 11:13:32 -04:00
Dan Funk 316b9e6e8d Fixing a stupid mistake - must have typed a ' mark during commit.
Script should inherit from object.
Updating the ids so I could more easily test and assure things were working correctly.
2020-04-08 13:39:42 -04:00
Dan Funk c79415a794 throw a sensible error when study is not found on get_study (don't 500)
some ugly fixes in the file_service for improving panda output from spreadsheet processing that I need to revist.
now that the spiff-workflow handles multi-instance, we can't have random multi-instance tasks around.
Improved tests around study deletion.
2020-04-08 13:28:43 -04:00
Aaron Louie 519a034d87 Updates last_updated when file data is saved. Returns last_updated as lastModified in response header for file data endpoint. 2020-04-08 12:58:55 -04:00
Dan Funk 534e8493a4 Deleting was broken because statistics were causing an integrity constraint.
Updating IDS bpmn to the latest for testing.
Adding a TODO
2020-04-02 12:58:52 -04:00
Dan Funk 34b6ec92bf updating the API
Removing the call for study/workflows - as workflow information is returned with the study by default.
Fixing a bug in the workflow spec model schema.
2020-03-30 10:12:10 -04:00
Dan Funk 4a916c1ee3 Created a "StudyService" and moved all complex logic around study manipulation out of the study api, and this service, as things were getting complicated. The Workflow Processor no longer creates the WorkflowModel, the study object handles that, and only passes the model into the workflow processor when it is ready to start the workflow.
Created a Study object (seperate from the StudyModel) that can cronstructed on request, and contains a different data structure than we store in the DB. This allows us to return underlying Categories and Workflows in a clean way.

Added a new status to workflows called "not_started", meaning we have not yet instantiated a processor or created a BPMN, they have no version yet and no stored data, just the possiblity of being started.

The Top Level Workflow or "Master" workflow is now a part of the sample data, and loaded at all times.

Removed the ability to "add a workflow to a study" and "remove a workflow from a study", a study contains all possible workflows by definition.

Example data no longer creates users or studies, it just creates the specs.
2020-03-30 08:00:16 -04:00
Dan Funk 57f1fa670e fixing a stupid mistake. 2020-03-27 14:55:53 -04:00
Dan Funk 907e1cbbb3 minor fixes that were breaking when connecting to the front end. 2020-03-27 14:27:50 -04:00
Dan Funk b61a35f956 Merge remote-tracking branch 'origin/master' into feature/status_refactor
Fixing adding a study so all workflows are again added, will add status on those workflows based on output from the master bpmn diagram, which is coming shortly.
2020-03-27 11:55:36 -04:00
Dan Funk c7d2c28178 Vastly more informative ApiError model that provides details on the underlying task where the error occured.
Added a validate_workflow_specification endpoint that allows you to check if the workflow will execute from beginning to end using random data.
Minor fixes to existing bpmns to allow them to pass.
All scripts must include a "do_task_validate_only" that restricts external calls and database modifications, but performs as much logic as possible.
2020-03-27 08:29:31 -04:00
Dan Funk e2c408b70d Removed all self-referential calls in the study_api. One api endpoint should never call another api endpoint. Moved the logic for updating a study to the study Model, rather than checking and setting dictionary values which will become very hard to maintain.
The protocol builder service now returns real models, not dictionaries, forcing proper validation and fail-fast behavior.
Changed the name of the "status" spec, to "top_level_workflow" and removing any connection a workflow or study has with this specification.  It is only unused to determine status in real time, and is not reused or tracked.
Modified the required documents script to return a dictionary and not an array, making it easier to speak to specific values in the BPMN and DMN.
Working on new ways to test the top_level_workflow in the context of updates, this is still a work in progress.
Making use of several modifications to the Spiff library that enables more complex expressions in DMN models. This is evident in the new DMN models for the top_level_workflow
2020-03-26 12:51:53 -04:00
Dan Funk f4342fc785 It became impossible to use the Swagger ui when we started adding authentication to all of the calls. I discovered Connexion and Swagger have a default way of handing JTW authentication and this cleans up our code quite a bit, moves the securing of endpoints into the API Definition, which is quite nice, and removes a whole library dependency (I never get to do that!) I added a SWAGGER_AUTH_KEY that can be used in non-production environments to allow users to quickly authenticate from the Swagger ui. I also removed all api calls to simple little happy api services, because that is just mean and pointless. 2020-03-24 14:15:21 -04:00
Dan Funk 0cd584880f Fixing a bug where the workflow specification can not be deleted. 2020-03-23 12:48:31 -04:00
Dan Funk 967ac65d0e Just raise the ApiError, no need to marshal it. 2020-03-23 12:22:26 -04:00
Dan Funk b35427523d Merge remote-tracking branch 'origin/master' into feature/reference_files
# Conflicts:
#	crc/models/file.py
#	crc/services/file_service.py
#	tests/data/reference/irb_documents.xlsx
#	tests/test_files_api.py
2020-03-20 11:07:55 -04:00
Dan Funk 6e3b6c2635 Assure that files uploaded through web forms and files generated from templates can be cross-referenced to known document requirements from the protocol builder. Configurators can control this by managing an XLS Spreadsheet called "irb_documents.xslx".
Required Documents is becoming complicated, so making this it's own script task, removing it from study_info.py
The file_service is now very aware of this irb_documents file, so it will always need to exist.  We seed this file
during setup, but it can be overwritten by the configurator.
2020-03-19 17:14:20 -04:00
Aaron Louie ac7c7b42c4 Revert "Merge remote-tracking branch 'origin/feature/reference_files' into feature/workflow_spec_categories"
This reverts commit 9a26fc8e80.
2020-03-18 17:05:13 -04:00
Aaron Louie 9a26fc8e80 Merge remote-tracking branch 'origin/feature/reference_files' into feature/workflow_spec_categories 2020-03-18 16:58:57 -04:00
Aaron Louie f0678b43d4 Minor fixes 2020-03-18 16:58:50 -04:00
Dan Funk 02be8ede75 Merge remote-tracking branch 'origin/master' into feature/reference_files 2020-03-18 15:16:34 -04:00
Aaron Louie 532d6b5dd6 Adds CRUD endpoints for workflow spec categories 2020-03-16 16:30:56 -04:00
Aaron Louie 4db456dcea Adds workflow spec category methods 2020-03-16 16:12:39 -04:00
Aaron Louie f9dbbefe89 Adds status spec to study when workflow is added. 2020-03-16 08:05:30 -04:00
Aaron Louie bdd07685c6 Adds status spec when adding a study, and adds/removes workflows from study based on output data from status spec. 2020-03-15 15:54:13 -04:00
Dan Funk 779674ab60 Add the ability to upload and request general reference files by name. These will be used across workflows and will frequently contain lookup tables that can be referenced by various script tasks. 2020-03-13 15:03:57 -04:00
Aaron Louie f177e33ad6 Tiny tweak 2020-03-11 15:31:23 -04:00
Aaron Louie 3262833102 Requires login 2020-03-11 15:27:22 -04:00
Aaron Louie e977ee9e0d Updates task counts when getting workflow 2020-03-11 15:16:42 -04:00
Aaron Louie bbfe9291e0 Adds endpoint to get workflow stats. Adds a test for the endpoint. 2020-03-11 14:28:53 -04:00
Aaron Louie be17c2159c Adds workflow stats and task events models and schemas. Updates workflow stats and logs task completion when task data is updated. 2020-03-10 21:29:57 -04:00
Aaron Louie f1b6718068 Adds task counts 2020-03-10 15:46:14 -04:00
Dan Funk e91d7aff2f Fixes #11: adding a delete endpoint for studies. It won't delete studies that have workflows, you have to delete those first)
Removing the "default" error response from the api.yml, it was all noise.
2020-03-09 15:12:40 -04:00
Aaron Louie acad0023d2 Fixes a bug in get_study_workflows and adds a test covering it 2020-03-06 10:52:41 -05:00
Dan Funk 9fcd6f38f4 Merge remote-tracking branch 'origin/master' into feature/pb_services 2020-03-05 17:13:41 -05:00
Dan Funk 906bacff6a Expose a flag on the workflow model in the api to shown if it is using the latest spec. Added a soft_reset and hard_reset onto the workflow endpoint that will allow you to cause a hard or soft reset. 2020-03-05 16:45:44 -05:00
Dan Funk 697d930eab Modify the workflow processor to accept a workflow model - so it can take on more of the responsibilities of updating this model and managing versions.
Changing the version information so that it includes the numbers of the files used to generate the serialized workflow.
2020-03-05 13:25:28 -05:00
Dan Funk 70611e2c1d Adding the version of the specification used to create a workflow to the workflow api endpoint. Though the exact content of this version is likely to change.
Split the API specific models out from the workflow models to help me keep this straight.
Added tests to help me understand the errors thrown the and resolution path when a workflow specification changes in the midst of a running workflow.
2020-03-05 11:18:20 -05:00
Dan Funk c5cee4761e Improve version handling of files. Consolidate more of this logic in FileService. Place the version on the actual data model, not the file model, so the file model remains the same, and we just version the data associated with it. 2020-03-04 13:40:25 -05:00
Aaron Louie f258090a0b Fixes broken unit test 2020-03-04 09:54:00 -05:00
Aaron Louie 8cbd6f892f WIP: Mocks study details PB endpoint response 2020-03-03 20:56:46 -05:00
Aaron Louie 01720a8bbf Replaces migration and fixes broken tests 2020-03-03 16:02:01 -05:00
Aaron Louie 93168482c2 Merge remote-tracking branch 'origin/master' 2020-03-03 15:46:24 -05:00
Aaron Louie 668de6c4dc Edits Protocol Builder statuses 2020-03-03 15:46:20 -05:00
Dan Funk 94f828dfd6 Adding a simple endpoint that describes what scripts are currently available, along with a brief description. 2020-03-03 15:30:42 -05:00
Dan Funk 7194d7d374 Standardizing the script tasks that can be executed on the server, adding tons of error messages for when things go wrong. All scripts must exist in side of the crc/scripts directory.
Adding a new script that script tasks can use to add in data about the study.

Moving all the test workflow specifications out of the main load.

fixing a pile of tests so they can find workflow specs that are now moved into the test directory.
2020-03-03 13:52:45 -05:00
Dan Funk 5e3fdaaa94 New set of "Tools" api endpoints, that provides a way to quickly render markdown or word documents by uploading json data and a template to populate.
Improved Error messages / Error processing.  You can now just throw an APIError anywhere, and it will be properly serialized and returned.
2020-02-29 17:22:38 -05:00
Aaron Louie 8611a23ad3 Renaming to snake case for consistency 2020-02-28 11:54:11 -05:00