60a10bb688
Marshmallow isn't the right tool when dealing with large models with lots of null values. Rather than fight the process of mamaging the Study Details, I'm letting that fall through, and we can test on an individual value or maybe set up a constants array when that becomes meaningful.
2020-04-03 16:24:38 -04:00
785918cb7f
Be sure the validation process examines the data located in the documentation and correctly handles boolean fields.
2020-04-02 14:47:20 -04:00
17796193de
fixing a bug that was causing failing tests.
...
Adding id and spec_version to the workflow metadata.
Refactoring the processing of the master_spec so that it doesn't polute the workflow database.
Adding tests to assure that the status and counts are updated on the workflow model as users make progress.
2020-03-30 14:01:57 -04:00
4a916c1ee3
Created a "StudyService" and moved all complex logic around study manipulation out of the study api, and this service, as things were getting complicated. The Workflow Processor no longer creates the WorkflowModel, the study object handles that, and only passes the model into the workflow processor when it is ready to start the workflow.
...
Created a Study object (seperate from the StudyModel) that can cronstructed on request, and contains a different data structure than we store in the DB. This allows us to return underlying Categories and Workflows in a clean way.
Added a new status to workflows called "not_started", meaning we have not yet instantiated a processor or created a BPMN, they have no version yet and no stored data, just the possiblity of being started.
The Top Level Workflow or "Master" workflow is now a part of the sample data, and loaded at all times.
Removed the ability to "add a workflow to a study" and "remove a workflow from a study", a study contains all possible workflows by definition.
Example data no longer creates users or studies, it just creates the specs.
2020-03-30 08:00:16 -04:00
c9900d787e
Every good deed goes punished.
2020-03-27 15:48:21 -04:00
c7d2c28178
Vastly more informative ApiError model that provides details on the underlying task where the error occured.
...
Added a validate_workflow_specification endpoint that allows you to check if the workflow will execute from beginning to end using random data.
Minor fixes to existing bpmns to allow them to pass.
All scripts must include a "do_task_validate_only" that restricts external calls and database modifications, but performs as much logic as possible.
2020-03-27 08:29:31 -04:00
b35427523d
Merge remote-tracking branch 'origin/master' into feature/reference_files
...
# Conflicts:
# crc/models/file.py
# crc/services/file_service.py
# tests/data/reference/irb_documents.xlsx
# tests/test_files_api.py
2020-03-20 11:07:55 -04:00
3eb1167b8e
Found a few errors in the sqlalchemy file definition that was causing failures, and had some consistency problems with the IRB Categories file name. The API was bailing out because we had restricted file types to bpmn,svg,or dmn in the connexion config file, I don't restrict this anymore we have plenty of checks elsewhere. Adding xlrd as a dependency - this didn't fail till a push to production.
2020-03-20 08:21:21 -04:00
6e3b6c2635
Assure that files uploaded through web forms and files generated from templates can be cross-referenced to known document requirements from the protocol builder. Configurators can control this by managing an XLS Spreadsheet called "irb_documents.xslx".
...
Required Documents is becoming complicated, so making this it's own script task, removing it from study_info.py
The file_service is now very aware of this irb_documents file, so it will always need to exist. We seed this file
during setup, but it can be overwritten by the configurator.
2020-03-19 17:14:20 -04:00
dbe6701bb2
Removing the doc_types from the protocol builder, as these will eventually contradict what is coming from IRB and should not be used as an reference. Also fixing a failing test and assuring that only one reference file ever exists with a given name.
2020-03-19 10:40:07 -04:00
83d859fd3a
Just merging stuff real quick.
2020-03-18 17:03:36 -04:00
02be8ede75
Merge remote-tracking branch 'origin/master' into feature/reference_files
2020-03-18 15:16:34 -04:00
0cc98616fd
Merge branch 'master' into feature/workflow_spec_categories
2020-03-16 10:25:03 -04:00
cd6a70b747
Fixes code smell issues identified by SonarCloud
2020-03-16 08:31:19 -04:00
779674ab60
Add the ability to upload and request general reference files by name. These will be used across workflows and will frequently contain lookup tables that can be referenced by various script tasks.
2020-03-13 15:03:57 -04:00
902dba7191
Adds is_status flag to workflow specs
2020-03-13 14:56:46 -04:00
05b39df745
Fixes #12 : Catching some specific common errors and re-raising as APIErrors with detailed codes and descriptions to improve debugging. In doing so, improving the error handling in the event a soft-reset causes an immediate error - and resetting to the original version of the specification in these events, to allow users the chance to try a hard reset instead.
2020-03-11 16:33:18 -04:00
906bacff6a
Expose a flag on the workflow model in the api to shown if it is using the latest spec. Added a soft_reset and hard_reset onto the workflow endpoint that will allow you to cause a hard or soft reset.
2020-03-05 16:45:44 -05:00
7b21b78987
Workflow Processor will deserialize workflows using the version of the BPMN files used during creation, but allows for both a soft and hard reset - soft resets will use the new workflow without a restart and will retain full history. A hard-reset will restart the workflow from scratch, but will retain the data from the last completed task. Workflows have a complex version number associated with them that is used during the deserialization to find the correct files.
2020-03-05 15:38:30 -05:00
697d930eab
Modify the workflow processor to accept a workflow model - so it can take on more of the responsibilities of updating this model and managing versions.
...
Changing the version information so that it includes the numbers of the files used to generate the serialized workflow.
2020-03-05 13:25:28 -05:00
70611e2c1d
Adding the version of the specification used to create a workflow to the workflow api endpoint. Though the exact content of this version is likely to change.
...
Split the API specific models out from the workflow models to help me keep this straight.
Added tests to help me understand the errors thrown the and resolution path when a workflow specification changes in the midst of a running workflow.
2020-03-05 11:18:20 -05:00
78b6f040eb
Add the ability to forcibly restart a workflow, while retaining that workflows data.
...
A workflow specification knows it's version number, which is generated by the version of the files that make it up.
A workflow specification version number is the primary file (the lead BPMN) followed by a consistency ordered version each extra file associated with the workflow. A change in any file modifies the specifications version.
2020-03-04 17:08:45 -05:00
c5cee4761e
Improve version handling of files. Consolidate more of this logic in FileService. Place the version on the actual data model, not the file model, so the file model remains the same, and we just version the data associated with it.
2020-03-04 13:40:25 -05:00
7194d7d374
Standardizing the script tasks that can be executed on the server, adding tons of error messages for when things go wrong. All scripts must exist in side of the crc/scripts directory.
...
Adding a new script that script tasks can use to add in data about the study.
Moving all the test workflow specifications out of the main load.
fixing a pile of tests so they can find workflow specs that are now moved into the test directory.
2020-03-03 13:52:45 -05:00
8611a23ad3
Renaming to snake case for consistency
2020-02-28 11:54:11 -05:00
0cc59d0974
Adds study inactive flag. Sets study to inactive if not found in Protocol Builder.
2020-02-27 11:17:58 -05:00
27d7afb656
Adds Protocol Builder models and schemas. Reorganizes and cleans up some files.
2020-02-27 09:54:46 -05:00
3ef4860391
Adds user_uid and investigator_ids fields to Study. Gets studies from protocol builder and adds them if they aren't already in the database
2020-02-26 18:06:51 -05:00
1e8a095760
Fixing a rogue comma that made something a tuple and not a string, which drives me CRAZY.
2020-02-25 12:01:25 -05:00
2cc6010c8d
Protocol builder connections
2020-02-20 13:30:04 -05:00
1a9b5b50e5
Merge branch 'master' of github.com:sartography/cr-connect-workflow
2020-02-18 16:39:11 -05:00
a642593e3d
Adding support to handle Single Sign On (Shibboleth) authentication using Flask SSO and an attribute map that has worked in the past with UVA's implementation. Aside from the new user endpoint, nothing requires authentication, but soon everything will expect it. I'm setting up a backdoor we can use for development and staging that will cause a round-robin affair that should make this relatively painless. Dropped "RestException" as we had two ways or raising errors, and that was silly.
2020-02-18 16:38:56 -05:00
b0b1a6e5e8
Saves form field key
2020-02-11 15:03:25 -05:00
709bae76b2
Removing a rogue comma that was causing havoc. Also, don't fail if a mock already exists in the test database.
2020-02-11 11:11:21 -05:00
9f0eb8477a
Fix for a bug in the File service where it was being overly restrictive.
2020-02-10 16:27:57 -05:00
1d24ebe382
Provide a script for generating word documents from template files. Refractored file managment into a service to make it easier to programatically add files. Modified the workflow_processor to inject the study_id and workflow_id into the running workflow so that this meta-information is avialable at the task level.
2020-02-10 16:19:23 -05:00
Dan Funk
Aaron Louie