Commit Graph

1028 Commits

Author SHA1 Message Date
Aaron Louie ea441dbff5 More updates 2020-06-01 00:28:37 -04:00
Aaron Louie 8092bbe682 Wrong file in wrong place. 2020-06-01 00:17:20 -04:00
Aaron Louie 7351dc4a43 Updates RRT workflow spec files for reals this time 2020-06-01 00:14:09 -04:00
Aaron Louie 2d5eb740ef Updates RRTworkflow spec files 2020-06-01 00:07:50 -04:00
Carlos Lopez bec11980eb Fixing broken test by using proper FileSchema 2020-05-31 22:00:52 -06:00
Carlos Lopez b2e56f797b Converting ApprovalModel to Approval in order to serialize properly the result 2020-05-31 21:02:47 -06:00
Aaron Louie cd16b984e0 Adds workflow spec title to workflow api schema 2020-05-31 22:46:56 -04:00
Aaron Louie f0bd8d4f9e Adds approvals to study schema. Adds approvals endpoint 2020-05-31 22:46:32 -04:00
Dan Funk 9c7de39b09 This adds additional file data details to the study model as well. 2020-05-31 21:15:40 -04:00
Carlos Lopez dd6c1d2b42 Renaming approval files 2020-05-31 18:16:42 -06:00
Carlos Lopez 2e54f07095 Adding more info to files and renaming approval waiting status 2020-05-31 17:24:23 -06:00
Aaron Louie c4a84ac509 Work in progress: Trying to get Swagger to use verify_token_admin to protect admin endpoints. Not working for some reason, though. 2020-05-31 18:01:08 -04:00
Aaron Louie be9b613bbb Refactors user authentication endpoints so we can use the Swagger UI in production mode 2020-05-31 16:49:39 -04:00
Carlos Lopez 26809d1470 Waiting status renaming 2020-05-31 13:35:42 -06:00
Dan Funk 2bc735a3f0 Kelly wrote a beautiful method for resetting the workflow that doesn't loose data when reset inside a parallel task, all I needed to do was use it.
Catching TypeErrors and reporting them back to the UI so we don't 500 in a bad way (but we still 500)
2020-05-31 13:48:00 -04:00
Dan Funk 98fb305868 Run the validation process twice, each time it is requested, first populating everything, and then a second time
using on the required form fields.
2020-05-30 18:43:20 -04:00
Dan Funk 1f0e8741ba Run the validation twice, once completing all of the data, and a second time, completing only the required fields.
Also, add a helper method to reduce boiler plate code around Workflow Exceptions.
2020-05-30 17:26:27 -04:00
Dan Funk 860c475b29 Fill out repeating sections during validation process.
Also, when returning error messages, attempt to include the task data for the task that caused the error.
Also, when attempting to delete any file, respond with an API error explaining the issue, and log the details.
2020-05-30 15:37:04 -04:00
Dan Funk d51fbe51e0 Merge branch 'dev' of github.com:sartography/cr-connect-workflow into dev 2020-05-29 15:18:41 -04:00
Dan Funk 4e4cc7884c Better ldap searching. 2020-05-29 15:17:51 -04:00
Aaron Louie 34b812b74e Updates RRT workflow spec files 2020-05-29 10:49:47 -04:00
Dan Funk 895d2f8852 Slight change to the way the approval model is connected. 2020-05-29 05:04:18 -04:00
Dan Funk 0ba2819867 Merge branch 'dev' into feature/approval_request_script 2020-05-29 04:51:50 -04:00
Dan Funk afb6be7c60 re-working the way the redirects function, so we pass arguments as a get parameter. Just trying to get rid of the weird lag on production.
I noticed the validation sometimes looks ahead for files, so looking at all the tasks now, not just the ready tasks for the lookup field.
Ran into an issue with validation where a workflow model was required, so I create one and delete it.  Another refactor for another day.
2020-05-29 04:42:48 -04:00
Dan Funk 11413838a7 Faster lookup fields. We were parsing the spec each time to get details about how to search. We're just grabbing the workflow id and task id now and building that straight into the full text search index for faster lookups. Should be peppy.
Another speed improvement - data in the FileDataModel is deferred, and not queried until it is specifically used, as the new data structures need to use this model frequently.
2020-05-29 01:39:39 -04:00
Dan Funk aea78de066 Merge remote-tracking branch 'origin/dev' into feature/file_refactor_part2 2020-05-28 20:06:32 -04:00
Dan Funk dba41f4759 Ludicrously stupid launch in a refactor of the way all files work in the system at a time where I crave sleep and peace above all other things.
Added a File class, that we wrap around the FileModel so the api endpoints don't change, but File no longer holds refences to versions or dates of the file_data model, we
figure this out based on a clean database structure.

The ApprovalFile is directly related to the file_data_model - so no chance that a reviewer would review the incorrect version of a file.py

Noticed that our FileType enum called "bpmn" "bpmm", hope this doesn't screw someone up.

Workflows are directly related to the data_models that create the workflow spec it needs.  So the files should always be there.  There are no more hashes, and thus no more hash errors where it can't find the files to rebuild the workflow.py

Not much to report here, other than I broke every single test in the system at one point.  So I'm super concerned about this, and will be testing it a lot before creating the pull request.
2020-05-28 20:03:50 -04:00
Aaron Louie dedec02d48 Updates RRT workflow spec files 2020-05-28 10:43:03 -04:00
Carlos Lopez 41b26e28bd Merge branch 'dev' into feature/approval_request_script 2020-05-28 08:11:31 -06:00
Dan Funk cd7f67ab48 A major refactor of how we search and store files, as there was a lot of confusing bits in here.
From an API point of view you can do the following (and only the following)

/files?workflow_spec_id=x
* You can find all files associated with a workflow_spec_id, and add a file with a workflow_spec_id
/files?workflow_id=x
* You can find all files associated with a workflow_id, and add a file that is directly associated with the workflow
/files?workflow_id=x&form_field_key=y
* You can find all files associated with a form element on a running workflow, and add a new file.
   Note: you can add multiple files to the same form_field_key, IF they have different file names. If the same name, the original file is archived,
   and the new file takes its place.

The study endpoints always return a list of the file metadata associated with the study.  Removed /studies-files, but there is an
endpoint called

/studies/all  - that returns all the studies in the system, and does include their files.

On a deeper level:
 The File model no longer contains:
  - study_id,
  - task_id,
  - form_field_key

Instead, if the file is associated with workflow - then that is the one way it is connected to the study, and we use this relationship to find files for a study.
A file is never associated with a task_id, as these change when the workflow is reloaded.
The form_field_key must match the irb_doc_code, so when requesting files for a form field, we just look up the irb_doc_code.
2020-05-28 08:27:26 -04:00
Aaron Louie 97cdbfce94 Deletes extra line break 2020-05-27 23:48:48 -04:00
Dan Funk 77f72e408f Lookup Service now raises exact matches to the top. Very hackish, but it works. 2020-05-27 14:36:10 -04:00
Carlos Lopez 54640988a7 Update endpoint fixes 2020-05-27 12:06:32 -06:00
Dan Funk d5e075db82 Order search results by relevancy in the lookup service. 2020-05-27 09:47:44 -04:00
Dan Funk 397cb23b52 Is true "true", yes it is true. So true, is "false", so true, it is true. 2020-05-26 23:38:57 -04:00
Dan Funk 0025931a2e Trying hard to figure out why the DCOS servers think the protocol builder is enabled. 2020-05-26 23:18:14 -04:00
Dan Funk 7869fa596e Protocol Builder isn't disabled on the dcos servers, trying to figure out why, and assure it isn't some sort of weird race condition. 2020-05-26 22:42:49 -04:00
Dan Funk eb15d172c1 I absolutely must fix our cascade deletes, but I sort of like the horror such chunks of code cause, we really need to protect these endpoints, and assure they never get called under normal circumstances, and raise some thoughtful errors. 2020-05-26 21:18:09 -04:00
Dan Funk ccbf374b40 Loads of bug fixes.
Modifed the request_approval to take a list of arguments, which works better for us... today.
UpdateStudy correctly handles validation.
WorkflowService correctly populates random values from lookup tables.
And several fixes down in Spiffworkflow, including a big bug where only the last item in a decision table made it through.
2020-05-26 20:06:50 -04:00
Dan Funk 987790164e use redirect, not redirect_url. 2020-05-26 15:09:57 -04:00
Carlos Lopez 7ed9411c74 Handling unavailability of ldap connection 2020-05-26 11:29:24 -06:00
Carlos Lopez 72b59deeaf Completing tests 2020-05-26 10:21:36 -06:00
Dan Funk a14168362a Merge branch 'feature/support_ui_dashboard' into dev 2020-05-25 21:31:16 -04:00
Carlos Lopez 1231b963d0 Enabling ldap lookup 2020-05-25 17:30:16 -06:00
Carlos Lopez 727274ae33 Using full approval payload to update record 2020-05-25 15:40:24 -06:00
Dan Funk 13186176ba Improved LDAP searches, allow filtering on last name as well as uva id. 2020-05-25 16:00:36 -04:00
Dan Funk be057e8758 Adding an "UpdateStudy" task that is able to update the data on the study model, useful for setting core data points on the model, such as setting the Primary Investigator, or altering the Study Title.
Fixing a bug where the validation of forms did not correctly process auto-complete fields.
Fixing a bug where the approvals script and the update study script could not process dot notation correctly.

Moved populate_random_data into the WorkflowService where it makes more sense.
2020-05-25 15:30:06 -04:00
Dan Funk 6cd4ef64d1 Fixing add_study api endpoint, so you can actually add a new "Study" with just some basic information.
Using the LDAP service for checking user details in development mode - even if you are using the back door.
Added a new Flask fucntion load-example-rrt-data that loads the rrt workflow, and not the CRC wrokflows.
Modified the "load-example-data" in the tests to use some test data, rather than loading up all the workflows[
in CRC each time, with a parameter to load crc data if that is required - which is enabled for just a handful of tests.
(Tests run in 1/4 the time now)
2020-05-25 12:29:05 -04:00
Dan Funk 971d9a58e9 As we now have an approval_service.py, I moved all the business logic into this service and out of the request_approval.py script. And moved all tests for these features into a test file for the service. Will make it easier to cross reference what is happening, as everything all happens in one file.
As many of the scripts need to know the workflow, and it's down in a weird parameter, moved this so it is passed in each time.
2020-05-24 16:13:15 -04:00
Carlos Lopez e9bd19b112 Fixing broken test 2020-05-24 01:22:14 -06:00
Carlos Lopez 49eb4b3f98 Making working endpoints for approvals 2020-05-23 23:53:48 -06:00
Aaron Louie 4eaee57076 Apparently, APPLICATION_ROOT does something. 2020-05-24 00:05:13 -04:00
Aaron Louie 68084a84cf Adds base href environment variable. Sets base path for API and all routes from BASE_HREF environment variable. 2020-05-23 22:07:22 -04:00
Carlos Lopez e5e7eb3dd3 Fixing broken tests by making version parameter optional in get file data endpoint 2020-05-23 15:37:03 -06:00
Dan Funk d5c91e575f stuff that might be broken. 2020-05-23 15:21:30 -04:00
Dan Funk d39ef658a2 Made some modifications to the Approval so that it knows exactly what versions of every file are being sent for approval
Added the following columns:
  * date_created - so we know when the file was created
  * renamed workflow_version to just "version", because everything has a version,  this is the version of the request.
  * workflow_hash - this is just a quick way to see what files and versions are associated with the request, it could be factored out.
  * study - a quick relationship link to the study, so that this model is easier to use.
  * workflow - ditto
  * approval_files - these is a list from a new link table that links an approval to specific files and versions.

The RequestApproval is logically sound, but still needs some additional pieces in place to be callable from a BPMN workflow diagram.

Altered the file service to pick up on changes to files vs adding new files, so that versions are picked up correctly as
users modify their submission - adding new files or replacing existing ones.  Deleting files worries me, and I will need to revisit this.

The damn base test keeps giving me a headache, so I made changes there to see if clearing and dropping the database each time won't allow the tests to pass more consistently.

Lots more tests around the file service to make sure it is versioning user uploaded files correctly.

The "Test Request Approval Script" tries to find to assure the correct behavior as this is likely to be called many times repeatedly and with little knowledge of the internal system.  So it should just "do the right thing".
2020-05-23 15:08:17 -04:00
Aaron Louie 6c14248ef9 Adds 'v1.0/' to login route 2020-05-23 14:49:02 -04:00
Aaron Louie 20bf01a888 Adds cascade to study relationship so data loader doesn't freak out. 2020-05-22 22:04:11 -04:00
Dan Funk 49c322177b
Merge pull request #76 from sartography/feature/rrp-endpoints
Feature/rrp endpoints
2020-05-22 19:38:26 -04:00
Dan Funk 148e86bb42 Building out the boilerplate code to make pushing forward on this a little friendlier.
There is an approval api file, and approval model file and an approval test file.
2020-05-22 18:25:00 -04:00
Dan Funk 571c1d7d24 Merge branch 'feature/rrp-endpoints' into feature/disable_protocol_builder 2020-05-22 16:18:33 -04:00
Dan Funk 1017bb1897 On the tools/render_docx api, allow sending the json data in the body, rather than as a ludicrous long get parameter. Silly Dan. 2020-05-22 15:30:22 -04:00
Dan Funk 503c1c8f18 Allow disabling the Protocol Builder
PB_ENABLED can be set to false in the configuration (either in a file called instance/config.py, or as an environment variable)

Added a check in the base_test, to assure that we are always running tests with the test configuration, and bail out otherwise.  Setting TESTING=true as an environment variable will get this, but so well the correct ordering of imports. Just be dead certain the first file every test file imports is base_test.py.

Aaron was right, and we call the Protocol Builder in all kinds of awful places.  But we don't do this now.  So Carlos, you should have the ability to reuse a lot of the logic in the study_service now.

I dropped the poorly named "study-update" endpoint completely.  We weren't using it. POST and PUT to Study still work just fine for doing exactly that.

All the tests now run and pass with the Protocol builder disabled. Tests that specifically check PB behavior turn it back on for the test, or mock it out.
2020-05-22 14:37:49 -04:00
Dan Funk 992a85e9a5 Rough idea of what the Approvals model will look like. 2020-05-22 11:56:43 -04:00
Carlos Lopez a0c884499e Merge branch 'feature/rrp-endpoints' of github.com:sartography/cr-connect-workflow into feature/rrp-endpoints 2020-05-22 09:46:25 -06:00
Carlos Lopez 1ed7930aab Endpoint for studies with files 2020-05-22 09:46:03 -06:00
Dan Funk 8b3dc8e0ca Merge branch 'dev' into feature/rrp-endpoints 2020-05-22 11:41:11 -04:00
Dan Funk b490005af7 dropping the remaining config stuff for flask_sso.
updaing the user 'sso' endpoint to provide additional information for debugging.
Pulling information from ldap to stay super consistent on where we get our information.
2020-05-22 09:50:18 -04:00
Dan Funk 1cc9a60bfe clearing out the remaining references to the flask_sso library. 2020-05-22 08:01:33 -04:00
Dan Funk 4627318818 Dropping flask_sso library in favor of reading from the headers directly. Updating login to read from ldap once it has the user_id. Adding more information to the sso endpoint. 2020-05-22 07:55:58 -04:00
Dan Funk 0265db7146 adding an /sso endpoint for testing. 2020-05-21 16:02:45 -04:00
Dan Funk 4628834106 just a few more logging details. 2020-05-21 12:11:35 -04:00
Carlos Lopez abc90cdcb1 Adding serialiazer for study files 2020-05-20 15:10:22 -06:00
Aaron Louie 48f9873548 Adding yet another flush, because Travis builds keep failing due to database race condition issues in this method. 2020-05-20 10:02:30 -04:00
Aaron Louie 58189285ad Cleans up 2020-05-20 00:12:48 -04:00
Aaron Louie 2b5687c3a3 Fixes pernicious bug where template document versions were not being updated properly, and template completion script was not honoring version numbers 2020-05-20 00:10:32 -04:00
Aaron Louie 481a9ed04c Gets image ids from task data and Injects images into jinja docx 2020-05-19 21:51:54 -04:00
Dan Funk c4f2bd8dc6 Quick cleanup, adding a space 2020-05-19 16:23:20 -04:00
Dan Funk 951710d762 ldap lookup.
Refactored calls into a new lookup_service to keep things tidy.

New keys for all enum/auto-complete fields:
    PROP_OPTIONS_FILE = "spreadsheet.name"
    PROP_OPTIONS_VALUE_COLUMN = "spreadsheet.value.column"
    PROP_OPTIONS_LABEL_COL = "spreadsheet.label.column"
    PROP_LDAP_LOOKUP = "ldap.lookup"
    FIELD_TYPE_AUTO_COMPLETE = "autocomplete"
2020-05-19 16:11:43 -04:00
Dan Funk 5c5c2a7312 Assure that new lines entered in text-fields are correctly added to the final word document. 2020-05-18 11:55:10 -04:00
Dan Funk fc080ce7a8 Don't reset the token to something that already has a token. Terrible things happen and we hit a recursion. 2020-05-17 13:29:36 -04:00
Dan Funk fafdf56ad2 Merge remote-tracking branch 'origin/testing' into feature/navigation 2020-05-16 15:46:14 -04:00
Dan Funk 43bf8f5337 Fixing a bug with navigation where elements went missing past exclusive gateways on subsequent forks. 2020-05-16 15:33:06 -04:00
Aaron Louie 091a1729f8 Removes CORS logging. Allows only specific origins 2020-05-16 11:36:48 -04:00
Dan Funk 7d3f139633 Merge branch 'feature/navigation' of github.com:sartography/cr-connect-workflow into feature/navigation 2020-05-15 16:39:25 -04:00
Dan Funk de435bd961 the heck with camel case, what the heck TypeScript? Get a grip. This is a python API. 2020-05-15 16:38:37 -04:00
Aaron Louie 0f1fbd0249 Merge branch 'dev' into feature/navigation 2020-05-15 16:10:40 -04:00
Dan Funk 53255ef35e massive overhaul of the Workflow API endpoint.
No Previous Task, No Last Task, No Task List.  Just the current task, and the Navigation.
Use the token endpoint to set the current task, even if it is a "READY" task in the api.
Previous Task can be set by identifying the prior task in the Navigation (I'm hoping)
Prefering camel case to snake case on all new apis.  Maybe clean the rest up later.
2020-05-15 15:54:53 -04:00
Dan Funk b63ee8159e We now only return the ready user tasks, not all tasks, and even then the ready user tasks don't come back with the forms and details, just the bare minimum. Speeds things up considerably, and most of this information wasn't used anyway. 2020-05-14 17:13:47 -04:00
Dan Funk f2c9fd5fc4 adding a default url. And some debugging information to see if we hit he endpoint in the logs. 2020-05-14 15:07:05 -04:00
Dan Funk 6d4348d644 Fixing some failing tests. Moved the task properties into a dictionary, but moving the form field properties to a dictionary will be a larger effort that we don't want to get into on either the back or front end right this moment. 2020-05-14 14:39:14 -04:00
Dan Funk 55a1850e7c adding a navigation component to the Workflow Model.
running all extension/properties through the Jinja template processor so you can have custom display names using data, very helpful for building multi-instance displays.
Properties was returned as an array of key/value pairs, which is just mean.  Switched this to a dictionary.
2020-05-14 13:43:23 -04:00
Aaron Louie 3ef07bd8c9 Allows all origins (for troubleshooting) 2020-05-13 12:32:57 -04:00
Aaron Louie caf791118f Allows all resources 2020-05-13 11:00:43 -04:00
Aaron Louie fbdff7a5e6 Merge branch 'master' into dev 2020-05-12 22:46:20 -04:00
Aaron Louie cdfe45dc00 Turns on flask_cors debugging to troubleshoot CORS issue 2020-05-12 22:42:02 -04:00
Aaron Louie 77b1908373 Escapes dot characters in CORS domains 2020-05-12 14:27:17 -04:00
Aaron Louie ac3f0b401b Converts list of allowed origins to regexes 2020-05-12 12:23:47 -04:00
Dan Funk e723992fde Found a number of bugs with the parallel multi-instance - pulling in some recent changes from Spiffworkflow to open things up a bit more to allow functional jumping between tasks. 2020-05-12 12:23:43 -04:00
Aaron Louie 73ce70b574 Allows CORS connections from frontend 2020-05-12 10:43:52 -04:00
Dan Funk 99503fb832 missed a file in the last commit. 2020-05-11 17:42:24 -04:00
Dan Funk b7c11fd893 Merge branch 'master' into feature/investigators_reference_file 2020-05-11 17:36:37 -04:00
Dan Funk 02f8764056 Updated to use the latest script engine / evaluation engine that creates a single location where all values used in BPMN/DMN are processed. Right now this is a python based interpreter, but we will eventually base this on FEEL expressions.
The validation process needs to take the api model into account so we catch errors with bad file names.
2020-05-11 17:04:05 -04:00
Dan Funk c34000d369
Merge branch 'master' into feature/investigators_reference_file 2020-05-07 16:47:24 -04:00
Dan Funk da7cae51b8 Adding a new reference file that provides greater details about the investigators related to a study.
Improving the study_info script documentation to provide detailed examples of values returned based on arguments.
Making the tests a little more targetted and less subject to breaking through better mocks.
Allow all tests to pass even when ther protocol builder mock isn't running locally.
Removing the duplication of reference files in tests and static, as this seems silly to me at the moment.
2020-05-07 13:57:24 -04:00
Aaron Louie 9afcd753c4 No longer requires task_id 2020-05-06 14:27:28 -04:00
Dan Funk 1571986c0e I had to give up and live with the idea that we can only render documentation on the current task, not on the previous or next tasks. I think this is ok. If you want to view a task, you need to make it the active task to assure all the parts and pieces are in place. 2020-05-06 13:01:38 -04:00
Dan Funk 8ded625c7d Merge remote-tracking branch 'origin/chore/update_specs' into feature/previous_task
# Conflicts:
#	Pipfile.lock

Assuring that all documents from the xls spreadsheet are loaded when doing validations.
Fixing some failed tests.
2020-05-06 11:46:19 -04:00
Dan Funk 07e58e923d Merge remote-tracking branch 'origin/chore/update_specs' into feature/previous_task
# Conflicts:
#	Pipfile.lock

Assuring that all documents from the xls spreadsheet are loaded when doing validations.
2020-05-06 11:25:50 -04:00
Dan Funk 9629b36e92 Setting JSON_SORT_KEYS to false, assuring that Flask does not resort all data returned to the front end.
Updating Spiff Workflow which has some critical behavioral changes around MultiInstance.
2020-05-06 10:59:49 -04:00
Aaron Louie 6c36a32411 Adds DSP 2020-05-06 10:37:43 -04:00
Aaron Louie a8827d78c4 Fixes typo 2020-05-05 20:49:07 -04:00
Aaron Louie 4ecb0cb3a3 Updates BPMN files 2020-05-05 16:15:38 -04:00
Dan Funk ce76b0d3e8 Merge branch 'feature/protocol_status' into feature/previous_task 2020-05-04 11:12:12 -04:00
Dan Funk 714b5f3be0 Merge branch 'feature/protocol_status' into feature/previous_task
# Conflicts:
#	crc/services/study_service.py
2020-05-04 11:08:36 -04:00
Dan Funk 2699f5c65c Refactor the stats models, and assure they are very correct across all tests with the workflow api.
I noticed we were saving the workflow every time we loaded it up, rather than only when we were making changes to it.  Refactored this to be a little more careful.
Centralized the saving of the workflow into one location in the processor, so we can make sure we update all the details about that workflow every time we save.
The workflow service has a method that will log any task action taken in a consistent way.
The stats models were removed from the API completely.  Will wait for a use case for dealing with this later.
2020-05-04 10:57:09 -04:00
Aaron Louie 28d5727851 Merge branch 'master' into feature/protocol_status 2020-05-01 22:55:59 -04:00
Aaron Louie 098cf10926 Fixes Jinja code to match new structure of StudyInfo documents 2020-05-01 22:49:52 -04:00
Aaron Louie 6a96ccffa2 Updates pipenv hashes. Fixes typo in SpiffWorkflow file name 2020-05-01 21:54:01 -04:00
Dan Funk 1f5002680a Initial work on a "Previous" task. 2020-05-01 12:11:39 -04:00
Dan Funk bec59a71d7 Deleteing stuff is a damn mess, but this is a little cleaner. 2020-04-29 16:07:39 -04:00
Dan Funk f1f8b91c9c Refactor the document details scripts. Now there is one script, it returns data in a consistent format, and has all the details required. The script is located in StudyInfo, with the argument documents. Make note that it returns a dictionary of ALL the documents, with a field to mark which ones are required according to the protocol builder. Others may become required if a workflow determines such, in which case the workflow will enforce this, and the document will have a count > 0, and additional details in a list of files within the document. I modified the XLS file to use lower case variable names, because it disturbed me, and we have to reference them frequently. Removed devious "as_object" variable on get_required_docs, so it behaves like the other methods all the time, and returns a dictionary. All the core business logic for finding the documents list now resides in the StudyService.
Because this changes the endpoint for all existing document details, I've modified all the test and static bpmn files to use the new format.
Shorting up the SponsorsList.xls file makes for slightly faster tests. seems senseless to load 5000 everytime we reset the data.
Tried to test all of this carefully in the test_study_details_documents.py test.
2020-04-29 15:08:11 -04:00
Aaron Louie beb86f0453 Adds protocol script to study service 2020-04-29 10:21:24 -04:00
Dan Funk 13333fb306 Adding an API endpoint that allows setting the token on the workflow to a specific task.
Added error checking such that attempting to submit data for a task that is not in the "READY" state throws an error message.
For some reason I'm getting lots of errors in the tests as they try to hit API endpoints they were not hitting before, so adding a number of mocks to some of the study service tests.
2020-04-28 17:25:53 -04:00
Dan Funk 0088364b1d Merge branch 'master' of github.com:sartography/cr-connect-workflow 2020-04-28 13:48:52 -04:00
Dan Funk 3e3a249e3c Verifying Sub-Process works, and adding a field to expose a hint as to the sub-process in which the task occurs.
Because the name field is now used to expose workflow/sub-process information on tasks, we can't use it to store the workflow_version, so that is now just stored on the database model.  Which is much cleaner and removes a duplication.
2020-04-28 13:48:44 -04:00
Aaron Louie 663a1a00a8 Fixes file name so it matches workflow spec ID 2020-04-28 11:50:10 -04:00
Aaron Louie 9311a2502a Fixes broken Formly expression 2020-04-28 09:25:44 -04:00
Aaron Louie 52fc3c2c2f Removes xlsx lock file. Adds LibreOffice lock files to gitignore 2020-04-27 22:58:12 -04:00
Aaron Louie 496e5b7719 Updates all workflow specs to match staging 2020-04-27 22:54:05 -04:00
Dan Funk 447f4013f8 Assure that a hard-reset sticks, and the system is properly updated. 2020-04-27 16:08:23 -04:00
Dan Funk 1b9743a4d1 Assure that if a form has an enumeration it errors out if that enumeration is empty. 2020-04-27 15:10:09 -04:00
Aaron Louie 8ed520c6f1 Removes hidden workflows 2020-04-24 09:45:55 -04:00
Aaron Louie c85173de88 Sorts approvals by display order 2020-04-24 08:54:14 -04:00
Dan Funk 1ccedbc9fd Merge branch 'master' of github.com:sartography/cr-connect-workflow 2020-04-24 07:01:40 -04:00
Dan Funk 12eb039bc9 Server isn't erroring out, but can't find the lookup table id in the database, so trying to use the in-memory model instead, to give things time to get to the database. Really unsure what is happening here. Hard to see in the database. 2020-04-24 07:01:32 -04:00
Aaron Louie af1bb9f80d Adds more useful metadata to approvals and documents status scripts. Fleshes out and pretties up Documents & Approvals screen 2020-04-23 23:32:20 -04:00
Aaron Louie 47de010a88 Puts data from sequential calls to StudyInfo into the right place. Sets the required document flag correctly. 2020-04-23 21:02:08 -04:00
Aaron Louie d91f690388 Adds documents_status StudyInfo script. Adds Documents & Approvals workflow spec. 2020-04-23 19:25:01 -04:00
Aaron Louie 2e601719ad Fixes broken spec 2020-04-23 17:10:10 -04:00
Dan Funk 08140eca17 Merge branch 'master' of github.com:sartography/cr-connect-workflow 2020-04-23 15:01:02 -04:00
Dan Funk 3aeb7ad116 Server isn't erroring out, but can't find the lookup table id in the database, so trying to use the in-memory model instead, to give things time to get to the database. Really unsure what is happening here. Hard to see in the database. 2020-04-23 14:58:17 -04:00
Aaron Louie 796c109611 Adds approvals to study service 2020-04-23 14:40:05 -04:00
Dan Funk b5b46b7c2c better overall search results for type ahead. Still dealing with stop words failing. 2020-04-23 12:05:08 -04:00
Dan Funk 65b29e1a9d Don't just bomb out as soon as someone types an empty string. 2020-04-23 09:44:11 -04:00
Dan Funk 7b085c9c9d Adding an API Endpoint that will return a list of LookupValues that match a given query - can be used to populate an auto-complete table. 2020-04-22 19:40:40 -04:00
Dan Funk 6de8c8b977 Create lookup tables for XSL files referenced in workflows so we can do full text searches and populate lists on the fly quickly. 2020-04-22 15:37:02 -04:00
Dan Funk 79d9a66ce3
Merge pull request #32 from sartography/feature/new_study_states
Updated the study status to use a different enumeration.  Migration c…
2020-04-21 17:21:10 -04:00
Dan Funk fd0adb1d43 Updated the study status to use a different enumeration. Migration correctly handles modifying the enum.
INCOMPLETE = 'Incomplete in Protocol Builder',
  ACTIVE = 'Active / Ready to roll',
  HOLD = 'On Hold',
  OPEN = 'Open - this study is in progress',
  ABANDONED = 'Abandoned, it got deleted in Protocol Builder'
2020-04-21 17:13:30 -04:00
Aaron Louie b10f72c085 Updates workflow spec. Removes old DMN files 2020-04-21 14:19:57 -04:00
Dan Funk 0a74bf8c44 We can now collect, and provide "extension properties" on a task as set in the camunda modeler.
These are provided as "properties" on a task, and are identical in structure to properties on a form field.
2020-04-21 12:07:59 -04:00
Dan Funk ec112f52be Make use of cleaner data provided by Spiffworkflow about multi-instance settings. 2020-04-21 11:43:43 -04:00
Dan Funk ee999a0f15 fixing a bunch of stupid mistakes because I am tried. 2020-04-20 20:28:12 -04:00
Dan Funk d2b31bb3d1
Merge pull request #31 from sartography/feature_personnel_multi_instance
Feature personnel multi instance
2020-04-20 16:17:43 -04:00
Dan Funk edbd75bb75 Connect LDAP Requests to the StudyInfo service so we get back additional details. 2020-04-20 16:02:13 -04:00
Dan Funk 2d3402a719 Ldap Service with Test and mocks.
LDAP_URL can be set in an environment variable.
2020-04-20 15:16:33 -04:00
Aaron Louie 194ed55801 Merge remote-tracking branch 'origin/master' 2020-04-20 12:04:18 -04:00
Aaron Louie 00312d244a Cleans up debugging config print 2020-04-20 12:04:10 -04:00
Dan Funk d3dd9dcc25 Functional multi-instance - works with no changes to the front end - though I've added some attributes to task so we could give people a sense of how many iterations they will go through. 2020-04-19 15:14:10 -04:00
Dan Funk b099cac174 Merge branch 'master' into feature_personnel_multi_instance 2020-04-17 14:26:23 -04:00
Dan Funk c8b817e98a Be careful not to clean out all the data when a form is submitted, just merge the data in as it comes. 2020-04-17 13:51:48 -04:00
Dan Funk 241980f98f If you name add a file to a workflow that has the exact same name as a Task Spec's ID, and an extension of "md", it wll use that file as the markdown content, and ignore the markdown in the documentation on the task spec.
Moving the primary process id from the workflow model to the file model, and assuring it is updated properly.  This was causing a bug that would "lose" the workflow.
2020-04-17 13:30:32 -04:00
Dan Funk 565ad01393 Merge branch 'master' into feature_personnel_multi_instance 2020-04-16 11:22:50 -04:00
Aaron Louie 35fa241a4c Printing out some more stuff for debugging 2020-04-15 14:43:09 -04:00
Dan Funk 678a45fa76 Merge branch 'master' into feature_personnel_multi_instance 2020-04-15 12:42:36 -04:00
Dan Funk a5f5468e9a
Merge branch 'master' into feature/customer_lookup 2020-04-15 12:41:35 -04:00
Dan Funk 288efff364 Merge branch 'master' into feature_personnel_multi_instance 2020-04-15 12:36:50 -04:00
Dan Funk dc2895cb05 Allow configurators to upload xls files into a workflow for defining enumrations of values for dropdown lists in forms. Fixing lots of tests.
Found a problem where the documentation for elements was being processed BEFORE data was loaded from a script.  There still may be some issues here.

Ran into an issue with circular dependencies - handling it with a new workflow_service, and pulling computational logic out of the api_models - it was the right thing to do.
2020-04-15 11:13:32 -04:00
Aaron Louie faf4c0df97 Updates BPMN and DMN files 2020-04-15 10:58:13 -04:00
Dan Funk aa17e30762 Updating libraries, modifying study_info script to save data in a consistent way, waiting on more fixes in spiffworkflow before continuing. 2020-04-13 16:23:31 -04:00
Aaron Louie 9a9922c473 Adds display order to Workflow Spec 2020-04-09 14:25:14 -04:00
Dan Funk 316b9e6e8d Fixing a stupid mistake - must have typed a ' mark during commit.
Script should inherit from object.
Updating the ids so I could more easily test and assure things were working correctly.
2020-04-08 13:39:42 -04:00
Dan Funk c79415a794 throw a sensible error when study is not found on get_study (don't 500)
some ugly fixes in the file_service for improving panda output from spreadsheet processing that I need to revist.
now that the spiff-workflow handles multi-instance, we can't have random multi-instance tasks around.
Improved tests around study deletion.
2020-04-08 13:28:43 -04:00
Aaron Louie 519a034d87 Updates last_updated when file data is saved. Returns last_updated as lastModified in response header for file data endpoint. 2020-04-08 12:58:55 -04:00
Dan Funk 9bf5c9e727 Fixing an error that would prevent the correct counts from displaying if the record didn't come back from the protcol builder. 2020-04-07 14:14:43 -04:00
Dan Funk 697127660f Assure that all script tasks place data in a dictionary that is named exactly the same as the class - which is also the same as the Script tag. 2020-04-07 14:09:21 -04:00
Dan Funk c6b6ee5d70 Renamed the required_docs script to just "documents", and it returns all documented in the irb_documents look up table indexed on the "Code" - so details become available in the task data like "documents.IRB_INFOSEC_DOC.required".
Updated the irb_documents with shorter code names, thanks to Alex. Re-worked the DMN models so they can properly read from this new datastructure.
2020-04-06 16:56:00 -04:00
Dan Funk d58a90a727 Merge branch 'master' of github.com:sartography/cr-connect-workflow 2020-04-06 13:24:01 -04:00
Aaron Louie e6550b7e39 Fixes Formly expression syntax error 2020-04-06 13:22:49 -04:00
Aaron Louie 419d619efa Updates BPMN & DMN files with latest from staging 2020-04-06 13:12:34 -04:00
Dan Funk e283b86466 Fixing a bug with deleting a study. 2020-04-06 13:08:17 -04:00
Dan Funk a322801c91 Allow a study to be deleted, even if some statistics are laying around. 2020-04-03 16:41:16 -04:00
Dan Funk 60a10bb688 Marshmallow isn't the right tool when dealing with large models with lots of null values. Rather than fight the process of mamaging the Study Details, I'm letting that fall through, and we can test on an individual value or maybe set up a constants array when that becomes meaningful. 2020-04-03 16:24:38 -04:00
Dan Funk fe40be752f fixing a bug in the Protocol builder schema, more to fix there, and deserves a test, but this corrects the investigators 2020-04-03 13:52:12 -04:00
Dan Funk b39e4b382f Catch more template processing errors. 2020-04-02 15:06:00 -04:00
Dan Funk 785918cb7f Be sure the validation process examines the data located in the documentation and correctly handles boolean fields. 2020-04-02 14:47:20 -04:00
Dan Funk 534e8493a4 Deleting was broken because statistics were causing an integrity constraint.
Updating IDS bpmn to the latest for testing.
Adding a TODO
2020-04-02 12:58:52 -04:00
Dan Funk 17796193de fixing a bug that was causing failing tests.
Adding id and spec_version to the workflow metadata.
Refactoring the processing of the master_spec so that it doesn't polute the workflow database.
Adding tests to assure that the status and counts are updated on the workflow model as users make progress.
2020-03-30 14:01:57 -04:00
Dan Funk 34b6ec92bf updating the API
Removing the call for study/workflows - as workflow information is returned with the study by default.
Fixing a bug in the workflow spec model schema.
2020-03-30 10:12:10 -04:00
Dan Funk 4a916c1ee3 Created a "StudyService" and moved all complex logic around study manipulation out of the study api, and this service, as things were getting complicated. The Workflow Processor no longer creates the WorkflowModel, the study object handles that, and only passes the model into the workflow processor when it is ready to start the workflow.
Created a Study object (seperate from the StudyModel) that can cronstructed on request, and contains a different data structure than we store in the DB. This allows us to return underlying Categories and Workflows in a clean way.

Added a new status to workflows called "not_started", meaning we have not yet instantiated a processor or created a BPMN, they have no version yet and no stored data, just the possiblity of being started.

The Top Level Workflow or "Master" workflow is now a part of the sample data, and loaded at all times.

Removed the ability to "add a workflow to a study" and "remove a workflow from a study", a study contains all possible workflows by definition.

Example data no longer creates users or studies, it just creates the specs.
2020-03-30 08:00:16 -04:00
Dan Funk c9900d787e Every good deed goes punished. 2020-03-27 15:48:21 -04:00
Aaron Louie 6ebd4dce42 WIP: Adds default workflow spec categories 2020-03-27 15:32:07 -04:00
Dan Funk 57f1fa670e fixing a stupid mistake. 2020-03-27 14:55:53 -04:00
Dan Funk 907e1cbbb3 minor fixes that were breaking when connecting to the front end. 2020-03-27 14:27:50 -04:00
Dan Funk b61a35f956 Merge remote-tracking branch 'origin/master' into feature/status_refactor
Fixing adding a study so all workflows are again added, will add status on those workflows based on output from the master bpmn diagram, which is coming shortly.
2020-03-27 11:55:36 -04:00
Dan Funk c7d2c28178 Vastly more informative ApiError model that provides details on the underlying task where the error occured.
Added a validate_workflow_specification endpoint that allows you to check if the workflow will execute from beginning to end using random data.
Minor fixes to existing bpmns to allow them to pass.
All scripts must include a "do_task_validate_only" that restricts external calls and database modifications, but performs as much logic as possible.
2020-03-27 08:29:31 -04:00
Dan Funk 6c832829b0 Merge remote-tracking branch 'origin' into feature/status_refactor 2020-03-26 20:47:37 -04:00
Aaron Louie f77ebe3177 Adds display order to categories 2020-03-26 15:29:52 -04:00
Dan Funk e2c408b70d Removed all self-referential calls in the study_api. One api endpoint should never call another api endpoint. Moved the logic for updating a study to the study Model, rather than checking and setting dictionary values which will become very hard to maintain.
The protocol builder service now returns real models, not dictionaries, forcing proper validation and fail-fast behavior.
Changed the name of the "status" spec, to "top_level_workflow" and removing any connection a workflow or study has with this specification.  It is only unused to determine status in real time, and is not reused or tracked.
Modified the required documents script to return a dictionary and not an array, making it easier to speak to specific values in the BPMN and DMN.
Working on new ways to test the top_level_workflow in the context of updates, this is still a work in progress.
Making use of several modifications to the Spiff library that enables more complex expressions in DMN models. This is evident in the new DMN models for the top_level_workflow
2020-03-26 12:51:53 -04:00
Aaron Louie dd5971bea9 Updates workflow specs for data loader 2020-03-25 11:13:52 -04:00
Dan Funk bd92c230e4 Fix the API so we are clear what is being returned when updating a file. 2020-03-25 08:06:58 -04:00
Dan Funk f4342fc785 It became impossible to use the Swagger ui when we started adding authentication to all of the calls. I discovered Connexion and Swagger have a default way of handing JTW authentication and this cleans up our code quite a bit, moves the securing of endpoints into the API Definition, which is quite nice, and removes a whole library dependency (I never get to do that!) I added a SWAGGER_AUTH_KEY that can be used in non-production environments to allow users to quickly authenticate from the Swagger ui. I also removed all api calls to simple little happy api services, because that is just mean and pointless. 2020-03-24 14:15:21 -04:00
Dan Funk 0cd584880f Fixing a bug where the workflow specification can not be deleted. 2020-03-23 12:48:31 -04:00
Dan Funk 967ac65d0e Just raise the ApiError, no need to marshal it. 2020-03-23 12:22:26 -04:00
Dan Funk b35427523d Merge remote-tracking branch 'origin/master' into feature/reference_files
# Conflicts:
#	crc/models/file.py
#	crc/services/file_service.py
#	tests/data/reference/irb_documents.xlsx
#	tests/test_files_api.py
2020-03-20 11:07:55 -04:00
Dan Funk 16c6ba9661 Removing unneeded files. 2020-03-20 08:33:37 -04:00
Dan Funk 3eb1167b8e Found a few errors in the sqlalchemy file definition that was causing failures, and had some consistency problems with the IRB Categories file name. The API was bailing out because we had restricted file types to bpmn,svg,or dmn in the connexion config file, I don't restrict this anymore we have plenty of checks elsewhere. Adding xlrd as a dependency - this didn't fail till a push to production. 2020-03-20 08:21:21 -04:00
Dan Funk 13a688048c Fixing failed tests, setting unknown=EXCLUDE on the workflow spec model schema so that it can handle getting back the dump-only workflow_spec_category and just ignore it. 2020-03-20 07:54:01 -04:00
Dan Funk e529e149f6 Assure that reference file exists on data-reload. 2020-03-20 07:41:21 -04:00
Dan Funk 6e3b6c2635 Assure that files uploaded through web forms and files generated from templates can be cross-referenced to known document requirements from the protocol builder. Configurators can control this by managing an XLS Spreadsheet called "irb_documents.xslx".
Required Documents is becoming complicated, so making this it's own script task, removing it from study_info.py
The file_service is now very aware of this irb_documents file, so it will always need to exist.  We seed this file
during setup, but it can be overwritten by the configurator.
2020-03-19 17:14:20 -04:00
Dan Funk dbe6701bb2 Removing the doc_types from the protocol builder, as these will eventually contradict what is coming from IRB and should not be used as an reference. Also fixing a failing test and assuring that only one reference file ever exists with a given name. 2020-03-19 10:40:07 -04:00
Dan Funk 560b8a8782 Mergers details from the irb_documents.xlsx into the values returned from the Protocol Builder to create a more complete picture of required document details. 2020-03-19 10:23:50 -04:00
Aaron Louie ac7c7b42c4 Revert "Merge remote-tracking branch 'origin/feature/reference_files' into feature/workflow_spec_categories"
This reverts commit 9a26fc8e80.
2020-03-18 17:05:13 -04:00
Dan Funk 83d859fd3a Just merging stuff real quick. 2020-03-18 17:03:36 -04:00
Aaron Louie 9a26fc8e80 Merge remote-tracking branch 'origin/feature/reference_files' into feature/workflow_spec_categories 2020-03-18 16:58:57 -04:00
Aaron Louie f0678b43d4 Minor fixes 2020-03-18 16:58:50 -04:00
Dan Funk 02be8ede75 Merge remote-tracking branch 'origin/master' into feature/reference_files 2020-03-18 15:16:34 -04:00
Aaron Louie 853b2b2502 Populates category 2020-03-18 15:16:15 -04:00
Aaron Louie 532d6b5dd6 Adds CRUD endpoints for workflow spec categories 2020-03-16 16:30:56 -04:00
Aaron Louie 4db456dcea Adds workflow spec category methods 2020-03-16 16:12:39 -04:00
Aaron Louie cbd8d2f46c Silences SQLALCHEMY_TRACK_MODIFICATIONS warning 2020-03-16 13:47:17 -04:00
Aaron Louie 40e12f5ab5 Resolves marshmallow_sqlalchemy.ModelSchema deprecation warning 2020-03-16 13:37:31 -04:00
Aaron Louie 7d31bc24eb Attempting to address "FutureWarning: Implicit (flask) JSON serialization" warning 2020-03-16 13:24:15 -04:00
Aaron Louie a98251237b Adds workflow spec category to API 2020-03-16 12:10:32 -04:00
Aaron Louie 6af1693bdc Fixes SonarCloud issue L23. Verifies that study ids match. 2020-03-16 10:37:06 -04:00
Aaron Louie 0cc98616fd Merge branch 'master' into feature/workflow_spec_categories 2020-03-16 10:25:03 -04:00
Aaron Louie 3885bc7624 Merge remote-tracking branch 'origin/master' into feature/delete_study 2020-03-16 08:49:44 -04:00
Aaron Louie cd6a70b747 Fixes code smell issues identified by SonarCloud 2020-03-16 08:31:19 -04:00
Aaron Louie f9dbbefe89 Adds status spec to study when workflow is added. 2020-03-16 08:05:30 -04:00
Aaron Louie bdd07685c6 Adds status spec when adding a study, and adds/removes workflows from study based on output data from status spec. 2020-03-15 15:54:13 -04:00
Aaron Louie e4af6f77d0 Adds is_status flag to workflow_spec, status_spec_id & version to study, and is_active flag to WorkflowApi 2020-03-15 15:52:59 -04:00
Dan Funk 779674ab60 Add the ability to upload and request general reference files by name. These will be used across workflows and will frequently contain lookup tables that can be referenced by various script tasks. 2020-03-13 15:03:57 -04:00
Aaron Louie 902dba7191 Adds is_status flag to workflow specs 2020-03-13 14:56:46 -04:00
Dan Funk 05b39df745 Fixes #12: Catching some specific common errors and re-raising as APIErrors with detailed codes and descriptions to improve debugging. In doing so, improving the error handling in the event a soft-reset causes an immediate error - and resetting to the original version of the specification in these events, to allow users the chance to try a hard reset instead. 2020-03-11 16:33:18 -04:00
Aaron Louie f177e33ad6 Tiny tweak 2020-03-11 15:31:23 -04:00
Aaron Louie 3262833102 Requires login 2020-03-11 15:27:22 -04:00
Aaron Louie e977ee9e0d Updates task counts when getting workflow 2020-03-11 15:16:42 -04:00
Aaron Louie bbfe9291e0 Adds endpoint to get workflow stats. Adds a test for the endpoint. 2020-03-11 14:28:53 -04:00
Aaron Louie be17c2159c Adds workflow stats and task events models and schemas. Updates workflow stats and logs task completion when task data is updated. 2020-03-10 21:29:57 -04:00
Aaron Louie f1b6718068 Adds task counts 2020-03-10 15:46:14 -04:00
Dan Funk e91d7aff2f Fixes #11: adding a delete endpoint for studies. It won't delete studies that have workflows, you have to delete those first)
Removing the "default" error response from the api.yml, it was all noise.
2020-03-09 15:12:40 -04:00
Dan Funk 95750d4fb0 Merge branch 'feature/pb_services' of github.com:sartography/cr-connect-workflow into feature/pb_services 2020-03-09 12:41:42 -04:00
Dan Funk 3b39f3875d Fix data being over-written in the study_info. 2020-03-09 12:41:35 -04:00
Aaron Louie acad0023d2 Fixes a bug in get_study_workflows and adds a test covering it 2020-03-06 10:52:41 -05:00
Dan Funk 9fcd6f38f4 Merge remote-tracking branch 'origin/master' into feature/pb_services 2020-03-05 17:13:41 -05:00
Dan Funk e501fb31fd Cleaning up swagger docs. 2020-03-05 17:10:00 -05:00
Dan Funk 906bacff6a Expose a flag on the workflow model in the api to shown if it is using the latest spec. Added a soft_reset and hard_reset onto the workflow endpoint that will allow you to cause a hard or soft reset. 2020-03-05 16:45:44 -05:00
Dan Funk 7b21b78987 Workflow Processor will deserialize workflows using the version of the BPMN files used during creation, but allows for both a soft and hard reset - soft resets will use the new workflow without a restart and will retain full history. A hard-reset will restart the workflow from scratch, but will retain the data from the last completed task. Workflows have a complex version number associated with them that is used during the deserialization to find the correct files. 2020-03-05 15:38:30 -05:00
Aaron Louie e07ad8d21c Updates training session BPMN files. Fixes PB status enums. 2020-03-05 14:21:03 -05:00
Dan Funk 697d930eab Modify the workflow processor to accept a workflow model - so it can take on more of the responsibilities of updating this model and managing versions.
Changing the version information so that it includes the numbers of the files used to generate the serialized workflow.
2020-03-05 13:25:28 -05:00
Dan Funk 70611e2c1d Adding the version of the specification used to create a workflow to the workflow api endpoint. Though the exact content of this version is likely to change.
Split the API specific models out from the workflow models to help me keep this straight.
Added tests to help me understand the errors thrown the and resolution path when a workflow specification changes in the midst of a running workflow.
2020-03-05 11:18:20 -05:00
Dan Funk 78b6f040eb Add the ability to forcibly restart a workflow, while retaining that workflows data.
A workflow specification knows it's version number, which is generated by the version of the files that make it up.
A workflow specification version number is the primary file (the lead BPMN) followed by a consistency ordered version each extra file associated with the workflow.  A change in any file modifies the specifications version.
2020-03-04 17:08:45 -05:00
Dan Funk c5cee4761e Improve version handling of files. Consolidate more of this logic in FileService. Place the version on the actual data model, not the file model, so the file model remains the same, and we just version the data associated with it. 2020-03-04 13:40:25 -05:00
Aaron Louie f258090a0b Fixes broken unit test 2020-03-04 09:54:00 -05:00
Aaron Louie 8cbd6f892f WIP: Mocks study details PB endpoint response 2020-03-03 20:56:46 -05:00
Aaron Louie 01720a8bbf Replaces migration and fixes broken tests 2020-03-03 16:02:01 -05:00
Aaron Louie 93168482c2 Merge remote-tracking branch 'origin/master' 2020-03-03 15:46:24 -05:00
Aaron Louie 668de6c4dc Edits Protocol Builder statuses 2020-03-03 15:46:20 -05:00
Dan Funk 94f828dfd6 Adding a simple endpoint that describes what scripts are currently available, along with a brief description. 2020-03-03 15:30:42 -05:00
Dan Funk 7194d7d374 Standardizing the script tasks that can be executed on the server, adding tons of error messages for when things go wrong. All scripts must exist in side of the crc/scripts directory.
Adding a new script that script tasks can use to add in data about the study.

Moving all the test workflow specifications out of the main load.

fixing a pile of tests so they can find workflow specs that are now moved into the test directory.
2020-03-03 13:52:45 -05:00
Aaron Louie f7416065da Merge remote-tracking branch 'origin/master' 2020-03-02 13:16:43 -05:00
Aaron Louie a24b67d5db Updates DSP BPMN 2020-03-02 13:16:36 -05:00
Aaron Louie b0c4996fb3 Populates documentation. 2020-03-02 13:14:03 -05:00
Dan Funk 5e3fdaaa94 New set of "Tools" api endpoints, that provides a way to quickly render markdown or word documents by uploading json data and a template to populate.
Improved Error messages / Error processing.  You can now just throw an APIError anywhere, and it will be properly serialized and returned.
2020-02-29 17:22:38 -05:00
Aaron Louie 5f944af0d7 Adds CR Connect training workflow specs 2020-02-28 15:39:44 -05:00
Aaron Louie 8611a23ad3 Renaming to snake case for consistency 2020-02-28 11:54:11 -05:00
Aaron Louie f4a0c32cec Fixes path to testing config file 2020-02-28 11:33:08 -05:00
Aaron Louie dc53939eb1 Merge remote-tracking branch 'origin/master' into feature/protocol-builder 2020-02-28 11:17:17 -05:00
Aaron Louie 4534b0c2df Updates study model to better match Protocol Builder. Deletes all migrations and starts over, since Alembic has problems migrating changes to existing column constraints. 2020-02-28 11:14:30 -05:00
Aaron Louie 0cc59d0974 Adds study inactive flag. Sets study to inactive if not found in Protocol Builder. 2020-02-27 11:17:58 -05:00
Aaron Louie f3068e68db Adds user to example data. Requires login before adding/updating a study. 2020-02-27 10:30:16 -05:00
Aaron Louie 27d7afb656 Adds Protocol Builder models and schemas. Reorganizes and cleans up some files. 2020-02-27 09:54:46 -05:00
Aaron Louie 3ef4860391 Adds user_uid and investigator_ids fields to Study. Gets studies from protocol builder and adds them if they aren't already in the database 2020-02-26 18:06:51 -05:00
Dan Funk c6eb2d937d Enable autoescape security feature for processing markdown data in descriptions from the BPMN Diagrams 2020-02-25 16:21:47 -05:00
Dan Funk 1e8a095760 Fixing a rogue comma that made something a tuple and not a string, which drives me CRAZY. 2020-02-25 12:01:25 -05:00
Aaron Louie 002207cbca Adds redirect URL to login handler 2020-02-24 16:59:16 -05:00
Aaron Louie 443d38a9a1 Prevents duplication of study on update. 2020-02-21 11:24:51 -05:00
Aaron Louie eb6354db6e Moves sso_backdoor parameters to query string. Prevents duplication of user on update. 2020-02-21 11:24:39 -05:00
Aaron Louie 9f4301712e Merge remote-tracking branch 'origin/master' 2020-02-20 17:23:17 -05:00
Aaron Louie a6e0809183 Adds SSO attributes. Prevents uid duplication errors. 2020-02-20 17:23:10 -05:00
Dan Funk d3c51af1f5 Merge branch 'master' of github.com:sartography/cr-connect-workflow 2020-02-20 16:04:44 -05:00
Dan Funk af35f182f6 Random Fact updated with a markdown set of examples 2020-02-20 16:04:38 -05:00
Aaron Louie 0cda7c66e5 Merge remote-tracking branch 'origin/master' 2020-02-20 15:43:34 -05:00
Aaron Louie 581434b453 Adds SSO header attributes 2020-02-20 15:43:29 -05:00
Dan Funk a94596112d Fixing a rogue comma that made something a tuple and not a string, which drives me CRAZY. 2020-02-20 14:35:21 -05:00
Dan Funk 2cc6010c8d Protocol builder connections 2020-02-20 13:30:04 -05:00
Dan Funk 1a9b5b50e5 Merge branch 'master' of github.com:sartography/cr-connect-workflow 2020-02-18 16:39:11 -05:00
Dan Funk a642593e3d Adding support to handle Single Sign On (Shibboleth) authentication using Flask SSO and an attribute map that has worked in the past with UVA's implementation. Aside from the new user endpoint, nothing requires authentication, but soon everything will expect it. I'm setting up a backdoor we can use for development and staging that will cause a round-robin affair that should make this relatively painless. Dropped "RestException" as we had two ways or raising errors, and that was silly. 2020-02-18 16:38:56 -05:00
Aaron Louie 6075a32d07 Edits default value 2020-02-18 10:17:14 -05:00
Aaron Louie 16db1e3504 Adds default_value to Task schema 2020-02-18 10:14:03 -05:00
Aaron Louie 5a53ddb88c Fixes vulnerability identified by SonarCloud 2020-02-12 11:07:01 -05:00
Aaron Louie 5f461d9e87 Fixes FactService 2020-02-11 22:13:46 -05:00
Aaron Louie b0b1a6e5e8 Saves form field key 2020-02-11 15:03:25 -05:00
Aaron Louie 0a3a932c50 Comments out study workflow for now. Fixes failing unit tests. 2020-02-11 13:40:14 -05:00
Aaron Louie 0ef1a178ed Merge branch 'master' into feature/crc_demo_workflows 2020-02-11 13:30:57 -05:00
Aaron Louie f58ed5b6ee Adds another user task at the end 2020-02-11 12:14:10 -05:00
Dan Funk 709bae76b2 Removing a rogue comma that was causing havoc. Also, don't fail if a mock already exists in the test database. 2020-02-11 11:11:21 -05:00
Dan Funk 879a248002 Adding a test to assure the file creation occurs as expected via the API. 2020-02-10 20:54:22 -05:00
Aaron Louie 359fb4a2f3 Adds draft demo workflow specs 2020-02-10 17:19:11 -05:00
Aaron Louie f128e7206b Adds placeholder scripts for testing 2020-02-10 17:18:51 -05:00
Dan Funk 9f0eb8477a Fix for a bug in the File service where it was being overly restrictive. 2020-02-10 16:27:57 -05:00
Dan Funk 1d24ebe382 Provide a script for generating word documents from template files. Refractored file managment into a service to make it easier to programatically add files. Modified the workflow_processor to inject the study_id and workflow_id into the running workflow so that this meta-information is avialable at the task level. 2020-02-10 16:19:23 -05:00
Aaron Louie fd9c685979 Handles processes with multiple top-level elements 2020-02-10 15:20:45 -05:00
Aaron Louie f142d02a30 Adds workflow_spec_id back in to WorkflowApi 2020-02-07 12:36:08 -05:00
Dan Funk eae6c8dd1a update to last commit, left an extra column on workflow model that does not need to exist. 2020-02-07 11:57:48 -05:00
Dan Funk e1148e1b3d Merge branch 'master' of github.com:sartography/cr-connect-workflow 2020-02-07 11:34:57 -05:00
Dan Funk 9bd93748be Returning better cleaner information about workflow with the workflow endpoint. Removes the get_all_tasks and get_user_tasks endpoints as tasks are returned with the workflow. Workflow endpoint also includes the last task and next_task, which may or may not be user tasks. The task "type" returned is now the class name of the task_spec, rather than just the word "task". 2020-02-07 11:34:44 -05:00
Aaron Louie 9fb3a96a50 Fixes instance path 2020-02-06 14:36:02 -05:00
Aaron Louie 71dac70e1f Updates list of allowed files. Removes unnecessary migrations. Note this will require a full wipe & reset of the database to work. 2020-02-05 17:23:37 -05:00
Aaron Louie 5a6d589e6d Adds form_field_key to API 2020-02-05 15:06:19 -05:00
Aaron Louie 58982ab306 Merge remote-tracking branch 'origin/feature/update-task-data' into feature/update-task-data 2020-02-05 14:55:59 -05:00
Aaron Louie 933a235471 Adds multiple and single file upload form controls 2020-02-05 14:55:52 -05:00
Aaron Louie 654f39e7fd Adds form field key to file model. 2020-02-05 14:55:31 -05:00
Dan Funk ff63978da0 read the instance/config.py file in IF it exists. 2020-02-05 13:43:59 -05:00
Aaron Louie e1e866a2b3 Gets docker-compose working.
You'll need to add a .env file to the same directory as the docker-compose.yml file that sets the values for DB_USER and DB_PASS.
2020-02-04 21:42:09 -05:00
Dan Funk ec4df2b3fa Cleaning up the tests and making it easier to test workflows without adding them to the example data structure. 2020-02-04 16:49:28 -05:00
Dan Funk b1a81957ef Merge branch 'feature/update-task-data' of github.com:sartography/cr-connect-workflow into feature/update-task-data 2020-02-04 15:45:05 -05:00
Dan Funk 9e6fbfee79 Followup on Postgres - had some failing tests that needed cleaning up now that we are using a real database. 2020-02-04 15:44:06 -05:00
Aaron Louie 3ee0426ba9 Adds workflow_id to file endpoints. 2020-02-04 14:26:53 -05:00
Aaron Louie 098ecb3e87 Wipes out old migrations, since changing foreign key constraints in SQLite requires a full database reset anyways. 2020-02-04 09:57:02 -05:00
Aaron Louie e717c25a25 Study ID is an integer. Task ID is a string. 2020-02-03 22:29:41 -05:00
Aaron Louie 0368657549 Adds ability to get FileModel records by study and task 2020-02-03 21:56:18 -05:00
Dan Funk 8fbbed4feb You must all "do engine steps" when completing a task so that it will kick off any automatic tasks/decisions/etc... this will update remaining tasks. Added some tests for parallel, since I wrote them trying to debug this. 2020-02-03 15:15:36 -05:00
Aaron Louie 8cc04aaad2 Adds task data to Task schema 2020-02-03 10:49:48 -05:00
Aaron Louie e4b2a7a641 Adds endpoint to retrieve all user tasks for a workflow, regardless of state 2020-01-31 12:40:08 -05:00
Aaron Louie 8d39a96606 Adds workflow spec & BPMN with parallel tasks 2020-01-31 12:01:41 -05:00
Aaron Louie 918a6d5ab7 Removes unsupported read-only flag on File schema for now 2020-01-31 11:33:31 -05:00
Aaron Louie a0e60ddd77 Renames endpoint methods for clarity. Adds endpoint for just updating file info. 2020-01-31 10:39:19 -05:00
Dan Funk 3fb831bae4 When you create a study, all possible workflows are then associated with that study. 2020-01-30 09:11:17 -05:00
Dan Funk acca1523b7 adding an api call for getting a single workflow spec. 2020-01-28 10:14:06 -05:00
Aaron Louie a3b6e4fc7f Fixes import paths 2020-01-24 11:52:52 -05:00
Aaron Louie 50dfa561a9 Fixes import paths, which were screwing up stuff. 2020-01-24 09:29:50 -05:00
Aaron Louie 6faabea8a4 Deletes related files and workflows when a workflow spec is deleted. 2020-01-24 09:26:17 -05:00
Aaron Louie 6aa3d711ce Adds endpoint to delete workflow spec 2020-01-23 16:05:09 -05:00
Dan Funk 45f2e52c86 Merge branch 'feature/update-task-data' of github.com:sartography/cr-connect-workflow into feature/update-task-data 2020-01-23 15:35:51 -05:00
Dan Funk 95b75f864d You can now add multiple files to a workflow spec, and if properly linked, you can associate a DMN file with a BPMN to process decision tables. 2020-01-23 15:32:53 -05:00
Aaron Louie 91e33a1768 Deletes associated file_data records when deleting a file. 2020-01-23 12:17:58 -05:00
Aaron Louie 42fa2880d0 Adds type hint 2020-01-23 10:55:07 -05:00
Aaron Louie 7fc2e87b84 Adds primary process ID to workflow spec 2020-01-23 10:54:41 -05:00
Aaron Louie 803e4be008 Stores workflow JSON 2020-01-23 10:53:59 -05:00
Aaron Louie fd21f4f301 Fixes breaking changes from SpiffWorkflow 2020-01-22 16:51:25 -05:00
Aaron Louie 6158243053 Fixes modify workflow method 2020-01-21 15:24:17 -05:00
Aaron Louie c11bf3d286 Adds workflow spec name field 2020-01-21 15:22:44 -05:00
Aaron Louie bf97095933 Adds type hinting back in 2020-01-21 15:21:33 -05:00
Dan Funk 2ffe32afdd Merge branch 'feature/update-task-data' of github.com:sartography/cr-connect-workflow into feature/update-task-data 2020-01-14 15:40:03 -05:00
Dan Funk 900819e647 fixing failed run 2020-01-14 15:39:54 -05:00
Aaron Louie 21fe50ed08 Import api before Connexion does its thing. 2020-01-14 15:26:39 -05:00
Aaron Louie 7ce48e8deb Removes type hinting, in case that's breaking something. 2020-01-14 15:10:43 -05:00
Aaron Louie 054bd88760 Fixes path to api methods 2020-01-14 14:55:10 -05:00
Aaron Louie 790b0dfc3a Adds endpoint for modifying existing workflow spec 2020-01-14 14:10:53 -05:00
Aaron Louie fe15799ee1 Adds type hinting to db and session to fix Pycharm autocomplete. Replaces db.session with session. 2020-01-14 11:45:12 -05:00
Dan Funk 6361e94af5 Allow submitting data and progressing to the next task in a workflow. 2020-01-14 11:02:44 -05:00
Aaron Louie 75881bf1d3 Adds endpoint for adding a workflow specification 2020-01-13 21:43:09 -05:00
Aaron Louie 53d72bdaae Includes workflow spec id 2020-01-13 17:52:37 -05:00
Dan Funk a3d7ba8114 WIP 2020-01-07 13:23:46 -05:00
Aaron Louie 872f3fca1b Returns workflow spec id with Workflow schema 2020-01-06 09:18:13 -05:00
Aaron Louie e75da611ca Adds endpoints for creating and updating a Study. 2020-01-03 11:44:24 -05:00
Dan Funk e316d02f02 Adding a missing file. 2020-01-02 12:22:15 -05:00
Dan Funk c4cbaeb64e Refactor models into seperate directories 2019-12-31 16:32:47 -05:00
Dan Funk 9a195bedad Fixing some caching issues and places where the updates were not being processed completed. And updates to the docker file. 2019-12-31 11:31:30 -05:00
Dan Funk 0685b282e8 Don't put foreign key constraints on primary keys. 2019-12-30 16:00:33 -05:00
Dan Funk 115a88347f Stupid tag 2019-12-30 13:34:18 -05:00
Dan Funk fb465531ca Add the ability to delete a workflow. 2019-12-30 13:15:39 -05:00
Dan Funk b54bd49cf4 Allow for clearing out the database. 2019-12-30 13:03:57 -05:00
Dan Funk 91a91e9677 Read workflow configuration from the database, and not from a file.
And dropping the API call in the Fact Service so we aren't hitting external apis for the time being.
2019-12-30 11:07:26 -05:00
Dan Funk 2ac2514e32 merge conflicts 2019-12-27 13:53:29 -05:00
Dan Funk 0b71833946 Adding file management. Files are stored in the database along with basic meta data. 2019-12-27 13:50:03 -05:00
Aaron Louie ddd0422392 Adds some fields with some Formly-specific logic 2019-12-20 15:39:40 -05:00
Aaron Louie 9b6599505d Adds task title and documentation 2019-12-20 15:39:02 -05:00
Aaron Louie dc13855593 Adds properties and validation to form 2019-12-20 14:32:04 -05:00
Dan Funk bb6ab4b8e8 Tasks can have forms and they are returned correctly. 2019-12-19 11:58:51 -05:00
Dan Funk bc9bfb658e Merge branch 'master' of github.com:sartography/cr-connect-workflow 2019-12-18 15:32:17 -05:00
Dan Funk 0dc422c7f6 we need a commit at the end, but I don't really believe that. 2019-12-18 15:32:06 -05:00
Aaron Louie bfff4f515a Auto stash before merge of "master" and "origin/master" 2019-12-18 15:24:01 -05:00
Dan Funk 53a8811769 Add workflow to study 2019-12-18 15:22:46 -05:00
Dan Funk fbbd6c6659 Adding a workflow from a spec 2019-12-18 14:42:40 -05:00
Dan Funk a734578327 re-building migrations. 2019-12-18 14:16:26 -05:00
Dan Funk 9861f6baf3 Lots and lots of random stuff as I organize code and get things to make sense. 2019-12-18 14:02:17 -05:00