Commit Graph

19 Commits

Author SHA1 Message Date
Dan Funk c790737bf0 The complete_template script might not grab the correct document when validating, fixed.
Updated to latest RRT files, and passing.
2020-06-01 12:33:58 -04:00
Dan Funk dba41f4759 Ludicrously stupid launch in a refactor of the way all files work in the system at a time where I crave sleep and peace above all other things.
Added a File class, that we wrap around the FileModel so the api endpoints don't change, but File no longer holds refences to versions or dates of the file_data model, we
figure this out based on a clean database structure.

The ApprovalFile is directly related to the file_data_model - so no chance that a reviewer would review the incorrect version of a file.py

Noticed that our FileType enum called "bpmn" "bpmm", hope this doesn't screw someone up.

Workflows are directly related to the data_models that create the workflow spec it needs.  So the files should always be there.  There are no more hashes, and thus no more hash errors where it can't find the files to rebuild the workflow.py

Not much to report here, other than I broke every single test in the system at one point.  So I'm super concerned about this, and will be testing it a lot before creating the pull request.
2020-05-28 20:03:50 -04:00
Dan Funk cd7f67ab48 A major refactor of how we search and store files, as there was a lot of confusing bits in here.
From an API point of view you can do the following (and only the following)

/files?workflow_spec_id=x
* You can find all files associated with a workflow_spec_id, and add a file with a workflow_spec_id
/files?workflow_id=x
* You can find all files associated with a workflow_id, and add a file that is directly associated with the workflow
/files?workflow_id=x&form_field_key=y
* You can find all files associated with a form element on a running workflow, and add a new file.
   Note: you can add multiple files to the same form_field_key, IF they have different file names. If the same name, the original file is archived,
   and the new file takes its place.

The study endpoints always return a list of the file metadata associated with the study.  Removed /studies-files, but there is an
endpoint called

/studies/all  - that returns all the studies in the system, and does include their files.

On a deeper level:
 The File model no longer contains:
  - study_id,
  - task_id,
  - form_field_key

Instead, if the file is associated with workflow - then that is the one way it is connected to the study, and we use this relationship to find files for a study.
A file is never associated with a task_id, as these change when the workflow is reloaded.
The form_field_key must match the irb_doc_code, so when requesting files for a form field, we just look up the irb_doc_code.
2020-05-28 08:27:26 -04:00
Dan Funk 971d9a58e9 As we now have an approval_service.py, I moved all the business logic into this service and out of the request_approval.py script. And moved all tests for these features into a test file for the service. Will make it easier to cross reference what is happening, as everything all happens in one file.
As many of the scripts need to know the workflow, and it's down in a weird parameter, moved this so it is passed in each time.
2020-05-24 16:13:15 -04:00
Aaron Louie 58189285ad Cleans up 2020-05-20 00:12:48 -04:00
Aaron Louie 2b5687c3a3 Fixes pernicious bug where template document versions were not being updated properly, and template completion script was not honoring version numbers 2020-05-20 00:10:32 -04:00
Aaron Louie 481a9ed04c Gets image ids from task data and Injects images into jinja docx 2020-05-19 21:51:54 -04:00
Dan Funk 5c5c2a7312 Assure that new lines entered in text-fields are correctly added to the final word document. 2020-05-18 11:55:10 -04:00
Dan Funk da7cae51b8 Adding a new reference file that provides greater details about the investigators related to a study.
Improving the study_info script documentation to provide detailed examples of values returned based on arguments.
Making the tests a little more targetted and less subject to breaking through better mocks.
Allow all tests to pass even when ther protocol builder mock isn't running locally.
Removing the duplication of reference files in tests and static, as this seems silly to me at the moment.
2020-05-07 13:57:24 -04:00
Dan Funk f1f8b91c9c Refactor the document details scripts. Now there is one script, it returns data in a consistent format, and has all the details required. The script is located in StudyInfo, with the argument documents. Make note that it returns a dictionary of ALL the documents, with a field to mark which ones are required according to the protocol builder. Others may become required if a workflow determines such, in which case the workflow will enforce this, and the document will have a count > 0, and additional details in a list of files within the document. I modified the XLS file to use lower case variable names, because it disturbed me, and we have to reference them frequently. Removed devious "as_object" variable on get_required_docs, so it behaves like the other methods all the time, and returns a dictionary. All the core business logic for finding the documents list now resides in the StudyService.
Because this changes the endpoint for all existing document details, I've modified all the test and static bpmn files to use the new format.
Shorting up the SponsorsList.xls file makes for slightly faster tests. seems senseless to load 5000 everytime we reset the data.
Tried to test all of this carefully in the test_study_details_documents.py test.
2020-04-29 15:08:11 -04:00
Dan Funk d3dd9dcc25 Functional multi-instance - works with no changes to the front end - though I've added some attributes to task so we could give people a sense of how many iterations they will go through. 2020-04-19 15:14:10 -04:00
Dan Funk dc2895cb05 Allow configurators to upload xls files into a workflow for defining enumrations of values for dropdown lists in forms. Fixing lots of tests.
Found a problem where the documentation for elements was being processed BEFORE data was loaded from a script.  There still may be some issues here.

Ran into an issue with circular dependencies - handling it with a new workflow_service, and pulling computational logic out of the api_models - it was the right thing to do.
2020-04-15 11:13:32 -04:00
Dan Funk c7d2c28178 Vastly more informative ApiError model that provides details on the underlying task where the error occured.
Added a validate_workflow_specification endpoint that allows you to check if the workflow will execute from beginning to end using random data.
Minor fixes to existing bpmns to allow them to pass.
All scripts must include a "do_task_validate_only" that restricts external calls and database modifications, but performs as much logic as possible.
2020-03-27 08:29:31 -04:00
Dan Funk 3eb1167b8e Found a few errors in the sqlalchemy file definition that was causing failures, and had some consistency problems with the IRB Categories file name. The API was bailing out because we had restricted file types to bpmn,svg,or dmn in the connexion config file, I don't restrict this anymore we have plenty of checks elsewhere. Adding xlrd as a dependency - this didn't fail till a push to production. 2020-03-20 08:21:21 -04:00
Dan Funk 6e3b6c2635 Assure that files uploaded through web forms and files generated from templates can be cross-referenced to known document requirements from the protocol builder. Configurators can control this by managing an XLS Spreadsheet called "irb_documents.xslx".
Required Documents is becoming complicated, so making this it's own script task, removing it from study_info.py
The file_service is now very aware of this irb_documents file, so it will always need to exist.  We seed this file
during setup, but it can be overwritten by the configurator.
2020-03-19 17:14:20 -04:00
Aaron Louie 6af1693bdc Fixes SonarCloud issue L23. Verifies that study ids match. 2020-03-16 10:37:06 -04:00
Dan Funk c5cee4761e Improve version handling of files. Consolidate more of this logic in FileService. Place the version on the actual data model, not the file model, so the file model remains the same, and we just version the data associated with it. 2020-03-04 13:40:25 -05:00
Dan Funk 94f828dfd6 Adding a simple endpoint that describes what scripts are currently available, along with a brief description. 2020-03-03 15:30:42 -05:00
Dan Funk 7194d7d374 Standardizing the script tasks that can be executed on the server, adding tons of error messages for when things go wrong. All scripts must exist in side of the crc/scripts directory.
Adding a new script that script tasks can use to add in data about the study.

Moving all the test workflow specifications out of the main load.

fixing a pile of tests so they can find workflow specs that are now moved into the test directory.
2020-03-03 13:52:45 -05:00