Commit Graph

88 Commits

Author SHA1 Message Date
Dan Funk 4bf13b0718 Doing everything I can imagine to get a CSV dump out tomorrow. 2020-06-02 22:01:49 -04:00
Dan Funk c2a1b0175a Tweaks to approvals 2020-06-01 21:49:30 -04:00
Aaron Louie f0bd8d4f9e Adds approvals to study schema. Adds approvals endpoint 2020-05-31 22:46:32 -04:00
Dan Funk 11413838a7 Faster lookup fields. We were parsing the spec each time to get details about how to search. We're just grabbing the workflow id and task id now and building that straight into the full text search index for faster lookups. Should be peppy.
Another speed improvement - data in the FileDataModel is deferred, and not queried until it is specifically used, as the new data structures need to use this model frequently.
2020-05-29 01:39:39 -04:00
Dan Funk cd7f67ab48 A major refactor of how we search and store files, as there was a lot of confusing bits in here.
From an API point of view you can do the following (and only the following)

/files?workflow_spec_id=x
* You can find all files associated with a workflow_spec_id, and add a file with a workflow_spec_id
/files?workflow_id=x
* You can find all files associated with a workflow_id, and add a file that is directly associated with the workflow
/files?workflow_id=x&form_field_key=y
* You can find all files associated with a form element on a running workflow, and add a new file.
   Note: you can add multiple files to the same form_field_key, IF they have different file names. If the same name, the original file is archived,
   and the new file takes its place.

The study endpoints always return a list of the file metadata associated with the study.  Removed /studies-files, but there is an
endpoint called

/studies/all  - that returns all the studies in the system, and does include their files.

On a deeper level:
 The File model no longer contains:
  - study_id,
  - task_id,
  - form_field_key

Instead, if the file is associated with workflow - then that is the one way it is connected to the study, and we use this relationship to find files for a study.
A file is never associated with a task_id, as these change when the workflow is reloaded.
The form_field_key must match the irb_doc_code, so when requesting files for a form field, we just look up the irb_doc_code.
2020-05-28 08:27:26 -04:00
Dan Funk 987790164e use redirect, not redirect_url. 2020-05-26 15:09:57 -04:00
Carlos Lopez e5e7eb3dd3 Fixing broken tests by making version parameter optional in get file data endpoint 2020-05-23 15:37:03 -06:00
Dan Funk d5c91e575f stuff that might be broken. 2020-05-23 15:21:30 -04:00
Dan Funk 49c322177b
Merge pull request #76 from sartography/feature/rrp-endpoints
Feature/rrp endpoints
2020-05-22 19:38:26 -04:00
Dan Funk 148e86bb42 Building out the boilerplate code to make pushing forward on this a little friendlier.
There is an approval api file, and approval model file and an approval test file.
2020-05-22 18:25:00 -04:00
Dan Funk 571c1d7d24 Merge branch 'feature/rrp-endpoints' into feature/disable_protocol_builder 2020-05-22 16:18:33 -04:00
Dan Funk 1017bb1897 On the tools/render_docx api, allow sending the json data in the body, rather than as a ludicrous long get parameter. Silly Dan. 2020-05-22 15:30:22 -04:00
Dan Funk 503c1c8f18 Allow disabling the Protocol Builder
PB_ENABLED can be set to false in the configuration (either in a file called instance/config.py, or as an environment variable)

Added a check in the base_test, to assure that we are always running tests with the test configuration, and bail out otherwise.  Setting TESTING=true as an environment variable will get this, but so well the correct ordering of imports. Just be dead certain the first file every test file imports is base_test.py.

Aaron was right, and we call the Protocol Builder in all kinds of awful places.  But we don't do this now.  So Carlos, you should have the ability to reuse a lot of the logic in the study_service now.

I dropped the poorly named "study-update" endpoint completely.  We weren't using it. POST and PUT to Study still work just fine for doing exactly that.

All the tests now run and pass with the Protocol builder disabled. Tests that specifically check PB behavior turn it back on for the test, or mock it out.
2020-05-22 14:37:49 -04:00
Carlos Lopez 1ed7930aab Endpoint for studies with files 2020-05-22 09:46:03 -06:00
Dan Funk de435bd961 the heck with camel case, what the heck TypeScript? Get a grip. This is a python API. 2020-05-15 16:38:37 -04:00
Dan Funk 53255ef35e massive overhaul of the Workflow API endpoint.
No Previous Task, No Last Task, No Task List.  Just the current task, and the Navigation.
Use the token endpoint to set the current task, even if it is a "READY" task in the api.
Previous Task can be set by identifying the prior task in the Navigation (I'm hoping)
Prefering camel case to snake case on all new apis.  Maybe clean the rest up later.
2020-05-15 15:54:53 -04:00
Dan Funk 55a1850e7c adding a navigation component to the Workflow Model.
running all extension/properties through the Jinja template processor so you can have custom display names using data, very helpful for building multi-instance displays.
Properties was returned as an array of key/value pairs, which is just mean.  Switched this to a dictionary.
2020-05-14 13:43:23 -04:00
Dan Funk 2699f5c65c Refactor the stats models, and assure they are very correct across all tests with the workflow api.
I noticed we were saving the workflow every time we loaded it up, rather than only when we were making changes to it.  Refactored this to be a little more careful.
Centralized the saving of the workflow into one location in the processor, so we can make sure we update all the details about that workflow every time we save.
The workflow service has a method that will log any task action taken in a consistent way.
The stats models were removed from the API completely.  Will wait for a use case for dealing with this later.
2020-05-04 10:57:09 -04:00
Dan Funk bec59a71d7 Deleteing stuff is a damn mess, but this is a little cleaner. 2020-04-29 16:07:39 -04:00
Dan Funk 13333fb306 Adding an API endpoint that allows setting the token on the workflow to a specific task.
Added error checking such that attempting to submit data for a task that is not in the "READY" state throws an error message.
For some reason I'm getting lots of errors in the tests as they try to hit API endpoints they were not hitting before, so adding a number of mocks to some of the study service tests.
2020-04-28 17:25:53 -04:00
Dan Funk 7b085c9c9d Adding an API Endpoint that will return a list of LookupValues that match a given query - can be used to populate an auto-complete table. 2020-04-22 19:40:40 -04:00
Dan Funk fd0adb1d43 Updated the study status to use a different enumeration. Migration correctly handles modifying the enum.
INCOMPLETE = 'Incomplete in Protocol Builder',
  ACTIVE = 'Active / Ready to roll',
  HOLD = 'On Hold',
  OPEN = 'Open - this study is in progress',
  ABANDONED = 'Abandoned, it got deleted in Protocol Builder'
2020-04-21 17:13:30 -04:00
Dan Funk d3dd9dcc25 Functional multi-instance - works with no changes to the front end - though I've added some attributes to task so we could give people a sense of how many iterations they will go through. 2020-04-19 15:14:10 -04:00
Dan Funk 17796193de fixing a bug that was causing failing tests.
Adding id and spec_version to the workflow metadata.
Refactoring the processing of the master_spec so that it doesn't polute the workflow database.
Adding tests to assure that the status and counts are updated on the workflow model as users make progress.
2020-03-30 14:01:57 -04:00
Dan Funk 34b6ec92bf updating the API
Removing the call for study/workflows - as workflow information is returned with the study by default.
Fixing a bug in the workflow spec model schema.
2020-03-30 10:12:10 -04:00
Dan Funk 4a916c1ee3 Created a "StudyService" and moved all complex logic around study manipulation out of the study api, and this service, as things were getting complicated. The Workflow Processor no longer creates the WorkflowModel, the study object handles that, and only passes the model into the workflow processor when it is ready to start the workflow.
Created a Study object (seperate from the StudyModel) that can cronstructed on request, and contains a different data structure than we store in the DB. This allows us to return underlying Categories and Workflows in a clean way.

Added a new status to workflows called "not_started", meaning we have not yet instantiated a processor or created a BPMN, they have no version yet and no stored data, just the possiblity of being started.

The Top Level Workflow or "Master" workflow is now a part of the sample data, and loaded at all times.

Removed the ability to "add a workflow to a study" and "remove a workflow from a study", a study contains all possible workflows by definition.

Example data no longer creates users or studies, it just creates the specs.
2020-03-30 08:00:16 -04:00
Dan Funk b61a35f956 Merge remote-tracking branch 'origin/master' into feature/status_refactor
Fixing adding a study so all workflows are again added, will add status on those workflows based on output from the master bpmn diagram, which is coming shortly.
2020-03-27 11:55:36 -04:00
Dan Funk c7d2c28178 Vastly more informative ApiError model that provides details on the underlying task where the error occured.
Added a validate_workflow_specification endpoint that allows you to check if the workflow will execute from beginning to end using random data.
Minor fixes to existing bpmns to allow them to pass.
All scripts must include a "do_task_validate_only" that restricts external calls and database modifications, but performs as much logic as possible.
2020-03-27 08:29:31 -04:00
Aaron Louie f77ebe3177 Adds display order to categories 2020-03-26 15:29:52 -04:00
Dan Funk bd92c230e4 Fix the API so we are clear what is being returned when updating a file. 2020-03-25 08:06:58 -04:00
Dan Funk f4342fc785 It became impossible to use the Swagger ui when we started adding authentication to all of the calls. I discovered Connexion and Swagger have a default way of handing JTW authentication and this cleans up our code quite a bit, moves the securing of endpoints into the API Definition, which is quite nice, and removes a whole library dependency (I never get to do that!) I added a SWAGGER_AUTH_KEY that can be used in non-production environments to allow users to quickly authenticate from the Swagger ui. I also removed all api calls to simple little happy api services, because that is just mean and pointless. 2020-03-24 14:15:21 -04:00
Dan Funk b35427523d Merge remote-tracking branch 'origin/master' into feature/reference_files
# Conflicts:
#	crc/models/file.py
#	crc/services/file_service.py
#	tests/data/reference/irb_documents.xlsx
#	tests/test_files_api.py
2020-03-20 11:07:55 -04:00
Dan Funk 3eb1167b8e Found a few errors in the sqlalchemy file definition that was causing failures, and had some consistency problems with the IRB Categories file name. The API was bailing out because we had restricted file types to bpmn,svg,or dmn in the connexion config file, I don't restrict this anymore we have plenty of checks elsewhere. Adding xlrd as a dependency - this didn't fail till a push to production. 2020-03-20 08:21:21 -04:00
Dan Funk 6e3b6c2635 Assure that files uploaded through web forms and files generated from templates can be cross-referenced to known document requirements from the protocol builder. Configurators can control this by managing an XLS Spreadsheet called "irb_documents.xslx".
Required Documents is becoming complicated, so making this it's own script task, removing it from study_info.py
The file_service is now very aware of this irb_documents file, so it will always need to exist.  We seed this file
during setup, but it can be overwritten by the configurator.
2020-03-19 17:14:20 -04:00
Aaron Louie ac7c7b42c4 Revert "Merge remote-tracking branch 'origin/feature/reference_files' into feature/workflow_spec_categories"
This reverts commit 9a26fc8e80.
2020-03-18 17:05:13 -04:00
Aaron Louie 9a26fc8e80 Merge remote-tracking branch 'origin/feature/reference_files' into feature/workflow_spec_categories 2020-03-18 16:58:57 -04:00
Dan Funk 02be8ede75 Merge remote-tracking branch 'origin/master' into feature/reference_files 2020-03-18 15:16:34 -04:00
Aaron Louie 532d6b5dd6 Adds CRUD endpoints for workflow spec categories 2020-03-16 16:30:56 -04:00
Aaron Louie 7d31bc24eb Attempting to address "FutureWarning: Implicit (flask) JSON serialization" warning 2020-03-16 13:24:15 -04:00
Aaron Louie a98251237b Adds workflow spec category to API 2020-03-16 12:10:32 -04:00
Dan Funk 779674ab60 Add the ability to upload and request general reference files by name. These will be used across workflows and will frequently contain lookup tables that can be referenced by various script tasks. 2020-03-13 15:03:57 -04:00
Aaron Louie bbfe9291e0 Adds endpoint to get workflow stats. Adds a test for the endpoint. 2020-03-11 14:28:53 -04:00
Aaron Louie f1b6718068 Adds task counts 2020-03-10 15:46:14 -04:00
Dan Funk e91d7aff2f Fixes #11: adding a delete endpoint for studies. It won't delete studies that have workflows, you have to delete those first)
Removing the "default" error response from the api.yml, it was all noise.
2020-03-09 15:12:40 -04:00
Dan Funk 9fcd6f38f4 Merge remote-tracking branch 'origin/master' into feature/pb_services 2020-03-05 17:13:41 -05:00
Dan Funk e501fb31fd Cleaning up swagger docs. 2020-03-05 17:10:00 -05:00
Dan Funk 906bacff6a Expose a flag on the workflow model in the api to shown if it is using the latest spec. Added a soft_reset and hard_reset onto the workflow endpoint that will allow you to cause a hard or soft reset. 2020-03-05 16:45:44 -05:00
Aaron Louie 01720a8bbf Replaces migration and fixes broken tests 2020-03-03 16:02:01 -05:00
Dan Funk 94f828dfd6 Adding a simple endpoint that describes what scripts are currently available, along with a brief description. 2020-03-03 15:30:42 -05:00
Dan Funk 5e3fdaaa94 New set of "Tools" api endpoints, that provides a way to quickly render markdown or word documents by uploading json data and a template to populate.
Improved Error messages / Error processing.  You can now just throw an APIError anywhere, and it will be properly serialized and returned.
2020-02-29 17:22:38 -05:00