157 Commits

Author SHA1 Message Date
Aaron Louie
7f0b35ff27 Merge branch 'dev' into feature/enum_lookup_data 2020-07-10 10:27:51 -04:00
Carlos Lopez
c7b864f9c7 Cleaning up old notes 2020-07-07 08:32:20 -06:00
Aaron Louie
1d9fef380a Merge branch 'fix/more-info-in-exceptions' into feature/enum_lookup_data 2020-07-03 14:22:11 -04:00
Aaron Louie
f344fe60c0 WIP: Adds enum field lookup option data 2020-07-03 12:20:10 -04:00
Carlos Lopez
817333f26e Adding proper exception information 2020-07-02 16:10:33 -06:00
Aaron Louie
340ae68eed Sets PB status to Active by default 2020-07-01 11:01:36 -04:00
Dan Funk
f183e12fe5 Provides some basic tools for getting additional information about a lookup field.
Adds an optional 'value' parameter to the lookup endpoint so you can find a specific entry in the lookup table.
Makes sure the data attribute returned on a lookup model is a dictionary, and not a string.
Fixes a previous bug that would crop up if double spaces were used when performing a search.
2020-06-30 10:34:16 -04:00
Dan Funk
d3ce1af1ce Provides some basic tools for getting additional information about a lookup field.
Adds an optional 'id' parameter to the lookup endpoint so you can find a specific entry in the lookup table.
Makes sure the data attribute returned on a lookup model is a dictionary, and not a string.
Fixes a previous bug that would crop up in double spaces were used when performing a search.
2020-06-30 10:00:22 -04:00
Carlos Lopez
5327b469f6 Merge branch 'rrt/dev' into feature/emails-enhancement 2020-06-24 21:46:53 -06:00
Dan Funk
6aec15cc7c Shifting to a different model, where the TaskEvents store ONLY the form data submitted for that task.
In order to allow proper deletion of tasks, we no longer merge data returned from the front end, we set it directly as the task_data.
When returning data to the front end, we take any previous form submission and merge it into the current task data, allowing users to keep their previous submissions.
There is now an "extract_form_data" method that does it's best job to calculate what form data might have changed from the front end.
2020-06-19 08:22:53 -04:00
Carlos Lopez
e947f40ec7 Merge branch 'rrt/dev' into feature/emails-enhancement 2020-06-17 20:10:11 -06:00
Carlos Lopez
896ba6b377 Email relies now on markdown content 2020-06-17 17:00:16 -06:00
Carlos Lopez
2ce2dc73b5 Rendering proper content & organizing file structure for email tests 2020-06-17 16:09:38 -06:00
Dan Funk
3b57adb84c Continuing a major refactor. Some important points:
* TaskEvents now contain the data for each event as it was when the task was completed.
* When loading a task for the front end, if the task was completed previously, we take that data, and overwrite it with the lastest data, allowing users to see previously entered values.
* Pulling in the Admin branch, as there are changes in that branch that are critical to seeing what is happening when we do this thing.
* Moved code for converting a workflow to an API ready data stricture into the Workflow service where it belongs, and out of the API.
* Hard resets just convert to using the latest spec, they don't try to keep the data from the last task.  There is a better way.
* Moving to a previous task does not attept to keep the data from the last completed task.
* Added a function that will fix all the existing RRT data by adding critical data into the TaskEvent model. This can be called with from the flask command line tool.
2020-06-17 17:11:15 -04:00
Dan Funk
1b9166dcb7 Cleaning up the merge, which resulted in some lost code. 2020-06-16 13:34:21 -04:00
Dan Funk
c7d8eaff30 Merge branch 'dev' into feature/refactor_data_loading 2020-06-16 13:15:43 -04:00
Kelly McDonald
706c4afda5 Create a parameter 'lane' since we are using the output from the nav-list as the input to the NavigationItem initializer.
NB - this doesn't do anything with the lane designation, it just keeps it from blowing up.
2020-06-15 12:33:51 -04:00
Aaron Louie
561e254315 Prevents non-admin users from editing each others' tasks. Fixes bug where test user uid was not being set from token. Moves complete form and get workflow API test utility methods into BaseTest. 2020-06-12 13:46:10 -04:00
Aaron Louie
cccff9b856 Fixes broken unit tests. But still broken. 2020-06-11 11:29:58 -04:00
Carlos Lopez
e9e805b2c9 Storing emails in database 2020-06-09 22:57:56 -06:00
Dan Funk
8cf420b781 Default mail user name and password to blank. 2020-06-08 14:15:56 -04:00
Dan Funk
6861991d8f Allow setting the type of approvals you want back, by status.
Some very minor performance enhancements, that will add up on the Approvers page.
2020-06-05 17:49:55 -04:00
Dan Funk
f0db5b70fc Adding some additional logic to the approval endpoint so that we take related approvals into account when setting the status. In addtion to prevous status options, there is a new status of "AWAITING" which means there are pending approvals before this approval that still need to be approved or canceled. 2020-06-05 14:33:00 -04:00
Dan Funk
fed6e86f92 Trying to fix LDAP issues on production. Changing LDAP to static only methods, caching the connection and calling bind before all connection requests.
Also assuring we don't load the documents.xls file over and over again.
2020-06-04 14:59:36 -04:00
Dan Funk
50d2acac9c Made a very stupid mistake with LDAP connections, pushing up quickly to production. 2020-06-04 11:57:00 -04:00
Dan Funk
68aeaf1273 BE VERY CAREFUL where you create a new LdapService() - construction is expensive.
Adding a few more details to the "csv" endpoint for RRT.
2020-06-04 10:33:17 -04:00
Dan Funk
bbcbfef1ba Fixing the migrations so I don't break the universe. 2020-06-04 10:09:36 -04:00
Dan Funk
217ecfc911 When you can't delete a file, mark it as archived. Don't include archived files in new approval requests. 2020-06-03 17:34:27 -04:00
Dan Funk
2424b9d78c Don't overwrite the approval, just allow minor changes. 2020-06-02 19:36:06 -04:00
Dan Funk
c7484267e1 For the main approval endpoints - we now group the approvals by study. So you get one record back for each study, but it may have other approvals along with it as "related_approvals".
We now cache the LDAP records - so we look in our own database for the record before calling out to ldap for the details when given a straight up computing id like dhf8r.

Added "date_approved" to the approval model.

And moved the approver and primary investigator into real associated models to make it easier to dump.

Fixed a problem with the validation that was causing it to throw incorrect errors on valid workflows. Getting it to behave a little more like the front end behaves, and respecting the read-only fields.  But it was mainly to do with always returning all the data with each form submission.
2020-06-02 18:17:00 -04:00
Dan Funk
d8329e9eff I completely missed pushing up my changes last night before handing back over to Carlos. Sorry about that.
Added checks to assure the approver matches g.uid of the current user - otherwise don't modify the approval record.
2020-06-02 07:43:19 -04:00
Dan Funk
4090667a5d Missed committing a pile of changes last night. 2020-06-02 07:08:29 -04:00
Dan Funk
c2a1b0175a Tweaks to approvals 2020-06-01 21:49:30 -04:00
Dan Funk
1db9401166 Don't put all the data into Spiff Tasks on a reload or backtrack, just store the data that gets submitted each time in the task log, and use that.
This should correct issues with parallel tasks and other complex areas - so we don't have tasks seeing data that isn't along their path.
2020-06-01 17:42:39 -04:00
Carlos Lopez
3c2849f40d Approval info addition 2020-06-01 15:29:59 -06:00
Carlos Lopez
bec11980eb Fixing broken test by using proper FileSchema 2020-05-31 22:00:52 -06:00
Aaron Louie
f0bd8d4f9e Adds approvals to study schema. Adds approvals endpoint 2020-05-31 22:46:32 -04:00
Dan Funk
9c7de39b09 This adds additional file data details to the study model as well. 2020-05-31 21:15:40 -04:00
Carlos Lopez
dd6c1d2b42 Renaming approval files 2020-05-31 18:16:42 -06:00
Carlos Lopez
2e54f07095 Adding more info to files and renaming approval waiting status 2020-05-31 17:24:23 -06:00
Carlos Lopez
26809d1470 Waiting status renaming 2020-05-31 13:35:42 -06:00
Dan Funk
1f0e8741ba Run the validation twice, once completing all of the data, and a second time, completing only the required fields.
Also, add a helper method to reduce boiler plate code around Workflow Exceptions.
2020-05-30 17:26:27 -04:00
Dan Funk
860c475b29 Fill out repeating sections during validation process.
Also, when returning error messages, attempt to include the task data for the task that caused the error.
Also, when attempting to delete any file, respond with an API error explaining the issue, and log the details.
2020-05-30 15:37:04 -04:00
Dan Funk
895d2f8852 Slight change to the way the approval model is connected. 2020-05-29 05:04:18 -04:00
Dan Funk
0ba2819867 Merge branch 'dev' into feature/approval_request_script 2020-05-29 04:51:50 -04:00
Dan Funk
11413838a7 Faster lookup fields. We were parsing the spec each time to get details about how to search. We're just grabbing the workflow id and task id now and building that straight into the full text search index for faster lookups. Should be peppy.
Another speed improvement - data in the FileDataModel is deferred, and not queried until it is specifically used, as the new data structures need to use this model frequently.
2020-05-29 01:39:39 -04:00
Dan Funk
dba41f4759 Ludicrously stupid launch in a refactor of the way all files work in the system at a time where I crave sleep and peace above all other things.
Added a File class, that we wrap around the FileModel so the api endpoints don't change, but File no longer holds refences to versions or dates of the file_data model, we
figure this out based on a clean database structure.

The ApprovalFile is directly related to the file_data_model - so no chance that a reviewer would review the incorrect version of a file.py

Noticed that our FileType enum called "bpmn" "bpmm", hope this doesn't screw someone up.

Workflows are directly related to the data_models that create the workflow spec it needs.  So the files should always be there.  There are no more hashes, and thus no more hash errors where it can't find the files to rebuild the workflow.py

Not much to report here, other than I broke every single test in the system at one point.  So I'm super concerned about this, and will be testing it a lot before creating the pull request.
2020-05-28 20:03:50 -04:00
Carlos Lopez
41b26e28bd Merge branch 'dev' into feature/approval_request_script 2020-05-28 08:11:31 -06:00
Dan Funk
cd7f67ab48 A major refactor of how we search and store files, as there was a lot of confusing bits in here.
From an API point of view you can do the following (and only the following)

/files?workflow_spec_id=x
* You can find all files associated with a workflow_spec_id, and add a file with a workflow_spec_id
/files?workflow_id=x
* You can find all files associated with a workflow_id, and add a file that is directly associated with the workflow
/files?workflow_id=x&form_field_key=y
* You can find all files associated with a form element on a running workflow, and add a new file.
   Note: you can add multiple files to the same form_field_key, IF they have different file names. If the same name, the original file is archived,
   and the new file takes its place.

The study endpoints always return a list of the file metadata associated with the study.  Removed /studies-files, but there is an
endpoint called

/studies/all  - that returns all the studies in the system, and does include their files.

On a deeper level:
 The File model no longer contains:
  - study_id,
  - task_id,
  - form_field_key

Instead, if the file is associated with workflow - then that is the one way it is connected to the study, and we use this relationship to find files for a study.
A file is never associated with a task_id, as these change when the workflow is reloaded.
The form_field_key must match the irb_doc_code, so when requesting files for a form field, we just look up the irb_doc_code.
2020-05-28 08:27:26 -04:00
Carlos Lopez
54640988a7 Update endpoint fixes 2020-05-27 12:06:32 -06:00