Commit Graph

259 Commits

Author SHA1 Message Date
Kelly McDonald 4fa0f9720a Allow frontend to honor 'view as' and fix problem with tests 2021-02-18 11:25:17 -05:00
Kelly McDonald 8c309e9a41 Add user display name to the response for task events 2021-02-18 08:44:44 -05:00
Kelly McDonald 118cf62172 Merge remote-tracking branch 'origin/dev' into STG-181_frontend-ux-model-changes 2021-02-16 10:07:34 -05:00
Kelly McDonald 003bf2f9b9 Remove Approvals per ticket 183 2021-02-16 09:05:29 -05:00
Kelly McDonald da0af14724 Changes to enable front-end changes
Added a human readable name to created user

added last updated task time and name,

added is_review on the workflow which is triggered by a swimlane with a name present.

Notably - it does NOT update or change any of the performance concerns that we were having with this.
2021-02-10 11:58:19 -05:00
mike cullerton 777cfbdecd We now test whether we have a valid StudyModel when getting a study by study model.
We raise an ApiError if the model is None or empty.
2021-02-09 17:37:55 -05:00
Dan b0dc834682 Don't error out on autocomplete_num 2021-01-04 13:41:57 -05:00
mike cullerton ede5df4df5 Studies can now be put on hold in the dashboard. We were overriding the status in StudyModel.update_from_protocol_builder. 2020-12-30 17:49:59 -05:00
Dan 1f9bf72c59 Merge branch 'dev' into bug/navigation 2020-12-14 11:46:44 -05:00
Dan 6bf24cc438 fixing some failing tests related to changes in the underlying spiffworkflow library. 2020-12-14 11:27:38 -05:00
Dan 02ea414b94 Additional fixes to Navigation to allow a nested navigation structure. 2020-12-14 10:07:19 -05:00
Kelly McDonald 0e1aa59fa1 Make a change to make sure that if there is a new file locally that is not present remotely when we pull from the remote, the new local file gets deleted.
Also: add several things to the requirements.txt that should have been there in the first place.
2020-12-08 13:42:01 -05:00
Dan 93b12a8e82 updates using new navigation from spiff workflow's navigation branch, all tests passing. 2020-12-04 17:56:12 -05:00
Kelly McDonald e9979efb0d Add some tests 2020-11-17 09:33:29 -05:00
Kelly McDonald 70425eee0d Add last_updated field - change the api signature to be closer to what we need going forward, but retain the old methods for testing the script get/set through the API 2020-11-13 12:03:29 -05:00
Kelly McDonald 7c2c725840 save before changing how API works 2020-11-13 09:58:21 -05:00
Kelly McDonald acaf633b45 Added ID as pk 2020-11-11 11:36:25 -05:00
Kelly McDonald 245124779e Add mostly-working elements of the data-store, compare with dev to see why db is not working after running tests 2020-11-10 08:21:46 -05:00
Dan Funk b544334f45 1. Updating Personnel BPMN diagram to debug some issues.
2. Disabling the token timeout for now, to see if this corrects the issues Alex is having with lost work.
3. Raising more thoughtful error messages for unknown lookup options.
4. Providing better validation of default values and injecting the correct value for defaults related to enum lists of all types.
5. Bumping Spiffworkflow library which contains some better error messages and checks.
2020-09-01 15:58:50 -04:00
Dan Funk 53d09303d8 Validating that field properties are valid - they must exist as constants on the Task model.
Making all the lookup field names consistent.
Fixing the lookup service which was failing at times trying to find the correct field to use for building the lookup table.
Updating validation to check for additional fields and properties.
When connexion level errors occur, wrapping it in an API Error to be consistent.
2020-08-27 14:00:14 -04:00
Dan Funk 85ad477b2b Some minor cleanup on the study status and automatic events. I wanted to avoid having one database model automatically generating other database models as a side effect. The study service now has full responsiblity to recording study events.
To help in running tests, adding __init__.py methods to all the test directories.
2020-08-17 14:56:00 -04:00
Carlos Lopez 369fe4bb35 Automatic events refactoring 2020-08-14 08:36:46 -06:00
Dan Funk 9a5c1d7cfb I may have finally wrapped my head around full text search in python. Now properly using an index based on simple rather than english dictionary which has far fewer stop words and stemming processes and plays much better to the type ahead search we are trying to provide.
Stop words are no longer excluded, so "other" is a valid search and gets a result.
2020-08-13 18:13:41 -04:00
Aaron Louie 9a31d591ae Adds user uid and cascades study event in study model. Fixes migration to properly downgrade enums. 2020-08-12 10:13:23 -04:00
Carlos Lopez a561a5cddc Adjusting study events attribute 2020-08-08 18:25:01 -06:00
Carlos Lopez bf3dae1885 Collecting user generated study changes 2020-08-05 20:29:05 -06:00
Aaron Louie b4595b345a Raises ApiError for 'display_name' extension property exception 2020-08-04 13:59:50 -04:00
Carlos Lopez 50b207307a Merge branch 'dev' into feature/update_study_status_latest 2020-07-31 14:26:24 -06:00
Carlos Lopez 2c2967e633 Normalizing enum values 2020-07-31 11:19:50 -06:00
Carlos Lopez 5d23223e51 New study status update 2020-07-30 21:03:11 -06:00
Aaron Louie faba0f55ab Adds AdminSession model and refactors impersonation methods to use it. 2020-07-30 12:40:53 -04:00
Aaron Louie 1b0ebecbf4 Uses Flask session to store impersonation state. 2020-07-30 10:17:02 -04:00
Aaron Louie 63537d7765 Adds is_admin boolean flag to user schema 2020-07-29 22:45:56 -04:00
Carlos Lopez de49397549 Adjustings tests for protocol builder status changes 2020-07-29 10:51:34 -06:00
Carlos Lopez 73a6b7adf1 Fixing tests 2020-07-29 10:25:57 -06:00
Carlos Lopez 6aa21638de Updating properly study status & fixing tests 2020-07-29 10:25:57 -06:00
Aaron Louie f897ee3aea Merge branch 'feature/approvals_dashboard_prototype' into feature/admin_impersonations 2020-07-29 10:32:46 -04:00
Dan Funk 0ea4c13d09 Convert protocol builder status to always be in lower case in order to better match the front end. And also fixing an issue with the multi_instance that is oddly broken suddenly, and I don't know why. 2020-07-28 17:16:48 -04:00
Dan Funk 4d11fc04a0 dropping the "read_only" flag in favor of a "do_engine_steps" flag, which more clearly defines what is happening. 2020-07-28 13:51:29 -04:00
Dan Funk f15626033d Allow the workflow to be requested without making changes to the workflow - requires that you specify a read_only flag of true, otherwise it assumes that you want a fully prepared workflow with the next ready task set to run. 2020-07-28 13:33:38 -04:00
Dan Funk 300026cbc8 Expanding the task events endpoint to accept workflow and study ids as additional filters.
Removing events from the study endpoint, too noisy.
2020-07-28 10:16:48 -04:00
Dan Funk 0cb480801b Provide event data on the Study api endpoint.
Speed up the tests a little, because that got out of hand.  Need to dig into what is causing this problem.
2020-07-27 17:05:01 -04:00
Dan Funk 452f2c3723 Building out a user service for getting the current user, it will provide a number of functions, one of which will allow administrative users to impersonate other users in some circumstances (but will assure that we log events correctly when an impersonation occures) 2020-07-27 14:38:57 -04:00
Aaron Louie b87f55fbd7 Exposes date in TaskEvent endpoint 2020-07-22 14:48:10 -04:00
Aaron Louie 855f5544e5 Adds enrollment_date to study model 2020-07-22 09:35:08 -04:00
Dan Funk 3d9eeab502 Add a few more details to the workflow metadata model. 2020-07-21 15:18:08 -04:00
Aaron Louie d67105332f Merge branch 'dev' into feature/lanes_and_roles 2020-07-16 14:00:53 -04:00
Dan Funk d85ca1ce51 Whenever a workflow is loaded or updated, add events to the TaskEvent table with assignments for the next set of ready tasks, along with who should complete those tasks.
Add the lane information to the Task model.
Drop the foreign key constraint on the user_uid in the task log, as we might create tasks for users before they ever log into the system.
Add a new endpoint to the API called task events.  It should be possible to query this and get a list of all tasks that need a users attention.
The task events returned include detailed information about the workflow and study as sub-models
Rename all the actions in event log to things that are easier to pass over the api as arguments, make this backwards compatible, updating existing names in the database via the migration.
Throughly test the navigation and task details as control of the workflow is passed between two lanes.
2020-07-14 22:16:44 -04:00
Dan Funk 9077ff3ebf It is not possible to use task_data for an auto-complete field. It's too expensive an operation to provide that feature on the backend, and the data already fully resides on the front end anyway. Task-data can be used to populate enum fields if needed, so it can populate dropdowns, radios and checkboxes, just not auto-complete. 2020-07-14 11:38:48 -04:00
Dan Funk a48322ef6a Partial work on CR Connect Roles.
Adding checks in the API to assure the correct person is completeing a task based on the task's lane.
Adding lane to the Navigation item.
Adding a check to assure that unique user ids can be identified using task.data
Added some additional ldap entries to make testing and development easier.
Removed a big chunk of duplicate code from task_tests_api
Modified some of the helper functions to make it easier to test as specific users.
Added some additional bpmn models to the tests for testing lanes and roles.
2020-07-14 10:29:25 -04:00
Aaron Louie b7920989ed WIP: Adds Camunda property for retrieving enum field options from task data. 2020-07-10 14:48:38 -04:00
Aaron Louie 1f454536e3 Renames TOKEN_AUTH_SECRET_KEY to SECRET_KEY 2020-07-10 11:26:15 -04:00
Aaron Louie 7f0b35ff27 Merge branch 'dev' into feature/enum_lookup_data 2020-07-10 10:27:51 -04:00
Carlos Lopez c7b864f9c7 Cleaning up old notes 2020-07-07 08:32:20 -06:00
Aaron Louie 1d9fef380a Merge branch 'fix/more-info-in-exceptions' into feature/enum_lookup_data 2020-07-03 14:22:11 -04:00
Aaron Louie f344fe60c0 WIP: Adds enum field lookup option data 2020-07-03 12:20:10 -04:00
Carlos Lopez 817333f26e Adding proper exception information 2020-07-02 16:10:33 -06:00
Aaron Louie 340ae68eed Sets PB status to Active by default 2020-07-01 11:01:36 -04:00
Dan Funk f183e12fe5 Provides some basic tools for getting additional information about a lookup field.
Adds an optional 'value' parameter to the lookup endpoint so you can find a specific entry in the lookup table.
Makes sure the data attribute returned on a lookup model is a dictionary, and not a string.
Fixes a previous bug that would crop up if double spaces were used when performing a search.
2020-06-30 10:34:16 -04:00
Dan Funk d3ce1af1ce Provides some basic tools for getting additional information about a lookup field.
Adds an optional 'id' parameter to the lookup endpoint so you can find a specific entry in the lookup table.
Makes sure the data attribute returned on a lookup model is a dictionary, and not a string.
Fixes a previous bug that would crop up in double spaces were used when performing a search.
2020-06-30 10:00:22 -04:00
Carlos Lopez 5327b469f6 Merge branch 'rrt/dev' into feature/emails-enhancement 2020-06-24 21:46:53 -06:00
Dan Funk 6aec15cc7c Shifting to a different model, where the TaskEvents store ONLY the form data submitted for that task.
In order to allow proper deletion of tasks, we no longer merge data returned from the front end, we set it directly as the task_data.
When returning data to the front end, we take any previous form submission and merge it into the current task data, allowing users to keep their previous submissions.
There is now an "extract_form_data" method that does it's best job to calculate what form data might have changed from the front end.
2020-06-19 08:22:53 -04:00
Carlos Lopez e947f40ec7 Merge branch 'rrt/dev' into feature/emails-enhancement 2020-06-17 20:10:11 -06:00
Carlos Lopez 896ba6b377 Email relies now on markdown content 2020-06-17 17:00:16 -06:00
Carlos Lopez 2ce2dc73b5 Rendering proper content & organizing file structure for email tests 2020-06-17 16:09:38 -06:00
Dan Funk 3b57adb84c Continuing a major refactor. Some important points:
* TaskEvents now contain the data for each event as it was when the task was completed.
* When loading a task for the front end, if the task was completed previously, we take that data, and overwrite it with the lastest data, allowing users to see previously entered values.
* Pulling in the Admin branch, as there are changes in that branch that are critical to seeing what is happening when we do this thing.
* Moved code for converting a workflow to an API ready data stricture into the Workflow service where it belongs, and out of the API.
* Hard resets just convert to using the latest spec, they don't try to keep the data from the last task.  There is a better way.
* Moving to a previous task does not attept to keep the data from the last completed task.
* Added a function that will fix all the existing RRT data by adding critical data into the TaskEvent model. This can be called with from the flask command line tool.
2020-06-17 17:11:15 -04:00
Dan Funk 1b9166dcb7 Cleaning up the merge, which resulted in some lost code. 2020-06-16 13:34:21 -04:00
Dan Funk c7d8eaff30 Merge branch 'dev' into feature/refactor_data_loading 2020-06-16 13:15:43 -04:00
Kelly McDonald 706c4afda5 Create a parameter 'lane' since we are using the output from the nav-list as the input to the NavigationItem initializer.
NB - this doesn't do anything with the lane designation, it just keeps it from blowing up.
2020-06-15 12:33:51 -04:00
Aaron Louie 561e254315 Prevents non-admin users from editing each others' tasks. Fixes bug where test user uid was not being set from token. Moves complete form and get workflow API test utility methods into BaseTest. 2020-06-12 13:46:10 -04:00
Aaron Louie cccff9b856 Fixes broken unit tests. But still broken. 2020-06-11 11:29:58 -04:00
Carlos Lopez e9e805b2c9 Storing emails in database 2020-06-09 22:57:56 -06:00
Dan Funk 8cf420b781 Default mail user name and password to blank. 2020-06-08 14:15:56 -04:00
Dan Funk 6861991d8f Allow setting the type of approvals you want back, by status.
Some very minor performance enhancements, that will add up on the Approvers page.
2020-06-05 17:49:55 -04:00
Dan Funk f0db5b70fc Adding some additional logic to the approval endpoint so that we take related approvals into account when setting the status. In addtion to prevous status options, there is a new status of "AWAITING" which means there are pending approvals before this approval that still need to be approved or canceled. 2020-06-05 14:33:00 -04:00
Dan Funk fed6e86f92 Trying to fix LDAP issues on production. Changing LDAP to static only methods, caching the connection and calling bind before all connection requests.
Also assuring we don't load the documents.xls file over and over again.
2020-06-04 14:59:36 -04:00
Dan Funk 50d2acac9c Made a very stupid mistake with LDAP connections, pushing up quickly to production. 2020-06-04 11:57:00 -04:00
Dan Funk 68aeaf1273 BE VERY CAREFUL where you create a new LdapService() - construction is expensive.
Adding a few more details to the "csv" endpoint for RRT.
2020-06-04 10:33:17 -04:00
Dan Funk bbcbfef1ba Fixing the migrations so I don't break the universe. 2020-06-04 10:09:36 -04:00
Dan Funk 217ecfc911 When you can't delete a file, mark it as archived. Don't include archived files in new approval requests. 2020-06-03 17:34:27 -04:00
Dan Funk 2424b9d78c Don't overwrite the approval, just allow minor changes. 2020-06-02 19:36:06 -04:00
Dan Funk c7484267e1 For the main approval endpoints - we now group the approvals by study. So you get one record back for each study, but it may have other approvals along with it as "related_approvals".
We now cache the LDAP records - so we look in our own database for the record before calling out to ldap for the details when given a straight up computing id like dhf8r.

Added "date_approved" to the approval model.

And moved the approver and primary investigator into real associated models to make it easier to dump.

Fixed a problem with the validation that was causing it to throw incorrect errors on valid workflows. Getting it to behave a little more like the front end behaves, and respecting the read-only fields.  But it was mainly to do with always returning all the data with each form submission.
2020-06-02 18:17:00 -04:00
Dan Funk d8329e9eff I completely missed pushing up my changes last night before handing back over to Carlos. Sorry about that.
Added checks to assure the approver matches g.uid of the current user - otherwise don't modify the approval record.
2020-06-02 07:43:19 -04:00
Dan Funk 4090667a5d Missed committing a pile of changes last night. 2020-06-02 07:08:29 -04:00
Dan Funk c2a1b0175a Tweaks to approvals 2020-06-01 21:49:30 -04:00
Dan Funk 1db9401166 Don't put all the data into Spiff Tasks on a reload or backtrack, just store the data that gets submitted each time in the task log, and use that.
This should correct issues with parallel tasks and other complex areas - so we don't have tasks seeing data that isn't along their path.
2020-06-01 17:42:39 -04:00
Carlos Lopez 3c2849f40d Approval info addition 2020-06-01 15:29:59 -06:00
Carlos Lopez bec11980eb Fixing broken test by using proper FileSchema 2020-05-31 22:00:52 -06:00
Aaron Louie f0bd8d4f9e Adds approvals to study schema. Adds approvals endpoint 2020-05-31 22:46:32 -04:00
Dan Funk 9c7de39b09 This adds additional file data details to the study model as well. 2020-05-31 21:15:40 -04:00
Carlos Lopez dd6c1d2b42 Renaming approval files 2020-05-31 18:16:42 -06:00
Carlos Lopez 2e54f07095 Adding more info to files and renaming approval waiting status 2020-05-31 17:24:23 -06:00
Carlos Lopez 26809d1470 Waiting status renaming 2020-05-31 13:35:42 -06:00
Dan Funk 1f0e8741ba Run the validation twice, once completing all of the data, and a second time, completing only the required fields.
Also, add a helper method to reduce boiler plate code around Workflow Exceptions.
2020-05-30 17:26:27 -04:00
Dan Funk 860c475b29 Fill out repeating sections during validation process.
Also, when returning error messages, attempt to include the task data for the task that caused the error.
Also, when attempting to delete any file, respond with an API error explaining the issue, and log the details.
2020-05-30 15:37:04 -04:00
Dan Funk 895d2f8852 Slight change to the way the approval model is connected. 2020-05-29 05:04:18 -04:00
Dan Funk 0ba2819867 Merge branch 'dev' into feature/approval_request_script 2020-05-29 04:51:50 -04:00
Dan Funk 11413838a7 Faster lookup fields. We were parsing the spec each time to get details about how to search. We're just grabbing the workflow id and task id now and building that straight into the full text search index for faster lookups. Should be peppy.
Another speed improvement - data in the FileDataModel is deferred, and not queried until it is specifically used, as the new data structures need to use this model frequently.
2020-05-29 01:39:39 -04:00
Dan Funk dba41f4759 Ludicrously stupid launch in a refactor of the way all files work in the system at a time where I crave sleep and peace above all other things.
Added a File class, that we wrap around the FileModel so the api endpoints don't change, but File no longer holds refences to versions or dates of the file_data model, we
figure this out based on a clean database structure.

The ApprovalFile is directly related to the file_data_model - so no chance that a reviewer would review the incorrect version of a file.py

Noticed that our FileType enum called "bpmn" "bpmm", hope this doesn't screw someone up.

Workflows are directly related to the data_models that create the workflow spec it needs.  So the files should always be there.  There are no more hashes, and thus no more hash errors where it can't find the files to rebuild the workflow.py

Not much to report here, other than I broke every single test in the system at one point.  So I'm super concerned about this, and will be testing it a lot before creating the pull request.
2020-05-28 20:03:50 -04:00
Carlos Lopez 41b26e28bd Merge branch 'dev' into feature/approval_request_script 2020-05-28 08:11:31 -06:00