Commit Graph

301 Commits

Author SHA1 Message Date
Kelly McDonald b857fddbad commit in intermediate changes to work on a bug 2021-07-26 13:00:11 -04:00
Kelly McDonald 48516e4009 Parent side working 2021-07-22 10:23:05 -04:00
Kelly McDonald 2455495457 got library side working - still need to have parent working 2021-07-22 10:20:34 -04:00
NWalker4483 4632c6374f Merge branch 'dev' into bug/missing_pi_name_246
# Conflicts:
#	crc/api.yml
#	crc/services/study_service.py
2021-07-20 11:44:11 -04:00
mike cullerton b0cf74fa3d Merge branch 'dev' into add-by-user-61
# Conflicts:
#	tests/files/test_files_api.py
2021-07-07 10:46:50 -04:00
mike cullerton c11b5cfb29 Add user_uid to api output 2021-07-07 08:18:02 -04:00
mike cullerton 23be257db0 Add user_uid column to file_data table 2021-07-06 17:07:47 -04:00
Dan cafdc4d100 Removing an unneeded modification to the database structure for lookup tables. 2021-07-06 14:02:07 -04:00
Dan 1b1a994360 Refactoring Reference files to use the lookup table, rather than parsing the results directly out of the spreadsheet, or attempting to cache them.
Adding a DocumentService to clean up the FileService, and get Documents well seperated, as it seems likely be pulled out or seperated in the future, there is now a Documents api file as well, for the same reason.
Some other minor changes are just fixing white space to assure our code is linting correctly.
I removed _create_study_workflow_approvals from the base test, as we don't use approvals like this anymore.
2021-07-06 13:10:20 -04:00
Kelly McDonald dce1d86e93 commit changes before looking at something else 2021-07-06 10:34:57 -04:00
Dan 61ad371680 Clean up the file api so we don't have a bunch of extra stuff in there. Just include the raw data from the document spraedsheet if it is available. 2021-06-22 15:08:08 -04:00
Dan 1f3febe2df Fix the panda warnings, which means we only support xlsx files now.
Include the document details when reutnring a file from the file model.
Improve the validation of file form elements.
2021-06-22 14:58:52 -04:00
Dan 07eb3f9ca8 Moving metrics into SpiffWorkflow so we can run the performance metrics deeply across both systems simultaniously.
Upgrading libraries.
Fixing deprication issue with Pandas and numpy.
We can only process xlsx files now, plain oldschool xls is fully removed.
2021-06-18 16:41:55 -04:00
Dan Funk 586a0f2f1a
Merge pull request #325 from sartography/feature/file_refactor
Prefer tasks that share a parent over just the the next available tas…
2021-06-08 11:28:23 -04:00
Dan 59f605c3df Prefer tasks that share a parent over just the the next available task when returning the next_task in the workflow processor. 2021-06-08 08:03:14 -04:00
mike cullerton 2e3d8c7343 Fix for DataStoreSchema missing the file_id column.
The file_id column is a foreign key.
2021-06-01 13:51:10 -04:00
Dan a1bb30e689 Switching from using "default" to "server_default" for all default time settings, and running migrations so that this is set at the database level rather than in python, to hopefully correct some issues with dates being 4 hours in the future. Having a very hard time replicating this issue locally. 2021-05-14 12:28:50 -04:00
Dan Funk 34759a2f3d
Merge branch 'dev' into feature/performance-refactor 2021-05-04 11:30:13 -04:00
Dan Funk d6054a9846
Merge branch 'dev' into 310-task-event-timezone 2021-05-04 11:21:36 -04:00
Dan Funk 77d9bfca43
Merge pull request #296 from sartography/launch-workflow-outside-study-204
Launch workflow outside study 204
2021-05-04 11:16:08 -04:00
Dan a719cf4bf9 When retrieving the study, only update the status of underlying workflows if specifically requested.
Record the size of a file in the database for quick access (this helps with a frontend refactor, so it isn't downloading the file just to see it's size)
Cleaning up the timing/performance metric reporting to make it easier to read.
Fixing a bug that prevented non-admins for getting the document-directory
2021-04-30 11:55:12 -04:00
mike cullerton 24c818bf31 Added study_id to workflow_api 2021-04-29 14:29:21 -04:00
Kelly McDonald c029dad688 Convert some datetime columns to include timezone 2021-04-28 10:00:22 -04:00
Dan Funk 21e95b106d
Merge pull request #295 from sartography/299-document-datastore
Add file_data_get and file_data_set functions that can be accessed in…
2021-04-26 12:49:46 -04:00
Dan 047c68b2b5 Just some quick tidying up. Dropping commented out code, and using the SQLAlchemyAutoSchema to handle serializing db models 2021-04-26 12:31:22 -04:00
Mike Cullerton 39fcab3730
Merge branch 'dev' into launch-workflow-outside-study-204 2021-04-26 10:27:32 -04:00
Kelly McDonald 40ee20ecca Add file_data_get and file_data_set functions that can be accessed in a bpmn script function
Fixes #299
2021-04-26 09:55:28 -04:00
mike cullerton 22432aaf2c study_id is no longer required for task events 2021-04-26 08:36:09 -04:00
mike cullerton c0655c9d03 Added `standalone` column to workflow_spec
Added `user_id` column to workflow
2021-04-26 08:33:55 -04:00
Dan ecff7d20d3 Adding a test for the new endpoint and cleaning up the API documenation 2021-04-21 14:00:19 -04:00
Kelly McDonald e2197ddab9 Changes to support the frontend data pane
fixes #277 and requires frontend branch 277-document-pane
2021-04-20 08:12:27 -04:00
mike cullerton d7894f43b6 These changes relate to changes in PB Mock ticket 273, where we removed Q_COMPLETE as a column in the PB Mock Study table. It is now a relationship.
Here, we removed Q_COMPLETE from ProtocolBuilderStudy and ProtocolBuilderStudySchema definitions to accommodate the PB Mock changes.

***This push will need to be coordinated with ticket 273***
2021-04-02 12:01:00 -04:00
Dan b64b52e7cb Assure that we properly return the new status message over the api calls. 2021-03-26 19:47:31 -04:00
Nile Walker a08c6b794f Updated Study Associate Endpoints 2021-03-22 17:30:49 -04:00
Dan 1000b22a3b Don't raise errors on non-ready tasks that calculate a title, the information may not be avaiable yet.
Also Allow autocomplete-num as a valid field property.
2021-03-12 15:40:23 -05:00
Dan 9e758c57d3 Merge branch 'dev' into bug/225_enum_lookup_same_field_name 2021-03-02 14:15:58 -05:00
Kelly McDonald 06dedd8a32 Add short_title to study, change update_study task to use kw arguments and add the short_title to the update_study script 2021-03-02 10:03:53 -05:00
Dan aac3d5c16e Bug #255, this requires the front end to pass in the name of the task, when doing a lookup. This will prevent a bug where we have multiple user tasks, with enum fields that set the same variable, but use different lookup tables to populate the dropbown or search feature. 2021-03-01 14:54:04 -05:00
Kelly McDonald c795dd7aea Merge branch 'dev' into 206-extend-study-access 2021-02-24 12:16:35 -05:00
Kelly McDonald 3dbe39c6fe initial checkin of changes for branch 2021-02-24 12:05:06 -05:00
Dan Funk 349c3d657a
Merge branch 'dev' into 201-changes-to-in-progress 2021-02-22 18:17:59 -05:00
Dan Funk 53fa49e668
Merge pull request #243 from sartography/183-remove-study-approval-code
Remove Approvals per ticket 183
2021-02-22 18:12:02 -05:00
Kelly McDonald 4fa0f9720a Allow frontend to honor 'view as' and fix problem with tests 2021-02-18 11:25:17 -05:00
Kelly McDonald 8c309e9a41 Add user display name to the response for task events 2021-02-18 08:44:44 -05:00
Kelly McDonald 118cf62172 Merge remote-tracking branch 'origin/dev' into STG-181_frontend-ux-model-changes 2021-02-16 10:07:34 -05:00
Kelly McDonald 003bf2f9b9 Remove Approvals per ticket 183 2021-02-16 09:05:29 -05:00
Kelly McDonald da0af14724 Changes to enable front-end changes
Added a human readable name to created user

added last updated task time and name,

added is_review on the workflow which is triggered by a swimlane with a name present.

Notably - it does NOT update or change any of the performance concerns that we were having with this.
2021-02-10 11:58:19 -05:00
mike cullerton 777cfbdecd We now test whether we have a valid StudyModel when getting a study by study model.
We raise an ApiError if the model is None or empty.
2021-02-09 17:37:55 -05:00
Dan b0dc834682 Don't error out on autocomplete_num 2021-01-04 13:41:57 -05:00
mike cullerton ede5df4df5 Studies can now be put on hold in the dashboard. We were overriding the status in StudyModel.update_from_protocol_builder. 2020-12-30 17:49:59 -05:00
Dan 1f9bf72c59 Merge branch 'dev' into bug/navigation 2020-12-14 11:46:44 -05:00
Dan 6bf24cc438 fixing some failing tests related to changes in the underlying spiffworkflow library. 2020-12-14 11:27:38 -05:00
Dan 02ea414b94 Additional fixes to Navigation to allow a nested navigation structure. 2020-12-14 10:07:19 -05:00
Kelly McDonald 0e1aa59fa1 Make a change to make sure that if there is a new file locally that is not present remotely when we pull from the remote, the new local file gets deleted.
Also: add several things to the requirements.txt that should have been there in the first place.
2020-12-08 13:42:01 -05:00
Dan 93b12a8e82 updates using new navigation from spiff workflow's navigation branch, all tests passing. 2020-12-04 17:56:12 -05:00
Kelly McDonald e9979efb0d Add some tests 2020-11-17 09:33:29 -05:00
Kelly McDonald 70425eee0d Add last_updated field - change the api signature to be closer to what we need going forward, but retain the old methods for testing the script get/set through the API 2020-11-13 12:03:29 -05:00
Kelly McDonald 7c2c725840 save before changing how API works 2020-11-13 09:58:21 -05:00
Kelly McDonald acaf633b45 Added ID as pk 2020-11-11 11:36:25 -05:00
Kelly McDonald 245124779e Add mostly-working elements of the data-store, compare with dev to see why db is not working after running tests 2020-11-10 08:21:46 -05:00
Dan Funk b544334f45 1. Updating Personnel BPMN diagram to debug some issues.
2. Disabling the token timeout for now, to see if this corrects the issues Alex is having with lost work.
3. Raising more thoughtful error messages for unknown lookup options.
4. Providing better validation of default values and injecting the correct value for defaults related to enum lists of all types.
5. Bumping Spiffworkflow library which contains some better error messages and checks.
2020-09-01 15:58:50 -04:00
Dan Funk 53d09303d8 Validating that field properties are valid - they must exist as constants on the Task model.
Making all the lookup field names consistent.
Fixing the lookup service which was failing at times trying to find the correct field to use for building the lookup table.
Updating validation to check for additional fields and properties.
When connexion level errors occur, wrapping it in an API Error to be consistent.
2020-08-27 14:00:14 -04:00
Dan Funk 85ad477b2b Some minor cleanup on the study status and automatic events. I wanted to avoid having one database model automatically generating other database models as a side effect. The study service now has full responsiblity to recording study events.
To help in running tests, adding __init__.py methods to all the test directories.
2020-08-17 14:56:00 -04:00
Carlos Lopez 369fe4bb35 Automatic events refactoring 2020-08-14 08:36:46 -06:00
Dan Funk 9a5c1d7cfb I may have finally wrapped my head around full text search in python. Now properly using an index based on simple rather than english dictionary which has far fewer stop words and stemming processes and plays much better to the type ahead search we are trying to provide.
Stop words are no longer excluded, so "other" is a valid search and gets a result.
2020-08-13 18:13:41 -04:00
Aaron Louie 9a31d591ae Adds user uid and cascades study event in study model. Fixes migration to properly downgrade enums. 2020-08-12 10:13:23 -04:00
Carlos Lopez a561a5cddc Adjusting study events attribute 2020-08-08 18:25:01 -06:00
Carlos Lopez bf3dae1885 Collecting user generated study changes 2020-08-05 20:29:05 -06:00
Aaron Louie b4595b345a Raises ApiError for 'display_name' extension property exception 2020-08-04 13:59:50 -04:00
Carlos Lopez 50b207307a Merge branch 'dev' into feature/update_study_status_latest 2020-07-31 14:26:24 -06:00
Carlos Lopez 2c2967e633 Normalizing enum values 2020-07-31 11:19:50 -06:00
Carlos Lopez 5d23223e51 New study status update 2020-07-30 21:03:11 -06:00
Aaron Louie faba0f55ab Adds AdminSession model and refactors impersonation methods to use it. 2020-07-30 12:40:53 -04:00
Aaron Louie 1b0ebecbf4 Uses Flask session to store impersonation state. 2020-07-30 10:17:02 -04:00
Aaron Louie 63537d7765 Adds is_admin boolean flag to user schema 2020-07-29 22:45:56 -04:00
Carlos Lopez de49397549 Adjustings tests for protocol builder status changes 2020-07-29 10:51:34 -06:00
Carlos Lopez 73a6b7adf1 Fixing tests 2020-07-29 10:25:57 -06:00
Carlos Lopez 6aa21638de Updating properly study status & fixing tests 2020-07-29 10:25:57 -06:00
Aaron Louie f897ee3aea Merge branch 'feature/approvals_dashboard_prototype' into feature/admin_impersonations 2020-07-29 10:32:46 -04:00
Dan Funk 0ea4c13d09 Convert protocol builder status to always be in lower case in order to better match the front end. And also fixing an issue with the multi_instance that is oddly broken suddenly, and I don't know why. 2020-07-28 17:16:48 -04:00
Dan Funk 4d11fc04a0 dropping the "read_only" flag in favor of a "do_engine_steps" flag, which more clearly defines what is happening. 2020-07-28 13:51:29 -04:00
Dan Funk f15626033d Allow the workflow to be requested without making changes to the workflow - requires that you specify a read_only flag of true, otherwise it assumes that you want a fully prepared workflow with the next ready task set to run. 2020-07-28 13:33:38 -04:00
Dan Funk 300026cbc8 Expanding the task events endpoint to accept workflow and study ids as additional filters.
Removing events from the study endpoint, too noisy.
2020-07-28 10:16:48 -04:00
Dan Funk 0cb480801b Provide event data on the Study api endpoint.
Speed up the tests a little, because that got out of hand.  Need to dig into what is causing this problem.
2020-07-27 17:05:01 -04:00
Dan Funk 452f2c3723 Building out a user service for getting the current user, it will provide a number of functions, one of which will allow administrative users to impersonate other users in some circumstances (but will assure that we log events correctly when an impersonation occures) 2020-07-27 14:38:57 -04:00
Aaron Louie b87f55fbd7 Exposes date in TaskEvent endpoint 2020-07-22 14:48:10 -04:00
Aaron Louie 855f5544e5 Adds enrollment_date to study model 2020-07-22 09:35:08 -04:00
Dan Funk 3d9eeab502 Add a few more details to the workflow metadata model. 2020-07-21 15:18:08 -04:00
Aaron Louie d67105332f Merge branch 'dev' into feature/lanes_and_roles 2020-07-16 14:00:53 -04:00
Dan Funk d85ca1ce51 Whenever a workflow is loaded or updated, add events to the TaskEvent table with assignments for the next set of ready tasks, along with who should complete those tasks.
Add the lane information to the Task model.
Drop the foreign key constraint on the user_uid in the task log, as we might create tasks for users before they ever log into the system.
Add a new endpoint to the API called task events.  It should be possible to query this and get a list of all tasks that need a users attention.
The task events returned include detailed information about the workflow and study as sub-models
Rename all the actions in event log to things that are easier to pass over the api as arguments, make this backwards compatible, updating existing names in the database via the migration.
Throughly test the navigation and task details as control of the workflow is passed between two lanes.
2020-07-14 22:16:44 -04:00
Dan Funk 9077ff3ebf It is not possible to use task_data for an auto-complete field. It's too expensive an operation to provide that feature on the backend, and the data already fully resides on the front end anyway. Task-data can be used to populate enum fields if needed, so it can populate dropdowns, radios and checkboxes, just not auto-complete. 2020-07-14 11:38:48 -04:00
Dan Funk a48322ef6a Partial work on CR Connect Roles.
Adding checks in the API to assure the correct person is completeing a task based on the task's lane.
Adding lane to the Navigation item.
Adding a check to assure that unique user ids can be identified using task.data
Added some additional ldap entries to make testing and development easier.
Removed a big chunk of duplicate code from task_tests_api
Modified some of the helper functions to make it easier to test as specific users.
Added some additional bpmn models to the tests for testing lanes and roles.
2020-07-14 10:29:25 -04:00
Aaron Louie b7920989ed WIP: Adds Camunda property for retrieving enum field options from task data. 2020-07-10 14:48:38 -04:00
Aaron Louie 1f454536e3 Renames TOKEN_AUTH_SECRET_KEY to SECRET_KEY 2020-07-10 11:26:15 -04:00
Aaron Louie 7f0b35ff27 Merge branch 'dev' into feature/enum_lookup_data 2020-07-10 10:27:51 -04:00
Carlos Lopez c7b864f9c7 Cleaning up old notes 2020-07-07 08:32:20 -06:00
Aaron Louie 1d9fef380a Merge branch 'fix/more-info-in-exceptions' into feature/enum_lookup_data 2020-07-03 14:22:11 -04:00
Aaron Louie f344fe60c0 WIP: Adds enum field lookup option data 2020-07-03 12:20:10 -04:00
Carlos Lopez 817333f26e Adding proper exception information 2020-07-02 16:10:33 -06:00
Aaron Louie 340ae68eed Sets PB status to Active by default 2020-07-01 11:01:36 -04:00