Commit Graph

976 Commits

Author SHA1 Message Date
Dan 620b9a5188 Fixing a regression. It's critical that Spiffworkflow's box implement deepcopy, as this is used by Jinga prior to generating a word document. 2021-05-05 21:36:57 -04:00
Dan 5a79b80f32 fixes #322, do not error out deleting files that have assocaited data stores, just remove the data store. 2021-05-05 20:21:33 -04:00
Dan 7831bef050 Don't fail the sync completely when a remote file does not exist. 2021-05-05 15:59:00 -04:00
Kelly McDonald cbd1d01203 Add URL to the study_info('documents') script
fixes #321 - I merged in branches that fix #320 and #297

320-add-default-for-file-data-get
297-filename-in-documents
2021-05-05 11:30:08 -04:00
Kelly McDonald 7ed30ef25a Merge branch '320-add-default-for-file-data-get' into 321-Markdown-Macro 2021-05-05 11:13:01 -04:00
Dan 8f28970f92 Resolving an alembic conflict.
Upgrading libraries, and resolving issues from that upgrade, including changes to how we manage tokens.  This seems to be working locally.
2021-05-04 13:39:49 -04:00
Dan Funk 8b0afa4437
Merge pull request #304 from sartography/irb-info-script-307
Irb info script #307
2021-05-04 11:40:52 -04:00
Kelly McDonald 7b3287f67b Save a sample file while I try to figure out how the JS is going to work 2021-05-04 11:32:34 -04:00
Dan Funk 34759a2f3d
Merge branch 'dev' into feature/performance-refactor 2021-05-04 11:30:13 -04:00
Dan Funk 4780240103
Merge pull request #302 from sartography/320-add-default-for-file-data-get
Add default capability to file_data_get script
2021-05-04 11:28:40 -04:00
Dan Funk d6054a9846
Merge branch 'dev' into 310-task-event-timezone 2021-05-04 11:21:36 -04:00
Dan Funk 81467d6a8d
Merge pull request #299 from sartography/297-filename-in-documents
Add filename to files listing.
2021-05-04 11:19:59 -04:00
Dan Funk d68f319cf7
Merge pull request #298 from sartography/316-Shield-Test-Error
Emulate checks on keyword arguments and then return. Validate only sh…
2021-05-04 11:18:36 -04:00
Dan Funk 77d9bfca43
Merge pull request #296 from sartography/launch-workflow-outside-study-204
Launch workflow outside study 204
2021-05-04 11:16:08 -04:00
mike cullerton 7689281d68 Fixed typo. Response should be list, not dict. 2021-05-03 14:52:29 -04:00
mike cullerton 804926dbac Fixed problem where WorkflowService.make_test_workflow did not add a study to the workflow_model 2021-05-01 15:55:20 -04:00
mike cullerton 060ee5076d New script to get IRB Info for a study 2021-05-01 15:52:41 -04:00
mike cullerton 22b3230243 Added service `get_irb_info` to `crc.services.protocol_builder.ProtocolBuilderService`
Added environment variable `PB_IRB_INFO_URL` for new `get_irb_info` service
2021-05-01 15:20:47 -04:00
Dan a719cf4bf9 When retrieving the study, only update the status of underlying workflows if specifically requested.
Record the size of a file in the database for quick access (this helps with a frontend refactor, so it isn't downloading the file just to see it's size)
Cleaning up the timing/performance metric reporting to make it easier to read.
Fixing a bug that prevented non-admins for getting the document-directory
2021-04-30 11:55:12 -04:00
Kelly McDonald 7608a889fb Add default capability to file_data_get
script
2021-04-30 10:11:50 -04:00
mike cullerton 24c818bf31 Added study_id to workflow_api 2021-04-29 14:29:21 -04:00
Kelly McDonald 2b9cee6b89 Update database to include timezone and change all points where we set the time on an event to be utc time. If we get something in the database with a timezone, it will display properly on the front end, but by default everything will be put in the database in UTC 2021-04-29 10:25:28 -04:00
Kelly McDonald cc77a9182c Add filename to files listing.
Fixes #297
2021-04-28 10:37:14 -04:00
Kelly McDonald dc6f1cc80d Emulate checks on keyword arguments and then return. Validate only should have no side effects so the previous behavior was a bug. 2021-04-28 10:03:52 -04:00
Kelly McDonald c029dad688 Convert some datetime columns to include timezone 2021-04-28 10:00:22 -04:00
Kelly McDonald 71a63c049d Merge remote-tracking branch 'origin/dev' into dev 2021-04-27 12:16:36 -04:00
Dan 248ca6a33a Don't require people to be admins just so they can see the list of documents, or we get recursive loads on Kelly's non admin account (and everyone elsees)
todo: Create a ticket to assure we don't get in an infinite loading loop.
2021-04-27 12:14:37 -04:00
Kelly McDonald 799747d638 Add filter for non-active studies so that the events for them do not get returned to the front end for the 'InProgress' data pane. 2021-04-27 12:13:49 -04:00
Dan Funk 21e95b106d
Merge pull request #295 from sartography/299-document-datastore
Add file_data_get and file_data_set functions that can be accessed in…
2021-04-26 12:49:46 -04:00
Dan 047c68b2b5 Just some quick tidying up. Dropping commented out code, and using the SQLAlchemyAutoSchema to handle serializing db models 2021-04-26 12:31:22 -04:00
Mike Cullerton 39fcab3730
Merge branch 'dev' into launch-workflow-outside-study-204 2021-04-26 10:27:32 -04:00
Kelly McDonald 40ee20ecca Add file_data_get and file_data_set functions that can be accessed in a bpmn script function
Fixes #299
2021-04-26 09:55:28 -04:00
Kelly McDonald d3d7eeb309 Add file_data_get and file_data_set functions that can be accessed in a bpmn script function
Fixes #299
2021-04-26 09:41:14 -04:00
mike cullerton 077b68c1e2 When creating a workflow for a study, set the user_id to None 2021-04-26 08:46:19 -04:00
mike cullerton f17a9dc0f4 Modified `get_users_assigned_to_task`.
If we are running a standalone workflow, only return the current user.
2021-04-26 08:45:08 -04:00
mike cullerton b6f5001687 Added methods for the 2 new api endpoints 2021-04-26 08:43:14 -04:00
mike cullerton 22432aaf2c study_id is no longer required for task events 2021-04-26 08:36:09 -04:00
mike cullerton c0655c9d03 Added `standalone` column to workflow_spec
Added `user_id` column to workflow
2021-04-26 08:33:55 -04:00
mike cullerton fa818bd751 Code for the 2 new api endpoints
Also modified `update_task` so that it no longer requires a study_id
2021-04-26 08:31:12 -04:00
mike cullerton 5936b2e8ef Created 2 api endpoints; one to get a workflow from a workflow spec, and one to list all standalone workflow specs.
Also added `standalone` argument to workflow_spec schema properties
2021-04-26 08:27:24 -04:00
Dan dd7c5fd1c1 Fixes #306 assure that we take the doc_code into account if passed in as an argument to the API for get_files. 2021-04-23 16:16:19 -04:00
Dan cf051a7d4f Merge branch 'dev' into 277-Document-pane
# Conflicts:
#	crc/api/file.py
2021-04-21 14:09:44 -04:00
Dan ecff7d20d3 Adding a test for the new endpoint and cleaning up the API documenation 2021-04-21 14:00:19 -04:00
Kelly McDonald 19e94da6b6 Fix slight bug in expansion code,
a non-expanded node was causing all parent levels to not expand.
2021-04-20 11:11:11 -04:00
Kelly McDonald e2197ddab9 Changes to support the frontend data pane
fixes #277 and requires frontend branch 277-document-pane
2021-04-20 08:12:27 -04:00
Dan Funk 277beb345f
Merge pull request #290 from sartography/auto-set-primary-bpmn-142
Auto set primary bpmn #142
2021-04-16 13:36:02 -04:00
mike cullerton 2374400542 Automatically set the first workflow for a workflow_spec to primary 2021-04-15 12:39:12 -04:00
mike cullerton 91e80d01bc Added delete_files argument to the workflow_processor reset method.
Added code to delete the files when delete_files is True.
2021-04-15 11:25:58 -04:00
mike cullerton 3332b9c7c0 Added delete_files argument to the restart_workflow api method, and pass it along to WorkflowProcessor.reset 2021-04-15 11:23:20 -04:00
mike cullerton 4615649938 Added delete_files argument to the workflow restart api endpoint 2021-04-15 11:21:51 -04:00
Kelly McDonald 949f3be403 give an endpoint to return all files associated with a study 2021-04-12 12:23:33 -04:00
Dan 6427a54d74 Set the correct default value on enumerations - if it is a multi-select enum (checkbox is set) then it should return a default value of an empty array, and not a dictionary. 2021-04-09 08:40:58 -04:00
Dan 41310b6879 Allow both a keyword argument, a single argument when calling delete_file. And call it "delete_file" to be more conssitent with existing scripts. 2021-04-08 13:42:17 -04:00
Dan Funk 611c72f431
Merge pull request #284 from sartography/delete-uploaded-file-script-283
Delete uploaded file script #283
2021-04-08 12:42:30 -04:00
mike cullerton f8428378e2 You can now delete a document or list of documents.
Also, refactored the fancy bits into `process_document_deletion`
2021-04-05 15:11:06 -04:00
mike cullerton f0f5cff8d7 Added some checks to do_task to look for valid irb_doc_code and whether or not we found a file
Added code for do_task_validate
2021-04-05 13:15:09 -04:00
Dan 4fd8fd13f8 Remove approvals from the study_info script. It was broken, and isn't really possible in this way anymore 2021-04-05 12:52:23 -04:00
Dan 501487b31c Assure we convert to box for validations as well, and test the bloodly snot out of this. 2021-04-05 12:47:50 -04:00
Dan 727aa72e18 Fix a bug from recent performance improvments - be sure that our study info script returns dicts that can be referenced immediately with dot notation, as this is a common pattern in our bpmn script tasks. 2021-04-05 11:31:06 -04:00
mike cullerton 9317e49e65 Merge branch 'dev' into delete-uploaded-file-script-283 2021-04-05 10:40:33 -04:00
mike cullerton 0eb6291740 Script to delete an IRB document. Takes an `irb_document` argument.
Deletes all files associated with the workflow with that IRB document code
2021-04-05 08:55:02 -04:00
Dan dcb4e7b736 fixing an error by taking a bit of the magic out of one of Mike's regex hinting tools. I figure the hint is still good, and a little less bittle, and all the information is already there if people need to dig. 2021-04-02 17:58:27 -04:00
Dan 3ad3a086ba Merge branch 'dev' into 263-optimize-dashboard 2021-04-02 17:12:02 -04:00
Dan 2100f27976 fixing the api.yml file which was starting to throw errors when updating the libraries.
Seems that sqlalchemy now has a hard time incrementing sequences, so putting in a fix for this.
Upgrading many of our libraries, to avoid any disconnects as we try to handle security patches from these automated bumps.
2021-04-02 17:10:05 -04:00
mike cullerton d7894f43b6 These changes relate to changes in PB Mock ticket 273, where we removed Q_COMPLETE as a column in the PB Mock Study table. It is now a relationship.
Here, we removed Q_COMPLETE from ProtocolBuilderStudy and ProtocolBuilderStudySchema definitions to accommodate the PB Mock changes.

***This push will need to be coordinated with ticket 273***
2021-04-02 12:01:00 -04:00
Kelly McDonald 116bf5e7aa Merge branch 'dev' into 263-optimize-dashboard
# Conflicts:
#	crc/services/study_service.py
2021-03-31 10:20:33 -04:00
Kelly McDonald 9084866015 Update error message - it may be that we should spend a bit and see if we can make the Box error 2021-03-31 10:17:08 -04:00
Kelly McDonald 60f5be1aef Check in - pending change from @cullerton 2021-03-30 12:10:49 -04:00
Dan Funk b5d4df595f
Merge pull request #271 from sartography/master-workflow-status-messages-222
Master workflow status messages 222
2021-03-26 20:52:24 -04:00
Dan ded11ddd82 fixing a code smell 2021-03-26 20:25:34 -04:00
Dan Funk e79e810ae1
Merge pull request #273 from sartography/modify-email-template-215
Change location of `CR Connect` string. Move it left, closer to the logo
2021-03-26 19:50:16 -04:00
Dan Funk 123ec50912
Merge pull request #272 from sartography/infinite-validations-262
Infinite validations 262
2021-03-26 19:49:42 -04:00
Dan b64b52e7cb Assure that we properly return the new status message over the api calls. 2021-03-26 19:47:31 -04:00
Dan ac4877fecf Merge remote-tracking branch 'origin/dev' into master-workflow-status-messages-222 2021-03-26 17:47:57 -04:00
mike cullerton d9bf2a8a81 Change location of `CR Connect` string. Move it left, closer to the logo 2021-03-23 13:12:13 -04:00
Kelly McDonald f1ca44a0e1 Add primitive cache on the fileservice is_review, halves the time of loading the list on the primary dashboard 2021-03-23 10:17:48 -04:00
mike cullerton a201e49d21 The master workflow now generates a dictionary of dictionaries, rather than a dictionary of strings.
Each entry in status is a dictionary with 'status' and 'message' keys.
We updated the StudyService._update_status_of_workflow_meta method to accommodate the master workflow changes.

Changed the order of some if statements so we move forward while we have good data, and run all the error states at the end.
Added some comments to explain the cascading if statements

We changed the names of private methods to begin with one underscore, so they work in the test environment.
Changed  some internal calls to accommodate the underscore change.
2021-03-22 17:46:39 -04:00
Dan Funk 6ab98c1dee
Merge pull request #267 from sartography/get-study-associates-validation-242
The `do_task_validate_only` method in the `get_study_associates` scri…
2021-03-19 15:41:46 -04:00
Dan Funk 544f4a4418
Merge pull request #266 from sartography/study-associated-email-207-223
We now perform an LDAP lookup for the associated user email addresses…
2021-03-19 15:40:31 -04:00
mike cullerton 938079cf59 Lowered count ceiling to 100, from 1000. This significantly increased validation time, from about 3 minutes to about 10 seconds.
Changed ApiError call to from_task, instead of from_workflow_exception.
2021-03-17 14:52:19 -04:00
mike cullerton e2ba19748d Added department chair to the mocked data for study_info script.
The missing department chair caused an infinite loop in a workflow that checked for a department chair.
2021-03-16 13:36:55 -04:00
mike cullerton 529bfc34b3 We now check for an infinite loop during validation.
This happened when testing for a Department Chair in a workflow.
Our mocked data for study_info script did not have a Department Chair
2021-03-16 13:35:51 -04:00
mike cullerton c05b879dd4 We want to modify the master workflow so that it returns both a status, and a message explaining the status for each workflow.
I.e., from status = {'ids_full_submission': 'required',
                              'enter_core_info': 'disabled',
                              ...}
       to     status = {'ids_full_submission': {'status': 'required', 'message': 'required because...'},
                              'enter_core_info': {'status': 'disabled', 'message': 'disabled because...'},
                              ...}

Modified study_service.__update_status_of_workflow_meta to accommodate this change.
2021-03-16 12:09:42 -04:00
mike cullerton 70efdb4788 The `do_task_validate_only` method in the `get_study_associates` script now returns a mocked list of associates.
It used to return `True`, and this caused shield validation to fail when looping over the results of `get_study_associates`.
(You can't loop over a boolean)

Added a for loop in `study_sponsors_associate.bpmn` to test for this.
Moved `BaseTest` import to the top of `test_study_associate_script` because debug was failing.
2021-03-16 08:57:30 -04:00
mike cullerton dfa65ac276 We now perform an LDAP lookup for the associated user email addresses, instead of looking in the DB user table. 2021-03-15 10:30:23 -04:00
Dan Funk 14386b8ba9
Merge pull request #264 from sartography/bug/243_complete_all
Bug/243 complete all
2021-03-14 12:34:55 -04:00
Dan Funk 0da07675b7
Merge pull request #258 from sartography/study-associated-email-207-223
Study associated email 207 223
2021-03-14 12:34:19 -04:00
Dan 6f80d816cd My privous efforts didn't take into account the fact that we use dot notation in field names.
I've re-worked the workflow form endpoint, so that it only accepts the data that should be in the form, and ignores any other values that come back from the front end.  It seems Formly has some bugs that were introducing confusing information, and I want everything to behave consistently.
I had to re-work some of the tests, which were relying on an ability to set data through a form post without having a corresponding form to do so.
2021-03-14 12:20:39 -04:00
Dan 15c64c862e Adding an argument to update task that will allow it to complete all remaining tasks through an "update_all" flag,
this will allow us to remove some fairly complex logic from the front end that isn't behaving properly.
2021-03-12 20:41:07 -05:00
Dan 1000b22a3b Don't raise errors on non-ready tasks that calculate a title, the information may not be avaiable yet.
Also Allow autocomplete-num as a valid field property.
2021-03-12 15:40:23 -05:00
mike cullerton ef26238127 added CustomBPMNScriptEngine back in 2021-03-11 14:33:35 -05:00
Mike Cullerton e7c78c8d0e
Merge branch 'dev' into study-associated-email-207-223 2021-03-11 12:38:47 -05:00
mike cullerton e58f1dc484 Remove unused mails service 2021-03-11 11:56:18 -05:00
mike cullerton fa4707811e Removed the test and templates from the unused mails service.
Mails was removed in previous commit.
2021-03-11 11:34:24 -05:00
mike cullerton 67697b5076 Lots of changes!
Email Script:
- The email script now accepts 3 keyword arguments; subject, recipients, and cc.
- Subject and recipients are required.
- Recipients (and cc) can be an email address or list of addresses.
- You can use the string 'associated' in place of an email address, in which case we look up the users associated with a study that have send_email set to True.
- Moved some helper methods to email_service.

Email Service:
- Added cc argument
- Added helper methods previously in email script.
- Modified get_rendered_content to accept message and data directly. Previously, we passed in the task and grabbed them from there. Now we can use the method outside a task.
2021-03-11 11:32:21 -05:00
mike cullerton 80f1fc276f Modify api.tools.send_email to accommodate the changes to the api endpoint in api.yml
We now accept subject, address, body (the email message), and possible data.
We now use email_service to send the message, and wrap it with the CR Connect html formatting.

**** We import DEFAULT_SENDER from config.default. Will this be a problem on testing and production? ****
2021-03-11 11:19:02 -05:00
mike cullerton 14e62a7ec2 Modify the send_email api endpoint (in Configurator Tools section) to accept subject, message, and data.
Subject, address and message are required.
Data is not required.
Message is in the body of the request.
Subject, address, and data are in the query string.
2021-03-11 11:10:10 -05:00
Kelly McDonald d250bac32e Merge remote-tracking branch 'origin/dev' into dev 2021-03-10 10:59:24 -05:00
Kelly McDonald 2ed825aef3 Fixes problem with swtiching users fixes #237 2021-03-10 10:59:15 -05:00
Mike Cullerton eaf52a21ca
Merge pull request #260 from sartography/bug/delete_study
Allows us to delete a study even if there are emails, and associated …
2021-03-09 13:51:25 -05:00