Commit Graph

196 Commits

Author SHA1 Message Date
Kelly McDonald 2b9cee6b89 Update database to include timezone and change all points where we set the time on an event to be utc time. If we get something in the database with a timezone, it will display properly on the front end, but by default everything will be put in the database in UTC 2021-04-29 10:25:28 -04:00
Kelly McDonald dc6f1cc80d Emulate checks on keyword arguments and then return. Validate only should have no side effects so the previous behavior was a bug. 2021-04-28 10:03:52 -04:00
Kelly McDonald 40ee20ecca Add file_data_get and file_data_set functions that can be accessed in a bpmn script function
Fixes #299
2021-04-26 09:55:28 -04:00
Kelly McDonald d3d7eeb309 Add file_data_get and file_data_set functions that can be accessed in a bpmn script function
Fixes #299
2021-04-26 09:41:14 -04:00
Dan 41310b6879 Allow both a keyword argument, a single argument when calling delete_file. And call it "delete_file" to be more conssitent with existing scripts. 2021-04-08 13:42:17 -04:00
Dan Funk 611c72f431
Merge pull request #284 from sartography/delete-uploaded-file-script-283
Delete uploaded file script #283
2021-04-08 12:42:30 -04:00
mike cullerton f8428378e2 You can now delete a document or list of documents.
Also, refactored the fancy bits into `process_document_deletion`
2021-04-05 15:11:06 -04:00
mike cullerton f0f5cff8d7 Added some checks to do_task to look for valid irb_doc_code and whether or not we found a file
Added code for do_task_validate
2021-04-05 13:15:09 -04:00
Dan 4fd8fd13f8 Remove approvals from the study_info script. It was broken, and isn't really possible in this way anymore 2021-04-05 12:52:23 -04:00
Dan 501487b31c Assure we convert to box for validations as well, and test the bloodly snot out of this. 2021-04-05 12:47:50 -04:00
Dan 727aa72e18 Fix a bug from recent performance improvments - be sure that our study info script returns dicts that can be referenced immediately with dot notation, as this is a common pattern in our bpmn script tasks. 2021-04-05 11:31:06 -04:00
mike cullerton 9317e49e65 Merge branch 'dev' into delete-uploaded-file-script-283 2021-04-05 10:40:33 -04:00
mike cullerton 0eb6291740 Script to delete an IRB document. Takes an `irb_document` argument.
Deletes all files associated with the workflow with that IRB document code
2021-04-05 08:55:02 -04:00
Kelly McDonald 116bf5e7aa Merge branch 'dev' into 263-optimize-dashboard
# Conflicts:
#	crc/services/study_service.py
2021-03-31 10:20:33 -04:00
Kelly McDonald 60f5be1aef Check in - pending change from @cullerton 2021-03-30 12:10:49 -04:00
Dan Funk 123ec50912
Merge pull request #272 from sartography/infinite-validations-262
Infinite validations 262
2021-03-26 19:49:42 -04:00
Dan Funk 6ab98c1dee
Merge pull request #267 from sartography/get-study-associates-validation-242
The `do_task_validate_only` method in the `get_study_associates` scri…
2021-03-19 15:41:46 -04:00
mike cullerton e2ba19748d Added department chair to the mocked data for study_info script.
The missing department chair caused an infinite loop in a workflow that checked for a department chair.
2021-03-16 13:36:55 -04:00
mike cullerton 70efdb4788 The `do_task_validate_only` method in the `get_study_associates` script now returns a mocked list of associates.
It used to return `True`, and this caused shield validation to fail when looping over the results of `get_study_associates`.
(You can't loop over a boolean)

Added a for loop in `study_sponsors_associate.bpmn` to test for this.
Moved `BaseTest` import to the top of `test_study_associate_script` because debug was failing.
2021-03-16 08:57:30 -04:00
mike cullerton dfa65ac276 We now perform an LDAP lookup for the associated user email addresses, instead of looking in the DB user table. 2021-03-15 10:30:23 -04:00
mike cullerton 67697b5076 Lots of changes!
Email Script:
- The email script now accepts 3 keyword arguments; subject, recipients, and cc.
- Subject and recipients are required.
- Recipients (and cc) can be an email address or list of addresses.
- You can use the string 'associated' in place of an email address, in which case we look up the users associated with a study that have send_email set to True.
- Moved some helper methods to email_service.

Email Service:
- Added cc argument
- Added helper methods previously in email script.
- Modified get_rendered_content to accept message and data directly. Previously, we passed in the task and grabbed them from there. Now we can use the method outside a task.
2021-03-11 11:32:21 -05:00
mike cullerton 7171219e8b removed unused import 2021-03-09 07:00:20 -05:00
mike cullerton 44ac55bc32 We can now send emails to users associated with a study, by using 'associated' as the recipient
We now use keyword arguments in the email script; subject and recipients
2021-03-09 06:49:39 -05:00
Kelly McDonald 06dedd8a32 Add short_title to study, change update_study task to use kw arguments and add the short_title to the update_study script 2021-03-02 10:03:53 -05:00
Dan Funk 3e0541fa15
Merge pull request #250 from sartography/206-extend-study-access
206 extend study access
2021-03-01 14:55:38 -05:00
mike cullerton 5c3d791bf3 UVA asked us to remove the extra whitespace in emails.
We removed the string replacement of '\n' -> '<br>' to solve this.
This means you can't purposely add blank lines any more.
Trade offs.
2021-02-25 11:08:08 -05:00
Kelly McDonald f592a8a0b0 Added a few more tests - I think I need to test one more script endpoint 2021-02-25 09:07:46 -05:00
Kelly McDonald 9bd3aa8dcb service and scripts with working test - still needs a few more tests to try out, but should work to get Mike going 2021-02-24 12:55:23 -05:00
Kelly McDonald c795dd7aea Merge branch 'dev' into 206-extend-study-access 2021-02-24 12:16:35 -05:00
Kelly McDonald 3dbe39c6fe initial checkin of changes for branch 2021-02-24 12:05:06 -05:00
Dan Funk 53fa49e668
Merge pull request #243 from sartography/183-remove-study-approval-code
Remove Approvals per ticket 183
2021-02-22 18:12:02 -05:00
Kelly McDonald 003bf2f9b9 Remove Approvals per ticket 183 2021-02-16 09:05:29 -05:00
mike cullerton d8bd203139 The email script was failing validation because of missing parameters in scripts.email.do_task_validate_only.
I added a test in tests.test_email_script to make sure we get the default email_address.
I also found and fixed a bug in workflow_service.populate_form_with_random_data where we would overwrite default_values with random text because of a missing continue.
(We don't want to continue if we have repeating fields though.)
2021-02-11 15:36:12 -05:00
mike cullerton d7a3afe8f4 changed the template to use the responsive html email template at https://github.com/leemunroe/responsive-html-email-template
added method get_cr_connect_wrapper to render our new html email template. This takes the email text generated from the task element documentation as the email_body.
2021-01-27 17:12:03 -05:00
mike cullerton 909af9b8d6 Change HTML emails so they look like the CR-Connect website. Add logo, color, style, etc.
Cleaned up email script a little.
Also, email script no longer uses services.mails. It uses email_service directly now.
2021-01-26 09:19:28 -05:00
mike cullerton 0c47d09972 Ticket 143. Test whether file is uploaded. 2021-01-04 10:53:21 -05:00
Dan Funk 386feddeed
Merge pull request #214 from sartography/validate-hide-expression-103
Modified study_info do_task_validate_only so that we take the default…
2020-12-31 10:53:27 -05:00
Dan efdbbcbc72 StudyInfo sometimes returns arrays of items, each item in the array should be coverted to a Box dictionary if possible. 2020-12-30 14:51:51 -05:00
mike cullerton f29429f0d0 Modified study_info do_task_validate_only so that we take the default path in ind_update workflow. 2020-12-30 10:04:10 -05:00
Dan e5a38874f6 A hard or soft reset should also cause a 'cancel_notify' which will kick off any CANCEL events. If this happens during
a hard or soft reset, and an error is thrown trying to fulfil a cancel event, the reset should still fire.
Sending emails still had a number of issues correctly parsing it's arguments.  This is corrected.
2020-12-29 18:05:13 -05:00
Dan Funk e54404e424
Merge pull request #205 from sartography/test-email-script
Test email script
2020-12-14 11:28:14 -05:00
mike cullerton 7defc2b02f Tests for uid in logs. Currently we test for uid in a response. This covers ApiError.
Currently, we don't have a test for Sentry. Unsure how to do this.
Also added a script, service and test workflow to help. (Also to learn about adding a script and service.)
2020-12-11 17:47:53 -05:00
mike cullerton 32c5060a31 No longer use eval on the email address. It is already parsed.
Tests for single and multiple emails, and email error checking.
Still need to figure out how to implement multiple emails.
2020-12-09 12:11:46 -05:00
mike cullerton 730d0ca18f Email script now uses an email address instead of a UVA LDAP user_id. 2020-12-07 16:23:41 -05:00
Dan b83ab1bcd1 Move data store base into its own file. 2020-11-18 15:34:50 -05:00
Kelly McDonald 70425eee0d Add last_updated field - change the api signature to be closer to what we need going forward, but retain the old methods for testing the script get/set through the API 2020-11-13 12:03:29 -05:00
Kelly McDonald 7c2c725840 save before changing how API works 2020-11-13 09:58:21 -05:00
Kelly McDonald 9a4d167dcd Add in api changes 2020-11-11 09:44:58 -05:00
Kelly McDonald cef221be32 Refactor to reuse code 2020-11-10 11:32:59 -05:00
Kelly McDonald 3ba11f5e6b add user_data_get/set and alter test so that it ensures that we don't grab study_data stuff from the user_data calls
change the test to mock up the user.
2020-11-10 10:32:37 -05:00
Kelly McDonald 245124779e Add mostly-working elements of the data-store, compare with dev to see why db is not working after running tests 2020-11-10 08:21:46 -05:00
Kelly McDonald e598994d34 Change the study_sponsors script so that we can actually call it :
var = study_info('sponsors')

to keep it more in line with other scripts that we have
2020-11-05 10:31:47 -05:00
Kelly McDonald 2931f45187 Add 'sponsors' to study_info call 2020-11-04 14:34:10 -05:00
Dan Funk b544334f45 1. Updating Personnel BPMN diagram to debug some issues.
2. Disabling the token timeout for now, to see if this corrects the issues Alex is having with lost work.
3. Raising more thoughtful error messages for unknown lookup options.
4. Providing better validation of default values and injecting the correct value for defaults related to enum lists of all types.
5. Bumping Spiffworkflow library which contains some better error messages and checks.
2020-09-01 15:58:50 -04:00
Dan Funk 5be35e8bab Adding a study sponsors script to return details about study sponsors 2020-08-25 10:30:46 -04:00
Dan Funk 8f5488d910 Don't raise an error when validating workflows that use the ldap script. 2020-08-17 15:45:37 -04:00
Dan Funk c12e2b97f3 calling ldap with no argument will return information about the current user. 2020-08-13 21:09:52 -04:00
Dan Funk e8067fe34e adding additional data to the study details returned during validation/shield testing. 2020-08-12 17:04:59 -04:00
Aaron Louie c25536dcca Makes Sonar happy. 2020-08-04 16:42:35 -04:00
Aaron Louie 2846bf9451 Fleshes out fake PB response, so workflows that pull from PB actually trigger validation errors 2020-08-04 15:50:29 -04:00
Dan Funk 9704cbcb26 Modifications to the ldap scripts to bring them back in line with what Kelly is doing with the evaluation process. 2020-07-30 13:35:20 -04:00
Dan Funk a6ec0c92c7 Merge remote-tracking branch 'origin/dev' into cr-connect-106-augment-eval-methods 2020-07-30 11:38:47 -04:00
Dan Funk 2905fc6173 Merge branch 'feature/ldap_lookup_script' of github.com:sartography/cr-connect-workflow into feature/ldap_lookup_script 2020-07-30 11:17:48 -04:00
Kelly McDonald d617af8565 All tests are passing - may need to refactor a bit, / remove comments 2020-07-28 11:02:49 -04:00
Kelly McDonald cc55aed89c Change exception name 2020-07-27 12:18:28 -04:00
Kelly McDonald a124e13c6a Replace all legacy style calls with new calls.
Still having issues where we try to eval an empty definition, not quite sure why there is a difference from what we had before. I may need to revert some of it and determine what is going on.
2020-07-24 14:33:24 -04:00
Kelly McDonald f5ab283538 Test of adding in the ability of augmenting the workflow to include internal scripts like StudyInfo
This is the first waypoint on a larger effort to make all of the 'special scripts' that currently require a shebang to be just another python function.
2020-07-24 12:08:46 -04:00
Carlos Lopez 41cbce8e01 Updating bpmn test 2020-07-21 21:08:08 -06:00
Carlos Lopez 522f848682 Addressing feedback 2020-07-21 20:54:03 -06:00
Carlos Lopez 313770d538 Ldap lookup script 2020-07-21 20:54:03 -06:00
Carlos Lopez a39cacdf00 Ldap lookup script 2020-07-19 21:53:18 -06:00
Dan Funk 6c6c5d315d Fixing a stupid mistake where I used an unquoted value in the mock data for validation, which created a dict that could not be serilized. 2020-07-16 13:28:00 -04:00
Dan Funk 419c05e46c
Merge branch 'dev' into fix/more-info-in-exceptions 2020-07-06 12:51:24 -04:00
Dan Funk a5cef8775e * Modifying the StudyInfo script to return both "investigators" and "roles", the investigators argument will return only the current active investigators according to the protocol builder. The "roles" argument returns all possible roles - it is what "investigators" use to be.
* The Task.title returned to the front end will now attempt to process the "display_name" property for dot-notation syntax, making it possible to use this for multi-instance tasks,
  but will work in all cases where we want he title to change based on values in the data model.
* Fixing a bug the test_study_api where it wasn't updated when we made recent changes to the different states of a study.
*
2020-07-06 12:09:21 -04:00
Carlos Lopez 817333f26e Adding proper exception information 2020-07-02 16:10:33 -06:00
Carlos Lopez 7116b582e8 Splitting commands properly without losing double quoted strings 2020-06-25 12:01:24 -06:00
Carlos Lopez 5d1ae402b6 Slight refactor on data passed to template rendering 2020-06-24 22:43:44 -06:00
Carlos Lopez e6d74aaa1a Removing extra index when parsing users info 2020-06-18 12:53:50 -06:00
Carlos Lopez 4db815a999 Handling incoming values from processor 2020-06-17 21:11:47 -06:00
Carlos Lopez 5ce279b663 Dropping silly pdb statement 2020-06-17 17:36:15 -06:00
Carlos Lopez 896ba6b377 Email relies now on markdown content 2020-06-17 17:00:16 -06:00
Carlos Lopez 2ce2dc73b5 Rendering proper content & organizing file structure for email tests 2020-06-17 16:09:38 -06:00
Carlos Lopez c730a7b1ec Sending subject and using default sender 2020-06-17 08:53:02 -06:00
Carlos Lopez 2ff836019f Sonarcloud fix 2020-06-16 18:55:18 -06:00
Carlos Lopez d4a285883f Email script 2020-06-16 18:42:36 -06:00
Dan Funk f0db5b70fc Adding some additional logic to the approval endpoint so that we take related approvals into account when setting the status. In addtion to prevous status options, there is a new status of "AWAITING" which means there are pending approvals before this approval that still need to be approved or canceled. 2020-06-05 14:33:00 -04:00
Dan Funk 59e91ec414 Don't add an empty approval record. 2020-06-02 08:35:19 -04:00
Dan Funk c790737bf0 The complete_template script might not grab the correct document when validating, fixed.
Updated to latest RRT files, and passing.
2020-06-01 12:33:58 -04:00
Dan Funk dba41f4759 Ludicrously stupid launch in a refactor of the way all files work in the system at a time where I crave sleep and peace above all other things.
Added a File class, that we wrap around the FileModel so the api endpoints don't change, but File no longer holds refences to versions or dates of the file_data model, we
figure this out based on a clean database structure.

The ApprovalFile is directly related to the file_data_model - so no chance that a reviewer would review the incorrect version of a file.py

Noticed that our FileType enum called "bpmn" "bpmm", hope this doesn't screw someone up.

Workflows are directly related to the data_models that create the workflow spec it needs.  So the files should always be there.  There are no more hashes, and thus no more hash errors where it can't find the files to rebuild the workflow.py

Not much to report here, other than I broke every single test in the system at one point.  So I'm super concerned about this, and will be testing it a lot before creating the pull request.
2020-05-28 20:03:50 -04:00
Dan Funk cd7f67ab48 A major refactor of how we search and store files, as there was a lot of confusing bits in here.
From an API point of view you can do the following (and only the following)

/files?workflow_spec_id=x
* You can find all files associated with a workflow_spec_id, and add a file with a workflow_spec_id
/files?workflow_id=x
* You can find all files associated with a workflow_id, and add a file that is directly associated with the workflow
/files?workflow_id=x&form_field_key=y
* You can find all files associated with a form element on a running workflow, and add a new file.
   Note: you can add multiple files to the same form_field_key, IF they have different file names. If the same name, the original file is archived,
   and the new file takes its place.

The study endpoints always return a list of the file metadata associated with the study.  Removed /studies-files, but there is an
endpoint called

/studies/all  - that returns all the studies in the system, and does include their files.

On a deeper level:
 The File model no longer contains:
  - study_id,
  - task_id,
  - form_field_key

Instead, if the file is associated with workflow - then that is the one way it is connected to the study, and we use this relationship to find files for a study.
A file is never associated with a task_id, as these change when the workflow is reloaded.
The form_field_key must match the irb_doc_code, so when requesting files for a form field, we just look up the irb_doc_code.
2020-05-28 08:27:26 -04:00
Dan Funk ccbf374b40 Loads of bug fixes.
Modifed the request_approval to take a list of arguments, which works better for us... today.
UpdateStudy correctly handles validation.
WorkflowService correctly populates random values from lookup tables.
And several fixes down in Spiffworkflow, including a big bug where only the last item in a decision table made it through.
2020-05-26 20:06:50 -04:00
Dan Funk be057e8758 Adding an "UpdateStudy" task that is able to update the data on the study model, useful for setting core data points on the model, such as setting the Primary Investigator, or altering the Study Title.
Fixing a bug where the validation of forms did not correctly process auto-complete fields.
Fixing a bug where the approvals script and the update study script could not process dot notation correctly.

Moved populate_random_data into the WorkflowService where it makes more sense.
2020-05-25 15:30:06 -04:00
Dan Funk 971d9a58e9 As we now have an approval_service.py, I moved all the business logic into this service and out of the request_approval.py script. And moved all tests for these features into a test file for the service. Will make it easier to cross reference what is happening, as everything all happens in one file.
As many of the scripts need to know the workflow, and it's down in a weird parameter, moved this so it is passed in each time.
2020-05-24 16:13:15 -04:00
Dan Funk d39ef658a2 Made some modifications to the Approval so that it knows exactly what versions of every file are being sent for approval
Added the following columns:
  * date_created - so we know when the file was created
  * renamed workflow_version to just "version", because everything has a version,  this is the version of the request.
  * workflow_hash - this is just a quick way to see what files and versions are associated with the request, it could be factored out.
  * study - a quick relationship link to the study, so that this model is easier to use.
  * workflow - ditto
  * approval_files - these is a list from a new link table that links an approval to specific files and versions.

The RequestApproval is logically sound, but still needs some additional pieces in place to be callable from a BPMN workflow diagram.

Altered the file service to pick up on changes to files vs adding new files, so that versions are picked up correctly as
users modify their submission - adding new files or replacing existing ones.  Deleting files worries me, and I will need to revisit this.

The damn base test keeps giving me a headache, so I made changes there to see if clearing and dropping the database each time won't allow the tests to pass more consistently.

Lots more tests around the file service to make sure it is versioning user uploaded files correctly.

The "Test Request Approval Script" tries to find to assure the correct behavior as this is likely to be called many times repeatedly and with little knowledge of the internal system.  So it should just "do the right thing".
2020-05-23 15:08:17 -04:00
Aaron Louie 58189285ad Cleans up 2020-05-20 00:12:48 -04:00
Aaron Louie 2b5687c3a3 Fixes pernicious bug where template document versions were not being updated properly, and template completion script was not honoring version numbers 2020-05-20 00:10:32 -04:00
Aaron Louie 481a9ed04c Gets image ids from task data and Injects images into jinja docx 2020-05-19 21:51:54 -04:00
Dan Funk 5c5c2a7312 Assure that new lines entered in text-fields are correctly added to the final word document. 2020-05-18 11:55:10 -04:00
Dan Funk 99503fb832 missed a file in the last commit. 2020-05-11 17:42:24 -04:00
Dan Funk b7c11fd893 Merge branch 'master' into feature/investigators_reference_file 2020-05-11 17:36:37 -04:00