Commit Graph

1313 Commits

Author SHA1 Message Date
mike cullerton be066b7d59 Add cc, bcc, and workflow_id to the email record, for reporting 2021-10-07 09:22:05 -04:00
mike cullerton 7551b99386 Add workflow_id to email_service call, for reporting
We now return the ID of the email model to the configurator, so they can track it.
2021-10-07 09:20:53 -04:00
mike cullerton dab233a07a Add columns to email model for reporting
Add Schema for email model
2021-10-07 09:19:37 -04:00
alicia pritchett 7b6beae9c1 fixes frontend bailing out
workflow id is workflow id
2021-10-06 14:33:09 -04:00
Dan 84680ea846 Fixing multiple issues that came out of Study Info, as we debugged issue #474 related to navigating back to a previous task.
There was a problem with the python script engine as well that wasn't handling the de-serialize properly and didn't correctly pick back up on the script engine, and the renaming of methods in PythonScriptEngine created some conflicts with the way we override functions.
We were not handling ldap looks up efficiently, and this was also breaking in Study Info.

Finally we had a bug in SpiffWorkflow that did not allow us to reset back to the previous task in some cases where nested call activities happen far later in the process and are currently active when the reset is created.
2021-10-06 12:17:57 -04:00
mike cullerton 572ca1db6b Fixes for backend code after removing `name` column from `workflow_spec` and `workflow_spec_category` tables 2021-10-05 14:17:41 -04:00
mike cullerton fcf538ca92 remove `name` column from workflow_spec and workflow_spec_category 2021-10-05 13:10:36 -04:00
Dan b02731df17 Add some additional checks in the workflow service so we assure the data we are creating during validation can be serialized and deserialized just as it would be through the API. (Was hanging us up on dates)
Assure that if we generate a default value for a date in the task data, it is stored as an ISO String.

remove any unserializable data from the task_data when an error is encountered, rather than just dropping all the task_data. This case seems to happen a lot and it leaves us with nothing to go on.
2021-10-04 15:07:47 -04:00
Dan 448b7b2773 respect proxy configurations. 2021-10-01 15:35:22 -04:00
Dan c3995bfc3e Another try at getting the url correct. This paired with an additional setting on the server maybe. 2021-10-01 14:25:00 -04:00
Dan 5924ee48f4 One more try at getting the currect url. 2021-09-30 16:56:27 -04:00
mike cullerton 5bf7bf3e2e Bug fix for adding master workflow to validation
We now check that a workflow is in the status dictionary before looking up its status.
2021-09-30 15:48:57 -04:00
mike cullerton 64e3cd044f Merge branch 'dev' into file-cleanup-466 2021-09-30 14:11:59 -04:00
mike cullerton f44b9836ce Add file cleanup to the scheduler 2021-09-30 14:11:33 -04:00
Dan Funk f898d6f502
Merge pull request #385 from sartography/ids-approval-validation-478
Ids approval validation #478
2021-09-30 13:59:26 -04:00
Dan Funk 6b5a55df4c
Merge pull request #383 from sartography/feature/476_file_url
URL for files
2021-09-30 13:47:37 -04:00
Dan Funk fbfb241c0b
Merge pull request #382 from sartography/feature/381_validate_field_types
fixes #381 Assure that the field type provided is supported.  Catches errors suc…
2021-09-30 13:46:40 -04:00
Dan Funk 615d0e7f89
Merge pull request #384 from sartography/chore/admin-cat-456
Admin flag on category + migration + updated test
2021-09-30 13:37:09 -04:00
Dan Funk 0353161402
Merge branch 'dev' into feature/381_validate_field_types 2021-09-30 13:19:36 -04:00
Dan Funk 1251202244
Merge pull request #379 from sartography/top-level-validation-460
Top level validation #460
2021-09-30 13:11:03 -04:00
mike cullerton 0202b409e1 My checkbox test was in a bad place. Moved it into each of the 2 blocks. 2021-09-30 12:34:46 -04:00
mike cullerton ba2741818d Added the Testing url as example data so we don't have to look it up and type it every time. 2021-09-30 11:36:39 -04:00
mike cullerton 73cd729cac Checkbox enums need to return a list. 2021-09-30 11:35:13 -04:00
Dan e002ffd363 Fixing a broken test. 2021-09-30 10:30:58 -04:00
alicia pritchett d5d4496cd0 Admin flag on category + migration + updated test 2021-09-29 16:53:59 -04:00
Dan 8a4a53f028 Assure that the field type provided is supported. Catches errors such as adding a custom field type of 'text_area' rather than "textarea" 2021-09-29 14:05:45 -04:00
Dan 8e00f16eaa Assure that the field type provided is supported. Catches errors such as adding a custom field type of 'text_area' rather than "textarea" 2021-09-29 11:43:08 -04:00
Dan d0c819dd5f sync was failing because it tried to add the libraries before adding the spec the libraries connect to. 2021-09-29 10:21:24 -04:00
Dan 92b9fea08d fixing a stupid mistake that made all the people I care about suffer yesterday afternoon. 2021-09-29 10:06:17 -04:00
mike cullerton 29798f1ba6 turned it on 2021-09-28 10:16:47 -04:00
mike cullerton c91e81e356 Clean up file data. Always keep at least 1 version. Do not delete if in lookup or dependency tables 2021-09-27 17:15:53 -04:00
mike cullerton ef5a0a7e45 Check whether a workflow spec is disabled by the master workflow before validating.
Return the master workflow message to the user
2021-09-24 11:05:33 -04:00
Dan Funk e4add14193
Merge pull request #378 from sartography/bug/246_user_ldap_refactor
Bug/246 user ldap refactor
2021-09-22 15:31:34 -04:00
Dan Funk 48d979dd5a
Merge pull request #377 from sartography/study-info-mislabeled-464
Study info mislabeled #464
2021-09-22 15:31:23 -04:00
Dan Funk deff293fff
Merge pull request #376 from sartography/bug/468_missing_file_date
fixes #448 - Missing  file date
2021-09-22 15:29:32 -04:00
Dan Funk e672d46dd0
Merge pull request #375 from sartography/study-info-adds-463
Study info adds #463
2021-09-22 15:28:40 -04:00
Dan 19104303de Refactor of the way we store and return details about users - All the details about a user, or individual associated with a study is returned within in an Ldap model. I've removed duplication between these models. This required some cleanup of the tests, and a migration that will drop the user details. 2021-09-22 13:16:25 -04:00
mike cullerton 8df6c78925 Cleaned up what we return from the description, for the tools/list_scripts API endpoint 2021-09-22 12:12:26 -04:00
mike cullerton d7b7f39128 Added a field `handle` that displays the filename, which is what configurators actually call in a script task. 2021-09-22 12:10:43 -04:00
Dan 4b591a076f Adding importlib-resournces as a dependency, which is suddenly failing on Travis, likely due to different versions of Python as I try to upgrade to 3.9 2021-09-21 14:36:57 -04:00
mike cullerton f101176b13 Add `short_name` and `proposal_name` to the update_study script 2021-09-17 11:54:22 -04:00
mike cullerton 8b8cfde6cb Add `short_name` and `proposal_name` to the study model 2021-09-17 11:53:45 -04:00
mike cullerton 829dc3661e Fixed name vs get_name bug where I had the wrong type of task (API model vs Spiff task) in my test
Also return good json
2021-09-16 13:43:45 -04:00
Dan Funk 72349cacf5
Merge pull request #373 from sartography/zip-documents-379
Bug fix #379
2021-09-15 15:42:35 -04:00
mike cullerton 6ca8ba3e69 Refactored to use Python tempfile 2021-09-15 13:33:55 -04:00
mike cullerton c335390025 Bug fix.
File can already exist if there was a previous problem running the script
2021-09-15 10:30:12 -04:00
Mike Cullerton 66bfbf0a49
Merge pull request #371 from sartography/zip-documents-379
Zip documents #379
2021-09-14 17:42:13 -04:00
Mike Cullerton 787614ee91
Merge pull request #369 from sartography/dmn-from-spreadsheet-395
Dmn from spreadsheet #395
2021-09-14 17:41:01 -04:00
Mike Cullerton 15efc2f3f1
Merge pull request #370 from sartography/library-listing-453
Library listing #453
2021-09-14 16:11:56 -04:00
alicia pritchett a7f032d372 Returns workflow spec categories ordered by display order
sorry to open a whole branch just for this little fix
2021-09-14 10:56:41 -04:00
mike cullerton 654c87489c New version of irb_documents spreadsheet, that includes the `zip_key_words` column 2021-09-14 10:40:47 -04:00
mike cullerton 512cf75df0 Modified to use the new `zip_key_words` column from irb_documents spreadsheet
Moved all the code inside the `if` statement
2021-09-14 10:39:53 -04:00
mike cullerton 444e3fa76d Merge branch 'dev' into zip-documents-379 2021-09-14 09:39:19 -04:00
mike cullerton 7b7c228a81 2 changes
- The `all_specifications` method is now explicit in what it returns; either `library`, `standalone`, or `regular` workflow specs

- When we add/edit a workflow spec, we make sure that `library` and `standalone` workflow specs do not get a category_id
2021-09-13 16:15:18 -04:00
mike cullerton 5ddff1d1be Merge branch 'dev' into add-reorder-api-endpoints-444 2021-09-09 13:10:54 -04:00
mike cullerton cc403f1fac Merge branch 'dev' into dmn-from-spreadsheet-395 2021-09-09 13:09:35 -04:00
mike cullerton 9bd012c822 Clean up display_order numbers when reordering.
This fixes the problem where we have don't have all unique display_order numbers.
I.e., 0, 1, 1, 3, 5
2021-09-08 11:16:49 -04:00
mike cullerton ce1af73d6e Remove unused import 2021-09-07 15:44:15 -04:00
mike cullerton 21987cf1f1 Remove old root definition.
We now use the 2019 spec
2021-09-07 15:38:53 -04:00
mike cullerton a690535c2d Removed script used in development 2021-09-07 15:36:50 -04:00
mike cullerton e630aeb94e Add DMNDI section.
Without it, the configurator could not parse the XML
2021-09-07 14:27:15 -04:00
mike cullerton 6e7b549136 Change root element to use 2019 spec 2021-09-07 12:11:27 -04:00
mike cullerton a3b3df1bbb Use flask send_file to return DMN file 2021-09-07 12:09:30 -04:00
Dan Funk 1e97f0ac31
Merge pull request #368 from sartography/feature/114_read_only_expression
#114 We just needed to allow read_only_expression
2021-09-03 14:32:56 -04:00
Dan Funk 86c017a6a6
Merge pull request #367 from sartography/value-error-expression-428
Value error expression #428
2021-09-03 14:32:46 -04:00
Dan Funk 1a008499c4
Merge pull request #366 from sartography/file-id-error-info-449
Added file_id to some error messages in api.file #449
2021-09-03 14:32:14 -04:00
mike cullerton 93e8015ddd Read the spreadsheet data sent from the front end. This allows pandas to see it as a file type object.
Fixed bug where column_count wouldn't get set if there isn't an Annotation column
2021-09-03 10:37:06 -04:00
mike cullerton e2b70da6b7 Not returning json. Returning stream of XML 2021-09-03 10:33:12 -04:00
mike cullerton 8677d0368b Grab the file data from the request, not as a parameter 2021-09-03 10:32:32 -04:00
Dan 5c0d30becb We just needed to allow read_only_expression 2021-09-03 09:48:16 -04:00
mike cullerton 1545226e06 We no longer save the DMN file, so we don't need the workflow_spec_id
(We will open the DMN in the editor. It will be saved there.)
2021-09-02 09:58:22 -04:00
mike cullerton 5f3968b582 Changed script to work with a file_id
This will all be deprecated, and we won't run this from a script
2021-09-02 09:55:31 -04:00
mike cullerton b5082b8fd3 Call FileService 2021-09-02 09:53:17 -04:00
mike cullerton 4fa68bd4c0 Move business logic to FileService 2021-09-02 09:52:58 -04:00
mike cullerton 0c31b091ee Merge branch 'dev' into dmn-from-spreadsheet-395 2021-09-01 09:38:03 -04:00
mike cullerton b88daef4fa Added task name to the error message, for configurators 2021-08-31 16:17:52 -04:00
mike cullerton 5db5c7ca43 Added file_id to some error messages in api.file 2021-08-31 15:28:21 -04:00
mike cullerton ed57993f8d Workflow spec cleanup needs a category_id 2021-08-31 12:17:25 -04:00
Dan 1140b21414 Adding a test that assures we pick up libraries on the remote system when syncing. 2021-08-31 12:07:41 -04:00
mike cullerton b92676ea75 Make sure we have good display_orders before adding specs and categories 2021-08-31 11:16:27 -04:00
mike cullerton 994ce4008f Don't let updates to workflow-spec and workflow-spec-category metadata change the display_order
Force display_order changes to use the new reorder API endpoints
2021-08-31 10:36:22 -04:00
mike cullerton f6816a2e42 Move display_order cleanup to their own methods 2021-08-31 10:07:38 -04:00
Dan d17a2c39ed A little cleanup to assure that when we sync we correctly pick up and associate librarires with a workflow when we sync. 2021-08-30 15:46:16 -04:00
mike cullerton b76f45368b Explicitly commit the changes 2021-08-30 15:29:49 -04:00
mike cullerton 7bbb64cf27 Reorder remaining categories after delete 2021-08-30 15:20:45 -04:00
mike cullerton c275c01526 Reorder remaining workflow specs upon deletion 2021-08-30 14:56:26 -04:00
mike cullerton eb6b73fde5 Fixed typo where I returned an ApiError, instead of raising it. 2021-08-30 13:55:21 -04:00
mike cullerton a99818b177 Make sure we don't try to move past the top or bottom of the list. 2021-08-30 11:42:44 -04:00
mike cullerton 070a9c9350 Code for reordering Workflow Spec Categories
Still need to work on edge cases where you try to move past top or bottom of list
Still need to work on tests
2021-08-30 11:27:07 -04:00
mike cullerton f2201a3cef Move business logic to WorkflowService
Return ordered list of specs
2021-08-30 10:41:08 -04:00
mike cullerton 41414e39c8 First pass at reorder spec code.
Still need to return list of specs
Still need to work on spec categories
2021-08-27 13:02:49 -04:00
mike cullerton f639e101b7 Commit the skeleton API endpoints, so we can use them while developing the BPMN side of things. 2021-08-27 11:40:38 -04:00
Dan 083abd1b58 Just being consistent in the names as we named it delete_task_data in the class, but not elsewhere, and since I'm here, using the name I prefer. 2021-08-26 14:22:12 -04:00
mike cullerton da914be3fa Add task_spec to data_store 2021-08-26 10:41:09 -04:00
mike cullerton 506157d58a Add task_spec_name to FileService.add_workflow_file, and script that calls it. 2021-08-26 10:40:47 -04:00
mike cullerton 4c41011299 Add task_spec_name to `add_file` API call 2021-08-26 10:37:05 -04:00
mike cullerton 3d7dadc319 Model changes and revision file for task_spec column to file and data_store models 2021-08-26 10:36:10 -04:00
mike cullerton ea08643bcd Script to delete files and task_events associated with a task 2021-08-26 10:34:21 -04:00
mike cullerton d990a5636f Grabbed the latest irb_documents spreadsheet, because of the new `zip` columns and updated categories 2021-08-26 08:43:11 -04:00
mike cullerton fb5b7f72d8 New script to created a zipped file of irb_documents for use as an email attachment.
The script accepts a list of file_ids, and generates a list of documents from these ids.
The documents are stored in a directory structure generated from the categories in the irb_documents spreadsheet
2021-08-26 08:31:01 -04:00
Mike Cullerton 961925e03e
Merge pull request #361 from sartography/430-email-enhancements
#430 email enhancements
2021-08-25 15:02:14 -04:00
Mike Cullerton fcbbeb4735
Merge pull request #360 from sartography/423-LibraryAppearingInDashboard
Fixes #423
2021-08-25 15:01:18 -04:00
Dan 4b9c5bd3e8 Fixing a stupid mistake. 2021-08-24 12:10:59 -04:00
Dan 0bd1c282a3 Merge branch 'dev' of github.com:sartography/cr-connect-workflow into dev 2021-08-20 13:46:08 -04:00
mike cullerton 32c72c5a40 Modified email script and email_service to accommodate new bcc, reply_to, and attachments arguments
Modified the email script description to add the new arguments
Cleaned up some import statements
2021-08-19 17:34:55 -04:00
mike cullerton ef9fd9514d This caused a key error if we had a repeating field that was unused 2021-08-19 17:32:24 -04:00
Kelly McDonald 76f8f5984e Fixes #413
Fixes #423 where a library was showing up in a study workflow

Alex said this used to work, but I don't see where I ever edited the code that needed a filter.
2021-08-19 10:47:13 -04:00
Mike Cullerton 5692f7d377
Merge pull request #359 from sartography/413-LibraryCleanup
413 library cleanup
2021-08-18 13:39:37 -04:00
Dan afaecfa21c Catching an error when variables conflict with repeating section variables. 2021-08-18 13:38:34 -04:00
Mike Cullerton feb72af32e
Update workflow.py
Just completing a comment so I understood
2021-08-18 13:27:28 -04:00
Kelly McDonald b4ecb0f97a Add backref cascade and make changes to the spot where we delete so it loads objects into memory and then uses the SQLAlchemy plumbing to make sure that the db object gets cleaned up. 2021-08-18 08:25:17 -04:00
Leashys 032aad9bc4 Fixes #383 - Timestamp on files is UTC?
also this includes the shield test thing for the boolean_type property whoopsie
2021-08-17 13:04:24 -04:00
Dan 7a8eeab820 Merge branch 'dev' into bug/missing_pi_name_246 2021-08-17 11:16:03 -04:00
Dan 6ce560cc2d Made a stupid mistake in a hot fix, causing the builds to fail. 2021-08-17 10:14:49 -04:00
Dan Funk c831ee7e4a
Merge pull request #357 from sartography/426-remove-hsr-number
remove hsr number #426
2021-08-17 09:58:56 -04:00
Dan 78f79a67b7 removing exectue flag. 2021-08-17 09:52:02 -04:00
Dan 4e0e162fd6 Removing the execution flag Nile keeps adding to files. 2021-08-17 09:50:57 -04:00
Dan 77de2a8f47 Merge branch 'dev' into bug/missing_pi_name_246 2021-08-17 09:47:09 -04:00
Dan 0a2597ae36 Just a quick hot fix so that when can still report errors back to the UI even when the task_data can't be serialized, which can happen when functions are defined. 2021-08-17 09:43:08 -04:00
nilez 07ffcd7c76 Dependencies In Testing are behind this avoids that issue. https://sonarcloud.io/project/issues?id=sartography_cr-connect-workflow&pullRequest=356&resolved=false&types=BUG 2021-08-16 23:46:39 -04:00
mike cullerton 2a2fd710c4 Merge branch 'dev' into dmn-from-spreadsheet-395 2021-08-16 14:22:02 -04:00
mike cullerton 3f44c202ca Merge branch '417-hsr-open-to-enrollment-bug' into 426-remove-hsr-number 2021-08-16 12:55:24 -04:00
mike cullerton b3fd72de8b Remove hsr_number from Study Schema 2021-08-16 12:51:48 -04:00
mike cullerton 1f82143aea Remove HSR Number from models 2021-08-16 12:50:09 -04:00
mike cullerton 46d7b13326 Start to remove HSR Number 2021-08-16 12:09:02 -04:00
nilez 0a232dace6 assertEquals -> assertEqual 2021-08-16 10:25:41 -04:00
nilez f8dd4c3eb9 Check Study Associates Endpoint 2021-08-16 09:32:02 -04:00
mike cullerton bd4a9cced3 We no longer use HSRNUMBER to automatically set Study Status to `Open for Enrollment` 2021-08-13 12:06:27 -04:00
nilez 4a2581e938 get studies raises and error when invalid studies are the only studies associated with the user 2021-08-12 14:03:44 -04:00
nilez 7027392a1b Merge remote-tracking branch 'origin/bug/missing_pi_name_246' into bug/missing_pi_name_246
# Conflicts:
#	crc/services/workflow_service.py
#	tests/study/test_study_associate_script.py
2021-08-12 13:01:41 -04:00
nilez e32c1db4c8 Merge remote-tracking branch 'origin/dev' into bug/missing_pi_name_246
# Conflicts:
#	crc/models/study.py
#	crc/services/study_service.py
#	crc/services/workflow_service.py
#	tests/study/test_study_associate_script.py
2021-08-12 12:59:15 -04:00
Dan 000de03733 Just a quick hot fix so that when can still report errors back to the UI even when the task_data can't be serialized, which can happen when functions are defined. 2021-08-12 12:49:19 -04:00
nilez 41e27cb454 Merge remote-tracking branch 'origin/dev' into bug/missing_pi_name_246
# Conflicts:
#	crc/models/study.py
#	crc/services/study_service.py
#	crc/services/workflow_service.py
#	tests/study/test_study_associate_script.py
2021-08-12 12:39:04 -04:00
nilez d20b68e483 Fixed Deprecation Warnings in Test Files 2021-08-12 12:33:27 -04:00
nilez 2732071585 Merge branch 'dev' into bug/missing_pi_name_246 2021-08-12 12:07:25 -04:00
Dan 40727c7ce6 avoid erroring out when unable to calculate a value expression as a part of finding a default value. 2021-08-12 11:35:05 -04:00
Dan Funk adb1b74822
Merge pull request #352 from sartography/post-process-empty-field-408
Post process empty field #408
2021-08-12 10:53:26 -04:00
mike cullerton d07bac27ca Fix for hidden file data field.
We were processing a field that didn't have a value.
Ticket 408
2021-08-11 16:43:50 -04:00
Dan 318cd34f81 bouncing to fix a bug in SpiffWorkflow that was holding outo outdated data deed within the python_script_engine.
This caused validation to fail for valid repeat sections, so including a fix to allow that validation to continue.
2021-08-11 14:21:23 -04:00
mike cullerton 05e7f5e253 Worksheet for Alex 2021-08-11 10:06:12 -04:00
mike cullerton 19e0caa700 API endpoint for creating a DMN table from a spreadsheet 2021-08-11 08:55:10 -04:00
Dan 848c2e622f Always use a schema to define what is being returned, it enforces consistency of the API and internally we can depend on well defined objects. 2021-08-10 16:16:08 -04:00
Dan 913c12c02d Merge branch 'dev' into bug/pi_name_missing_246 2021-08-10 15:06:52 -04:00
mike cullerton dce95189cb Merge branch 'dev' into dmn-from-spreadsheet-395 2021-08-10 09:39:42 -04:00
Kelly McDonald 960d302c03 Finish #367 - Make sure that we only use a primary_id if it is in the workflow that we are currently working on - otherwise it is a library and might have its own primary id. 2021-08-10 09:15:38 -04:00
mike cullerton c153c5e344 Another test spreadsheet 2021-08-09 13:27:58 -04:00
mike cullerton deebc49a82 Removed test files - don't need under version control 2021-08-03 12:20:25 -04:00
mike cullerton 0e20480f52 Removed test files - don't need under version control 2021-08-03 12:17:54 -04:00
mike cullerton 64576ee8fd It now adds the dmn file to the DB.
Added annotations.

Still need to move it out of a script.
Maybe a new API endpoint and a new method in file_service
2021-08-03 11:55:50 -04:00
Kelly McDonald 6e1fedb704 Add tests for libraries, remove an API endpoint that didn't make sense after recent changes and remove some commented out sections that were not needed after some decisions about the api endpoints were made 2021-08-03 10:02:22 -04:00