Commit Graph

1751 Commits

Author SHA1 Message Date
alicia pritchett b0c723f021 Merge remote-tracking branch 'origin/feature/get-spec-script-613' into chore/variables-in-forms-550 2022-02-17 10:35:06 -05:00
alicia pritchett 8d49e2bfbb Merge branch 'dev' into chore/variables-in-forms-550 2022-02-17 10:32:40 -05:00
mike cullerton 7885481b84 Ignore hidden files 2022-02-16 15:55:42 -05:00
mike cullerton fca6eb1582 Script to get workflow spec information from a workflow spec id 2022-02-16 15:54:30 -05:00
Dan 0ad9843569 Dealing with some changes in SpiffWorkflow's latest release. I think this gets all tests passing again, and behaving as expected. 2022-02-16 10:46:46 -05:00
Dan cc6f80bea9 fixing some paths. 2022-02-15 16:25:52 -05:00
Dan c0af75a010 Merge branch 'dev' into feature/upgrade_spiff 2022-02-15 16:25:13 -05:00
Dan 093741fa90 By default the system will start up the scheduled tasks, which is usually what you want, but in development
it can be kind of irritating for this stuff to be spinning up when you are trying to debug something, so just set
PROCESS_WAITING_TASKS to false in instance/config.py and voila!!
2022-02-15 14:07:09 -05:00
Dan 7a91363db1 By default the system will start up the scheduled tasks, which is usually what you want, but in development
it can be kind of irritating for this stuff to be spinning up when you are trying to debug something, so just set
PROCESS_WAITING_TASKS to false in instance/config.py and voila!!
2022-02-15 11:06:11 -05:00
mike cullerton 63de48ccb6 Merge branch 'feature/spec_files_wthout_db' into git-integration-fixes-596 2022-02-11 12:57:57 -05:00
mike cullerton de797c4f32 Added examples for changes and untracked 2022-02-11 12:45:29 -05:00
mike cullerton 0e51def09f Cleanup - removed old unused methods 2022-02-11 12:18:20 -05:00
mike cullerton 88b662ea38 modified get_remote_url to use configuration, instead of hard-coding the remote host 2022-02-11 12:17:32 -05:00
Dan 648383f2ea for some reason jenkins doesn't like the spec_category_reorder test, so just doing it a different way. 2022-02-11 11:37:25 -05:00
Dan bf4f6909b9 Don't show the .git directory. 2022-02-11 11:06:26 -05:00
Dan 7a99859108 fixing some failing test 2022-02-11 11:00:42 -05:00
Dan 1b993ea227 updating readme 2022-02-11 09:47:49 -05:00
Dan 3993297360 minor fixes to the study api to get things working locally when we pull down a database from testing. 2022-02-10 18:19:57 -05:00
Dan 1858cd314f Assure we can call get_spec and it can return the master spec correctly. 2022-02-10 18:04:32 -05:00
Dan a047c63c32 Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-10 16:53:22 -05:00
Dan 1e4ec6f395 fixes to the workflow spec schema, so we can load null things without stuff blowing up. 2022-02-10 16:53:15 -05:00
alicia pritchett b7893688ae tweak 2022-02-10 16:21:59 -05:00
Dan edaf6e205e Not certain if this works, but this should help finding data maybe. 2022-02-10 16:10:35 -05:00
Dan eb7760fa55 Adding the migration - very destructive, but required for running the app.
minor fixes - there is still some weird problems with study id and user id being null that I need to track down, but the issue is sporadic, and hard to track down.
2022-02-10 11:50:31 -05:00
Dan 3ebffeed33 Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-10 11:34:56 -05:00
Dan 4280fa6610 Assure the spec has the corect category id on it. 2022-02-10 11:34:50 -05:00
alicia pritchett 3ee95b3c39 gets master spec 2022-02-10 11:29:38 -05:00
Dan f73b29cda2 Removing all the old unused outdated specs in the static directory. 2022-02-10 10:47:05 -05:00
Dan df3f67601c performance improvements. and last rements of load_example_data() 2022-02-09 23:29:39 -05:00
Dan f12c4aba52 Refactor the workflow_spec_service so it doesn't cache anything, it always reads what it needs from the file system. 2022-02-09 21:06:55 -05:00
Dan cf71f68ad9 all tests passing!!!!!!!!!!!!! 2022-02-09 15:30:07 -05:00
Dan 2cc20d4b9d really need to avoid any kind of caching in the WorkflowSpecService 2022-02-09 14:43:32 -05:00
mike cullerton 77b1692e2f typo 2022-02-09 13:52:48 -05:00
alicia pritchett c830c468b6 Merge remote-tracking branch 'origin/feature/spec_files_wthout_db' into feature/spec_files_wthout_db 2022-02-09 13:41:53 -05:00
alicia pritchett b788bbf4f4 revert this back 2022-02-09 13:41:50 -05:00
Dan 89e02c6207 really need to avoid any kind of caching in the WorkflowSpecService 2022-02-09 13:37:57 -05:00
alicia pritchett b5099cf5a2 Merge remote-tracking branch 'origin/feature/spec_files_wthout_db' into feature/spec_files_wthout_db 2022-02-09 13:09:49 -05:00
alicia pritchett 595b866f41 return empty list if you dont find standalones or libraries 2022-02-09 13:09:46 -05:00
Dan 7e7821a5e7 more test passing 2022-02-09 13:01:26 -05:00
Dan db93f9be4e fixing the test_spec_files 2022-02-09 12:44:54 -05:00
Dan e9fd76ed99 lookup service tests passing, but I need to revist this. 2022-02-09 12:11:51 -05:00
Dan de6d7eee84 nearly all study tests work. 2022-02-09 12:03:45 -05:00
Dan f72b25d9be Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-09 11:37:58 -05:00
Dan 8e7c8c87c3 down to just 38 failed tests 2022-02-09 11:37:54 -05:00
alicia pritchett 5bd25b9f60 get all specs change 2022-02-09 11:34:17 -05:00
alicia pritchett 148796d311 fixes add_category 2022-02-09 11:24:41 -05:00
alicia pritchett 38d64b1ffd fixes a workflow model related test
yes i call a service from a service whoops
2022-02-09 11:07:09 -05:00
mike cullerton 1c033ce7f2 Merge branch 'feature/spec_files_wthout_db' of https://github.com/sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-09 10:55:19 -05:00
mike cullerton 9576089e84 Reorder categories 2022-02-09 10:55:02 -05:00
alicia pritchett 8c5f3a8cfb slight model change thing for categories 2022-02-09 10:27:46 -05:00
Dan cc915ac25a refactoring the study service. 2022-02-09 08:50:00 -05:00
mike cullerton 4058078c12 index, not index_of 2022-02-08 17:45:14 -05:00
mike cullerton 2e4bab9d04 Add TODO 2022-02-08 15:04:47 -05:00
mike cullerton 65d6728be7 pass specs to _add_all_workflow_specs_to_study 2022-02-08 15:03:00 -05:00
mike cullerton 420f50f44a Defaults for WorkflowSpecInfo 2022-02-08 14:40:33 -05:00
Dan f1a75ca6bf getting default display_order working correctly 2022-02-08 14:10:56 -05:00
Dan 25ecc1593a test_workflow_spec_api passing. 2022-02-08 13:35:27 -05:00
mike cullerton df85e5cd7f Merge branch 'feature/spec_files_wthout_db' of https://github.com/sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-08 12:57:10 -05:00
mike cullerton 916418dcf3 For Dan 2022-02-08 12:53:20 -05:00
Dan 082ba18c2d got one more test to work. 2022-02-08 11:34:07 -05:00
Dan 6c0e38c0a5 Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-08 11:31:56 -05:00
Dan dc040f190e got one more test to work. 2022-02-08 11:30:13 -05:00
alicia pritchett 38c30adbfc i think this fixes adding and removing libraries to a spec 2022-02-08 11:27:20 -05:00
mike cullerton 3f856355e2 Merge branch 'git-integration-596' into feature/spec_files_wthout_db
# Conflicts:
#	config/testing.py
#	crc/services/lookup_service.py
2022-02-08 10:38:03 -05:00
alicia pritchett 7b031c26dc small change to libraries in workflow.py 2022-02-08 10:06:20 -05:00
mike cullerton 265381f48c Delete TODO comment. Done 2022-02-07 16:44:59 -05:00
mike cullerton 3444a34527 Merge branch 'feature/spec_files_wthout_db' of https://github.com/sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-07 16:34:03 -05:00
mike cullerton 2bd833446e get_spec not static 2022-02-07 16:33:42 -05:00
alicia pritchett df67572549 puts specs into categories 2022-02-07 16:27:45 -05:00
mike cullerton 2bd0647cd4 No longer pass category_name 2022-02-07 16:13:38 -05:00
mike cullerton 80322b392f Get category_name from fields 2022-02-07 16:09:51 -05:00
alicia pritchett aa9807a543 heres the categories 2022-02-07 15:45:09 -05:00
Dan f40fb26a8f a few more tests passing 2022-02-07 15:17:32 -05:00
Dan 1d7ecad51f Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-07 14:58:42 -05:00
Dan 4b8d193fb9 a few more tests passing 2022-02-07 14:58:25 -05:00
alicia pritchett 6c2690d27f rm duplicate method 2022-02-07 14:12:29 -05:00
alicia pritchett d4b04f5b80 fixes the import thing for real 2022-02-07 14:10:10 -05:00
mike cullerton e24a81b924 Merge branch 'feature/spec_files_wthout_db' of https://github.com/sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-07 13:35:51 -05:00
Dan d64853fa5b two passing tests around the WorkflowSpecService 2022-02-07 13:34:13 -05:00
mike cullerton b6d6e21360 Merge branch 'feature/spec_files_wthout_db' of https://github.com/sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-07 13:01:36 -05:00
Dan 056f901f9a WIP - its just caos right now. 2022-02-07 12:59:48 -05:00
Dan ad095aaaf5 don't try to load the workflowspecservice too early. 2022-02-07 12:38:00 -05:00
Dan abc17a8aff load the workflow_spec_service from app to avoid frequent file scans. 2022-02-07 12:30:20 -05:00
Dan e4f7e14a4d Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-07 12:18:36 -05:00
Dan 1c384a78e9 the tests kind of try and run now ... 2022-02-07 12:18:32 -05:00
alicia pritchett c78140e60c fixed spec_file.py 2022-02-07 12:17:49 -05:00
alicia pritchett 2a6578b785 fixed workflow.py 2022-02-07 12:10:57 -05:00
mike cullerton 2a6e8315a1 *** WIP ***
Committing to deal w/ the migration
2022-02-07 11:22:45 -05:00
Dan 8ec39bca15 Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-07 11:00:28 -05:00
Dan ed07907ee2 at least allow the tests to spin up, even if they all fail. 2022-02-07 11:00:19 -05:00
alicia pritchett 66906c3786 fix a couple things 2022-02-07 10:57:34 -05:00
Dan 1abb2490e5 Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-07 10:33:28 -05:00
Dan 4ba122fff3 WIP - completely broken. 2022-02-07 09:12:11 -05:00
alicia pritchett 699d0ab797 updated method calls in workflow.py
except for reordering
2022-02-04 15:31:36 -05:00
Dan 2383c7d76d WIP - lots of random changes, trying to swtich over 2022-02-04 14:50:31 -05:00
alicia pritchett 2fb6746f6b new methods 2022-02-04 14:45:52 -05:00
mike cullerton 36911d35ac Added description of the service 2022-02-04 14:03:37 -05:00
Dan 6765afbd02 WIP - Move the categories and specifications out of the database. 2022-02-04 13:51:39 -05:00
mike cullerton 2706632cda Fix origin/origin problem 2022-02-03 07:14:16 -05:00
mike cullerton 6fa729900c Bug? origin/origin/brach, rather than origin/branch 2022-02-03 07:10:38 -05:00
mike cullerton 64ffbde3d9 Remove unused code 2022-02-03 06:47:41 -05:00
mike cullerton 2d48151976 Fix when checkout branch does not exist yet
First attempt at merge
2022-02-03 06:47:02 -05:00
mike cullerton e9c7f3d9b6 Force checkout of branch when loading repo 2022-02-02 14:48:47 -05:00
mike cullerton 21b5d034e2 authentication
variable name changes for readability
removed unused code
2022-02-02 14:11:41 -05:00
Dan e103845218 fixing failing tests around default primary processes
Removing a ton of ununsed code from example_data.
2022-02-02 13:30:54 -05:00
mike cullerton a45c5959ef First attempt at adding credentials 2022-02-02 13:24:54 -05:00
Dan 4ec6e403f5 1. Created a UserFileService, so it's clear what we use when for File Services, UserFiles, SpecFiles, and ReferenceFiles each function differently.
2. Reference Files and Spec Files are written to disk, they do not exist in the database at all.
2022-02-02 12:59:56 -05:00
mike cullerton 6f233a9b79 removed unused code 2022-02-02 12:51:14 -05:00
mike cullerton 35e2a53b09 Actually use the model 2022-02-02 12:35:08 -05:00
mike cullerton 55697c6c88 Return the schema 2022-02-02 12:04:28 -05:00
mike cullerton ec054bbdf1 set tracking on the active branch 2022-02-02 11:48:05 -05:00
mike cullerton 558d6f47e2 removed unused code from get_repo
started pull_from_remote
2022-02-02 11:30:05 -05:00
mike cullerton 2a0d71aab6 Status was removed from the model 2022-02-02 10:04:52 -05:00
mike cullerton 1cb6465f22 Return the schema 2022-02-02 10:04:02 -05:00
mike cullerton 8a8c3de8c4 clone repo if it doesn't exist 2022-02-01 16:25:46 -05:00
alicia pritchett c1a5032014 WIP modifying workflow service to remove value expression 2022-02-01 15:44:55 -05:00
mike cullerton 070c3cfd5d *** WIP ***
New API endpoints, and associated methods.
Services for the new endpoints
Minimal test
2022-02-01 10:49:35 -05:00
mike cullerton c07c429ae1 Fix for new version of Pandas. They check data type now. 2022-02-01 10:42:18 -05:00
mike cullerton c9b7d95a83 *** WIP ***
Beginning of Git integration
- added gitpython to Pipfile
- started additions for api, model, service, and a test
2022-01-28 16:11:36 -05:00
Dan 4c00a5762f partial commit - new spec_file_service, and new spec_file_api endpoints that use spec and file name, not file id.
removed worklow_sync
cleaned up file and workflow models
most of the test are broken.
2022-01-28 06:42:37 -05:00
Dan Funk 97c29bf77c
Merge pull request #449 from sartography/files-to-filesystem-572
Files to filesystem 572
2022-01-26 10:11:26 -05:00
mike cullerton d82367ab2a Move temp_migration_service code into migration 7225d990740e
Delete test_files_to_filesystem and test_files_from_filesystem--they were only used to give me access to the migration methods
2022-01-26 09:58:46 -05:00
Dan f3637ee6a0 resolving todos, as they are done, and removing commented out code that is no longer in use. 2022-01-25 16:37:17 -05:00
Dan 8529465322 Removed the method get_spec_data_files completly - using get_spec_files and get_spec_data to get this information instead.
Only load the spec data files if you are creating a new workflow, otherwise just deserialize the json.
Removed the stuff about calculaing the version of the spec, as we don't use it.
2022-01-25 16:10:54 -05:00
mike cullerton ae86ab4790 Make erroring workflows visible
The `process_erroring_workflows` method is called by our scheduler in __init__.py
2022-01-25 12:18:29 -05:00
Dan Funk 58b3378fbf
Merge pull request #448 from sartography/get-spec-from-workflow-id-589
Get spec from workflow id #589
2022-01-24 16:40:53 -05:00
Dan 9690ebf883 minor minor stuff. 2022-01-24 14:43:36 -05:00
mike cullerton 1fe65ff6b7 Script to get a workflow spec from a workflow id. 2022-01-24 12:40:50 -05:00
mike cullerton 662014d378 Merge branch 'fix-login-590' into missing-bpmn-534 2022-01-21 15:31:13 -05:00
mike cullerton 7da2fa2171 Fix for failing tests 2022-01-21 15:13:05 -05:00
mike cullerton 65345a8e91 We can now set our own user_id in instance.config so that user dhf8r isn't always logged in. 2022-01-21 14:42:36 -05:00
mike cullerton 91fa44b2ab ** WIP **
Committing so I can work on another ticket.
2022-01-21 14:37:11 -05:00
mike cullerton 58658d9507 Create separate tags for reference files and spec files 2022-01-20 15:58:31 -05:00
mike cullerton bc65f1c64a Limit modifying reference files to admins 2022-01-20 15:43:32 -05:00
mike cullerton 76b09a418e Created separate api files for spec_files and reference_files 2022-01-20 15:22:47 -05:00
mike cullerton ca54b850cd UNDO my previous commit regarding `get_files needs to have workflow_spec_id`
We need to call `get_spec_files` instead now
2022-01-20 14:57:32 -05:00
Dan f815add699 1. Add a default directory for the location of SYNC files.
2. Added a last_updated column to the lookup table
3. The Lookup service now uses the above, and compares it to the actual file date, we can then rebuild the lookup if needed.
4. That 755 migration loads up the models, so when you change the models, the migration starts to fail.  Not really sure what to do here, but modify the migration while we are in process.
2022-01-20 13:05:58 -05:00
mike cullerton 9ea1399578 get_files needs to have workflow_spec_id 2022-01-20 09:54:22 -05:00
mike cullerton 68820c67cb Removed (almost) all references to WorkflowSpecDependencyFile
(There is still a call in the lookup service, but we need to decide how to fix that)
2022-01-19 16:12:54 -05:00
mike cullerton 16a9a18c5b Fixed get reference file info, and added test 2022-01-19 15:17:35 -05:00
mike cullerton 091d71eb0f Cleaned up code around differences between file info and file data
Cleaned up some api code around differences between file, spec_file, and reference_file
Cleaned up some api code around differences between file info and file data
Fixed some tests for file api
2022-01-19 13:47:14 -05:00
mike cullerton d83f28e1a7 Pushed so Dan can play 2022-01-18 15:55:49 -05:00
mike cullerton 68ac056457 Merge branch 'dev' into files-to-filesystem-572 2022-01-18 08:18:14 -05:00
mike cullerton 6da6e05171 *** WIP ***
code for updating spec files
some cleanup

tests pass, but we don't have all the tests we need.
2022-01-13 15:24:29 -05:00
mike cullerton b99ed73951 Remove unused imports 2022-01-12 15:00:26 -05:00
mike cullerton cfa9f00bf3 *** WIP ***
Moved reference files to their own service
2022-01-12 14:37:33 -05:00
mike cullerton b6b9482a64 *** WIP ***
Incremental commit. Mainly cleaning up failing tests.
2022-01-12 13:16:09 -05:00
Dan e67a1554ea Always but always but ALWAYS, use server_default=func.now() when setting up a datetime field we autopopulate. 2022-01-12 12:33:59 -05:00
Dan c2a11a7d95 fixing an error where I accidentally deleted a bit of the API. 2022-01-12 09:58:34 -05:00
Dan d346589e07 1. added the ability to filter on level and user.
2. added a total to the response, so you can see what the total count of users is.
3. adding user_uid to the task_log table.
2022-01-12 08:58:50 -05:00
mike cullerton 9cc91f92c3 *** WIP ***
cleanup - removing commented code
2022-01-11 15:55:08 -05:00
mike cullerton 4df2ed6ce4 *** WIP ***
Failing tests, and missing functionality.
Committing to get stuff on Github.
2022-01-11 15:30:22 -05:00
Dan 7176452fe3 forgot the migration, also fixing an error in the TaskLogQuerySchema. 2022-01-10 13:26:16 -05:00
Dan 962d05c875 1. Created a TaskLoggingService to encapsulate what we are doing with Task Logging through BPMN Script calls and API calls from the front end.
2. Added two api endpoints that will allow us to get all the logs for a specific workflow or study.
3. Assured that requests for logs are paginated, sortable, and can be limited to a specific code if needed.
4. Assure that we only use logging levels that match the log levels of Python.
2022-01-10 13:16:54 -05:00
mike cullerton f537fd7fbf *** WIP *** 2022-01-07 15:58:23 -05:00
mike cullerton dc27f795c8 *** WIP ***
Committing because it is Friday afternoon, and my computer is acting flaky
2022-01-07 15:34:51 -05:00
mike cullerton 86a6039dc8 *** WIP ***
**Many** tests are failing!

Committing so I can merge dev into this branch
2022-01-06 11:46:54 -05:00
mike cullerton 110e2c56f8 cli entry point for temp_migration_service.
Ultimately, this will call the real service.
2022-01-04 07:28:23 -05:00
mike cullerton 7f9519736c fixed some issues with how we look up records. Try to depend on the data_obj 2022-01-04 07:26:12 -05:00
mike cullerton 2cd0f84a47 Merge branch 'dev' into files-to-filesystem-572 2022-01-03 08:06:17 -05:00
Dan 1446302a04 A quick fix for an error we are seeing in testing. 2021-12-30 15:27:31 -05:00
Dan 3480e1c8ab use a signal, in addition to firing off a cancel notify when a workflow is restarted. 2021-12-23 15:30:09 -05:00
Dan bd3176eea6 Fixes #578 - issue with us displaying an incorrect end event in some cases. 2021-12-22 16:27:52 -05:00
Dan c83b047881 Fixing some bugs I introduced with the latest fix. 2021-12-21 16:22:41 -05:00
Dan f129509802 Looks like reording the deletes allows us to remove workflow specs a little easier. Tested this with libraries, and seems to work ok. 2021-12-20 14:36:45 -05:00
mike cullerton 3fa5505970 Cleanup - remove print statements 2021-12-17 12:30:10 -05:00
mike cullerton 6073412d82 Clean up imports 2021-12-17 12:03:45 -05:00
mike cullerton aa27df7840 Process the spec files and their metadata 2021-12-17 11:46:21 -05:00
alicia pritchett a81f25f0ee Just kidding. We should make deepcopy and put in fake data 4 valdiation
see above. previously i deleted this, because i didnt want unitializsed data in task data. well heres a hint. don't put it in task data. make a deep copy and hold on to that for validation pursposes
2021-12-16 14:47:04 -05:00
mike cullerton 5c90ce01a6 *** WIP ***
Temp services `ToFilesystemService` and `FromFilesystemService` for migrating files To and From the filesystem - Not sure where these classes will end up
Tests to call the two services.
upgrade method in migration that writes workflow spec files and metadata to filesystem
2021-12-16 14:22:37 -05:00
mike cullerton 338352017b *** WIP ***
Moving files to the filesystem
Tools for reading/writing metadata for categories and specs
2021-12-16 08:41:49 -05:00
Dan 421e4fd9aa Fixes related to resent commit to SpiffWorkflow that provides better handling for workflows that loop back on themselves. 2021-12-15 16:00:31 -05:00
Dan Funk e62a99af96
Merge pull request #438 from sartography/bug/fix-task-data
Bug/fix task data
2021-12-13 13:31:37 -05:00
Dan Funk b5f2fe05dc
Merge pull request #439 from sartography/workflow-spec-delete-217
Problem deleting workflow spec #217
2021-12-13 13:19:56 -05:00
Dan c46800171d Merge remote-tracking branch 'origin/dev' into bug/fix-task-data 2021-12-13 13:18:00 -05:00
Dan ad9a480f93 Merge remote-tracking branch 'origin/dev' into waiting-workflow-errors-566 2021-12-13 13:05:02 -05:00
Dan Funk 0bb06f380a
Merge pull request #436 from sartography/more-study-statuses-562
More study statuses #562
2021-12-13 13:02:57 -05:00
Dan Funk 407cc8dd69
Merge pull request #435 from sartography/bug/scheduler_task_recursion_depth
#557 - scheduler recursion because each call to 'process_witing_tasks…
2021-12-13 13:01:31 -05:00
alicia pritchett cdaad5dcc0 Allow 'none' for progress_status
Well this fixes tests but also, I dont think you necessarily have this right off the bat if you havent done anything in a study, which i handle on the frontend. so null is fine
2021-12-13 12:34:32 -05:00
alicia pritchett 8b27423172 Updating study model schema to include Progress Status 2021-12-13 12:29:15 -05:00
mike cullerton daebf42a30 Move some of the work in `crc.api.workflow.delete_workflow_specification` to separate workflow_service methods
Added a session.commit to `delete_workflow_spec_task_events`
2021-12-13 11:04:28 -05:00
mike cullerton acbb8898e4 Merge branch 'more-study-statuses-562' into waiting-workflow-errors-566 2021-12-10 16:52:07 -05:00
mike cullerton f748fa3db8 Removed unused imports 2021-12-10 16:48:28 -05:00
mike cullerton d30405226e Script to get the new progress_status value 2021-12-10 16:16:33 -05:00
mike cullerton 100e822f2e Fixes for set_study_progress_status to use new ProgressStatus enum 2021-12-10 16:15:49 -05:00
mike cullerton 9cd5022bf0 Set progress_status to in_progress when creating a new study.
-- I'm not sure if this is what we want.
2021-12-10 16:00:33 -05:00
mike cullerton 6d09aa1817 We moved the new statuses to a new column and created a separate enum class for them.
They really are sub_statuses of the in_progress status
2021-12-10 15:57:02 -05:00
mike cullerton 61975093f9 Renamed some files to include the word `progress` 2021-12-10 15:47:34 -05:00
alicia pritchett f9fbb32641 Take this garbage out
I can say that because I originally wrote that garbage
2021-12-10 12:10:08 -05:00
alicia pritchett 962cf7c32c Fixes problem with initializing some task data fields
we had a problem where sometimes, task data gets some values set in its fields to None, which conflicts with default values. this fixes that
2021-12-10 11:33:26 -05:00
mike cullerton 52baf25459 In `WorkflowService.do_waiting`, if a workflow causes an exception, change its status from `waiting` to `erroring` so we don't continue to call it in `do_waiting`. 2021-12-09 15:33:39 -05:00
Dan 06960644fa #557 - scheduler recursion because each call to 'process_witing_tasks' would kick off an additional scheduler by settting up a new app_context. 2021-12-09 08:54:44 -05:00
Dan Funk 23fa28bc23
Merge pull request #434 from sartography/bug/ldap_service_rollback_expoded_the_world
The LDAP service didn't do a good enough check to see if the record w…
2021-12-09 06:42:02 -05:00
Dan 412290f2ab The LDAP service didn't do a good enough check to see if the record was already in the database. This caused an error, which left the workflow in a waiting state. And that waiting state meant the error would just happen a million times over again until we pushed up this code change. 2021-12-09 06:33:30 -05:00
mike cullerton 6db9cdabc7 New study statuses 2021-12-08 10:50:30 -05:00
Dan Funk 0cfa59bfea
Merge pull request #429 from sartography/chore/clear-form-data-553
Fixes #553 - Don't load Form data
2021-12-08 10:37:44 -05:00
Dan Funk b28ddef842
Merge pull request #430 from sartography/study-status-scripts-556
Study status scripts #556
2021-12-08 10:36:53 -05:00
Dan ffafc1674a assure we don't overwrite task data with a previous form submission if that data already exists. 2021-12-07 17:04:20 -05:00
mike cullerton 8813098190 Updated to use new ProtocolBuilderCreatorStudy model and schema 2021-12-06 14:37:42 -05:00
mike cullerton db1f3ba6ab Model and schema for new StudyCreator api endpoint
Update StudyModel to use new model
2021-12-06 14:36:34 -05:00
mike cullerton 791328d520 Don't need to ask the question twice 2021-12-03 12:18:17 -05:00
mike cullerton 6be9529b22 Fixed the class names 2021-12-03 11:53:21 -05:00
mike cullerton 34b4913542 Restructured `do_task`
Fixed up `do_task_validate_only`
2021-12-03 11:45:02 -05:00
mike cullerton 52bfa848ac *** WIP ***
Script to set study status
2021-12-02 17:11:28 -05:00
mike cullerton 8c66a3d2ef Script to get the status of the current script 2021-12-02 15:30:47 -05:00
Dan 5cf011f1d9 Quick fix for ldap. 2021-12-02 12:19:36 -05:00
alicia pritchett 7524e97c9a Fixes #553 - Don't load Form data
adds an extension so that you can opt to clear out form data on a given task
2021-12-01 16:11:52 -05:00
Dan ad93b5fae6 We were not correctly handing the possibility of there only being a waiting task (and no ready tasks).
We also had a problem with the validator throwing a 500 when it was trying tell us it would never finish validation.
2021-11-30 12:44:35 -05:00
Mike Cullerton ec9d8f8a5c
Merge pull request #425 from sartography/bug/slow_ldap
Remove wild-card from uid search, it performs very badly.  Add a bit …
2021-11-30 11:09:42 -05:00
Mike Cullerton 8cc581611e
Merge pull request #426 from sartography/feature/performance_improvement
Performance upgrades - study dashboard in less than 2 seconds now
2021-11-30 11:09:28 -05:00
Dan Funk 8c30df011b
Merge pull request #427 from sartography/workflow-status-script-548
Workflow status script #548
2021-11-30 10:51:57 -05:00
Dan Funk c8c8c8b244
Merge pull request #424 from sartography/xls-xlsx-issue-540
Xls xlsx issue #540
2021-11-30 10:50:33 -05:00
mike cullerton 0f1ccd7db5 New script to check workflow status.
Includes test and test workflow
2021-11-30 08:54:15 -05:00
Dan 455c1035f1 Performance upgrade to script.py - reduce the number of times we generate the augmented list of script functions for running python evals.
Performance upgrade to study_service.py - Don't query on the whole WorkflowModel, just get the status that we need.  Saves lots of complex joins.
Performance upgrade to SpiffWorkflow - don't convert to box unless we have to do so.
2021-11-29 17:48:05 -05:00
Dan e5b1daa3d9 Remove wild-card from uid search, it performs very badly. Add a bit of logging so we can easily see performance in the future. 2021-11-29 11:03:14 -05:00
Dan 8ec58fd6d7 Assure better error messages when parsing word documents with jinja syntax.
I also have a pull request into the folks that run the python docx library to improve their error messages that we depend on here.
2021-11-23 14:38:48 -05:00
Dan 4a1285b8e0 When validating a workflow, don't run the master workflow multiple times, its a huge waste of time.
Provide decent data when calling get_study_associate(s) scripts.
2021-11-22 17:17:19 -05:00
Dan 5bb7dbfb19 Improve the error messages returned from jinja template processing. Try to find the actual line with an error in it. 2021-11-22 14:46:33 -05:00
Dan 12b78c1f99 Value Errors should also alert us that we can't serialize the json.
Enum Label was depending on the options attribute not existing in some situtations, which is a bad assumption.  Rather, check for specific properties, and call back to using options as the default.
2021-11-22 12:37:10 -05:00
Dan 4d1f4429ff Fixing a bug in enum_label that was not working when lookup up an select list from a spreadsheet.
Assure we raise more thoughtful error messages when running getting exceptions in engine tasks.
Field Options should always be available now due to a fix in Spiffworkflow.
2021-11-22 12:26:30 -05:00
Dan 7b02f46403 During validation, options were not being handled correctly for certain types of lookups. 2021-11-17 13:38:41 -05:00
mike cullerton cb77db26a3 Minor edit, for clarity 2021-11-16 12:05:20 -05:00
mike cullerton 9f18484ebb Grab exception when reading older xls spreadsheet into pandas
Renamed `xls` variable to `xlsx`, so it makes more sense
Added a hint to error_service for validation
2021-11-16 11:54:31 -05:00
Dan Funk 8c166bd029
Merge pull request #423 from sartography/test-protocol-builder-519
Test protocol builder #519
2021-11-11 11:49:57 -05:00
Dan f4f3842059 Fixing a stupid mistake. 2021-11-11 11:45:35 -05:00
Dan 197b75c6bf Quickfix - validation was failing if we had a date object in the task data, but this isn't really a problem. We now use the same JSON serializer as flask does, when doing validations, to better verify behavior with the front end. 2021-11-11 11:18:58 -05:00
mike cullerton c7147274f0 Resolving issues from code review 2021-11-10 17:43:34 -05:00
mike cullerton a8b7cd8bba Merge branch 'dev' into test-protocol-builder-519 2021-11-10 16:32:34 -05:00
Dan Funk 037692f42b
Merge pull request #422 from sartography/chore/group-repeating-validation-536
Fixes #536 - Group + Repeating property validation
2021-11-10 15:28:04 -05:00
Dan Funk e764bf937d
Merge pull request #421 from sartography/bug/validation_of_enum_label_failing_in_sub_workflows
Validation was failing for enum_label() expressions when called withi…
2021-11-10 15:26:03 -05:00
Dan Funk 81f8aed68b
Merge pull request #420 from sartography/drop-tasklog-on-delete
Delete Task log model when deleting a study
2021-11-10 15:20:11 -05:00
Dan Funk 84f364ce37
Merge branch 'dev' into chore/view-as-382 2021-11-10 15:16:20 -05:00
alicia pritchett accdd281fa Fixes #536 - Group + Repeating property validation
For now, we should not allow both group and repeat properties on a field together.
2021-11-09 13:21:44 -05:00
mike cullerton 5cf742d955 remove unused import 2021-11-09 13:07:01 -05:00
Dan d1eae3c15a Validation was failing for enum_label() expressions when called within a sub-process. Possible (but unlikely) that this would occur outside validation. 2021-11-09 12:55:06 -05:00
alicia pritchett aa1ba59bb1 Delete Task log model when deleting a study
just a minor fix
2021-11-09 10:42:47 -05:00
mike cullerton b193f56f35 Minor cleanup for readability. 2021-11-08 10:13:07 -05:00
mike cullerton 6f09b0d7dd Added comment to _is_valid_study.
Fixed get_required_docs. The data returned for get_required_dcs from UVA PB was *very* different than what we had.
2021-11-05 14:37:41 -04:00
mike cullerton d186b24de3 UVA PB returns the info in a list. We want the dictionary in the list 2021-11-05 14:26:21 -04:00
mike cullerton 271d4da5ff Merge branch 'dev' into test-protocol-builder-519 2021-11-05 10:29:20 -04:00
mike cullerton 6d548c2a74 *** WIP ***
Committing to work on another ticket
There are TODOs in study_service
2021-11-05 09:59:14 -04:00
Dan 9376d3deaf Don't error out trying to send an error about invalid review types. 2021-11-04 13:32:57 -04:00
Dan Funk 3a36a48b88
Merge pull request #418 from sartography/convert-timestamp-499
Added ordered arguments to the get_localtime script #499
2021-11-03 14:28:57 -04:00
mike cullerton b03e596dcf One thing at a time; remove HSRNUMBER for now 2021-11-03 12:58:03 -04:00
mike cullerton 9c8ae36ba3 UVA IRB passes back `Q_COMPLETE` and `HSRNUMBER` 2021-11-03 12:40:43 -04:00
mike cullerton 7f163285c3 Test w/Alex's ID 2021-11-03 11:56:07 -04:00
mike cullerton 8be9a1968b configure logging 2021-11-03 11:21:39 -04:00
mike cullerton 50d4159168 info log didn't show up. Maybe debug? 2021-11-03 10:36:27 -04:00
mike cullerton 8b6c8f9518 Logging to troubleshoot API call 2021-11-03 10:14:11 -04:00
mike cullerton 1f9c80d70d Added ordered arguments to the get_localtime script
Modified the workflow so the first task is a form with booleans that allows me to test all the permutations.
Modified tests to cover all the permutations for calling the script
2021-11-03 08:36:24 -04:00