Commit Graph

891 Commits

Author SHA1 Message Date
mike cullerton a830657ebd raise ApiError when isExecutable tag is missing.
Unsure how this would actually happen, since it should get set to False if box is unchecked.
2022-02-21 10:17:01 -05:00
Dan 24f33ec9c9 Merge branch 'chore/variables-in-forms-550' of github.com:sartography/cr-connect-workflow into chore/variables-in-forms-550 2022-02-21 09:57:19 -05:00
alicia pritchett bb3d3a5907 Merge remote-tracking branch 'origin/chore/variables-in-forms-550' into chore/variables-in-forms-550 2022-02-21 08:56:11 -05:00
Dan 06b9b653b2 don't overwrite the field.label during an eval. It should remain as it is, just assure that it /can/ be evaluated. 2022-02-19 17:43:12 -05:00
Dan 4f32cf753c Merge branch 'chore/variables-in-forms-550' of github.com:sartography/cr-connect-workflow into chore/variables-in-forms-550 2022-02-18 17:20:10 -05:00
Dan c3e52f960b Minor stuff to help digging into validation of current workflows. 2022-02-18 17:20:04 -05:00
Dan a7f591d2ac Merge branch 'dev' into chore/variables-in-forms-550 2022-02-18 17:19:01 -05:00
alicia pritchett 3b48435411 Merge branch 'master' into chore/variables-in-forms-550 2022-02-18 13:23:33 -05:00
alicia pritchett 4b92b0ee71 oops 2022-02-18 11:57:13 -05:00
alicia pritchett e7c0e66f86 Merge remote-tracking branch 'origin/chore/variables-in-forms-550' into chore/variables-in-forms-550 2022-02-18 11:41:15 -05:00
alicia pritchett a912e8ef49 validate checks/catches default value and label value 2022-02-18 11:41:07 -05:00
Dan b90be07bca Remove these bloodly log files from the repo. 2022-02-18 11:40:10 -05:00
Dan e91be2a54d Merge branch 'dev' into chore/variables-in-forms-550 2022-02-18 11:27:58 -05:00
Dan 635a112796 still trying to fix a rogue test. 2022-02-18 10:41:24 -05:00
Dan 6adf1107fe Trying to figure out why these files are not the same on testing. 2022-02-18 10:12:36 -05:00
alicia pritchett 06dc20657b Fixing various BPMN's for tests 2022-02-17 17:29:44 -05:00
alicia pritchett a5acd9b07e Merge branch 'dev' into chore/variables-in-forms-550 2022-02-17 15:04:41 -05:00
Dan c2c6410bbb Assure we update the workflow model, so we don't keep reporting the error over and over again. 2022-02-17 14:12:31 -05:00
Dan 3704279f20 Quick fix, assure we can reset studies. 2022-02-17 14:07:33 -05:00
Dan Funk be48f86440
Merge pull request #459 from sartography/bug/619_lookup_tables
Another re-work to fix 619 - and to assure that we aren't rebuilding …
2022-02-17 13:15:37 -05:00
Dan c2dbec1e3f Merge remote-tracking branch 'origin/dev' into bug/619_lookup_tables 2022-02-17 12:51:24 -05:00
Dan e0a8e7300c Merge remote-tracking branch 'origin/dev' into get-erroring-workflows-574 2022-02-17 12:37:34 -05:00
Dan Funk 9d74ee5790
Merge pull request #458 from sartography/bug/date-field-validation-622
Bug/date field validation #622
2022-02-17 12:18:39 -05:00
Dan Funk 484b9a9c7c
Merge pull request #457 from sartography/feature/get-spec-script-613
Feature/get spec script #613
2022-02-17 12:17:12 -05:00
alicia pritchett 08c1571803 Remove value_expression 2022-02-17 12:16:15 -05:00
Dan Funk b91372171c
Merge pull request #454 from sartography/git-integration-fixes-596
Git integration fixes #596
2022-02-17 12:15:13 -05:00
Dan Funk 5def436602
Merge branch 'dev' into get-erroring-workflows-574 2022-02-17 12:12:42 -05:00
Dan b72ecb8375 Another re-work to fix 619 - and to assure that we aren't rebuilding the lookup tables too frequently. 2022-02-17 11:59:48 -05:00
mike cullerton 31f724b0c3 Seed date fields correctly during validation.
We were seeding them with random strings.
2022-02-17 11:48:08 -05:00
Dan f2b6008e5f Fixes 619 - look up models were being built incorrectly, and repeatedly, and sometimes bombed out all together.
bonus: resond with a valid error message when an invalid task_id is requested (don't just 500)
2022-02-17 11:04:50 -05:00
alicia pritchett b0c723f021 Merge remote-tracking branch 'origin/feature/get-spec-script-613' into chore/variables-in-forms-550 2022-02-17 10:35:06 -05:00
alicia pritchett 8d49e2bfbb Merge branch 'dev' into chore/variables-in-forms-550 2022-02-17 10:32:40 -05:00
mike cullerton 7885481b84 Ignore hidden files 2022-02-16 15:55:42 -05:00
Dan 0ad9843569 Dealing with some changes in SpiffWorkflow's latest release. I think this gets all tests passing again, and behaving as expected. 2022-02-16 10:46:46 -05:00
Dan cc6f80bea9 fixing some paths. 2022-02-15 16:25:52 -05:00
Dan 093741fa90 By default the system will start up the scheduled tasks, which is usually what you want, but in development
it can be kind of irritating for this stuff to be spinning up when you are trying to debug something, so just set
PROCESS_WAITING_TASKS to false in instance/config.py and voila!!
2022-02-15 14:07:09 -05:00
mike cullerton 63de48ccb6 Merge branch 'feature/spec_files_wthout_db' into git-integration-fixes-596 2022-02-11 12:57:57 -05:00
mike cullerton 0e51def09f Cleanup - removed old unused methods 2022-02-11 12:18:20 -05:00
mike cullerton 88b662ea38 modified get_remote_url to use configuration, instead of hard-coding the remote host 2022-02-11 12:17:32 -05:00
Dan 648383f2ea for some reason jenkins doesn't like the spec_category_reorder test, so just doing it a different way. 2022-02-11 11:37:25 -05:00
Dan bf4f6909b9 Don't show the .git directory. 2022-02-11 11:06:26 -05:00
Dan 7a99859108 fixing some failing test 2022-02-11 11:00:42 -05:00
Dan 1b993ea227 updating readme 2022-02-11 09:47:49 -05:00
Dan 3993297360 minor fixes to the study api to get things working locally when we pull down a database from testing. 2022-02-10 18:19:57 -05:00
Dan 1858cd314f Assure we can call get_spec and it can return the master spec correctly. 2022-02-10 18:04:32 -05:00
alicia pritchett b7893688ae tweak 2022-02-10 16:21:59 -05:00
Dan edaf6e205e Not certain if this works, but this should help finding data maybe. 2022-02-10 16:10:35 -05:00
Dan eb7760fa55 Adding the migration - very destructive, but required for running the app.
minor fixes - there is still some weird problems with study id and user id being null that I need to track down, but the issue is sporadic, and hard to track down.
2022-02-10 11:50:31 -05:00
Dan 3ebffeed33 Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-10 11:34:56 -05:00
Dan 4280fa6610 Assure the spec has the corect category id on it. 2022-02-10 11:34:50 -05:00
alicia pritchett 3ee95b3c39 gets master spec 2022-02-10 11:29:38 -05:00
Dan df3f67601c performance improvements. and last rements of load_example_data() 2022-02-09 23:29:39 -05:00
Dan f12c4aba52 Refactor the workflow_spec_service so it doesn't cache anything, it always reads what it needs from the file system. 2022-02-09 21:06:55 -05:00
Dan cf71f68ad9 all tests passing!!!!!!!!!!!!! 2022-02-09 15:30:07 -05:00
Dan 2cc20d4b9d really need to avoid any kind of caching in the WorkflowSpecService 2022-02-09 14:43:32 -05:00
mike cullerton 77b1692e2f typo 2022-02-09 13:52:48 -05:00
alicia pritchett c830c468b6 Merge remote-tracking branch 'origin/feature/spec_files_wthout_db' into feature/spec_files_wthout_db 2022-02-09 13:41:53 -05:00
alicia pritchett b788bbf4f4 revert this back 2022-02-09 13:41:50 -05:00
Dan 89e02c6207 really need to avoid any kind of caching in the WorkflowSpecService 2022-02-09 13:37:57 -05:00
alicia pritchett 595b866f41 return empty list if you dont find standalones or libraries 2022-02-09 13:09:46 -05:00
Dan e9fd76ed99 lookup service tests passing, but I need to revist this. 2022-02-09 12:11:51 -05:00
Dan de6d7eee84 nearly all study tests work. 2022-02-09 12:03:45 -05:00
Dan 8e7c8c87c3 down to just 38 failed tests 2022-02-09 11:37:54 -05:00
alicia pritchett 148796d311 fixes add_category 2022-02-09 11:24:41 -05:00
alicia pritchett 38d64b1ffd fixes a workflow model related test
yes i call a service from a service whoops
2022-02-09 11:07:09 -05:00
mike cullerton 9576089e84 Reorder categories 2022-02-09 10:55:02 -05:00
Dan cc915ac25a refactoring the study service. 2022-02-09 08:50:00 -05:00
mike cullerton 4058078c12 index, not index_of 2022-02-08 17:45:14 -05:00
mike cullerton 2e4bab9d04 Add TODO 2022-02-08 15:04:47 -05:00
mike cullerton 65d6728be7 pass specs to _add_all_workflow_specs_to_study 2022-02-08 15:03:00 -05:00
Dan f1a75ca6bf getting default display_order working correctly 2022-02-08 14:10:56 -05:00
Dan 25ecc1593a test_workflow_spec_api passing. 2022-02-08 13:35:27 -05:00
mike cullerton df85e5cd7f Merge branch 'feature/spec_files_wthout_db' of https://github.com/sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-08 12:57:10 -05:00
mike cullerton 916418dcf3 For Dan 2022-02-08 12:53:20 -05:00
Dan 082ba18c2d got one more test to work. 2022-02-08 11:34:07 -05:00
Dan 6c0e38c0a5 Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-08 11:31:56 -05:00
Dan dc040f190e got one more test to work. 2022-02-08 11:30:13 -05:00
mike cullerton 3f856355e2 Merge branch 'git-integration-596' into feature/spec_files_wthout_db
# Conflicts:
#	config/testing.py
#	crc/services/lookup_service.py
2022-02-08 10:38:03 -05:00
mike cullerton 2bd0647cd4 No longer pass category_name 2022-02-07 16:13:38 -05:00
Dan f40fb26a8f a few more tests passing 2022-02-07 15:17:32 -05:00
Dan 1d7ecad51f Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-07 14:58:42 -05:00
Dan 4b8d193fb9 a few more tests passing 2022-02-07 14:58:25 -05:00
alicia pritchett 6c2690d27f rm duplicate method 2022-02-07 14:12:29 -05:00
Dan d64853fa5b two passing tests around the WorkflowSpecService 2022-02-07 13:34:13 -05:00
Dan 056f901f9a WIP - its just caos right now. 2022-02-07 12:59:48 -05:00
Dan e4f7e14a4d Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-07 12:18:36 -05:00
Dan 1c384a78e9 the tests kind of try and run now ... 2022-02-07 12:18:32 -05:00
alicia pritchett 2a6578b785 fixed workflow.py 2022-02-07 12:10:57 -05:00
Dan 8ec39bca15 Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-07 11:00:28 -05:00
Dan ed07907ee2 at least allow the tests to spin up, even if they all fail. 2022-02-07 11:00:19 -05:00
alicia pritchett 66906c3786 fix a couple things 2022-02-07 10:57:34 -05:00
Dan 1abb2490e5 Merge branch 'feature/spec_files_wthout_db' of github.com:sartography/cr-connect-workflow into feature/spec_files_wthout_db 2022-02-07 10:33:28 -05:00
Dan 4ba122fff3 WIP - completely broken. 2022-02-07 09:12:11 -05:00
alicia pritchett 699d0ab797 updated method calls in workflow.py
except for reordering
2022-02-04 15:31:36 -05:00
Dan 2383c7d76d WIP - lots of random changes, trying to swtich over 2022-02-04 14:50:31 -05:00
alicia pritchett 2fb6746f6b new methods 2022-02-04 14:45:52 -05:00
mike cullerton 36911d35ac Added description of the service 2022-02-04 14:03:37 -05:00
Dan 6765afbd02 WIP - Move the categories and specifications out of the database. 2022-02-04 13:51:39 -05:00
mike cullerton 2706632cda Fix origin/origin problem 2022-02-03 07:14:16 -05:00
mike cullerton 6fa729900c Bug? origin/origin/brach, rather than origin/branch 2022-02-03 07:10:38 -05:00
mike cullerton 2d48151976 Fix when checkout branch does not exist yet
First attempt at merge
2022-02-03 06:47:02 -05:00
mike cullerton e9c7f3d9b6 Force checkout of branch when loading repo 2022-02-02 14:48:47 -05:00
mike cullerton 21b5d034e2 authentication
variable name changes for readability
removed unused code
2022-02-02 14:11:41 -05:00
Dan e103845218 fixing failing tests around default primary processes
Removing a ton of ununsed code from example_data.
2022-02-02 13:30:54 -05:00
mike cullerton a45c5959ef First attempt at adding credentials 2022-02-02 13:24:54 -05:00
Dan 4ec6e403f5 1. Created a UserFileService, so it's clear what we use when for File Services, UserFiles, SpecFiles, and ReferenceFiles each function differently.
2. Reference Files and Spec Files are written to disk, they do not exist in the database at all.
2022-02-02 12:59:56 -05:00
mike cullerton ec054bbdf1 set tracking on the active branch 2022-02-02 11:48:05 -05:00
mike cullerton 558d6f47e2 removed unused code from get_repo
started pull_from_remote
2022-02-02 11:30:05 -05:00
mike cullerton 8a8c3de8c4 clone repo if it doesn't exist 2022-02-01 16:25:46 -05:00
alicia pritchett c1a5032014 WIP modifying workflow service to remove value expression 2022-02-01 15:44:55 -05:00
mike cullerton 070c3cfd5d *** WIP ***
New API endpoints, and associated methods.
Services for the new endpoints
Minimal test
2022-02-01 10:49:35 -05:00
mike cullerton c07c429ae1 Fix for new version of Pandas. They check data type now. 2022-02-01 10:42:18 -05:00
mike cullerton c9b7d95a83 *** WIP ***
Beginning of Git integration
- added gitpython to Pipfile
- started additions for api, model, service, and a test
2022-01-28 16:11:36 -05:00
Dan 4c00a5762f partial commit - new spec_file_service, and new spec_file_api endpoints that use spec and file name, not file id.
removed worklow_sync
cleaned up file and workflow models
most of the test are broken.
2022-01-28 06:42:37 -05:00
Dan Funk 97c29bf77c
Merge pull request #449 from sartography/files-to-filesystem-572
Files to filesystem 572
2022-01-26 10:11:26 -05:00
mike cullerton d82367ab2a Move temp_migration_service code into migration 7225d990740e
Delete test_files_to_filesystem and test_files_from_filesystem--they were only used to give me access to the migration methods
2022-01-26 09:58:46 -05:00
Dan f3637ee6a0 resolving todos, as they are done, and removing commented out code that is no longer in use. 2022-01-25 16:37:17 -05:00
Dan 8529465322 Removed the method get_spec_data_files completly - using get_spec_files and get_spec_data to get this information instead.
Only load the spec data files if you are creating a new workflow, otherwise just deserialize the json.
Removed the stuff about calculaing the version of the spec, as we don't use it.
2022-01-25 16:10:54 -05:00
mike cullerton ae86ab4790 Make erroring workflows visible
The `process_erroring_workflows` method is called by our scheduler in __init__.py
2022-01-25 12:18:29 -05:00
mike cullerton 91fa44b2ab ** WIP **
Committing so I can work on another ticket.
2022-01-21 14:37:11 -05:00
mike cullerton ca54b850cd UNDO my previous commit regarding `get_files needs to have workflow_spec_id`
We need to call `get_spec_files` instead now
2022-01-20 14:57:32 -05:00
Dan f815add699 1. Add a default directory for the location of SYNC files.
2. Added a last_updated column to the lookup table
3. The Lookup service now uses the above, and compares it to the actual file date, we can then rebuild the lookup if needed.
4. That 755 migration loads up the models, so when you change the models, the migration starts to fail.  Not really sure what to do here, but modify the migration while we are in process.
2022-01-20 13:05:58 -05:00
mike cullerton 9ea1399578 get_files needs to have workflow_spec_id 2022-01-20 09:54:22 -05:00
mike cullerton 68820c67cb Removed (almost) all references to WorkflowSpecDependencyFile
(There is still a call in the lookup service, but we need to decide how to fix that)
2022-01-19 16:12:54 -05:00
mike cullerton 16a9a18c5b Fixed get reference file info, and added test 2022-01-19 15:17:35 -05:00
mike cullerton 091d71eb0f Cleaned up code around differences between file info and file data
Cleaned up some api code around differences between file, spec_file, and reference_file
Cleaned up some api code around differences between file info and file data
Fixed some tests for file api
2022-01-19 13:47:14 -05:00
mike cullerton d83f28e1a7 Pushed so Dan can play 2022-01-18 15:55:49 -05:00
mike cullerton 68ac056457 Merge branch 'dev' into files-to-filesystem-572 2022-01-18 08:18:14 -05:00
mike cullerton 6da6e05171 *** WIP ***
code for updating spec files
some cleanup

tests pass, but we don't have all the tests we need.
2022-01-13 15:24:29 -05:00
mike cullerton b99ed73951 Remove unused imports 2022-01-12 15:00:26 -05:00
mike cullerton cfa9f00bf3 *** WIP ***
Moved reference files to their own service
2022-01-12 14:37:33 -05:00
mike cullerton b6b9482a64 *** WIP ***
Incremental commit. Mainly cleaning up failing tests.
2022-01-12 13:16:09 -05:00
Dan d346589e07 1. added the ability to filter on level and user.
2. added a total to the response, so you can see what the total count of users is.
3. adding user_uid to the task_log table.
2022-01-12 08:58:50 -05:00
mike cullerton 9cc91f92c3 *** WIP ***
cleanup - removing commented code
2022-01-11 15:55:08 -05:00
mike cullerton 4df2ed6ce4 *** WIP ***
Failing tests, and missing functionality.
Committing to get stuff on Github.
2022-01-11 15:30:22 -05:00
Dan 962d05c875 1. Created a TaskLoggingService to encapsulate what we are doing with Task Logging through BPMN Script calls and API calls from the front end.
2. Added two api endpoints that will allow us to get all the logs for a specific workflow or study.
3. Assured that requests for logs are paginated, sortable, and can be limited to a specific code if needed.
4. Assure that we only use logging levels that match the log levels of Python.
2022-01-10 13:16:54 -05:00
mike cullerton f537fd7fbf *** WIP *** 2022-01-07 15:58:23 -05:00
mike cullerton dc27f795c8 *** WIP ***
Committing because it is Friday afternoon, and my computer is acting flaky
2022-01-07 15:34:51 -05:00
mike cullerton 86a6039dc8 *** WIP ***
**Many** tests are failing!

Committing so I can merge dev into this branch
2022-01-06 11:46:54 -05:00
mike cullerton 7f9519736c fixed some issues with how we look up records. Try to depend on the data_obj 2022-01-04 07:26:12 -05:00
mike cullerton 2cd0f84a47 Merge branch 'dev' into files-to-filesystem-572 2022-01-03 08:06:17 -05:00
Dan 1446302a04 A quick fix for an error we are seeing in testing. 2021-12-30 15:27:31 -05:00
Dan 3480e1c8ab use a signal, in addition to firing off a cancel notify when a workflow is restarted. 2021-12-23 15:30:09 -05:00
Dan bd3176eea6 Fixes #578 - issue with us displaying an incorrect end event in some cases. 2021-12-22 16:27:52 -05:00
mike cullerton 3fa5505970 Cleanup - remove print statements 2021-12-17 12:30:10 -05:00
mike cullerton 6073412d82 Clean up imports 2021-12-17 12:03:45 -05:00
mike cullerton aa27df7840 Process the spec files and their metadata 2021-12-17 11:46:21 -05:00
alicia pritchett a81f25f0ee Just kidding. We should make deepcopy and put in fake data 4 valdiation
see above. previously i deleted this, because i didnt want unitializsed data in task data. well heres a hint. don't put it in task data. make a deep copy and hold on to that for validation pursposes
2021-12-16 14:47:04 -05:00
mike cullerton 5c90ce01a6 *** WIP ***
Temp services `ToFilesystemService` and `FromFilesystemService` for migrating files To and From the filesystem - Not sure where these classes will end up
Tests to call the two services.
upgrade method in migration that writes workflow spec files and metadata to filesystem
2021-12-16 14:22:37 -05:00
mike cullerton 338352017b *** WIP ***
Moving files to the filesystem
Tools for reading/writing metadata for categories and specs
2021-12-16 08:41:49 -05:00