Commit Graph

603 Commits

Author SHA1 Message Date
Dan Funk bf3066a8b1 Upgrading Spiff to throw better error messages. 2020-04-07 17:15:05 -04:00
Dan Funk 9bf5c9e727 Fixing an error that would prevent the correct counts from displaying if the record didn't come back from the protcol builder. 2020-04-07 14:14:43 -04:00
Dan Funk c7a18709ee Merge branch 'master' of github.com:sartography/cr-connect-workflow 2020-04-07 14:09:28 -04:00
Dan Funk 697127660f Assure that all script tasks place data in a dictionary that is named exactly the same as the class - which is also the same as the Script tag. 2020-04-07 14:09:21 -04:00
Aaron Louie d23b5e747b Replaces deprecated assertEquals with assertEqual 2020-04-06 18:15:32 -04:00
Aaron Louie 0f22c8474b Removes duplicate pytest command. Runs sonar scanner only if tests pass. 2020-04-06 18:14:18 -04:00
Dan Funk c6b6ee5d70 Renamed the required_docs script to just "documents", and it returns all documented in the irb_documents look up table indexed on the "Code" - so details become available in the task data like "documents.IRB_INFOSEC_DOC.required".
Updated the irb_documents with shorter code names, thanks to Alex. Re-worked the DMN models so they can properly read from this new datastructure.
2020-04-06 16:56:00 -04:00
Dan Funk d58a90a727 Merge branch 'master' of github.com:sartography/cr-connect-workflow 2020-04-06 13:24:01 -04:00
Aaron Louie e6550b7e39 Fixes Formly expression syntax error 2020-04-06 13:22:49 -04:00
Aaron Louie 419d619efa Updates BPMN & DMN files with latest from staging 2020-04-06 13:12:34 -04:00
Dan Funk e283b86466 Fixing a bug with deleting a study. 2020-04-06 13:08:17 -04:00
Aaron Louie e866c2c4bb Moves awscli install to install step 2020-04-03 16:43:25 -04:00
Dan Funk a322801c91 Allow a study to be deleted, even if some statistics are laying around. 2020-04-03 16:41:16 -04:00
Dan Funk fae07f289f Merge branch 'master' of github.com:sartography/cr-connect-workflow 2020-04-03 16:24:45 -04:00
Dan Funk 60a10bb688 Marshmallow isn't the right tool when dealing with large models with lots of null values. Rather than fight the process of mamaging the Study Details, I'm letting that fall through, and we can test on an individual value or maybe set up a constants array when that becomes meaningful. 2020-04-03 16:24:38 -04:00
Aaron Louie 07551ed4a8 Moves AWS CLI install step into deploy script 2020-04-03 15:53:24 -04:00
Aaron Louie 222e4635c3 Merge remote-tracking branch 'origin/master' 2020-04-03 15:38:32 -04:00
Aaron Louie 846b64d93a Skips cleanup in deploy step 2020-04-03 15:38:23 -04:00
Dan Funk 5de850cfdb Merge branch 'master' of github.com:sartography/cr-connect-workflow 2020-04-03 13:52:19 -04:00
Dan Funk fe40be752f fixing a bug in the Protocol builder schema, more to fix there, and deserves a test, but this corrects the investigators 2020-04-03 13:52:12 -04:00
Aaron Louie 71d9bbec02 Replaced password with token 2020-04-03 11:29:33 -04:00
Aaron Louie 13542d78c7 Moves awscli install to after script success 2020-04-03 10:14:28 -04:00
Aaron Louie f501fc47fb Removes duplicate build command 2020-04-03 10:09:06 -04:00
Aaron Louie b0e49943a0 Sends message to DCOS when Docker Hub image is updated 2020-04-03 09:53:47 -04:00
Dan Funk b39e4b382f Catch more template processing errors. 2020-04-02 15:06:00 -04:00
Dan Funk 785918cb7f Be sure the validation process examines the data located in the documentation and correctly handles boolean fields. 2020-04-02 14:47:20 -04:00
Dan Funk 534e8493a4 Deleting was broken because statistics were causing an integrity constraint.
Updating IDS bpmn to the latest for testing.
Adding a TODO
2020-04-02 12:58:52 -04:00
Dan Funk af5ab6a2c2
Merge pull request #25 from sartography/feature/status_refactor
Feature/status refactor
2020-03-31 09:31:53 -04:00
Dan Funk c86f3321c5 reworking the test_study_service to run from a clean state, this is growing so complex. Tests really need to be isolated better. 2020-03-30 15:39:50 -04:00
Dan Funk 17796193de fixing a bug that was causing failing tests.
Adding id and spec_version to the workflow metadata.
Refactoring the processing of the master_spec so that it doesn't polute the workflow database.
Adding tests to assure that the status and counts are updated on the workflow model as users make progress.
2020-03-30 14:01:57 -04:00
Dan Funk 34b6ec92bf updating the API
Removing the call for study/workflows - as workflow information is returned with the study by default.
Fixing a bug in the workflow spec model schema.
2020-03-30 10:12:10 -04:00
Dan Funk 2cd5d70a77 missing another migration. This should fix it. 2020-03-30 09:40:56 -04:00
Dan Funk f8cca274d4 Adding a migration for the last commit. 2020-03-30 08:36:10 -04:00
Dan Funk 4a916c1ee3 Created a "StudyService" and moved all complex logic around study manipulation out of the study api, and this service, as things were getting complicated. The Workflow Processor no longer creates the WorkflowModel, the study object handles that, and only passes the model into the workflow processor when it is ready to start the workflow.
Created a Study object (seperate from the StudyModel) that can cronstructed on request, and contains a different data structure than we store in the DB. This allows us to return underlying Categories and Workflows in a clean way.

Added a new status to workflows called "not_started", meaning we have not yet instantiated a processor or created a BPMN, they have no version yet and no stored data, just the possiblity of being started.

The Top Level Workflow or "Master" workflow is now a part of the sample data, and loaded at all times.

Removed the ability to "add a workflow to a study" and "remove a workflow from a study", a study contains all possible workflows by definition.

Example data no longer creates users or studies, it just creates the specs.
2020-03-30 08:00:16 -04:00
Dan Funk c9900d787e Every good deed goes punished. 2020-03-27 15:48:21 -04:00
Aaron Louie 6ebd4dce42 WIP: Adds default workflow spec categories 2020-03-27 15:32:07 -04:00
Dan Funk 57f1fa670e fixing a stupid mistake. 2020-03-27 14:55:53 -04:00
Dan Funk 907e1cbbb3 minor fixes that were breaking when connecting to the front end. 2020-03-27 14:27:50 -04:00
Dan Funk b5fca2f683 Forgot a migration 2020-03-27 12:19:32 -04:00
Dan Funk b61a35f956 Merge remote-tracking branch 'origin/master' into feature/status_refactor
Fixing adding a study so all workflows are again added, will add status on those workflows based on output from the master bpmn diagram, which is coming shortly.
2020-03-27 11:55:36 -04:00
Dan Funk c7d2c28178 Vastly more informative ApiError model that provides details on the underlying task where the error occured.
Added a validate_workflow_specification endpoint that allows you to check if the workflow will execute from beginning to end using random data.
Minor fixes to existing bpmns to allow them to pass.
All scripts must include a "do_task_validate_only" that restricts external calls and database modifications, but performs as much logic as possible.
2020-03-27 08:29:31 -04:00
Dan Funk 6c832829b0 Merge remote-tracking branch 'origin' into feature/status_refactor 2020-03-26 20:47:37 -04:00
Aaron Louie f77ebe3177 Adds display order to categories 2020-03-26 15:29:52 -04:00
Dan Funk e2c408b70d Removed all self-referential calls in the study_api. One api endpoint should never call another api endpoint. Moved the logic for updating a study to the study Model, rather than checking and setting dictionary values which will become very hard to maintain.
The protocol builder service now returns real models, not dictionaries, forcing proper validation and fail-fast behavior.
Changed the name of the "status" spec, to "top_level_workflow" and removing any connection a workflow or study has with this specification.  It is only unused to determine status in real time, and is not reused or tracked.
Modified the required documents script to return a dictionary and not an array, making it easier to speak to specific values in the BPMN and DMN.
Working on new ways to test the top_level_workflow in the context of updates, this is still a work in progress.
Making use of several modifications to the Spiff library that enables more complex expressions in DMN models. This is evident in the new DMN models for the top_level_workflow
2020-03-26 12:51:53 -04:00
Aaron Louie 4ed89eec5b Mocks Protocol Builder response 2020-03-26 10:30:15 -04:00
Aaron Louie 3fb9fb9b22 Simplifies status BPMN & DMN. 2020-03-26 10:06:12 -04:00
Aaron Louie 8752a81042 Disables broken spec 2020-03-26 08:52:53 -04:00
Aaron Louie dd5971bea9 Updates workflow specs for data loader 2020-03-25 11:13:52 -04:00
Dan Funk bd92c230e4 Fix the API so we are clear what is being returned when updating a file. 2020-03-25 08:06:58 -04:00
Dan Funk 37a8b69ffe Cleaning up a failing test, not sure why this isn't failing locally? 2020-03-24 14:40:24 -04:00