Commit Graph

81 Commits

Author SHA1 Message Date
Dan Funk b2aede2b8d Glob on windows seems to return directories where in linux it does not, so now assuring we don't try to add files from the sub-directory. 2020-10-05 17:35:35 -04:00
Aaron Louie 2846bf9451 Fleshes out fake PB response, so workflows that pull from PB actually trigger validation errors 2020-08-04 15:50:29 -04:00
Dan Funk 4d11fc04a0 dropping the "read_only" flag in favor of a "do_engine_steps" flag, which more clearly defines what is happening. 2020-07-28 13:51:29 -04:00
Dan Funk f15626033d Allow the workflow to be requested without making changes to the workflow - requires that you specify a read_only flag of true, otherwise it assumes that you want a fully prepared workflow with the next ready task set to run. 2020-07-28 13:33:38 -04:00
Dan Funk 0cb480801b Provide event data on the Study api endpoint.
Speed up the tests a little, because that got out of hand.  Need to dig into what is causing this problem.
2020-07-27 17:05:01 -04:00
Aaron Louie d67105332f Merge branch 'dev' into feature/lanes_and_roles 2020-07-16 14:00:53 -04:00
Dan Funk 9077ff3ebf It is not possible to use task_data for an auto-complete field. It's too expensive an operation to provide that feature on the backend, and the data already fully resides on the front end anyway. Task-data can be used to populate enum fields if needed, so it can populate dropdowns, radios and checkboxes, just not auto-complete. 2020-07-14 11:38:48 -04:00
Dan Funk a48322ef6a Partial work on CR Connect Roles.
Adding checks in the API to assure the correct person is completeing a task based on the task's lane.
Adding lane to the Navigation item.
Adding a check to assure that unique user ids can be identified using task.data
Added some additional ldap entries to make testing and development easier.
Removed a big chunk of duplicate code from task_tests_api
Modified some of the helper functions to make it easier to test as specific users.
Added some additional bpmn models to the tests for testing lanes and roles.
2020-07-14 10:29:25 -04:00
Aaron Louie 07066b8a16 Looks up enum options from task data 2020-07-13 17:46:28 -04:00
Aaron Louie 3c8e1e5c37 Tests for presence of data property in enum options 2020-07-13 11:02:29 -04:00
Dan Funk 6a79fb3581 There may be multiple investigators of the same type that come back from the protocol builder, adding some tests and additional code to handle this, but still keep the list flat, currently appends a number to the investigator type when there is more than one. 2020-07-07 17:16:33 -04:00
Dan Funk a5cef8775e * Modifying the StudyInfo script to return both "investigators" and "roles", the investigators argument will return only the current active investigators according to the protocol builder. The "roles" argument returns all possible roles - it is what "investigators" use to be.
* The Task.title returned to the front end will now attempt to process the "display_name" property for dot-notation syntax, making it possible to use this for multi-instance tasks,
  but will work in all cases where we want he title to change based on values in the data model.
* Fixing a bug the test_study_api where it wasn't updated when we made recent changes to the different states of a study.
*
2020-07-06 12:09:21 -04:00
Dan Funk f183e12fe5 Provides some basic tools for getting additional information about a lookup field.
Adds an optional 'value' parameter to the lookup endpoint so you can find a specific entry in the lookup table.
Makes sure the data attribute returned on a lookup model is a dictionary, and not a string.
Fixes a previous bug that would crop up if double spaces were used when performing a search.
2020-06-30 10:34:16 -04:00
Dan Funk d3ce1af1ce Provides some basic tools for getting additional information about a lookup field.
Adds an optional 'id' parameter to the lookup endpoint so you can find a specific entry in the lookup table.
Makes sure the data attribute returned on a lookup model is a dictionary, and not a string.
Fixes a previous bug that would crop up in double spaces were used when performing a search.
2020-06-30 10:00:22 -04:00
Aaron Louie fe61333b7b Fixes typo 2020-06-26 12:31:15 -04:00
Carlos Lopez dc5ffd29d0 Refactoring shared code 2020-06-22 14:07:57 -06:00
Dan Funk 6aec15cc7c Shifting to a different model, where the TaskEvents store ONLY the form data submitted for that task.
In order to allow proper deletion of tasks, we no longer merge data returned from the front end, we set it directly as the task_data.
When returning data to the front end, we take any previous form submission and merge it into the current task data, allowing users to keep their previous submissions.
There is now an "extract_form_data" method that does it's best job to calculate what form data might have changed from the front end.
2020-06-19 08:22:53 -04:00
Dan Funk c7d8eaff30 Merge branch 'dev' into feature/refactor_data_loading 2020-06-16 13:15:43 -04:00
Aaron Louie 561e254315 Prevents non-admin users from editing each others' tasks. Fixes bug where test user uid was not being set from token. Moves complete form and get workflow API test utility methods into BaseTest. 2020-06-12 13:46:10 -04:00
Carlos Lopez 16ca4fa2c0 Merge branch 'rrt/dev' into feature/mail-system 2020-06-05 12:35:05 -06:00
Carlos Lopez a6758fd555 Removing deprecation warnings 2020-06-05 12:08:46 -06:00
Dan Funk 1324533865 Some additional cleanup - when a file is "archived" it is no longer returned for any endpoints about files, but it
is directly accessible via id, in the event some request is made for it at a later date.
2020-06-04 09:49:42 -04:00
Dan Funk c2a1b0175a Tweaks to approvals 2020-06-01 21:49:30 -04:00
Dan Funk 1db9401166 Don't put all the data into Spiff Tasks on a reload or backtrack, just store the data that gets submitted each time in the task log, and use that.
This should correct issues with parallel tasks and other complex areas - so we don't have tasks seeing data that isn't along their path.
2020-06-01 17:42:39 -04:00
Dan Funk 11413838a7 Faster lookup fields. We were parsing the spec each time to get details about how to search. We're just grabbing the workflow id and task id now and building that straight into the full text search index for faster lookups. Should be peppy.
Another speed improvement - data in the FileDataModel is deferred, and not queried until it is specifically used, as the new data structures need to use this model frequently.
2020-05-29 01:39:39 -04:00
Dan Funk dba41f4759 Ludicrously stupid launch in a refactor of the way all files work in the system at a time where I crave sleep and peace above all other things.
Added a File class, that we wrap around the FileModel so the api endpoints don't change, but File no longer holds refences to versions or dates of the file_data model, we
figure this out based on a clean database structure.

The ApprovalFile is directly related to the file_data_model - so no chance that a reviewer would review the incorrect version of a file.py

Noticed that our FileType enum called "bpmn" "bpmm", hope this doesn't screw someone up.

Workflows are directly related to the data_models that create the workflow spec it needs.  So the files should always be there.  There are no more hashes, and thus no more hash errors where it can't find the files to rebuild the workflow.py

Not much to report here, other than I broke every single test in the system at one point.  So I'm super concerned about this, and will be testing it a lot before creating the pull request.
2020-05-28 20:03:50 -04:00
Dan Funk 7869fa596e Protocol Builder isn't disabled on the dcos servers, trying to figure out why, and assure it isn't some sort of weird race condition. 2020-05-26 22:42:49 -04:00
Dan Funk 6cd4ef64d1 Fixing add_study api endpoint, so you can actually add a new "Study" with just some basic information.
Using the LDAP service for checking user details in development mode - even if you are using the back door.
Added a new Flask fucntion load-example-rrt-data that loads the rrt workflow, and not the CRC wrokflows.
Modified the "load-example-data" in the tests to use some test data, rather than loading up all the workflows[
in CRC each time, with a parameter to load crc data if that is required - which is enabled for just a handful of tests.
(Tests run in 1/4 the time now)
2020-05-25 12:29:05 -04:00
Dan Funk 503c1c8f18 Allow disabling the Protocol Builder
PB_ENABLED can be set to false in the configuration (either in a file called instance/config.py, or as an environment variable)

Added a check in the base_test, to assure that we are always running tests with the test configuration, and bail out otherwise.  Setting TESTING=true as an environment variable will get this, but so well the correct ordering of imports. Just be dead certain the first file every test file imports is base_test.py.

Aaron was right, and we call the Protocol Builder in all kinds of awful places.  But we don't do this now.  So Carlos, you should have the ability to reuse a lot of the logic in the study_service now.

I dropped the poorly named "study-update" endpoint completely.  We weren't using it. POST and PUT to Study still work just fine for doing exactly that.

All the tests now run and pass with the Protocol builder disabled. Tests that specifically check PB behavior turn it back on for the test, or mock it out.
2020-05-22 14:37:49 -04:00
Dan Funk 951710d762 ldap lookup.
Refactored calls into a new lookup_service to keep things tidy.

New keys for all enum/auto-complete fields:
    PROP_OPTIONS_FILE = "spreadsheet.name"
    PROP_OPTIONS_VALUE_COLUMN = "spreadsheet.value.column"
    PROP_OPTIONS_LABEL_COL = "spreadsheet.label.column"
    PROP_LDAP_LOOKUP = "ldap.lookup"
    FIELD_TYPE_AUTO_COMPLETE = "autocomplete"
2020-05-19 16:11:43 -04:00
Dan Funk dff92c6e4c fixing a test. 2020-05-18 20:49:50 -04:00
Dan Funk 7a380fdeb4 Forgot to fix another test and to add the example file used for a previous test. 2020-05-16 15:38:15 -04:00
Dan Funk 43bf8f5337 Fixing a bug with navigation where elements went missing past exclusive gateways on subsequent forks. 2020-05-16 15:33:06 -04:00
Dan Funk de435bd961 the heck with camel case, what the heck TypeScript? Get a grip. This is a python API. 2020-05-15 16:38:37 -04:00
Dan Funk 53255ef35e massive overhaul of the Workflow API endpoint.
No Previous Task, No Last Task, No Task List.  Just the current task, and the Navigation.
Use the token endpoint to set the current task, even if it is a "READY" task in the api.
Previous Task can be set by identifying the prior task in the Navigation (I'm hoping)
Prefering camel case to snake case on all new apis.  Maybe clean the rest up later.
2020-05-15 15:54:53 -04:00
Dan Funk b63ee8159e We now only return the ready user tasks, not all tasks, and even then the ready user tasks don't come back with the forms and details, just the bare minimum. Speeds things up considerably, and most of this information wasn't used anyway. 2020-05-14 17:13:47 -04:00
Dan Funk 6d4348d644 Fixing some failing tests. Moved the task properties into a dictionary, but moving the form field properties to a dictionary will be a larger effort that we don't want to get into on either the back or front end right this moment. 2020-05-14 14:39:14 -04:00
Dan Funk 55a1850e7c adding a navigation component to the Workflow Model.
running all extension/properties through the Jinja template processor so you can have custom display names using data, very helpful for building multi-instance displays.
Properties was returned as an array of key/value pairs, which is just mean.  Switched this to a dictionary.
2020-05-14 13:43:23 -04:00
Dan Funk b7c11fd893 Merge branch 'master' into feature/investigators_reference_file 2020-05-11 17:36:37 -04:00
Dan Funk 02f8764056 Updated to use the latest script engine / evaluation engine that creates a single location where all values used in BPMN/DMN are processed. Right now this is a python based interpreter, but we will eventually base this on FEEL expressions.
The validation process needs to take the api model into account so we catch errors with bad file names.
2020-05-11 17:04:05 -04:00
Dan Funk da7cae51b8 Adding a new reference file that provides greater details about the investigators related to a study.
Improving the study_info script documentation to provide detailed examples of values returned based on arguments.
Making the tests a little more targetted and less subject to breaking through better mocks.
Allow all tests to pass even when ther protocol builder mock isn't running locally.
Removing the duplication of reference files in tests and static, as this seems silly to me at the moment.
2020-05-07 13:57:24 -04:00
Dan Funk 1571986c0e I had to give up and live with the idea that we can only render documentation on the current task, not on the previous or next tasks. I think this is ok. If you want to view a task, you need to make it the active task to assure all the parts and pieces are in place. 2020-05-06 13:01:38 -04:00
Dan Funk 9629b36e92 Setting JSON_SORT_KEYS to false, assuring that Flask does not resort all data returned to the front end.
Updating Spiff Workflow which has some critical behavioral changes around MultiInstance.
2020-05-06 10:59:49 -04:00
Dan Funk 2699f5c65c Refactor the stats models, and assure they are very correct across all tests with the workflow api.
I noticed we were saving the workflow every time we loaded it up, rather than only when we were making changes to it.  Refactored this to be a little more careful.
Centralized the saving of the workflow into one location in the processor, so we can make sure we update all the details about that workflow every time we save.
The workflow service has a method that will log any task action taken in a consistent way.
The stats models were removed from the API completely.  Will wait for a use case for dealing with this later.
2020-05-04 10:57:09 -04:00
Dan Funk bec59a71d7 Deleteing stuff is a damn mess, but this is a little cleaner. 2020-04-29 16:07:39 -04:00
Dan Funk 13333fb306 Adding an API endpoint that allows setting the token on the workflow to a specific task.
Added error checking such that attempting to submit data for a task that is not in the "READY" state throws an error message.
For some reason I'm getting lots of errors in the tests as they try to hit API endpoints they were not hitting before, so adding a number of mocks to some of the study service tests.
2020-04-28 17:25:53 -04:00
Dan Funk 3e3a249e3c Verifying Sub-Process works, and adding a field to expose a hint as to the sub-process in which the task occurs.
Because the name field is now used to expose workflow/sub-process information on tasks, we can't use it to store the workflow_version, so that is now just stored on the database model.  Which is much cleaner and removes a duplication.
2020-04-28 13:48:44 -04:00
Dan Funk 447f4013f8 Assure that a hard-reset sticks, and the system is properly updated. 2020-04-27 16:08:23 -04:00
Dan Funk 7b085c9c9d Adding an API Endpoint that will return a list of LookupValues that match a given query - can be used to populate an auto-complete table. 2020-04-22 19:40:40 -04:00
Dan Funk 0a74bf8c44 We can now collect, and provide "extension properties" on a task as set in the camunda modeler.
These are provided as "properties" on a task, and are identical in structure to properties on a form field.
2020-04-21 12:07:59 -04:00