4ec6e403f5
1. Created a UserFileService, so it's clear what we use when for File Services, UserFiles, SpecFiles, and ReferenceFiles each function differently.
...
2. Reference Files and Spec Files are written to disk, they do not exist in the database at all.
2022-02-02 12:59:56 -05:00
6f233a9b79
removed unused code
2022-02-02 12:51:14 -05:00
2a0d71aab6
Status was removed from the model
2022-02-02 10:04:52 -05:00
070c3cfd5d
*** WIP ***
...
New API endpoints, and associated methods.
Services for the new endpoints
Minimal test
2022-02-01 10:49:35 -05:00
c9b7d95a83
*** WIP ***
...
Beginning of Git integration
- added gitpython to Pipfile
- started additions for api, model, service, and a test
2022-01-28 16:11:36 -05:00
4c00a5762f
partial commit - new spec_file_service, and new spec_file_api endpoints that use spec and file name, not file id.
...
removed worklow_sync
cleaned up file and workflow models
most of the test are broken.
2022-01-28 06:42:37 -05:00
f815add699
1. Add a default directory for the location of SYNC files.
...
2. Added a last_updated column to the lookup table
3. The Lookup service now uses the above, and compares it to the actual file date, we can then rebuild the lookup if needed.
4. That 755 migration loads up the models, so when you change the models, the migration starts to fail. Not really sure what to do here, but modify the migration while we are in process.
2022-01-20 13:05:58 -05:00
68820c67cb
Removed (almost) all references to WorkflowSpecDependencyFile
...
(There is still a call in the lookup service, but we need to decide how to fix that)
2022-01-19 16:12:54 -05:00
68ac056457
Merge branch 'dev' into files-to-filesystem-572
2022-01-18 08:18:14 -05:00
b99ed73951
Remove unused imports
2022-01-12 15:00:26 -05:00
e67a1554ea
Always but always but ALWAYS, use server_default=func.now() when setting up a datetime field we autopopulate.
2022-01-12 12:33:59 -05:00
d346589e07
1. added the ability to filter on level and user.
...
2. added a total to the response, so you can see what the total count of users is.
3. adding user_uid to the task_log table.
2022-01-12 08:58:50 -05:00
4df2ed6ce4
*** WIP ***
...
Failing tests, and missing functionality.
Committing to get stuff on Github.
2022-01-11 15:30:22 -05:00
7176452fe3
forgot the migration, also fixing an error in the TaskLogQuerySchema.
2022-01-10 13:26:16 -05:00
962d05c875
1. Created a TaskLoggingService to encapsulate what we are doing with Task Logging through BPMN Script calls and API calls from the front end.
...
2. Added two api endpoints that will allow us to get all the logs for a specific workflow or study.
3. Assured that requests for logs are paginated, sortable, and can be limited to a specific code if needed.
4. Assure that we only use logging levels that match the log levels of Python.
2022-01-10 13:16:54 -05:00
dc27f795c8
*** WIP ***
...
Committing because it is Friday afternoon, and my computer is acting flaky
2022-01-07 15:34:51 -05:00
86a6039dc8
*** WIP ***
...
**Many** tests are failing!
Committing so I can merge dev into this branch
2022-01-06 11:46:54 -05:00
ad9a480f93
Merge remote-tracking branch 'origin/dev' into waiting-workflow-errors-566
2021-12-13 13:05:02 -05:00
cdaad5dcc0
Allow 'none' for progress_status
...
Well this fixes tests but also, I dont think you necessarily have this right off the bat if you havent done anything in a study, which i handle on the frontend. so null is fine
2021-12-13 12:34:32 -05:00
8b27423172
Updating study model schema to include Progress Status
2021-12-13 12:29:15 -05:00
acbb8898e4
Merge branch 'more-study-statuses-562' into waiting-workflow-errors-566
2021-12-10 16:52:07 -05:00
f748fa3db8
Removed unused imports
2021-12-10 16:48:28 -05:00
6d09aa1817
We moved the new statuses to a new column and created a separate enum class for them.
...
They really are sub_statuses of the in_progress status
2021-12-10 15:57:02 -05:00
52baf25459
In `WorkflowService.do_waiting`, if a workflow causes an exception, change its status from `waiting` to `erroring` so we don't continue to call it in `do_waiting`.
2021-12-09 15:33:39 -05:00
6db9cdabc7
New study statuses
2021-12-08 10:50:30 -05:00
0cfa59bfea
Merge pull request #429 from sartography/chore/clear-form-data-553
...
Fixes #553 - Don't load Form data
2021-12-08 10:37:44 -05:00
db1f3ba6ab
Model and schema for new StudyCreator api endpoint
...
Update StudyModel to use new model
2021-12-06 14:36:34 -05:00
7524e97c9a
Fixes #553 - Don't load Form data
...
adds an extension so that you can opt to clear out form data on a given task
2021-12-01 16:11:52 -05:00
6d548c2a74
*** WIP ***
...
Committing to work on another ticket
There are TODOs in study_service
2021-11-05 09:59:14 -04:00
b03e596dcf
One thing at a time; remove HSRNUMBER for now
2021-11-03 12:58:03 -04:00
9c8ae36ba3
UVA IRB passes back `Q_COMPLETE` and `HSRNUMBER`
2021-11-03 12:40:43 -04:00
c4f3a3ad0d
Merge pull request #413 from sartography/logging-script-513
...
Logging script #513
2021-11-01 15:41:38 -04:00
b0c479420d
New task_log model with schema, and migration
2021-10-27 12:19:09 -04:00
7090a22a23
Remove content_html from the data we return from the email and get_email_data scripts
2021-10-26 16:28:50 -04:00
1c84bd9f1b
Merge branch 'dev' into refactor/enums_as_key_value
2021-10-19 12:02:45 -04:00
5429e7da7d
All enumerated lists used in web forms should contain a single value, not a dictionary of value/labels.
...
Removing the spreadsheet.value.column and data.value.column so we just have value.column for both.
Improving the __str__ function in the ApiError class, to make debugging a little easier.
Adding a "validate_all" flask command, to help us track down any issues with current workflows in production (use this in concert with sync_with_testing)
Fixed logs of tests.
removed fact_runner.py, a very early and crufty bit of code.
2021-10-19 10:13:43 -04:00
dcb06f3753
Schema for returning email model
2021-10-13 11:42:01 -04:00
699fc7daac
Change column in email table from workflow_id to workflow_spec_id
2021-10-12 13:42:11 -04:00
24932072dc
Merge branch 'dev' into email-data-script-487
2021-10-07 12:30:13 -04:00
10abf51f70
Added workflow_spec_id to WorkflowMetadata model, because the id is the workflow id, not the workflow_spec id.
...
Then, we use workflow_spec_id in StudyService when looking for workflow status.
2021-10-07 10:02:13 -04:00
dab233a07a
Add columns to email model for reporting
...
Add Schema for email model
2021-10-07 09:19:37 -04:00
7b6beae9c1
fixes frontend bailing out
...
workflow id is workflow id
2021-10-06 14:33:09 -04:00
572ca1db6b
Fixes for backend code after removing `name` column from `workflow_spec` and `workflow_spec_category` tables
2021-10-05 14:17:41 -04:00
fcf538ca92
remove `name` column from workflow_spec and workflow_spec_category
2021-10-05 13:10:36 -04:00
c3995bfc3e
Another try at getting the url correct. This paired with an additional setting on the server maybe.
2021-10-01 14:25:00 -04:00
5924ee48f4
One more try at getting the currect url.
2021-09-30 16:56:27 -04:00
6b5a55df4c
Merge pull request #383 from sartography/feature/476_file_url
...
URL for files
2021-09-30 13:47:37 -04:00
fbfb241c0b
Merge pull request #382 from sartography/feature/381_validate_field_types
...
fixes #381 Assure that the field type provided is supported. Catches errors suc…
2021-09-30 13:46:40 -04:00
e002ffd363
Fixing a broken test.
2021-09-30 10:30:58 -04:00
d5d4496cd0
Admin flag on category + migration + updated test
2021-09-29 16:53:59 -04:00
8a4a53f028
Assure that the field type provided is supported. Catches errors such as adding a custom field type of 'text_area' rather than "textarea"
2021-09-29 14:05:45 -04:00
8e00f16eaa
Assure that the field type provided is supported. Catches errors such as adding a custom field type of 'text_area' rather than "textarea"
2021-09-29 11:43:08 -04:00
e4add14193
Merge pull request #378 from sartography/bug/246_user_ldap_refactor
...
Bug/246 user ldap refactor
2021-09-22 15:31:34 -04:00
deff293fff
Merge pull request #376 from sartography/bug/468_missing_file_date
...
fixes #448 - Missing file date
2021-09-22 15:29:32 -04:00
19104303de
Refactor of the way we store and return details about users - All the details about a user, or individual associated with a study is returned within in an Ldap model. I've removed duplication between these models. This required some cleanup of the tests, and a migration that will drop the user details.
2021-09-22 13:16:25 -04:00
4b591a076f
Adding importlib-resournces as a dependency, which is suddenly failing on Travis, likely due to different versions of Python as I try to upgrade to 3.9
2021-09-21 14:36:57 -04:00
8b8cfde6cb
Add `short_name` and `proposal_name` to the study model
2021-09-17 11:53:45 -04:00
5c0d30becb
We just needed to allow read_only_expression
2021-09-03 09:48:16 -04:00
3d7dadc319
Model changes and revision file for task_spec column to file and data_store models
2021-08-26 10:36:10 -04:00
5692f7d377
Merge pull request #359 from sartography/413-LibraryCleanup
...
413 library cleanup
2021-08-18 13:39:37 -04:00
b4ecb0f97a
Add backref cascade and make changes to the spot where we delete so it loads objects into memory and then uses the SQLAlchemy plumbing to make sure that the db object gets cleaned up.
2021-08-18 08:25:17 -04:00
032aad9bc4
Fixes #383 - Timestamp on files is UTC?
...
also this includes the shield test thing for the boolean_type property whoopsie
2021-08-17 13:04:24 -04:00
1f82143aea
Remove HSR Number from models
2021-08-16 12:50:09 -04:00
46d7b13326
Start to remove HSR Number
2021-08-16 12:09:02 -04:00
848c2e622f
Always use a schema to define what is being returned, it enforces consistency of the API and internally we can depend on well defined objects.
2021-08-10 16:16:08 -04:00
913c12c02d
Merge branch 'dev' into bug/pi_name_missing_246
2021-08-10 15:06:52 -04:00
41ad7935f8
Merge branch 'dev' into 367-Call-Activity
...
# Conflicts:
# Pipfile.lock
2021-07-29 09:28:21 -04:00
b857fddbad
commit in intermediate changes to work on a bug
2021-07-26 13:00:11 -04:00
48516e4009
Parent side working
2021-07-22 10:23:05 -04:00
2455495457
got library side working - still need to have parent working
2021-07-22 10:20:34 -04:00
4632c6374f
Merge branch 'dev' into bug/missing_pi_name_246
...
# Conflicts:
# crc/api.yml
# crc/services/study_service.py
2021-07-20 11:44:11 -04:00
b0cf74fa3d
Merge branch 'dev' into add-by-user-61
...
# Conflicts:
# tests/files/test_files_api.py
2021-07-07 10:46:50 -04:00
c11b5cfb29
Add user_uid to api output
2021-07-07 08:18:02 -04:00
23be257db0
Add user_uid column to file_data table
2021-07-06 17:07:47 -04:00
cafdc4d100
Removing an unneeded modification to the database structure for lookup tables.
2021-07-06 14:02:07 -04:00
1b1a994360
Refactoring Reference files to use the lookup table, rather than parsing the results directly out of the spreadsheet, or attempting to cache them.
...
Adding a DocumentService to clean up the FileService, and get Documents well seperated, as it seems likely be pulled out or seperated in the future, there is now a Documents api file as well, for the same reason.
Some other minor changes are just fixing white space to assure our code is linting correctly.
I removed _create_study_workflow_approvals from the base test, as we don't use approvals like this anymore.
2021-07-06 13:10:20 -04:00
dce1d86e93
commit changes before looking at something else
2021-07-06 10:34:57 -04:00
61ad371680
Clean up the file api so we don't have a bunch of extra stuff in there. Just include the raw data from the document spraedsheet if it is available.
2021-06-22 15:08:08 -04:00
1f3febe2df
Fix the panda warnings, which means we only support xlsx files now.
...
Include the document details when reutnring a file from the file model.
Improve the validation of file form elements.
2021-06-22 14:58:52 -04:00
07eb3f9ca8
Moving metrics into SpiffWorkflow so we can run the performance metrics deeply across both systems simultaniously.
...
Upgrading libraries.
Fixing deprication issue with Pandas and numpy.
We can only process xlsx files now, plain oldschool xls is fully removed.
2021-06-18 16:41:55 -04:00
586a0f2f1a
Merge pull request #325 from sartography/feature/file_refactor
...
Prefer tasks that share a parent over just the the next available tas…
2021-06-08 11:28:23 -04:00
59f605c3df
Prefer tasks that share a parent over just the the next available task when returning the next_task in the workflow processor.
2021-06-08 08:03:14 -04:00
2e3d8c7343
Fix for DataStoreSchema missing the file_id column.
...
The file_id column is a foreign key.
2021-06-01 13:51:10 -04:00
a1bb30e689
Switching from using "default" to "server_default" for all default time settings, and running migrations so that this is set at the database level rather than in python, to hopefully correct some issues with dates being 4 hours in the future. Having a very hard time replicating this issue locally.
2021-05-14 12:28:50 -04:00
34759a2f3d
Merge branch 'dev' into feature/performance-refactor
2021-05-04 11:30:13 -04:00
d6054a9846
Merge branch 'dev' into 310-task-event-timezone
2021-05-04 11:21:36 -04:00
77d9bfca43
Merge pull request #296 from sartography/launch-workflow-outside-study-204
...
Launch workflow outside study 204
2021-05-04 11:16:08 -04:00
a719cf4bf9
When retrieving the study, only update the status of underlying workflows if specifically requested.
...
Record the size of a file in the database for quick access (this helps with a frontend refactor, so it isn't downloading the file just to see it's size)
Cleaning up the timing/performance metric reporting to make it easier to read.
Fixing a bug that prevented non-admins for getting the document-directory
2021-04-30 11:55:12 -04:00
24c818bf31
Added study_id to workflow_api
2021-04-29 14:29:21 -04:00
c029dad688
Convert some datetime columns to include timezone
2021-04-28 10:00:22 -04:00
21e95b106d
Merge pull request #295 from sartography/299-document-datastore
...
Add file_data_get and file_data_set functions that can be accessed in…
2021-04-26 12:49:46 -04:00
047c68b2b5
Just some quick tidying up. Dropping commented out code, and using the SQLAlchemyAutoSchema to handle serializing db models
2021-04-26 12:31:22 -04:00
39fcab3730
Merge branch 'dev' into launch-workflow-outside-study-204
2021-04-26 10:27:32 -04:00
40ee20ecca
Add file_data_get and file_data_set functions that can be accessed in a bpmn script function
...
Fixes #299
2021-04-26 09:55:28 -04:00
22432aaf2c
study_id is no longer required for task events
2021-04-26 08:36:09 -04:00
c0655c9d03
Added `standalone` column to workflow_spec
...
Added `user_id` column to workflow
2021-04-26 08:33:55 -04:00
ecff7d20d3
Adding a test for the new endpoint and cleaning up the API documenation
2021-04-21 14:00:19 -04:00
e2197ddab9
Changes to support the frontend data pane
...
fixes #277 and requires frontend branch 277-document-pane
2021-04-20 08:12:27 -04:00
d7894f43b6
These changes relate to changes in PB Mock ticket 273, where we removed Q_COMPLETE as a column in the PB Mock Study table. It is now a relationship.
...
Here, we removed Q_COMPLETE from ProtocolBuilderStudy and ProtocolBuilderStudySchema definitions to accommodate the PB Mock changes.
***This push will need to be coordinated with ticket 273***
2021-04-02 12:01:00 -04:00
b64b52e7cb
Assure that we properly return the new status message over the api calls.
2021-03-26 19:47:31 -04:00
a08c6b794f
Updated Study Associate Endpoints
2021-03-22 17:30:49 -04:00
1000b22a3b
Don't raise errors on non-ready tasks that calculate a title, the information may not be avaiable yet.
...
Also Allow autocomplete-num as a valid field property.
2021-03-12 15:40:23 -05:00
9e758c57d3
Merge branch 'dev' into bug/225_enum_lookup_same_field_name
2021-03-02 14:15:58 -05:00
06dedd8a32
Add short_title to study, change update_study task to use kw arguments and add the short_title to the update_study script
2021-03-02 10:03:53 -05:00
aac3d5c16e
Bug #255 , this requires the front end to pass in the name of the task, when doing a lookup. This will prevent a bug where we have multiple user tasks, with enum fields that set the same variable, but use different lookup tables to populate the dropbown or search feature.
2021-03-01 14:54:04 -05:00
c795dd7aea
Merge branch 'dev' into 206-extend-study-access
2021-02-24 12:16:35 -05:00
3dbe39c6fe
initial checkin of changes for branch
2021-02-24 12:05:06 -05:00
349c3d657a
Merge branch 'dev' into 201-changes-to-in-progress
2021-02-22 18:17:59 -05:00
53fa49e668
Merge pull request #243 from sartography/183-remove-study-approval-code
...
Remove Approvals per ticket 183
2021-02-22 18:12:02 -05:00
4fa0f9720a
Allow frontend to honor 'view as' and fix problem with tests
2021-02-18 11:25:17 -05:00
8c309e9a41
Add user display name to the response for task events
2021-02-18 08:44:44 -05:00
118cf62172
Merge remote-tracking branch 'origin/dev' into STG-181_frontend-ux-model-changes
2021-02-16 10:07:34 -05:00
003bf2f9b9
Remove Approvals per ticket 183
2021-02-16 09:05:29 -05:00
da0af14724
Changes to enable front-end changes
...
Added a human readable name to created user
added last updated task time and name,
added is_review on the workflow which is triggered by a swimlane with a name present.
Notably - it does NOT update or change any of the performance concerns that we were having with this.
2021-02-10 11:58:19 -05:00
777cfbdecd
We now test whether we have a valid StudyModel when getting a study by study model.
...
We raise an ApiError if the model is None or empty.
2021-02-09 17:37:55 -05:00
b0dc834682
Don't error out on autocomplete_num
2021-01-04 13:41:57 -05:00
ede5df4df5
Studies can now be put on hold in the dashboard. We were overriding the status in StudyModel.update_from_protocol_builder.
2020-12-30 17:49:59 -05:00
1f9bf72c59
Merge branch 'dev' into bug/navigation
2020-12-14 11:46:44 -05:00
6bf24cc438
fixing some failing tests related to changes in the underlying spiffworkflow library.
2020-12-14 11:27:38 -05:00
02ea414b94
Additional fixes to Navigation to allow a nested navigation structure.
2020-12-14 10:07:19 -05:00
0e1aa59fa1
Make a change to make sure that if there is a new file locally that is not present remotely when we pull from the remote, the new local file gets deleted.
...
Also: add several things to the requirements.txt that should have been there in the first place.
2020-12-08 13:42:01 -05:00
93b12a8e82
updates using new navigation from spiff workflow's navigation branch, all tests passing.
2020-12-04 17:56:12 -05:00
e9979efb0d
Add some tests
2020-11-17 09:33:29 -05:00
70425eee0d
Add last_updated field - change the api signature to be closer to what we need going forward, but retain the old methods for testing the script get/set through the API
2020-11-13 12:03:29 -05:00
7c2c725840
save before changing how API works
2020-11-13 09:58:21 -05:00
acaf633b45
Added ID as pk
2020-11-11 11:36:25 -05:00
245124779e
Add mostly-working elements of the data-store, compare with dev to see why db is not working after running tests
2020-11-10 08:21:46 -05:00
b544334f45
1. Updating Personnel BPMN diagram to debug some issues.
...
2. Disabling the token timeout for now, to see if this corrects the issues Alex is having with lost work.
3. Raising more thoughtful error messages for unknown lookup options.
4. Providing better validation of default values and injecting the correct value for defaults related to enum lists of all types.
5. Bumping Spiffworkflow library which contains some better error messages and checks.
2020-09-01 15:58:50 -04:00
53d09303d8
Validating that field properties are valid - they must exist as constants on the Task model.
...
Making all the lookup field names consistent.
Fixing the lookup service which was failing at times trying to find the correct field to use for building the lookup table.
Updating validation to check for additional fields and properties.
When connexion level errors occur, wrapping it in an API Error to be consistent.
2020-08-27 14:00:14 -04:00
85ad477b2b
Some minor cleanup on the study status and automatic events. I wanted to avoid having one database model automatically generating other database models as a side effect. The study service now has full responsiblity to recording study events.
...
To help in running tests, adding __init__.py methods to all the test directories.
2020-08-17 14:56:00 -04:00
369fe4bb35
Automatic events refactoring
2020-08-14 08:36:46 -06:00
9a5c1d7cfb
I may have finally wrapped my head around full text search in python. Now properly using an index based on simple rather than english dictionary which has far fewer stop words and stemming processes and plays much better to the type ahead search we are trying to provide.
...
Stop words are no longer excluded, so "other" is a valid search and gets a result.
2020-08-13 18:13:41 -04:00
9a31d591ae
Adds user uid and cascades study event in study model. Fixes migration to properly downgrade enums.
2020-08-12 10:13:23 -04:00
a561a5cddc
Adjusting study events attribute
2020-08-08 18:25:01 -06:00
bf3dae1885
Collecting user generated study changes
2020-08-05 20:29:05 -06:00
b4595b345a
Raises ApiError for 'display_name' extension property exception
2020-08-04 13:59:50 -04:00
50b207307a
Merge branch 'dev' into feature/update_study_status_latest
2020-07-31 14:26:24 -06:00
2c2967e633
Normalizing enum values
2020-07-31 11:19:50 -06:00
5d23223e51
New study status update
2020-07-30 21:03:11 -06:00
faba0f55ab
Adds AdminSession model and refactors impersonation methods to use it.
2020-07-30 12:40:53 -04:00
1b0ebecbf4
Uses Flask session to store impersonation state.
2020-07-30 10:17:02 -04:00
63537d7765
Adds is_admin boolean flag to user schema
2020-07-29 22:45:56 -04:00
de49397549
Adjustings tests for protocol builder status changes
2020-07-29 10:51:34 -06:00
73a6b7adf1
Fixing tests
2020-07-29 10:25:57 -06:00
6aa21638de
Updating properly study status & fixing tests
2020-07-29 10:25:57 -06:00
f897ee3aea
Merge branch 'feature/approvals_dashboard_prototype' into feature/admin_impersonations
2020-07-29 10:32:46 -04:00
0ea4c13d09
Convert protocol builder status to always be in lower case in order to better match the front end. And also fixing an issue with the multi_instance that is oddly broken suddenly, and I don't know why.
2020-07-28 17:16:48 -04:00
4d11fc04a0
dropping the "read_only" flag in favor of a "do_engine_steps" flag, which more clearly defines what is happening.
2020-07-28 13:51:29 -04:00
f15626033d
Allow the workflow to be requested without making changes to the workflow - requires that you specify a read_only flag of true, otherwise it assumes that you want a fully prepared workflow with the next ready task set to run.
2020-07-28 13:33:38 -04:00
300026cbc8
Expanding the task events endpoint to accept workflow and study ids as additional filters.
...
Removing events from the study endpoint, too noisy.
2020-07-28 10:16:48 -04:00
Dan
mike cullerton
alicia pritchett
Mike Cullerton
Kelly McDonald
NWalker4483
Carlos Lopez
Aaron Louie