Removing the spreadsheet.value.column and data.value.column so we just have value.column for both.
Improving the __str__ function in the ApiError class, to make debugging a little easier.
Adding a "validate_all" flask command, to help us track down any issues with current workflows in production (use this in concert with sync_with_testing)
Fixed logs of tests.
removed fact_runner.py, a very early and crufty bit of code.
Adding a DocumentService to clean up the FileService, and get Documents well seperated, as it seems likely be pulled out or seperated in the future, there is now a Documents api file as well, for the same reason.
Some other minor changes are just fixing white space to assure our code is linting correctly.
I removed _create_study_workflow_approvals from the base test, as we don't use approvals like this anymore.
Record the size of a file in the database for quick access (this helps with a frontend refactor, so it isn't downloading the file just to see it's size)
Cleaning up the timing/performance metric reporting to make it easier to read.
Fixing a bug that prevented non-admins for getting the document-directory
Added a human readable name to created user
added last updated task time and name,
added is_review on the workflow which is triggered by a swimlane with a name present.
Notably - it does NOT update or change any of the performance concerns that we were having with this.
Adds an optional 'value' parameter to the lookup endpoint so you can find a specific entry in the lookup table.
Makes sure the data attribute returned on a lookup model is a dictionary, and not a string.
Fixes a previous bug that would crop up if double spaces were used when performing a search.
Adds an optional 'id' parameter to the lookup endpoint so you can find a specific entry in the lookup table.
Makes sure the data attribute returned on a lookup model is a dictionary, and not a string.
Fixes a previous bug that would crop up in double spaces were used when performing a search.
Another speed improvement - data in the FileDataModel is deferred, and not queried until it is specifically used, as the new data structures need to use this model frequently.
Added a File class, that we wrap around the FileModel so the api endpoints don't change, but File no longer holds refences to versions or dates of the file_data model, we
figure this out based on a clean database structure.
The ApprovalFile is directly related to the file_data_model - so no chance that a reviewer would review the incorrect version of a file.py
Noticed that our FileType enum called "bpmn" "bpmm", hope this doesn't screw someone up.
Workflows are directly related to the data_models that create the workflow spec it needs. So the files should always be there. There are no more hashes, and thus no more hash errors where it can't find the files to rebuild the workflow.py
Not much to report here, other than I broke every single test in the system at one point. So I'm super concerned about this, and will be testing it a lot before creating the pull request.
From an API point of view you can do the following (and only the following)
/files?workflow_spec_id=x
* You can find all files associated with a workflow_spec_id, and add a file with a workflow_spec_id
/files?workflow_id=x
* You can find all files associated with a workflow_id, and add a file that is directly associated with the workflow
/files?workflow_id=x&form_field_key=y
* You can find all files associated with a form element on a running workflow, and add a new file.
Note: you can add multiple files to the same form_field_key, IF they have different file names. If the same name, the original file is archived,
and the new file takes its place.
The study endpoints always return a list of the file metadata associated with the study. Removed /studies-files, but there is an
endpoint called
/studies/all - that returns all the studies in the system, and does include their files.
On a deeper level:
The File model no longer contains:
- study_id,
- task_id,
- form_field_key
Instead, if the file is associated with workflow - then that is the one way it is connected to the study, and we use this relationship to find files for a study.
A file is never associated with a task_id, as these change when the workflow is reloaded.
The form_field_key must match the irb_doc_code, so when requesting files for a form field, we just look up the irb_doc_code.
Added the following columns:
* date_created - so we know when the file was created
* renamed workflow_version to just "version", because everything has a version, this is the version of the request.
* workflow_hash - this is just a quick way to see what files and versions are associated with the request, it could be factored out.
* study - a quick relationship link to the study, so that this model is easier to use.
* workflow - ditto
* approval_files - these is a list from a new link table that links an approval to specific files and versions.
The RequestApproval is logically sound, but still needs some additional pieces in place to be callable from a BPMN workflow diagram.
Altered the file service to pick up on changes to files vs adding new files, so that versions are picked up correctly as
users modify their submission - adding new files or replacing existing ones. Deleting files worries me, and I will need to revisit this.
The damn base test keeps giving me a headache, so I made changes there to see if clearing and dropping the database each time won't allow the tests to pass more consistently.
Lots more tests around the file service to make sure it is versioning user uploaded files correctly.
The "Test Request Approval Script" tries to find to assure the correct behavior as this is likely to be called many times repeatedly and with little knowledge of the internal system. So it should just "do the right thing".
Improving the study_info script documentation to provide detailed examples of values returned based on arguments.
Making the tests a little more targetted and less subject to breaking through better mocks.
Allow all tests to pass even when ther protocol builder mock isn't running locally.
Removing the duplication of reference files in tests and static, as this seems silly to me at the moment.
Moving the primary process id from the workflow model to the file model, and assuring it is updated properly. This was causing a bug that would "lose" the workflow.
Required Documents is becoming complicated, so making this it's own script task, removing it from study_info.py
The file_service is now very aware of this irb_documents file, so it will always need to exist. We seed this file
during setup, but it can be overwritten by the configurator.
Dan
mike cullerton
Kelly McDonald
Carlos Lopez
Aaron Louie