Commit Graph

26 Commits

Author SHA1 Message Date
NWalker4483 a73d1794eb Fixed Associate API Endpoint description 2021-07-22 14:09:24 -04:00
Dan fb54edac1c Adding additional details to error messages, and cleaning up the cruft around these messages to keep them clear and succinct.
Most noteable is the addition of the line on which the error occurs for script tasks.  It will report the line number and pass back the content of
the line that failed.
The validator only returns the first error it encounters, as it's clear that all we ever get right now is two of the same error.
Did a lot of work between this and spiffworkflow to remove all the places where we obfuscate or drop details as we converted between workflowExceptions and APIExceptions.
Dropped the python levenshtein dependency, in favor of just rolling a simple one ourselves in Spiffworkflow.
2021-07-07 00:53:49 -04:00
Dan 1b1a994360 Refactoring Reference files to use the lookup table, rather than parsing the results directly out of the spreadsheet, or attempting to cache them.
Adding a DocumentService to clean up the FileService, and get Documents well seperated, as it seems likely be pulled out or seperated in the future, there is now a Documents api file as well, for the same reason.
Some other minor changes are just fixing white space to assure our code is linting correctly.
I removed _create_study_workflow_approvals from the base test, as we don't use approvals like this anymore.
2021-07-06 13:10:20 -04:00
Dan 07eb3f9ca8 Moving metrics into SpiffWorkflow so we can run the performance metrics deeply across both systems simultaniously.
Upgrading libraries.
Fixing deprication issue with Pandas and numpy.
We can only process xlsx files now, plain oldschool xls is fully removed.
2021-06-18 16:41:55 -04:00
Dan c8f5a44050 adding a warning in the logger so we can see when expensive calls are made to rebuild searches for enumerations. But all looks good. 2021-03-02 12:21:51 -05:00
Dan aac3d5c16e Bug #255, this requires the front end to pass in the name of the task, when doing a lookup. This will prevent a bug where we have multiple user tasks, with enum fields that set the same variable, but use different lookup tables to populate the dropbown or search feature. 2021-03-01 14:54:04 -05:00
Dan Funk b544334f45 1. Updating Personnel BPMN diagram to debug some issues.
2. Disabling the token timeout for now, to see if this corrects the issues Alex is having with lost work.
3. Raising more thoughtful error messages for unknown lookup options.
4. Providing better validation of default values and injecting the correct value for defaults related to enum lists of all types.
5. Bumping Spiffworkflow library which contains some better error messages and checks.
2020-09-01 15:58:50 -04:00
Dan Funk 53d09303d8 Validating that field properties are valid - they must exist as constants on the Task model.
Making all the lookup field names consistent.
Fixing the lookup service which was failing at times trying to find the correct field to use for building the lookup table.
Updating validation to check for additional fields and properties.
When connexion level errors occur, wrapping it in an API Error to be consistent.
2020-08-27 14:00:14 -04:00
Dan Funk 9a5c1d7cfb I may have finally wrapped my head around full text search in python. Now properly using an index based on simple rather than english dictionary which has far fewer stop words and stemming processes and plays much better to the type ahead search we are trying to provide.
Stop words are no longer excluded, so "other" is a valid search and gets a result.
2020-08-13 18:13:41 -04:00
Dan Funk 9077ff3ebf It is not possible to use task_data for an auto-complete field. It's too expensive an operation to provide that feature on the backend, and the data already fully resides on the front end anyway. Task-data can be used to populate enum fields if needed, so it can populate dropdowns, radios and checkboxes, just not auto-complete. 2020-07-14 11:38:48 -04:00
Aaron Louie 463660f185 Merge branch 'dev' into feature/dynamic_enum_list 2020-07-13 17:47:56 -04:00
Aaron Louie 07066b8a16 Looks up enum options from task data 2020-07-13 17:46:28 -04:00
Dan Funk 9e29a43785 Correct for a race condition where multiple lookup tables are built for the same field and workflow specification, causing it to appear that the models are not updating correctly. 2020-07-13 12:45:51 -04:00
Aaron Louie b7920989ed WIP: Adds Camunda property for retrieving enum field options from task data. 2020-07-10 14:48:38 -04:00
Dan Funk 84973d2351 resolving comments from pull request. 2020-06-30 12:24:48 -04:00
Dan Funk 93bf46354b A last minute change to make the API a little clearer and cleaner broke some tests. 2020-06-30 11:12:28 -04:00
Dan Funk f183e12fe5 Provides some basic tools for getting additional information about a lookup field.
Adds an optional 'value' parameter to the lookup endpoint so you can find a specific entry in the lookup table.
Makes sure the data attribute returned on a lookup model is a dictionary, and not a string.
Fixes a previous bug that would crop up if double spaces were used when performing a search.
2020-06-30 10:34:16 -04:00
Dan Funk d3ce1af1ce Provides some basic tools for getting additional information about a lookup field.
Adds an optional 'id' parameter to the lookup endpoint so you can find a specific entry in the lookup table.
Makes sure the data attribute returned on a lookup model is a dictionary, and not a string.
Fixes a previous bug that would crop up in double spaces were used when performing a search.
2020-06-30 10:00:22 -04:00
Dan Funk fed6e86f92 Trying to fix LDAP issues on production. Changing LDAP to static only methods, caching the connection and calling bind before all connection requests.
Also assuring we don't load the documents.xls file over and over again.
2020-06-04 14:59:36 -04:00
Dan Funk e102214809 minor cleanup of error codes. 2020-06-03 15:03:22 -04:00
Dan Funk c7484267e1 For the main approval endpoints - we now group the approvals by study. So you get one record back for each study, but it may have other approvals along with it as "related_approvals".
We now cache the LDAP records - so we look in our own database for the record before calling out to ldap for the details when given a straight up computing id like dhf8r.

Added "date_approved" to the approval model.

And moved the approver and primary investigator into real associated models to make it easier to dump.

Fixed a problem with the validation that was causing it to throw incorrect errors on valid workflows. Getting it to behave a little more like the front end behaves, and respecting the read-only fields.  But it was mainly to do with always returning all the data with each form submission.
2020-06-02 18:17:00 -04:00
Dan Funk 11413838a7 Faster lookup fields. We were parsing the spec each time to get details about how to search. We're just grabbing the workflow id and task id now and building that straight into the full text search index for faster lookups. Should be peppy.
Another speed improvement - data in the FileDataModel is deferred, and not queried until it is specifically used, as the new data structures need to use this model frequently.
2020-05-29 01:39:39 -04:00
Dan Funk 77f72e408f Lookup Service now raises exact matches to the top. Very hackish, but it works. 2020-05-27 14:36:10 -04:00
Dan Funk d5e075db82 Order search results by relevancy in the lookup service. 2020-05-27 09:47:44 -04:00
Dan Funk c4f2bd8dc6 Quick cleanup, adding a space 2020-05-19 16:23:20 -04:00
Dan Funk 951710d762 ldap lookup.
Refactored calls into a new lookup_service to keep things tidy.

New keys for all enum/auto-complete fields:
    PROP_OPTIONS_FILE = "spreadsheet.name"
    PROP_OPTIONS_VALUE_COLUMN = "spreadsheet.value.column"
    PROP_OPTIONS_LABEL_COL = "spreadsheet.label.column"
    PROP_LDAP_LOOKUP = "ldap.lookup"
    FIELD_TYPE_AUTO_COMPLETE = "autocomplete"
2020-05-19 16:11:43 -04:00